Multidisciplinary Teams and UX Prototyping Strategy
The idea of multidisciplinary teams and UX wireframing/prototyping has been discussed by the UX Team for quite some time, but it remains unclear how can this process could work at Dick’s Sporting Goods.
Executive Summary The idea of multidisciplinary teams and UX wireframing/prototyping has been discussed by the UX Team for quite some time, but it remains unclear how can this process could work at Dick’s Sporting Goods. To analyze the efficacy and efficiency of both multidisciplinary teams and UX wireframing/prototyping – since the process could/should be potentially interrelated – we employed the following assessments: • Survey and analyze current members of the Front-End Development, User Interface, and User Experience teams at Dick’s Sporting Goods about their workflows, what interest they may have in working with other teams, and how this process could work for everyone in the future. • Survey and analyze external corporate workflows, processes, and practices outside of Dick’s Sporting Goods to determine what others feel works for their design and development process. • Examine quantifiable benefits for Dick’s Sporting Goods concerning blind optimization testing, versus conducting user research with wireframes/prototypes, and basing optimization tests on user data. • Analyze, catalog, and explore lean software suites, educational opportunities, and processes/workflows that eliminate waste, and increase efficacy and efficiency.
Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure Balsamic InVision Bitbucket Bitnami GitHub Vagrant HTML CSS SCSS AJAX Angular JavaScript jQuery Node React Aria Roles Color Contrast WCAG 2.0 Research Design Interaction Workflow Web Development Accessibility Expert Semi-Expert Familiar Some Experience No Experience UI Skill Set Visualization This infographic is a visualization of the User Interface team’s skill set. They’re aligned vertically by level of experience – ranging from No Experience to Expert – and distributed horizontally by a project life-cycle/development methodology (and alphabetically), wherein a project proceeds through the steps of research, design, interaction, web, development, and accessibility (with workflow/work management being placed in the middle). Feel free to zoom-in to see more detail. * Each bar represents a single survey response. Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure Balsamic InVision Bitbucket Bitnami GitHub Vagrant HTML CSS SCSS AJAX Angular JavaScript jQuery Node React Aria Roles Color Contrast WCAG 2.0 Research Design Interaction Workflow Web Development Accessibility Expert Semi-Expert Familiar Some Experience No Experience
Front-End Dev Skill Set Visualization This infographic is a visualization of the Front-End Development team’s skill set. They’re aligned vertically by level of experience – ranging from No Experience to Expert – and distributed horizontally by a project life-cycle/development methodology (and alphabetically), wherein a project proceeds through the steps of research, design, interaction, web, development, and accessibility (with workflow/work management being placed in the middle). Feel free to zoom-in to see more detail. * Each bar represents a single survey response. Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure Balsamic InVision Bitbucket Bitnami GitHub Vagrant HTML CSS SCSS AJAX Angular JavaScript jQuery Node React Aria Roles Color Contrast WCAG 2.0 Research Design Interaction Workflow Web Development Accessibility Expert Semi-Expert Familiar Some Experience No Experience Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure Balsamic InVision Bitbucket Bitnami GitHub Vagrant HTML CSS SCSS AJAX Angular JavaScript jQuery Node React Aria Roles Color Contrast WCAG 2.0 Research Design Interaction Workflow Web Development Accessibility Expert Semi-Expert Familiar Some Experience No Experience
UX Skill Set Visualization This infographic is a visualization of the User Experience team’s skill set. They’re aligned vertically by level of experience – ranging from No Experience to Expert – and distributed horizontally by a project life-cycle/development methodology (and alphabetically), wherein a project proceeds through the steps of research, design, interaction, web, development, and accessibility (with workflow/work management being placed in the middle). Feel free to zoom-in to see more detail. Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure Balsamic InVision Bitbucket Bitnami GitHub Vagrant HTML CSS SCSS AJAX Angular JavaScript jQuery Node React Aria Roles Color Contrast WCAG 2.0 Research Design Interaction Workflow Web Development Accessibility Expert Semi-Expert Familiar Some Experience No Experience * Each bar represents a single survey response. Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure Balsamic InVision Bitbucket Bitnami GitHub Vagrant HTML CSS SCSS AJAX Angular JavaScript jQuery Node React Aria Roles Color Contrast WCAG 2.0 Research Design Interaction Workflow Web Development Accessibility Expert Semi-Expert Familiar Some Experience No Experience
1 1.5 2 2.5 3 3.5 4 4.5 5 Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure InVision Balsamic Bitbucket GitHub Bitnami Vagrant HTML CSS SCSS JavaScript jQuery AJAX Angular React Node Aria Roles Color Contrast WCAG 2.0 User Interface Front-End Development User Experience Expert Semi-Expert Familiar Some Experience No Experience Research Design Interaction Workflow Web Development Accessibility Team Skill Set Visualization This infographic is a visualization of the averaged Front-End Development, User Interface, and User Experience team’s skill sets. They’re aligned vertically by level of experience – ranging from No Experience to Expert – and distributed horizontally by a project life-cycle/development methodology, wherein a project proceeds through the steps of research, design, interaction, web, development, and accessibility (with workflow/work management being placed in the middle). Feel free to zoom-in to see more detail. 1 1.5 2 2.5 3 3.5 Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure InVision Balsamic Bitbucket GitHub Bitnami Vagrant HTML CSS SCSS User Interface Front-End Development User Experience Familiar Some Experience No Experience
1 1.5 2 2.5 3 3.5 4 4.5 5 Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure InVision Balsamic Bitbucket GitHub Bitnami Vagrant HTML CSS SCSS JavaScript jQuery AJAX Angular React Node Aria Roles Color Contrast WCAG 2.0 User Interface Front-End Development User Experience Expert Semi-Expert Familiar Some Experience No Experience Research Design Interaction Workflow Web Development Accessibility Team Skill Set Discrepancies This infographic represent the most noticeable gaps in skill sets between the Front-End Dev, UI, and UX Teams, and the following are three different levels of gap severity, each reflecting a greater distance between skill sets from team to team: H(high) 50% – 75% Gap M(medium) 25% – 50% Gap L(low) 12.5% – 25% Gap H H M M M M L L 1 1.5 2 2.5 3 3.5 Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure InVision Balsamic Bitbucket GitHub Bitnami Vagrant HTML CSS SCSS User Interface Front-End Development User Experience Familiar Some Experience No Experience
1 1.5 2 2.5 3 3.5 4 4.5 5 Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure InVision Balsamic Bitbucket GitHub Bitnami Vagrant HTML CSS SCSS JavaScript jQuery AJAX Angular React Node Aria Roles Color Contrast WCAG 2.0 User Interface Front-End Development User Experience Expert Semi-Expert Familiar Some Experience No Experience Research Design Interaction Workflow Web Development Accessibility Team Skill Set Deficiencies This infographic represent the noticeable deficiencies in all three team’s skill sets. Individual software and languages withstanding, highlighted are the skills or areas of project life-cycle/ development methodology where at least two teams or more have a low level of proficiency. 1). User Data/User Research: Tap-able Areas, Links, Functionality, IA, etc. 2). Workflow Alignment: Project Management Systems, Repositories, Virtual Servers, etc. 3). Medium Alignment: HTML, CSS/SCSS, JavaScript, etc. 4). Accessibility Education: Aria Roles, Code Practice, Developer Tools, etc. 1 2 3 4 1 1.5 2 2.5 3 3.5 Qualitative Research & Analysis Quantitative Research & Analysis User Testing Competitive Benchmarking Content Strategy Adobe Illustrator Adobe InDesign Adobe Photoshop GIMP Sketch Adobe XD Axure InVision Balsamic Bitbucket GitHub Bitnami Vagrant HTML CSS SCSS User Interface Front-End Development User Experience Familiar Some Experience No Experience
DSG and Multidisciplinary Teams Team Total User Interface Front-End Dev User Experience In the past have you worked in a different role/team other than what you are currently (e.g. now a Front-End Developer, but formerly a UI Designer, etc.)? Yes 50% No 50% Yes 33% No 67% Yes 56% No 44% Yes 67% No 33% Have you ever worked in a multidisciplinary role/team before (e.g. you were responsible for both UI and UX, or you did UI and built websites)? A lot 30% A little bit 46% Not so much 12% Not at all 12% A lot 22% A little bit 56% Not so much 22% Not at all 0% A lot 45% A little bit 11% Not so much 11% Not at all 33% A lot 17% A little bit 83% Not so much 0% Not at all 0% Was working in a multidisciplinary role/team before successful? Yes 89% No 11% Yes 100% No 0% Yes 60% No 40% Yes 100% No 0% What are your thoughts about working multidisciplinary or cross-functionally with other teams at DICK’s Sporting Goods? It’d be awesome 62% It could work 17% I’m skeptical 21% It wouldn’t work 0% It’d be awesome 56% It could work 11% I’m skeptical 33% It wouldn’t work 0% It’d be awesome 67% It could work 22% I’m skeptical 11% It wouldn’t work 0% It’d be awesome 67% It could work 16% I’m skeptical 17% It wouldn’t work 0%
DSG Team Qualitative Responses How/Why was working in a multidisciplinary team successful? • Communication was key. We talked every little issue out. Nothing was assumed or a guess. • Overall it was a smooth process to design and develop since I knew my limits before designing. • Key takeaway was consistency. When an individual is involved in all aspects of the project at some level the idea/voice is heard all the way through keeping a connection of consistency benefiting the team and user. • We had a templated system • Being able to collaborate with team members with different expertise and skill sets makes the design process much easier. UX, UI and front end (and back end) development should work much more closely. Development specifically can be included in conversations early on to determine level or effort. This inclusion can prevent rework for the UX and UI teams once it gets to development if it turns out to be too large of an effort. Why do you think working in a multidisciplinary teams at DICK’s Sporting Goods could work, or even be awesome? • The more collaboration the better. This way we could bounce ideas off of each other and possibly come up with some great solutions. • Cross function helps the team grow and become stronger and avoids creative/ knowledge stagnation. Allows the teams to lean on others when times become stressful and no one project is solely on the backs of one individual. • Better work comes from teammates not being in ‘silos’ • Not only are projects more fun as a collaborative effort, they are more efficient as well. Including UX, UI and development in conversations such as feature mapping and story writing allow for all of the proper people to be in the room and speak to their specific part, as well as make any necessary clarifications on design and/or make adjustments based on development feedback. • I think it could help us grow in our own position. It’s helpful to at least know what other teams do.
DSG Learning and Tools User Interface Front-End Dev User Experience Which of these areas would you like to learn more about? Select all that apply. Front-End Dev 78% User Experience 78% User Interface 33% Front-End Dev 22% User Experience 89% User Interface 100% Front-End Dev 83% User Experience 33% User Interface 50% Are there any other tools you use or would like to use in your job? Adobe Cloud Sketch * Framer Adobe XD * Axure Front-End Development * Animation * Automated Testing Illustrator Wireframing * InDesign * Development Tools Adobe XD * Sketch TechSmith Morae * Are there any other skills you use or would like to use in your job? Animation * Design Regulations/Standards Wireframing * Illustrator InDesign * Human Centered Design Front-End Development * Iconography Typography Email Marketing Social Media Animation * Less Restricting CMS Wireframing * Automated UI Testing UX Skills Web Development Skills Prototyping * Wireframing * Stakeholder Negotiations TechSmith Morae * Front-End Development * * Responses that were mentioned by more than one team
External Participants The following demographic visualizations are from professionals recruited through social platforms (LinkedIn, Twitter, etc.), and represent roughly three weeks of recruitment. Individuals that participated are from Adobe, Boeing, Home Depot, and other entities. 1-9 Team Members 48% 10-20 Team Members 14% 20-49 Team Members 21% 50-75 Team Members 7% 75+ Team Members 10% Aerospace 3% Chemical 3% Computer 14% Construction 3% eCommerce 17% Education 21% Entertainment 3% Healthcare 17% Information 17% User Experience 55% User Interface 59% Development 41% Other 34% What is your industry? What are you primarily? Data visualization above is based on 29 survey participants How big is your company’s UI, UX, Design, Web Design, and/or Development team?
External Practices and Workflow Participants Participants Now or in the past have you worked in a different role/team other than what you are currently? Yes 72% No 28% Do you blur the lines between design and development/engineering? Yes 42% Somewhat 47% No 10% Have you ever worked in a multidisciplinary role/team before? A lot 72% A little bit 12% Not so much 8% Not at all 8% Do you have a mediation process for conflict/process resolution (e.g. voting, or I’m a 2 out of 10 on this)? Yes 43% No 47% What this? 10% Was it successful working in a multidisciplinary role/team? Yes 83% No 16% What is the design/development process in which you currently work? Select all that apply. Kanban 26% Waterfall 26% Agile 79% Lean 21% Scrum 42% How much does design inform your overall process? A Leading Voice 47% A Contributing Voice 53% A Voice Not Considered 0% How well does Agile work? It works well 61% It kind of works 33% It doesn’t really work 6% Do you use any of the following for your processes and products? Select all that apply. CSS Framework 82% Content Management System 82% Pattern Library 41% Style Guide 76% How well does Waterfall work? It works well 40% It kind of works 60% It doesn’t really work 0%
External Qualitative Responses How/Why was working in a multidisciplinary team successful? • I had support from other team members but more importantly from people higher up in the chain who had a general idea of what I did and why it took the time it does to implement a well functioning and well designed product. • I’m diversely trained. Work demanded it. • Had a good team around me. • I’ve performed UX and UI on many projects, sometimes there is benefit to not worrying about visuals while you’re trying to structure functionality or to not have your head clouded with output while you’re performing research. However, I often find that on projects where I’m involved at every stage up to development, I find it helpful to have a full picture view at all times. • Was able to oversee the full scope of the project, it’s timeliness and easily communicate in all aspects. What are the pain points, if any, with your current process? • Offshore communication. • My team lives in a waterfall world. We get sprints interrupted frequently. • Not as much of a goal/ending place to achieve - always work to be done, but where are we going?? • Creative agency vs development agency and merging platforms they use. • My scrum project has gone well. Other projects don’t have focus and so didn’t matter what style it was. Do you think your team and process works well? If not, how would you design it differently? • Have a scrum master that has teeth. • I would like more of an organized leader/guiding voice for goals and deadlines. • Could use more data and IT support. • It’d be better if we actually worked together sometimes instead of everyone having separate parts which evolve independently. • I would keep the process less siloed with more interactions with design/layout usability testings and development process with front-end development mostly involved all along the process and back-end brought in as needed. • Yes. Currently adjusting to new methods and getting everyone on board. • It’s nice to have a more granular view into what everyone is working on, rather than just having a vague idea. • For the most pet yes we are a well oiled machine. Only downfall is turnover in people so there is always change. • Our team works really well together. Our process works reasonably well. I would like us to have a more iterative process where we advance to new levels of fidelity only after we’ve explored concepts at the current level, identifying risk and attempting to “de-risk” our designs at each level. • We need to be able to do rapid user testing, with minimal ramp-up time for the test and minimal analysis time. That would allow us to build confidence at each step instead of moving ahead in “faith”, hoping we’re on the right track, until we can do a study to confirm our direction, breath a sigh of relief (hopefully!) and repeat that whole process.
CSS 11% CSS 11% CSS 26% CSS 16% CSS 37% HTML 11% HTML 5% HTML 32% HTML 16% HTML 37% JavaScript 11% JavaScript 16% JavaScript 37% JS 5% JavaScript 32% Not Important Slightly Important Important Fairly Important Very Important JavaScript HTML CSS External Skill Set Importance How important do you think it is for all UI, UX, and Front- End Dev employees on your team to know CSS, HTML, and JavaScript?
DSG Optimization Dick’s Sporting Goods optimization campaign features anywhere from 2 to 4 splits on average – one of which is an existing design (which functions as a control) – but sometimes can contain upwards of 8 splits. 2016 Results According to Senior eCommerce Analysts in Web Optimization: • 44 optimization campaigns were launched in 2016. • The success rate for optimization campaigns to increase revenue was 13.6%, whereas the industry standard is around 25% per year. • Optimization has no metric for loss per split/average revenue loss for unsuccessful campaigns. • The average annual revenue gain was around $8M per successful campaign, and the 6 successful campaigns totaled over $50M in additional revenue. Going forward, and with the addition of both Field and Stream and Golf Galaxy, it’s likely that the number of optimization campaigns will increase in 2017. Unsuccessful Campaigns 86% Successful Campaigns 14% Room for Improvement to Industry Standard 11%
Prototyping and Optimization Currently Dick’s Sporting Goods involves several resource intensive steps before we LEARN * anything from our users. However, prototyping capabilities would allow us to LEARN more about our users sooner, cheaper, and with less time investment. Benefits of LEARN-ing Sooner • Saving money and time in optimization by using data to inform splits • User data can potentially improve campaign outcomes • Reducing overall design and development time • Providing more robust sources of truth for development • Creating excitement and sharing vision within organization • Making overall enhancements easier Prototyping Case Studies • Over a 8 week span in 2012, Kids.gov ran 2 sets of iterative prototype tests with users. The prototyping process increased site traffic by 51%. • Using prototype testing within the UserTesting platform, Evernote was able to increase user retention by 15% and saw a “dramatic increase in user engagement.” • By using UX before optimization, and targeting areas customers actually struggled with, Lovehoney.com increased conversion rates by 25% (despite a 60% increase in mobile traffic) and increased overall revenue by 115%. *
Team Recommendations Companies like GE, Intuit, Toyota, Buzzfeed, and more work in multidisciplinary and cross-functional teams to accomplish innovative work. The following are suggestions based on the research presented in this report. Multidisciplinary Team Willingness • 89% of the UI, UX, and Front-End Dev teams feel that it was successful working in a multidisciplinary role/team in the past. • 79% of the UI, UX, and Front-End Dev teams feel that working in a multidisciplinary role/team at Dick’s Sporting Goods would either be awesome or could work. Team Skill Set Deficiencies & Discrepancies • User Data/User Research There is a large gap between the UI/Front-End Dev and the UX Team’s knowledge and skill sets concerning user data, user research, qualitative research, and quantitative research. Recommendation: The UX Team should incorporate the UI and Front-End Dev teams in the research process, and even conduct lunch-and-learn sessions concerning user expectations and standards for web interaction. • Workflow and Medium Alignment The Front-End Dev, UI, and UX team are not working in the same project management systems, softwares, repositories, virtual servers, or workflows. This leads to duplication of work, recreation of design artifacts, and work not being performed in the closest medium to the final product. Recommendation: The UX and UI teams should learn more about code through online courses/coding sessions at work, all teams could work in project repositories, and wireframes and comps could be built in HTML, CSS/SCSS, and JavaScript. • Accessibility Education All teams have a lack of knowledge and understanding concerning accessibility and web standards as it pertains to aging, disabilities, peripheral devices, and best code and design practices. Recommendation: All teams should attend general training concerning accessibility guidelines, and individual teams should develop a deeper understanding of accessibility through training specific to their areas of focus.
Learning Recommendations Workforce training is an indispensable way to keep an organization competitive. According to an Execu-Search report about hiring in 2017, 76% of millennial employees want opportunities for career growth. The following are suggestions based on the research presented in this report: Desire for Learning • A proportion of each team – UI 33%, FED 22%, and UX 33% – wants to grow their knowledge of their own team’s skills. • The majority of each team – 50% to 100% – want to grow their knowledge of other team’s skills. • 78% of the UI Team want to learn more about Front-End Dev and UX teams. • 89% of the Front-End Dev Team want to learn more about the UX Team, and 100% want to learn more about the UI Team. • 83% of the UX Team wants to learn more about the Front-End Dev Team, and 50% want to learn more about the UI Team. Learning Resources • Lunch-and-learns/learn from each other • Job Shadowing • Online Learning: Code Academy, Code School, Treehouse, Lynda, YouTube, etc. • Design Tweaking Sessions - Get all the JIRA bugs, low hanging fruit, and CSS cleanup to have a few UX and UI Designers help fix those bugs. Desired Team Skills and Software SOFTWARE Adobe XD Sketch TechSmith Morae SKILL SETS Animation Front-End Development Prototyping Wireframing * For full team-specific results see page 14.
Prototyping Recommendations As DSG merges Project Eagle and Current Business groups into one, and takes full control of the site’s functionality, it’s important to take this opportunity to focus on enhancements for the users. Making good decisions about those enhancements requires robust user data. Prototyping will allow DSG to gather better user data to inform design choices, feature validity, and the optimization process. Prototyping will also help identify critical areas for improvement, and overall provide DSG customers with an improved eCommerce experience. DSG needs both the skills to prototype, as well as the space, time, and tools to exercise a prototyping capability. This could be tested via the following: Pilot Projects (Enhancements or New Features) • Identify a specific project to prototype a solution for UX testing, that could be built with a cross-functional team utilizing the same set of tools. • Utilize people from UI, UX, and Front-End Dev who have expressed a desire to learn more about other teams’ work, and who are willing to be teachers of their own. • Use real working prototypes to get user data, feedback, and insights to inform optimization tests.
timbroadwater
bengo4com
shimacos
kongmingstrap
z63d
tarotaro0129
cybozuinsideout
yamaguchitk333
matthewlujp
ysknsid25
yabakokobayashi
maruto
rvirus0817
bkeepers
kastner
jlugia
addyosmani
eileencodes
tammyeverts
mojombo
eitanlees
sachag
csswizardry
gr2m
