Just What Is A Quality Engineer?

A great plurality of companies today, who deal with the delivery of software or providing software services should have a team responsible for checking the quality of their products before they get into their customers’ hands. This team, far too often, is comprised of a single person, usually labeled with the term “QA”, and is responsible for making sure that all known issues with the product are verified and that some level of testing is performed before it can be ‘shipped’ to customers. Not everyone really knows what happens during this phase, but it is well-known that someone, somehow, installs, configures and ‘tests’ all the facets and features of a product before it ‘goes out the door’.

The stereotype is that somewhere in the building where you work, there is at least one person, “not suitable” to be a developer, who is relegated to pushing buttons and clicking elements on web ui elements to make sure that things work as advertised. These are the ‘button pushers’.

If you have some of this species inhabiting your work environment, good for you. But if you really want to deliver quality with your products, then what you need is a Quality Engineer, a rare hybrid species that merges software development, forensics, DEVOPS and creativity skills into a super being.

Transcript

1. @maaretp It is not about proving that QE > DEV. Finding issues early is only part of the solution.
2. Just What Is A Quality Engineer? Lessons From The Trenches Og Maciel <omaciel@redhat.com>
3. 4 Objectives Things that I hope you’ll get out of this presentation • How to pronounce my name :) • What is QA • What is QE • Skills for becoming QE
4. My Name
5. Pronounced “Augie” Mah-see-el • My mother was a huge fan of the American author Augustine “Og” Mandino • Wrote the bestselling book The Greatest Salesman in the World. • “Og” has cool spelling but pronunciation sounded better as “Augie” • Wrote The Gift Of Acabar which was translated to Brazilian Portuguese as “The Best Gift in the World” • Yup, I was the best gift in the world :)
6. Name: Og Maciel Work: Red Hat since 2011; rPath previously Job: Senior Manager, Quality Engineering Projects: • RH Satellite 5 and 6 • Pulp • Sonar
7. Nobody Knows QE
8. Me: “I’m a Quality Engineer.” Others: “Oh, you mean QA?” Me: “No, QE!!!”
9. The problem • “Quality Assurance” is a process, not a profession! • “Quality Engineer” is not a well known term • Google shows: ◦ 5,390,000 hits for “Quality Engineer” ◦ 83,100,000 hits for “Quality Assurance” ◦ 86,900,000 hits for “Quality Control” Starts Late • QE is late to the party ◦ Usually DEV is dragged kicking and screaming into ‘exciting’ new role • “What does QA do again?” • “Testers don’t write code” mentality
10. (Broken) Software Life Cycle model QA
11. Late to the Party • No visibility until the end • Testing a black box • No chance to be part of design • No room or time for changes • “Gate Keepers” role leads to negative image Starts Late
12. (BETTER) Software Life Cycle model QE QE QE QE QE QE
13. Quality Early, Quality Often • Provide constructive feedback throughout, based on experience with product and customer. • Participate of design, define user stories, write test cases and explore areas outside the ‘happy path’. • Automate whatever and whenever it makes sense to automate! • Do exploratory testing too!
14. Quality Assurance
15. “The terms "quality assurance" and "quality control" are often used interchangeably to refer to ways of ensuring the quality of a service or product.” “Quality assurance comprises administrative and procedural activities implemented in a quality system so that requirements and goals for a product, service or activity will be fulfilled. It is the systematic measurement, comparison with a standard, monitoring of processes and an associated feedback loop that confers error prevention.” https://en.wikipedia.org/wiki/Quality_assurance Quality Assurance
16. Keywords • Requirements fulfilled • Measurement • Comparison • Monitoring • Feedback loop • Error prevention • It is a PROCESS and not a PROFESSION • Validate requirements (if you’re lucky to have them) to make sure that the product conforms with them • Compare results with previous ones • Catch errors/issues before customers do… but late in the game • Reactive mode: ON
17. Quality Assurance == Superman • Reacts to events • Acts on instinct • No deep analysis of problems • “Solutions” based on “gut feeling” and involving a lot of “flying around” and “beating things down” until resolved • Easily defeated by Kryptonite Different roles involved in Quality Assurance: • Requirements engineer • Test designer • Test executor
18. Quality Engineering
19. “Quality Engineering integrates methods and tools from Enterprise architecture-management, software-product management, IT service management, software engineering and Systems Engineering, and from software quality management and information security management. This means that quality engineering goes beyond the classic disciplines of software engineering, information security management or software product management since it integrates management issues (such as business and IT strategy, risk management, business process views, knowledge and information management, operative performance management), design considerations (including the software development process, requirements analysis, software testing) and operative considerations (such as configuration, monitoring, IT service management).” https://en.wikipedia.org/wiki/Quality_engineering Quality Engineering
20. Keywords • Goes beyond • Business and IT strategy • Risk management • Business process views • Knowledge and information management • Operative performance management • Development process • Requirements analysis • Software testing • Configuration • Monitoring • It is a PROFESSION with specific SKILLS • Part of planning, design and testing phases • Opinion matters ◦ Issues and customers ◦ Design and patterns ◦ Scalability and performance • Proactive mode: ON
21. Quality Engineer == Batman • Thinks before acts • Deep analysis of problems • Solutions found based on real evidence and data • No “gut feeling” involved; hard code data • Utility belt!!! • When everything else fails, finds solution with what’s at hand Different roles involved in Quality Engineering: • Business architect • IT architect • Security officer • Requirements engineer • Software quality manager • Test manager • Project manager • Product manager • Security architect
22. Conclusion
23. Batman > Superman
24. QE Utility Belt, or Sharpening your Bat Saws! Tools of the Trade
25. Be Your Customers Champion • Learn about customer pain points and issues • File bug reports if needed • Collect the data and find trends • Propose testing strategy based on trends ◦ Increase customer satisfaction ◦ Increase code coverage • Know when/how to turn down RFEs • Always deliver quality!
26. Doing The Leg Work • Be customer #1 (make no assumptions and follow docs) • No shortcuts allowed (do as the customers would) • Make note of usability pain points and if it hurts, let people know about it (provide feedback!) • Don’t be afraid to explore • Avoid the “Not My Job” attitude
27. Use Real Data • You cannot quantify “gut” • You cannot argue against real data • Easier to make logical decisions • Use issues tracker and customer data trends • Report your findings (show the data!)
28. Part of the Solution
29. Write Awesome Bug Reports • Use positive, constructive words • Be clear and concise • Provide context and background • Write clear, well tested steps to reproduce the issue • List of packages, build number, Operating System, web browser versions • Include logs, screenshots/videos • Submit a patch!!! • Example: https://github.com/ansible/ansible-mo dules-extras/issues/3358
30. Follow Up • Set up ‘reminders’ for your issues • ‘Check in’ and provide more relevant feedback after the original issue discovery • Schedule a quick meeting (if needed) • Offer a system where folks can do forensics • Don’t take ‘No’ for an answer • Use positive, constructive words • Assume ‘positive intent’ • Offer to test patches
31. Wrap It Up • Provide clear steps as to how issue was resolved • Include scripts used (if any) • Include screenshots/videos • Write down what the solution was :)
32. Automate (Almost) Everything
33. @brainpicker
34. http://xkcd.com/303/
35. Test Pyramids
36. Rest API • Best “bang for your buck” • Easy to automate and increase code coverage • Easily add automation for CRUD operations • Fall in LOVE with Python Requests package (http://docs.python-req uests.org/en/master/) (playground) ~ » pip install requests Successfully installed requests-2.12.1 (playground) ~ » python Python 2.7.12 (default, Jun 29 2016, 14:05:02) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> r = requests.get('http://httpbin.org/get') >>> assert r.status_code == 200 >>> r = requests.post('http://httpbin.org/post', data = {'Quality Engineer':'Batman'}) >>> r.json() {u'files': {}, u'origin': u'107.15.52.113', u'form': {u'Quality Engineer': u'Batman'}, u'url': u'http://httpbin.org/post', u'args': {}, u'headers': {u'Content-Length': u'23', u'Accept-Encoding': u'gzip, deflate', u'Accept': u'*/*', u'User-Agent': u'python-requests/2.12.1', u'Host': u'httpbin.org', u'Content-Type': u'application/x-www-form-urlencoded'}, u'json': None, u'data': u''}
37. Command Line • Usually the easiest endpoint to automate • Much easier to automate and increase code coverage • May require some data parsing to handle results • Fall in LOVE with the Paramiko package (http://www.paramiko.or g/) (playground) ~ » pip install paramiko Successfully installed paramiko-2.0.2 (playground) ~ » python Python 2.7.12 (default, Jun 29 2016, 14:05:02) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import paramiko >>> client = paramiko.SSHClient() >>> client.load_system_host_keys() >>> client.connect('omaciel.fedorapeople.org', password=’*******”) >>> stdin, stdout, stderr = client.exec_command('ls -l') >>> print stdout <paramiko.ChannelFile from <paramiko.Channel 0 (closed) -> <paramiko.Transport at 0x4ddfbd0L (cipher aes128-ctr, 128 bits) (active; 0 open channel(s))>>> >>> print stdout.read() total 808 drwxr-xr-x. 3 omaciel omaciel 38 Oct 28 18:50 bagoftricks -rw-r--r--. 1 omaciel omaciel 46417 Nov 18 13:48 bagoftricks.tar.gz
38. Web User Interface • Web User Interface is hard! • Requires a lot of creativity • Relies on existing CSS IDs and/or XPATH to locate elements • Brittle! Small changes can break EVERYTHING! • Fall in LOVE with Python Selenium WebDriver package (http://www.seleniumhq. org/docs/03_webdriver.j sp) (playground) ~ » pip install selenium Successfully installed selenium-3.0.1 (playground) ~ » python Python 2.7.12 (default, Jun 29 2016, 14:05:02) [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from selenium import webdriver >>> driver = webdriver.Firefox() >>> driver.get("http://www.google.com") >>> inputElement = driver.find_element_by_name("q") >>> inputElement.send_keys("quality engineering") >>> assert 'quality engineering' in driver.title
39. Looking at CSS and XPATH information for a web element
40. Finding Web Elements >>> from selenium.webdriver.common.by import By >>> element = driver.find_element(by=By.ID, value="lst-ib") >>> element.send_keys("quality engineering") >>> element = driver.find_element(By.CSS_SELECTOR, "input#lst-ib.gsfi") >>> element = driver.find_element(By.XPATH, "//input[@id='lst-ib']") >>> element = driver.find_element(By.TAG_NAME, “”)
41. Continuous Integration • Tie it all together • Ad-hoc, on-demand test execution or scheduled tasks • Triggered execution • Pipelines and permutations • Fall in LOVE with Jenkins (https://jenkins.io/ind ex.html)
42. Jenkins dashboard showing automation trends across different test runs
43. System Administration • MUST know Linux! Period! • You can never know enough $SHELL scripting • Emacs/Vim/Nano. Pick one and get good at it • Ssh, scp • Tail, grep, sed, awk
44. Collaborate
45. Sharing is Caring • Nobody is an island ◦ Learn from others ▪ Developers ▪ Support ▪ Documentation • Sharpen your saw ◦ Learning every day • Share what you know ◦ Talks and demos • Feedback is a gift! ◦ Growth requires constructive feedback
46. Q & A max growth
47. Thank You • Blog: https://omaciel.github.io • Twitter: @OgMaciel • Github: omaciel • Speaker Deck: https://speakerdeck.com/omaciel/ just-what-is-a-quality-engineer