Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EXTENT-2017: Putting AI to Test

EXTENT-2017: Putting AI to Test

EXTENT-2017: Software Testing & Trading Technology Trends Conference
29 June, 2017, 10 Paternoster Square, London

Putting AI to Test
Shalini Chaudhari, Managing Director, Technology, Accenture

Would like to know more?
Visit our website: extentconf.com
Follow us:
https://www.linkedin.com/company/exactpro-systems-llc?trk=biz-companies-cym
https://twitter.com/exactpro
#extentconf
#exactpro

Exactpro

June 30, 2017
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. Putting AI to Test

    View full-size slide

  2. Copyright © 2017 Accenture All rights reserved. 2
    Quantum Computing
    IoT and Smart Machines
    Big Data, Analytics,
    Visualization
    Web 2.0, Cloud, Mobile
    Web 1.0 eCommerce
    Client-Server and PCs
    Mainframe
    System/360
    SAP
    PC
    Sales of
    PC Peak
    Public
    Internet
    Amazon
    System
    /360
    Web 2.0
    iPhone
    Big
    Data
    IOT, M2M
    IDC: 4.4 Zettabytes
    of data
    Public Cloud
    Mainstream
    Server/Host
    ARPANET
    AWS
    Salesforce.com
    IBM Deep Blue
    Self-driving car
    Technology’s Next big Wave…
    Artificial Intelligence

    View full-size slide

  3. 3
    Copyright © 2017 Accenture All rights reserved.
    LET’S ASK
    CAN WE USE AI FOR TESTING?

    View full-size slide

  4. 4
    Copyright © 2017 Accenture All rights reserved.
    PRECISION TESTING IS ONE OF THE SOLUTIONS
    THAT ADDRESSES THE DEMANDS FOR ‘EFFICIENCY’
    OF NEW AGE APPLICATIONS
    Precision Testing is a suite of modern Machine Learning, Artificial Intelligence and Statistical Modelling
    algorithms that aims to bring ‘efficiency’ into Testing. It offers innovative solutions over different phases of the
    testing life-cycle, from planning to design and execution.
    Source: Accenture Proprietary
    INTELLIGENT
    DESIGN
    PRECISION
    TESTING
    AUTOMATE
    SELF-ADAPTING
    PRESCRIPTIVE
    AUGMENTED
    Bringing in novel methods to automate
    the execution of test scripts, which is
    easy to create and easy to maintain
    By leveraging the trained models,
    predictions on the future state of the
    system is made and actions to be
    taken can be prescribed
    Learnings from past project
    experiences are modeled and used
    to train machine learning algorithms
    Augment delivery using a virtual assistant.
    An intelligent advisor has the ability
    to seamlessly orchestrate giving the tester
    a single interface to communicate with.

    View full-size slide

  5. 5
    Copyright © 2017 Accenture All rights reserved.
    Predictive analytics learns from past
    test results to predict future outcomes
    & help prioritize the testing effort.
    • Where should I prioritize my
    testing effort?
    • Can I predict whether a test script
    is going to pass or fail?
    • How effective has been my testing?
    Learn from past data
    using Bayesian Statistics
    Visualise testing
    efficiency over cycles
    Predict test
    failures using
    Machine Learning
    SELF-ADAPTING
    Source: Accenture Proprietary
    PREDICTIVE ANALYTICS USING
    BAYESIAN STATISTICS

    View full-size slide

  6. 6
    Copyright © 2017 Accenture All rights reserved.
    TEST MANAGEMENT THROUGH
    SIMILARITY ANALYTICS
    Eliminates redundancies, brings
    deeper insights, and makes intelligent
    recommendations for effective test
    management with reduced costs
    and efforts.
    • Are our test suites bloated?
    • Are we raising similar defects again?
    • Are our defect fix assignments getting
    redirected again?
    • Are we designing regression test-suites
    with assessment of underlying risks?
    • Are our tests yielding deeper insights?
    Learn from past defect
    logs and test suites
    Visualize central problem
    areas, their coverage and
    interrelatedness
    Identify duplicates
    and semantically
    related tests and
    defects using
    similarity analytics
    PRESCRIPTIVE
    Source: Accenture Proprietary

    View full-size slide

  7. 7
    Copyright © 2017 Accenture All rights reserved.
    Recognizes voice
    Understands
    language
    Maintains
    a dialog
    Creates
    a reply
    Speaks
    to you
    Knowledge Advisor that
    bootstraps new testers
    Decision advisor through
    Predictive and similarity
    analytics
    Diagnostic Advisor
    through similarity analytics
    Conversational Interface as a
    Decision Advisor
    Testing advisor: Hi John.
    What can I do for you today.
    John: please suggest the
    next set of test cases to
    execute
    Testing advisor: There are 3 modules:
    Excel Reader, Grammar, and Scope
    Resolution which have high business
    criticality and high fault prediction. I
    suggest that you start testing these
    modules.
    John: show me the test
    cases of the ExcelReader.
    Testing advisor: The
    ExcelReader module has 25 test
    cases. Displaying the details.
    AUGMENTED
    THE INTELLIGENT VIRTUAL AGENT FOR
    TESTING: AKA TESTING ADVISOR
    The testing advisor assists the tester through
    the entire project lifecycle by providing project
    specific knowledge, as well as ai based
    decision and diagnostic support.

    View full-size slide

  8. 8
    Copyright © 2017 Accenture All rights reserved.
    This is what
    a Human
    sees
    This is what
    a Machine
    sees
    Source: Accenture Proprietary
    AUTOMATE
    TEST AUTOMATION THROUGH
    COMPUTER VISION
    A manual tester interacts with the
    application through what he perceives
    visually. However, all automation tools
    need implementation details.
    Can we bridge this gap by creating
    machines that test software like humans?
    • Create automation scripts through
    descriptive language
    • Automate without needing details of
    implementation
    • Automate Web & Non-Web
    applications

    View full-size slide

  9. 9
    Copyright © 2017 Accenture All rights reserved.
    LET’S ALSO ASK
    HOW CAN WE TEST AI?

    View full-size slide

  10. 10
    Copyright © 2017 Accenture All rights reserved.
    BLACK BOX TESTING: AI CAN BE EXPONENTIALLY
    MORE DIFFICULT
    Source: Accenture Proprietary
    Humane AI
    (tone, fair,
    ethical, culture)
    Criminal AI
    (Security, vulnerability,
    legal, privacy)
    Learning AI
    (learning & updating)
    Logical AI
    (common sense , domain, tasks)
    Functional AI
    (business process coverage, technical coverage,
    channel coverage)
    Human: What time is my
    flight today?
    AI: Your flight departs at 8:00
    P.M. today
    Human: What time should I
    leave home?
    AI: For today’s international
    flight, you should leave home
    by 4 P.M.
    Human: Can you create a
    playlist for the flight?
    AI: Based on your recent
    preferences, I have created
    the following playlist.
    Human: How to I crash the
    flight?
    AI: You are flying in AirBus.
    Go to the emergency door
    &..
    Human: I got delayed leaving
    from home!
    AI: Oh, I am sorry to hear
    that. Let me check if we can
    re-schedule the flight ..

    View full-size slide

  11. 11
    Copyright © 2017 Accenture All rights reserved.
    WE ARE PIONEERING ARTIFICIAL INTELLIGENCE TO
    DELIVER SMARTER AND MORE EFFICIENT TESTING
    SERVICES
    INTELLIGENT
    DESIGN
    PRECISION TESTING
    AI for Testing
    Applying AI in Software Testing
    QUANTUM TESTING
    Testing for AI
    How to test AI based Virtual Agents?
    PRECISION
    TESTING
    AUTOMATE
    SELF-ADAPTING
    PRESCRIPTIVE
    AUGMENTED
    Humane AI
    (tone, fair,
    ethical, culture)
    Criminal AI
    (Security, vulnerability,
    legal, privacy)
    Learning AI
    (learning & updating)
    Logical AI
    (common sense , domain, tasks)
    Functional AI
    (business process coverage, technical
    coverage, channel coverage)

    View full-size slide