Quantitative Regulatory Science (RegOps) for Software Intensive Medical Devices and SaMD

www.netspective.com
© 2021 Netspective. All Rights Reserved.
1
Quantitative Regulatory Science
(“RegOps”) for FDA Digital
Health/Therapeutics Compliance
By Shahid N. Shah, Founder, Netspective Communications LLC
This Photo by Unknown Author is licensed under CC BY

View Slide

www.netspective.com
2
This deck is available at
http://www.speakerdeck.com/shah
@ShahidNShah

View Slide

www.netspective.com
3
• Medical Devices, Gov’t Tech & Security Advisor
• 15+ years of safety-critical compliance and cybersecurity
expertise (in healthcare, government, and other sectors)
• 18+ years of healthcare IT and medical devices experience
(blog at https://healthcareguy.com and publisher at
https://medigy.com)
• 20+ years of technology management experience (government,
non-profit, commercial)
• 30 years of software engineering and multi-discipline complex
IT implementations (Gov., defense, health, finance, insurance)
Author of two chapters: “Understanding Medical Practice
Cybersecurity Risks” and “How to Conduct a Health-Care
Environment Electronic Risk Assessment”
Who is Shahid?

View Slide

www.netspective.com
4
What’s this talk about?
What we will discuss
FDA is looking to innovate around how
digital therapeutic solutions are regulated
(“Software as a Medical Device”, SaMD).
Software systems are not “manufactured”
so traditional CGMP standards are less
applicable.
Unique challenges of applying regulatory
science to software-intensive medical
devices.
Key takeaways
• Software has a well-defined systems
development lifecycle (SDLC). If
regulatory science is applied properly to
those lifecycle steps, SaMD solutions
have a chance of being even safer and
more compliant than their
software+hardware cousins.
• Consider creating an OKR-based
compliance strategy at your
organization.
• Deterministic reproducibility,
observability, and transparency
assurance through RegOps (regulatory
operations) tied to DevOps, DevSecOps,
DataOps, and SRE.

View Slide

www.netspective.com
5
https://www.fda.gov/media/145022/download https://www.fda.gov/media/80958/download

View Slide

www.netspective.com
6
Most
Popular
“AI” in
Healthcare
Beware of “AI
Washing” –
it’s all just
software
https://towardsdatascience.com/ethical-ai-its-implications-for-enterprise-ai-use-cases-and-governance-81602078f5db

View Slide

www.netspective.com
7
Software is
often non-
deterministic
Reproducible
reliability
difficult to
achieve.

View Slide

www.netspective.com
8
Most software is easy to build these days, especially
software driven by machine learning.
Challenge moves from building to testing.
https://insights.sei.cmu.edu/blog/the-challenges-of-testing-in-a-non-deterministic-world/

View Slide

www.netspective.com
9
Areas of non-
determinism
that must be
addressed

View Slide

www.netspective.com
10
Lack of AI “explainability” and auditability increase
challenges of deterministic reproducibility
A model agnostic explainability frameworks, Local Interpretable Model-Agnostic Explanations (LIME), is “a novel explanation technique
that explains the predictions of any classifier in an interpretable and faithful manner by learning an interpretable model locally around the
prediction.”
https://towardsdatascience.com/ethical-ai-its-implications-for-enterprise-ai-use-cases-and-governance-81602078f5db

View Slide

www.netspective.com
11
NASA Software
Safety Guidebook

View Slide

www.netspective.com
12

View Slide

www.netspective.com
13
FHIR-capable
systems (EHRs)
Non-HL7/FHIR
EHRs, RCMs, etc.
Government or any
Reporting Agencies
(CDC, CMS, FDA, et. al.)
RegOps Participant 1 (Manufacturer, Health System, HIE, Insurer, etc.)
RegOps Participant 2
RegOps Participant N
RegOps
Participant 3
RegOps
Outcomes
Observatory
HL7-capable
systems (EHRs)
Source
Databases
Any
Database
Multiple source systems
within a physician practice,
in a hospital department,
across a health system,
inside a payer, state health
departments, or an entire
HIE can generate
standardized Metrics and
Measures through existing
or custom RegOps
Exporters
RegOps
Gateways
Local
Dashboards,
Alerts, Reports
RegOps
Dashboards
Participants can join multiple bespoke
RegOps business networks over blockchain
or DNS-style chains
Publishing can be alerts
or documents
Medical Devices
(Class 1, 2, or 3)
Safety/Error
Reporting Systems
RegOps
Publishers
Specifications providers such as CMS, NQF, NCQA, FDA, and
others can create executable specifications that live in open
source or commercial exporters (actual software)
Given the time-series database
kept by RegOps, full claims
records and complete EHR
exchange is not required.
RegOps Reference Architecture
Everything we need is available as Open Source Software (OSS). RegOps approach allows regulatory artifacts to be
generated as byproducts of the SaMD development effort. Start from the real-world and look inward, not the other
way around.
Auto-generate
compliance artifacts

View Slide

www.netspective.com
14
Trend: Software is
eating medical
devices1
FDA intends to apply its regulatory oversight to only those
software functions that are medical devices and whose
functionality could pose a risk to a patient’s safety if the
device were to not function as intended.
FDA Policy for Device Software Functions and Mobile Medical Applications (September 2019)
1 https://a16z.com/2011/08/20/why-software-is-eating-the-world/
FDA says there are no question that these are regulated:
• Software functions that are an extension of one or more medical devices by
connecting to such device(s) for purposes of controlling the device(s) or analyzing
medical device data.
• Software functions (typically, mobile apps) that transform the mobile platform into a
regulated medical device by using attachments, display screens, or sensors or by
including functionalities similar to those of currently regulated medical devices.
• Software functions that become a regulated medical device by performing patient-
specific analysis and providing patient-specific diagnosis, or treatment
recommendations.

View Slide

www.netspective.com
15
Trend: Software is
increasingly
working together
What happens when software and systems that were
never tested together are forced to live and work together
in the “wild”?
What does “intended use” mean after integration?
FDA discretion as to whether these are regulated:
• Software functions that supplement professional clinical care by facilitating behavioral
change or coaching patients with specific diseases or identifiable health conditions in their
daily environment.
• Software that provides contextually-relevant information to users by matching patient-
specific information (e.g., diagnosis, treatments, allergies, signs, or symptoms) to reference
information routinely used in clinical practice (e.g., practice guidelines) to facilitate a user’s
assessment of a specific patient.
• Software functions that are specifically marketed to help patients communicate with
healthcare providers by supplementing or augmenting the data or information by capturing
an image for patients to convey to their healthcare providers about potential medical
conditions.
• Software functions that perform simple calculations routinely used in clinical practice.

View Slide

www.netspective.com
16
Trend: Software is
eating medical
devices1
FDA intends to apply its regulatory oversight to only those
software functions that are medical devices and whose
functionality could pose a risk to a patient’s safety if the
device were to not function as intended.
FDA Policy for Device Software Functions and Mobile Medical Applications (September 2019)
1 https://a16z.com/2011/08/20/why-software-is-eating-the-world/
FDA says many things are
unregulated, but many will be at
their discretion.
https://www.fda.gov/media/80958/download
Think of 510(k) and other
regulatory compliance as an
opportunity, not a cost.

View Slide

www.netspective.com
17
Trend: Compliance “Theatre”
We want to sound smart, so we talk about risk-based compliance but we’re no better than airport security theatre

View Slide

www.netspective.com
18
Example: Is the MDS2 Security Theatre?
Is this third-party risk-based or just making us feel good?
http://www.himss.org/resourcelibrary/MDS2

View Slide

www.netspective.com
19
https://atos.net/en/blog/applying-artificial-intelligence-in-existing-business-processes

View Slide

www.netspective.com
20
Compliance: often binary (yes/no)
Safety and security: must be continuous
You can be compliant and not safe, safe
but not compliant, or both
Compliant but unsafe is pretty common
Trend: Compliance does not lead to safety
Unless generated from real-world evidence, compliance leads to a false sense of security
Safe &

View Slide

www.netspective.com
21
There is a safe software development crisis,
especially with Machine Learning / AI.
Our software development lifecycles, languages, and tools are built
for deterministic environments with systems that stand alone, not
for fully integrated non-deterministic environments.
Safe development lifecycles and modern software supply chain techniques
must be implemented.
Building regulatory practices directly into the systems development
lifecycle could ease the inspection and audit burdens but create more
reliable results.

View Slide

www.netspective.com
22
Preparing annual controls catalogs and
compliance documentation or passing
audits doesn’t mean you’re safe.
Not enough organizations differentiate between point in time
assessments versus continuous monitoring.
Only continuous monitoring of each safety-critical asset,
from the bottom-up, ensures safety. This is what I call
RegOps – it’s an extension of DevOps, DevSecOps, and SRE.

View Slide

www.netspective.com
23
▪ concurrency defects:
▪ classic defects, such as deadlock, livelock, starvation, suspension, priority inversion, (data) race conditions,
order violations, atomicity violations, and lock and semaphore defects
▪ multicore defects, such as interference between cores due to sharing common resources (e.g., L2/L3 caches,
RAM and disc memory, I/O, system bus), improper allocation of software to hardware components, and
unacceptable jitter in processing times
▪ virtual machine defects, such as VM escapes and interference between VMs, improper allocation of SW to
VMs, and hypervisor defects
▪ performance defects that cause missed deadlines (latency and response time), unacceptable jitter, and incorrect
order of processing and events
▪ reliability defects that cause buffer overflow and bugs due to ill-timed automatic garbage collection
▪ robustness defects that cause missing, inadequate, or incorrect error, fault, failure, and environmental tolerance
▪ security defects including vulnerabilities and defects in security controls (e.g., incorrect configurations)
▪ rare alignments of cyclic behaviors that result in intermittent and non-reproducible failures
▪ positive feedback under stress propagated through a system's environment triggering showstoppers or
dangerous loss of control
▪ bizarre responses that are internally consistent but externally dangerous
Common defects in non-deterministic software

View Slide

www.netspective.com
24
WHAT IS THE
SOLUTION?
OKRs, ML, AUTOMATION
AUTO SCORING
AUTO ASSESSMENTS
DISTRIBUTED
COMPLIANCE
COLLABORATIVE
EVIDENCE

View Slide

www.netspective.com
25
Move from an oversight and validation regulatory
focus to a continuous assurance strategy
grounded in machine verifiable transparency and
observability.
The risk of a device and the excellence of a company should
be related to the transparency/opacity of its measurable
data which can easily be compared with other products /
companies.

View Slide

www.netspective.com
26
Move from an “intended use” reporting model to
one tied to “intended results” model built around
the concept of objectives and key results (OKRs).
Today we describe what a device does, but we should be
describing what it’s designed to accomplish for patients,
through objective measures that can be independently
verified.

View Slide

www.netspective.com
27
Digitize compliance into
structured data and integrate
with software engineering
THE NEXT CONSULTANT TO GIVE ADVICE ABOUT
YET ANOTHER DOCUMENT OR TEMPLATE SHOULD BE FIRED

View Slide

www.netspective.com
28
Use Objectives & Key Results (OKRs)
Setting quantifiable goals with objective measures should be the focus
INTEL CORPRORATE OBJECTIVE in 1980
Establish the 8086 as the highest
performance 16-bit microprocessor family, as
measured by:
KEY RESULTS (Q2 1980)
1. Develop and publish five benchmarks
showing superior 8086 family performance
(Applications).
2. Repackage the entire 8086 family of
products Marketing).
3. Get the 8MHz part into production
(Engineering, Manufacturing).
4. Sample the arithmetic coprocessor no later
than June 15 (Engineering).

View Slide

www.netspective.com
29
How OKRs would work in safety critical software
Setting measurables goals with objective measures should be the focus
Useful Software Safety Objective
Automatically notify device operator and cloud service desk when a software
exception is thrown.
Potential Key Results
1. All configuration checks are automatically performed, with violation findings recorded in
Jira.
2. At least five members of IT are trained to understand and act on violation findings by
March 15.
Useful Cybersecurity Objective
Reduce the likelihood of “A production AWS credential was exposed to the
public in Q3”.
Potential Key Results
1. Commits mentioning AWS_SECRET_KEY show up in the #security slack.
2. The photobackup pipeline will be moved to an AWS role.
3. Complete Security Monkey alerting pipeline towards our detection on-call.
4. Complete a before & after forecast, and CloudTrail hunt.
https://medium.com/starting-up-security/how-to-measure-risk-with-a-better-okr-c259bccf359e

View Slide

www.netspective.com
30
Good Safety Critical Professionals use OKRs
Tie your safety and compliance goals with corporate or agency goals and make your bosses sign off
https://medium.com/starting-up-security/how-to-measure-risk-with-a-better-okr-c259bccf359e
https://community.isc2.org/t5/Tech-Talk/OKR-s-for-a-Security-Team/td-p/16048

View Slide

www.netspective.com
31
If you’re not sure, start with FAIR
FAIR, short for “Factor Analysis of Information Risk,” is an international standard quantitative model for
operational risk.

View Slide

www.netspective.com
32
Asset management
–Requirements management
–Procurement authority
–Intake procedures
–Asset ownership/custodian gap
–Charging & storage locations
Operations
–Tracking MD by end users
–Disposal procedures
Maintenance management
–Patch management
–Alerts
Access management
–Unauthorized remote access
–Unauthorized local access
Loss/theft
–Audit logs w/user history
Incident management
–Incident reporting
–Incident response
–Forensics capabilities
System engineering process
–Software vulnerabilities
oSecure software development
oCommercial Operating Systems
oOpen source software
–Hardware vulnerabilities
oPatching limitations
oHardcoding generic credentials
oEasily accessible USB ports on portable
devices
Software maintenance process
–Secure coding environment
oSeparation of Dev & Ops
oCode test & validation
oCMDB
–Patch distribution process
oPatch integrity checks
Cloud operations
–Access controls
oSeparation of Dev & Ops
–Supply chain
oCode test & validation
Insider threats
Supply Chain
Supply Chain
Physical
Insider
LAN
OS
Data
Are current frameworks & tools useful?
Source: Cynergistek
Development Risks (Manufacturers) Production Risks (Manufacturers & Customers)
How are things supposed to improve when each manufacturer is independent, but customer risks are collaborative?

View Slide

www.netspective.com
33
https://securityintelligence.com/how-stix-taxii-and-cybox-can-help-with-standardizing-threat-information/
http://www.redzonetech.net/podcast/aharon-chernin/
Structured threat sharing is a must
Trusted Automated Exchange of Indicator Information (TAXII)
Cyber Observable Expression (CybOX)
Structured Threat Information Expression (STIX)

View Slide

www.netspective.com
34
Secure the
Software Supply
Chain
• Identify all the stuff in your pipeline; from people, code,
dependencies, to infrastructure
• Ensure a consistent and quality build process
• Protect the product while in storage and transit
• Guarantee and validate the final product at delivery against
a bill of materials
https://m-square.com.au/securing-the-enterprise-software-supply-chain-using-docker/

View Slide

www.netspective.com
35
Let’s reimagine
compliance so it’s focused
on continuous assurance
of reproducible reliability

View Slide

www.netspective.com
36
Requirements
OKRs
Structured, machine and human readable, machine computable, and machine
assurable requirements.
• It’s not a set of requirements we’re after but after a hierarchical set of objectives
that the device is designed to achieve.
• Need to be able to compare multiple devices using similar definitions (if two
devices are looking to accomplish the same thing, we should be able to specify
that).

View Slide

www.netspective.com
37
Objectives – what the
device will accomplish
(not what it does)
•
Explicit
•
Qualitative
•
Address a target user
•
Align with the manufacturer’s long
term mission, KPIs, product vision,
and business goals
•
Evaluated for efficacy during every
product release cycle
Key Results – how the
accomplishment is judged
•
Specific
•
Quantitative
•
Instrumentable
•
Measurable
•
No change for disagreement when
achieved
•
Explicitly define the metric by which
they will be monitored (e.g.,
OpenMetrics exposition format)
•
Accompanied by a monitoring
dashboard (e.g., open source JSON
definition)

View Slide

www.netspective.com
38
Opaque ➔ Transparent
Instrumented
Structured, machine and human readable, machine
verifiable, model driven development for engineering
(AADL, Architecture Analysis & Design Language and
contract driven programming).
FDA and regulators should be able to test the design
of a device by executing its models.

View Slide

www.netspective.com
39
Discrete ➔ Continuous
Migrate from SDLC and quality systems-based
evaluation to deterministic reproducibility through
machine veriable structured definitions and models
(machines can verify cybersecurity, requirements,
design, architecture, QA, and artifacts).
• Measure the outcome of a process, not the
process (SDLC) itself
• FDA and regulators should be able to run
simulators automatically without any custom
work

View Slide

www.netspective.com
40
Reportable ➔ Actionable
Live, real-world feedback
for continuous assurance

View Slide

www.netspective.com
41
Consensus-driven
Data-driven
Observable Telemetry
and Metrics

View Slide

www.netspective.com
42
Every suggestion being made is
already being done in production
across many, even non-safe-critical,
software systems
What’s our excuse?

View Slide

www.netspective.com
43

View Slide

www.netspective.com
44

View Slide

www.netspective.com
45
Eminence-driven
Evidence-driven
Why do we need to wait
for research papers?
Ivory tower
Real-world

View Slide

www.netspective.com
46
Compliance ➔ Value
Provable patient safety
Not conjecture BS

View Slide

www.netspective.com
47
“Paper” ➔ Digital Native
Microsoft Word ➔ Markdown
Microsoft Excel ➔ JSON or XML
Manual Reviews ➔ Instrumentation APIs
Test Plans ➔ Machine Assurance

View Slide

www.netspective.com
48
Human ➔ Machine Generated
Structured, machine and human readable, machine
computable, and machine assurable QA metrics (OSS:
OpenMetrics, OpenTelemetry style).
• Need to specify what’s experimental, what’s
confirmation, and what’s observed
• FDA and government bodies should be able to receive
anonymous telemetry with ability reidentify when
necessary

View Slide

www.netspective.com
49
Retrospective reporting
Interactive telemetry
EVENT DRIVEN SDKs

View Slide

www.netspective.com
50
Institution-framed
Patient-framed

View Slide

www.netspective.com
51
Population-based
Personalized

View Slide

www.netspective.com
52
Measurement
Process Improvement
Continuous Assurance

View Slide

www.netspective.com
53
Every suggestion being made is
already being done in production
across many, even non-safe-critical,
software systems
What’s our excuse?

View Slide

www.netspective.com
54
What we learned
What we will discuss
FDA is looking to innovate around how
digital therapeutic solutions are regulated
(“Software as a Medical Device”, SaMD).
Software systems are not “manufactured”
so traditional CGMP standards are less
applicable.
Unique challenges of applying regulatory
science to software-intensive medical
devices.
Key takeaways
• Software has a well-defined systems
development lifecycle (SDLC). If
regulatory science is applied properly to
those lifecycle steps, SaMD solutions
have a chance of being even safer and
more compliant than their
software+hardware cousins.
• Consider creating an OKR-based
compliance strategy at your
organization.
• Deterministic reproducibility,
observability, and transparency
assurance through RegOps (regulatory
operations) tied to DevOps, DevSecOps,
DataOps, and SRE.

View Slide

www.netspective.com
55
Quantitative Regulatory Science
(“RegOps”) for FDA Digital
Health/Therapeutics Compliance
This Photo by Unknown Author is licensed under CC BY
THANK YOU
Shahid N. Shah ([email protected])
@ShahidNShah

View Slide

Quantitative Regulatory Science (RegOps) for Software Intensive Medical Devices and SaMD

Quantitative Regulatory Science (RegOps) for Software Intensive Medical Devices and SaMD

Shahid N. Shah

More Decks by Shahid N. Shah

Featured

Transcript