Functional validation means that a client presents to a prospective service provider a sequence of challenges. The service provider replies to these challenges with corresponding answers. Only after the client is satisfied that the service provider’s answers are consistent with the client’s expectations is an actual commitment made to using the service.
Challenge the service provider with some test cases x1, x2, ..., xk . The remote service provider R offers the corresponding answers fR(x1), fR(x2), ..., fR(xk). The client C may or may not have independent access to the answers fC(x1), fC(x2), ..., fC(xk).
Possible situations and machine learning models:
C “knows” fC(x) and R provides fR(x).
PAC learning and Chernoff bounds theory
C “knows” fC(x) and R does not provide fR(x).
C does not “know” fC(x) and R provides fR(x).
Simulation-based learning and reinforcement learning