June 28, 2023 Werner Keil Jakarta Data Standardized Data Access for SQL and NoSQL Databases 

Werner Keil Jakarta EE Specification Committee Member Let’s meet @emilyfhjiang @wernerwedge Emily Jiang Cloud Native Architect, IBM 

3 5 trends in 2023 

New digital trends create new technical challenges @emilyfhjiang @wernerwedge 

What are these challenges? @emilyfhjiang @wernerwedge • Agile development process • High performance and availability • Manage huge data volumes 

NoSQL FTW! 

Advantages of NoSQL @emilyfhjiang @wernerwedge • Handles large volumes of data at high-speed • Stores unstructured, semi-structured, or structured data • Easy updates to schemas and fields • Developer-friendly • Takes advantage of the cloud to deliver zero downtime 

Why this talk Tons of persistence frameworks Which one performs best for your case? JVM can cope with heavy loads Why is there no common standard? 

NoSQL 01 Database 02 Doesn't use (semi)structure 03 No transactions 04 BASE Five different types 05 

Key-Value stores AmazonS3 Apollo Ares Aphrodite Sun War Love Beauty AmazonDynamo Hazelcast Redis 

Column-Oriented Apollo Aphrodite Ares Kratos Duty Duty Duty Dead Gods Love, happy Sun War 13 Color weapon Sword Row-key Columns HBase Scylla SimpleDB Cassandra DynamoDB Clouddata 

Document stores { "name":"Diana", "duty":[ "Hunt", "Moon", "Nature" ], "siblings":{ "Apollo":"brother" } } ApacheCouchDB MongoDB Couchbase 

Graph databases Apollo Ares Kratos was killed by was killed by killed killed Neo4j InfoGrid Sones HyperGraphDB 

Time series InfluxDB KairosDB TimescaleDB 

Multi-Model 01 02 03 04 OrientDB (graph, document) Couchbase (key value, document) Elasticsearch (document, graph) ArangoDB (document, graph, key-value) 

SQL vs NoSQL SQL KEY-VALUE COLUMN DOCUMENTS GRAPH Table Bucket Column family Collection Row Key/value pair Column Documents Vertex Column Key/value pair Key/value pair Vertex and Edge property Relationship Link Edge 

BASE vs ACID • Basically Available • Soft state • Eventual consistency • Atomicity • Consistency • Isolation • Durability 

CAP 

Scalability vs Complexity Scalability Complexity key-value Column Document Graph 

Relational Application NoSQL Application Logic Tier Logic Tier DAO DAO JPA JPA JPA JPA JDBC JDBC JDBC JDBC Data Tier API API API Data Tier 

Our comparison model 

JPA problem for NoSQL 01 02 03 04 05 06 Saves Async Async Callback Time to Live (TTL) Consistency Level SQL based Diversity in NoSQL 

Annotated Entities 01 02 03 Mapped Superclass Entity Column @Entity(”person") public class Person { @Column private String name; @Column private long age; @Column private Set powers; } 

Template Person artemis = ...; DocumentTemplate template = … template.insert(artemis); template.update(artemis); DocumentQuery query = ... List people = template.select(query); 

Repository interface PeopleRepository extends MongoRepository { List findByNameAndAge (String name, Integer age); } 

Micronaut Data NoSQL 

Repository @MongoRepository interface BookRepository extends CrudRepository { Book find(String title); } Or @MongoRepository public abstract class AbstractBookRepository implements CrudRepository { public abstract List findByTitle(String title); } 

Entity @MappedEntity public class Book { @Id @GeneratedValue private ObjectId id; private String title; private int pages; public Book(String title, int pages) { this.title = title; this.pages = pages; } // ... } 

Query by Text @MongoFindQuery(filter = "{title:{$regex: :t}}", sort = "{title: 1}") List customFind(String t); @MongoAggregateQuery("[{$match: {name:{$regex: :t}}}, {$sort: {name: 1}}, {$project: {name: 1}}]") List customAggregate(String t); @MongoUpdateQuery(filter = "{title:{$regex: :t}}", update = "{$set:{name: 'tom'}}") List customUpdate(String t); @MongoDeleteQuery(filter = "{title:{$regex: :t}}", collation = "{locale:'en_US', numericOrdering:true}") void customDelete(String t); 

GORM NoSQL 

Entity @Entity class User { ObjectId id String emailAddress String password String fullname Date dateCreated Date lastUpdated static constraints = { emailAddress email: true password nullable: true fullname blank: false } }} 

@Document(collection = ”people") public class person { … } interface GodRepository extends MongoRepository { … } What about the Controller? org.springframework.boot spring-boot-starter-data-mongodb spring.data.mongodb.database=mythology spring.data.mongodb.port=27017 Logistics Domain 

Our example 

MovieEntity MovieRepository PersonEntity Roles … Other reactive dependencies … org.neo4j.springframework.data spring-data-neo4j-rx-spring-boot-starter 1.1.1 org.neo4j.driver.uri=bolt://localhost:7687 org.neo4j.driver.authentication.username=neo4j org.neo4j.driver.authentication.password=secret Logistics Domain 

Repository public interface MovieRepository extends ReactiveNeo4jRepository { Mono findOneByTitle(String title); } 

Entities @Node("Person") public class PersonEntity { @Id private final String name; private final Integer born; } @Node("Movie") public class MovieEntity { @Id private final String title; @Property("tagline") private final String description; @Relationship(type = "ACTED_IN", direction = INCOMING) private Map actorsAndRoles = new HashMap<>(); @Relationship(type = "DIRECTED", direction = INCOMING) private List directors = new ArrayList<>(); } 

Relationships @RelationshipProperties public class Roles { private final List roles; public Roles(List roles) { this.roles = roles; } } 

Quarkus NoSQL 

Entity public class Person extends PanacheMongoEntity { public String name; public LocalDate birthDate; public Status status; // return name as uppercase in the model public String getName(){ return name.toUpperCase(); } // store all names in lowercase in the DB public void setName(String name){ this.name = name.toLowerCase(); } } 

Repository @ApplicationScoped public class PersonRepository implements PanacheMongoRepository { // put your custom logic here as instance methods public Person findByName(String name){ return find("name", name).firstResult(); } public List findAlive(){ return list("status", Status.ALIVE); } public void deleteLoics(){ delete("name", "Loïc"); } } 

Motivation BaseDocument baseDocument = new BaseDocument(); baseDocument.addAttribute(name, value); Document document = new Document(); document.append(name, value); JsonObject jsonObject = JsonObject.create(); jsonObject.put(name, value); ODocument document = new ODocument(“collection”); document.field(name, value); 

Make NoSQL easier @Inject Template template; … template.insert(book); List books = template.select(Book.class) .where("title").eq("Effective Java").list(); 

Now What? Jakarta Data is to unify the data access world! @emilyfhjiang @wernerwedge 

Jakarta Data NoSQL JPA Rest Client 

Jakarta Data relational database business logic repository interface repository interface Jakarta Data Provider NoSQL database Jakarta Data / Jakarta NoSQL Provider JPA or JDBC 

Jakarta Data Key Elements ENTITY REPOSITY Define methods to perform CRUD RESTful Service Performing CRUD 

Entity @Entity public class Person{ @Id private String id; @Column private String name; @Column private String city; } @Entity public record Book(@Id String id, @Column("title") String title, @Column("edition") int edition){} 

Entity @Entity public class Person{ @Id private String id; @Column private String name; @Column private String city; } key value key key key value value value Column Family Graph Document Key Value 

Template @Inject Template template; ... Car ferrari = template.insert(ferrari); Optional car = template.find(Car.class, 1L); List cars = template.select(Car.class).where("city").eq("Rome").result(); template.delete(Car.class).where("id").eq(1L).execute(); Optional result = template.singleResult("select * from Car where id = 1"); 

Templates Template DocumentTemplate KeyValueTemplate VendorTemplate VendorTemplate GraphTemplate ColumnTemplate 

Basic Entity and Repository ▪ DataRepository doesn’t come with any built-in methods. To further simplify… @Inject Products products; … products.save(new Product(1, "$25 gift card", 25.0f)); List found = products.findByPriceLessThan(100.0f); @Repository public interface Products extends DataRepository { List findByPriceLessThan(float maxPrice); void save(Product product); ... } public record Product( long id, String name, float price ); entity class key type 

Basic Entity and CRUD Repository @Inject Products products; … products.save(new Product(1, "$25 gift card", 25.0f)); List found = products.findByPriceLessThan(100.0f); @Repository public interface Products extends CrudRepository { List findByPriceLessThan(float maxPrice); ... } public record Product( long id, String name, float price ); entity class key type save(E entity); saveAll(Iterable entities); findAll(); findById(K id); existsById(K id); delete(E entity); deleteById(K id); ... Inherited via CrudRepository: 

Pageable Repository @Repository public interface ProductRepository extends PageableRepository { Page findByTypeOrderByName(CarType type, Pageable pageable); } 

Naming Conventions Compose your own save, findBy, deleteBy, countBy & more methods by following precise naming conventions with reserved keywords and entity property names within the method name. Product[] findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice); find...By indicates a query returning results And keyword separates NameLike and PriceBetween conditions Name and Price are entity property names Like keyword is a type of condition, requires 1 parameter Between keyword is a type of condition, requires 2 parameters 

Repository with Queries Some queries are too complex to be defined within a method name. @Query gives you a way to supply a query written in: JPQL (for Jakarta Persistence-based providers) @Repository public interface Products extends CrudRepository { @Query("UPDATE Product o SET o.price = o.price - o.price * ?1") int discountAll(float discountRate); } 

Named Parameters If you prefer to use named parameters, you can do that with @Param, @Repository public interface Products extends CrudRepository { @Query("UPDATE Product o SET o.price = o.price - o.price * :rate") int discountAll(@Param("rate") float discountRate); } 

Sorting of Results Reserved keywords OrderBy, Asc, and Desc enable sorting of results. Product[] findByNameLikeAndPriceBetweenOrderByPriceDescNameAsc(String namePattern, float minPrice, float maxPrice); OrderBy keyword indicates the start of the sorting criteria Asc keyword indicates ascending sort Desc keyword indicates descending sort 

Sorting of Results – better ways @OrderBy(value = "price", descending = true) @OrderBy("name") Product[] findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice); Method names can get a bit lengthy, so there is also @OrderBy annotation for sorting criteria that is known in advance Sort parameters for dynamic sorting Product[] findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice, Sort...); found = products.findByNameLikeAndPriceBetween(namePattern, 10.00f, 20.00f, Sort.desc("price"), Sort.asc("name")); 

Without Method Name Magic? @Filter(by = "price", op = Compare.Between) @Filter(by = "name", fn = Function.IgnoreCase, op = Compare.Contains) @OrderBy("price") Product[] inPriceRange(float min, float max, String namePattern); @Filter(by = "name") @Update(attr = "price", op = Operation.Multiply) boolean inflatePrice(String productName, float rate); @Delete @Filter(by = "reviews", op = Compare.Empty) int removeUnreviewed(); It could be possible to define queries entirely with annotations. • This idea was deferred to post v1.0, but Open Liberty has it working in a prototype: Javadoc: https://ibm.biz/JakartaData 

Limiting the Number of Results @OrderBy("price") Product[] findFirst10ByNameLike(String namePattern); Sometimes you don’t want to read the entire matching dataset and only care about the first several results. First keyword indicates the start of the sorting criteria 10 numeric value optionally indicates how many. When absent, only the very first result is returned. Another way: @OrderBy("price") Product[] findByNameLike(String namePattern, Limit limit); found = products.findByNameLike(namePattern, Limit.of(10)); 

Offset Pagination @OrderBy("price") @OrderBy("name") Page findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice, Pageable pagination); For large datasets, you can read data in pages, defined by the Pageable parameter, Offset pagination is convenient to users jumping multiple pages ahead or behind. But it’s inefficient in making the database fetch unwanted results, and if data is modified, some results might be missed or duplicated between pages! for (Pageable p = Pageable.ofSize(50); p != null; ) { Page page = products.findByNameLikeAndPriceBetween(pattern, 40.0f, 60.0f, p); ... p = page.nextPageable(); } 

Keyset Pagination Reduces scenarios where results are missed or duplicated across pages. • Entity properties that serve as the sort criteria must not be modified. Gives the Pageable awareness of cursor position from a prior page. • Jakarta Data provider automatically adds conditions to the query making the previous cursor the starting point for the next page. Can be more efficient because it does not require fetching and skipping large numbers of results. Unwanted results never match to begin with! 

Keyset Pagination Examples @OrderBy("lastName") @OrderBy("id") KeysetAwarePage findByHoursWorkedGreaterThanEqual(int minHours, Pageable pagination); Traversing all results, Or relative to a specific position, for (Pageable p = Pageable.ofSize(100); p != null; ) { KeysetAwarePage page = employees.findByHoursWorkedGreaterThanEqual(1500, p); ... p = page.nextPageable(); } Pageable p = Pageable.ofSize(100).afterKeyset(employee.lastName, employee.id); page = employees.findByHoursWorkedGreaterThanEqual(1500, p); Order of keyset keys matches the order of sort criteria 

Keyset Pagination – How it Works @OrderBy("lastName") @OrderBy("firstName") @OrderBy("id") KeysetAwarePage findByHoursWorkedGreaterThanEqual(int minHours, Pageable pagination); Let’s visualize what this could look like if transformed to JPQL: SELECT o FROM Employee o WHERE (o.hoursWorked >= ?1) AND ( (o.lastName > ?2) OR (o.lastName = ?2 AND o.firstName > ?3) OR (o.lastName = ?2 AND o.firstName = ?3 AND o.id > ?4) ) pagination.getKeysetCursor() provides the values for ?2, ?3, ?4 

Demo Time @emilyfhjiang @wernerwedge https://github.com/OpenLiberty/sample-jakarta-data 

Demo Entity: CrewMember Repository: CrewMembers CRUD Service CrewService @Inject CrewMembers crewMembers; … @DELETE @Path("/{id}") public String remove(@PathParam("id") String id) { crewMembers.deleteByCrewID(id); return ""; } @Repository public interface CrewMembers extends DataRepository { @OrderBy("crewID") List findAll(); void deleteByCrewID(String crewID); void save(CrewMember a); } @Entity public class CrewMember { @NotEmpty(message = "All crew members must have a name!") private String name; @Pattern(regexp = "(Captain|Officer|Engineer)", message = "Crew member must be one of the listed ranks!") private String rank; @Id @Pattern(regexp = "^\\d+$", message = "ID Number must be a non-negative integer!") private String crewID; 

▪ GitHub repositories ▪ https://github.com/OpenLiberty/sample-jakarta-data ▪ OpenLiberty Beta ▪ https://openliberty.io/blog/2023/05/16/23.0.0.5-beta.html ▪ Specifications ▪ Jakarta Data: https://jakarta.ee/specifications/data/ @ wernerwedge Links @emilyfhjiang 

• Flaticon.com • Michael Simons • Jean Tessier • Teo Bais • Nathan Rauh • Mark Swatosh • Otavio Santana Credits @emilyfhjiang @wernerwedge 

Thank You @emilyfhjiang @wernerwedge