DWX23: Jakarta Data – Standardized Data Access for SQL and NoSQL Databases

The amount of data collected by applications nowadays is growing rapidly. Many of them need to support both relational SQL and non-relational NoSQL databases. Jakarta Data provides an API to allow easy data access. Developers can split the persistence mechanism and the model using common features like the Repository pattern and seamlessly switch between SQL and NoSQL databases or even use both in the same application.

First we'll compare popular Java data access frameworks for SQL/JPA and NoSQL databases. We are going to look at Jakarta NoSQL/JNOSQL, Spring Data, Micronaut Data, Quarkus/Panache or DeltaSpike Data. What do they have in common? How do they compare, what are the differences and similarities? We'll show each of them with a selection of different NoSQL or SQL databases to learn how they work and what concepts like CRUD operations, the Repository pattern, pagination or sorting they support - followed by an overview of Jakarta Data and how it can help migrate your data applications to a vendor-neutral future, becoming greater than the sum of all existing solutions.

Transcript

1. June 28, 2023 Werner Keil Jakarta Data Standardized Data Access for SQL and NoSQL Databases
2. Werner Keil Jakarta EE Specification Committee Member Let’s meet @emilyfhjiang @wernerwedge Emily Jiang Cloud Native Architect, IBM
3. 3 5 trends in 2023
4. New digital trends create new technical challenges @emilyfhjiang @wernerwedge
5. What are these challenges? @emilyfhjiang @wernerwedge • Agile development process • High performance and availability • Manage huge data volumes
6. NoSQL FTW!
7. Advantages of NoSQL @emilyfhjiang @wernerwedge • Handles large volumes of data at high-speed • Stores unstructured, semi-structured, or structured data • Easy updates to schemas and fields • Developer-friendly • Takes advantage of the cloud to deliver zero downtime
8. Why this talk Tons of persistence frameworks Which one performs best for your case? JVM can cope with heavy loads Why is there no common standard?
9. NoSQL 01 Database 02 Doesn't use (semi)structure 03 No transactions 04 BASE Five different types 05
10. Key-Value stores AmazonS3 Apollo Ares Aphrodite Sun War Love Beauty AmazonDynamo Hazelcast Redis
11. Column-Oriented Apollo Aphrodite Ares Kratos Duty Duty Duty Dead Gods Love, happy Sun War 13 Color weapon Sword Row-key Columns HBase Scylla SimpleDB Cassandra DynamoDB Clouddata
12. Document stores { "name":"Diana", "duty":[ "Hunt", "Moon", "Nature" ], "siblings":{ "Apollo":"brother" } } ApacheCouchDB MongoDB Couchbase
13. Graph databases Apollo Ares Kratos was killed by was killed by killed killed Neo4j InfoGrid Sones HyperGraphDB
14. Time series InfluxDB KairosDB TimescaleDB
15. Multi-Model 01 02 03 04 OrientDB (graph, document) Couchbase (key value, document) Elasticsearch (document, graph) ArangoDB (document, graph, key-value)
16. SQL vs NoSQL SQL KEY-VALUE COLUMN DOCUMENTS GRAPH Table Bucket Column family Collection Row Key/value pair Column Documents Vertex Column Key/value pair Key/value pair Vertex and Edge property Relationship Link Edge
17. BASE vs ACID • Basically Available • Soft state • Eventual consistency • Atomicity • Consistency • Isolation • Durability
18. CAP
19. Scalability vs Complexity Scalability Complexity key-value Column Document Graph
20. Relational Application NoSQL Application Logic Tier Logic Tier DAO DAO JPA JPA JPA JPA JDBC JDBC JDBC JDBC Data Tier API API API Data Tier
21. Our comparison model
22. JPA problem for NoSQL 01 02 03 04 05 06 Saves Async Async Callback Time to Live (TTL) Consistency Level SQL based Diversity in NoSQL
23. Annotated Entities 01 02 03 Mapped Superclass Entity Column @Entity(”person") public class Person { @Column private String name; @Column private long age; @Column private Set<String> powers; }
24. Template Person artemis = ...; DocumentTemplate template = … template.insert(artemis); template.update(artemis); DocumentQuery query = ... List<Person> people = template.select(query);
25. Repository interface PeopleRepository extends MongoRepository<Person, String> { List<Person> findByNameAndAge (String name, Integer age); }
26. Micronaut Data NoSQL
27. Repository @MongoRepository interface BookRepository extends CrudRepository<Book, ObjectId> { Book find(String title); } Or @MongoRepository public abstract class AbstractBookRepository implements CrudRepository<Book, ObjectId> { public abstract List<Book> findByTitle(String title); }
28. Entity @MappedEntity public class Book { @Id @GeneratedValue private ObjectId id; private String title; private int pages; public Book(String title, int pages) { this.title = title; this.pages = pages; } // ... }
29. Query by Text @MongoFindQuery(filter = "{title:{$regex: :t}}", sort = "{title: 1}") List<Book> customFind(String t); @MongoAggregateQuery("[{$match: {name:{$regex: :t}}}, {$sort: {name: 1}}, {$project: {name: 1}}]") List<Person> customAggregate(String t); @MongoUpdateQuery(filter = "{title:{$regex: :t}}", update = "{$set:{name: 'tom'}}") List<Book> customUpdate(String t); @MongoDeleteQuery(filter = "{title:{$regex: :t}}", collation = "{locale:'en_US', numericOrdering:true}") void customDelete(String t);
30. GORM NoSQL
31. Entity @Entity class User { ObjectId id String emailAddress String password String fullname Date dateCreated Date lastUpdated static constraints = { emailAddress email: true password nullable: true fullname blank: false } }}
32. @Document(collection = ”people") public class person { … } interface GodRepository extends MongoRepository<Person, String> { … } What about the Controller? <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-mongodb</artifactId> </dependency> spring.data.mongodb.database=mythology spring.data.mongodb.port=27017 Logistics Domain
33. Our example
34. MovieEntity MovieRepository PersonEntity Roles … Other reactive dependencies … <dependency> <groupId>org.neo4j.springframework.data</groupId> <artifactId>spring-data-neo4j-rx-spring-boot-starter</artifactId> <version>1.1.1</version> </dependency> org.neo4j.driver.uri=bolt://localhost:7687 org.neo4j.driver.authentication.username=neo4j org.neo4j.driver.authentication.password=secret Logistics Domain
35. Repository public interface MovieRepository extends ReactiveNeo4jRepository<MovieEntity, String> { Mono<MovieEntity> findOneByTitle(String title); }
36. Entities @Node("Person") public class PersonEntity { @Id private final String name; private final Integer born; } @Node("Movie") public class MovieEntity { @Id private final String title; @Property("tagline") private final String description; @Relationship(type = "ACTED_IN", direction = INCOMING) private Map<PersonEntity, Roles> actorsAndRoles = new HashMap<>(); @Relationship(type = "DIRECTED", direction = INCOMING) private List<PersonEntity> directors = new ArrayList<>(); }
37. Relationships @RelationshipProperties public class Roles { private final List<String> roles; public Roles(List<String> roles) { this.roles = roles; } }
38. Quarkus NoSQL
39. Entity public class Person extends PanacheMongoEntity { public String name; public LocalDate birthDate; public Status status; // return name as uppercase in the model public String getName(){ return name.toUpperCase(); } // store all names in lowercase in the DB public void setName(String name){ this.name = name.toLowerCase(); } }
40. Repository @ApplicationScoped public class PersonRepository implements PanacheMongoRepository<Person> { // put your custom logic here as instance methods public Person findByName(String name){ return find("name", name).firstResult(); } public List<Person> findAlive(){ return list("status", Status.ALIVE); } public void deleteLoics(){ delete("name", "Loïc"); } }
41. Motivation BaseDocument baseDocument = new BaseDocument(); baseDocument.addAttribute(name, value); Document document = new Document(); document.append(name, value); JsonObject jsonObject = JsonObject.create(); jsonObject.put(name, value); ODocument document = new ODocument(“collection”); document.field(name, value);
42. Make NoSQL easier @Inject Template template; … template.insert(book); List<Book> books = template.select(Book.class) .where("title").eq("Effective Java").list();
43. Now What? Jakarta Data is to unify the data access world! @emilyfhjiang @wernerwedge
44. Jakarta Data NoSQL JPA Rest Client
45. Jakarta Data relational database business logic repository interface repository interface Jakarta Data Provider NoSQL database Jakarta Data / Jakarta NoSQL Provider JPA or JDBC
46. Jakarta Data Key Elements ENTITY REPOSITY Define methods to perform CRUD RESTful Service Performing CRUD
47. Entity @Entity public class Person{ @Id private String id; @Column private String name; @Column private String city; } @Entity public record Book(@Id String id, @Column("title") String title, @Column("edition") int edition){}
48. Entity @Entity public class Person{ @Id private String id; @Column private String name; @Column private String city; } key value key key key value value value Column Family Graph Document Key Value
49. Template @Inject Template template; ... Car ferrari = template.insert(ferrari); Optional<Car> car = template.find(Car.class, 1L); List<Car> cars = template.select(Car.class).where("city").eq("Rome").result(); template.delete(Car.class).where("id").eq(1L).execute(); Optional<Car> result = template.singleResult("select * from Car where id = 1");
50. Templates Template DocumentTemplate KeyValueTemplate VendorTemplate VendorTemplate GraphTemplate ColumnTemplate
51. Basic Entity and Repository ▪ DataRepository doesn’t come with any built-in methods. To further simplify… @Inject Products products; … products.save(new Product(1, "$25 gift card", 25.0f)); List<Product> found = products.findByPriceLessThan(100.0f); @Repository public interface Products extends DataRepository<Product, Long> { List<Product> findByPriceLessThan(float maxPrice); void save(Product product); ... } public record Product( long id, String name, float price ); entity class key type
52. Basic Entity and CRUD Repository @Inject Products products; … products.save(new Product(1, "$25 gift card", 25.0f)); List<Product> found = products.findByPriceLessThan(100.0f); @Repository public interface Products extends CrudRepository<Product, Long> { List<Product> findByPriceLessThan(float maxPrice); ... } public record Product( long id, String name, float price ); entity class key type save(E entity); saveAll(Iterable<E> entities); findAll(); findById(K id); existsById(K id); delete(E entity); deleteById(K id); ... Inherited via CrudRepository:
53. Pageable Repository @Repository public interface ProductRepository extends PageableRepository<Product, Long> { Page<Car> findByTypeOrderByName(CarType type, Pageable pageable); }
54. Naming Conventions Compose your own save, findBy, deleteBy, countBy & more methods by following precise naming conventions with reserved keywords and entity property names within the method name. Product[] findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice); find...By indicates a query returning results And keyword separates NameLike and PriceBetween conditions Name and Price are entity property names Like keyword is a type of condition, requires 1 parameter Between keyword is a type of condition, requires 2 parameters
55. Repository with Queries Some queries are too complex to be defined within a method name. @Query gives you a way to supply a query written in: JPQL (for Jakarta Persistence-based providers) @Repository public interface Products extends CrudRepository<Product, Long> { @Query("UPDATE Product o SET o.price = o.price - o.price * ?1") int discountAll(float discountRate); }
56. Named Parameters If you prefer to use named parameters, you can do that with @Param, @Repository public interface Products extends CrudRepository<Product, Long> { @Query("UPDATE Product o SET o.price = o.price - o.price * :rate") int discountAll(@Param("rate") float discountRate); }
57. Sorting of Results Reserved keywords OrderBy, Asc, and Desc enable sorting of results. Product[] findByNameLikeAndPriceBetweenOrderByPriceDescNameAsc(String namePattern, float minPrice, float maxPrice); OrderBy keyword indicates the start of the sorting criteria Asc keyword indicates ascending sort Desc keyword indicates descending sort
58. Sorting of Results – better ways @OrderBy(value = "price", descending = true) @OrderBy("name") Product[] findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice); Method names can get a bit lengthy, so there is also @OrderBy annotation for sorting criteria that is known in advance Sort parameters for dynamic sorting Product[] findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice, Sort...); found = products.findByNameLikeAndPriceBetween(namePattern, 10.00f, 20.00f, Sort.desc("price"), Sort.asc("name"));
59. Without Method Name Magic? @Filter(by = "price", op = Compare.Between) @Filter(by = "name", fn = Function.IgnoreCase, op = Compare.Contains) @OrderBy("price") Product[] inPriceRange(float min, float max, String namePattern); @Filter(by = "name") @Update(attr = "price", op = Operation.Multiply) boolean inflatePrice(String productName, float rate); @Delete @Filter(by = "reviews", op = Compare.Empty) int removeUnreviewed(); It could be possible to define queries entirely with annotations. • This idea was deferred to post v1.0, but Open Liberty has it working in a prototype: Javadoc: https://ibm.biz/JakartaData
60. Limiting the Number of Results @OrderBy("price") Product[] findFirst10ByNameLike(String namePattern); Sometimes you don’t want to read the entire matching dataset and only care about the first several results. First keyword indicates the start of the sorting criteria 10 numeric value optionally indicates how many. When absent, only the very first result is returned. Another way: @OrderBy("price") Product[] findByNameLike(String namePattern, Limit limit); found = products.findByNameLike(namePattern, Limit.of(10));
61. Offset Pagination @OrderBy("price") @OrderBy("name") Page<Product> findByNameLikeAndPriceBetween(String namePattern, float minPrice, float maxPrice, Pageable pagination); For large datasets, you can read data in pages, defined by the Pageable parameter, Offset pagination is convenient to users jumping multiple pages ahead or behind. But it’s inefficient in making the database fetch unwanted results, and if data is modified, some results might be missed or duplicated between pages! for (Pageable p = Pageable.ofSize(50); p != null; ) { Page<product> page = products.findByNameLikeAndPriceBetween(pattern, 40.0f, 60.0f, p); ... p = page.nextPageable(); }
62. Keyset Pagination Reduces scenarios where results are missed or duplicated across pages. • Entity properties that serve as the sort criteria must not be modified. Gives the Pageable awareness of cursor position from a prior page. • Jakarta Data provider automatically adds conditions to the query making the previous cursor the starting point for the next page. Can be more efficient because it does not require fetching and skipping large numbers of results. Unwanted results never match to begin with!
63. Keyset Pagination Examples @OrderBy("lastName") @OrderBy("id") KeysetAwarePage<Employee> findByHoursWorkedGreaterThanEqual(int minHours, Pageable pagination); Traversing all results, Or relative to a specific position, for (Pageable p = Pageable.ofSize(100); p != null; ) { KeysetAwarePage<Employee> page = employees.findByHoursWorkedGreaterThanEqual(1500, p); ... p = page.nextPageable(); } Pageable p = Pageable.ofSize(100).afterKeyset(employee.lastName, employee.id); page = employees.findByHoursWorkedGreaterThanEqual(1500, p); Order of keyset keys matches the order of sort criteria
64. Keyset Pagination – How it Works @OrderBy("lastName") @OrderBy("firstName") @OrderBy("id") KeysetAwarePage<Employee> findByHoursWorkedGreaterThanEqual(int minHours, Pageable pagination); Let’s visualize what this could look like if transformed to JPQL: SELECT o FROM Employee o WHERE (o.hoursWorked >= ?1) AND ( (o.lastName > ?2) OR (o.lastName = ?2 AND o.firstName > ?3) OR (o.lastName = ?2 AND o.firstName = ?3 AND o.id > ?4) ) pagination.getKeysetCursor() provides the values for ?2, ?3, ?4
65. Demo Time @emilyfhjiang @wernerwedge https://github.com/OpenLiberty/sample-jakarta-data
66. Demo Entity: CrewMember Repository: CrewMembers CRUD Service CrewService @Inject CrewMembers crewMembers; … @DELETE @Path("/{id}") public String remove(@PathParam("id") String id) { crewMembers.deleteByCrewID(id); return ""; } @Repository public interface CrewMembers extends DataRepository<CrewMember, String> { @OrderBy("crewID") List<CrewMember> findAll(); void deleteByCrewID(String crewID); void save(CrewMember a); } @Entity public class CrewMember { @NotEmpty(message = "All crew members must have a name!") private String name; @Pattern(regexp = "(Captain|Officer|Engineer)", message = "Crew member must be one of the listed ranks!") private String rank; @Id @Pattern(regexp = "^\\d+$", message = "ID Number must be a non-negative integer!") private String crewID;
67. ▪ GitHub repositories ▪ https://github.com/OpenLiberty/sample-jakarta-data ▪ OpenLiberty Beta ▪ https://openliberty.io/blog/2023/05/16/23.0.0.5-beta.html ▪ Specifications ▪ Jakarta Data: https://jakarta.ee/specifications/data/ @ wernerwedge Links @emilyfhjiang
68. • Flaticon.com • Michael Simons • Jean Tessier • Teo Bais • Nathan Rauh • Mark Swatosh • Otavio Santana Credits @emilyfhjiang @wernerwedge
69. Thank You @emilyfhjiang @wernerwedge