Criando um endpoint de streaming

Transcript

1. Criando um endpoint de streaming DANIEL MEZZATTO @mezzatto FELIPE MAMUD

   @mamud

2. Então vc quer criar um endpoint stream?

3. AGENDA Stream.of("Motivação", "Desafios", "Endpoint") .map(VivaReal::techTalk) .collect(Collectors.knowledge());

4. MOTIVAÇÃO • Time to first byte (TTFB) • Deep paging

5. DESAFIOS • Qual implementação utilizar? ◦ Java? outra linguagem? ◦

   Server-Sent Events ◦ StreamingResponseBody* ◦ Custom • ES SearchScroll (com shards) • Sized stream

6. CONTROLLER @RequestMapping(value = "/{index}/stream", method = GET) public StreamingResponseBody stream(

   FilterableApiRequest request, HttpServletResponse httpServletResponse) { httpServletResponse .setContentType("application/x-ndjson;charset=UTF-8"); return out -> searchService.stream(request, out); }

7. @Component public class ElasticSearchStream { public void stream(OutputStream stream) {

   ResponseStream.iterate(stream, new SearchApiIterator<>(esClient, requestBuilder(), scroll -> scroll.setScroll(keepAlive) .actionGet(streamTimeout), count), searchHit -> toBytes(searchHit.getSourceRef())); } } ELASTICSEARCH STREAM

8. RESPONSE STREAM public static <T> void iterate(OutputStream stream, Iterator<T[]> iterator,

   Function<T, byte[]> byteFn) { while (iterator.hasNext()) { for (T hit: iterator.next()) { stream.write(byteFn.apply(hit)); stream.write('\n'); } stream.flush(); } }

9. SEARCH ITERATOR public class SearchApiIterator<T> implements Iterator<T[]> { @Override public

   boolean hasNext() { return hits() > 0 && count <= size; } @Override public T[] next() { T[] result = (T[]) response.getHits().getHits(); response = loop.apply(client.prepareSearchScroll(response.getScrollId())); this.count += hits(); return result; } }|
10. None