Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Criando um endpoint de streaming

Criando um endpoint de streaming

Como criar um endpoint stream utilizando Spring Boot.

3c1b9c8d515093124b8c081107d4830c?s=128

Felipe Mamud

October 19, 2017
Tweet

Transcript

  1. Criando um endpoint de streaming DANIEL MEZZATTO @mezzatto FELIPE MAMUD

    @mamud
  2. Então vc quer criar um endpoint stream?

  3. AGENDA Stream.of("Motivação", "Desafios", "Endpoint") .map(VivaReal::techTalk) .collect(Collectors.knowledge());

  4. MOTIVAÇÃO • Time to first byte (TTFB) • Deep paging

  5. DESAFIOS • Qual implementação utilizar? ◦ Java? outra linguagem? ◦

    Server-Sent Events ◦ StreamingResponseBody* ◦ Custom • ES SearchScroll (com shards) • Sized stream
  6. CONTROLLER @RequestMapping(value = "/{index}/stream", method = GET) public StreamingResponseBody stream(

    FilterableApiRequest request, HttpServletResponse httpServletResponse) { httpServletResponse .setContentType("application/x-ndjson;charset=UTF-8"); return out -> searchService.stream(request, out); }
  7. @Component public class ElasticSearchStream { public void stream(OutputStream stream) {

    ResponseStream.iterate(stream, new SearchApiIterator<>(esClient, requestBuilder(), scroll -> scroll.setScroll(keepAlive) .actionGet(streamTimeout), count), searchHit -> toBytes(searchHit.getSourceRef())); } } ELASTICSEARCH STREAM
  8. RESPONSE STREAM public static <T> void iterate(OutputStream stream, Iterator<T[]> iterator,

    Function<T, byte[]> byteFn) { while (iterator.hasNext()) { for (T hit: iterator.next()) { stream.write(byteFn.apply(hit)); stream.write('\n'); } stream.flush(); } }
  9. SEARCH ITERATOR public class SearchApiIterator<T> implements Iterator<T[]> { @Override public

    boolean hasNext() { return hits() > 0 && count <= size; } @Override public T[] next() { T[] result = (T[]) response.getHits().getHits(); response = loop.apply(client.prepareSearchScroll(response.getScrollId())); this.count += hits(); return result; } }|
  10. None