IBM Bluemix Platform Logging with the Elastic Stack

IBM Bluemix platform logs to the Elastic stack. Learn about the flexibility of our logging solutions in microservice environments. See Docker and Cloud Foundry logging – no user configuration necessary. Runtime logs are easily added.

In this session, Charlie and Ray will show an example logging from a Watson service. They'll also share details on their capabilities as well as how they deliver cloud scale logging built on the Elastic Stack.

Charlie Comiskey l Product Manager, IBM Cloud l IBM
Ray Harris l Software Engineer l IBM

Transcript

1. IBM March 8, 2017 IBM Bluemix platform logging with the Elastic stack Charlie Comiskey, Offering Manager Linkedin: charlie-c Ray Harris, Software Engineer Linkedin: rayharris1
2. 2 3/20/17 We Enable Logging in Bluemix { } …and many more Automated log collection for your interconnected cloud-hosted microservices. Spend less time instrumenting your app More time enhancing your app value. Multi-tenant enabled to offer large scale personalized logging experience 10s of thousands
3. 3 3/20/17 Built on the Elastic Stack Interact with your microservices in Kibana Heterogeneous workloads or hybrid, One log location Build applications based on VMs | Docker | Cloud Foundry Bluemix Public | Dedicated | Local configurations Send logs from application workloads outside the cloud Data & Workload co-location Little to zero configuration + =
4. 4 Go Live in Seconds APIs and Services On-Prem Integration Analytics Compute Security Storage Watson Mobile DevOps Bluemix hosts a rich catalog of Applications and Services What is Bluemix Bluemix is an open-standard, cloud-based platform for building, managing, and running applications of all types (web, mobile, big data, new smart devices, and so on). | | | | | |
5. Bluemix Public Services Compute Storage Networking Block Storage Object Storage Auto-scaling CDN Load Balancer for Containers, and Virtual Servers VPN Cloud Foundry Runtimes Docker Containers OpenStack Virtual Servers OpenWhisk Swift Runtime Mobile IBM Push Notifications Mobile Analytics Mobile App Content Manager Mobile Client Access Mobile Foundation Mobile Quality Assurance 5 Security Access Trail App Security on Cloud IBM Identity Mixer Key Protect Single Sign-On DevOps Active Deploy Auto-Scaling Availability Monitoring Automated Access.Tester Delivery Pipeline Delivery Pipeline Next Deployment Risk Analytics Digital Content Checker Globalization Pipeline IBM Alert Notification IBM Operations Analytics Advanced Insights IBM Runbook Automation Integration Tester Monitoring & Analytics Open Toolchain Fabric – DevOps Console Track & Plan Application Integrate Blockchain Business Rules Data Cache Document Generation Message Hub Message Hub Incubator Session Cache WebSphere App Server Workload Scheduler API Connect API Harmony Message Connect Secure Gateway Secure Broker Service Discovery Service Proxy Watson IoT Context Mapping Driver Behavior IoT Platform IoT for Automotive IoT for Electronics IoT for Insurance IoT Workbench Watson Cognitive AlchemyAPI Conversation Discovery Document Conversion Language Translation Language Translator Natural Language Classifier Personality Insights Retrieve and Rank Speech to Text Text to Speech Tone Analyzer Tradeoff Analytics Visual Recognition Data & Analytics Analytics Exchange – not a tile Apache Spark BigInsights for Apache Hadoop Cloudant NoSQL DB Compose for Elasticsearch Compose for etcd Compose for MongoDB Compose for PostgreSQL Compose for RabbitMQ Compose for Redis Compose for RethinkDB dashDB (D), dashDB Tx DataWorks Decision Optimization Geospatial Analytics IBM DataStage on Cloud IBM DB2 on Cloud IBM Graph IBM Information Server IBM Master Data Mgmt on Cloud Informix on Cloud Insights for Twitter Lift Predictive Analytics Streaming Analytics Weather Company Data Xpages NoSQL Database Logging is a Bluemix platform offering. Enabled by default.
6. User Scenarios • Bluemix Public Cloud users • Large number of groups ( Bluemix spaces ) 10s of thousands per instance • Generating a variety of log data – platform, application, and external • Daily ingest volumes from kilobytes to gigabytes • Many users engage in microservices development projects and DevOps • Bluemix Local and Dedicated Cloud users • Dedicated private clouds hosted @IBM or @Client-premise • Bluemix local runs on VM. • Internal IBM Infrastructure • Small number of groups • Each with heavy log requirements (avg 3-5 TB/day ) • Data from all compute types
7. Implementations Bluemix Public Cloud users • Multiple stacks deployed in 3 WW datacenters • >3TB/day per Logging stack Bluemix Local and Dedicated Cloud users • Deployed in 3 hypervisors for HA • 2 logging instances for IBM/Client data isolation • Supporting a variety of daily log volumes Internal IBM Infrastructure • Node size depends on team • Customized retention and tenant ingestion policies • <3TB/day per Logging instance
8. Demo of logging in Bluemix using the Elastic Stack Randy was tasked to build their banking app in the cloud, using microservices, then extend it with cognitive APIs. In this demo we will see • Application logging with different runtimes • Across different compute types • Extended to include a Watson API • Visualized with a consolidated Kibana dashboard • All with little to zero configuration
9. Demo Summary • Node.js and Liberty runtime logging • Cloud Foundry and Docker Containers – into same space • Sentiment analysis with Watson Tone Analyzer • Log Analysis with Kibana - debugged to line of code • All with little to zero configuration
10. How we use the Elastic stack Load Balancer MT Lumberjack Logstash cluster Kafka Cluster Logstash Indexer cluster Elasticsearch cluster MT API proxy ngnx crawlers Logstash Bluemix Microservices
11. Thank you! Learn more about Logging in Bluemix Docs Charlie Comiskey Linkedin: charlie-c Ray Harris Linkedin: rayharris1
12. 12 3/20/17 Notices and disclaimers Copyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.” Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
13. 13 3/20/17 Notices and disclaimers continued Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non- IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular, purpose. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services®, Global Technology Services®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli® Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.