Integrating Splunk into your Spring Applications

Editor's Notes

1. Developer Evangelist at SplunkBuild various apps,add-ons & tools across the Splunk developer landscape , talk about it , get other developers knowledgeable and interested in our various developer hooksSplunk architect/admin and community collaboratorI was once a customer and was spending a lot of time on Splunkbase (still do !)CoderMostly in the Java ecosystem, but have several languages in my batbelt, generally interested in coding anything that’s useful to someone, use the right language for the job.Born and raised in Aotearoa (New Zealand)
2. Tee shirtDividiedloyaltysKudos to elison for a great spectacle
3. Massive amounts of “data” are being generated everydayWe are often blind to where that data is and what that data is telling us , it lives buried away in a dark caveWe can navigate our way through the murk and make discoveries if we have the right equipment and know how (big data tools and techniques )Not a teeshirt company , not a twitter analysis company , not a twitter competition platform 
4. Where we started founders , where we areIntegrated platform for data collection, searching & correlation and visualizationReal time visibility of events, Real time alertingNo need to write Map Reduce jobsOpen and extensibleNumerous ways to get your data into Splunk (TCP,UDP,REST,File monitoring, custom inputs etc..)“Schema on the fly”, no need to define/enforce structure up frontHighly available distributed architecture allows Splunk to scale out to TB’s per day , Index replicationStream in data directly from your application codeProvides comprehensive controls for data security, retention and integritySplunkbase community , heaps of free apps and add-ons, community supportInternet of things , elevators , SCADA , cars, medical devices , sensors, wifi , camera , mics , Mobile , rfid, network packets , SIEM quadrant , vending machines, fitbitetc..Splunk’sflagship product is Splunk Enterprise. Splunk Enterprise is a fully featured, powerful platform for collecting, searching, monitoring and analyzing machine data.Splunk collects machine data securely and reliably from wherever it’s generated. It stores and indexes the data in real time in a centralized location and protects it with role-based access controls. You can even leverage other data stores. Splunk lets you search, monitor, report and analyze your real-time and historical data. Now you have the ability to quickly visualize and share your data, no matter how unstructured, large or diverse it may be. Troubleshoot problems and investigate security incidents in minutes (not hours or days). Monitor your end-to-end infrastructure to avoid service degradation or outages. Gain real-time visibility and critical insights into customer experience, transactions and behavior. Use Splunk and make your data accessible, usable and valuable across the enterprise.
5. Unlike traditional structured data or multi-dimensional data– for example data stored in a traditional relational database for batch reporting – machine data is non-standard, highly diverse, dynamic and high volume. You will notice that machine data events are also typically time-stamped – it is time-series data. Take the example of purchasing a product on your tablet or smartphone: the purchase transaction fails, you call the call center and then tweet about your experience. All these events are captured - as they occur - in the machine data generated by the different systems supporting these different interactions. Each of the underlying systems can generate millions of machine data events daily. Here we see small excerpts from just some of them. Volume , velocity , variety , veracity ,valueStitchSingle pain of glass
6. When we look more closely at the data we see that it contains critical information – customer id, order id, time waiting on hold, twitter id … what was tweeted. What’s important is first of all the ability to actually see across all these disparate data sources, but then to correlate related events across disparate sources, to deliver meaningful insight.
7. Semantic logging:Create Human Readable EventsClearly Timestamp EventsUse Key-Value Pairs (JSON Logging)Separate Multi-Value EventsLog Unique Identifiers
8. What have developers been building using Splunk Enterprise? Examples include the following:Run searches and retrieve Splunk data from existing Customer Service/Call Center applications (Comcast use case) Integrate Splunk data into existing BI tools and dashboard (Tableau, MS Excel)Build mobile applications with KPI dashboards and alerts powered by Splunk (Otto Group use case)Log directly to Splunk from remote devices (Bosch use cases)Build customer-facing dashboards powered by user-specific data in Splunk (Socialize, Hurricane Labs use cases)Programmatically extract data from Splunk for long-term data warehousing
9. Code examples from Eclipse
10. Service is a wrapper that facilitates access to all Splunk REST endpointsCollections use a common mechanism to create and remove entitiesEntities use a common mechanism to retrieve and update property values, and access entity metadataArgs provide a POJO getter/setter paradigm fro REST API endpoint arguments
11. David TuranskiJarrod leiDemo : Spring outbound twitter + splunk search + create dashboards + splunk booth exampleSpring inbound search , console output
12. Writing to file it nice , buffers things , but sometimes there are constraints that forbid this , disk space etc..Bureaucratic constraints to depoing collectors / forwarders.
13. Demo : Splunk java logging code example + Splunk search
14. Demo : Spring integration code + active MQ with Splunk search
15. Splunk Java Agent demo from Eclipse with Splunk searches
16. Demo :Show shome data off my Nexus going into splunkcowboy.comSimple search exampleHaversine
17. Splunk Enterprise is simple to deploy, scales from a single server deployment to global large-scale operations and delivers fast payback. Download Splunk Enterprise for free, install it in 5 minutes on your laptop or on any commodity server, point it at any machine data and start using it. Splunk software is often deployed for the first time while under fire. A serious service outage or security incident in progress is stressful, but with Splunk Enterprise, you can complete your investigation in a few minutes versus hours or days.
18. Developer track , hackathon , and vegas !.conf2013 war our 3rd annual conferenceHeld in Las Vegas at The Cosmopolitan Hotel in September.Goal here is to make our customers smarter, because smarter customers find new ways to use Splunk and tell their colleagues to use Splunk. Specific conference goals:Help customers answer: Where will your data take you?Empower customers with knowledgeFoster deep, supportive relationships within the Splunk communityGarner rich feedback and input to create a better SplunkReinforce Splunk CommunityEquip Customers and Partners with skills for successCreate channel for sharing best practices—expanding use casesLive, in-person venue for trainingFoundation for everything Splunk--future Users’ Conferences, regional user groups, fueling Splunkbase and Splunk Answers…