Talk Abstract: The architecture of modern E-commerce companies typically revolves around micro-services, this often causes a challenge for Data Scientists who need to pull data from disparate sources to carry out their day to day work. This pushes the typical 80% of time spent on data exploration, closer to 100%, resulting in less time spent solving the customer problem. At Trainline, we have solved this using event sourcing and streaming technology. We will talk about how this is practically carried out from a data engineering to data product development perspective. We will then explore advancing this technology, giving the ability to put realtime data products into production and the hands of our customers.
Bio: Dan runs the Data Engineering team at Trainline. He’s passionate about applying engineering principles to data products: building cool stuff that _actually runs in production_. He’s headed up Data Engineering teams in such exciting fields as insurance comparison, telecoms network optimisation and supermarket refrigeration, and asserts that 90% of data challenges are the same across all industries.