Velocity Conference - Notes & Takeaways, pt.3

Finally got a chance to tidy up the final part of my notes from this year's Velocity conference!

The Definition Of Normal: An Intro and guide to anomaly detection (Alois Reitbauer)

As anomaly detection has a nice role to play in spotting issues early - ideally before any really bad things happen - I was really excited about this talk. Here is my high-level overview: Anomalies are defined as events or observations that don’t conform to an expected pattern. As such, the anomaly detection workflow is:

1. Use actual data to define / calculate what is ‘normal’, i.e. define your ‘normal model.’
2. The ‘normal model’ is continuously updated with new data.
3. Hypotheses are derived from the ‘normal model.’
4. Events are checked against your hypotheses, applying a likeliness judgement.
5. How the event performs against this likeliness judgement translates into whether it is an anomaly or not.

But how should we best go about setting the baselines which define our normal model? One thing to bear in mind that some of them (such as mean / average or median) don’t learn very well. Reitbauer recommended using exponential smoothing instead, since it is both easy to calculate and learns very well.

The full slides are here and include more detailed recommendations, as well as presenting of a number of different options for doing anomaly detection, including all the juicy maths if you fancy them!

A Real-Life Account Of Moving 100% To A Public Cloud (Julien Simon and Antoine Guy)

Their company Viadeo moved to AWS for reasons commonly cited by other companies making a similar move, and like many other companies facing the challenge they moved over gradually. Rather than recounting the full steps and stages of their transition, I wanted to highlight some of the key lessons they learned in the process, which really resonated with my experience of our current move to the cloud at 7digital, and the challenges it presents for us:

1. Outline your key objectives! - You will need all the focus and direction you can get your hands on during a potentially sprawling transition like this.
2. Plan and build with a temporary hybrid run in mind - be able to roll back etc. 
3. Ahead of the move, have a thorough report of your infrastructure - estimate equivalent cost in cloud; evaluate each for replacement (PaaS, Saas or leave as is?); identify pain points (tech debt; relevance of moving legacy apps). 
4. Define a high-level migration plan - once again, for focus and direction.
5. Tech is only half the work - identify all stakeholders and their goals; involve Legal / Finance early, especially if you might have to battle early terminations of legacy infrastructure contracts, ensure you work on awareness and knowledge transfer across teams at all key stages of the transition.

The slides for this talk have been made available, if you're interested in the full account.

Further resources

Following the close of the conference, O'Reilly have put together a really neat collection of relevant free eBooks, covering a range of subjects around web operations, distributed systems, performance optimisation, resilience and scalability.

All the short keynote talks of the conference can be watched here; unfortunately it doesn’t currently look like they’ll make the full-length talks available to the public, and slides of all the sessions (where speakers have chosen to share them) have been collated here.