API World 2018

AI has been a hot topic, but there has not been much discussion of the infrastructure and scaling challenges that come with it. How do you support dozens of different languages & frameworks, and make them interoperate invisibly? How do you elastically run abstract code from thousands of different developers with under 15ms overhead? In my time at Algorithmia, we’ve built, deployed, and scaled thousands of algorithms and machine learning models, using every kind of framework. We’ve seen many of the challenges faced in this area; in this talk I’ll share insights into the problems you’re likely to face, and how to approach solving them. In brief, we’ll examine the need for, and implementations of, a complete “OS for AI”: a common interface for different algorithms to be used and combined, and a general architecture for serverless ML which is discoverable, versioned, scalable & sharable.