Zack Proser

Talk @ a16z: Taking AI applications to Production


Navigating from Jupyter Notebooks to Production 1

On December 6th, 2023, Pinecone and Cohere held a joint meetup at the Andreesen Horowitz offices in downtown San Francisco.

I was one of the speakers, along with Jay Alammar from Cohere and Kat Morgan from Pulumi.

I gave a talk titled, "Navigating from Jupyter Notebooks to production" that announced the Pinecone AWS Reference Architecture with Pulumi.

I used a an extended mountaineering metaphor to compare getting the kernel of your application working in a Jupyter Notebook to arriving at base camp.

Navigating from Jupyter Notebooks to Production 2

In some ways, you're so close to production and yet still so far away, because you need to come up with good solutions for secrets, logging, monitoring, autoscaling, networking, and on and on.

Navigating from Jupyter Notebooks to Production 3 Navigating from Jupyter Notebooks to Production 4

I then announced the Pinecone AWS Reference Architecture using Pulumi and talked through what it was and how it can help folks get into production with high-scale use cases for Pinecone faster.

Navigating from Jupyter Notebooks to Production 5

Using Pulumi, I was able to build the initial version of the Pinecone AWS Reference Architecture in about a month, start to finish.

Navigating from Jupyter Notebooks to Production 6 Navigating from Jupyter Notebooks to Production 7

I gave a brief introduction to Infrastructure as Code (IaC) and explained why it's a very powerful tool for working with cloud providers such as AWS or GCP.

Navigating from Jupyter Notebooks to Production 8 Navigating from Jupyter Notebooks to Production 9

I also candidly explained the pitfalls of introducing IaC to a team that doesn't have much cloud development experience, and explained how easy it is for cognitive load and overall complexity to skyrocket past the point of the offering being useful.

I related my experiences around mitigating the pitfall of added complexity by intentionally cross-training your entire team in working with IaC and deploying and destroying any architectures you may produce together.

The default is for this knowledge to become siloed across one or two developers who originally worked on the system, but the key is to ensure everyone on the team is equally confident in working with IaC and cloud providers, via cross-training and practice.

Navigating from Jupyter Notebooks to Production 10 Navigating from Jupyter Notebooks to Production 11

The event was great and we had nearly 125 folks show up to network, eat and listen to our talks. A huge thank you to the folks at Andreesen and Horowitz for their hospitality, support and excellent venue.