Neurelo Takes an API-First Route to Programming with Databases
In cloud native computing, abstractions in compute, network and storage have made developers’ lives easier in many ways. But with data? Not so much.
Developers still have to weave together ways to define data, integrate, program, monitor, scale and secure it. In MongoDB’s 2022 Report on Data and Innovation, 73% of the 2,000 IT professionals polled said that working with data is the hardest part of building applications.
“This whole layer is an iceberg of complexities and challenges of itself,” said Chirag Shah, co-founder and CEO of the startup Neurelo, which is taking an API-first approach to making it simpler
Neurelo aims to bring a single layer abstracting complexities of dependencies between the application and the database to those building applications using PostgreSQL, MongoDB and My SQL.
APIs Built for Your Data Model
Shah and co-founder Guru Kamat, both of whom come from database backgrounds, saw this “iceberg” first-hand in their work with customers.
Guru Kamat
Chirag Shah
Shah previously was vice president of products at MongoDB and led product and go-to-market functions at VMware, Hewlett Packard Enterprise and other firms. Kamat was head of engineering for Stripe Security Foundations and vice president of engineering at Palo Alto Networks for Prisma Cloud.
That experience led them to create Neurelo, based in Los Altos, California, about a year and a half ago. Looking at the future, they decided it would be API-first and asked themselves, “What if every single read and every single write from your software to your database becomes a standard API purpose-built for your data model? In minutes, not days, not weeks. Now, the very first day, when developer starts …They go to start writing the programs in a minute.”
With its 1.0 release of its Cloud Data API Platform, Neurelo is offering those custom auto-generated APIs based on your data model, eliminating the need to rely on drivers, driver query languages, object-relational mapping (ORM) or Node.js-based object data modeling (ODM) for MongoDB. It also automatically creates documentation for the API and handles version control, changing code and schemas as the application demands.
“The API by itself is a massive developer velocity boost because all your gestation period of starting with database programming, query language, drivers, all of those things are completely neutralized. They can just expose [their] data models and get started in a minute,” Shah said.
In a post from his experience with an early access version of Neurelo, software developer and coding instructor Sean R. Reid wrote that setting up an Express or Django API would at minimum require six or seven steps, more likely double that, which wouldn’t necessarily be daunting if he enjoyed backend work. But he doesn’t.
“My ‘happy place’ is the frontend, React, Remix, Next, Vue, etc. … Building an API was like that one chore you keep pushing to the bottom of your to-do list….”
He details resurrecting a long-dormant project named BOOMKARKS after a spelling error:
“There are many upsides to Neurelo, but let’s start with the biggest: I added my database as a data source and BOOM 💥, I had an API!
“Seriously, that’s all there was to get up and running with the BoomKarks API. Neurelo read my schema and conjured up GET, POST, PUT and DELETE routes for all my primary tables. Within minutes I could throw out some fetch() queries and was able to do all the basic CRUD. No Node, no Python, no PHP, just point the data source as my PostgreSQL server, flip the switch and BOOMKARKS LIVES!”
The APIs are generated on both REST and GraphQL formats and can handle myriad create, read, update, and delete (CRUD) operations across individual tables or collections as as well as JOIN queries across multiple entities in the database. The APIs, which Shah describes as very similar in technology to that of Stripe or Twilio, are stored on its fully managed cloud platform.
Custom Complex Queries and More
Neurelo also can auto-generate custom complex queries using natural language prompts. This release is based on OpenAI technology, though the company has been experimenting with Bard and other AI models. It’s been training large language models on all the database-specific syntax and semantics so you can just ask questions and it generates optimized queries in SQL (Structured Query Language) or MQL (Mongo Query Language) as needed for the specific database, taking in the context of your application from the data model.
Though abstraction creates a certain opacity, Neurelo aims to show what’s happening with every read and write taking place through the API.
“The industry misses the mark when they give you query-level observability. Every database vendor gives you that, right? But the real understanding is, ‘Give me my data access pattern. What are my reads? What are my hotspots? Which column is getting a 90-percent read traffic?’ Things like that [are] profound, that changes how you actually do a lot of things in production,” he said, explaining that if a particular column is getting heavy traffic, for instance, that could determine the way you scale.
Neurelo’s observability can be set up across different environments — dev, test, prod — to track API and database query performance.
It also offers a Schema-as-Code feature to treat database schema like source code to track changes with version control. It provides a codified repository — a central source of truth — for automated collection and management of metadata across all aspects of the database, helping to consistently enforce standards and improve collaboration among team members.
It has released SDKs for TypeScript, Go, Java, PHP and Python, though Shah said Neurelo works with any language.
“If you’re a frontend developer, building an application into, let’s say, HTMX … you can just start making restful calls to your database through us, you don’t even have to worry about database drivers not supporting those languages,” he said.
The technology itself is built in Rust. Founding software engineer James Shockley explained why they chose Rust in this video:
Analyst Torsten Volk, managing research director at Enterprise Management Associates (EMA), weighed in on the security aspects of Neurelo.
“The combination of centrally managed access tokens and the ability to whitelist specific IPs and IP ranges aligns Neurelo’s security measures with those of other hosted data services such as MongoDB or AWS DynamoDB. Additionally, Neurelo allows organizations to host its runners — dedicated agents responsible for data retrieval and manipulation — on-prem. This on-prem hosting option enables organizations to apply the ‘least privilege’ security principle, effectively narrowing the runners’ permissions to minimize the risk and impact of a security breach,” he said.
When used in conjunction with microsegmentation of the organization’s network, Neurelo’s approach offers additional security by isolating workloads, which restricts the lateral movement of threats within the system. This architectural separation between the Neurelo-hosted control plane and the locally managed execution agents (runners) addresses a broad range of security concerns, he explained.
“As modern distributed microservices applications increasingly depend on application-centric security strategies rather than solely on infrastructure hardening, Neurelo’s model of a cloud-hosted control plane with on-prem hosted agents is well-suited for secure integration into corporate environments. It’s important to note, however, that certain use cases, particularly those with stringent regulatory requirements, may not be compatible with a cloud-based control plane. Nevertheless, these cases represent a relatively small fraction of enterprise applications, suggesting that Neurelo’s platform could be applicable and secure for the vast majority — potentially over 90% — of enterprise workloads.”
Roadmap for the Enterprise
“Database programming can be a daunting task, especially for those new to the field. Understanding the intricacies of data types, constraints, indexing and transactions is a prerequisite for building efficient and reliable database applications. Even experienced developers can find themselves spending hours or even days debugging and optimizing queries to achieve the desired performance,” according to a company white paper.
Some companies have thrown a lot of different tools at the problem — it’s not unusual to find five or more tools for each database — while a few tech giants like PayPal, X (formerly Twitter) and Facebook have built their own abstraction layers, projects that are expensive to build and maintain.
This release is only the first milestone along Neurelo’s roadmap. It’s working toward more enterprise-level features like field tagging and other access controls of sensitive information and plug-n-play integration with caching, search and other systems. There also are plans to develop an on-premises version.
“The utilization of APIs is a key part in the development life cycle, allowing Neurelo to create, modify, and enhance functionalities swiftly. This approach accelerates developer productivity and acts as a catalyst for innovation,” said analyst Paul Nashawaty of The Futurum Group by email.
“… Neurelo’s API abstraction layer plays a role in simplifying the complexities associated with database management. Developers can interact with the underlying database systems through well-defined API endpoints, reducing the intricacies traditionally associated with direct database manipulation. This abstraction layer enhances developer understanding and optimizes the overall development process. Additionally, by leveraging the managed cloud services, Neurelo ensures the security and reliability of its APIs, offering a robust and scalable environment. This strategic integration not only results in a faster time to production but also affords better control, predictability and cost-effectiveness, ultimately contributing to Neurelo’s technological advancement and competitive edge in the market.”
