When companies build these embedded intelligence experiences, or internal analytics dashboards, engineers might use an open source library like Chart.js or D3 for the visualization, axios to fetch it from the backend, and build software to query the data from storage. They usually leverage the ORM framework provided by their backend (ActiveRecord for Ruby on Rails, Django ORM for Python, Sequelize for Node.js, etc) to access the data, rather than writing pure SQL queries (View Highlight)
Querying performance: as the amount of data being analyzed grows, it becomes harder to deliver results in a timely manner. This usually leads to a degradation of the user experience, which can negatively impact your business. A frontend/full stack engineer might not be proficient enough in SQL to rewrite their ORM queries in a more efficient manner.
Caching complexity: some of these queries begin to get pretty expensive and the data isn’t very dynamic, so you resort to caching. As the application grows, managing cache expiration becomes challenging.
Multi-data store analysis: when running queries, you might need to join data from multiple databases or data warehouses. This isn’t trivial to do in a performant way, and it might not be supported at all by the ORM you are using. This limitation also applies to traditional BI tools. For example, Tableau didn’t have support for hosted MySQL and PostgreSQL databases until 2015. (View Highlight)
Note: Embedded analytics pain points
When thinking about embedded intelligence, there’s two axes that we categorize these tools on: on one axis, there’s products focused on the developer experience, contrasted by data science-focused ones. On the other axis, the spectrum goes from single player to multiplayer. (View Highlight)
