The Fruits of good Data Management
What is the difference between data engineering and data science ? Read on to understand thow they play crucial role in data management
MongoDB Atlas has a host of features that make a developer's life easier. One of them is the Atlas Search, introduced by MongoDB for general availability in 2023. Atlas Search brings with it features like semantic search, query analytics, fuzzy search, custom scoring, sorting, auto complete, synonyms, fast faceting and most recently, search nodes. In this article, we will learn what search nodes are and what problem they solve.
What is Atlas Search?
Atlas Search is an advanced search feature that allows text indexing and querying data on Atlas clusters. This is done via the $search stage of the MongoDB aggregation pipeline. Atlas Search provides several types of analyzers for search, the default being the standard analyzer for all Atlas search indexes and queries. Some other options are the whitespace analyzer which splits the text into smaller search terms whenever it finds a whitespace, language analyzer, which provides a set of language-specific analyzers and many others.
Architecture of MongoDB Atlas Search
As we already know, MongoDB uses the daemon process mongod to handle all database requests, manage data access and perform background tasks. Similarly, MongoDB Atlas search uses the mongot Java web process alongside the mongod process to manage the Atlas search indexes on all the database collections. When you configure your index definition for the collection, mongot creates the search indexes for you. It also processes any index changes for the collections that have search indexes defined, processes the search queries and returns the matching documents.
Apache Lucene
Apache Lucene is one of the most powerful search engines, and is optimized for text indexing, like synonyms, plurals and similarity functions. Generally, transactional databases (like MongoDB) do not have the search capability. With Lucene, application developers can create rich search capabilities for their Atlas database. Lucene provides low-level APIs that enable creating complex queries programmatically and retrieving keyword based starches in large data quite fast.
Why do we need search nodes
As we observed in the above setup, both mongot and mongod have to work together. If a search query and another database query require the same resource, there might be a conflict. Further, the non-availability of Atlas search or mongot might affect the mongod availability too.
So, MongoDB decided to eliminate the dependency between Atlas Search and other database requirements.
Search nodes provide dedicated infrastructure to specific workloads that require search functionality, like Atlas Search and Vector Search. Search nodes optimize compute resources (RAM, CPU) and fully scale search requirements independent of the database, as they consist of only indexes defined by search collections.
Search Nodes dedicated architecture
MongoDB Atlas deploys search nodes for each cluster or with each shard on the cluster. With this architecture, the storage and query load become independent of the MongoDB cluster. With the general availability, MongoDB has also introduced a low CPU option optimal for vector search, particularly in the production environment - this solves any service interruption resulting due to the shared resources between database operations and search.
Search uses MongoDB Change streams to know about the collections that define Atlas search indexes, and accordingly update the search indexes. You can configure each search node independent of other search nodes and run mongot processes separately from the database nodes (running the mongod process) on the Atlas cluster. Once you deploy the search nodes, Atlas automatically assigns a mongod for each of the search nodes (mongot). The specific mongot then communicates with the mongod to observe, listen and sync index changes for the indexes it stores.
Conclusion
In this article, we have learnt how creating separate search nodes can provide better performance, and enable workload isolation through better resource usage. Having search nodes also creates high availability by eliminating dependencies and creating separation between the search nodes and mongodb nodes. Currently, search nodes are available only on AWS and Google Cloud. You can enable search nodes using the MongoDB Atlas UI during new cluster creation, in a few simple steps. When creating the cluster, you can enable the “Multi-Cloud, Multi-Region & Workload Isolation” option:
Once you do this, if search nodes are enabled for your region, the slider for “Search Nodes ” will be enabled for you and you can turn it on.
You can configure the number of search nodes as per your requirement. Before clicking the Create cluster button, make sure to read all the considerations for creating the cluster with Sear
You can configure the number of search nodes as per your requirement. Before clicking the Create cluster button, make sure to read all the considerations for creating the cluster with Search nodes.
