How do you store test data into nosql document database for quick read?

Question

I am currently migrating our currently test data that is stored in excel files to a database. Because the type of data in the test results are going to grow we decided to go with nosql instead of sql.

I was planning on using a document database to store the data in but realized I'm not entirely sure how to store the data for it to be efficiently read.

The idea is that you have unique keys that help you find the data you are looking for quicker.

The data columns in the excel are:

• Date
• Model of device
• Version of software
• Type of test
• A bunch of other columns that include the test data (% of something, Count of something, etc...)

So these are the columns and there are multiple rows of his kind of data.

Would a nested document be used here?

i.e. [Type of test] -> [Date] -> [Model of device] -> [Version of software] -> [Bunch of test data]

Am I even using the right database type?

Answer 1

Not clear what "[Bunch of test data]" looks like so this is hard to answer. I'd likely have each document include the other items you mention as keys and index those fields so you can query against them quickly.

If "Bunch of test data" is JSON, you could achieve similar results using a modern DB that supports JSON (e.g. Postgres or SQL Server)

Whether you shuld use SQL or NoSQL is a much bigger question.

Answer 2

For storing test data into a NoSQL document database, you can use a key-value pair or a nested document structure. In this case, as you want to search and retrieve the data quickly, it's better to use a nested document structure, which provides easy querying and indexing.

You can create a NoSQL document structure as follows:

{
  "type_of_test": "test_name",
  "date": "dd-mm-yyyy",
  "model_of_device": "device_name",
  "version_of_software": "software_version",
  "test_data": [
      {
          "name": "% of something",
          "value": "50"
      },
      {
          "name": "count of something",
          "value": "10"
      }
  ]
}

Here, each test result is represented by a document that contains all the columns of the Excel sheet. The test_data field is an array that contains the name and value of the test data. You can add more fields to the document as per your requirements.

You can store these documents in a NoSQL document database like MongoDB or Couchbase. You can then index the fields on which you want to search and retrieve the data quickly. For example, you can index the type_of_test, date, model_of_device, and version_of_software fields.

NoSQL databases are a good fit for storing unstructured and semi-structured data. If your test data is constantly changing, NoSQL databases offer more flexibility as you don't need to define a schema beforehand. However, if your data is structured and you require complex queries or transactions, SQL databases may be a better fit.

Overall, using a nested document structure in a NoSQL database can be an efficient way to store test data and retrieve it quickly.