0

I am currently migrating our currently test data that is stored in excel files to a database. Because the type of data in the test results are going to grow we decided to go with nosql instead of sql.

I was planning on using a document database to store the data in but realized I'm not entirely sure how to store the data for it to be efficiently read.

The idea is that you have unique keys that help you find the data you are looking for quicker.

The data columns in the excel are:

  • Date
  • Model of device
  • Version of software
  • Type of test
  • A bunch of other columns that include the test data (% of something, Count of something, etc...)

So these are the columns and there are multiple rows of his kind of data.

Would a nested document be used here?

i.e. [Type of test] -> [Date] -> [Model of device] -> [Version of software] -> [Bunch of test data]

Am I even using the right database type?

1
  • Are test data updated only when creating/updating a test?
    – dzieciou
    May 4, 2019 at 9:51

2 Answers 2

0

Not clear what "[Bunch of test data]" looks like so this is hard to answer. I'd likely have each document include the other items you mention as keys and index those fields so you can query against them quickly.

If "Bunch of test data" is JSON, you could achieve similar results using a modern DB that supports JSON (e.g. Postgres or SQL Server)

Whether you shuld use SQL or NoSQL is a much bigger question.

3
  • I updated my document a bit more to make it more clear. Are you saying you would create a document for each data entry? Because the test results contains say 100 rows of the column I mentioned, using that method would generate 100 documents. Is there a way to consolidate the data into one document?
    – tyleax
    Dec 4, 2018 at 22:38
  • I was suggesting a document for each row in your spreadsheet. But, if you have fixed columns in every row, then NoSQL doesn't feel like a good fit, unless there are other reasons you want to leverage a document store?
    – ernie
    Dec 5, 2018 at 19:29
  • The fixed columns are not really fixed. I suspect over time as requirements come in there will be more columns and some will disable. I'm worried if I use sql that a table will not be normalized anymore and will have lots of null entries.
    – tyleax
    Dec 5, 2018 at 22:32
0

For storing test data into a NoSQL document database, you can use a key-value pair or a nested document structure. In this case, as you want to search and retrieve the data quickly, it's better to use a nested document structure, which provides easy querying and indexing.

You can create a NoSQL document structure as follows:

{
  "type_of_test": "test_name",
  "date": "dd-mm-yyyy",
  "model_of_device": "device_name",
  "version_of_software": "software_version",
  "test_data": [
      {
          "name": "% of something",
          "value": "50"
      },
      {
          "name": "count of something",
          "value": "10"
      }
  ]
}

Here, each test result is represented by a document that contains all the columns of the Excel sheet. The test_data field is an array that contains the name and value of the test data. You can add more fields to the document as per your requirements.

You can store these documents in a NoSQL document database like MongoDB or Couchbase. You can then index the fields on which you want to search and retrieve the data quickly. For example, you can index the type_of_test, date, model_of_device, and version_of_software fields.

NoSQL databases are a good fit for storing unstructured and semi-structured data. If your test data is constantly changing, NoSQL databases offer more flexibility as you don't need to define a schema beforehand. However, if your data is structured and you require complex queries or transactions, SQL databases may be a better fit.

Overall, using a nested document structure in a NoSQL database can be an efficient way to store test data and retrieve it quickly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.