Caching is a technique for storing data in a temporary storage area called a cache. It aims to retrieve data quickly and avoid costly processes such as recalculating or searching it in the database, especially for frequently asked resources. In the first article, I introduced the different possibilities for caching data in JavaScript and how to choose the appropriate caching possibility according to your needs. In a previous article, I introduced an example of using the Web Storage API (localStorage and sessionStorage) to cache data. Then, I dedicated another article to using objects and data-* attributes to temporarily cache data in JavaScript. Also, I elucidated the use of service workers to cache static resources like HTML, CSS, and JavaScript files or API responses. In this article, I talk about caching data using the IndexedDB database.
1. IndexedDB Database and IndexedDB API
IndexedDB database is a client-side database that stores large amounts of structured data including files and blobs in browsers. This means that a web application can use IndexedDB to cache data so that it can be accessed quickly and avoid frequent requests to the server.
IndexedDB API is a low-level API that manages the corresponding database. It is good for complex applications that need to cache large JSON responses or images and allows working offline.
To avoid blocking web applications, IndexedDB operations are executed asynchronously.
2. IndexedDB Operations
You can perform several operations on IndexedDB. Some can be applied to the database itself, some can be applied to object stores, and some operations are used to manage the data stored in the database. Let’s look at the different IndexedDB operations based on their types.
Database Operations and Event Handling: Database operations are asynchronous and allow you to create, manage, and delete the database. Here is the list of major operations and events for managing the IndexedDB database.
Database Operations |
||
Operation |
Definition |
Syntax |
Create/Open |
Opens a database if it exists. Otherwise, it is created. |
indexedDB.open() |
Delete |
Deletes a database. |
deleteDatabase() |
Close |
Closes the connection to the database. |
close() |
Events |
||
DB Structure upgrade or change |
Triggered when the structure of the database changes. |
|
Operation successfully performed |
Triggered upon successful operation completion. |
onsuccess |
Error occurred |
Triggered when an error occurs. |
onerror |
DB version changed |
- Triggered when the database version changes. - It is used for schema updates. |
onupgradeneeded |
Object Store Operations: Object stores are analogous to relational database tables, serving as a structured repository for data organization. The following table explains the main operations of creating and deleting object stores.
Object Store Operations |
||
Operation |
Definition |
Syntax |
Create |
- Creates an object store as part of the database upgrade process. - This method can only be invoked within the context of a version change transaction. |
createObjectStore() |
Delete |
Deletes an object store. |
deleteObjectStore() |
Transaction Operations: Transactions allow you to perform operations atomically. Here is the list of the main ones you can apply to transactions.
Transaction Operations |
||
Operation |
Definition |
Syntax |
Create a transaction |
Establishes a transaction for reading, writing, or a combination of both, guaranteeing data consistency. Two modes are defined: - readonly: read-only operations. - readwrite: read and write operations. |
transaction() |
CRUD (Create, Read, Update, Delete) Operations: Crud operations refer to the four basic functions that can be performed on data in the data storage system.
CRUD (Create, Read, Update, Delete) Operations |
||
Operation |
Definition |
Syntax |
Read |
- Retrieves a record by its key (unique identifier). - Retrieves all records present in the object store. - Traverses each record in the object store or index. |
- get() - getAll() - openCursor() |
Insert |
- Adds a new record. - Fails if a record with the same key already exists. |
add() |
Update |
Inserts a new record or updates an existing one in the object store. |
put() |
Delete |
- Deletes a record using its key. - Deletes all records in an object store. |
- delete() - clear() |
Index Operations: Indexes enable fast and optimized querying by providing a quick way to locate specific data. The following table summarizes the principal operations on indexes.
Index Operations |
||
Operation |
Definition |
Syntax |
Create index |
Creates an index on a particular field or attribute of records. |
createIndex() |
Delete Index |
Deletes an index. |
deleteIndex() |
Get Index |
Utilizes an existing index to retrieve and filter data. |
index() |
Key Range Queries: Key ranges are used to filter data during queries. They allow you to retrieve only the specific subset of data that falls within a defined range.
Key Range Queries |
||
Operation |
Definition |
Syntax |
Define a range |
Specifies a key range boundary (inclusive or exclusive), allowing for targeted data retrieval within a defined scope. |
IDBKeyRange.bound() |
Define a range with a lower bound |
Specifies a range with a lower bound. |
IDBKeyRange.lowerBound() |
Define a range with an upper bound |
Defines a range with an upper limit. Specifies a range with an upper bound |
IDBKeyRange.upperBound() |
Define a specific key |
Specifies an exact key value, allowing for the retrieval of a single, specific record. |
IDBKeyRange.only() |
3. Example of Caching Data Using IndexedDB Database
IndexedDB database is a suitable mechanism for caching large amounts of data. Let’s take an example of how you can cache the result of getting data from the back-end using IndexedDB. For this, we need some back-end code to return data. An easy way to do this is to use Python’s Flask framework. Additionally, we develop HTML, CSS, and JavaScript code for the front-end.
Note. We only write the relevant part here to explain how to use IndexedDB to cache data. If you want the full example, you can find it on GitHub here:
https://github.com/noura-github/indexeddb-app-blog
Python Code
In the back-end code, we first define ‘company_data’ which stores data about companies, departments, and employees. The dictionary has three main keys: ‘companies’, ‘departments’, and ‘employees’. Each dictionary in the ‘companies’ list represents a company with an ‘id’ and a ‘name’. Each dictionary in the ‘departments’ list represents a department with an ‘id’, a ‘name’, and a ‘companyId’ that references its company. Finally, each dictionary in ‘employees’ is an employee with an ‘id’, a ‘firstname’, a ‘lastname’, an ‘email’, a ‘departmentId’, a ‘departmentName’, and a ‘companyName’.
company_data = {
"companies": [
{"id": 1, "name": "TechCorp"},
{"id": 2, "name": "TechMicro"}
],
"departments": [
{"id": 1, "name": "Engineering", "companyId": 1},
{"id": 2, "name": "Marketing", "companyId": 1},
{"id": 3, "name": "Sales", "companyId": 2},
{"id": 4, "name": "HR", "companyId": 2}
],
"employees": [
{"id": 1, "firstname": "Alice", "lastname": "Smith", "email": "alice@techcorp.com", "departmentId": 1, "departmentName": "Engineering", "companyName": "TechCorp"},
{"id": 2, "firstname": "Bob", "lastname": "Brown", "email": "bob@techcorp.com", "departmentId": 1, "departmentName": "Engineering", "companyName": "TechCorp"},
{"id": 3, "firstname": "Charlie", "lastname": "Davis", "email": "charlie@techcorp.com", "departmentId": 2, "departmentName": "Marketing", "companyName": "TechCorp"},
{"id": 4, "firstname": "David", "lastname": "Wilson", "email": "david@techmicro.com", "departmentId": 3, "departmentName": "Sales", "companyName": "TechMicro"},
{"id": 5, "firstname": "Mary", "lastname": "Johnson", "email": "mary@techmicro.com", "departmentId": 4, "departmentName": "HR", "companyName": "TechMicro"}
]
}
In this Python code, we need a ‘get_data’ function as a Flask route that handles GET requests to the /data endpoint by returning the ‘company_data’ dictionary as a JSON object.
@app.route('/data', methods=['GET'])
def get_data():
return jsonify(company_data)
HTML Code
In HTML code, we need mainly a table to display the data:
<h1>Company Data</h1>
<table id="dataTable">
<thead>
<tr>
<th>First Name</th>
<th>Last Name</th>
<th>Email</th>
<th>Department Name</th>
<th>Company Name</th>
</tr>
</thead>
<tbody>
<!-- Data will be dynamically inserted here -->
</tbody>
</table>
JavaScript Code
In the JavaScript code, we show how to initialize an IndexedDB by opening it and creating its stores. We will also show how we store the data in IndexedDB, and how to retrieve and display it.
To initialize an IndexedDB database, we create a function called ‘initializeDatabase()’ which we call once the document is ready as follows:
$(document).ready(function () {
initializeDatabase();
});
Here is an overview of the purpose of the ‘initializeDatabase()’ function:
Opening the Database: The function starts by opening the IndexedDB database using the ‘indexedDB.open()’ method. The database name is set to ‘CompanyDB’ and the version is set to 1.
Handling the ‘onupgradeneeded’ Event: The ‘onupgradeneeded’ event is triggered when the database is created for the first time or when the version number is incremented. In this case, the function creates three object stores: ‘companyStore’, ‘departmentStore’, and ‘employeeStore’. Each object store has a key path set to ‘id’.
Handling the ‘onsuccess’ Event: When the database is successfully opened, the ‘onsuccess’ event is triggered. The function checks if data already exists in the database by calling the ‘checkExistingData()’ function. If data exists, it calls the ‘loadCachedData()’ function to display the cached data. If no data exists, it calls the ‘fetchDataFromBackend()’ function to fetch data from the back-end.
Handling the ‘onerror’ Event: If an error occurs while opening the database, the ‘onerror’ event is triggered. The function logs the error to the console.
function initializeDatabase() {
const request = indexedDB.open("CompanyDB", 1);
request.onupgradeneeded = function (event) {
const db = event.target.result;
if (!db.objectStoreNames.contains("companyStore")) {
db.createObjectStore("companyStore", { keyPath: "id" });
}
if (!db.objectStoreNames.contains("departmentStore")) {
db.createObjectStore("departmentStore", { keyPath: "id" });
}
if (!db.objectStoreNames.contains("employeeStore")) {
db.createObjectStore("employeeStore", { keyPath: "id" });
}
};
request.onsuccess = function (event) {
const db = event.target.result;
checkExistingData(db).then(function (dataExists) {
if (dataExists) {
console.log("Using cached data");
loadCachedData(db);
} else {
console.log("Fetching data from backend...");
fetchDataFromBackend(db);
}
});
};
request.onerror = function (event) {
console.error("Error opening database:", event.target.error);
};
}
The ‘checkExistingData()’ Function: This function checks if the ‘companyStore’ object store contains data of a given IndexedDB database (parameter ‘db’). The function creates a read-only transaction on the ‘companyStore’ object store. It gets a reference to the ‘companyStore’ object store. Then, it generates a Promise that will be resolved with a boolean value. It sends a count() request to the object store to retrieve the number of records.
If the request is successful, it resolves the Promise with true if there is at least one record (request.result > 0), and false otherwise. If the request fails (e.g., due to an error), it resolves the Promise with false.
function checkExistingData(db) {
const transaction = db.transaction(["companyStore"], "readonly");
const store = transaction.objectStore("companyStore");
return new Promise(function (resolve) {
const request = store.count();
request.onsuccess = function () {
resolve(request.result > 0);
};
request.onerror = function () {
resolve(false);
};
});
}
The function ‘fetchDataFromBackend()’: This is an asynchronous function that takes an IndexedDB database object (‘db’) as an argument. In summary, the function sends a GET request to the /data endpoint using the fetch API. It waits for the response and parses it as JSON using ‘response.json()’. The resulting data is expected to be an object with three properties: companies, departments, and employees, each containing an array of objects. It creates a read-write transaction on three object stores: ‘companyStore’, ‘departmentStore’, and ‘employeeStore’.
It gets references to each of these object stores. It loops through each array of data (companies, departments, employees) and adds each object to its corresponding object store using the ‘add()’ method. It sets up an event handler for when the transaction is complete (‘transaction.oncomplete’). When the transaction is complete, it calls another function named ‘loadCachedData()’, passing the ‘db’ object as an argument.
If any error occurs during the execution of this function, it catches the error and logs it to the console with a message.
async function fetchDataFromBackend(db) {
try {
const response = await fetch("/data");
const data = await response.json();
const transaction = db.transaction(["companyStore", "departmentStore", "employeeStore"], "readwrite");
const companyStore = transaction.objectStore("companyStore");
const departmentStore = transaction.objectStore("departmentStore");
const employeeStore = transaction.objectStore("employeeStore");
data.companies.forEach((company) => companyStore.add(company));
data.departments.forEach((department) => departmentStore.add(department));
data.employees.forEach((employee) => employeeStore.add(employee));
transaction.oncomplete = () => loadCachedData(db);
} catch (error) {
console.error("Error fetching data from backend:", error);
}
}
The function ‘loadCachedData()’: This function loads data from an IndexedDB database (‘db’) to display it. In summary, this function creates a read-only transaction on three object stores: ‘companyStore’, ‘departmentStore’, and ‘employeeStore’. It gets references to each of these object stores. It retrieves all data from each object store using ‘getAll()’. When the data is retrieved, it calls the ‘displayData()’ function to display the data, passing the type of data (‘Company’, ‘Department’, or ‘Employee’) and the retrieved data as arguments.
function loadCachedData(db) {
const transaction = db.transaction(["companyStore", "departmentStore", "employeeStore"], "readonly");
const companyStore = transaction.objectStore("companyStore");
const departmentStore = transaction.objectStore("departmentStore");
const employeeStore = transaction.objectStore("employeeStore");
companyStore.getAll().onsuccess = function (event) {
displayData("Company", event.target.result);
};
departmentStore.getAll().onsuccess = function (event) {
displayData("Department", event.target.result);
};
employeeStore.getAll().onsuccess = function (event) {
displayData("Employee", event.target.result);
};
}
The function ‘displayData()’: This function displays data in an HTML table. It takes two arguments: type (not used in the function) and data (an array of objects). It clears the contents of a table body element using ‘empty()’. It loops through each object in the data array, and for each object, it creates a table row (<tr>) with five columns containing the object’s properties: ‘firstname’, etc. Finally, it appends each table row to the table body element using ‘append()’.
function displayData(type, data) {
const $tableBody = $("#dataTable tbody");
$tableBody.empty();
$.each(data, function(index, employee) {
const row = `
<tr>
<td>${employee.firstname}</td>
<td>${employee.lastname}</td>
<td>${employee.email}</td>
<td>${employee.departmentName}</td>
<td>${employee.companyName}</td>
</tr>
`;
$tableBody.append(row);
});
}
Let’s take an example of how to query data from an indexedDB database.
The function ‘queryCompanyDB_getAllEmployees()’: This function retrieves all employees from ‘CompanyDB’. First, it opens the ‘CompanyDB" database. When the database is opened, the function creates a read-only transaction on the ‘employeeStore’ object store. It retrieves all employees from the ‘employeeStore’ using ‘getAll()’. When the data is retrieved, it logs the array of employees to the console.
function queryCompanyDB_getAllEmployees() {
const dbRequest = indexedDB.open("CompanyDB");
dbRequest.onsuccess = function(event) {
const db = event.target.result;
const tx = db.transaction("employeeStore", "readonly");
const store = tx.objectStore("employeeStore");
const request = store.getAll();
request.onsuccess = function(event) {
const employees = event.target.result;
console.log(employees);
};
};
}
4. Running the example
If you run the example for the first time, you will see the following web page. The data is initially retrieved from the back-end.
If you go to Chrome DevTools ‘Application’, you can perform some operations like checking the contents of the stores or deleting the database.