Scribd
Upload
13 views19 pages

NodeJS Part6 Databases

This document is part 6 of a 10-part series on integrating databases with Node.js applications, focusing on MongoDB and PostgreSQL. It covers essential topics such as schema design, CRUD operations, relationships, and best practices for data handling in production. The document provides practical code examples for setting up and managing both NoSQL and SQL databases within a Node.js environment.

Uploaded by

alvinregi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
13 views19 pages

NodeJS Part6 Databases

This document is part 6 of a 10-part series on integrating databases with Node.js applications, focusing on MongoDB and PostgreSQL. It covers essential topics such as schema design, CRUD operations, relationships, and best practices for data handling in production. The document provides practical code examples for setting up and managing both NoSQL and SQL databases within a Node.js environment.

Uploaded by

alvinregi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Node.

js Complete Guide
Part 6: Working with Databases
MongoDB and PostgreSQL Integration
Part 6 of 10-Part Series
Modern web applications require robust data persistence solutions. In this part, we'll explore how to integrate
databases with your [Link] applications, focusing on both NoSQL (MongoDB with Mongoose) and SQL
(PostgreSQL) databases. You'll learn schema design, CRUD operations, relationships, transactions, and best
practices for working with data in production applications.

1. Introduction to Databases
1.1 SQL vs NoSQL
Aspect SQL (Relational) NoSQL (Document)

Schema Fixed, predefined Flexible, dynamic

Structure Tables with rows/columns Collections with documents

Relationships Foreign keys, JOINs Embedded or referenced

Scaling Vertical (scale up) Horizontal (scale out)

Transactions ACID compliant Eventual consistency

Best For Complex queries, reporting Rapid dev, unstructured data

1.2 When to Use Each


Choose SQL when: You need complex relationships, strict data integrity, ACID transactions, or well-defined
schemas.
Choose NoSQL when: You need rapid development, flexible schemas, horizontal scalability, or large volumes
of unstructured data.
2. MongoDB with Mongoose
2.1 Setting Up MongoDB and Mongoose
# Install packages
npm install mongoose dotenv

# .env file
MONGODB_URI=mongodb://localhost:27017/myapp
PORT=3000
// config/[Link]
const mongoose = require('mongoose');

const connectDB = async () => {


try {
const conn = await [Link]([Link].MONGODB_URI);
[Link](`MongoDB Connected: ${[Link]}`);
} catch (error) {
[Link](`Error: ${[Link]}`);
[Link](1);
}
};

[Link]('connected', () => [Link]('Mongoose connected'));


[Link]('error', (err) => [Link]('Connection error:', err));
[Link]('disconnected', () => [Link]('Mongoose disconnected'));

[Link]('SIGINT', async () => {


await [Link]();
[Link](0);
});

[Link] = connectDB;

2.2 Creating Schemas and Models


// models/[Link]
const mongoose = require('mongoose');
const bcrypt = require('bcryptjs');

const userSchema = new [Link]({


name: {
type: String,
required: [true, 'Name is required'],
trim: true,
minlength: [2, 'Name must be at least 2 characters']
},
email: {
type: String,
required: true,
unique: true,
lowercase: true,
match: [/^\S+@\S+\.\S+$/, 'Invalid email']
},
password: {
type: String,
required: true,
minlength: 8,
select: false
},
role: {
type: String,
enum: ['user', 'admin', 'moderator'],
default: 'user'
},
profile: {
bio: String,
website: String
}
}, { timestamps: true });
// Hash password before saving
[Link]('save', async function(next) {
if (![Link]('password')) return next();
[Link] = await [Link]([Link], 12);
next();
});

// Compare passwords
[Link] = async function(candidatePassword) {
return await [Link](candidatePassword, [Link]);
};

[Link] = [Link]('User', userSchema);


2.3 CRUD Operations
// controllers/[Link]
const User = require('../models/User');

// CREATE
[Link] = async (req, res) => {
try {
const user = await [Link]([Link]);
[Link](201).json({ status: 'success', data: { user } });
} catch (error) {
[Link](400).json({ status: 'error', message: [Link] });
}
};

// READ ALL
[Link] = async (req, res) => {
try {
const users = await [Link]();
[Link]({ status: 'success', results: [Link], data: { users } });
} catch (error) {
[Link](500).json({ status: 'error', message: [Link] });
}
};

// READ ONE
[Link] = async (req, res) => {
try {
const user = await [Link]([Link]);
if (!user) {
return [Link](404).json({ status: 'error', message: 'User not found' });
}
[Link]({ status: 'success', data: { user } });
} catch (error) {
[Link](500).json({ status: 'error', message: [Link] });
}
};

// UPDATE
[Link] = async (req, res) => {
try {
const user = await [Link]([Link], [Link], {
new: true,
runValidators: true
});
if (!user) {
return [Link](404).json({ status: 'error', message: 'User not found' });
}
[Link]({ status: 'success', data: { user } });
} catch (error) {
[Link](400).json({ status: 'error', message: [Link] });
}
};

// DELETE
[Link] = async (req, res) => {
try {
const user = await [Link]([Link]);
if (!user) {
return [Link](404).json({ status: 'error', message: 'User not found' });
}
[Link](204).json({ status: 'success', data: null });
} catch (error) {
[Link](500).json({ status: 'error', message: [Link] });
}
};

2.4 Querying and Filtering


// Query examples
// Basic find
const users = await [Link]({ isActive: true });
// Comparison operators
const adults = await [Link]({ age: { $gte: 18 } });
const range = await [Link]({ age: { $gte: 18, $lte: 30 } });

// Logical operators
const admins = await [Link]({
$or: [{ role: 'admin' }, { role: 'moderator' }]
});

// Pattern matching (regex)


const johns = await [Link]({ name: /^John/i });

// Field selection
const users = await [Link]().select('name email -_id');

// Sorting
const sorted = await [Link]().sort({ createdAt: -1 });

// Pagination
const page = 2, limit = 10;
const paginated = await [Link]()
.skip((page - 1) * limit)
.limit(limit);

// Count
const count = await [Link]({ isActive: true });

// Aggregation
const stats = await [Link]([
{ $match: { isActive: true } },
{ $group: { _id: '$role', count: { $sum: 1 } } },
{ $sort: { count: -1 } }
]);
2.5 Relationships and Population
// models/[Link]
const postSchema = new [Link]({
title: { type: String, required: true },
content: { type: String, required: true },
author: {
type: [Link],
ref: 'User',
required: true
},
comments: [{
user: { type: [Link], ref: 'User' },
text: String,
createdAt: { type: Date, default: [Link] }
}]
}, { timestamps: true });

const Post = [Link]('Post', postSchema);

// Population examples
// Basic populate
const post = await [Link](postId).populate('author');

// Populate with field selection


const post = await [Link](postId)
.populate('author', 'name email');

// Populate nested fields


const post = await [Link](postId)
.populate({
path: 'author',
select: 'name email'
})
.populate({
path: '[Link]',
select: 'name'
});

// Populate with conditions


const posts = await [Link]()
.populate({
path: 'author',
match: { isActive: true },
select: 'name email'
});

2.6 Middleware Hooks


// Pre and post hooks
[Link]('save', function(next) {
[Link]('About to save user:', [Link]);
next();
});

[Link]('save', function(doc, next) {


[Link]('User saved:', [Link]);
next();
});

// Pre remove hook


[Link]('remove', async function(next) {
// Delete all posts by this user
await [Link]({ author: this._id });
next();
});

// Query middleware
[Link](/^find/, function(next) {
[Link]({ isActive: { $ne: false } });
next();
});
// Aggregation middleware
[Link]('aggregate', function(next) {
[Link]().unshift({ $match: { isActive: true } });
next();
});
3. PostgreSQL with [Link]
3.1 Setting Up PostgreSQL
# Install packages
npm install pg dotenv

# .env file
DATABASE_URL=postgresql://username:password@localhost:5432/myapp
# Or
DB_HOST=localhost
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=password
DB_NAME=myapp
// config/[Link]
const { Pool } = require('pg');

const pool = new Pool({


connectionString: [Link].DATABASE_URL,
// Or individual properties:
// host: [Link].DB_HOST,
// port: [Link].DB_PORT,
// user: [Link].DB_USER,
// password: [Link].DB_PASSWORD,
// database: [Link].DB_NAME,
max: 20, // Maximum connections in pool
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000
});

// Test connection
[Link]('connect', () => {
[Link]('Connected to PostgreSQL database');
});

[Link]('error', (err) => {


[Link]('Unexpected error on idle client', err);
[Link](-1);
});

[Link] = {
query: (text, params) => [Link](text, params),
getClient: () => [Link]()
};

3.2 Creating Tables


// migrations/001_create_users_table.sql
CREATE TABLE IF NOT EXISTS users (
id SERIAL PRIMARY KEY,
name VARCHAR(100) NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL,
role VARCHAR(50) DEFAULT 'user',
is_active BOOLEAN DEFAULT true,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_users_email ON users(email);


CREATE INDEX idx_users_role ON users(role);

-- Trigger to update updated_at


CREATE OR REPLACE FUNCTION update_updated_at_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = CURRENT_TIMESTAMP;
RETURN NEW;
END;
$$ language 'plpgsql';
CREATE TRIGGER update_users_updated_at BEFORE UPDATE ON users
FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
// Run migration
const db = require('./config/database');
const fs = require('fs').promises;

async function runMigration() {


try {
const sql = await [Link]('./migrations/001_create_users_table.sql', 'utf8');
await [Link](sql);
[Link]('Migration completed successfully');
} catch (error) {
[Link]('Migration failed:', error);
}
}

runMigration();
3.3 CRUD Operations with PostgreSQL
// models/[Link]
const db = require('../config/database');
const bcrypt = require('bcryptjs');

class User {
// CREATE
static async create({ name, email, password, role = 'user' }) {
const hashedPassword = await [Link](password, 12);
const query = `
INSERT INTO users (name, email, password, role)
VALUES ($1, $2, $3, $4)
RETURNING id, name, email, role, created_at
`;
const values = [name, email, hashedPassword, role];
const result = await [Link](query, values);
return [Link][0];
}

// READ ALL
static async findAll({ page = 1, limit = 10, role = null }) {
let query = 'SELECT id, name, email, role, created_at FROM users WHERE 1=1';
const values = [];
let paramCount = 1;

if (role) {
query += ` AND role = $${paramCount}`;
[Link](role);
paramCount++;
}

query += ` ORDER BY created_at DESC LIMIT $${paramCount} OFFSET $${paramCount + 1}`;


[Link](limit, (page - 1) * limit);

const result = await [Link](query, values);


return [Link];
}

// READ ONE
static async findById(id) {
const query = 'SELECT id, name, email, role, created_at FROM users WHERE id = $1';
const result = await [Link](query, [id]);
return [Link][0];
}

// READ BY EMAIL
static async findByEmail(email) {
const query = 'SELECT * FROM users WHERE email = $1';
const result = await [Link](query, [email]);
return [Link][0];
}

// UPDATE
static async update(id, { name, email, role }) {
const query = `
UPDATE users
SET name = COALESCE($1, name),
email = COALESCE($2, email),
role = COALESCE($3, role)
WHERE id = $4
RETURNING id, name, email, role, updated_at
`;
const values = [name, email, role, id];
const result = await [Link](query, values);
return [Link][0];
}

// DELETE
static async delete(id) {
const query = 'DELETE FROM users WHERE id = $1 RETURNING id';
const result = await [Link](query, [id]);
return [Link][0];
}
// SEARCH
static async search(searchTerm) {
const query = `
SELECT id, name, email, role, created_at
FROM users
WHERE name ILIKE $1 OR email ILIKE $1
ORDER BY name
`;
const result = await [Link](query, [`%${searchTerm}%`]);
return [Link];
}

// COUNT
static async count(role = null) {
let query = 'SELECT COUNT(*) FROM users';
const values = [];

if (role) {
query += ' WHERE role = $1';
[Link](role);
}

const result = await [Link](query, values);


return parseInt([Link][0].count);
}
}

[Link] = User;
3.4 Complex Queries and JOINs
// Create posts table
const createPostsTable = `
CREATE TABLE IF NOT EXISTS posts (
id SERIAL PRIMARY KEY,
title VARCHAR(255) NOT NULL,
content TEXT NOT NULL,
author_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
status VARCHAR(50) DEFAULT 'draft',
published_at TIMESTAMP,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
`;

// JOIN queries
class Post {
// Get posts with author information
static async findAllWithAuthors() {
const query = `
SELECT
[Link], [Link], [Link], p.published_at,
[Link] as author_id, [Link] as author_name, [Link] as author_email
FROM posts p
INNER JOIN users u ON p.author_id = [Link]
WHERE [Link] = 'published'
ORDER BY p.published_at DESC
`;
const result = await [Link](query);
return [Link];
}

// Get user with post count


static async getUsersWithPostCount() {
const query = `
SELECT
[Link], [Link], [Link],
COUNT([Link]) as post_count
FROM users u
LEFT JOIN posts p ON [Link] = p.author_id AND [Link] = 'published'
GROUP BY [Link], [Link], [Link]
ORDER BY post_count DESC
`;
const result = await [Link](query);
return [Link];
}

// Complex aggregation
static async getStatsByAuthor() {
const query = `
SELECT
[Link],
COUNT([Link]) as total_posts,
COUNT(CASE WHEN [Link] = 'published' THEN 1 END) as published_posts,
COUNT(CASE WHEN [Link] = 'draft' THEN 1 END) as draft_posts,
MAX(p.published_at) as last_published
FROM users u
LEFT JOIN posts p ON [Link] = p.author_id
GROUP BY [Link], [Link]
HAVING COUNT([Link]) > 0
ORDER BY total_posts DESC
`;
const result = await [Link](query);
return [Link];
}
}
4. Transactions
4.1 MongoDB Transactions
// MongoDB transactions (requires replica set)
const session = await [Link]();
[Link]();

try {
// Create user
const user = await [Link]([{
name: 'John Doe',
email: 'john@[Link]',
password: 'password123'
}], { session });

// Create post for user


await [Link]([{
title: 'My First Post',
content: 'Hello World',
author: user[0]._id
}], { session });

// Commit transaction
await [Link]();
[Link]('Transaction successful');
} catch (error) {
// Rollback on error
await [Link]();
[Link]('Transaction failed:', error);
throw error;
} finally {
[Link]();
}

4.2 PostgreSQL Transactions


// PostgreSQL transactions
const db = require('./config/database');

async function createUserWithPost(userData, postData) {


const client = await [Link]();

try {
await [Link]('BEGIN');

// Insert user
const userResult = await [Link](
'INSERT INTO users (name, email, password) VALUES ($1, $2, $3) RETURNING id',
[[Link], [Link], [Link]]
);
const userId = [Link][0].id;

// Insert post
await [Link](
'INSERT INTO posts (title, content, author_id) VALUES ($1, $2, $3)',
[[Link], [Link], userId]
);

await [Link]('COMMIT');
[Link]('Transaction successful');
return userId;

} catch (error) {
await [Link]('ROLLBACK');
[Link]('Transaction failed:', error);
throw error;
} finally {
[Link]();
}
}
// Usage
await createUserWithPost(
{ name: 'Jane', email: 'jane@[Link]', password: 'hashed' },
{ title: 'First Post', content: 'Content here' }
);
5. Database Best Practices
5.1 Connection Pooling
// MongoDB connection pooling (configured in connect options)
[Link](uri, {
maxPoolSize: 10,
minPoolSize: 2,
socketTimeoutMS: 45000
});

// PostgreSQL connection pooling


const pool = new Pool({
max: 20, // Max clients
min: 5, // Min clients to maintain
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000
});

5.2 Indexing
// MongoDB indexes
[Link]({ email: 1 }, { unique: true });
[Link]({ name: 1 });
[Link]({ createdAt: -1 });
[Link]({ role: 1, isActive: 1 });

// Text index for search


[Link]({ title: 'text', content: 'text' });

// PostgreSQL indexes
CREATE INDEX idx_users_email ON users(email);
CREATE INDEX idx_users_name ON users(name);
CREATE INDEX idx_posts_author ON posts(author_id);
CREATE INDEX idx_posts_status_published ON posts(status, published_at);

5.3 Error Handling


// Centralized error handler
class DatabaseError extends Error {
constructor(message, originalError) {
super(message);
[Link] = 'DatabaseError';
[Link] = originalError;
}
}

async function safeQuery(fn) {


try {
return await fn();
} catch (error) {
if ([Link] === '23505') { // PostgreSQL duplicate key
throw new DatabaseError('Record already exists', error);
}
if ([Link] === 11000) { // MongoDB duplicate key
throw new DatabaseError('Record already exists', error);
}
throw new DatabaseError('Database operation failed', error);
}
}

// Usage
try {
const user = await safeQuery(() => [Link](userData));
[Link](user);
} catch (error) {
if (error instanceof DatabaseError) {
return [Link](400).json({ error: [Link] });
}
[Link](500).json({ error: 'Internal server error' });
}
5.4 Data Validation
// Mongoose validation
const userSchema = new [Link]({
email: {
type: String,
required: [true, 'Email is required'],
validate: {
validator: async function(email) {
const user = await [Link]({ email });
return !user || user._id.equals(this._id);
},
message: 'Email already exists'
}
},
age: {
type: Number,
min: [0, 'Age must be positive'],
max: [150, 'Age must be realistic'],
validate: {
validator: [Link],
message: 'Age must be an integer'
}
}
});

// PostgreSQL constraints
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
age INTEGER CHECK (age >= 0 AND age <= 150),
role VARCHAR(50) CHECK (role IN ('user', 'admin', 'moderator'))
);

5.5 Query Optimization


// Use projection to limit fields
const users = await [Link]().select('name email'); // MongoDB
const result = await [Link]('SELECT name, email FROM users'); // PostgreSQL

// Use pagination
const page = 2, limit = 10;
const users = await [Link]()
.skip((page - 1) * limit)
.limit(limit);

// Use lean() for read-only MongoDB queries (faster)


const users = await [Link]().lean();

// Use prepared statements for PostgreSQL


const query = 'SELECT * FROM users WHERE email = $1';
const result = await [Link](query, [email]);

// Avoid N+1 queries with population/joins


// MongoDB
const posts = await [Link]().populate('author');
// PostgreSQL
const query = `
SELECT p.*, [Link], [Link]
FROM posts p
JOIN users u ON p.author_id = [Link]
`;
6. Summary
In this part, we covered comprehensive database integration for [Link] applications:
• Understanding SQL vs NoSQL databases and when to use each
• MongoDB with Mongoose: schemas, models, CRUD operations, and queries
• PostgreSQL integration with node-postgres (pg) library
• Database relationships and population/joins
• Transaction handling in both MongoDB and PostgreSQL
• Best practices: connection pooling, indexing, error handling
• Query optimization and performance tuning
• Data validation and schema design patterns
With your [Link] knowledge from Part 5 and database skills from Part 6, you can now build complete,
data-driven web applications with robust data persistence.

7. What's Next in Part 7?


In Part 7, we'll explore Authentication and Security, covering:
• Password hashing and salting with bcrypt
• JWT (JSON Web Tokens) authentication
• Session-based authentication
• OAuth 2.0 and third-party authentication (Google, GitHub)
• Role-based access control (RBAC)
• Security best practices: CORS, CSRF, XSS prevention
• Rate limiting and DDoS protection
• HTTPS and SSL/TLS certificates
This concludes Part 6 of the [Link] Complete Guide. Practice creating schemas, performing queries, and
optimizing database operations to build efficient, scalable applications.

You might also like