JSON Data Transformation: Master jq, Lodash & Modern Techniques 2026

# JSON Data Transformation: Master jq, Lodash & Modern Techniques 2026

Transforming JSON data is a daily task for developers—reshaping API responses, filtering large datasets, aggregating analytics, or converting between formats. This guide covers the most powerful tools and techniques for JSON transformation.

Table of Contents

---

Native JavaScript Transformations

Array Methods (map, filter, reduce)

const users = [
  { id: 1, name: 'Alice', age: 30, role: 'admin' },
  { id: 2, name: 'Bob', age: 25, role: 'user' },
  { id: 3, name: 'Charlie', age: 35, role: 'user' }
];

// Extract names
const names = users.map(u => u.name);
// ['Alice', 'Bob', 'Charlie']

// Filter users
const admins = users.filter(u => u.role === 'admin');
// [{ id: 1, name: 'Alice', age: 30, role: 'admin' }]

// Calculate average age
const avgAge = users.reduce((sum, u) => sum + u.age, 0) / users.length;
// 30

// Group by role
const byRole = users.reduce((acc, user) => {
  if (!acc[user.role]) acc[user.role] = [];
  acc[user.role].push(user);
  return acc;
}, {});
// { admin: [...], user: [...] }

Reshaping Objects

const apiResponse = {
  data: {
    user: {
      id: 123,
      attributes: {
        firstName: 'Alice',
        lastName: 'Smith',
        email: 'alice@example.com'
      }
    }
  }
};

// Flatten structure
const flatUser = {
  id: apiResponse.data.user.id,
  ...apiResponse.data.user.attributes
};
// { id: 123, firstName: 'Alice', lastName: 'Smith', email: 'alice@example.com' }

// Rename keys
const renamedUser = {
  userId: flatUser.id,
  name: ${flatUser.firstName} ${flatUser.lastName},
  email: flatUser.email
};
// { userId: 123, name: 'Alice Smith', email: 'alice@example.com' }

Deep Cloning & Merging

// Shallow clone
const shallowClone = { ...user };

// Deep clone (simple objects)
const deepClone = JSON.parse(JSON.stringify(user));

// Deep clone (with functions, dates, etc.)
import { cloneDeep } from 'lodash';
const properClone = cloneDeep(user);

// Merge objects
const defaults = { theme: 'light', notifications: true };
const userSettings = { theme: 'dark' };
const merged = { ...defaults, ...userSettings };
// { theme: 'dark', notifications: true }

Array to Object Conversion

const users = [
  { id: 'user1', name: 'Alice' },
  { id: 'user2', name: 'Bob' }
];

// Array to object (keyed by id)
const usersById = Object.fromEntries(
  users.map(user => [user.id, user])
);
// { user1: { id: 'user1', name: 'Alice' }, user2: {...} }

// Alternative with reduce
const usersById2 = users.reduce((acc, user) => {
  acc[user.id] = user;
  return acc;
}, {});

---

jq Command-Line Tool

Powerful JSON processor for command line and shell scripts.

Installation

# macOS
brew install jq

# Ubuntu/Debian
sudo apt-get install jq

# Windows (via Chocolatey)
choco install jq

Basic jq Operations

# Pretty print JSON
echo '{"name":"Alice","age":30}' | jq '.'

# Extract field
echo '{"name":"Alice","age":30}' | jq '.name'
# "Alice"

# Extract multiple fields
echo '{"name":"Alice","age":30,"email":"alice@example.com"}' | jq '{name, age}'
# { "name": "Alice", "age": 30 }

# Array access
echo '[1,2,3,4,5]' | jq '.[2]'
# 3

# Array slicing
echo '[1,2,3,4,5]' | jq '.[1:3]'
# [2,3]

Filtering Arrays

# users.json
# [
#   {"name":"Alice","age":30,"role":"admin"},
#   {"name":"Bob","age":25,"role":"user"},
#   {"name":"Charlie","age":35,"role":"user"}
# ]

# Filter by condition
jq '.[] | select(.age > 28)' users.json
# {"name":"Alice","age":30,"role":"admin"}
# {"name":"Charlie","age":35,"role":"user"}

# Filter and extract field
jq '.[] | select(.role == "admin") | .name' users.json
# "Alice"

# Count matching items
jq '[.[] | select(.age > 28)] | length' users.json
# 2

Mapping & Transformation

# Transform array
jq 'map({userId: .id, fullName: .name})' users.json

# Add new field
jq 'map(. + {isAdult: (.age >= 18)})' users.json

# Sort by field
jq 'sort_by(.age)' users.json

# Reverse sort
jq 'sort_by(.age) | reverse' users.json

# Group by field
jq 'group_by(.role)' users.json

Advanced jq Examples

# Nested field extraction
echo '{"data":{"users":[{"name":"Alice"}]}}' | jq '.data.users[0].name'
# "Alice"

# Multiple outputs (flatten)
echo '{"a":1,"b":2}' | jq '.a, .b'
# 1
# 2

# Conditional (if-then-else)
jq 'map(if .age >= 30 then . + {senior: true} else . end)' users.json

# String interpolation
jq '.[] | "Name: \(.name), Age: \(.age)"' users.json
# "Name: Alice, Age: 30"

# Unique values
echo '[1,2,2,3,3,3]' | jq 'unique'
# [1,2,3]

# Keys of object
echo '{"name":"Alice","age":30}' | jq 'keys'
# ["age","name"]

# Merge objects
echo '[{"a":1},{"b":2}]' | jq 'add'
# {"a":1,"b":2}

jq in Node.js

npm install node-jq

import jq from 'node-jq';

const data = [
  { name: 'Alice', age: 30 },
  { name: 'Bob', age: 25 }
];

const result = await jq.run('.[] | select(.age > 28)', data, {
  input: 'json',
  output: 'json'
});

console.log(result); // [{ name: 'Alice', age: 30 }]

---

Lodash for Complex Transformations

Installation

npm install lodash

Deep Access & Manipulation

import _ from 'lodash';

const data = {
  user: {
    profile: {
      name: 'Alice',
      settings: {
        theme: 'dark'
      }
    }
  }
};

// Safe deep get
const theme = _.get(data, 'user.profile.settings.theme', 'light');
// 'dark'

const missing = _.get(data, 'user.profile.settings.language', 'en');
// 'en' (default)

// Deep set
_.set(data, 'user.profile.settings.notifications', true);

// Deep clone
const clone = _.cloneDeep(data);

Array Utilities

const users = [
  { id: 1, name: 'Alice', tags: ['admin', 'developer'] },
  { id: 2, name: 'Bob', tags: ['user'] },
  { id: 3, name: 'Charlie', tags: ['developer'] }
];

// Group by field
const byFirstTag = _.groupBy(users, user => user.tags[0]);
// { admin: [...], user: [...], developer: [...] }

// Key by unique field
const usersById = _.keyBy(users, 'id');
// { 1: {...}, 2: {...}, 3: {...} }

// Partition (split into groups)
const [admins, others] = _.partition(users, user => user.tags.includes('admin'));

// Uniq by field
const uniqueByName = _.uniqBy(users, 'name');

// Sort by multiple fields
const sorted = _.orderBy(users, ['name', 'id'], ['asc', 'desc']);

// Flatten nested arrays
const allTags = _.flatMap(users, 'tags');
// ['admin', 'developer', 'user', 'developer']

const uniqueTags = _.uniq(allTags);
// ['admin', 'developer', 'user']

Object Manipulation

const user = {
  id: 1,
  name: 'Alice',
  email: 'alice@example.com',
  password: 'secret',
  role: 'admin'
};

// Pick fields
const publicUser = _.pick(user, ['id', 'name', 'email']);
// { id: 1, name: 'Alice', email: 'alice@example.com' }

// Omit fields
const safeUser = _.omit(user, ['password']);

// Map object values
const uppercased = _.mapValues(user, val => 
  typeof val === 'string' ? val.toUpperCase() : val
);

// Invert (swap keys and values)
const roleToId = _.invert({ alice: 'admin', bob: 'user' });
// { admin: 'alice', user: 'bob' }

Complex Transformations

const orders = [
  { id: 1, userId: 'alice', items: [{ price: 10 }, { price: 20 }] },
  { id: 2, userId: 'bob', items: [{ price: 15 }] },
  { id: 3, userId: 'alice', items: [{ price: 5 }] }
];

// Calculate total per user
const totals = _(orders)
  .groupBy('userId')
  .mapValues(userOrders => 
    _.sumBy(userOrders, order => 
      _.sumBy(order.items, 'price')
    )
  )
  .value();
// { alice: 35, bob: 15 }

// Top 3 most expensive orders
const top3 = _(orders)
  .map(order => ({
    ...order,
    total: _.sumBy(order.items, 'price')
  }))
  .orderBy('total', 'desc')
  .take(3)
  .value();

---

Ramda Functional Approach

Immutable, functional alternative to Lodash.

npm install ramda

import  as R from 'ramda';

const users = [
  { name: 'Alice', age: 30, role: 'admin' },
  { name: 'Bob', age: 25, role: 'user' },
  { name: 'Charlie', age: 35, role: 'user' }
];

// Compose transformations
const getAdultUserNames = R.pipe(
  R.filter(R.propSatisfies(R.gte(R.__, 18), 'age')),
  R.map(R.prop('name')),
  R.uniq
);

const names = getAdultUserNames(users);
// ['Alice', 'Bob', 'Charlie']

// Partial application
const isAdmin = R.propEq('role', 'admin');
const admins = R.filter(isAdmin, users);

// Lens for deep access
const themeLens = R.lensPath(['settings', 'theme']);
const data = { settings: { theme: 'dark' } };

const theme = R.view(themeLens, data); // 'dark'
const updated = R.set(themeLens, 'light', data); // { settings: { theme: 'light' } }

npm install jsonpath

import jp from 'jsonpath';

const data = {
  store: {
    book: [
      { category: 'reference', author: 'Nigel Rees', price: 8.95 },
      { category: 'fiction', author: 'Evelyn Waugh', price: 12.99 },
      { category: 'fiction', author: 'Herman Melville', price: 8.99 }
    ],
    bicycle: {
      color: 'red',
      price: 19.95
    }
  }
};

// All authors
jp.query(data, '$..author');
// ['Nigel Rees', 'Evelyn Waugh', 'Herman Melville']

// All prices
jp.query(data, '$..price');
// [8.95, 12.99, 8.99, 19.95]

// Books cheaper than 10
jp.query(data, '$.store.book[?(@.price < 10)]');

// First book
jp.query(data, '$.store.book[0]');

// Last book
jp.query(data, '$.store.book[-1:]');

// GraphQL-style response -> flat structure
const graphqlResponse = {
  data: {
    user: {
      id: '123',
      name: 'Alice',
      posts: {
        edges: [
          { node: { id: 'p1', title: 'Post 1' } },
          { node: { id: 'p2', title: 'Post 2' } }
        ]
      }
    }
  }
};

const normalized = {
  user: {
    id: graphqlResponse.data.user.id,
    name: graphqlResponse.data.user.name,
    postIds: graphqlResponse.data.user.posts.edges.map(e => e.node.id)
  },
  posts: Object.fromEntries(
    graphqlResponse.data.user.posts.edges.map(e => [e.node.id, e.node])
  )
};
// {
//   user: { id: '123', name: 'Alice', postIds: ['p1', 'p2'] },
//   posts: { p1: {...}, p2: {...} }
// }

import Papa from 'papaparse';

const csv = name,age,role

Alice,30,admin
Bob,25,user;

const result = Papa.parse(csv, { header: true });
console.log(result.data);
// [
//   { name: 'Alice', age: '30', role: 'admin' },
//   { name: 'Bob', age: '25', role: 'user' }
// ]

const users = [
  { name: 'Alice', age: 30, role: 'admin' },
  { name: 'Bob', age: 25, role: 'user' }
];

const csv = Papa.unparse(users);
console.log(csv);
// name,age,role
// Alice,30,admin
// Bob,25,user

const users = Array.from({ length: 100000 }, (_, i) => ({
  id: i,
  name: User ${i},
  age: Math.floor(Math.random()  60) + 18
}));

// Native filter + map
console.time('native');
const result1 = users
  .filter(u => u.age > 30)
  .map(u => u.name);
console.timeEnd('native'); // ~5ms

// Lodash chain
console.time('lodash');
const result2 = _.chain(users)
  .filter(u => u.age > 30)
  .map('name')
  .value();
console.timeEnd('lodash'); // ~8ms

// Winner: Native (but Lodash more readable for complex ops)

Lazy Evaluation with Lodash

// Regular (executes all operations immediately)
const result1 = _.map(
  _.filter(users, u => u.age > 30),
  'name'
);

// Lazy (evaluates only when needed)
const result2 = _(users)
  .filter(u => u.age > 30)
  .map('name')
  .take(10) // Only processes first 10 matching items!
  .value();

---

Conclusion

Choose the right tool for JSON transformation:

Start with native methods, reach for libraries when complexity increases.

Related Resources

• Large JSON Files
• JavaScript & JSON
• JSON in Node.js