Beautiful Filter

Changing the way we use filters in Dashboards/APIs

Recently I came across this UI library called ReUi. And while randomly going through their components, I found this interesting way of handling filters.

It was just so modular, with so many options to choose from. They had the frontend for it, what it needed was a modular enough backend to support it.

Check out their Filters Documentation for more information.

Important Links:

ReUi Filters Documentation
MongoDB Specific Filter Implementation Code
How to use this function in an API Endpoint : Code Snippet

What were they offering?

The component was simple - Import a filter, Create a Filter config (which will hold the filter options and their types), and then pass the config to the filter component. It would then generate a json object with :

A key
Field on which the filter is applied
Operator (like equals, not equals, contains, etc)
Values (given by the user)

You can try the debug mode here.

A sample output for the options Email - Contains - abc.com & Score - Greater than - 50 would be :

[
  {
    "id": "1761907057152-zf96sxgwe",
    "field": "email",
    "operator": "contains",
    "values": [
      "abc.com"
    ]
  },
  {
    "id": "1762003041254-qi5p3qfc6",
    "field": "score",
    "operator": "greater_than",
    "values": [
      "50"
    ]
  }
]

Cool, right? Hold on, it gets better. They have a tonne of filter types like Text, Number, Date, Select, Multi-Select, Boolean, etc. And also a lot of operators for each type such as equals, not equals, contains, does not contain, greater than, less than, between, etc.

The entire list of filter types and their operators :

{
  select: [
    { value: 'is', label: ---.is },
    { value: 'is_not', label: ---.isNot },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  multiselect: [
    { value: 'is_any_of', label: ---.isAnyOf },
    { value: 'is_not_any_of', label: ---.isNotAnyOf },
    { value: 'includes_all', label: ---.includesAll },
    { value: 'excludes_all', label: ---.excludesAll },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  date: [
    { value: 'before', label: ---.before },
    { value: 'after', label: ---.after },
    { value: 'is', label: ---.is },
    { value: 'is_not', label: ---.isNot },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  daterange: [
    { value: 'between', label: ---.between },
    { value: 'not_between', label: ---.notBetween },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  text: [
    { value: 'contains', label: ---.contains },
    { value: 'not_contains', label: ---.notContains },
    { value: 'starts_with', label: ---.startsWith },
    { value: 'ends_with', label: ---.endsWith },
    { value: 'is', label: ---.isExactly },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  number: [
    { value: 'equals', label: ---.equals },
    { value: 'not_equals', label: ---.notEquals },
    { value: 'greater_than', label: ---.greaterThan },
    { value: 'less_than', label: ---.lessThan },
    { value: 'between', label: ---.between },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  numberrange: [
    { value: 'between', label: ---.between },
    { value: 'overlaps', label: ---.overlaps },
    { value: 'contains', label: ---.contains },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  boolean: [
    { value: 'is', label: ---.is },
    { value: 'is_not', label: ---.isNot },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  email: [
    { value: 'contains', label: ---.contains },
    { value: 'not_contains', label: ---.notContains },
    { value: 'starts_with', label: ---.startsWith },
    { value: 'ends_with', label: ---.endsWith },
    { value: 'is', label: ---.isExactly },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  url: [
    { value: 'contains', label: ---.contains },
    { value: 'not_contains', label: ---.notContains },
    { value: 'starts_with', label: ---.startsWith },
    { value: 'ends_with', label: ---.endsWith },
    { value: 'is', label: ---.isExactly },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  tel: [
    { value: 'contains', label: ---.contains },
    { value: 'not_contains', label: ---.notContains },
    { value: 'starts_with', label: ---.startsWith },
    { value: 'ends_with', label: ---.endsWith },
    { value: 'is', label: ---.isExactly },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  time: [
    { value: 'before', label: ---.before },
    { value: 'after', label: ---.after },
    { value: 'is', label: ---.is },
    { value: 'between', label: ---.between },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
  datetime: [
    { value: 'before', label: ---.before },
    { value: 'after', label: ---.after },
    { value: 'is', label: ---.is },
    { value: 'between', label: ---.between },
    { value: 'empty', label: ---.empty },
    { value: 'not_empty', label: ---.notEmpty },
  ],
}

The Backend

Now, the question is how do we implement this in the backend? The json is very modular, it has a lot of types with each type having its own operators. Also some operators can be applied to multiple types (like equals can be applied to text, number, date, etc).

I work a lot with MongoDB, so the implementation is based on that. But the same logic can be applied to SQL or any other database.

So, what I thought was to use the operators as switches and handle the appropraite action (again based on the type). For example - is operator can have $eq operator in MongoDB for number type, while we have to create date ranges for date type (saying that user's query is equal to a date would mean checking from date's start of the day to end of the day).

case "is": {
    if (fieldType === "date") {
        if (!isValueValid(value))
            throw new Error("A valid date value is required.");
 
        const startDate = new Date(value);
        startDate.setHours(0, 0, 0, 0);
        const endDate = new Date(value);
        endDate.setHours(23, 59, 59, 999);
 
        if (isNaN(startDate.getTime()))
            throw new Error("Invalid date format provided.");
 
        // Logic: "is between the start and end of the day"
        queryParts.push({ [field]: { $gte: startDate, $lte: endDate } });
    } else {
        queryParts.push({ [field]: { $eq: value } });
    }
    break;
}

Similarly, we can handle other operators as well.

Error Handling

While implementing this, I realised that there can be a lot of errors that can occur due to invalid inputs from the user.

Fun fact, I didn't realise anything on my own. I was bombarded with errors while testing & I had to handle them one by one.

So, there is a simple function which generates the query based on the filter config, and validates the inputs as well.

case "is": {
    if (fieldType === "date") {
        if (!isValueValid(value))
            throw new Error("A valid date value is required.");
 
        const startDate = new Date(value);
        startDate.setHours(0, 0, 0, 0);
        const endDate = new Date(value);
        endDate.setHours(23, 59, 59, 999);
 
        if (isNaN(startDate.getTime()))
            throw new Error("Invalid date format provided.");
 
        // Logic: "is between the start and end of the day"
        queryParts.push({ [field]: { $gte: startDate, $lte: endDate } });
    } else {
        queryParts.push({ [field]: { $eq: value } });
    }
    break;
}

Putting It All Together

Finally the function looks something like this :

Input : Filter Config (the json object)
Input : Field Types Map (a map of field names to their types)
Output : Object (MongoDB Query + Error)

The entire code can be found here

Now, how to use it in an API endpoint?

import {
  buildMongoQuery,
  FieldTypeMap,
  FilterCondition,
} from "@/utils/filter";
 
...
 
const { filters }: { filters: FilterCondition[] } = req.body || {};
const fieldTypeMap: FieldTypeMap = {
    name: "array",
    email: "string",
    dateOfBirth: "date",
    score: "number",
    isPriority: "boolean"
};
 
const { query, error } = buildMongoQuery(filters, fieldTypeMap);
 
if (error) {
    res.status(400).json({ message: "Invalid filter conditions", error: error.message });
    return;
}
 
const data = await DataModel.find(query || {});

Testing

Checkout the complete code which I used for testing this here

Create environment variables in the backend.
Run the backend server & use the script in scripts/db.seed.js to seed some data.
Run the frontend & navigate to /filterexample page to test the debug UI.
Navigate to /filter to see the filter in action (connected to the backend).

For Some Other Night

I don't know man, it can be anything. I'll see what interesting thing graces me with its presence next time.