How to Create a Search Engine With SearXNG

Are public search engines too intrusive for you?

Searxng Install 00 Featured Image

SearXNG is a powerful meta-search engine that you can host anywhere. Unlike traditional search engines, SearXNG is an engine aggregator: instead of processing searches, it uses other engines to do the searching. Further, SearXNG is incredibly lightweight and easy to set up, and you can host it on a system as simple as a Raspberry Pi!

Why Install SearXNG

Search engines make it incredibly convenient to browse and discover the Web. However, using a public search engine comes with a price: modern search companies such as Google use your search data to create a highly accurate profile of you.

Searxng Install 03 Google Myactivity

This behavior can feel intrusive to individuals who want to keep their searches private and anonymous. For example, journalists who are reporting about sensitive and controversial topics may not want to have their search queries associated with their names.

Searxng Install 04 Sample Classified Information

One way to deal with this privacy issue is to install and host your own instance of SearXNG to ensure that only you will have access to your search logs.

What You’ll Need

SearXNG requires you to have a few things ready before you can install it:

  • A Fully Qualified Domain Name (FQDN): to use SearXNG, you need to host it through a proper web server with a domain.
  • A machine accessible from outside your home network: this can either be a computer at your home that has a Static IP address or a dedicated Virtual Private Server (VPS) that you are renting.
  • Root access to your machine: SearXNG requires you to install some system tools to configure and host it properly.

This tutorial focuses on installing and setting up SearXNG on a Debian 11 VPS from DigitalOcean.

Searxng Install 05 Digitalocean Droplet

Installing SearXNG’s Dependencies

Before you can install SearXNG, you need to create a separate user account in your system:

sudo useradd -s /bin/bash -d /home/searx -m -G sudo searx
sudo passwd searx
Searxng Install 06 Create New User

Doing this allows you to isolate all the commands and files when installing and configuring SearXNG. It can be especially useful if you intend to host the search engine on a VPS with multiple services.

Switch to the new user account by running su searx, then install all the necessary dependencies for SearXNG:

sudo apt install git nginx iptables iptables-persistent ufw certbot python3-certbox-nginx
Searxng Install 07 Install Dependencies

Configuring Your Firewall

While this is not a necessary step, configuring your firewall ensures that any outside system will only be able to access ports that you authorize to help prevent any malicious actors from flooding your machine with endless requests.

Searxng Install 08 Install Firewall Rules

You can use both iptables and ufw to enable only the ports that SearXNG needs:

sudo iptables -I INPUT -m state --state NEW -p tcp --dport 80 -j ACCEPT
sudo iptables -I INPUT -m state --state NEW -p tcp --dport 443 -j ACCEPT
sudo netfilter-persistent save
sudo ufw allow 80
sudo ufw allow 443
  • The two iptables commands create a new rule that accepts any new incoming HTTP and HTTPS connections, allowing your system to serve the SearXNG webpage to your users.
  • The netfilter-persistent command saves and reloads any changes that you make to your firewall to ensure that your machine has the right settings in between system restarts.
  • The two ufw commands ensure that any current connection in either HTTP or HTTPS remains open for the duration of the session.

Cloning and Installing SearXNG

SearXNG’s developers offer a simple installation script that handles most of the complicated configuration steps.

Searxng Install 09 Searxng Github Page

Obtain this installation script by cloning the program’s GitHub repository:

git clone https://github.com/searxng/searxng searxng && cd searxng
Searxng Install 10 Git Clone Repository

Once the cloning is complete, start the installation process:

sudo -H ./utils/searxng.sh install all

While this process is largely automatic, there are instances where the script asks you to confirm the changes it is making. For example, when the SearXNG script prints a list of programs that it will install, you need to press Y to continue.

Searxng Install 11 Sample Prompt Install

Configuring SearXNG

You need to configure a web server that will broadcast SearXNG, as the search engine is just a process that takes requests and posts results. Without a proper web server, it is not possible to interact with and use SearXNG.

Searxng Install 12 Search Engine Running

Currently, SearXNG officially supports both Apache and Nginx. This section focuses on configuring SearXNG as a standalone service using Nginx.

Configuring Nginx

Create a new configuration file under “/etc/nginx/sites-available/”:

sudo nano /etc/nginx/sites-available/searxng

Write a server block inside the file that will contain the web server’s configuration. For example, this is a basic block that I am using for my SearXNG instance:

server {
 
    # Ports.
    listen 80;
    listen [::]:80;
 
    # Hostname.
    server_name yetanothersearxserver.xyz;
 
    # Logging.
    access_log /dev/null;
    error_log  /dev/null;
 
    # Searx Redirect.
    location / {
        uwsgi_pass unix:///usr/local/searxng/run/socket;
 
        include uwsgi_params;
 
        uwsgi_param    HTTP_HOST             $host;
        uwsgi_param    HTTP_CONNECTION       $http_connection;
 
        # see flaskfix.py
        uwsgi_param    HTTP_X_SCHEME         $scheme;
        uwsgi_param    HTTP_X_SCRIPT_NAME    /searxng;
 
        # see limiter.py
        uwsgi_param    HTTP_X_REAL_IP        $remote_addr;
        uwsgi_param    HTTP_X_FORWARDED_FOR  $proxy_add_x_forwarded_for;
 
    }
}
  • The listen variables tell Nginx that the web server broadcasts a service at port 80. It is important to change this value if you are already running a different virtual host in your system.
  • server_name is a variable that should contain an FQDN that points to your machine. In my case, my webserver uses “yetanothersearxserver.xyz.”
  • The access_log and error_log variables tell Nginx where to save information about your search queries. For example, setting it to “/dev/null” ensures that you are not saving any logs in the machine.
  • The location block tells Nginx what to do whenever a user connects to the web server.
Searxng Install 13 Sample Nginx Config

Installing SSL and Enabling SearXNG

Creating an SSL certificate for your SearXNG instance ensures that every connection you make to your website is secure.

Searxng Install 14 Sample Ssl Certificate

Easily enable SSL for free by signing up with Lets Encrypt‘s certbot. For example, I can run the following command to issue a certificate for my SearXNG host:

certbot --nginx yetanothersearxserver.xyz

Enable your website through Nginx by creating a symbolic link to the web server’s “sites_enabled” folder:

sudo ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/

Reload both SearXNG and Nginx to apply and enable your configurations:

sudo systemctl reload nginx
sudo service uwsgi restart searxng

Frequently Asked Questions

Is it possible to update SearXNG once I install it?

Yes! While you cannot install SearXNG through a traditional package manager, it is possible to update and migrate the search engine from the command line. For example, you can update SearXNG by running the following command: sudo /home/$USER/searxng/utils/searxng.sh instance update.

The installation script is telling me that Redis is not available. Did my installation fail?

No. But this error will most likely cause your SearXNG instance to be unstable, as a Redis failure in SearXNG means it is running in a non-SystemD system and that its anti-bot filtering daemon is not running.

uninstall SearXNG by running the command sudo /home/$USER/searxng/utils/searxng.sh remove allto fix the problem and ensure that you are using SystemD by running sudo systemctl --version.

Is it possible to see what SearXNG is currently doing?

Yes! While SearXNG is a headless daemon that runs in the background, it is possible to check on what it is currently doing. It’s useful in instances where SearXNG is encountering some problems, and you are not sure what is causing issues. For example, I can tap into SearXNG’s debug screen by running sudo /home/$USER/searxng/utils/searxng.sh instance inspect.

Image credit: Isaac Quesada via Unsplash, altered by Ramces Red with SearXNG Logo and Wordmark. All screenshots by Ramces Red.

Subscribe to our newsletter!

Our latest tutorials delivered straight to your inbox