Drupal and Lucidworks Managed Search: Speedy, Secure, and Always the Latest Version of Solr

Solr and PHP

Solr has always been very important to SMBs operating websites. Even some of the largest websites in the world rely on Drupal for content management. The Solarium plugin and the Drupal search_api module have been the driving forces behind its adoption. Lucidworks Managed Search now includes out-of-the-box integration with existing Drupal (and WordPress) deployments to support users of the popular CMS platforms through extension of these well known modules. Drupal users can now leverage the fastest most performant scale of the latest stable version of Apache Solr. In this post, I will cover the steps for getting your Drupal installation on Solr from its creators and the sponsoring company behind the Apache Lucene/Solr Project. 

Lucidworks Managed Search (LMS) is our fully managed distribution of Apache Solr, running on GCP, AWS, and soon Azure. It is open source API-compliant and comes with a host of scalability, security, and reliability features so that engineers can rest easy when it comes to Solr and focus on what they do best. LMS is currently in incubation, slated for general availability this summer. If you are interested in an early preview to try out the Drupal integration, contact us today. 

Step 1: Configuring Drupal and LMS

Drupal users will recognize many of the steps to getting up and running because not a lot of steps are needed. Here are the prerequisites for getting started with Drupal on LMS:

PHP >= 7.2
An LMS account: fill out this contact form to request one today!

On a server or local workstation with PHP installed, to setup LMS with Drupal 8 you can run a command as simple as:

composer create-project drupal/recommended-project drupal-site-search

This will create a project in ‘drupal-site-search’ and execute composer install to download the latest stable version of Drupal 8 and all its dependencies. If you need a primer on Composer, see here.

Prior to installing the package, do the following:

export COMPOSER_MEMORY_LIMIT=-1
composer require symfony/event-dispatcher:"4.3.4 as 3.4.35"

A brief explanation of the above steps:

1. The default memory limit imposed by composer may be too low as we install additional dependencies. The first command unsets memory limit constraint.
2. The latest version of search_api_solr recommends this particular step. More info on it over here.
3. This reads the composer.json file from the current directory, processes it, and updates, removes or installs all the dependencies.

We will now install the managed-search-drupal project.

composer require lucidworks/managed-search-drupal

Once you’re done with the above, you can also run:

composer update drupal/core --with-dependencies

This will process your site’s composer.json file and update/remove/install the relevant dependencies.

After you complete the install, navigate to localhost or the IP of your server and port 8080. You should see this page: 

Next, we will install the Lucidworks fork of the search_api_solr module:

composer require drupal/search_api_solr

Next, you will install the following three modules by going to the Extend menu:

1. Search API
2. Search API Solr
3. Search API Solr Defaults

The Search API Solr Search Defaults module provides a default setup of Search API which would save you time to create one from scratch. We will also uninstall the default Search module of Drupal because we’re going to configure the Solr Search module. Here’s how the process looks: 

Step 2: Configuring the Search API Module

Go to the “Configuration” tab and select the “Search API” menu within it:

Select the “Server” Option. The status might read as “Unavailable,” but here is the state we are after and the following steps will help you reach it:

Click on the “Edit” operation for the server. Enter a server name and select “Solr Cloud” under “Configure Solr Backend.” 

Enter the following configuration options:

1. HTTP Protocol – HTTPS
2. Solr Node – enter your host name (e.g. pg01.us-west1.cloud.lucidworks.com)
3. Port: 443
4. Solr Path: /:customer_id/:cluster_id
5. Set your OAuth2 ClientID and Client Secret values
6. Set your default Solr Collection to one that exists in your LMS cluster
7. Set HTTP Method to GET for improved performance. 

Step 3: Set the Schema via Configset

On your Drupal View tab for Solr, you will likely see an error about an incompatible Solr Schema. That’s what we will need to fix.

Here is a Solr Configuration zip prepared for you so that you can use Drupal with LMS, as we do not permit uploads of  untrusted configsets. If you would like us to scan and validate your configset, you can send it over and add it to the approved list. 

To upload the configset we provided to your LMS cluster, you need to run the following command:

curl --location --request POST
'https://pg01.us-west1.cloud.lucidworks.com/php-test/php-test/solr/admin/
configs?action=UPLOAD&name=drupal-new-search' \
--header 'Content-Type: application/octet-stream' \
--header "Authorization: Bearer $TOKEN" \
--data-binary '@drupal-new-search-configset.zip'

You will see that this call requires your LMS Bearer token. Steps to get a new token can be viewed here. In the LMS UI, you can also obtain the command with all your credentials pre-filled in each command after you provide your Client ID and Client Secret.

Once you have uploaded your configset, you can visit the Solr Admin UI to create a collection with the configset applied. If you have already set a collection in the Drupal site, delete the collection prior to re-creating it with the config set.

After this is completed, you can navigate back to Drupal and check that your configset has been correctly applied as seen here:

Step 4: Generating Site Data and Indexing It

First, we need to add some data. Install the devel module which provides utilities for generating site data. Here’s the command: 

composer require drupal/devel

Head back over to the “Extend” tab in Drupal and add the Devel Generate Module.

Next, head to the “Configuration” tab and click the “Generate Content” menu.

Select some options and once you save, your site will have dummy content generated. Here’s what it will look like configured, followed by what it will look like once the generation is done:

Once done:

Since our Search API module is already configured, this data would’ve been indexed in our Solr backend already.

We can confirm this by going to the Configuration > Search API > Solr Index menu:

We can also confirm this by making a search query from Solr Admin UI:

Step 5: Search Content

Our search is already configured, thanks to the Search Api Solr Defaults module. We can confirm this by navigating to “Structure” > “Views” tab:

Then head to the “Solr Search Content”  view:

Here, you can edit the page path from the default /solr-search/content/ to something more obvious like /search (if you are not using the search path for something else):

Great! Now, let’s visit the search page. Our search is configured to work with Lucidworks Managed Search.

Stay tuned for my next post, where I will cover how to get set up on WordPress and LMS to drive more meaningful engagement through improved content discovery. You can reach out to me at marcus.eagan@lucidworks.com!