Securing Solr: Tips and Tricks You Really Need to Know

As of release 5.2, Solr comes out-of-the-box with both authentication and authorization APIs, allowing you to define users, roles and permissions, using the RuleBasedAuthorizationPlugin and the BasicAuthPlugin.  That’s the good news.  The not-as-good-news is that these plug-ins, while powerful, are a bit counter-intuitive when it comes to configuration.  Thus I took it upon myself to spend some quality time with the Solr security architecture and understand a) just how this framework operates; and  b) to identify it’s various idiosyncrasies.  To that end, I’ve compiled a handy list of things to keep in mind when setting up and/or managing your Solr security.

(Shameless plug):  The security features native to Lucidworks Fusion are much easier to use, and offer a more granular implementation and enhanced access control.  With that said, we move on with securing your Solr instance.

The first step in utilizing Solr authorization/authentication is enabling Solr security.  While you can use the APIs to update your security once it has been enabled, you need to use the ZooKeeper script to upload your security.json file.  To do this, you’ll want to open a command line window and ‘cd’ to your Solr install directory.

Once there, you need to start Solr:

bin/solr start -cloud

From there, you’ll want to make sure your security.json file is in the root, and then run this command.  This will install your security.json in ZooKeeper and all your running nodes.  It will also cause the Solr security framework to be enabled.

server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:9983 -cmd putfile /security.json security.json

Sample security.json

{
"authentication":{
   "class":"solr.BasicAuthPlugin",
   "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
   "class":"solr.RuleBasedAuthorizationPlugin",
   "permissions":[{"name":"security-edit",
      "role":"admin"}]
   "user-role":{"solr":"admin"}
}}

Anatomy of a security.json file

In the above example, we see two top-level attribute declarations: ‘authentication’ and ‘authorization’.   These represent the combined data sent to both APIs, respectively.   The ‘class’ attribute defines the handler class for these APIs.  The BasicAuthPlugin and the RuleBasedAuthorizationPlugin are both packaged with Solr.  You can add your own custom handlers.

In the case of authentication, you’ll want to:

extend AuthenticationPlugin implements ConfigEditorPlugin, SpecProvider

In the case of authorization, you’ll want to:

implements AuthorizationPlugin,ConfigEditablePlugin,SpecProvider

Moving on, in the authentication object you see a ‘credentials’ attribute specified.  This contains the list of authorized users and their SHA-256 hashed passwords.   User management will be discussed later in this article.

In the authorization object, you see a ‘permissions’ object and a list of user-roles, defined by username:role.  Here is where you’ll associate (and effectively create) the roles that will be used in your environment with the users you’ve declared under ‘authentication’.   You can declare as many user:role relationships as you’d like, but keep in mind that:

a) All users must be registered users in the system.

b) The more users and roles you declare, the more complex will be your permissions matrix.

The permissions array is really the endpoint of a security definition.  It uses the previously described definitions of users and roles to bind to a single permission.  This is important, and really where Solr security can get a little hairy.  There is a one-to-one (1:1) relationship between a permission and a role.  The only exceptions to this rule are the wildcard (*), which may be used in any permission role definition, and means that all authenticated users will have this permission; and the ‘before’ keyword, which will for a permission to be evaluated prior to another named permission.  Otherwise, a permission is limited to a single role.  What this means, ultimately, is that you need to lend great consideration to how you construct your permissions matrix.   For example:  if you want to give all users in your system true ‘read’ access, you need to define more than just {“name”: “read”, “role”: “*”}.  At first blush, or rather, intuitively, you would think that this would cover all things needing to be read.  However, this is not the case.  To apply ‘read’ permissions for all users you need to declare:

{"name":"read","role":"*"},{"name":"schema-read","role":"*"},{"name":"config-read","role":"*"},{"name":"collection-admin-read","role":"*"},{"name":"metrics-read","role":"*"},{"name":"core-admin-read","role":"*"}

This will effectively give all your users unfettered read access to all aspects of your Solr admin console.

A real-world working example

Recently, we had a client asking about implementing document level security in Solr.   Solr’s permission framework will allow you to implement granular security from the console to the collection level, however for document level security you’ll need to write your own service to apply an ACP (Access Control Policy) to each document.  That said, Solr is capable, as mentioned,  of providing security from the console to the collection level.   So what if we wanted to go ahead and create such a permissions matrix?

{"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=","devuser":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{"solr":"admin","devuser":"dev"},
"permissions":[{"name":"read","role":"*"},{"name":"schema-read","role":"*"},{"name":"config-read","role":"*"},{"name":"collection-admin-read","role":"*"},{"name":"metrics-read","role":"*"},{"name":"core-admin-read","role":"*"},{ "name":"secure-collection1-permission",
                      "collection":"securecollection",
                      "path":"/select",
                      "before": "collection-admin-read",
                      "role": "admin"
   }]
}}

As you can see in the above example, I’ve defined two users (solr/devuser) and two roles (admin/dev). I’ve then assigned full ‘read’ access to all users, with the exception of one collection, which may only be accessed by a user with the ‘admin’ role. According to that definition, this permission will be applied prior to the collection-admin-read role. This means that only an admin will be able to see the collection named ‘securecollection’. I’ve defined the authorized path as ‘/select’ which means the user will be able to query the collection. The name of this permission is completely arbitrary, with the only constraint being that it cannot be given the name of a built-in permission (E.g. ‘read’, ‘update’, etc). In this case I named it ‘secure-collection1-permission’.

The predefined ‘known’ permissions names, from the class PermissionNameProvider:

COLL_EDIT_PERM("collection-admin-edit", null), 
COLL_READ_PERM("collection-admin-read", null), 
CORE_READ_PERM("core-admin-read", null), 
CORE_EDIT_PERM("core-admin-edit", null), 
READ_PERM("read", "*"), UPDATE_PERM("update", "*"), 
CONFIG_EDIT_PERM("config-edit", "*"), 
CONFIG_READ_PERM("config-read", "*"), 
SCHEMA_READ_PERM("schema-read", "*"), 
SCHEMA_EDIT_PERM("schema-edit", "*"), 
SECURITY_EDIT_PERM("security-edit", null), 
SECURITY_READ_PERM("security-read", null), 
METRICS_READ_PERM("metrics-read", null), 
ALL("all", unmodifiableSet(new HashSet<>(asList("*", null))))

• collection-admin-edit:  Allows users with this role to update collections.
• collection-admin-read:  Allows users with this role read access to collections.
• core-admin-read: Allows users with this role to read certain items in the admin console.
• core-admin-edit: Allows users with this role the permission to edit certain aspects of the admin interface (NOT security).
• core-admin-read: Allows users with this role to read certain aspects of the admin console.
• read: Provides rudimentary ‘read’ access to users with this role.  NOTE: assigning the ‘read’ permission doesn’t actually give the role full read access.  
• config-read: Gives authenticated users ‘read’ access to configuration information.
• config-edit: Gives users with this role permission to access to edit Solr’s configuration.
• config-read: Gives users with this role permission to read configuration settings.
• schema-read: Allows users with this role to read collection schema information.
• schema-edit: Allows users with this role to edit collection schema information.
• metics-read: Allows users with this permission to read Solr metric data.
• all: Provides the associated role with ‘all’ permissions.

… and the allowed attribute values for a permission:

"collection", "role", "params", "path", "method", "name","index","before"

• ‘collection‘ is the given name of a collection in your environment.
• ‘role‘ is the name of a predefined role (defined in ‘user-roles’).
• ‘params‘ specifies query parameters (E.g. &wt=json) which will be filtered.
• ‘path‘ specifies a url path to filter (E.g. ‘/select).  The available paths in Solr are:Registered paths: /admin/mbeans,/browse,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/elevate,/update/json,/admin/threads,/query,/analysis/field,/analysis/document,/spell,/update/csv,/sql,/graph,/tvrh,/select,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms,/debug/dump,/update/extract
• ‘method‘ refers to the HTTP protocol used, one of  the supported protocols: GET,PUT,POST,DELETE,HEAD.  Note that ‘OPTIONS’ is not a supported protocol, and neither are ‘TRACE’, ‘CONNECT’ nor ‘PATCH’.
• ‘name‘ is the given name of the permission, either a known name or a custom name.
• ‘index‘ is the index of the permission once it has been registered in the system.  NOTE: Some older documentation specifies using the permission name for the ‘delete-permission’ action.  However, you need to use the permission index (an integer) rather than the name (a string).  You can find the index of a stored permission by calling the authorization API
• ‘before’ specifies that this permission will be applied prior to the named permission.

You can download the full rules api specification here.

Managing Users

Solr’s user management API is Spartan, is arguably lacking some necessary actions.  You can add, edit, delete users and passwords, however once you’ve uploaded your security.json file and you start using the API, you’re initial security.json is now out of date, and if you were to upload it again, it would overwrite whatever user management changes had been made previously.  This is why it’s a good idea to maintain an update-to-date security.json file.  This way if you ever need to migrate your node clusters, you don’t have to worry about setting up the user base from scratch.  You would just upload your security.json file and off you go!

To that end, let’s go over a best-practice user management workflow.

1. Upload your security.json
2. Add a user:

   curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
     "set-user": {"tom" : "TomIsCool"}}'
   

3. Now, let’s get tom’s SHA-256 password hash:

   curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication 
   
   

   and the response will look something like:

   {
     "responseHeader":{
       "status":0,
       "QTime":3},
     "authentication.enabled":true,
     "authentication":{
       "class":"solr.BasicAuthPlugin",
       "credentials":{
         "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
         "tom":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}}}
   
   

4. Now, copy the ‘tom’ entry, and add it to your security.json file.  This way, you maintain a current list of authorized users, and you won’t have to recreate them if you ever have to change a permission, change your security.json file, or migrate your environment.   Admittedly, this is something of a roundabout method, but it will save you time and in many cases, effort.

Troubleshooting Your Permissions Implementation

Security implementations are, more often than not, very difficult to troubleshoot.  Often these implementations don’t provide cogent error messages, or worse, no message at all.  Such is not the case with Solr.  Solr returns a 403 ‘Unauthorized’ error in general.  However, if you upload a security.json that doesn’t behave the way you would expect, go to your solr.log file, and look for error messages.  They will be your best friends in debugging a Solr permission matrix.  Here are some common errors:

• No Authorized User: 
  2017-03-08 15:47:55.876 INFO (qtp1348949648-16) [ ] o.a.s.s.HttpSolrCall USER_REQUIRED auth header null context : [FAILED toString()]
• Invalid Role:
  “role”:”admin”}, The principal [principal: devuser] does not have the right role
  This will be your most common exception when dealing with user permissions!

In a Nutshell: Important Tips and Tricks when using Solr Security

1. Your Solr installation should be secured behind a Firewall.
2. The user under which Solr is started should only have write access to the Solr home directory ( by default $SOLR_INSTALL/server/solr )
3. Permissions are FIFO (First In, First Out).  That is, they are evaluated in the order in which they are declared.
4. There is a 1:1 relationship between roles and permissions. Any given permission (E.g. ‘read’) is only evaluated once, the first encountered in the list. If you have a second ‘read’ defined down the line, it will not be honored.
5. You can use an asterisk (*) to assign all permissions to all users. { “name”: “read”, “role”: “*”}
6. The ‘role’ attribute takes either a wildcard ‘*’ or single role name string as an argument.  You cannot pass arrays or comma-delimited strings.
7. You can use the permission ‘all’ to assign all permissions to a given role. { “name”: “all”, “role”: “admin”}
8. You should err on the side of fewer permissions.  Only use just what you need.  “Less is more.”  That said, there is a fine balance between what is required, and otherwise going overboard.
9. When deleting permissions, you MUST to use the ‘index’ integer, rather than the permission name.
10.  The wildcard permission is your friend.
11. If you assign the generic ‘read’ role for the console, you need to also assign collection-admin-read and core-admin-read to the same role.  Likewise, for ‘update’, you’ll need to specify the core-admin-edit and collection-admin-edit to the same role.
12. When defining a permission, only the ‘role’ parameter is required.
13. Use a library such as SolrJ to communicate with ZooKeeper.
14. When using a pre-specified (or ‘named’) permission, the only allowed attributes are: name, role, collection, index. 

Happy (Secure) Searching!