Monday, 16 April 2018 06:29

Make a new document type available to search

By 

Any document type that the publishing-API knows about can be added to our internal search. By default, all document types in internal search also get included in the GOV.UK sitemap, which tells external search engines about our content.

The app responsible for search is Rummager. Rummager listens to RabbitMQ messages about published documents to know when to index documents. For the new document type to be indexed, you need to add it to a whitelist.

Rummager has its own concept of document type, which represents the schema used to store documents in Elasticsearch (the search engine).

Normally, you’ll map your document type an existing rummager document type. If in doubt, use “edition” - this is used for most documents.

Then, modify mapped_document_types.yml with the mapping from the publishing API document type.

If you want a search to be able to use metadata that isn’t defined in any rummager document type, then you’ll need to add new fields to rummager.

Rummager knows how to handle most of the core fields from the publishing platform, like title, description, and public_updated_at. It looks at the body or parts fields to work out what text to make searchable. If your schema uses different fields to render the text of the page, update the IndexableContentPresenter as well.

The part of rummager that translates between publishing API fields and search fields are elasticsearch_presenter.rb. Modify this if there is anything special you want a search to do with your documents (for example: appending additional information to the title).

2. Add the document type to migrated_formats.yaml

Add the document_type name to the migrated list in rummager.

3. Reindex

Reindex the govuk index following the instructions in Reindex an Elasticsearch index

4. Republish all the documents

Republish all the documents. If they have been published already, you can republish them with the publishing-api represent_downstream rake task:

rake represent_downstream:document_type[new_document_type]

You can test that the documents appear in search through the API using a query such as:

Source: This article was published docs.publishing.service.gov.uk 

AOFIRS

World's leading professional association of Internet Research Specialists - We deliver Knowledge, Education, Training, and Certification in the field of Professional Online Research. The AOFIRS is considered a major contributor in improving Web Search Skills and recognizes Online Research work as a full-time occupation for those that use the Internet as their primary source of information.

Get Exclusive Research Tips in Your Inbox

Receive Great tips via email, enter your email to Subscribe.