Why does Search matter?

The Post-Covid Working Environment

The pandemic changed working environments dramatically. In the UK in 2019, the ONS stated that 12% of people worked at home for at least part of the week. This has increased to an average of around 41%. New working practices mean that people store and find information differently, and this is certainly contributing to many organisations focussing closely on their search system.

A good Search system means that you can get the job done quickly and efficiently, which is great for business.

Amount of stored data

The amount of data that organisations store increases every year. Some statistics suggest that amounts increased by more than three times between 2019 to 2023 and recent research suggests that data growth is now averaging at 63% a year. This is increasingly cloud-based storage, with organisations typically storing less than 10% on-premise.

As the amount of data that organisations are storing and processing increases, it is vital that the ability to search and find that information is powerful enough and can scale to meet the organisation's needs. This is one of the main factors driving organisations to recognise Search as a key part of their information usage strategy.

Taxonomy breakdown

We looked at some of the disadvantages of taxonomies in our previous article; however, many organisations have encountered taxonomy breakdowns.

This is a similar effect as was encountered by the early internet search engines such as Yahoo, which were entirely taxonomy-driven. This model was quickly proven not to be the best solution for large and growing volumes of data. When Google came on the scene with their intelligent search solution, the taxonomy-based solutions all but disappeared practically overnight.

Organisations are now facing the same challenges and as a result, their Search solution has become a key corporate system.

Global data privacy regulations have forced organisations to understand their data through better data mapping.

Increasing data mapping, data privacy and data governance requirements

Global data privacy regulations such as GDPR (UK and EU), PIPEDA (Canada), CCPA (California), VCDPA (Virginia), ColoPA (Colorado) and many others, have forced organisations to understand their data through better data mapping.

In some cases, this has led to the discovery of repositories of unstructured data that need to be brought under the corporate governance umbrella. As part of that process, it has to be made available to the organisation for use, thus making it searchable and discoverable once again.

The single repository fallacy

Earlier this century, there was a belief that it would be possible to store all documents in a single system. Most organisations who adopted such an approach have unfortunately found that there is not, in most cases, a single system that meets the needs of all of its user communities. This leads to the creation and propagation of the shadow systems that we looked at in a previous article.

SimSage allows users to search from a single search box and perform a federated search to find and display results from their multi-terabyte data estate, in less than half a second.

This ‘single repository’ approach also fails for businesses that are acquisitive, and more likely to have multiple repositories as companies merge. The process of consolidating all systems is challenging, resulting in document management and document storage solutions. These are often more complex to rationalise and consolidate due to their being tightly integrated with other corporate systems.

In these cases, where a single repository has been found to be impossible or impractical to achieve, organisations have sought highly connected search solutions, such as SimSage. This allows users to search from a single search box and perform a federated search to find and display results from their multi-terabyte data estate, in less than half a second.