First Release of a Data Catalog Product That Helped a Swiss Startup Win Its First Client

Summary

A Swiss data-management startup had an idea for a data catalog platform and a potential client who wanted to see it in action before committing. Lacking the in-house resources to build it, the startup engaged INNERLUXES. By customizing the open-source DataHub platform, INNERLUXES delivered a first release in eight weeks — with metadata management, a custom UI tailored to the client's needs, and Azure AD single sign-on — and that release won the startup its first paying customer.

About the Client

The Client is a Swiss startup developing innovative data management solutions.

Proving a Data Catalog Idea to Win a First Client

The Client wanted to build a data catalog platform that would streamline enterprise-wide data search through efficient metadata management and automated access control. It already had a potential customer who liked the idea but wanted to see it working, and — lacking in-house development resources — turned to INNERLUXES for its data management and analytics expertise. Through several interviews, INNERLUXES captured the vision: a SaaS product that stores clients' metadata, supports both conventional and AI-powered search across enterprise data objects, and automates user access management. The team also studied the potential customer — a Swiss medical center that wanted smooth data search across 50+ research papers and BI reports for more than 2,500 employees.

Choosing DataHub to Build a Future-Proof Data Catalog

To avoid costly custom development and roll out faster while still delivering all the required capabilities, INNERLUXES recommended customizing an open-source data catalog platform. After comparing several options, the team chose DataHub for:

• High scalability for stable performance in complex data environments.
• Broad customization capabilities.
• Native integration with a wide range of data tools (such as Amazon S3, Azure Synapse Analytics, MySQL, Google Cloud Storage, and Apache Kafka).
• Ease of integration with AI tools for AI-powered data search.
• The best cost-to-performance ratio.

Since the Client wanted to show the platform to its customer as soon as possible, the team agreed to deliver a first release with native DataHub capabilities and a custom UI tailored to the medical center, leaving a general-purpose UI and AI-based search for a later full-featured version.

A Data Catalog First Release With a Custom UI and SSO

INNERLUXES deployed a DataHub-based data catalog within an AWS Lightsail instance, in a Kubernetes environment with a Goharbor instance to store and maintain deployment configurations. To strengthen security and simplify access, the team implemented single sign-on based on Microsoft Azure Active Directory, aligned with the client's data environment and user roles, and provided a detailed user guide covering login, data upload, search, and more. The catalog works like this:

• Users upload their databases to the platform.
• The platform automatically gathers and structures the relevant metadata (for example, file owner name and research domain).
• Data stewards review the automated metadata and adjust it manually when needed — adding or removing metadata, creating new table columns, and so on.
• The system uses the metadata to power the search.

The custom UI was built to the client's needs — for instance, surfacing research domains in the navigation pane. Users type a request and get the relevant data objects back, with a button to send an automated access request to the object's owner. Once a data object is open, users can search within it, modify tags, leave comments, rate data quality, or navigate the domain tree it belongs to.

Winning the Startup's First Paying Client

In just eight weeks, the Client received a data catalog first release that streamlines enterprise-wide data search and access. Building it on DataHub reduced development costs and sped up the launch, and the tailored features and UI met the medical center's needs, supported by a detailed software guide for smooth adoption. After trying the release, the customer was very satisfied, purchased the software subscription, and expressed interest in the full-featured version. As of June 2024, the Client is planning to add AI-powered search and a universal UI to appeal to a wider range of customers — and the chosen tech stack is set up to avoid scalability or customization limits as the product evolves.

Technologies and Tools

DataHub, Java, React.js, AWS Lightsail, Microsoft Azure Active Directory, Goharbor.