A Short Guide to Streamlining the Last Mile

By David Thain, Product Marketing Manager at Privitar

Data’s last mile

Every competitive enterprise has invested in leveraging their data for competitive advantage and improved performance. Technology solutions promise the uninterrupted end-to-end flow from collection into the data warehouse and out to BI, reporting, and analytics tools where end users create value. In reality, the seamless process hits a buffer in the last mile because individual projects need approval from compliance stakeholders and then governance, security, and privacy measures that fit their unique context.

This technical guide describes streamlined processes and user experiences that improve collaboration, remove last mile obstacles, and enable self-service access to data without friction or delay.

Where data shopping runs into obstacles

Data consumers are the end users who access data to provide business insights and innovation, whether data scientists, analysts, or non-specialist business users. The last mile is only complete when they receive data.

Many organizations have made moves to implement data shopping user experiences built on a data catalog. They’re trying to meet data consumers’ desire to quickly access meaningful data that’s already been approved for use, choosing from an inventory that might hold hundreds or thousands of assets. 

Typically, assets are easy to find and understand in the catalog’s user interface and supported by accurate data classifications, tags, and other descriptors. Users can often add “data products” to a basket and request access, but then they run into obstacles:

• The data catalog doesn’t necessarily distinguish between assets that are already approved and readily available versus those that require more steps.
• The process to agree and approve access is manual and involves multiple interactions and touchpoints, taking considerable time and effort.
• It’s often unclear to business users how regulations apply to a use case or who has the authority to determine compliance steps.
• The steps to apply data protection measures to rows, columns, and cells are manual and may require custom coding.
• Where data catalogs have “policy” features, these are typically a manual checklist of requirements that place a burden on data teams to prepare data.
• These processes are inconsistent, depending on the data, systems, stakeholders, and use cases involved. 

Without automation and defined workflows, the steps to access data proliferate. It’s hard to enforce policies if they aren’t automated within the data stack.

Ironically, from the end user’s point of view, the only things that aren’t restricted are the number of interactions needed to get a decision and the amount of time before they can access data!

Streamlining the last mile 

The solution to the last mile problem is streamlined approval workflows and automated data protections, tightly integrated with your data catalog and other metadata systems.

Privitar’s Data Security Platform gives data consumers a streamlined process to explore, evaluate, and access approved and protected data:

1. Create a project
2. Add data to the project
3. Submit it for approval
4. Access the data

Quicker than ever, you can get on with realizing value from data.

Create a project

As a data consumer, you start by creating a project.

Projects allow you to gather the data you want, describe your use case, and request access for yourself or your team. You’ll have instant access to the data within a project the minute a data guardian approves your request.

Data guardians are the stakeholders, usually in the governance or compliance team, who ensure their organization uses data legally, ethically, and responsibly.

To create a new project, enter a title and a description that explains the goal of your project, then select your project’s purpose from a pre-approved list that aligns with regulatory requirements. Selecting a “purpose” records the intended use of data in simple terms, but it’s also an attribute the platform uses in rules and policies to ensure data is protected for the intended use.

You choose whether access to data in the project should expire on a specific date, which is the recommended best practice, preventing access after that.

Transparency is important for user confidence and good governance. All objects created on the platform—projects, policies, rules, datasets, data assets, and connections—should be given a title, description, terms, and tags that explain their purpose, making it easy for other users to understand what each item is for.

Tags are keywords you can define to describe objects, group objects together, or add context to those objects. For example, you might want to define tags that correspond to geography, line of business, or degrees of sensitivity. Tags help facilitate search and filtering.

Terms are words used within your organization to describe business concepts in plain language. Adding them to the platform ensures consistent use of those words throughout your organization. Terms also lend meaning and give context to physical assets and their fields. When data consumers browse assets, terms allow them to understand the business meaning and semantics of the physical asset. Examples of terms could be “account type,” “customer level,” or “credit risk rating.”

The platform uses one other type of classification to describe fields (columns) specifically: data classes.

Data classes identify particular categories of data, such as date of birth, postal code, and types of national identity number. They ensure that rules can be applied consistently to fields in the same category because they classify data logically even if the fields are labeled differently in their original table.

If you can’t find a tag, term, or data class that matches your criteria, data guardians can create new ones.

Add data to the project

Once you’ve created a project, you search for the data you need, using the Data Exchange in Privitar’s Data Security Platform to explore datasets using an intuitive shop front. You can search or filter by tag to narrow down your options.

Every dataset contains one or more assets. You evaluate the details of each asset, including the fields in the asset and the data class, terms, and tags assigned to each. These are classifications that can be pulled through automatically by synchronizing the platform with a data catalog.

When you find an asset you need, simply click “Add to Project” to add it to your project. You can request as many assets as you need for your project, including assets from different datasets.

Submit it for approval

Once you’ve requested all the data you need for your project, you submit the project for approval. A data guardian is the role which approves or denies your project request for the purpose you’ve requested. 

A data guardian reviews all the information associated with your project and makes sure they have already set up appropriate rules and policies in the platform to protect the assets in the project for the purpose you’ve requested. 

We’ll look at policies and rules shortly, but first, let’s see how you complete the last mile.

Access the data

Often the fastest way to access the data in approved projects is through a SQL query tool, like Microsoft SQL Server, or a business intelligence (BI) tool, like Tableau​. Once your project is approved, you are provided a unique URL for your project to connect to the data assets and retrieve data that’s automatically protected based on policies and rules defined on the platform.

Privitar’s intuitive user interface provides repeatability and consistency, but wider solutions can also leverage all the platform’s functions through comprehensive APIs if you want to build data shopping experiences around the needs of users in other tools. 

Privitar reduces complicated processes and prolonged approval cycles to a matter of a few clicks, simple interactions, and documented decisions, all in one place. Data users get access to the right data at the right time, with secure data provisioned at speed and scale.

Prepare policies for the last mile

Privitar’s Data Security Platform​ provides a system of policies and rules, defined by your governance and compliance users, to meet all sorts of requests for data access in a scalable solution. 

Policies contain flexible, logical rules that allow you to align data protection with approved business use, risk appetite, and regulatory compliance. You could write a policy around a particular regulation or business use case (like GDPR or provisioning data for marketing analytics).

Rather than creating specific policies for each dataset or data source, you create reusable policies based on business context and shared attributes. These policies execute automatically on an unlimited number of data requests. You can reuse policies across different environments and multiple data releases to ensure data is treated consistently.

To protect data for every scenario, data protection must be conditioned by context and applied specifically for each scenario. Given the variety of regulatory requirements, it’s unlikely you want to apply the same treatment to every field in every dataset regardless of user, purpose, or the type of data. 

Rules use conditional logic to manage how data should be treated in specific circumstances. They use data classes, tags, and terms alongside contextual attributes such as user groups, locations, purposes, and more to determine the treatment for data in each field. Rules can obtain these attributes from your data catalog’s metadata. 

Leverage your contextual metadata by synchronizing Privitar’s Data Security Platform with your data catalog.

Pulling data classifications, business terms, and tags from your data catalog streamlines the registration of new data assets on the platform and provides additional context and meaning that helps users understand the data better.

The platform uses these data classifications in combination with  contextual attributes to enforce dynamic policies. Attributes can also be drawn from your data catalog and supplemented your catalog metadata from other sources and systems, including:
— Business, technical, and contextual metadata from, business glossaries, more than one data catalog, and other governance tools.
— Metadata from a data catalog or provided by a user describing the intended purpose for data.
— User and location information available as extensible attributes from identity and access management (IAM) and other systems.
— Attributes derived from data itself, such as a data subject’s location held within a record.

Data security and privacy controls should preserve the business value of data while minimizing the risk of exposing identifiable information. Collectively, the use of your data catalog metadata and other contextual information ensures data protections are appropriate for each use case, while the overall solution is scalable when you’re managing hundreds of use cases. 

The platform provides two types of policy: access control policies and transformation policies. Access control policies filter in or out the records (rows) that are appropriate under particular conditions. Transformation policies protect data (de-identify, anonymize, pseudonymize) but maintain its usefulness for analyses.

Transformations define behaviors or actions—like redaction, generalization, substitution, encryption, or tokenization—that the platform can apply to a field or cell. You manage a library of available transformations in the platform and customize individual actions to ensure particular data classes and formats are handled with subtlety.

Create a Policy

As a data guardian, you’ll use the intuitive policy management interface in Privitar’s Data Security Platform to create policies without needing any coding skills. 

You can add tags to your policies to help other users understand how different policies might be related. For example, you could tag a policy using the name of the regulation it helps your organization comply with. You can add a trigger to a transformation policy so that it only applies under conditions specified by you.

Next you add rules to this policy. 

Create an Access Control Rule

You create access control rules to specify when the platform must deny access to a record (typically rows in a table):

• Conditions deny access based on the value in a contextual attribute.
• Filters deny access based on the value in a data class (field or column).

You create a condition by selecting all the attributes that must be present for this rule to activate. Attributes include tags, terms, purposes, user groups, locations, and more, all of which can be drawn from your data catalog as described earlier. You select values for each attribute. For example, if you select the “User Location” attribute, you can choose from a list of countries.

You create a filter by selecting the data class you want to deny access to. Select an operator, such as “Is any of” or “Is none of” and add as many values as you need. For example, if you select a data class called “Customer Country,” select the “Is any of” operator, and enter “Switzerland,” the platform will drop any record with “Switzerland” in the “Customer Country” column.

Repeat these processes until you’ve specified all the conditions you need.

Create a Transformation Rule

To create a transformation rule, you start by specifying the conditions when transformations will apply to data classes. Then select all the attributes and corresponding values that must be present for the rule to activate. 

Next, you assign transformations at the field level or at the cell level: 

• To add a field-level transformation, select the data class you want to be transformed and assign a transformation, like tokenize, generalize, or redact. 
• To add a cell-level transformation, start by creating a condition. Select a data class and then an operator, such as “Is greater than”, “Is less than”, “Is any of”, or “Is none of”. Then enter a string value for the data class, such as a year or a country. Next, select the data class you want to be transformed and assign a transformation.

Submit it for approval

When you’ve added all the rules you need, you submit the policy for approval by a peer. You can’t approve your own policies. An approved policy will be enforced automatically whenever its conditions are met.

Prepare data assets for the last mile

Data owners are responsible for systems that collect and share data. Whether in lines of business or operations teams, they understand where data comes from, its quality, and what it can be used for. They also own the risk associated with the data’s use.

Privitar’s Data Security Platform gives data owners a simple process to add data and make it available to data consumers:

1. Create a dataset
2. Add assets to the dataset
3. Submit it for approval

Create a dataset

Datasets are logical containers for the data in one or more individual assets. Their purpose is to group data and facilitate an easier search experience. You can think of each dataset on Privitar’s Data Security Platform as a “data product.” 

As a data owner, you create datasets in the Data Exchange. You give each dataset a title, description, and add tags that help other users understand its subject area or domain.

Add assets to the dataset

The next step is to add assets to the dataset you’ve created, describing the assets with as much meaningful context for your data consumers as possible.

Assets are data structures. Examples are the tables in an Oracle or PostgreSQL database. 

To add an asset, simply select a connection and enter the names of the database table and schema containing the asset you want to register. You can create connections between the data sources you own and the platform. Privitar’s Data Security Platform​ returns the structural metadata of the asset (table), such as its fields (column) and their data types. 

You assign a data class, terms, and tags to each field, choosing from the options provided. You can edit multiple fields simultaneously. 

Submit it for approval

When you are happy that all the asset’s details are correct, click “Register”. The new asset is sent to a data guardian for approval.

A data guardian checks that you’ve properly classified the asset, reviewing the data classes, terms, and tags assigned to each field. They can ensure there are appropriate rules for the protection of the data for each potential intended purpose before the asset can be requested for a project.

Once a data guardian approves the asset, it becomes accessible in Privitar’s Data Exchange for data consumers to find and request. Your organization starts to realize its value within minutes rather than months.

Let’s solve your last mile problem

Privitar focuses on reducing the time it takes data consumers to find and access the data they need. 

We solve the last mile problem with streamlined approval processes, automated data protection, and tight integration with your data catalog to drive frictionless access to data.

Are you ready to talk about addressing the last mile problem in your organization? Let’s arrange today to start the conversation.