What is data management? Sensible data handling

Data management is a relatively new field within information technology, emerging in response to growing connectivity and the ever-increasing flood of data in modern businesses. The topic has also become a focus of research. Data science, in particular, is a young discipline dedicated to the sensible storage and integration of data, as well as efficient methods for searching through large datasets.

Definition of data management

Data management defines clear requirements for handling digital information and refers to a process rather than isolated actions. Data should be systematically organized right from the point of collection and entry. Data minimization and quality are key aspects to keep in mind. In addition to safeguarding the content, the data must remain practical and usable for its intended purpose. Another important consideration is determining which data should be archived and for how long. Any data that is no longer needed must be easy to locate and securely deleted without delay.

Types of data

When developing a data management plan, begin by identifying the different types of data your business produces. Categorizing this information allows you to work in a structured way and ensures that no important area is overlooked.

• Personal data: Information that directly relates to specific individuals; classic examples include names, phone numbers, and addresses. Measurement data and shopping habits are also included. It can involve customer data, data of one’s own employees, or third parties. This data requires special protection.
• Protected company data: Internal company information such as accounting data, tax documents, and trade secrets; every company naturally has a keen interest in handling this data carefully. However, it’s quite sensible to define what information belongs to this area as part of data management.
• Secondary data: Data generated for a different purpose during an action; an example would be video surveillance, which is typically installed for protection against burglary and theft. This might also record customer vehicle license plates. Another example is log files in a company network that might store visitors’ IP addresses.
• Public data: Intentionally published and distributed data; this includes information on the website and in company brochures. It’s important to comply with copyright regulations and protect your own data, such as published images, advertising slogans, and company logos. The latter can be protected under the Design Act (formerly the Design Patent Act).

Tasks and implementation of data management

The role of data management is to integrate all processes from data collection to storage or deletion, with a focus on efficiency. It takes into account the entire “lifetime” of the data — the concept behind Data Life Management (DLM).

Data collection

Data processing begins with data collection. Two aspects play a central role here:

1. Data minimization. Only the necessary information should be collected. This obligation is now established in the General Data Protection Regulation (GDPR) should your company do business in the EU. There are also many state laws that govern data collection, such as the CCPA (California Consumer Privacy Act). Data can only be processed if the individuals concerned have given their consent or if it is necessary for legal reasons, such as contract drafting.

2. Data quality: Carefully capturing data saves unnecessary queries and rework. The information should also be stored in the format in which it will later be needed. Any transfer or conversion can lead to errors in the data inventory.

Data storage

The selection of the storage location and format is important. As a storage location, a backup in cloud storage alongside local storage is a good idea. Both solutions have pros and cons, so a combined solution is advisable for very important data:

For large amounts of data, databases are the preferred choice for storage. If specialized software is used, such as for accounting or inventory management, the question of storage location generally doesn’t arise. However, for the latter, attention should be paid to compatibility with external systems and export options, such as the GOBD interface for audits by the IRS.

Data security

Data security is a critical and complex part of data management. Data should be protected from loss, unauthorized changes, and unauthorized access. The National Institute of Standards and Technology (NIST) offers comprehensive guidance, including the continually updated Cybersecurity Framework that outlines potential threats and best practices. In addition to identifying risks, it provides process-based recommendations for securing systems and data. The framework is free to access, and many certifications — such as ISO 27001 — align with its principles.

Possible threats are:

• Hardware damage due to fire, water, or power surge
• Data loss due to improper operation
• Data loss or system malfunction due to malware (ransomware, data theft)
• Data loss due to software errors
• Loss due to theft

To address the various risks, solutions include not only software-based protection mechanisms but also organizational measures such as fire and burglar alarm systems.

Keep these principles in mind:

• Regular updates: Consider the pros and cons of automated versus manual updates. Automated updates have the advantage of running consistently without being forgotten, while manual updates reduce the risk of faulty installations.
• Secure passwords: There are various strategies to consider here. One effective approach is to set guidelines requiring employees to use complex passwords. Regular password changes are also recommended. However, placing too much emphasis on complexity and frequent changes can backfire, leading employees to write passwords down and leave them at their workspace.
• Antivirus/Firewall: Current antivirus protection is essential for any IT system. Depending on the complexity of the network, it may be advisable to use a firewall and, if needed, an Intrusion Detection System (IDS).
• Backup strategy: One of the most important aspects is undoubtedly the proper backup strategy. Relevant data should be thoroughly and regularly backed up on media located in different places. A particular challenge is backing up databases. It may not be possible to simply copy open files during operation. Instead, backups must be performed from within the application used or by using specialized software like MySQL Dump.

Data protection

Data protection is to be distinguished from data security, although there are overlaps. The goal here is simply to ensure that unauthorized persons have no access to confidential data. Data protection includes the following areas:

1. Prevent external access: This requires data security measures

2. Restrict internal access to personal data: This requires a permissions management system in the software used. This way, individual employees are denied access or data sets are only partially displayed. Additional protection is provided by encrypted transmission and storage of data. This ensures that sensitive data is protected from hardware access, such as during a break-in or by unauthorized employees.

Company requirements

Data management must be as practical and intuitive as possible within business processes. This ensures the best acceptance among employees and the greatest effectiveness. Some of the outlined goals are also sensible regarding efficiency improvement. Gathering unnecessary data wastes time and could annoy customers. The organized and secure storage of data enhances productivity.

It may therefore be beneficial for a company to implement a data governance policy that establishes how data should be handled within the company. This particularly concerns data quality and potential improvements through aids like autocorrections. Additionally, consistent terminology and terms are defined.

Archiving

Archiving data that is not currently needed is also an important responsibility for companies. This includes information that must be retained by law, such as invoices and tax records. For this reason, archiving should be an integral part of any data management strategy.

• Separate storage: This reduces the volume of current data backups and ensures data protection.
• Suitable storage medium: Not all storage media are ideal for archiving. For example, hard drives should be powered on regularly to maintain functionality. Optical media such as CDs are vulnerable to external factors and have a limited lifespan. Tape drives with magnetic tapes offer excellent durability and are well-suited for archiving. However, they come with high initial costs for the drives and can be cumbersome to handle. On the plus side, the tapes themselves are inexpensive and have a long service life.

Deleting

Data that is no longer needed should be deleted, which also removes the obligation to ensure its security. A solid data management strategy should therefore make it possible to identify and delete such data separately. Personal data, in particular, must be securely and irreversibly erased.

Deleting with operating system functions typically only makes the data available for overwriting. In fact, they still exist on the hard drive until the storage space is randomly needed and they are overwritten.

Other legal requirements

In addition to federal and state privacy laws, companies in the U.S. are subject to various regulations that require the protection of personal data. If a failure to safeguard such data leads to misuse, business owners or executives can be held liable. Relevant laws and frameworks include:

• California Privacy Rights Act (CPRA)
• Health Insurance Portability and Accountability Act (HIPAA) for health-related data
• Gramm–Leach–Bliley Act (GLBA) for financial institutions

In some countries, a data protection officer must be designated if a company reaches a certain size. In the U.S., it is not a universal federal requirement for private companies, but certain state laws, industry standards, or contractual obligations may require one, especially for larger organizations or those handling sensitive data.

Types of data management

How data management is organized depends on the size of each company. Various approaches to integrated solutions are available on the market. Possible forms include:

• Enterprise Resource Planning Systems (ERP): These systems offer the most comprehensive approach. All resources of the company are recorded and considered. This includes personnel, equipment, and materials. Well-known commercial vendors are SAP, Sage, Oracle, and Microsoft. However, there are also free software solutions like Odoo and OpenZ.
• Master Data Management (Central Master Data Administration): Centralization and revision of a company’s core data. This includes employee data, customer data, and information about equipment. The goal is a uniform data quality that leads to improved usability. This approach is usually considered in ERP systems.
• Content Management Systems (CMS): Predominantly information management systems, for example, in the form of a central intranet for the company. Due to their great flexibility, other aspects like form management and the integration of databases are also possible.
• Document Management Systems (DMS): A subset of data management. They provide forms and offer functions such as filing and archiving.

Challenges of data management

Data management is a dynamic process and must be continually adjusted to current requirements. This results in new challenges each time.

Big data

Data volumes are constantly increasing. Consequently, there are high demands on data management, especially in the following areas:

• Scalability of storage and backup capacities
• Organization and findability of required data
• Data minimization and filtering of important information

Security

Network administrators constantly face new dangers. Information theft through social engineering and sabotage via ransomware are just some scenarios. The more a company digitizes its data resources, the more dependent it becomes on the functionality of the system used. Therefore, it’s essential to stay informed about new risks and make preparations for hardware failure or lack of access to their own systems.

Legal requirements

The introduction of the GDPR has caused a lot of uncertainty and resulted in significant effort for companies that do business in the EU. Still, it is not unlikely that further regulations will follow or changes to existing laws will require adjustments. This may also affect the concept of data management.

Changes in the company environment

Changes in the structure or operations of a company must also be considered when it comes to data management. Provisions can be made for this by using expandable systems or those that are easy to migrate. Regular employee training on internal data governance means additional effort.