Survey Design

Four surveys (see appendixes 1–4, available at https://pubs.usgs.gov/publication/ofr20231069) were designed to collect information on the current level of participation (as of March 2021), implementation, challenges, and opportunities for improvement in relation to data management planning and the development and curation of DMPs. The surveys were divided into three main sections. The first section consisted of questions related to project documentation and processing through proposals, work plans, DMPs, approvals and reviews, and project tracking systems. The second section consisted of questions specifically about DMPs, their usefulness, and related challenges. The third section focused on questions about demographics, affiliations, and awareness of DMPs and related project planning topics. Each survey provided an opportunity for the participants to share additional feedback by volunteering to be interviewed on the topics addressed in the survey and by writing their feedback in a freeform text field. Microsoft Forms was used as the survey development and deployment tool, and a Power Automate workflow was used to export the response data from each survey to a Microsoft Excel workbook.

Survey Groups

To cater the survey questions to the participant’s role, survey participants were categorized into four distinct groups: researchers, data managers and information technologists, Center Directors, and Program Coordinators and Bureau Approving Officials. Survey questions were rephrased for each group but tailored to collect the same type of information regarding data management planning and DMPs. The distribution lists for the four survey groups were determined as follows:

Researchers (1,484 individuals):

Data managers and information technologists (hereafter discussed as “data managers”) (529 individuals):

Center Directors (92 individuals):

Program Coordinators (PCs) and Bureau Approving Officials (BAOs) (26 individuals):

All survey group distribution lists were cross-referenced to ensure that individuals would receive only a single email invitation to respond to the survey most appropriate to their role. Subsequent recommendations from survey participants to invite additional individuals to respond to a survey were also collected. A total of 2,131 individuals were invited to take the surveys.

Survey Pilot Testing

Each survey was pilot tested by at least one representative from each survey group. Invitations were sent to survey pilot testers, and their survey responses and feedback were received February 21–23, 2021. The information gathered during this pilot testing phase was used to clarify and validate the surveys' structure, formatting of questions, and response options.

Survey Distribution

On February 24, 2021, the Associate Director of the USGS Core Science Systems Mission Area informed the USGS Executive Leadership Team of the intent to distribute the surveys to USGS staff. Formal written permission from the Executive Leadership Team to distribute the surveys was received on March 1, 2021. Microsoft Word's Mail Merge function was used to create and send standardized individual email notifications to each recipient. Invitations to complete the surveys were emailed to each of the survey groups on March 2 and 3, 2021. On March 17, 2021, reminder emails were sent to those who had not yet completed the survey. The survey completion deadline was March 26, 2021.

Personally Identifiable Information

Where free-text responses included information that could potentially reveal a respondent's identity, affiliation(s), or any other information that might affect their anonymity, the information was manually replaced by USGS Science Data Management staff with a generic phrase (for example, [Center name], [program name], [system name], [person name]). Sensitive information (for example, first and last names, email addresses, and Science Centers) was also redacted from the supporting data as part of the processing and preparation for formal data release (Langseth and others, 2023).

Data Analyses

Jupyter Notebooks (Langseth and others, 2023) were used to perform data manipulation and analyses, basic summary statistics on survey responses, and Pearson’s chi-square tests of independence (χ²) to examine the relations between factors associated with DMP creation, maintenance, and usefulness. In all surveys, the response “I don’t know” was provided as an additional option to Yes or No questions. For analysis of responses to this question type, particularly in the case of the researcher survey, “No” and “I don’t know” responses were treated as a single category. Additionally, the lack of a resource and the lack of knowledge about a resource’s availability were considered to have the same effect.

Interviewee Selection

After the survey closed, interviews were performed with willing participants to follow up on notable themes emerging from the researchers and data managers surveys. Each of the surveys gave participants the opportunity to indicate their willingness to be interviewed (appendixes 1 and 2, question [Q] 29). Interviews were initiated to learn more about the following: (1) what data management resources and training were used by researchers and their preferred formats for them; (2) how and why researchers are using DMPs as living documents and their processes for updating them, if applicable; and (3) the perception of and culture around DMPs as experienced by researchers and data managers (appendixes 5 and 6, available at https://pubs.usgs.gov/publication/ofr20231069).

Interview participants were selected based on their responses to certain survey questions. Ten data managers were interviewed based on their response to the question “Have you found project DMPs to be useful to your work?”; five who responded that DMPs are very or somewhat useful and five who responded that DMPs are not useful at all, somewhat not useful, or neutral. Researchers from each of the categories described in table 1 were interviewed for a total of 10 researcher interviews.

Overall Demographics

The distribution of respondents across all survey groups corresponded roughly to the distribution of USGS staff across DOI Active Directory departments, which equate to USGS Regions or Mission Areas (fig. 1). The DOI Active Directory is a system used to manage USGS personnel information.

The initial email distribution lists consisted of representatives from 107 USGS Science Centers. Researcher respondents represented 75 Science Centers. Data manager respondents represented 62 Science Centers, and Center Director respondents represented 33 Science Centers. A total of 80 out of 114 Science Centers were represented in the responses received for all three surveys. The eight PCs that responded to the survey represented the USGS Ecosystems (four participants), Natural Hazards (one participant), and Water Resources (three participants) Mission Areas.

Roles Related to Data Management Plans

Data managers and Center Directors were asked what their roles were with respect to DMPs at their Centers (appendix 2, Q13, and appendix 3, Q20). The question was multiple choice for both groups, but data managers were also able to enter free-text responses to this question. Center Directors were not given the free-text option because their role is already defined by USGS FSP (USGS, 2017b).

Data managers who reported having roles related to DMPs represented 50 USGS Science Centers (43.9 percent of all Science Centers). Of the data manager respondents who considered themselves to have some sort of role related to DMPs at their Centers, the highest percentage (38.2 percent) reported that they “Ensure project DMPs are created” and “Review or approve project DMPs” (fig. 2). Free-text answers from data managers primarily included more detailed information on their specific roles related to DMPs. For example,

Data managers who reported having some role related to DMPs represented 37 different DOI Active Directory position titles (table 2), and the most common was “IT Specialist.” A minority (ten individuals) had position titles that included the word “data,” indicating that data managers are likely taking on this DMP role as collateral duty.

According to USGS SM 502.6, Center Directors or their designees are responsible for “ensuring the development of data management plans for all new research proposals and the updating of these plans.” When asked about their role(s) with respect to DMPs at their Center, most Center Directors (61.3 percent) responded that they delegate this DMP authority (see USGS, 2017b) to another person or team (fig. 3).

Creation of Data Management Plans

When responding to the first section of the questionnaire, researchers were asked to think about one of their most recent projects. A little over one-half of researchers (51.9 percent) reported that a DMP had been created for the project, either by the respondent (31.9 percent) or by another person (20.0 percent). Fewer respondents (8.8 percent) said that their project work plan referenced an existing DMP. Finally, 39.3 percent of respondents reported that no DMP existed for their project.

In subsequent analyses, responses were grouped into two categories: “DMP created” (including existing DMPs) and “No DMP created.” These two categories were also used for branching within the survey (fig. 4). For example, respondents in the “DMP created” category were asked several followup questions related to access and maintenance of the DMPs (appendix 1, Q8–10).

Data managers were asked if projects at their Centers generally have DMPs (appendix 2, Q7). Data manager respondents predominantly reported that “Most” (31.5 percent) or “A few” (31.5 percent) projects at their Center have DMPs (fig. 5).

Center Directors were asked whether they require projects at their Centers to have DMPs (appendix 3, Q16). Most Center Directors (64.7 percent) answered that “Yes, all of them (projects)” are required to have a DMP (fig. 6).

For those Center Directors who responded that “Most” or “A few” of their Center’s projects have DMPs, a followup question was asked to determine how they decide which projects need a DMP. Some free-text responses to this followup question indicated that the primary deciding factor on whether projects have DMPs is based on the funding source and whether the funder requires a DMP, whereas others stated that the type of project affects whether there is a DMP. For example, fieldwork-based projects will have a DMP, but laboratory-based projects may not, or projects funded by different USGS Mission Areas may have different levels of requirements. Other responses indicated that all projects should have DMPs, but a lack of resources or Center guidance causes them to fall short of this requirement.

Center Data Management Plan Processes, Policies, and Resources

In each survey, respondents were asked several questions about their awareness of DMP processes and resources at their Center. One of the first questions was about whether their Science Center has a documented process for creating DMPs. Thirty-six of the 49 Science Centers represented in the data manager responses reported their Center does have a documented process for creating DMPs. Twenty of the 34 Science Centers (approximately 58 percent) represented in the Center Director responses reported their Center does have a documented process for creating DMPs. When considered along with responses from researchers, 64 out of the 80 Science Centers (80 percent) have at least one person reporting their Center has a documented process for creating DMPs. However, responses from within one-fourth of Centers (20) represented in survey responses indicated there is some disagreement among respondents from the same Center regarding whether documented processes exist. For example, in some cases, a researcher may have reported a positive response, but a data manager or Center Director from the same Center may have reported a negative response regarding whether there is a documented process for creating DMPs at their Center.

One-half of researcher respondents (50.5 percent) indicated their Center does have a documented process for creating DMPs, whereas 41.2 percent of researchers were “not sure” and 8.2 percent answered “No” (their Center does not have a documented process for creating DMPs; fig. 7).

Data Management Plan Creation Support

Less than one-half of researchers (48.2 percent) indicated someone is available at their Center to help them create a DMP (fig. 8). Slightly fewer researchers (37.9 percent) were not sure if help is available, and 13.8 percent reported that help is not available. Researchers who reported having help represent 61 unique Science Centers across the USGS. Data managers and Center Directors were not asked about the availability of someone to help create DMPs because data managers often fulfill this role for their Center and Center Directors are not expected to be involved in DMP creation.

Data Management Plan Update Policy

When asked if their Center has a documented policy (versus a process or procedures) for updating DMPs, most researchers did not know (65.6 percent), less than one-fourth of respondents (22.1 percent) answered “Yes,” and 12.4 percent answered “No” (fig. 9). Researchers who reported having a documented policy for updating DMPs represented 47 unique USGS Science Centers. Data managers and Center Directors were not asked about whether their Center has a documented policy for updating DMPs because this knowledge is assumed for those roles.

Data Management Plan Review and Approval Process

Data managers from 26 Science Centers and Center Directors from 13 Science Centers reported that their Center has a documented process for reviewing and approving DMPs. When these results are combined, they indicate there are a total of 33 unique USGS Science Centers with documented processes for reviewing and approving DMPs. Researchers were not asked about processes for reviewing and approving DMPs because by policy, it does not fall within their area of responsibility.

Even though many researchers were not sure if there is a documented policy for updating DMPs at their Center (fig. 9), approximately one-half of them (49.3 percent) reported project progress reviews do include an update on the status of planned data management activities (fig. 10).

Data Management Plan Access

Of those researcher respondents who had a DMP for their most recent project, the majority (62.6 percent) reported their project DMP was in a shared folder, drive, or database internally available for their Center; however, almost one-third of researchers (32.4 percent) said that the DMP was only available by request from a project team member’s computer (fig. 11). Most data managers (91.9 percent) reported that DMPs for their Center are available in a shared folder, drive, or database, with only 13.5 percent reporting that DMPs are only available by request from a project team member’s computer (fig. 11). Center Directors were not asked this question because they are not directly involved in DMP access and maintenance.

Ideal Location for Data Management Plans

When asked about the ideal location for USGS project DMPs to be stored and accessed, the most common response from researchers (45.0 percent), data managers (65.5 percent), and Center Directors (58.8 percent) was that DMPs should be internally available in a shared folder, drive, or database for their Center (fig. 12).

Data Management Plan Maintenance

Researchers were asked how their DMPs are maintained, either as a living document that receives updates throughout the lifecycle of the project or as a static document that is not changed or updated (appendix 1, Q9). More researchers (42.0 percent) reported their DMPs are maintained as living documents (fig. 13).

Similar results were present when data managers were asked the same question regarding how DMPs for their Center are primarily maintained (appendix 2, Q10). More than one-half (56.0 percent) of data manager respondents indicated their Center’s DMPs are maintained as living documents. Less than one-third (30.7 percent) reported DMPs are maintained as static documents, and 13.3 percent reported they did not know how DMPs are maintained. Center Directors were not asked this question because they are not directly involved in maintaining DMPs.

A Pearson’s χ² test of independence was performed to examine the relation between providing updates about data management during project progress reviews (answer options were “Yes,” “No,” or “I don’t know”) and how DMPs are maintained (answer options were “Living Document,” “Static Document,” or “I don’t know”). The association between these variables was statistically significant, χ² (4, N = 290) = 41.02, p = 0.000. There is a positive association between providing updates about data management during project reviews and maintaining DMPs as living documents (standard residual = 3.93, p = 0.000). There also is a positive association between knowing a policy for updating DMPs exists and maintaining DMPs as living documents, χ² (2, N = 293) = 23.69, p = 0.000 (standard residual = 4.79, p = .000).

Researchers were interviewed to explore the idea of maintaining DMPs as living documents and knowledge of DMP update policies. Interviews of researchers who had reported maintaining DMPs as living documents and having documented DMP update policies in their survey responses revealed that they were often unsure or unaware of policies around updating DMPs, or they disclosed that no formal policy exists. Several interviewees from this group also mentioned they did not update their DMPs but thought it was a good idea or had plans to do so in the future.

Data Management Plan Perceptions

In the second section of the survey, researchers were asked to consider their prior projects when responding to questions related to their perceptions of interactions with DMPs. When asked if they had ever created, or helped to create, a project DMP (appendix 1, Q13), 59.1 percent of researchers answered “Yes” and 40.9 percent answered “No.” Researchers who responded “No” were asked why they had not created a project DMP and were allowed to choose multiple responses from a list, as well as provide free-text responses (appendix 1, Q22). Most of these respondents (62.2 percent) indicated they had never been required to create a DMP for any of their projects, and nearly one-fourth of respondents (23.5 percent) reported someone else had always created the DMP for them. Many free-text responses indicated researchers were not sure about DMP requirements or the purpose of a DMP. For example,

Some researchers also noted they publish data releases and articles describing methods and data management activities, indicating the belief that this fulfills the requirements or purpose of a DMP. For example,

Data Management Plan Challenges

In each survey, participants (researchers, data managers, and Center Directors) were asked to select all the challenges that they face when creating project DMPs, as well as to indicate their greatest challenge (appendix 1, Q19; appendix 2, Q19; appendix 3, Q6). The two challenges that were selected by most researchers and data managers were that no one ever reads the DMP and DMPs take too much time to create (fig. 14, fig. 15). The two challenges selected by the most Center Directors were that a DMP tracking mechanism is needed and that they don’t have a formal process for data management planning (fig. 16).

The top challenge selected by most researchers, data managers, and Center Directors was that DMPs take too much time. The second most common response for researchers was “I don’t face any challenges,” followed by “No one ever reads it.” (fig. 14). The second most common response for data managers was “we don’t have a formal process for data management planning,” followed by “No one ever reads it.” (fig. 15). The second most common response for Center Directors was a three-way tie among the need for a DMP tracking mechanism, tools and templates, and a formal process for data management planning (fig. 16).

Researchers also submitted many other challenges to DMP creation as free-text responses. Some of these responses indicated that researchers need more assistance when it comes to creating DMPs. For example,

“I could use an experienced facilitator or clear templates to help create the DMP.”

USGS researcher

Some researchers also noted that current tools and templates (as of March 2021) are “clunky,” and they feel uncertain how to respond to some of the questions in DMP templates. Other respondents described there is uncertainty at the beginning of the project, which makes them wary of putting something in writing that they may not be able to deliver.

Data managers also included free-text responses when asked about the various challenges faced when creating DMPs. Several respondents reported the value of DMPs is unclear and they are an “administrative burden on researchers.” Data manager respondents also indicated there is “no Center leadership support” for DMPs and “no accountability from Center management for completing or updating a DMP.” Other respondents noted the DMP templates they have used in the past do “not always fit the project.”

When asked about the top challenges faced when creating or encouraging researchers to create DMPs, data managers noted in their free-text responses that USGS researchers “think that [DMPs] have little benefit” and may “need a few cycles of reviews * * * to embrace [the] benefit of DMPs.” Other respondents also noted there is “little guidance on how to complete a DMP, where the DMP templates can be found, or how to view the inventory of already established DMPs.”

Center Directors reported other challenges, including a poor understanding of data management and DMPs among project leads contributing to their reluctance to create them. Additionally, they reported a lack of understanding among project leads of records management tasks, such as archiving, to properly complete and execute DMPs. They also noted that, when created, DMPs are not updated or followed.

Data Management Plan Usefulness

The following sections describe the survey results that are related to DMP usefulness. These results include who DMPs are most useful for, how they are useful, what elements are most useful, what variables might be influencing the perceived usefulness of DMPs, and respondents’ suggestions for making them more useful.

Usefulness by Role

Researchers, data managers, and Center Directors were asked if DMPs were useful to their work (appendix 1, Q15; appendix 2, Q15, appendix 3, Q2), and how useful they thought DMPs are to people in other roles (appendix 1, Q20; appendix 2, Q20, appendix 3, Q8). Available responses to these questions were presented as Likert scales (Joshi and others, 2015). In general, all respondents believe DMPs are most useful for data managers (figs. 17, 18, and 19). Most researchers (fig. 17) reported DMPs were less useful to themselves and others, compared to data managers (fig. 18) and Center Directors (fig. 19), most of whom reported DMPs were somewhat or very useful to themselves and others.

How Data Management Plans are Useful

Researchers, data managers, and Center Directors were asked to describe the ways that DMPs are useful to their projects or work (appendix 1, Q16; appendix 2, Q16; and appendix 3, Q3). Many of the researcher's free-text responses revolved around planning for data release and archiving data at the end of projects, as well as ensuring there is awareness of FSP requirements. For example,

“We have had to figure out where our data will be published if they don't fit into NWIS [National Water Information System]—Science Base is where our team is putting those data.”

USGS researcher

Data manager and Center Director responses echoed this theme, noting DMPs help to catch potential oversights that may cause issues later in a project, especially related to FSP. For example,

“Many * * * project chiefs don't know all there is to know about USGS data collection requirements * * *—leading to FSP issues. The DMP reviews during proposal review process catches these issues early.”

USGS data manager

“[DMPs] also help connect researchers with our database management and support team early in the project life cycle which helps prevent problems later on.”

USGS Center Director

Researchers, data managers, and Center Directors also noted DMPs help ensure they have budgeted time, money, and staff to complete data release and archiving activities. For example,

“[A DMP] Allows me to set aside time/resources to ensure that data management is occurring.”

USGS researcher

“[DMPs are a] very important planning tool. DMPs force management to understand how much time will actually be required to complete the project.”

USGS data manager

“[A DMP] Helps project chiefs think through where all their data are going at the inception of a project, so they can properly budget or estimate staff time for generating data releases and metadata.”

USGS Center Director

In many cases, respondents reported DMPs have been helpful for outlining data roles, responsibilities, and agreements, including documenting when someone is not responsible for releasing data.

“[A DMP] Serves as a disclaimer for data that I shouldn't be responsible for.”

USGS researcher

“DMPs help to get [sic] ensure that all personnel responsible for the data are identified and kept informed from initial creation through project completion and that the necessary resources are planned for.”

USGS data manager

“DMPs can make investigators more aware of the need to plan for the final disposition of data, which can involve more advanced planning about ownership of data, data sharing agreements, nondisclosure agreements, etc.”

USGS Center Director

Although many of the responses revolved around data management processes at the end of the data lifecycle, such as publishing, there were examples of DMPs being useful for processes closer to the beginning of the data lifecycle, such as acquiring, storing, and organizing data.

“Developing the DMP requires thinking about what data will be collected, how it will be organized, and where it will be stored in advance of the onset of data collection.”

USGS researcher

Understanding the anticipated storage size for IT infrastructure purposes was of particular interest to data managers and one Center Director.

“[DMPs have been useful] in planning with our IT department for storage of particularly large datasets, such as imagery.”

USGS data manager

“[DMPs provide] some understanding of local server needs and requirements.”

USGS Center Director

In the free-text question about DMP usefulness, 47 of the 214 researcher respondents (22.0 percent) described why they felt DMPs were a waste of time and not useful to them.

“My research group already has strict data management procedures, processes, and detailed SOPs, so the additional USGS required DMPs do not have much added benefit for my group.”

USGS researcher

Within these responses, there were many statements about how DMPs are only useful for people besides the respondent, such as new researchers, researchers with large projects or datasets, or people in different roles.

“* * * [DMPs are] useful for new * * * scientists. * * * Those of us who have been doing science and research for many years know how to manage data and it is an intrinsic part of what we do.”

USGS researcher

“Usually, I'm the only one doing the work and have a good sense of where data are or where products will be published.”

USGS researcher

“I feel it's probably more useful for IT to receive a heads up for projects for which large amounts of disk or server space may be required.”

USGS researcher

Data Management Plan Element Usefulness

Researchers, data managers, and Center Directors were asked how useful various elements or content sections within a DMP are to their work (figs. 20–22). Researchers, data managers, and Center Directors rated the following as the top three most useful elements that may be included in a DMP:

1. 1) Where the data will be stored and backed up;

2. 2) Plans for data preservation and disposition; and

3. 3) Where the data will be released.

Data managers indicated most of the elements in a DMP are relatively equally important, whereas researcher and Center Director responses showed more variation in the level of importance across elements.

Variables Related to Data Management Plan Usefulness

A series of Pearson’s χ² tests of independence were performed to test which, if any, responses to other questions were associated with researcher respondents reporting that DMPs were useful to their work. The variables that were tested against DMP usefulness were as follows:

• • Whether their Center has a documented process for creating DMPs;

• • Whether their Center has a documented policy for updating DMPs;

• • Whether someone is available at their Center to help with DMP creation;

• • How DMPs are maintained; and

• • Whether project progress reviews include updates on planned data management activities.

There is no statistically significant association (p greater than 0.05) between having a process or policy for creating or updating DMPs and how useful researchers find DMPs to be to their work, χ² (2, N = 286) = 3.77, p = 0.152 and χ² (2, N = 286) = 5.28, p = 0.071, respectively. However, there are statistically significant associations (p less than or equal to 0.05) between whether someone is available to assist with DMP creation, how DMPs are maintained, and whether project progress reviews include updates on planned data management activities and how useful researchers find DMPs to be to their work.

According to the survey responses, whether someone is available to help with DMP creation is significantly associated with DMP usefulness, χ² (2, N = 286) = 7.24, p = 0.027). There is a statistically significant negative association between someone being available to help with DMP creation and researchers reporting that DMPs are not useful at all or somewhat not useful (standard residual = –2.49, p = 0.006).

The way DMPs are maintained also is significantly associated with DMP usefulness, χ² (4, N = 221) = 27.15, p = 0.000. Maintaining DMPs as living documents (those that are updated) is significantly associated with researchers reporting DMPs are very or somewhat useful (standard residual = 4.98, p = 0.000). In addition, the inclusion of data management updates during project progress reviews also is significantly associated with DMP usefulness, χ² (4, N = 283) = 12.79, p = 0.012). Providing updates about data management activities during project progress reviews is significantly associated with researchers reporting the DMPs are very or somewhat useful (standard residual = 3.20, p = 0.001).

Data managers indicated that DMPs are very or somewhat useful in more than 82 percent of responses; therefore, Pearson’s χ² tests of independence were not statistically significant in determining factors related to DMP usefulness for this survey group. Given the comparatively small sample size of Center Director responses (34), nearly three-fourths of which indicated that DMPs are very or somewhat useful, Pearson’s χ² tests of independence were also not statistically significant in determining factors related to DMP usefulness for this survey group.

Project Delays

Researchers, data managers, and Center Directors were asked if they have encountered challenges or delays at the end of a project and if earlier planning and documentation could have improved or mitigated those delays (appendix 1, Q28; appendix 2, Q28, appendix 3, Q22).

Causes of delays commonly described in free-text responses to this question were related to unawareness that a data release was required by USGS policy and uncertainty on how to complete a data release or how much time it would take to release data.

“After manuscripts are approved by the Bureau Approval Official, I am informed a Data Release is required before moving forward. I wish I was told that earlier in the process to make concurrent dissemination. I wouldn't mind doing it, but it was a surprise when I thought everything was done.”

USGS researcher

Data managers noted researchers they worked with were either “surprised” they had to make their data public or that starting the data release process was often left to the last minute. They also noted that the benefits of better documentation throughout the lifecycle of a project was only understood after researchers began the data release process. One Center Director noted failure to plan for the earlier stages in the data lifecycle, such as acquisition, has caused project delays:

“We've had a few occasions over the years where someone gets out in the field with a new acquisition system and they have not thought through how the data will be curated, processed, and disseminated, or who will be able to do the work and how much time it will take.”

USGS Center Director

Additionally, researchers cited a need for help with completing data releases.

“We do not have adequate support to help organize and release the data we collect.”

USGS researcher

Other common causes of delay mentioned by researchers are time-consuming reviews that hold up the release of data and publications and that they didn't properly budget for data management activities. For example,

“* * * the data go through so many reviews, lots of nit-picking, etc., to the point that the data are 1) not quickly released, and 2) hinder significant publications.”

USGS researcher

“* * * [Project delays were caused by the] lack of budget for data cleanup and metadata [generation], lack of plan for timely release of metadata.”

USGS researcher

Researchers and Center Directors reported some challenges were the result of evolving policies or projects. For example,

“The ever increasing burden of changing rules makes it impossible to prepare ahead for these products.”

USGS researcher

“[A project delay] occurs when the project needs evolve after initiation of the project, which happens because we don't always know what will be found during the investigation.”

USGS Center Director

When asked about the ways in which DMPs are useful, (appendix 1, Q16), some researcher respondents described DMPs as being useful for establishing who will be responsible for releasing data at the end of the project. This specific function of DMPs was also mentioned in response to the question regarding project delays (appendix 1, Q28; appendix 2, Q28; appendix 3, Q22) four times by researchers, seven times by data managers, and two times by Center Directors.

“The lack of a data sharing agreement or DMP outlining data responsibilities really hindered their ability to get responses from the data producer.”

USGS data manager

“It is extremely common that data ownership issues are not properly resolved at the beginning of a project, and this becomes an issue when it’s time to release data. Also, issues of data that needs [sic] to be protected or not disclosed is [sic] often not identified at the beginning of the project.”

USGS Center Director

Policy and Resource Awareness

Researchers and data managers were asked if DMPs are required by USGS policy (appendix 1, Q24, appendix 2, Q23). Most researchers (61.7 percent) and data managers (84.3 percent) answered that project DMPs are required, but approximately one-third of researchers (36.9 percent) and fewer data managers (15.7 percent) answered that they did not know if DMPs are required. A small portion of researchers (1.4 percent) and no data managers answered that project DMPs are not required.

Researchers, data managers, and Center Directors were asked which resources they are familiar with related to the creation of DMPs (appendix 1, Q25; appendix 2, Q24; appendix 3, Q21; table 5). The USGS Data Management website resource was selected the most by all three of the survey groups.

In followup interviews, several data managers expressed a need for a primary, common location for resources related to data management and DMPs. Although many data managers knew where to find resources for DMPs and other data management tasks, they mentioned that the resources were widespread across several platforms, websites, and systems and could be difficult to find. For example,

Data Management Plan Purpose and Benefits

Smale and others (2018) describe four purposes for DMPs: (1) to meet funder requirements for data sharing, (2) to gather institutional business intelligence, (3) to educate researchers or change behavior, and (4) to help researchers with project management planning. They argue that organizations often try to make DMPs serve all these purposes, which is nontrivial and likely an impossible task. Results from these surveys have demonstrated that the purpose for DMPs at the USGS is not clear and has been inconsistent in how it has been communicated. The USGS Survey Manual chapter 502.6 (USGS, 2017a) does not explicitly state the purpose for the DMP. The USGS Public Access Plan (USGS, 2016, p. 13) describes what should be in the DMP but also does not explicitly state the purpose of DMPs. It states, “Prior to initiating research, intra- or extramural, approved plans must identify appropriate methods for digital data management, data release, and appropriate preservation in accordance with the USGS Records Disposition Schedules. The plans must also address making data available in appropriate long-term repositories (refer to section 8.1.6 [in the USGS Public Access Plan]) and stress the importance of nonproprietary, open formats for improved accessibility.” Given that the Public Access Plan was developed in response to Federal requirements to increase public access to results from Federally funded research, it may be implied that the purpose of DMPs for intramural and extramural researchers is to ensure compliance with data management requirements, particularly data release and preservation.

The purpose of DMPs is not well-communicated across the Bureau. Many researchers in our survey were unsure about DMP requirements or the purpose and benefits of DMPs and were mainly motivated to create them because they were required by USGS policy or their Center Director (table 3). Data managers and Center Directors, on the other hand, believe that DMPs help researchers define how they will manage and track data throughout their projects, whereas a little less than one-half of the surveyed researchers stated they are motivated to create DMPs because they help them define how they will manage their data throughout the lifecycle of the project. Although this may be an added benefit of DMPs, perhaps it is not or should not be the main purpose.

There does seem to be agreement among these three groups that DMPs do not help researchers learn about new data management practices and requirements. For the most part, USGS researchers believe data management is important, and they do manage their data, but they may not document their plan or at least document it in the way USGS requires. Completing a DMP is not necessarily an educational experience (Smale and others, 2018) and the construction of DMPs should not be used as a data management training proxy. USGS will need to develop a different strategy for promoting leading data management practices and providing education on new requirements. For example, mandatory training on USGS FSP requirements rather than training on DMP development would help researchers know in advance that a USGS data release may be required. This awareness would help resolve some of the delays identified by survey respondents when they found themselves unprepared and lacking resources at the end of their project for comfortably accommodating this requirement.

Data management plans are variously described in existing literature as providing a written record of the data lifecycle within a project including, but not limited to the following: data collection and acquisition, processing, organization and storage (including data-related financial and IT-related requirements), documentation (metadata), quality assurance, access rights, data sharing, publication or release, and archiving and preserving (for example, Jones, 2011; Bishoff and Johnston, 2015; Hudson-Vitale and Moulaison-Sandy, 2019; Smale and others, 2020). However, Smale and others (2018) questioned the view that good DMPs need to address the entire data lifecycle of a project. They also questioned the perceived or assumed benefits of DMP use (as distinct from the purpose[s] of DMPs). They suggested that rather than DMPs, organizations might consider requiring data sharing plans, or DSPs, which focus on describing compliance with data sharing policies. In addition to data sharing, there are various USGS policies that researchers are expected to adhere to, such as for records management (USGS, 2019a) and the Paperwork Reduction Act (USGS, 2019b). Focusing DMPs strictly on data sharing policies may be too narrow for USGS; however, narrowing the focus of DMPs to describe how project teams will meet specific USGS data-related policy requirements could reduce the amount of effort, time, and content they require.

The amount of time that it takes to create a DMP was listed as a major challenge by both researchers and data managers in our surveys. Implementation of more narrowly focused USGS DMPs may also make DMPs easier for data managers and Center Directors to evaluate. Researchers, data managers, and Center Directors all identified the most useful DMP section or element as the one that describes plans for data storage and backup, followed by data preservation and disposition plans and data release and publication (figs. 20–22). In free-text responses, survey respondents reported DMPs are most useful for ensuring FSP requirements are met, such as describing plans for data release and archiving. Some researchers and data managers also noted DMPs as being useful for aspects of their project planning such as budgeting time, money, and staff; however, these uses seemed to be an added benefit that some, but not most, researchers and data managers realize with DMPs.

Recommendations—

Human Infrastructure to Support Data Management Plans

Data management planning often focuses first on development of the workflows and cyberinfrastructure or machinery to facilitate it (Lowe, 1995). And although survey respondents suggested various machine-based solutions for improving DMP implementation at their Science Centers, their responses also highlighted the importance of an equally supported human infrastructure. Humans are necessary for feeding the machinery of data management (Lowe, 1995). Parr and McCarthy (2019) noted that data curators are critical links between end users (for example, which we interpret as both researchers and final users of our data in the USGS context) and developers of data curation services and software (for example, the Science Data Management Branch of the USGS Science Analytics and Synthesis Program).

Researchers benefit from receiving help with data management, sharing, and archiving (Brandt, 2007; Smale and others, 2020; Tenopir and others, 2020). Data management planning and DMP creation should be a team effort that involves staff (beyond the principal investigator[s] and research team) with expertise in the data management, science, and technology domains (Peng and others, 2016, Miksa and others, 2019). Data manager involvement centers around ensuring DMP creation and review and approval. But where data managers are involved or help in DMP creation, researchers are significantly more likely to have a DMP associated with their most recent project and are also significantly more likely to view DMPs positively (as very or somewhat useful). Additionally, when there is a documented process for creating DMPs (that staff are aware of), researchers are significantly more likely to have a DMP associated with their most recent project. Similarly, researchers noted a need for dedicated staff who can provide support in creating DMPs and consultation on data management for specific data types.

The act of writing a DMP urges the writer to think about and try to anticipate their project's resource needs, including those necessary for generating and curating the products and data resulting from their project, and any possible challenges, risks, or dependencies this may entail. When researchers create a DMP alongside a data manager, the process may help researchers and data managers think about, discuss, and document decisions and solutions. As a result, project delays associated largely with a lack of awareness of what is expected of researchers throughout and especially at the end of the project may be avoided or at least mitigated. Having a data manager involved in creating or reviewing a DMP at the beginning of a project, even prior to its approval, would ensure there is a plan to meet USGS information management policy requirements. A review of a project’s work plan and budget by a data manager might also reveal more time or resources are needed for the data release review and approval process. An analysis of the average time USGS data releases take from draft submission through the review and approval process to publication could also help researchers and data managers produce more accurate project schedules and budgets.

Researchers and Center Directors reported that some challenges in maintaining DMPs were the result of constantly evolving policies or projects. For events that cannot be anticipated, planning may not be precise, but involvement of a data manager in DMP creation at the beginning of a project, as well as regular reviews throughout the project lifecycle, could help address potential delays by increasing the overall data management skillset and domain awareness of the project team. In many of these scenarios, simply having a DMP may not be sufficient. Having someone with knowledge and expertise in USGS data management requirements review DMPs can successfully expose issues early in the project lifecycle. This concept of early identification of issues was one of the major themes from the surveys for how DMPs were useful to data managers. Regular reviews of DMPs with a data manager could also address the Center Directors’ concerns that DMPs are not being followed.

The role of a data manager includes the ability to assess, label, and manage data quality (Lowe, 1995). They must be equipped to help authors meet Bureau publishing requirements while still maintaining the integrity, provenance, and correct characterization of information products as they are variously tagged, stored, indexed for search, and published for public consumption. Those in data management roles must also, to some extent, have good communication skills to effectively critique and deliver constructive feedback and instruction to policy makers, approving officials, researchers, and other stakeholders during the project and data management lifecycles.

Information technologists, for the sake of this study, were grouped in with data managers. This grouping is due, in part, to the responsibilities for data management and DMP-related roles (fig. 2) and activities at USGS often falling to staff with other, preexisting primary roles (for example, table 2) including IT professionals. Repository operators or IT professionals and those who provide other cyberinfrastructure and computing resources are often not involved nor even informed at the outset of a project about potential service demands (Miksa and others, 2019 and 2022). Simms and others (2017) also noted that repositories, or personnel involved in managing them, such as IT professionals and library scientists, rarely play an active role in the data management planning process. Information technologists should be more involved in USGS data management planning and DMP creation because they can provide guidance on cyberinfrastructure considerations and requirements (Miksa and others, 2019).

Recommendations—

Resource Needs for Researchers, Data Managers, and Center Directors

Alongside training, USGS researchers, data managers, and Center Directors would benefit from the development of additional resources to facilitate and support DMP creation, maintenance, and curation.

Data Management Plan Accessibility and Storage

Bureau Approving Officials expressed the need for DMPs to be in a location that is accessible to them to support their review and evaluation responsibilities. Researcher, data manager, and Center Director survey responses, as well as some interviewees, also indicated the need for a Center-level, shared location for DMP storage and access, and guidance about its use. This infrastructure solution would need to be accessible by all stakeholders, including BAOs, it would need a complementary communication and awareness-raising campaign about its existence (fig. 12), and it may also need to accommodate DMP maintenance as living documents.

The FAIR guiding principles were developed to clarify what good data management means (Wilkinson and others, 2016). The data lifecycle implementation choices made and documented in a DMP can affect the degree of conformity with the FAIR guiding principles of a project's products (Wilkinson and others, 2016). If all research objects should follow FAIR principles, there is increasing recognition that in addition to data, DMPs should also partially or completely follow FAIR principles (Jones and others, 2020; Miksa and others, 2019; Simms and others, 2017; Wilkinson and others, 2016). Although the Bureau provides guidance on officially recognized solutions for data publishing and storage in trusted digital repositories (for example, Hutchison and others, 2021), USGS lacks the same type of guidance for DMPs. Also, whether maintained as static or living documents, the accessibility of USGS DMPs to the stakeholders identified in this study is limited temporally (for example, to specific periods of the project lifecycle such as proposal evaluation or data publication) or physically (for example, storage on individuals' computers, or on limited-membership network locations). Simms and others (2017) suggested that increasing the “FAIRness” of DMPs, even in unstructured formats lacking persistent identifiers or versioning, can be valuable and incentivize the creation and maintenance of good DMPs.

Survey respondents noted the value of being able to see and emulate other examples of good DMPs. This notion indicates another reason for improving DMP access at least within the Bureau. There was strong support from survey respondents for increasing the accessibility of DMPs internal to the Bureau, but equally strong resistance to, and disagreement with, the idea of publishing or making USGS DMPs publicly available. Some survey respondents expressed hesitation in documenting their DMPs in writing over concerns that they might be penalized for not following them.

Prior to the survey, the survey administrator’s exposure to USGS DMPs was limited to a few examples on the USGS Data Management website and anecdotes from other data managers. However, the volume of survey repondents indicates a higher level of DMP adoption by USGS stakeholders than survey administrators expected (fig. 4). Survey administrators’ lack of evidence or awareness of DMPs may be explained, in part, by the lack of transparency and accessibility of USGS DMP storage and management mechanisms. The discrepancy between researchers (32.4 percent) and data managers (13.5 percent) reporting on availability of DMPs as “upon request from a project team member's computer” (fig. 11) may be because data manager responses represent Centers where data managers have a role related to DMPs. This result may indicate that as data managers get involved with DMP development, the number of DMPs that are only available through request from a team member’s computer may decrease. Moreover, DMP accessibility becomes more aligned with FAIR principles when data managers are involved in the process.

Recommendation—

Data Management Plans as Static versus Living Documents

The maintenance of DMPs as static documents can be difficult and cumbersome (Hudson-Vitale and Moulaison-Sandy, 2019; Miksa and others, 2019 and 2022; Simms and others, 2017; Smale and others, 2020). Survey respondents who said DMPs are useful also expressed that one of the biggest challenges is maintaining DMPs, particularly in a static document format (fig. 14). A lack of usefulness and user-friendliness in processes and workflows can discourage the adoption, use, and user participation in DMPs. Although Mannheimer (2018) found that principal investigators did not use or reference their DMPs as guiding documents during their research projects, at least one data manager in this study noted that DMPs might be more useful if principal investigators had more incentive to complete and maintain them throughout the project lifecycle. Hudson-Vitale and Moulaison-Sandy (2019) assert that an inability to edit DMPs may be one reason why some of the early research into DMP adoption and compliance has shown them to be ineffective. Good document management and control skills and having the time to perform these activities are critical to the successful creation and maintenance of DMPs, especially if they are multiauthored static documents. Researchers need the ability to update their DMPs throughout the project lifecycle to accommodate changes to their project or USGS policies.

Smale and others (2020) observed a gap between the ideal versus actual data management practices of researchers. Many researchers’ survey responses described their Centers as having a policy for updating DMPs and that DMPs are much more useful as living documents, but it became evident in subsequent interviews of survey participants that the actual updating of DMPs was often not happening. Good intentions to maintain DMPs as living documents are evident and formally recognized in Center policies, but perhaps the necessary time, tools, workflows, and motivations are unavailable. Although respondents indicated maintaining DMPs as living documents was a move in the right direction, some also recognized that better mechanisms than those typically used in basic document editing are needed to facilitate efficiencies and benefits of living DMPs. To accommodate changes in RDM that occur during research projects and increase the usefulness of DMPs (as measured by how accurately they represent the data management activities of a project over time), we recommend that USGS DMPs be maintained as living documents if not as partial to fully machine-actionable data management plans (maDMPs; Jones and others, 2020; Simms and others, 2017; Miksa and others, 2019; Miksa and others, 2022). Machine-actionable data management plans are those that “improve the experience for all involved by exchanging information across research tools and systems and embedding DMPs in existing workflows” (Miksa and others, 2019).

Recommendations—