A kind-of brief shared early history of US-RSE

Published: Feb 6, 2022 by Vanessa Sochat, Daniel S. Katz, Ian Cosden, Christina Maimone, Charles Ferenbaugh, Sandra Gesing, Jeffrey Carver, Chris Hill, Nicole Brewer, Peter Vaillancourt, Mark Carroll, Peter Elmer

The history of a community is a valuable thing to know, including everything from early correspondence to how events unfolded and the culture at the time. We, the US Research Software Engineer Association, are finishing up our third year and starting our fourth, and while our history is still somewhat fresh in our minds, we see the importance of an effort to capture it. While events may be clear to us now, details will be buried in oblivion with many activities going on. The future community will not be able to see into our experiences, and it will become increasingly more challenging to understand how US-RSE grew from just a handful of people to a thriving community. This document aims to help with that, including not just written accounts but also historical documents to go along with them. This is a “kind-of-brief” history of US-RSE, the title inspired by the similar post about the UK RSE group, and it is a living document that we hope will be expanded as our community continues to grow.

The Prehistoric Era

Before there was US-RSE, there was a need for sustainable software. Those who developed research software in universities seldom had long term careers, or positions that recognized their value. A lot of software was developed in research projects, but not shared, leading others to later develop similar software rather than working together. And the reproducibility crisis [ref] that happened in the second decade of the 21st century was rooted in the idea that the results of research could not be trusted, because studies could not be reproduced, or even replicated. As software plays an essential part in research [ref], it comes as no surprise that groups such as the Software Sustainability Institute in the UK were championing both the importance of software and the people creating it, research software engineers (RSEs) as early as 2012. Indeed, before the term was known in the United States, there was a vibrant community growing across the pond [ref].

A Research Software Signal

Even before any establishment of US-RSE, or awareness of the term, there was both an awareness of needing research software, and a desire to connect with others to work on it. These smaller stories, in chronological order, from our community members offer glimpses into this awareness, and further detail on each event can be explored in the links provided.

• When they were created in 1985, the five original NSF supercomputer centers (NCSA at Illinois, SDSC at UCSD, PSC at CMU and Pittsburgh, CTC at Cornell, and JVNC at Princeton) had base funding from NSF that allowed them to hire software developers and applications consultants, many of whom were what we today would call RSEs.
• Mark Lundstrom at Purdue University had the vision for easy-to-use graphical user interfaces for simulations for nanotechnology and created Punch in 1996 that led to the development of nanoHUB and HUBzero developed by an RSE team growing further under Gerhard Klimeck and Mike Zentner. Sandra Gesing joined the HUBzero team part-time in 2017 to explore diverse applications and funding possibilities for supporting the research software engineering team. Mike Zentner transitioned the team and product in 2019 to SDSC.
• Dan Katz created and led a group of proto-RSEs, called the Cyberinfrastructure Development (CyD) Division at the LSU Center for Computation and Technology (CCT) starting in 2006, which was later led by Shantenu Jha and Steven Brandt.
• Jarek Nabrzyski started to lead the Center for Research Computing (CRC) at the University of Notre Dame in 2009 and expanded its mission from an HPC Center to a Research Computing Center. He successfully increased the staff from 7 positions to about 50 positions in a little bit over 10 years with about 25 research software engineers and 5 computational scientists. Sandra Gesing joined the CRC in 2013 as research software engineer and transitioned to a computational scientist and research faculty in 2014 integrating RSE into her proposal writing and research.
• While working at NSF, Dan Katz started interacting with the Software Sustainability Institute in 2013, as the initial UK RSE groups were beginning to make an impact, which Dan observed and starting thinking about how to bring this model to the US.
• Vanessa Sochat had vision for a layer of academic software developers at an institution in January of 2016, although she had never heard of research software engineering [ref].
• Ian Cosden founded the Princeton Research Software Engineering group in 2016, with the initial goal to help researchers optimize their software running on Princeton HPC systems.
• Dan Katz met Ian Cosden at an NSF-funded workshop on research software collaborations between high energy physics and computer science in Princeton on May 2, 2017 [ref]. There they discussed Ian’s group, a nascent RSE group, telling Ian about the RSE concept and term. Dan had met or collaborated with several individuals that would turn into the initial leaders of US-RSE over a decade later.
• NSF started to fund conceptualizations of sustainability institutes in 2015 and Karthik Ram, Jeffrey Carver, Sandra Gesing, Dan Katz and Nic Weber are the co-PIs on the conceptualization of the US Research Software Sustainability Institute (URSSI) that was funded from 2017-2021. Activities include community building via workshops, a large-scale survey and ethnographic studies. The implementation plan for URSSI can be found here.

This message about sustainability and research software trickled quickly into the United States. As the community in the UK was flourishing, this provided an opportunity for some of us in the United States to see what was happening, and to participate in these UK events, including (but not limited to) the first RSE conference in the fall of 2016. These early connections and discussions at these events are what led to the first stages of what would become the US-RSE, discussed next.

The Idea of a US-RSE

The ultimate formation of US-RSE was driven by people. It came down to individuals finding one another via shared ideas and goals, and then engaging in specific correspondence and collaborations that resulted from these opportunities for interaction. The early formation of US-RSE was exactly that - a few individuals in the United States engaging with other RSE groups, and within themselves to eventually lead to the founding of US-RSE. This section will detail some of this correspondence.

The International RSE Survey

In September 2017, after attending the first international RSE conference, Dan Katz volunteered to customize the questions and answers for US participants in the first international RSE survey. This is a great example of an early effort to participate in the international RSE movement, and Dan wanted others involved too. As a result of thinking about this survey in context of a session on science gateways [ref] for the Research Data Alliance Plenary (RDA) [ref] Dan reached out to Sanda Gesing on September 26, 2017:

Hi Sandra,

To follow-up on a brief discussion at the gateways meeting at RDA:

I’ve “volunteered” to work with the UK Software Sustainability Institute (SSI) to create a US-flavored version of the RSE survey that they’ve done for 2 years now. I wonder if you might be interested in working on this as well?

Basically, I’m working with Simon Hettrick to “translate” the survey questions and answers, which is all done in GitHub. I also plan to add another note that says if the respondents are interested [and] want to be part of a possible RSE group, or if they want to organize such a group, they should do something (maybe volunteer through a separate Google form, so that the data in the survey itself will remain anonymous.)

The survey will be run by Southampton, and the data will be published, so I don’t believe there are any IRB issues to worry about, but I welcome your opinions on this too.

Does this sound at all interesting?

If so, do you have suggestions for other people who might also be interested?

Thanks,

Dan

Although Dan doesn’t explicitly say it, in his words he speculates on creation of a “possible RSE group” and formulates his message around the idea of collaborating with the UK RSE movement. Sandra Gesing received the email, and was excited about the topic and eager to join Dan to do the work. Dan also reached out to other organizations that might have interest, such as Campus Champions on November 20 of that same year:

Subject: [campuschampions] Research Software Engineer survey

Are you an Research Software Engineer in the United States? Do you work professionally in software in a research environment, at least partly writing code for other people? Your job title might be research programmer, facilitator, postdoctoral researcher, or research assistant. If so, please fill out this short survey, being run by the UK Software Sustainability Institute with the idea of understanding and helping to organize the US RSE community: http://bit.ly/US-RSE-2017-survey (Please feel free to forward this to others, particularly possible RSEs in your center or at your university.)

– Daniel S. Katz

By the end of the survey, 10 people had volunteered to work on a US RSE organization.

The Founding Emails of US-RSE

In the new year after the survey (2018), the first International RSE Leaders Workshop would be held in the UK at the Alan Turing Institute in London. Sandra Gesing (Notre Dame), Ian Cosden (Princeton), Chris Hill (MIT), David Bernholdt (Oak Ridge National Laboratory), and Daniel Smith (Virginia Tech) were the US people who attended.

To continue the momentum from the survey, Dan and Sandra followed up with an email on January 25, 2018 to introduce folks and encourage further discussion at the international meeting. With two more folks who attended the international leaders meeting but hadn’t volunteered via the survey, plus Dan and Sandra, there were now 14 people in this small group.

Hi,

The first 10 of you are the people who went through the US RSE survey to the end, and then took the extra step of filling out the form saying that you were interested in helping to organize a US RSE organization.

[10 names/emails, including Ian Cosden, Chris Hill, and Lance Parsons]

In addition, there are 5 US people going to an international RSE meeting in the UK next week, including Chris and Ian and Sandra (who worked with me to create the US version of the survey). The other two are:

[2 names/emails]

I’m now writing to introduce you to each other, at least electronically. It might be useful for each person to write a short email saying who they are and why there are interested.

After that, I want to encourage you to talk to each other, and try to find a way to move forward, in particular after some of you participate in next week’s international meeting.

Sandra Gesing and I, as the organizers of the US version of the RSE survey, offer you at least moral support.

We are also co-PIs on the just starting US Research Software Sustainability Institute Conceptualization project (http://urssi.us), and we will likely cover related topics in our project workshops and survey; we look forward to working with a US RSE community, or helping one form.

I can help make connections to the UK RSE organization as well.

Finally, I can also help make connections to public and private US funding agencies, if you can conceive of activities where funding is needed.

But at this point, I want to turn this over to you to move forward. Feel free to copy Sandra and I on any emails, and to ask for help if you think we can provide it.

Dan

The potential new organization was also mentioned at an RSE BOF at SC18, and a few more people expressed interest in participating.

The Early US-RSE

Those initial conversations led Chris Hill in early 2018 to establish a Slack space for ongoing conversation and a basic website via GitHub [ref]. From there, work continued slowly to build a space for US-RSE.

Defining the US-RSE

Inspired by conversations with others in the RSE community at SC18, Ian Cosden took charge to create a more functional website in late 2018 that is still viewable on the Wayback Machine [ref].

Part of establishing the new website was deciding what it would say. One of the first questions for the nascent US-RSE in early 2019 was what the focus of the organization should be. Many of the pre-US-RSE efforts noted above focused on the research software itself and the work to create that software. Yet there was also a recognition of the need for support for the people creating the software. Essentially, the question was what should the “E” in US-RSE stand for: engineering or engineers?

Early members agreed to focus on the engineers, which is reflected in “community” being the first part of the US-RSE mission.

As the website content developed, Charles Ferenbaugh created a handful of candidates for an official US-RSE logo [ref] and a vote within slack chose the winner. Chris Hill purchased the us-rse.org domain name, and the new site with the new logo was published [ref].

Promoting US-RSE

The horses were off to publicize US-RSE! In early 2019, a Google Group and Drive were created, and a handful of members used the drive to collaborate on a document describing the mission for the new organization. In April 2019 Sandra Gesing created a mailing list (via the Google Group) to promote the association, and Christina Maimone created the Twitter handle us_rse and membership form in June of that same year [ref]. Indeed, it cannot be overlooked that the success of an effort or organization is hugely influenced by the ability of that organization to reach others. Reaching others can be on a small scale such as giving talks, but more public and easy to see venues like social media also play a role.

Leadership

Ian Cosden continued to take the lead in pushing US-RSE forward to a more formal organization. An email from May 14, 2019 established a steering committee for US-RSE:

Dear all [Jay Jay Billings, Jeff Carver, Charles Ferenbaugh, Sandra Gesing, Chris Hill, Dan Katz, Christina Maimone, Lance Parsons, Jordan Perr-Sauer],

In the last couple months the US-RSE group has clearly seen a rise in interest, membership, and engagement. The growth is extremely encouraging, and with a small amount of effort, it’s only a matter of time before we reach a significant level of activity.

I wanted to reach out, not via slack, but in a more composed email. The nine of you (plus me) are all listed on the website as the current steering committee, for the sole reason that you either (a) wrote your name as a contributor in the document that helped define our mission or (b) expressed interest in participating around the same time. When I put the new website together, I just ran with it and appointed us the “Steering Committee.” For better or worse, that’s all it took in December.

It’s unlikely that we can continue with a completely ad-hoc organization forever. New members are (hopefully) going to be continually joining. I’d like to propose that we decide on some semi-formal organizational issues, now, rather than wait for actual problems to arise. Perhaps it’s too early. But better too soon, than too late.

At some point, there are decisions that will have to be made that will affect the direction of the community - some already have been made (I’m certainly guilty of this myself). One big question we need to answer: Who gets to make these decisions, and on what basis?

I know we are all volunteering our time for this, and everyone is busy. So whatever we decide will only work if it’s low maintenance and something that can evolve over time.

So as a starting point, let me propose the following:

1. We decide on a steering committee. At this point, I think the 10 of us should have the right of first refusal – so basically, as long as you are willing. As a reasonable expectation, perhaps “willing” would mean agreeing to commit a few hours a month to US-RSE related work.
2. We get agreement on the basic mission, set some realistic goals, then set some aspirational goals.
3. We start to develop a flexible set of “bylaws” or some other agreement of how this endeavor is going to work. This will require us to answer lots of questions that start with who and how.
4. We set up a regular (monthly?) zoom call for the steering committee members as a way to check in, discuss issues, and assign roles/tasks.

These things obviously aren’t going to happen overnight, but we can get the ball rolling. I don’t think we want to be closed off from the rest of the group, or purposely hide things, but having individuals run wild or randomly throwing things out to the group at large for input in slack isn’t going to work forever.

If I’m way off the mark here let me know.

Best regards,

Ian

The US-RSE Takes Off!

With the foundation established, everything was in place to start seriously inviting new members. The new website was announced on the UKRSE Slack and added to the international site researchsoftware.org in March 2019, bringing several new members on board and giving the organization its first growth spurt. The early members started announcing the group in presentations and generating additional interest. Likely the first public announcement of US-RSE was at an invited talk by Ian at The University of Delaware in early May 2019. Soon after, events and publicity were organized at several mainstream conferences, the first being PEARC19:

The PEARC19 BOF (Birds of a Feather), “Building a Community of Research Software Engineers” was the first large event/conference to announce the existence of US-RSE. In preparation for PEARC19 and future conferences, Ian Cosden had 2,000 stickers printed with the US-RSE logo (see picture below). Always the optimist, he thought it necessary to bring the entire box of stickers to PEARC19. It almost didn’t make it, because in the Philadelphia airport his carryon was flagged as highly suspicious. It seems that a box of stickers looks very similar to explosives. After a surprisingly thorough search the TSA agent laughed and let the stickers make the trip to Chicago. Unfortunately for Ian, he was only able to give out a handful of stickers, which meant he had to return with a nearly full box. He was stopped and searched again in the Chicago airport on the way home.

Another major venue was SC19, where a workshop was held, and a number of workshop attendees, both US and international RSEs, met socially in the evening:

and interest and growth of the organization rose quickly after these events, and has risen steadily ever since [ref].

By way of having open channels for communication, the US-RSE was set up for organic growth, or for community members to start and champion efforts that they care about. Eventual community members could see US-RSE advertised, and then easily join the Slack and say “Hello World, I’ve found my people!” It was arguably a good balance between top down and bottom up leadership. The steering committee provided an early governance body to develop structure for the organization, structure that the community members could fill with passion and ideas. These sections include efforts from community members from the founding to present day of US-RSE.

Continued Community Growth

A strong reflection of the growth of our community can be seen on the US-RSE join page [ref], which shows monthly membership growth, which is also shown in the figure below.

This automated graphic was added in May 2019 by Vanessa Sochat [ref], who also created and added the community map [ref] that is now maintained on its own site [ref]. The chart shows change over time, while the map shows a single snapshot in time of membership based on those who choose to report their city and state. In a larger sense, what these small initiatives capture is the general excitement that was held by the community - not just to find others like themselves, but to feel like they can be a part of a larger community and movement. And because many community members are software engineers, it was fairly natural to have things like automation, continuous integration, and new web interfaces added to make this possible.

In mid-2019, Ian Cosden, Jeff Carver, and Dan Katz, in an effort to seek funding to grow the organization, began conversations with The Alfred P. Sloan Foundation. The three wrote a proposal for a dedicated, in-person, US-RSE community building workshop designed to bring together community members to plan the best path forward for US-RSE. The grant was funded and a workshop was scheduled for April 2020 in Princeton, NJ. Unfortunately, it was bad timing, and COVID-19 forced the repeated postponement of the workshop; as of Feb 2022, it is planned for April 2022, COVID-permitting.

Community Voices

In July of 2019, after joining US-RSE in May, community member Vanessa Sochat was frustrated about the message being sent about what constituted an RSE. Her first effort to send a message of a diversity of roles was to make a video, “The Story of the Research Software Engineer” [ref], however any video can only have limited impact. She woke up one morning in September, and knew that she needed to start a podcast, and lift up research software engineers from all around the community to tell their stories directly. This became the RSE Stories podcast [ref] which eventually brought on a host from the UK, and inspired the creation of other RSE-run podcasts [ref].

Community Calls and Newsletters

Community calls began as an experiment in moderated online discussion with two broad goals:

• sharing ideas, knowledge and practices, and also
• giving everyone a chance to talk to each other, listen to each other and learn about and from one another.

Over time they evolved to a base one hour format with focussed discussion on some topical theme (technical or organizational) and usually mixed a short presentation with small group discussions around topical prompts. Small group discussions are kept deliberately small (3-4 people) to ensure that conversations are not overly monopolized by single voices and a broad spectrum of experience and knowledge is mutually shared. Brief plenary reviews after group discussions provide a way for all participants to compare conversations and communicate collectively. The community call notes archive and video recordings provide a growing raw resource for RSE learning and knowledge sharing [Google drive folder] from practicing RSEs across disciplines, experience levels and economic sectors. They cover all manner of topics, spanning developing professional career profiles to tools for code review.

US-RSE started creating and distributing newsletters in July 2019, led by Sandra Gesing, and has generally tried to issue them every two months, as one of our means of sharing information among members.

Working Groups

The US-RSE governance document established a need and role for working groups, but these were informal at first. Between 2020 and 2021, an early set of working groups [ref] including a website working group, diversity, equity, and inclusion (DEI), and education and training were more formally established with slack channels, mailing lists, and regular meetings. As the groups formalized, they added information on themselves to the website. The 2021 website refresh gave working groups a more prominent location so others could find them to join.

The DEI Working Group

In early 2019, Sandra Gesing mentioned diversity in a brainstorming document [ref]. Initial progress on the idea was made in a September 2019 steering committee meeting. During this committee meeting, there was discussion about creating a diversity statement on the US-RSE website. Dan Katz had suggested this topic, writing to the steering committee channel on the US-RSE slack before the meeting:

Note that we don’t say anything about diversity in https://us-rse.org or https://us-rse.org/about/ or https://us-rse.org/mission/ I think we probably should have increasing diversity of the US-RSE community (not our just members, but the larger community) as a goal.

While DEI wasn’t a new idea as US-RSE had talked about diversity from an early point, it was a gap in our community to not explicitly do anything about it. These early discussions led to diversity, equity, and inclusion being the topic of our October 2020 community call, and this in turn led to the DEI slack channel and the DEI working group led by Lance Parsons. The working group put together a DEI mission statement, with contributions from the entire group and leadership on the part of Alex Koufos, and an addition of DEI to the US-RSE Mission Statement.

In January 2021, community members Sajid Ali took the lead to establish a DEI Speaker Series [ref], which runs quarterly. In May, Nicole Brewer initiated the DEI book club club [ref] that started with a reading of Better Allies: Everyday Actions to Create Inclusive, Engaging Workplaces” by Karen Catlin. The book club was later expanded by Miranda Mundt to include all forms of media, such as podcasts. With these efforts in place, US-RSE had diversity, equity and inclusion on the radar, and a regular means to encourage discussion.

The Website Working Group

As US-RSE grew, it became apparent that a blog template would not be suitable for all the kinds of content we wanted on our site. The website working group was established to critique and work on this new template. Sandra Gesing took charge to arrange a design contest for a new logo, Ian Cosden took the lead on running website meetings, and an original draft of the new site was created by community member Vanessa Sochat. Over several months it was worked on extensively by the website working group. On October 11 2021 [ref], the group released a revamped, updated version of the website.

The Education and Training Working Group

As a result of the February 2021 Community Call, we determined there was a need for a working group to focus on Education and Training. In April 2021, a group of US-RSE members organized by Jeffrey Carver began meeting with a goal of identifying the types of education and training activities RSEs need and gathering resources to support those needs. The group began by developing an initial list of existing training resources [ref], which is still growing. Next the group enumerated a list of skills that various types of RSEs may possess [ref]. The group is currently working on establishing a webinar series to provide relevant training to members of US-RSE. Eventually, the group plans to map the RSE skills into a more structured training program that could eventually be adopted by workshops or academic institutions.

Financial Status

In November of 2020, community member Vanessa Sochat won a Science Slam, and wanted to contribute her prize to US-RSE. When it was apparent that there was no means to do this, the steering committee put their heads together to figure out how to create a financial entity as part of US-RSE to not only be able to use the contribution, but to allow regular contributions from other community members. Dan and Chris investigated a set of options, and proposed to the steering committee in March 2021 that US-RSE establish a US-RSE Open Collective [ref] and create a Treasurer role, which was proposed to the full US-RSE group in April 2021 [ref], without any dissension. After this, Chris Hill was elected to be the first treasurer, and our financial information was added to the website [ref]. Through the Open Collective umbrella US-RSE was established as a non-profit 501c3 activity with a public service focus. A first set of financial goals for the organization was discussed at the 2021 Annual General Meeting [ref]. These are aimed at helping establish the organization with funds to support basic member services and to ultimately support profession advancing activities. The 2021 Annual General Meeting also launched the first ever US-RSE fund raising event, with a t-shirt sale!

Governance

Although the first steering committee was created “just as a way to get things going” one of its early efforts was to create governance and code of conduct guidelines for US-RSE. Initial guidelines were drafted and shared with the community for input and discussion in July 2019. The Code of Conduct was established in October 2019, and the US-RSE governance document was approved by the Steering Committee in June 2020. The governance document established a process for elections for the Steering Committee, and, after Elections Chair Christina Maimone established election logistics, the first elections took place in December 2020. Each election so far has brought a new member to the Steering Committee. It has since evolved into a democratic process that we hope to further improve upon. In particular, we recognize that our election process can be improved to further give opportunity to community members that have not yet had influence.

Annual General Meetings

An annual meeting was an early goal for US-RSE. While the initial hope was for an in-person event or conference, the pandemic put planning for such gatherings on hold. Yet as the organization grew and processes for elections and finances were established, the need for some type of membership meeting became clear. The first Annual General Meeting was held in 2020, with a second in 2021. These meetings have become an opportunity to reflect on the yearly progress of the US-RSE, address the business of the organization, hear from Steering Committee candidates, and bring together the general membership for a discussion.

There’s a lot more that could be written about our US-RSE community, and we are sure other people will have other views of their own parts in leading us forward. We urge them to write additional posts. And in the meantime, a good source of information is Ian’s talks in our 2020 and 2021 annual general meetings.

US-RSE Future Timeline

As we start the new year, 2022, although our society faces unprecedented challenges with a global pandemic, as a distributed community we are accustomed to remote work, and optimistic for the new year. We hope to continue learning, growing, and writing a post like this once in a while to not lose track of our history.