Comments and feedback from full review of paper - August 2024

[jcohen02]

I'm opening this issue to track comments and feedback from the full review of the paper that I've just completed.

There are a few minor fixes that I'll submit in a PR, as well as some (mostly minor) more general points. I'll detail these here and they can then be linked to other issues if more discussion is required. I'll open separate PRs for suggested updates to separate them from minor fixes such as typos in order to simplify integrating, altering or rejecting proposed changes on a case-by-case basis.

[jcohen02]

A set of suggested minor edits and updates have been submitted in #347. The following are more general comments and suggestions around further edits:

• Introduction: In paragraph 5, “This community paper…” - it states that the paper defines core values and foundational competencies “…agnostic of specific technical capabilities…”. I’m not sure if this is a fair statement since many elements of the competencies are highlighting specific technical skills and requirements. I guess things are mostly programming language-agnostic but I don’t know that I’d say agnostic of technical capabilities. (Assigned to @jpthiele, Rework par. 5 of introduction #353)

• Terminology: The definition of “Researchers” here seems too simplistic - it’s effectively saying that use the term “Researchers” to refer to anyone who makes use of the services provided by RSEs. I wonder if something slightly more detailed might be preferable? e.g.

  “Researchers: Postgraduate research students and faculty members at all levels who are undertaking or leading work, within a single subject domain or across multiple domains, to develop new ideas and conclusions, verify and build upon existing findings and discover new knowledge. In the context of this paper, we consider these researchers to be consumers of the services provided by Research Software Engineers.
  Discussed in the meeting - we agreed this is broadly OK as it was before but might need brief explanation (@CaptainSifff to check)

• Start of 3.1.2: “RSEs are often experienced professionals who instruct and work closely with early career researchers.”. I’m not sure if I entirely agree that RSEs “instruct” ECRs? Maybe say: “RSEs are often highly experienced professionals who work closely with and provide technical training and guidance to early career researchers.” (Add suggestion to replace instruct #357).

• Section 4: Foundational competencies - the start of this section provides a description of what we see as the role (or the positioning of the role) of the RSE. I think that in general, the paper takes a view of RSEs that falls very much towards the research end of the “RSE continuum” that spans from a role that looks the same as that of a researcher at one end to one that looks the same as a professional software engineer working in an industry environment at the other. While I don’t disagree with this and can identify with it from a personal perspective, we’re also increasingly seeing RSEs whose roles look quite different and are not so connected to the research lifecycle, even though they are writing code to support or undertake research.

  Do we want to make some comment here, at the start of section 4 about RSE roles and the routes into the RSE profession changing as it develops? We could say that “we work with the model of the RSE as it is described here, because it fits best with the current environment in the research community in Germany, but as things develop, we recognise that roles may be more focused towards the (R) or (SE) side of RSE and training and skills development efforts will need to take this into account.”? (@jcohen02 to investigate updated text to address this)

• Section 4: Three pillars - Also in the intro to section 4, there is mention of “the three pillars”, I’m keen that these are not confused with the 4 pillars of RSE (which we haven’t referenced, but could) - I realise that the 4 pillars are taking a slightly different view of RSE looking at related activities beyond SE, and from the basis of what works from the perspective of institutional provision of RSE activities and support. Nonetheless, I think they’re still relevant in the context of the paper. (@jcohen02 - could add a reference to the four pillars of RSE paper in section 4 where I noted the changes to RSE graduates)

• Section 4.1: With the increasing interest and awareness around “green computing” and efficiency, should we also have a sustainability section in 4.1, highlighting knowledge around efficient algorithms and code optimisation as an important competency for managing efficient and effective usage of resources that often have a major carbon footprint, especially in the context of, for example, the training of LLMs? (from discussion, we could look to add a sentence to 4.4 as noted below - @jcohen02 to come up with some text).

• Section 4.4, paragraph 4: There is mention here of environment costs via associated energy consumption. If we decide to add a section about sustainability, this should be linked here. (see above)

• I think the content in 5.1 is spot on and provides a great example of how RSE roles are different at different levels of seniority/experience.

• Table 1: MOD, Principal RSE: The text in this section feels a little too general, I feel that this describes something that all RSEs should have some understanding of - I think we could improve the text in this section. (@jpthiele to look at updating text here, Rework MOD table for PI RSE #354)

• Section 6: From the first sentence, it feels like there’s a little bit of a confusion between whether an RSE needs ALL the skills listed or may only need some of them. While we class them as core competencies, I don’t think it’s realistic to say every RSE must have all these skills (or at least must be an expert in them)? In some places we highlight that, in others, like below, we seem to suggest the opposite.

• The Future Work and Conclusions sections could benefit from some editing - I’m happy to look at this next.

[CaptainSifff]

Item 2 is maybe addressed by f23dfcd

[CaptainSifff]

@jcohen02 Do you think we can close that?

[jcohen02]

We talked about the this at the meeting last week since I was wondering if we should close this. It was suggested that we should wait until all the tickbox items above are addressed before closing.

In principle I agree with this and if we're going to try and address these items before submitting an updated version of the manuscript to a journal then this should remain open. On the other hand, if we're not planning to address these items, it doesn't make sense to leave this open indefinitely.

I'd say leave it open for now if you're OK with that.

[CaptainSifff]

OK, item 5 and 6 look like sth. that could be tackled with a small MR.

[CaptainSifff]

@jcohen02 Do you want to pick up your feedback again, now we are in the phase where we are addressing issues?