[PRE REVIEW]: Pythagora: A Python Package for Modeling the Impact of Social Networks on Market Outcomes

[editorialbot]

Submitting author: @AnnieKLamar (Annie K. Lamar)
Repository: https://github.com/stwilker/pythagora
Branch with paper.md (empty if default branch):
Version: 1.1.0
Editor: @sbenthall
Reviewers: @dataspider, @pitmonticone
Managing EiC: Samuel Forbes

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/8091165ad2038a4a4cab0f1f4593ea6f"><img src="https://joss.theoj.org/papers/8091165ad2038a4a4cab0f1f4593ea6f/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/8091165ad2038a4a4cab0f1f4593ea6f/status.svg)](https://joss.theoj.org/papers/8091165ad2038a4a4cab0f1f4593ea6f)

Author instructions

Thanks for submitting your paper to JOSS @AnnieKLamar. Currently, there isn't a JOSS editor assigned to your paper.

@AnnieKLamar if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- None

🟡 SKIP DOIs

- No DOI given, and none found for title: The Social Life of Ancient Markets: Using Formal N...
- No DOI given, and none found for title: Toasting Across Empires: Drinking Vessels, Product...
- No DOI given, and none found for title: Netlogo
- No DOI given, and none found for title: Exploring network structure, dynamics, and functio...

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.08 s (686.6 files/s, 243796.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
HTML                            20           4674             75           6632
JavaScript                       6            578           1411           2488
CSS                              2            338            114           1483
Python                          10            237            728           1166
XML                              7              0              0            309
Markdown                         2             78              0            184
TeX                              1              4              0             30
TOML                             1              2              0             21
YAML                             1              1              4             19
SVG                              6              0              0             16
JSON                             2              0              0              2
-------------------------------------------------------------------------------
SUM:                            58           5912           2332          12350
-------------------------------------------------------------------------------

Commit count by author:

    14	AnnieKLamar
     5	Annie Lamar
     3	Sarah T. Wilker
     2	Annie K. Lamar

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1217

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🟡 License found: Other (Check here for OSI approval)

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[editorialbot]

Five most similar historical JOSS papers:

pylattica: a package for prototyping lattice models in chemistry and materials science
Submitting author: @mcgalcode
Handling editor: @richardjgowers (Active)
Reviewers: @riesben, @amkrajewski
Similarity score: 0.6888

epyc: Computational experiment management in Python
Submitting author: @simoninirelland
Handling editor: @ajstewartlang (Active)
Reviewers: @zbeekman, @lorenzo-rovigatti, @amritagos
Similarity score: 0.6795

EoN (Epidemics on Networks): a fast, flexible Python package for simulation, analytic approximation, and analysis of epidemics on networks
Submitting author: @joelmiller
Handling editor: @lpantano (Active)
Reviewers: @acolum, @pholme, @hagberg
Similarity score: 0.6750

PyMarket - A simple library for simulating markets in Python
Submitting author: @gus0k
Handling editor: @alexhanna (Retired)
Reviewers: @igarizio, @taqtiqa-mark
Similarity score: 0.6694

Raphtory: The temporal graph engine for Rust and Python
Submitting author: @narnolddd
Handling editor: @luizirber (Active)
Reviewers: @abhishektiwari, @arashbm
Similarity score: 0.6595

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

[samhforbes]

@AnnieKLamar it's fairly typical for us to check a submission is within scope for a package of that size, so I'll proceed with that now.

[samhforbes]

@editorialbot query scope

[editorialbot]

Submission flagged for editorial review.

[AnnieKLamar]

Hello there, thank you for the above information! Here are a few suggestions of reviewers that would be well suited to evaluate our submission:

• Corinna Coupette (Digital Humanities, Python, Networks)
• jkbren (Agent-Based Models, Network Science)
• Sam Hames (CS, DS, IS, Digital Humanities, Computational Humanities)
• Pietro Monitcone (CS, IS, network science)
• Moritz Makowski (Data pipelines, data generation, networks)

[AnnieKLamar]

@samhforbes Happy Thursday! Are you able to give us an estimated timeline for when this article will move to review or be assigned an editor?

[samhforbes]

Sorry for the delay while I was off @AnnieKLamar. Now that the checks are complete I am trying to assign you an editor.

[samhforbes]

@editorialbot invite @sbenthall as editor

@sbenthall might this be one you are able to look at?

[editorialbot]

Invitation to edit this submission sent!

[sbenthall]

Hello. Yes, I can serve as an editor on this one.

[sbenthall]

@editorialbot assign me as editor

[editorialbot]

Assigned! @sbenthall is now the editor

[sbenthall]

@jkbren @dataspider Can you review this submission to JOSS?

[dataspider]

@sbenthall Okay
@AnnieKLamar I skimmed the repo and the write-up, and it seems that you have a bunch of files in the repo that should have been gitignored, so please add a python .gitignore to your repo – GitHub has templates for that – and remove all the things that shouldn't have gotten into your repo in the first place. Also, you mention networkx in a specific version as a dependency, which probably should be networkx >= 3.2 in your pyproject.toml (see the pyproject.toml of this recent JOSS publication as inspiration on how to specify dependencies in your pyproject.toml). It also seems that you don't have tests. In brief, I'd recommend you check the JOSS Reviewer Guidelines and make sure that your submission conforms with the requirements listed therein, because reviewers will get this as a checklist and if some things are evidently missing, this will just prolong the review.

[AnnieKLamar]

@dataspider Hi there, thanks for quick response!

• I have added all the contents of the .idea folder to the .gitignore file.
• I have added NetworkX 3.2 to the .toml file requirements.
• We included "manual steps described so that the functionality of the software can be verified" in the QuickStart instructions--are the automated tests required as well?

[sbenthall]

Thanks @dataspider for agreeing to review, and for your comments here!

JOSS has a somewhat more formal review process, which involves a checklist. That process will begin once we find another reviewer.

I'll add you as a first reviewer -- you might want to review the guidelines:
https://joss.readthedocs.io/en/latest/reviewer_guidelines.html

Please let me know if you have any questions or concerns about this, especially if this comes to you as a surprise.

[sbenthall]

@editorialbot add @dataspider as reviewer

[editorialbot]

@dataspider added to the reviewers list!

[sbenthall]

@alanlujan91 Would you be interested in and able to review this submission to JOSS?

[sbenthall]

@pitmonticone would you be willing to review this submission to JOSS?

[pitmonticone]

@sbenthall Yes, but I won’t be able to start until about a week from now.

[dataspider]

@sbenthall Could you share your thoughts on this issue? I am aware that this submission was previously on scope review (just as the other submission I reference there, which I am now handling separately with the editor and AEiC of that submission), and the concerns I detail in the linked issue are even more pertinent to this present submission.

[sbenthall]

@pitmonticone That's perfectly fine. Thank you. I'll add you as a reviewer.

[sbenthall]

@editorialbot add @pitmonticone as reviewer

[editorialbot]

@pitmonticone added to the reviewers list!

[sbenthall]

Thanks for your question @dataspider

I believe @samhforbes removed the 'query-scope' label, indicating that he believed this submission is in scope as is.

But I'm happy to discuss the scope issue before beginning formal review if you have concerns.

Which criterion do you think this submission fails? Is it this:

"- Be feature-complete (no half-baked solutions) and be designed for maintainable extension (not one-off modifications)."

?

What indicators in the code are the basis of this evaluation?

[sbenthall]

@dataspider I think that maybe your question will be answered once we start the formal review process.

You will be presented with a checklist of things to look for and comment on in your review.
Many of these relate to the quality of the code.
You will also have the opportunity to raise additional issues which can support the quality of the submission, including the robustness of the code.

I will start the formal review process shortly.

Thanks for your patience, @AnnieKLamar

[sbenthall]

@editorialbot start review

[editorialbot]

OK, I've started the review over in #7549.

[dataspider]

@sbenthall My understanding is that once I get the checklist, there is no way to reject the submission as out of scope, which is precisely why I opened the issue that was now closed. Imho the submission is problematic regarding both of the criteria I quoted, and I don't want to have to sign off on something that doesn't meet these criteria. I understand that JOSS wants authors to work with reviewers to meet the checklist criteria, but when meeting the scope criteria would essentially require a complete revamp...? In any event, if you want to proceed with the review as in-scope, I would kindly ask you to assign this to someone else, and I'll make sure to do my own internal scope review before accepting any review requests in the future.

[sbenthall]

@dataspider Sorry for the confusion. I didn't get a response from you earlier, so I thought perhaps you had withdrawn your objection.

It sounds like you are asking the editors to do a scope review.

@samhforbes I see that you earlier assigned this submission for editorial review, but then later removed that label before you assigned me as editor.

Did an editorial review of this submission happen? Why did you remove that label?

I thought you had reviewed the scope and found it to be within scope. Am I mistaken?

---

My own opinion: I don't see at the moment how this submission is obviously out of scope. I'm curious what @dataspider would want to see to see the package more convincingly display substantial scholarly effort.

One thing I see is a lack of automated tests, which is on the reviewer checklist.

Another thing is that the paper has only 4 references, which perhaps does not place it well within the broader range of literature and tools. I.e, the 'State of the field: Do the authors describe how this software compares to other commonly-used packages?' requirement, which is on the reviewers checklist.

[samhforbes]

Thanks @sbenthall. You can see from the dashboard there was a full scope review including the AEiCs where it was decided it falls within our scope.

[sbenthall]

Thanks @samhforbes for that clarification.

It looks like the editors have determined this submission to be in scope!

Does that change your views on reviewing this submission @dataspider ?

As you point out, the purpose of JOSS reviews is to improve the submission, and that's even more true when there's constructive feedback to be given. Since scope was determined by the editors to not be the issue here, I wonder what the other underlying issues are. Is it possible to articulate those? You would be welcome to raise those in your review, even if they are not on the checklist.

[dataspider]

Let me start by articulating that "substantial scholarly effort" is a terrible name for a checklist criterion (who am I to judge what may have been a substantial effort for others).
I guess my underlying intuition is that this feels more like a move to get a quick additional publication with associated citations than like a software contribution. While this is subjective and might be misguided (let's assume everyone has heard best intentions), there are more objective problems that would make it highly problematic for this software to be used by others for doing simulations. For example, unless I missed something, none of the randomness is seeded, which makes the results computationally unreproducible. From the code it's pretty apparent that no attempt was made to follow elementary software-engineering principles. This is widespread (although still problematic) for the throwaway code people write for their one-off publications, but unacceptable for software "stamped" to be reused by others, especially in fields with low computational literacy. It simply creates huge research-validity problems. Again, I am not in principle opposed to helping others improve their software, but this just seems more like a start-from-scratch than an evolutionary improvement to what's there. Does that help?

[sbenthall]

@dataspider Thanks for clarifying your position.

As a purely procedural matter, it seems to me that:

• the scope check was asked for
• the editors determined the submission was in scope

There are two ways to proceed:

• (A) we continue on to the formal review, which is typically when reviewers assess the submission and recommend, or not, acceptance into the journal. It appears you are well on your way to doing that already.
• (B) you withdraw as a reviewer (because you disagree with the editors about scope), and we look for a replacement reviewer.

It's up to you. Would you prefer (A) or (B)?

[dataspider]

@sbenthall Let's do (B) then. If I simulate (A) (assuming away the scope issue), then my conclusion is "major revision", but I don't want to hold up the acceptance process.

[sbenthall]

Ok, thanks for participating @dataspider