🌸Joining the BigLAM hackathon🌸

[davanstrien]

We're excited you want to get involved!

To join the hackathon:

• 👋 comment below with an introduction of yourself
• 🤗 let us know your Hugging Face username (this is so we can respond to requests to join the biglam organization more quickly

[cceyda]

This is great! LAMS have so much good (curated) data with open licenses but not always simple apis or UIs, would be great to have them on the hub! I have only recently discovered the power of museum data while adding this for a hackathon https://huggingface.co/datasets/ceyda/smithsonian_butterflies will do a better job with dataset card tomorrow

[davanstrien]

Hi @cceyda 🤗 thanks for joining and the 🦋 dataset looks amazing. I'll have a think about some other datasets/image collections that could be nice for training GANs.

[shamikbose]

Hi! I'm Shamik Bose. I got my PhD in making deep learning systems more transparent and explainable. I have been involved with the BigScience workshop for data creation and also with the biomedical hackathon by contributing a number of datasets and bug fixes. I would love to contribute to this effort as well. My huggingface hub username is 'shamikbose89'

[davanstrien]

Hey @shamikbose, thanks for joining 🤗. The biomedical hackathon was a big inspiration for this — happy to have you on board!

I've added you to our Hugging Face organization. Feel free to use #self-assign to assign yourself to any datasets you think would be fun to work on (or suggest any others you know about. Feel free to ping me if you have any questions.

[marianna13]

Hi 👋 My name is Marianna. I am a Machine Learning Engineer and I would love to help you with this!

[davanstrien]

@marianna13 thanks for joining, glad to have you on board. Feel free to assign yourself to any datasets that look of interest on the current project board. Feel free to ping me if you have any questions 🙂

[marianna13]

Thanks!

[albertvillanova]

Hi @marianna13, please do not forget to request joining our BigLAM org, so that you can contribute to the hackathon: https://huggingface.co/biglam

[marianna13]

Done ✅

[thebooort]

Hi everyone!
My name is Bart Ortiz, I am a PhD Candidate in Machine Learning and Recommendation systems. This is going to be my first time collaborating in a Hugging Face hackaton but I have been following some BigScience's projects closely these days. My nickname is thebooort and I am planning to start by adding some datasets from Cervantes Institute to have more spanish datasets! :) Probably will start with the Sonnets Dataset. However I am going to take easy and read through all the docs first!

[davanstrien]

@thebooort nice to meet you 🤗 It would be great to have some Spanish datasets included! Feel free to give me a shout if you need help with anything.

[albertvillanova]

Great to hear Spanish datasets will be added @thebooort 😉

[Skorkmaz88]

Hi everyone, I am Semih, several years of ML experience , but this field moves so fast I keep staying as a beginner, and always have a LAM nerd :D Would love to join and contribute ! HF username is : skorkmaz88

[albertvillanova]

Hi @Skorkmaz88, welcome to this hackathon!

[davanstrien]

Great to have you here @Skorkmaz88!

[apjanco]

Hello, my name is Andy Janco and I am a research software engineer at the University of Pennsylvania. I'm a big fan of 🤗 and look forward to the hackathon!
My username on Huggingace is ajanco
Here's a collection of illustrated art periodicals I published with friends: https://huggingface.co/datasets/ajanco/pesp

[albertvillanova]

Welcome @apjanco : just added you to out Hub org. You can start contributing: https://github.com/orgs/bigscience-workshop/projects/8

[davanstrien]

Hi @apjanco great to have you here! The dataset you shared looks great 🙂

[aih]

Hi All,
I'm Ari Hershowitz (https://aih.github.io/), a lawyer, software developer and general troublemaker working to modernize document processing at the House of Representatives, Library of Congress, and other public organizations. I'm particularly interested in helping government organizations to make their datasets available in machine-readable and easily consumable formats. Our projects are featured priorities of the House Committee on Modernization.

I'm recruiting ML researchers for related projects so if this sounds interesting, get in touch! My Hugging Face profile is https://huggingface.co/arihers

[albertvillanova]

Thanks for your interest in our initiative, @aih.

[marianna13]

Hi! Sounds interesting to me. I would love to help you with your projects!

[aih]

@marianna13 Sorry, I did not see this earlier. What is the best way to get in touch with you? I can be reached at arihershowitz at gmail

[zaidalyafeai]

Hey, I am Zaid . A a PhD student at kfupm in Saudi Arabia. Co-founded arbml an effort to demcratize arabic nlp. Co-authored masader for Arabic NLP datasets cataloging at bigscience, we can use the metadata to extract datasets related to lam. I thought this effort might be relevant to us at arbml. We have a community on discord so I will share this there. My hf name is Zaid.

[zaidalyafeai]

Not sure if this is about English data only.

[albertvillanova]

Thanks @zaidalyafeai, great initiatives so far. And really interesting to sync efforts. Please, feel free to join our org if willing to contribute any dataset: https://huggingface.co/biglam

[davanstrien]

@zaidalyafeai that sounds great. We're definitely interested in non-English materials too! I'm a big fan of masader, I know of a few LAM-related Arabic datasets but it would be really nice to identify more.

[kallewesterling]

I would love to join as an opportunity to get more involved into machine learning + LAM.

Currently I am a Digital Humanities Research Software Engineer at the British Library and The Alan Turing Institute on the Living with Machines research project, which combines large-scale digitised historical collections with data science and computational methods.

Not very active on HuggingFace but here I am :) https://huggingface.co/kallewesterling

[davanstrien]

Hey @kallewesterling! Feel free to request to join the Hugging Face org here: https://huggingface.co/biglam

[albertvillanova]

Great @kallewesterling to have you interested in collaborating. For sure your contributions will be very appreciated.

[mialondon]

Hello!

I'm currently the Digital Curator for Western Heritage Collections at the British Library, and a Co-Investigator on the Living with Machines project. I tend to work on access to digitised collections, including UX and audience research, with a focus on participatory digital access (basically, crowdsourcing for cultural heritage).

According to the edit dates on http://museum-api.pbworks.com/w/page/21933420/Museum%C2%A0APIs I maintained a list of sources of 'museum, gallery, library, archive, archaeology and assorted sources for machine-readable data' for 12 or so years, and I'd love to know how many of those resources might be suitable, and how best to contact the institution to let them know about HuggingFace and what it means for them if their data is available on it.

This seems like a great way for institutions to have some involvement with machine learning without the overhead of figuring it out for themselves.

[mialondon]

And I have some suggestions for the FAQ, starting with 'What is HuggingFace?'. Where should I add them?

[davanstrien]

Hello!

I'm currently the Digital Curator for Western Heritage Collections at the British Library, and a Co-Investigator on the Living with Machines project. I tend to work on access to digitised collections, including UX and audience research, with a focus on participatory digital access (basically, crowdsourcing for cultural heritage).

According to the edit dates on http://museum-api.pbworks.com/w/page/21933420/Museum%C2%A0APIs I maintained a list of sources of 'museum, gallery, library, archive, archaeology and assorted sources for machine-readable data' for 12 or so years, and I'd love to know how many of those resources might be suitable, and how best to contact the institution to let them know about HuggingFace and what it means for them if their data is available on it.

This seems like a great way for institutions to have some involvement with machine learning without the overhead of figuring it out for themselves.

Thanks for sharing those. I think there are a bunch of datasets from that list that could be interesting. I'll add it to the discussion thread on 'dataset' inspiration (#31). I'm planning to add some more possible datasets to the backlog this week so I'll definitely take a look in there for inspiration!

[davanstrien]

And I have some suggestions for the FAQ, starting with 'What is HuggingFace?'. Where should I add them?

For the FAQs, I've opened a new thread for suggested questions (and answers!) here: #51

[mialondon]

And I've just joined HuggingFace as miah

[albertvillanova]

Hi @mialondon, welcome and feel free to request to join our dedicated organization on the Hugging Face Hub: https://huggingface.co/biglam

[librarianrafia]

Hi,
My name is Rafia, I am librarian leaning about ML to help in offering workshops & instruction.

[davanstrien]

Great to have you here @librarianrafia! I think some of the datasets we might add as part of this sprint are a great resource for workshops/teaching. I think it's nice to try and use "real" data when learning ml so hopefully this effort helps with this!

[AyeshaShafique]

Hi everyone,

This is Ayesha, Data Science Manger @CancerClarity USA. Working in AI field for than 5 years with several international publications. Would love to contribute.

Thanks

[davanstrien]

Hey @AyeshaShafique! Thanks for joining in this effort. I'm working on adding some more datasets to the queue today so feel free to assign yourself to a dataset from the project board https://github.com/orgs/bigscience-workshop/projects/8?query=is%3Aopen+sort%3Aupdated-desc. Also, feel free to request to join Hugging Face org we're using for the hack here: https://huggingface.co/biglam

[albertvillanova]

Hi @AyeshaShafique, welcome and feel free to request to join our dedicated organization on the Hugging Face Hub: https://huggingface.co/biglam

[epoz]

Hello!

I am @epoz T - the creator and maintainer of https://iconclass.org/
Have been working on GLAM and (meta)data for many years, and recently quite a bit to do with the semantic web, linked data and the intersection with AI and cultural heritage.
Was delighted to see the post by @davanstrien adding the Iconclass testset

The current Iconclass search interface also support visual similarity searches using CLIP. Still working on adding more multi-modal searches, but very excited with the results already.

Looking forward to hear more about others interested in AI, Art & Iconography.

[davanstrien]

Hey @epoz great to have you here! I'm a big fan of your work.

Would love to hear more about the CLIP work. Is that using the existing CLIP model? One plan I had (but didn't find time for) was to fine tune a CLIP style model on iconclass labels. It's probably too small as a training set but potentially the more systematic nature of the label structure make it easier for the text encoder part the learn the relevant patterns (this is pure speculation!)

[epoz]

Hi @davanstrien , thx! So nice to hear.
This is with "plain" CLIP, no further training. The results are already very good, I would be really curious to see what can be achieved with more focussed training.

[davanstrien]

I'm doing a few experiments with fine-tuning CLIP at the moment - if those aren't a complete failure (quite likely!), I'd be happy to experiment a bit with trying to fine-tune CLIP with Iconclass.

[bmschmidt]

Hi all,

I'm Ben Schmidt, I work at NYU doing history and digital humanities. (At least for the time being! Let me know if anyone's looking for.a hired gun...) I've worked a lot with large textual corpora in the US, especially at the Hathi Trust and Internet Archive. I spent some time last year sketching out a format for sharing of texts and metadata especially for nonconsumptive research that I'm looking to develop. From what I see there are a lot of homologies between that and the format used on HF, so there are a number of text collections that I might able to ingest quickly.

I'm especially interested in sharing feature representations (token counts, learned embeddings, and random projections) on data that can't be shared publicly for copyright or other reasons.

I'm also FWIW, obsessed with the possibilities for visualization on historical texts with metadata.

Just set up a shiny new HF username at https://huggingface.co/benmschmidt.

[aih]

@bmschmidt I'll send you a message! I think I have a project you'll be interested in

[davanstrien]

Hey @bmschmidt! Thanks again for joining. I'll ping you soon about the Europeana data 🙂

[peregilk]

Hi All,
I am Per Kummervold. I am a researcher at the AiLab at the National Library of Norway. Our lab has previously contributed datasets lik the Norwegian Colossal Corpus that should be of interest to BigLAM. We hope to contribute with more datasets in the future.

[davanstrien]

Hey @peregilk, great to have you here! Big fan of all the work you've been doing at the National Library of Norway 🤗

[nabsiddiqui]

Hello, my name is Nabeel Siddiqui, and I am an Assistant Professor of Digital Media at Susquehanna University. I work on a variety of topics around the history of computing and cultural analytics. This summer is fairly busy for me, but I would love to be involved and help where I can. I have already sent a request to the BigLAM hugging face group with the same username I use on GitHub. I am fairly good at cleaning up numerical datasets, joining datasets, together, etc. I usually work in R. I would also be happy to run a 4cat instance if someone has idea for social media data that might be useful or perhaps image datasets online. In short, I am happy to do the work on any dataset if someone comes up with the idea lol.

Finally, if anyone is interested, I would love to collaborate to publish a dataset for the Journal of Open Humanities Data. I really appreciate their initiative and am using some of their datasets for a book I am working on. This seems like a good opportunity to contribute to that project since we are already creating datasets. Look forward to working with everyone.

[shamikbose]

Hi Nabeel! The JOHD looks really interesting. I would love to collaborate on that effort!

[davanstrien]

@nabsiddiqui hi Nabeel, great to have you here! I'm planning to add a bunch more datasets to the backlog this week but feel to prose others you think may be of interest :)

[giganttheo]

Hi all! My name is Théo Gigant (@gigant on HuggingFace), I just started my PhD in multimodal deep learning. I previously contributed the Wikiart dataset on the hub as part of the HugGan sprint. I also contributed to BigScience via the BigBio hackathon, and I look forward to work for open source projects again!

I didn't really check the datasets for now, but if that's relevant for the scope, I might like to add the illustrations from https://www.oldbookillustrations.com/ to the hub. The task is already halfway done since I already wrote a scraping script to get all the infos + images for the illustrations.

[davanstrien]

Hey @giganttheo, thanks for joining the hackathon! I've just approved you on the Hugging Face org.

I didn't really check the datasets for now, but if that's relevant for the scope, I might like to add the illustrations from oldbookillustrations.com to the hub. The task is already halfway done since I already wrote a scraping script to get all the infos + images for the illustrations.

Feel free to open an issue suggesting this dataset.

[cneud]

Hi everyone, my name is Clemens Neudecker and I lead a small research team working on AI/ML for digital cultural heritage at the Berlin State Library.

We have created a number of AI/ML-based tools and corrsponding datasets/models in our Qurator project, see an overview here: https://ravius.sbb.berlin. I am already on Hugginface as https://huggingface.co/cneud and have experimented with (but not completed) distributing some of our project outputs via the Huggingface hub (see https://huggingface.co/SBB). Additonally I would like to look into the possibility of sharing some datasets from our SBB-LAB or the EuropeanaNewspapers project.

[davanstrien]

Hey @cneud! Great to see SBB on the hub. It would be great to see some of those datasets on the hub. I have various lists of NER datasets for historical material (including IIRC one from a paper you wrote). I'll try and get those turned into dataset issues this week.

[bmschmidt]

Hi @cneud ! I had a discussion open about some Europena newspapers that hasn't yet turned into a ticket--would love any thoughts on the metadata ontology and whether my uploading this stuff would be redundant: https://github.com/bigscienc. Discussion here: workshop/lam/discussions/50

[cneud]

@bmschmidt Nice, I'll look into it and will comment there asap!

[MikeTrizna]

Hi All,
My name is Mike Trizna and I am a Data Scientist at the Smithsonian Institution, within the Office of the Chief Information Officer Data Science Lab (yep, that's a mouthful!). Our group at the Smithsonian partners with subject matter experts at the various units of the institution (21 museums, 21 libraries, 9 research centers, and 1 zoo) to apply machine learning and other data science techniques to research and workflows. The Smithsonian makes over 4 million collections items available through its Open Access program both through an API and for bulk download via AWS Open Datasets. The variety of data types, as well as the deeply-nested JSON format make this data source kind of hard to work with, but I would be happy to help anyone extract and process interesting subsets for this project.

I know @davanstrien from the AI4LAM organization, which I encourage everyone here to check out. Let me know if you have any questions about that!

[nabsiddiqui]

@MikeTrizna Do you have the metadata and tags with them? Can you upload them to Hugging Face with the associated tags, metadata?

@shamikbose I was hoping to just run them through VGGNet16 and VGGNet19. If @MikeTrizna has the metadata, we can use that as one column and then the tags that VGGNet16 and VGGNet19 give on the ImageNet data. I would also be willing to use R to just do some basic analysis to add to the metadata, such as color, etc.

[MikeTrizna]

Yep, @nabsiddiqui, I have the metadata and tags as well. I will try to get the full dataset filtered to the butterflies and get them uploaded tomorrow, but I will be out of the office on Friday.

As far as the VGG ImageNet inferences, I would recommend against going that route. We have found that ImageNet classifier models perform quite poorly on art and illustration images, and the ImageNet1K classes are not very comprehensive either. For that sideproject I mentioned, I generated CLIP embeddings which seemed to perform well with text queries following the pattern of "illustration of X". Would those embeddings be useful to include as well?

[nabsiddiqui]

Let me start a new issue and we can discuss it there.

[cceyda]

@davanstrien another example GAN project making beetles using Biodiversity Heritage Library images on flickr images https://www.cunicode.com/works/confusing-coleopterists

[davanstrien]

@davanstrien another example GAN project making beetles using Biodiversity Heritage Library images on flickr images cunicode.com/works/confusing-coleopterists

This is super cool!

[ericleasemorgan]

My name is Eric Morgan (@ericleasemorgan). I work at the University of Notre Dame in a digital scholarship center.

My most recent research interest surrounds distant reading, and thus I support a Python-based library used to create data sets from (almost) arbitrarily large sets of textual corpora. The library is called the Distant Reader Toolbox. Put almost any number of files of any type into a directory. Point the Toolbox to the directory. Allow the Toolbox to do its work. And the result is a platform- and network-independent file system filled with: 1) the original data, 2) plain text versions of the original data, 3) tabular representations of extracted features, and 4) a relational database file. I call the resulting data sets "study carrels". You know, those little spaces in libraries where one can bring their items of interest, arrange them on shelves, and use them for study.

Given a study carrel -- a data set which is easy to compute against -- the command-line interface to the Toolbox supports all sorts of different modeling processes: ngram analysis, concordancing, collocations, a few different clustering techniques, full text indexing, semantic indexing, part-of-speech analysis, named-entity analysis, and information extraction through the use of grammars. In the end, the Toolbox is intended to allow students, researchers, or scholars the ability to articulate a research question, assemble a set of texts which might answer the question, and then compute against the texts to address the question. For example, a research question might be "How did Ralph Waldo Emerson define what it means to be a man?"

Finally, I have created a set of more than 3,000 study carrels, and they are available as a sort of collection at http://library.distantreader.org. Many of the study carrels surround topics from the humanities: love, honor, justice, etc. And these same carrels' content comes a lot from Project Gutenberg. Others from the HathiTrust, and so on. Another large portion of the study carrels surround the topic of COVID-19 and were originally disseminated through a project called CORD-19.

In short, I have a whole lot of pre-computed datasets, and they are amenable to additional computation by just about any other platform. To contribute, what should I do next?

[davanstrien]

Hey, great to have you here @ericleasemorgan!

Distant Reader sounds like a super exciting project! I will look closely at the links you sent, but I think there are a few potential ways you could share this material. Since you already have a library of material you have prepared, it might make the most sense to consider creating a dataset loading script that can load any of the study carrels in the library. This would then allow someone to load a dataset based on one of these libraries:

ds = load_dataset('load_distant_reader', name='homer')

I think the main question to consider is what overlap (if any) you want with the existing behaviour of the Python library. The datasets library https://huggingface.co/docs/datasets/ is primarily focused on supporting machine learning workflows, so there are likely to be some things that your library does that datasets won't do well. Essentially when you write a loading script, you get the data to fit some set of features https://huggingface.co/docs/datasets/about_dataset_features you specify. 'under the hood' these map to Apache Arrow datatypes.

It might already be useful to write a loading script that can load any library items and load the previously cleaned texts with metadata about that text?

[sarahciston]

Hi! I'm Sarah Ciston (@sarahciston on huggingface), a PhD Candidate at USC Media Arts + Practice researching how to bring intersectional feminist, anti-racist, queer approaches to AI/ML and how to work critically with datasets. I've been collaborating with USC Libraries and researchers here on more inclusive collections-as-datasets. I'm excited to see the BigLAM initiative, and I'd love to be involved!

[davanstrien]

Hi! I'm Sarah Ciston (@sarahciston on huggingface), a PhD Candidate at USC Media Arts + Practice researching how to bring intersectional feminist, anti-racist, queer approaches to AI/ML and how to work critically with datasets. I've been collaborating with USC Libraries and researchers here on more inclusive collections-as-datasets. I'm excited to see the BigLAM initiative, and I'd love to be involved!

Hi @sarahciston your work sounds super interesting! If there are any datasets you think we should be included, let us know!

One of the challenges we'll face with a lot (most?) of the datasets we add is their historical biases. Both with the content itself and the baggage of LAM collecting and metadata practices. If you have learned any useful ways of trying to describe or mitigate this bias would be really useful to hear about those.

[ericleasemorgan]

On Jul 29, 2022, at 12:20 PM, Daniel van Strien ***@***.***> wrote: Distant Reader sounds like a super exciting project! I will look closely at the links you sent, but I think there are a few potential ways you could share this material. Since you already have a library of material you have prepared, it might make the most sense to consider creating a dataset loading script [1] that can load any of the study carrels in the library. This would then allow someone to load a dataset based on one of these libraries: ds = load_dataset('load_distant_reader', name='homer') I think the main question to consider is what overlap (if any) you want with the existing behaviour of the Python library. The datasets library https://huggingface.co/docs/datasets/ is primarily focused on supporting machine learning workflows, so there are likely to be some things that your library does that datasets won't do well. Essentially when you write a loading script, you get the data to fit some set of features https://huggingface.co/docs/datasets/about_dataset_features you specify. 'under the hood' these map to Apache Arrow datatypes. It might already be useful to write a loading script that can load any library items and load the previously cleaned texts with metadata about that text? [1] https://huggingface.co/docs/datasets/dataset_script

Daniel, thank you for the prompt reply, and likewise, very interesting. Just to re-iterate, the next step is for me to write a little Python script with a specific name and in a specific location. The script is really a set of classes and objects denoting features of my data sets. I then run another script that looks for the script I just wrote, and it will output data files of some sort, and those data files will go to 🌸? Finally, somebody will then be able to run something like above (ds=load_dataset('load_distant_reader', name='homer') to actually load my data sets. Correct? If so, then many of the examples in [1] allude to training sets, splits, and testing. To what degree is this required for me to denote? All of my data (plain text) file are saved in a single directory, and it is not divided into training and testing sets.

…

-- Eric Morgan

[davanstrien]

I've opened an issue to discuss possible ways of including this dataset here: #80.

[ymaurer]

Hi my name is Yves Maurer and I'm deputy head of IT at the national library of Luxembourg.

At our library we have released a few open data sets based off historical newspapers at data.bnl.lu and we'd like to make more available. I've seen that Daniel van Strien has already added one of our datasets for text processing to the BigLAM project, but I think that the ground truth packs of manually hand-corrected OCR should maybe also be added. They are smaller than the article fulltexts, but since they are hand-corrected (double-keyed), they definitely have a higher quality.

The datasets in question are:
(https://data.bnl.lu/wp-content/uploads/2021/07/bnl-ground-truth-newspapers-before-1878.zip)
and:
(https://data.bnl.lu/wp-content/uploads/2021/09/bnl-ground-truth-newspapers-before-1878-raw.zip)

And my huggingface username is ymaurer

[davanstrien]

Hi @ymaurer, great to have you here! I'm a massive fan of how the data is presented on data.bnl.lu.

It would be great to include those other datasets! Also if you prefer to have your own namespace/organization for these datasets let me know, and we can always transfer things over to that namespace once they are ready.

[ymaurer]

Thanks @davanstrien, that is nice to hear.

That's a good suggestion for the namespace, I'll set something up for the bnl and let you know.

[hawc2]

Hi my name is Alex Wermer-Colan and I'm the Academic Director at Temple University's Scholars Studio. I support a lot of faculty and student projects that may be relevant to this project, such as researchers who are building large Reddit datasets on specific subjects.

Our department's focus is on digitizing and developing copyrighted and open-access datasets with a focus on literary texts, especially science fiction and underepresented writers and genres. We've ingested materials into HathiTrust Digital Library, and I'm looking for other ways to make restricted data useful to machine learning researchers outside our institution, for example through extracted features by pre-generating models without sharing the original datasets.

We also have scraped lesser known public domain works from relatively unknown digital archives could be incorporated into existing literary datasets such as those available on Project Gutenberg.

[davanstrien]

Hi @hawc2 great to have you join!

Our department's focus is on digitizing and developing copyrighted and open-access datasets with a focus on literary texts, especially science fiction and underepresented writers and genres. We've ingested materials into HathiTrust Digital Library, and I'm looking for other ways to make restricted data useful to machine learning researchers outside our institution, for example through extracted features by pre-generating models without sharing the original datasets.

This all sounds exciting. For this hackathon, we're focused on datasets that don't have copyright/have an open licence. It would be great to explore possible datasets with extracted features. I'm hoping to do more work on using tools to visualize/describe the datasets being included in this sprint later in the hackathon. These datasets could fit nicely into this. It would also be great to train models on some of this data. A GPT-esque model trained on science fiction sounds a lot of fun!

I'm also pinging @meg-huggingface and @yjernite. They are co-chairing the BigScience Data Governance working group. Part of the focus of this group is to explore models for data governance for data that can't be shared freely but might be possible to share in particular situations. In particular, the goal is to establish models for 'data providers' and 'data hosts' to responsibly host and share (under particular access controls) datasets that can't be freely shared. If this sounds of interest, this might be something to discuss a bit further offline.

[hawc2]

That sounds great - I'd love to be part of those broader conversations around Data Governance. I'm doing a lot of work in that area within the context of libraries and research centers, and would like to see better and more widely available technical infrastructure for controlled access and curation of restricted data

[davanstrien]

Awesome, the data governance group is taking a break during the summer but I'll keep you in the loop about this. @meg-huggingface and @yjernite would be good to pick this up in September.

[kiymetakdemir]

Hi, I am Kiymet. I am a computer engineering and mathematics student at Bogazici University. Currently, I am a trainee in KNAW Humanities Cluster, Digital Humanities Lab. I would like to contribute by adding Odeuropa benchmark which is based on historical texts in 7 languages.
My HuggingFace username is kiymetakd :)

[davanstrien]

Hey @kiymetakdemir, great to have you join. It would be amazing to have the Odeuropa benchmark dataset added! Let me know if you need any support with adding it!

[Jim-Salmons]

Hi all, in the event I am not too late--as it is now Nov 1st--I'd like to throw my proverbial hat into the ring with regard to making an ML-appropriate #DigitalHumanities dataset available to the #GLAM research community. I am a 71-yo cancer-surviving indie #CitizenScientist and unfortunately #DisabledDeveloper (due to a 2020 spinal cord injury) with a dataset I would like to make more widely available to the digitization research community.

My dataset consists of over 7,000 bounding-box dimensions for all the advertisements in the 48-issues of in the digital collection of Softalk magazine at the Internet Archive. Beyond the raw dimensions of these ads which document the size, shape, and position within the 2- and 3-column page grids of these ads' pages, the ad specifications are contextualized by my development of the #MAGAZINEgts ground-truth storage format based on an ontological stack of CIDOC-CRM, FRBRoo, PRESSoo, and PAGExml. This format provides an integrated complex document structure and content depiction model using a metamodel subgraph design pattern.

My dataset is currently found here on GitHub: https://github.com/SoftalkAppleProject/datasets_ml_all_ads_1M. The READme/home page of this repository has addition information about the dataset, the MAGAZINEgts format, associated DATeCH posters, and relevant links to articles, etc. The project description is a bit dated and has been affected by my recent disabling injury. The READme page does not currently include a link to this relevant article: https://bit.ly/pressoo-magazines.

I could use a helping hand to determine the best way to format, document, and contribute this dataset for inclusion in the #BigLAM Hackathon project/community. I can most easily be reached here in reply or via Twitter at: https://twitter.com/Jim_Salmons

[shamikbose]

Hi Jim, it’s great to have you here. I’m not sure if the hackathon is still on, but I’d be happy to help you out with putting the dataset on the organization page and building a dataloader for it

[Jim-Salmons]

Hi Samik! 👋🤓👍 That assistance and advice will be so helpful. As I am sure you are otherwise busy, don't worry about this needing to be a quick focused response. I have a very low-key and flexible schedule. In addition, I am somewhat focused on trying to get critical mass pulled together of a "#GitHub #Copilot for #DisabledDevelopers" research study and support program (https://bit.ly/Copilot4DisDev).

To start if you can direct me to any instructive content to better understand the requirements for listing on the organization page and creating a dataloader. I'll do my best to minimize need for your help.

Thank you.

[davanstrien]

Hi @Jim-Salmons, really nice to have you here! This dataset sounds super relevant, and I can see a lot of work went into it)
We'll slowly keep working on adding datasets to the BigLAM, so also happy to offer you help to get this into the datasets hub.

If you want to include the dataset under the BigLAM hugging face org (https://huggingface.co/biglam), you will need to create an account on Hugging Face: https://huggingface.co/join. You can then request to join the BigLAM org.

What format are the annotations for the data in? We have a few object detection datasets (https://huggingface.co/datasets?task_categories=task_categories:object-detection&sort=downloads&search=biglam) that might be a useful starting point for loading scripts.

[Jim-Salmons]

Thanks @shamikbose & @davanstrien - I have an as-yet unused-as-contributor account at HuggungFace and have submitted my request to join the BigLAM organization. I will begin exploring the datasets there and get a better idea of what is required for submission.

I'll shortly follow up here with a short explanation of what I have and its formats etc. I will be able to more cogently explain this once I look at some of what is already in the BigLAM datasets collection.