library(readr)
library(dplyr)
library(tibble)
library(purrr)
library(stringr)
library(jsonlite)
package_names <- c(
"onet2r",
"cmapr",
"huggingfaceR",
"foundryR"
)
package_versions <- package_names |>
set_names(package_names) |>
map_chr(\(package) {
as.character(
utils::packageVersion(package)
)
})
locked_packages <- fromJSON(
"renv.lock",
simplifyVector = FALSE
)$Packages
package_commits <- package_names |>
set_names(package_names) |>
map_chr(\(package) {
locked_packages[[package]]$RemoteSha
})
version_table <- tibble(
package = names(package_versions),
version = unname(package_versions),
commit = unname(package_commits)
)Inspecting provider schemas and provenance
Check schema, units, provenance, and rights before combining records
training data
data providers
workforce research
Use onet2r, cmapr, and huggingfaceR fixtures to inspect what an outside source records and what it cannot establish.
Choosing two uncertain sentences for review did not broaden the Riverton Workforce Lab’s source material. It still has 28 fictional local text units. That is enough to practice, not enough to describe a labor market. The sentences cannot tell workers what options exist beyond those notices. The team therefore inspects three outside resources: occupation tasks from the O*NET database, career transitions from CMap, and a catalog of machine-learning dataset tasks from the Hugging Face Hub.
A training data provider supplies data or access to data. Provider does not mean neutral authority. Before combining records, the team must understand who created them, why, from whom, under which license, and what one row represents.
This lesson is an inspection, and the distinction matters more than it might sound. Everything examined below is a small example file shipped inside an R package for demonstration, written by the author of these lessons. Reading such a file can show what fields a source records, what one row means, and which rights questions it raises. It cannot support any statement about how good, complete, or accurate O*NET, CMap, or Hugging Face datasets are, and no such statement is made or implied here.
Note
Riverton Workforce Lab is fictional. The package fixtures are executable examples, but this lesson does not assume that every row is an authenticated extract from the provider named by its schema. Each fixture’s status is stated below.
TipWhat you will learn
By the end of this lesson, you will be able to:
- distinguish a package from the data source it helps access;
- inspect versioned O*NET task records with onet2r;
- load CMap example transitions with cmapr;
- inspect Hugging Face task names with huggingfaceR;
- test whether two sources share a unit and a join key; and
- say what an unauthenticated example file cannot establish.
Record the software versions
The R packages are tools written by the same author as this teaching site. The underlying data comes from separate providers with separate documentation and licenses. The next chunk loads table, file, JSON, repetition, and text helpers, then records package versions; provider calls stay qualified by package name.
| Package | Version | Git commit |
|---|---|---|
| onet2r | 0.5.0.9000 | b21c1d7430e6 |
| cmapr | 0.0.0.9000 | eb2d0b3810e0 |
| huggingfaceR | 2.1.0 | 56b2c416ff81 |
| foundryR | 0.1.0 | bdeccd40b211 |
The project lockfile also records the Git commit for packages installed from GitHub. A version number alone is not enough when development code can change.
Inspect O*NET task statements
The onet2r package includes a miniature archive for examples. We ask the parser to treat it as release 30.3 and supply a release date. Those arguments describe the fixture to the function; they do not independently authenticate its bytes as an official O*NET database extract.
onet_archive <- system.file(
"extdata",
"onet-mini",
"db_30_3_text",
package = "onet2r"
)
onet_tasks <- onet2r::onet_archive_read(
version = "30.3",
table = "Task Statements",
path = onet_archive,
release_date = "2026-05-01"
)
onet_display <- onet_tasks |>
select(
task_id,
onet_soc_code,
title,
task,
release_version,
source_date
)| Task ID | O*NET-SOC code | Occupation | Task statement | Release | Source date |
|---|---|---|---|---|---|
| 1001 | 15-1252.00 | Software Developers | Analyze user needs and software requirements. | 30.3 | 2025-07-01 |
| 1002 | 15-1252.00 | Software Developers | Prepare reports on software testing status. | 30.3 | 2025-07-01 |
| 2001 | 29-1141.00 | Registered Nurses | Monitor patient health and record signs. | 30.3 | 2025-08-01 |
One row describes one occupation task. The occupation code, supplied release label, and source date travel with the text. The release date supplied to the function is not the same as the source dates displayed in individual rows. The three-row fixture demonstrates the onet2r schema; this lesson does not use it for claims about occupations or the full O*NET database. Task statements could inform a study of work or skills. They do not establish whether a particular Riverton notice offers training or evening classes.
Inspect CMap career transitions
The cmapr package includes ten example transitions. Each row links one job title to a later title within a sector and region. The file is a package teaching fixture; it does not contain upstream record identifiers that would allow this lesson to authenticate it as an exact CMap subset.
cmap_path <- cmapr::cmap_example_data()
cmap_transitions <- read_csv(
cmap_path,
col_types = cols(
sector = col_character(),
region = col_character(),
job_title_from = col_character(),
job_title_to = col_character(),
transition_weighted_count = col_double(),
si_from = col_double(),
si_to = col_double()
)
)| Sector | Region | Earlier title | Later title |
|---|---|---|---|
| Technology | North America | Software Engineer | Senior Software Engineer |
| Technology | North America | Senior Software Engineer | Engineering Manager |
| Technology | Europe | Data Analyst | Data Scientist |
| Healthcare | North America | Registered Nurse | Nurse Manager |
| Healthcare | Europe | Medical Assistant | Registered Nurse |
| Finance | North America | Financial Analyst | Senior Financial Analyst |
| Finance | Asia | Accountant | Finance Manager |
| Technology | Asia | Junior Developer | Software Engineer |
| Healthcare | North America | Physician Assistant | Medical Director |
| Finance | Europe | Investment Analyst | Portfolio Manager |
These rows demonstrate title-transition columns, not occupation task statements. A title change does not prove a promotion, wage gain, or improvement in job quality. The full CMap documentation distinguishes validated and model-inferred transitions; the fixture lacks that status and does not support population estimates.
Inspect the Hugging Face task vocabulary
Hugging Face hosts many independently contributed datasets. huggingfaceR contains a local list of task names for Hub searches. Filtering that list does not download a dataset or verify any dataset card.
classification_tasks <- huggingfaceR::hf_list_tasks(
pattern = "classification"
)
classification_tasks[1] "text-classification" "token-classification"
[3] "zero-shot-classification" "image-classification"
[5] "audio-classification" The list shows the vocabulary bundled with this version of huggingfaceR, not a live statement about the Hub. A researcher still needs to inspect each candidate dataset’s card, license, languages, collection method, labels, and known limits. Being searchable on a platform is not evidence that a dataset is suitable.
huggingfaceR can also build a zero-shot classification payload. This prepares a request body for inspection only. It does not choose a model or contact a provider.
zero_shot_payload <- huggingfaceR::hf_zero_shot_classification_payload(
"Paid training is provided.",
candidate_labels = c(
"training",
"requirement",
"schedule",
"skill",
"other"
),
multi_label = FALSE
)
str(zero_shot_payload)List of 2
$ inputs : chr "Paid training is provided."
$ parameters:List of 2
..$ candidate_labels: chr [1:5] "training" "requirement" "schedule" "skill" ...
..$ multi_label : logi FALSEDo not join unlike rows
The three resources use different units:
provider_units <- tibble(
resource = c(
"O*NET teaching archive",
"CMap teaching fixture",
"Hugging Face task vocabulary"
),
row_or_item = c(
"occupation task statement",
"job-title transition",
"platform task name"
),
records_seen = c(
nrow(onet_display),
nrow(cmap_transitions),
length(classification_tasks)
)
)| Resource | What one item means | Items inspected |
|---|---|---|
| O*NET teaching archive | occupation task statement | 3 |
| CMap teaching fixture | job-title transition | 10 |
| Hugging Face task vocabulary | platform task name | 5 |
A shared topic is not a shared unit of analysis. A title-transition row cannot answer whether a posting offers evening classes, and a task statement cannot be merged with a local sentence without a documented relationship. The team first needs a research question and a documented crosswalk between occupation codes, titles, or tasks.
Test the join instead of assuming it
The claim that these tables do not join is checkable, and checking it is more convincing than asserting it. Two tests are enough: do the tables share a column name, and do their labels for people’s work ever line up?
onet_titles <- unique(onet_display$title)
cmap_titles <- unique(
c(
cmap_transitions$job_title_from,
cmap_transitions$job_title_to
)
)
shared_column_names <- intersect(
names(onet_display),
names(cmap_transitions)
)
exact_title_matches <- intersect(onet_titles, cmap_titles)
loosen <- function(titles) {
titles |>
str_to_lower() |>
str_remove("s$") |>
str_squish()
}
loose_title_matches <- intersect(
loosen(onet_titles),
loosen(cmap_titles)
)
join_report <- tibble(
test = c(
"shared column names",
"titles that match exactly",
"titles that match after lowercasing and dropping a final s",
"occupation codes present on both sides"
),
result = c(
length(shared_column_names),
length(exact_title_matches),
length(loose_title_matches),
as.integer(
"onet_soc_code" %in% names(cmap_transitions)
)
)
)| Test | Count |
|---|---|
| shared column names | 0 |
| titles that match exactly | 0 |
| titles that match after lowercasing and dropping a final s | 1 |
| occupation codes present on both sides | 0 |
No column name is shared and no title matches exactly. One pair survives a loosened comparison: the O*NET fixture’s Registered Nurses and the CMap fixture’s Registered Nurse.
That single match is the trap, not the solution. On the O*NET side each row is an occupation-task statement: 29-1141.00 identifies the occupation and the task field identifies one activity associated with it. On the CMap side each row is a transition between job titles, and the table carries no occupation code. Making the strings equal does not make the units equal, and a join on the loosened string would attach task statements to title transitions on the strength of a dropped letter.
A real link needs a crosswalk: a documented mapping from titles to occupation codes, with rules for the many-to-many cases, published by whoever is responsible for the classification. Building one from string similarity and calling the result a crosswalk is how a plausible table acquires an unsupportable claim.
Questions for any provider
The team records what it can establish and leaves unsupported fields as unknown.
| Resource inspected | What supplied it | Unit shown | Coverage represented here | License and rights note | Status of teaching fixture |
|---|---|---|---|---|---|
| onet2r miniature archive | Installed onet2r package | Occupation task statement | 3 rows with U.S. occupation codes | O*NET database material requires CC BY 4.0 attribution; this lesson does not verify the fixture against an official archive | Schema demonstration; exact upstream extraction is not authenticated here |
| cmapr example transitions | Installed cmapr package | Job-title transition | 10 stylized rows across sectors and regions | Full CMap record is CC BY 4.0; license alone does not answer consent questions about underlying source records | Package example; exact relationship to the full dataset is unknown here |
| huggingfaceR task vocabulary | Installed huggingfaceR package | Platform task name | 5 locally bundled classification names | Package code is MIT; each Hub dataset has its own contributor-declared terms and underlying rights still need review | Local software vocabulary; no dataset was downloaded |
- Who collected or generated the data?
- What population, language, place, and time does it cover?
- What does one row represent?
- Which records were excluded or suppressed?
- Which labels were human-reviewed or model-inferred?
- Which version and taxonomy were used?
- What license, attribution, and use restrictions apply?
- Can the provider change or remove records later?
- Does the dataset contain personal or sensitive information?
An open license does not establish that the people represented in source data consented to every later use. A contributor-declared license also does not by itself prove that the contributor held every necessary right.
This page includes information from the O*NET Resource Center by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). Used under the CC BY 4.0 license. O*NET® is a trademark of USDOL/ETA. The onet2r teaching fixture and this lesson may modify or add information; USDOL/ETA has not approved, endorsed, or tested those modifications.
What to remember
- A package is a tool; it is not the authority that created the data.
- Preserve provider, version, date, license, and row meaning.
- Small teaching fixtures demonstrate structure, not population results.
- An example file cannot support a judgment about the source it imitates.
- Platform availability does not establish quality or permission.
- Test a join before believing it, and treat a string match as a question.
- Combine sources only after their units and taxonomies are reconciled.
Three inspections produced three schemas, three units, three rights notes, and no combined table. That is the finding. Nothing here ranks these sources, and nothing here could: the files read above are demonstration data, and a comparison of what O*NET, CMap, or a Hub dataset actually delivers would need authenticated extracts, a stated research question, and evaluation against something other than itself.
Back in the local label table, another comparability problem remains. Three reviewers may read the same requirement differently, so their answers stay separate for the agreement analysis that follows.