This release fixes bugs that returned wrong or incomplete
data without any error or warning. Most important:
get_enade() returned rows that mixed different
students in every edition; any analysis that crossed ENADE
variables at the student level must be redone (see Breaking changes). If
you used get_igc() for 2008-2016,
get_fundeb_distribution() for 2007-2022 or 2025-2026, or
the sub_area column of get_cpc(2008), please
re-run it: see the bug fixes below.
get_enade() now reads one file per
call, chosen with the new arquivo argument (the
number in the file name: arq3 is arquivo = 3).
INEP publishes every ENADE edition (2004-2023) as 27 to 53 files, each
sorted by different variables so that they cannot be joined by student
(LGPD; INEP’s user manual: “nao e possivel agrupar as informacoes no
nivel de estudante”). get_enade() used to glue the files
side by side, so every row combined variables of different students: in
2023, the course code of file 2 differed from file 1 in 406,163 of
406,294 rows. Without arquivo, file 1 is read with a
warning. Files can still be combined at the course level
(co_curso).list_enade_files() lists the files of an edition
and the variables in each, to find the arquivo that holds a
variable.get_conceito_enade() downloads the Conceito Enade,
the course-level ENADE indicator, for 2004-2019, 2021-2023 and 2025
(type = "licenciaturas" or "medicina").
get_enade() returns student microdata; until now the
official indicator itself, with its 1-5 faixa, was only reachable inside
the CPC table up to 2011. For 2007-2011, INEP publishes it inside the
Enade/CPC table, which is returned as is.get_enem(2025, type = "resultados") works. INEP
compressed RESULTADOS_2025.csv with Deflate64, which R’s
built-in unzip cannot read: the file was skipped with only a “zip file
is corrupt” warning and the call failed with “no ENEM data file found”.
ZIP extraction now checks that every file in the archive was extracted
and, if not, falls back to 7-Zip (or PowerShell on Windows,
unzip elsewhere), erroring clearly if the archive is still
incomplete. This applies to every function that downloads a
ZIP.
CSV columns that are empty in their first 1,000 rows no longer
lose their later values. readr guessed those columns as logical and
turned every later value into NA, with only a generic
“parsing issues” warning: get_censo_escolar() lost values
in 1995 (234 of vem2206), 1997 (567 of
vpe626), 1998 and 1999, among others. Columns with parsing
problems are now re-read and typed from all their values. This applies
to every function that reads INEP CSV files.
Columns holding only zeros, which readr read as
FALSE (Censo Escolar 1995-2006 counts), come back
numeric.
get_censo_escolar() returns
dt_ano_letivo_inicio and dt_ano_letivo_termino
as dates in 2008-2021 too. INEP writes “0” for a missing date, so the
columns stayed text; “0” is now read as missing (2015: 188,689 dates,
49,190 missing).
Answer letters “T” and “F” are no longer turned into
TRUE/FALSE. readr reads a column holding only
those letters as logical, so in
get_saeb(2021, type = "diretor") 17,427 answers “F” to
question 106 (and three other questions) came out as FALSE.
INEP files have no logical columns; such columns are now kept as
text.
get_saeb() gains type = "secretario"
(municipal education secretary questionnaire, 2019 onwards), which no
type reached before. type = "escola" no longer falls back
on that file: with level = "educacao_infantil" (2021) it
returned the secretary questionnaire labelled as the school one; it now
stops and lists the files of the edition.
latin1 files whose first accent comes late are no longer read as
UTF-8. The encoding was guessed from the first 10 KB; in
get_censo_escolar(2000) the first accent is 238 MB into the
file, and 34,882 values of uf came out with the accented
letter replaced by “�” (“S?o Paulo”). The encoding is now decided on the
first part of the file that holds a non-ASCII byte.
get_fundeb_distribution() now returns every FUNDEB
source. It read only the tax-share sheets and left out the
Complementacao da Uniao ("COUN"), the Lei Kandir
compensation ("LC8796"), the adjustment sheets
("AJUSTE") and, from 2022, the EC 123/2022 aid and the FTI
share. With source = NULL the sum of valor was
3% to 13% below STN’s own total in 2007-2020 (R$ 11.25 billion missing
for municipalities in 2019). It now equals STN’s TOTAL sheet in every
year and destination, 2007-2026. Footnote rows (“Obs.: …”) are no longer
returned as a 28th state with empty values (2009, 2012, 2013, 2020,
2021).
get_fundeb_enrollment() returns the same column
types on the first call and on cached calls (ano_censo and
qtd_matricula were integer from the API and double from the
cache), and keeps paging the FNDE API until an empty page instead of
stopping at the first short page, which would truncate a year if the
server capped its page size. Counts were checked against an independent
paging of the API: 104,358 records and 45,656,716 enrolments in 2017,
103,560 and 43,400,372 in 2018.
get_capes() now reaches every year it announces
(2013-2024) for every type. CAPES publishes several of
those years under package names that do not follow the usual pattern
(including a “stricto-census” typo), so 28 of the 60 type x year
combinations failed with an error: programas, docentes and cursos
2013-2016, discentes 2013-2020 and catalogo 2013-2020. The file is also
matched on its reference year: in the 2013-2016 packages every file name
contains “2016”, and matching on “contains the year” could pick the
wrong year’s file.
get_ideb(level = "estado" | "regiao", stage = "anos_iniciais", metric = "indicador")
returns the 2025 “Indicador de Rendimento”. INEP’s 2025 spreadsheet
codes that column VL_INDICADOR_REND_2023, so it was
returned as a second 2023 value and 2025 had none. Repeated codes now
take their year from the block header above them (“Taxa de Aprovacao -
2025”). Checked with IDEB = N x P, which now holds to rounding for every
level, stage and edition.
Downloads and API calls (INEP, CAPES, FNDE) now retry when the
connection drops (“server closed abruptly”), not only on HTTP error
codes: httr2::req_retry() does not retry connection
failures unless asked to. Requires httr2 >= 1.0.0.
Date columns (dt_*, dh_*) are no longer
erased when they are not in SAS format. They were parsed with
“12FEB2024:00:00:00” only, so “12/12/2013 00:00:00” became
NA: get_capes(type = "catalogo") lost every
dt_titulacao and dt_matricula in 2013-2016
(67,534 of 67,534 in 2013) and most of four other date columns. SAS,
“dd/mm/yyyy” and ISO dates are now read; a column that no format reads
completely is kept as text.
get_censo_superior(type = "alunos") and
type = "docentes" are deprecated and stop with an
explanation: INEP no longer publishes student or faculty microdata (the
archive of every year, 2009-2024, holds only the IES and course files),
and the old error (“no data file found”) suggested a broken download.
"ies" and "cursos" were checked in every year:
row counts equal the files and no value is lost.
get_enade() 2004-2009 works again: INEP replaced
those files with _LGPD versions and the old addresses no
longer exist.
get_idd(2022) works. INEP serves that edition as a
RAR archive under a .zip name, and the download check
rejected it as “not a valid ZIP archive”. Archives are now identified by
their signature (ZIP, 7z or RAR), both when checking a download and when
extracting it.
Excel columns are no longer emptied when their first 1,000 rows
are blank. readxl guessed the column type from those rows only, read the
column as logical and turned every later value into NA:
get_cpc(2008) lost 1,785 values of sub_area
and get_cpc(2017) 28 of observacao. Types are
now guessed from every row.
get_igc() now returns every institution for
2008-2016. INEP splits those editions into one sheet per type of
institution (universities, university centers, faculties), and
get_igc() read only the first one: about 180-230
universities instead of about 2,000-2,200 institutions per year. All
institution sheets are now read and stacked; course- and program-level
sheets in the same file are still ignored.
get_igc(2007) works again. The 2007 archive holds
three CSV files, one per type of institution, not an Excel file, and
get_igc() stopped with “no Excel file found”. It now reads
and stacks the three CSVs (1,840 institutions).
get_cpc() and get_igc() no longer
return blank, footnote or total rows as data (CPC 2021 and 2022, IGC
2021), nor a sub-header row as the first record (IGC 2008, now with
igc_continuo and igc_faixas columns). A title
row above the header (IGC 2009) is skipped.
get_cpc() 2007 and 2008 no longer return duplicated
column names: “% Docentes Doutores” and similar columns become
pct_docentes_doutores instead of colliding with
docentes_doutores.
On macOS, column names built from accented headers no longer
differ from Windows and Linux: “Município” became
munic_ipio instead of municipio, in every
get_*() function, because macOS’s
iconv(to = "ASCII//TRANSLIT") writes “í” as “’i”. Names are
now folded with ICU (stringi) and are identical on every platform; on
Windows and Linux they are unchanged (checked against 260 accented INEP
headers).
On macOS, latin1 CSV files are no longer read as UTF-8, which
garbled accented text and column names. The encoding check relied on
iconv() returning NA for invalid UTF-8, which
macOS’s iconv does not do; it now uses
validUTF8().
igc_continuo, *_faixas and other
*_continuo columns stored as text are converted to numeric,
like *_faixa already were; markers such as “SC” and “sub
judice” become NA.
This release fixes two bugs that returned the wrong data
without any error or warning. If you used
get_saeb(type = "aluno") or get_encceja() with
educabR 1.1.0 or earlier, please re-check your results: see the first
two items below.
get_saeb() gains a serie argument and no
longer loads the wrong grade silently (#21). Since 2013 INEP ships one
student file per grade (TS_ALUNO_2EF.csv,
TS_ALUNO_5EF.csv, TS_ALUNO_9EF.csv, …), and
type = "aluno" used to read whichever came first
alphabetically: the 2nd grade in 2019-2023, the 3rd year of high school
in 2013-2017, so the 5th and 9th grades were unreachable in every
edition. Now serie = "5ef" (or 5) picks the
file by exact name; when an edition has several grades and
serie is missing, get_saeb() stops and lists
them. For 2011, type = "aluno" now reads
TS_RESULTADO_ALUNO (proficiency results) instead of the
student questionnaire.get_encceja() gains
type = c("regular", "ppl") and now reads the national
regular exam by default. Each edition ships a regular file
(REG_NAC / REGULAR, 260-940 MB) and a much
smaller file for people deprived of liberty (PPL_NAC /
PPL), and get_encceja() used to load whichever
matched first: the PPL file in 2014, 2017-2020 and 2022-2025.
type = "ppl" keeps that file reachable.get_encceja() rejects them up front instead of failing at
download.get_enem() and get_enem_itens() now
support the 2025 edition, published by INEP in 2026. The 2025 zip keeps
the split layout introduced in 2024
(PARTICIPANTES_2025.csv, RESULTADOS_2025.csv,
ITENS_PROVA_2025.csv), so only the known-years list
changed.get_encceja() now supports the 2025 edition (same file
set as 2024: REG_NAC, PPL_NAC,
PPL_NAC_QSE, ITENS_PROVA).Both editions were detected by the portfolio source monitor and validated against the live INEP files.
get_ideb() now supports the IDEB 2025 edition
(released 2026-08-05). INEP changed the packaging for 2025: spreadsheets
now come inside a .zip archive, which the pipeline
downloads and extracts transparently. The cached file remains the inner
.xlsx, so existing cache handling is unchanged.
get_ideb() gains
stage = "ensino_medio_integrado", covering the “Ensino
médio mais educação profissional técnica integrada” spreadsheets first
published with IDEB 2025 — high school IDEB including students
of integrated technical-professional education (EPT integrada), which
the traditional high school cut excludes for historical comparability.
INEP publishes this cut only for level = "brasil" (editions
2019-2025) and level = "estado" (2017-2025), state networks
only, with no official targets (metric = "meta" is rejected
with a clear error). list_ideb_available() now lists these
combinations.
get_ideb() now raises a clear error when the
year filter matches no IDEB edition (listing the editions
actually present in the file), instead of the misleading “file may be
corrupted” message.
Dynamic year discovery no longer removes known years from
available_years() when a verification request fails
transiently — a network hiccup could make a valid year (e.g. 2022 for
the Censo Escolar) unavailable for the whole session (#17). Discovery
now only adds newly published years to the known list.
get_ideb() has a new signature:
get_ideb(level, stage, metric, year, quiet). The old
positional usage get_ideb(year, level, stage) still works
with a deprecation warning, but the year parameter now
filters IDEB editions instead of selecting which file to download.get_ideb() now returns data in tidy long format instead
of wide format. Output columns depend on the metric
parameter ("indicador", "aprovacao",
"nota", "meta").get_ideb() now supports 5 geographic levels:
"escola", "municipio", "estado",
"regiao", and "brasil" (previously only escola
and municipio).get_ideb() always downloads the most recent IDEB file
available, which contains the full historical series. The
year parameter filters editions.get_ideb_series() is deprecated. Use
get_ideb(level, stage, metric) instead.list_ideb_available() now returns level,
stage, and metric columns (previously returned
year, level, stage).uf parameter has been removed from
get_ideb(). Filter the result with
dplyr::filter() instead.download_inep_file() timeout is now configurable via
options(educabR.download_timeout = N) (seconds; default
600). Raise it when downloading large microdata (e.g. ENEM
participantes at ~1.6 GB) over a slow link (issue #7).read_inep_file() now warns before reading files larger
than 500 MB entirely into memory, suggesting n_max or UF
filters to reduce memory pressure. Suppressible with
quiet = TRUE; all get_*() callers propagate
their quiet argument (issue #5).get_ideb() no longer consumes several GB of RAM for
school-level reads (issue #1). The xlsx is now read with column
projection: only the vl_* columns matching the requested
metric (and year, when given) are parsed; the
others are skipped at the readxl C++ layer. INEP’s NA tokens
("", "-", "ND") are also passed
to read_excel(na = ...) so the missing-value strings never
get allocated as R character vectors. For
level = "escola", stage = "anos_iniciais", metric = "indicador",
this cuts the in-memory result from ~133 MB to ~37 MB (4 years) or ~19
MB (1 year), with proportional drops in peak memory during reshape.read_ideb_excel(), read_excel_safe()) and the
FUNDEB enrollment OData fetcher (fetch_fundeb_enrollment())
are now normalized to UTF-8 NFC, matching the behavior already in
read_inep_file(). Previously, equality comparisons against
literals such as filter(rede == "Pública") could silently
return zero rows on Windows because the source-file encoding produced
non-canonical strings. The shared helper
normalize_utf8_nfc() is now applied at every read
entrypoint so all four code paths agree. Affects
get_ideb(), get_cpc(), get_igc(),
get_fundeb_enrollment().read_excel_safe() (used by get_cpc() and
get_igc()) now passes INEP’s missing-value tokens
("", "-", "ND", en/em dashes) to
readxl::read_excel(na = ...) so those cells are loaded as
NA instead of character strings cleaned up post-hoc (issue
#4). Previously, a column whose first rows were all "-"
could be inferred as logical and later numeric values
silently dropped. clean_dash_values() remains as a safety
net but is now largely redundant for CPC/IGC.download_inep_file() now verifies downloaded files
before caching them (issue #3). Three checks run after the bytes hit
disk: file size against the server’s Content-Length (1%
tolerance, catches truncated downloads), HTML-masquerade detection on
the first 64 bytes (catches INEP maintenance pages served with HTTP
200), and ZIP magic-bytes (PK\x03\x04) for
.zip destinations (catches proxy corruption). On any
failure the corrupt file is deleted and the user gets a clear error
telling them to retry, instead of a cryptic readxl /
readr failure on the next call.validate_year() now rejects vectors and non-numeric
input with a clear error pointing at purrr::map_dfr() for
multi-year composition (issue #2). Previously, passing
c(2017, 2019) to any of the 13 affected getters
(get_cpc, get_idd, get_igc,
get_capes, get_saeb, get_enem,
get_enem_itens, get_enade,
get_encceja, get_fundeb_distribution,
get_fundeb_enrollment, get_censo_escolar,
get_censo_superior) hit either a cryptic
length > 1 error (R ≥ 4.2) or silently used only the
first element (R < 4.2). get_ideb() is unaffected — it
intentionally accepts year vectors.extract_zip() cleaned up: removed dead
if (TRUE) branch and an unreachable
cli_abort(); the muffle on extraction warnings was
tightened from the broad erro|error pattern to the two
specific messages that motivated it (issue #6).RoxygenNote bumped to 8.0.0 and man/*.Rd
regenerated; systemfonts and textshaping
declared in Suggests: to silence the cosmetic
R CMD check NOTE about packages pulled transitively by
pkgdown.R CMD check warnings cleared: em-dashes in
cli_abort() message strings in
R/utils-download.R are now written with Unicode escapes (R
requires ASCII-only in code strings; comments are exempt).available_years() now dynamically discovers available
years by querying data sources (HEAD requests for INEP, OData queries
for FNDE). Results are cached per session. Falls back to a hardcoded
list when offline.available_years() now accepts
"fundeb_enrollment" as a separate dataset name. Previously,
"fundeb" was shared between distribution and
enrollment.CO_*, CD_*) are now read as
character instead of numeric across all datasets. This prevents loss of
leading zeros in codes like municipality codes, course codes, and
institution codes.get_enade() failing for 9 of 19 available years.
INEP uses inconsistent URLs for ENADE: _LGPD suffix for
2012-2019, .rar format for 2022. Added hardcoded URL map
(enade_urls) with all 19 correct URLs.get_fundeb_enrollment() accepting years with no
data in the FNDE API. The API currently only has data for
2017-2018.clear_cache() failing to delete files on Windows
when they were memory-mapped by readr. Now deletes entire directories
and warns about locked files..rar archive extraction support via 7-Zip.
find_7z() searches common Windows install paths when
7z is not in PATH.strip_diacriticals() internal helper for
encoding-safe text matching.read_inep_file() now auto-detects code columns
(CO_*, CD_*) from the file header and reads
them as character. No user action required.read_ideb_excel() and read_excel_safe()
(CPC/IGC) now convert code columns to character after reading.get_fundeb_distribution(): Download FUNDEB resource
distribution data (years 2007-2026). Reads all sheets from STN Excel
files and returns tidy long-format data with monthly transfer amounts by
state, funding source, destination (states/municipalities), and table
type (fundeb/adjustment).get_fundeb_enrollment(): Download FUNDEB enrollment
data. Fetches from FNDE OData API with automatic pagination. Results
cached as CSV.uf,
source (FPE, FPM, ICMS, etc.), and destination
(“uf” or “municipio”).https://www.tesourotransparente.gov.br) and FNDE
(https://www.fnde.gov.br).get_capes(): Download CAPES graduate education data
(years 2013-2024)."programas"), students
("discentes"), faculty ("docentes"), courses
("cursos"), and theses/dissertations catalog
("catalogo").https://dadosabertos.capes.gov.br).get_cpc(): Download CPC data (years 2007-2019,
2021-2023; no 2020 edition).readxl package.get_igc(): Download IGC data (years 2007-2019,
2021-2023; no 2020 edition).read_excel_safe(): Internal helper to read Excel files
with error handling.get_enem_escola(): Download ENEM results aggregated by
school (2005-2015).get_idd(): Download IDD microdata (years 2014-2019,
2021-2023; no 2020 edition).extract_archive() utility.get_encceja(): Download ENCCEJA microdata (years
2014-2024).get_enade(): Download ENADE microdata.get_censo_superior(): Download Higher Education Census
microdata (years 2009-2024)."ies"),
courses ("cursos"), students ("alunos"), and
faculty ("docentes").list_censo_superior_files(): List available files in a
downloaded census.uf parameter.get_saeb(): Download SAEB microdata (years 2011, 2013,
2015, 2017, 2019, 2021, 2023)."aluno"), school ("escola"), principal
("diretor"), and teacher ("professor")
questionnaires.level parameter.iconv()
instead of validEnc()."Latin-1" encoding name to "latin1"
for Windows codepage compatibility.type parameter for split
files ("participantes", "resultados").dt_*).vl_* columns from character to numeric,
handling "-", "ND", and comma decimals.get_ideb_series() now shows per-year progress
indication (e.g., “processing IDEB 2017 (1/4)”) and propagates the
quiet parameter to inner get_ideb()
calls.get_enem_itens() now has keep_zip
parameter for consistency with get_enem() and
get_censo_escolar().README.md) as default; Portuguese
version renamed to README.pt-br.md with cross-links between
both.@param year ranges in documentation to match
available_years():
get_enem() / get_enem_itens(): 2009-2023
-> 1998-2024get_censo_escolar(): 2007-2024 -> 1995-2024@family tags to group related functions in help
pages (ENEM, IDEB, School Census, cache).getting-started.Rmd).README.pt-br.md.enem_summary(): statistics calculation,
NA handling, grouping by variable, and error on missing score
columns.validate_data(): empty data, few
columns, missing expected columns per dataset.\donttest with \dontrun in all
examples per CRAN request.set_cache_dir() example that created a directory
in the user’s home (~/educabR_cache) during CRAN checks.
Now uses tempdir() in examples.First public release.
get_ideb(): Download IDEB data (years 2017, 2019, 2021,
2023).get_ideb_series(): Download IDEB historical series
across multiple years.list_ideb_available(): List available year/stage/level
combinations.get_enem(): Download ENEM microdata (years
1998-2024).get_enem_itens(): Download ENEM item response
data.enem_summary(): Calculate summary statistics for ENEM
scores.get_censo_escolar(): Download School Census microdata
(years 1995-2024).list_censo_files(): List available files in a
downloaded census.set_cache_dir(): Set custom cache directory.get_cache_dir(): Get current cache directory.clear_cache(): Clear cached files.list_cache(): List cached files with metadata.available_years(): Get available years for each
dataset.