dm 1.0.11.9014
Continuous integration
- Avoid failure in fledge workflow if no changes (#2273).
dm 1.0.11.9013
Continuous integration
Debug fledge.
Fetch tags for fledge workflow to avoid unnecessary NEWS entries (#2270).
Documentation
- Restore empty space removed by styler (#2269).
dm 1.0.11.9012
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Chore
Ignore.
Fix creation of documentation in the RStudio IDE [ci skip].
Bump RMariaDB version (#2244).
Auto-update from GitHub Actions (#2231).
-
Auto-update from GitHub Actions.
Use dev RMariaDB.
-
Auto-update from GitHub Actions.
Roxygenize.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
Use CRAN dbplyr.
Continuous integration
Use larger retry count for lock-threads workflow (#2267).
Ignore errors when removing pkg-config on macOS (#2257).
Explicit permissions (#2254).
Use styler from main branch (#2252).
Need to install R on Ubuntu 24.04 (#2250).
Use Ubuntu 24.04 and styler PR (#2248).
Configure token.
Count rulesets to understand if branch protection is enabled.
Remove Aviator.
Use stable pak (#2239).
Trigger run (#2238).
Use pkgdown branch (#2237).
Install via R CMD INSTALL ., not pak (#2230).
Install local package for pkgdown builds.
Improve support for protected branches with fledge.
Improve support for protected branches, without fledge.
Sync with latest developments.
Use v2 instead of master.
Inline action.
Use dev roxygen2 and decor.
Fix on Windows, tweak lock workflow.
Avoid checking bashisms on Windows.
Better commit message.
Harmonize workflows.
Use value after
source()
.Bump versions, better default.
Recent updates.
Prepare for dynamic build matrix.
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.11.9011
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Chore
Ignore.
Fix creation of documentation in the RStudio IDE [ci skip].
Bump RMariaDB version (#2244).
Auto-update from GitHub Actions (#2231).
-
Auto-update from GitHub Actions.
Use dev RMariaDB.
-
Auto-update from GitHub Actions.
Roxygenize.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
Use CRAN dbplyr.
Continuous integration
Ignore errors when removing pkg-config on macOS (#2257).
Explicit permissions (#2254).
Use styler from main branch (#2252).
Need to install R on Ubuntu 24.04 (#2250).
Use Ubuntu 24.04 and styler PR (#2248).
Configure token.
Count rulesets to understand if branch protection is enabled.
Remove Aviator.
Use stable pak (#2239).
Trigger run (#2238).
Use pkgdown branch (#2237).
Install via R CMD INSTALL ., not pak (#2230).
Install local package for pkgdown builds.
Improve support for protected branches with fledge.
Improve support for protected branches, without fledge.
Sync with latest developments.
Use v2 instead of master.
Inline action.
Use dev roxygen2 and decor.
Fix on Windows, tweak lock workflow.
Avoid checking bashisms on Windows.
Better commit message.
Harmonize workflows.
Use value after
source()
.Bump versions, better default.
Recent updates.
Prepare for dynamic build matrix.
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.11.9010
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Chore
Ignore.
Fix creation of documentation in the RStudio IDE [ci skip].
Bump RMariaDB version (#2244).
Auto-update from GitHub Actions (#2231).
-
Auto-update from GitHub Actions.
Use dev RMariaDB.
-
Auto-update from GitHub Actions.
Roxygenize.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
Use CRAN dbplyr.
Continuous integration
Ignore errors when removing pkg-config on macOS (#2257).
Explicit permissions (#2254).
Use styler from main branch (#2252).
Need to install R on Ubuntu 24.04 (#2250).
Use Ubuntu 24.04 and styler PR (#2248).
Configure token.
Count rulesets to understand if branch protection is enabled.
Remove Aviator.
Use stable pak (#2239).
Trigger run (#2238).
Use pkgdown branch (#2237).
Install via R CMD INSTALL ., not pak (#2230).
Install local package for pkgdown builds.
Improve support for protected branches with fledge.
Improve support for protected branches, without fledge.
Sync with latest developments.
Use v2 instead of master.
Inline action.
Use dev roxygen2 and decor.
Fix on Windows, tweak lock workflow.
Avoid checking bashisms on Windows.
Better commit message.
Harmonize workflows.
Use value after
source()
.Bump versions, better default.
Recent updates.
Prepare for dynamic build matrix.
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.11.9009
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Chore
Ignore.
Fix creation of documentation in the RStudio IDE [ci skip].
Bump RMariaDB version (#2244).
Auto-update from GitHub Actions (#2231).
-
Auto-update from GitHub Actions.
Use dev RMariaDB.
-
Auto-update from GitHub Actions.
Roxygenize.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
Use CRAN dbplyr.
Continuous integration
Ignore errors when removing pkg-config on macOS (#2257).
Explicit permissions (#2254).
Use styler from main branch (#2252).
Need to install R on Ubuntu 24.04 (#2250).
Use Ubuntu 24.04 and styler PR (#2248).
Configure token.
Count rulesets to understand if branch protection is enabled.
Remove Aviator.
Use stable pak (#2239).
Trigger run (#2238).
Use pkgdown branch (#2237).
Install via R CMD INSTALL ., not pak (#2230).
Install local package for pkgdown builds.
Improve support for protected branches with fledge.
Improve support for protected branches, without fledge.
Sync with latest developments.
Use v2 instead of master.
Inline action.
Use dev roxygen2 and decor.
Fix on Windows, tweak lock workflow.
Avoid checking bashisms on Windows.
Better commit message.
Harmonize workflows.
Use value after
source()
.Bump versions, better default.
Recent updates.
Prepare for dynamic build matrix.
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.11.9008
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Chore
Ignore.
Fix creation of documentation in the RStudio IDE [ci skip].
Bump RMariaDB version (#2244).
Auto-update from GitHub Actions (#2231).
-
Auto-update from GitHub Actions.
Use dev RMariaDB.
-
Auto-update from GitHub Actions.
Roxygenize.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
Use CRAN dbplyr.
Continuous integration
Ignore errors when removing pkg-config on macOS (#2257).
Explicit permissions (#2254).
Use styler from main branch (#2252).
Need to install R on Ubuntu 24.04 (#2250).
Use Ubuntu 24.04 and styler PR (#2248).
Configure token.
Count rulesets to understand if branch protection is enabled.
Remove Aviator.
Use stable pak (#2239).
Trigger run (#2238).
Use pkgdown branch (#2237).
Install via R CMD INSTALL ., not pak (#2230).
Install local package for pkgdown builds.
Improve support for protected branches with fledge.
Improve support for protected branches, without fledge.
Sync with latest developments.
Use v2 instead of master.
Inline action.
Use dev roxygen2 and decor.
Fix on Windows, tweak lock workflow.
Avoid checking bashisms on Windows.
Better commit message.
Harmonize workflows.
Use value after
source()
.Bump versions, better default.
Recent updates.
Prepare for dynamic build matrix.
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.11.9007
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Chore
Ignore.
Fix creation of documentation in the RStudio IDE [ci skip].
Bump RMariaDB version (#2244).
Auto-update from GitHub Actions (#2231).
-
Auto-update from GitHub Actions.
Use dev RMariaDB.
-
Auto-update from GitHub Actions.
Roxygenize.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
-
Auto-update from GitHub Actions.
Use CRAN dbplyr.
Continuous integration
Ignore errors when removing pkg-config on macOS (#2257).
Explicit permissions (#2254).
Use styler from main branch (#2252).
Need to install R on Ubuntu 24.04 (#2250).
Use Ubuntu 24.04 and styler PR (#2248).
Configure token.
Count rulesets to understand if branch protection is enabled.
Remove Aviator.
Use stable pak (#2239).
Trigger run (#2238).
Use pkgdown branch (#2237).
Install via R CMD INSTALL ., not pak (#2230).
Install local package for pkgdown builds.
Improve support for protected branches with fledge.
Improve support for protected branches, without fledge.
Sync with latest developments.
Use v2 instead of master.
Inline action.
Use dev roxygen2 and decor.
Fix on Windows, tweak lock workflow.
Avoid checking bashisms on Windows.
Better commit message.
Harmonize workflows.
Use value after
source()
.Bump versions, better default.
Recent updates.
Prepare for dynamic build matrix.
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.11
CRAN release: 2024-11-25
Features
- Add support for Redshift connections (@owenjonesuob, #2215).
Testing
- Fix compatibility with waldo >= 0.6.0 (#2240).
dm 1.0.10
CRAN release: 2024-01-21
Chore
Establish compatibility with igraph >= 2.0.0 (#2187) and withr 3.0.0 (#2184).
Reexport
tibble::glimpse()
instead ofpillar::glimpse()
to avoid pillar dependency with roxygen2 7.3.0 (#2179).
dm 1.0.9
CRAN release: 2024-01-08
Features
-
dm_sql()
now processestable_names
withdbplyr::escape()
, therefore also accepting dbplyr objects (#2129).
dm 1.0.8
CRAN release: 2023-11-02
Bug fixes
-
copy_dm_to()
creates string columns of necessary lengths for MariaDB and SQL Server. This worked before for SQL Server in dm 1.0.5, now also works on MariaDB (#311, #2066, #2082).
Chore
- Make
check_suggested()
a standalone (#2054).
Performance
- Speed up
dm()
,new_dm()
,as_dm()
anddm_validate()
.dm()
andas_dm()
no longer calldm_validate()
(#2108).
dm 1.0.7
CRAN release: 2023-10-24
Features
copy_dm_to()
now warns unconditionally on unsupported arguments, and fails ifcopy_to
is provided (#1944). Use the newdm_sql()
function as a replacement forcopy_dm_to(copy_to = )
(#1915, #2011, @jangorecki).New
json_unnest()
andjson_unpack()
, currently implemented for data frames only (#991, #997).dm_rows_append()
also works for local dm, with support for autoincrement primary keys (#1727, #1745).Breaking change: Add
check_dots_empty()
calls (#1929, #1943).Test MySQL on GHA (#1940).
Improve MySQL compatibility regarding learning of database schemas and checking of constraints (#1938).
Bug fixes
Compatibility with duckdb 0.9.1.
Minor fixes in
dm_pack_tbl()
anddm_unwrap_tbl()
(#1947).
Documentation
Use
rlang::check_installed()
internally to install missing suggested packages on the fly (@olivroy, #1348, #2036, #2039, #2040).Use vectorized
rlang::is_installed()
to decide if examples should be run (@olivroy, #2043).Recategorize and describe function reference.
Better error and information messages when querying keys.
collect.zoomed_dm()
shows a more helpful error message (#1929, #1945).Add
db-*
rules to Makefile to simplify Docker-based database setup.
dm 1.0.6
CRAN release: 2023-07-21
Bug fixes
Compare version returned by
getRversion()
with string instead of number.Work around vctrs bug in jsonlite 1.8.5.
Features
dm_from_con()
gains.names
argument for pattern-based construction of table names in the dm object (@owenjonesuob, #1790).New
dm_set_table_descriptions()
,dm_get_table_descriptions()
anddm_reset_table_descriptions()
to set table labels as persistent attributes of the table object (#1888).dm_from_con()
can retrieve multiple schemas, pass a character vector to theschema
argument (@owenjonesuob, #1533, #1789).build_copy_queries()
anddb_learn_from_db()
improvements (@samssann, #1642, #1677, #1739).Allow for additional description of tables in dm_draw() (#1875, #1876).
Chore
Establish compatibility with dbplyr 2.3.3 and 2.4.0 (@mgirlich, #1919).
In
copy_dm_to()
, callcollect()
only when copying data, table by table (@jangorecki, #1900).Use roxyglobals (#1838).
Documentation
Add table description to diagram in README.
Tweak testing instructions. Mention
Makefile
in CONTRIBUTING.md. Describe Docker setup (#1898).Vignette corrections (@MikeJohnPage, #1882).
Avoid tidyverse package.
Testing
- Fix local tests (#1921).
dm 1.0.5
CRAN release: 2023-03-16
Features
- Progress bars for
dm_wrap_tbl()
anddm_unwrap_tbl()
(#835, #1450).
dm 1.0.4
CRAN release: 2023-02-11
Features
dm_add_pk()
gainsautoincrement
argument (#1689), autoincrement primary keys are configured on the database withcopy_dm_to()
(#1696).New
dm_add_uk()
,dm_rm_uk()
anddm_get_all_uks()
functions for explicit support of unique keys (#622, #1716).dm_get_all_pks()
anddm_get_all_fks()
return output in the order oftable
orparent_table
argument (#1707).Improve error message for
dm_add_pk()
when thecolumns
argument is missing (#1644, #1646).
Breaking changes
-
dm_get_all_pks()
,dm_get_all_fks()
, anddm_get_all_uks()
require unquoted table names as input, for consistency with other parts of the API (#1741).
Bug fixes
-
dm_examine_constraints()
works fordm
objects on the database with compound keys (#1713).
Documentation
Update pkgdown URL to https://dm.cynkra.com/ (#1652).
Fix link rot (#1671).
Internal
Require dplyr >= 1.1.0 and lifecycle >= 1.0.3 (#1771, #1637).
Checks pass if all suggested packages are missing (#1659).
Fix r-devel builds (#1776).
dm_unpack_tbl()
sets PK before FK (#1715).Clean up
dm_rows_append()
implementation (#1714).dm()
accepts tables that are of class"tbl_sql"
but not"tbl_dbi"
(#1695, #1710).Use correctly typed missing value for lists (@DavisVaughan, #1686).
dm 1.0.2
CRAN release: 2022-09-20
Bug fixes
-
dm_paste()
gives correct output for factor columns with many levels (#1510, #1511).
Chore
Refactor
dm_unwrap_tbl()
so it builds a “unwrap plan” first (#1446, #1447).Reenable
dm_rows_update()
test (#1437).
dm 1.0.1
CRAN release: 2022-08-06
Features
- New
dm_deconstruct()
creates code to deconstruct adm
object into individual keyed tables viapull_tbl(keyed = TRUE)
(#1354).
Bug fixes
- Use
dm_ptype()
indm_gui()
, generate better code (#1353).
dm 1.0.0
CRAN release: 2022-07-21
Features
New
dm_gui()
for interactive editing ofdm
objects (#1076, #1319).dm_get_tables()
andpull_tbl()
gain a newkeyed = FALSE
argument. If set toTRUE
, table objects of class"dm_keyed_tbl"
are returned. These objects inherit from the underlying data structure (tibble or lazy table), keep track of primary and foreign keys, and can be used later on in a call todm()
to recreate a dm object with the keys (#1187).New
by_position
argument tocheck_subset()
,check_set_equality()
,check_cardinality_...()
andexamine_cardinality()
(#1253).dm_examine_constraints()
honors implicit unique keys defined by foreign keys (#1131, #1209).
Breaking changes
dm_filter()
is now stable, with a new API that avoids exposing an intermediate state with filters not yet applied, with a compatibility wrapper (#424, #426, #1236).check_cardinality_...()
,examine_cardinality()
,check_subset()
andcheck_set_equality()
are now stable and consistently use a common interface with arguments namedx
,y
,x_select
andy_select
, with compatibility wrappers (#1194, #1229).dm_examine_cardinalities()
anddm_examine_constraints()
are now stable with a new signature and a compatibility wrapper (#1193, #1195).dm_apply_filters()
,dm_apply_filters_to_tbl()
anddm_get_filters()
are deprecated (#424, #426, #1236).dm_disambiguate_cols()
adds table names as a suffix by default, and gains a.position
argument to restore the original behavior. Argumentssep
andquiet
are renamed to.sep
and.quiet
(#1293, #1327).dm_squash_to_tbl()
is deprecated in favor of the new.recursive
argument todm_flatten_to_tbl()
. Argumentsstart
andjoin
are renamed to.start
and.join
(#1272, #1324).dm_rm_tbl()
is deprecated in favor ofdm_select_tbl()
(#1275).dm_bind()
anddm_add_tbl()
are deprecated in favor ofdm()
(#1226).rows_truncate()
anddm_rows_truncate()
are deprecated, because they use DDL as opposed to all other verbs that use DML (#1031, #1321).All internal S3 classes now use the
"dm_"
prefix (#1285, #1339).Add ellipses to all generics (#1298).
API
dm_ptype()
,dm_financial()
anddm_pixarfilms()
are stable now (#1254).Turn all “questioning” functions to “experimental” (#1030, #1237).
Performance
is_unique_key()
usesvctrs::vec_count()
on local data frames for speed (@eutwt, #1247).check_key()
usesvctrs::vec_duplicate_any()
on local data frames for speed (@eutwt, #1234).
Bug fixes
Don’t print rule in
glimpse.dm()
for emptydm()
(#1208).
Documentation
Work around ANSI escape issues in CRAN rendering of vignette (#1156, #1330).
Fix column names in
?dm_get_all_pks
(#1245).Improve contrast for display of
dm_financial()
(#1073, #1250).Add contributing guide (#1222).
dm 0.3.0
CRAN release: 2022-07-06
Features
Implement
glimpse()
forzoomed_df
(@IndrajeetPatil, #1003, #1161).Remove message about automated key selection with the
select
argument in joins onzoomed_df
(@IndrajeetPatil, #1113, #1176).dm_from_con(learn_keys = TRUE)
works for MariaDB (#1106, #1123, #1169, @maelle), and for compound keys in Postgres (#342, #1006, #1016) and SQL Server (#342).New
json_pack_join()
,json_nest_join()
,json_pack()
andjson_nest()
, similar topack_join()
,dplyr::nest_join()
,tidyr::pack()
andtidyr::nest()
, but create character columns (#917, #918, #973, #974).nest_join()
andpack_join()
supportzoomed_df
objects (#1119, @IndrajeetPatil).
API
Marked stable functions as stable, in particular
dm()
and related functions (#1032, #1040).Remove own
rows_*()
implementation for lazy tables, they are now available in dbplyr >= 2.2.0 (#912, #1024, #1028).Deprecate
dm_join_to_tbl()
,dm_is_referenced()
anddm_get_referencing_tables()
(#1038).New
dm_validate()
replaces now deprecatedvalidate_dm()
(#1033).dm_get_con()
anddm_get_filters()
usedm
as argument name (#1034, #1036).Mark
...
indm_flatten_to_tbl()
as experimental (#1037).Add ellipses to
dm_disambiguate_cols()
,dm_draw()
,dm_examine_constraints()
,dm_nycflights13()
anddm_pixarfilms()
(#1035).New
dm_from_con()
, soft-deprecateddm_from_src()
(#1014, #1018, #1044).Moved
pack_join()
arguments past the ellipsis for consistency (#920, #921).
Documentation
The pkgdown site now uses BS5 for greater readability (#1067, @maelle).
Better message for
dm_rows_...()
functions if thein_place
argument is missing (@IndrajeetPatil, #414, #1160).Better message for learning error (#1081).
Greatly improved consistency, content, and language across all articles (@IndrajeetPatil, #1056, #1132, #1157, #1166, #1079, #1082, #1098, #1100, #1101, #1103, #1112, #1120, #1158, #1175).
Tweaks of intro vignette and README (#1066, #1075, @maelle).
Document
glimpse()
S3 method fordm
(@IndrajeetPatil, #1121).Update credentials to fallback databases for
dm_financial()
hosted on pacha.dev (#916, @pachadotdev), also used now for vignettes (#1118) and indm_from_con()
example (#993).Update license year (#1029).
dm 0.2.8
CRAN release: 2022-04-08
Features
-
pack_join()
works correctly ifname
is the same as an existing column in either table. In some cases a column is overwritten, this is consistent withnest_join()
behavior (#864, #865). - Messages that suggest the installation of optional packages are shown only once per session (#852).
-
dm_insert_zoomed()
uses the color from the zoomed table for the new table (#750, #863). -
copy_dm_to()
consumes less memory and is faster when writing to SQL Server (#855).
Bug fixes
- Remove extra spaces in output when examining constraints with compound keys (#868).
- Fix column tracking for foreign keys (#856, #857).
-
copy_dm_to()
shows progress bars again (#850, #855). - Progress bars use the console width (#853).
- Avoid calling
dbAppendTable()
for zero-row tables (#847).
Internal
- Require rlang 1.0.1 (#840).
dm 0.2.7
CRAN release: 2022-02-03
Features
- New
dm_wrap_tbl()
,dm_unwrap_tbl()
,dm_nest_tbl()
,dm_unnest_tbl()
,dm_pack_tbl()
anddm_unpack_tbl()
(#595, #733, #737). - New
dm_examine_cardinality()
(#264, #735). - New
pack_join()
generic and method for data frames, the same totidyr::pack()
asdplyr::nest_join()
is totidyr::nest()
(#721, #722). -
dm_pixarfilms()
is exported and gains aconsistent = FALSE
argument; ifTRUE
the data is modified so that all referential constraints are satisfied (#703, #707, #708, @erictleung).
Bug fixes
-
db_schema_...()
functions no longer pro-actively check for schema existence (#672, #815, #771). -
db_schema_list.Microsoft SQL Server
no longer ignoring schemas for which the owner cannot be found (#815, #771). -
copy_dm_to()
works with DuckDB again, the SQL statements to define the tables and indexes are now created by us (#701, #709).
dm 0.2.6
CRAN release: 2021-11-21
Features
- New
dm_pixarfilms()
creates a dm object with data from the {pixarfilms} package (#600, @erictleung). -
check_cardinality_0_1()
,check_cardinality_0_n()
,check_cardinality_1_1()
,check_cardinality_1_n()
, andexamine_cardinality()
now support compound keys (#524). -
check_subset()
andcheck_set_equality()
support compound keys (#523). -
dm_paste()
adds theon_delete
argument todm_add_fk()
(#673). -
dm_disambiguate_cols()
also disambiguates columns used in keys, to support correct disambiguation for compound keys (#662). -
dm_disambiguate_cols()
now emits the source code equivalent of a renaming operation (#684). -
dm_examine_constraints()
uses backticks to surround table names (#687).
Bug fixes
-
decompose_table()
now avoids creatingNA
values in the key column (#580). -
dm_draw()
works with empty tables (#585).
Internal
- Fix compatibility with dplyr 1.0.8 (#698).
dm 0.2.5
CRAN release: 2021-10-15
Features
-
mutate()
,transmute()
,distinct()
andsummarize()
now supportdplyr::across()
and extra arguments (#640). - Key tracking for the first three verbs is less strict and based on name equality (#663).
-
relocate()
now works on zoomeddm
objects (#666). -
dm_add_fk()
gainson_delete
argument whichcopy_dm_to()
picks up and translates to anON DELETE CASCADE
orON DELETE NO ACTION
specification for the foreign key (#649). -
dm_copy_to()
defines foreign keys during table creation, for all databases except DuckDB. Tables are created in topological order (#658). For cyclic relationship graphs, table creation is attempted in the original order and may fail (#664). -
waldo::compare()
shows better output for dm objects (#642). -
dm_paste()
output uses trailing commas in thedm::dm()
andtibble::tibble()
calls, and sorts column attributes by name, for better modularity (#641).
Breaking changes
- New
db_schema_create()
,db_schema_drop()
,db_schema_exists()
anddb_schema_list()
replace the correspondingsql_schema_*()
functions, the latter are soft-deprecated (#670). The connection argument todb_schema_*()
is calledcon
, notdest
(#668).
Bug fixes
-
copy_dm_to()
andsql_create_schema()
no longer actively check for schema existence (#644, #660). - Add newline after
OUTPUT
clause for SQL Server (#647). - Fix
sql_rows_delete()
withreturning
argument for SQL Server (#645).
dm 0.2.4
CRAN release: 2021-09-30
Features
-
rows_insert()
,rows_update()
androws_delete()
gainreturning
argument. In combination within_place = TRUE
this argument makes the newly inserted rows accessible viaget_returning_rows()
after the operation completes (#593, @mgirlich). - Implement
rows_patch()
for DBI connections (#610, @mgirlich). - Use
NO ACTION
instead ofCASCADE
in foreign key constraints to permit self-references. -
dm_from_src()
supportspool::Pool
objects (#599, @moodymudskipper). - Better error message for
dm_rows_update()
and related functions for dm objects with tables without primary key (#592). -
glimpse()
is implemented fordm
objects (#605). - Support DuckDB in
rows_insert()
,rows_update()
androws_delete()
(#617, @mgirlich).
Bug fixes
- Fix
dm_zoom_to()
fordm
objects with an empty table (#626, @moodymudskipper). - Avoid generating invalid
dm
objects in some corner cases (#596).
Internal
-
sql_schema_list()
supportspool::Pool
objects (#633, @brancengregory). - Establish compatibility with pillar 1.6.2, vctrs > 0.3.8 and rlang > 0.4.11 (#613).
- Use
check_suggested()
everywhere (#572, @moodymudskipper). - Add CI run for validating all new
dm
objects (#597).
dm 0.2.3
CRAN release: 2021-06-20
Bug fixes
- Fix
rows_truncate()
in interactive mode (#588).
Features
- Implement
rows_delete()
for databases (#589).
dm 0.2.2
CRAN release: 2021-06-13
Features
-
dm_examine_constraints()
,dm_rows_insert()
and related,copy_dm_to()
andcollect.dm()
show progress bars in interactive mode via the progress package. The newprogress = NA
argument controls the behavior (#262, @moodymudskipper). -
copy_dm_to()
gains acopy_to
argument to support other ways of copying data to the database (#582).
Internal
- Always run database tests on sqlite for df source.
- Establish compatibility with testthat > 3.0.2 (#566, @moodymudskipper).
dm 0.2.1
CRAN release: 2021-05-11
Breaking changes
-
dm_get_all_fks()
returns a data frame with aparent_key_cols
instead of aparent_pk_cols
column (introduced in dm 0.2.0), to reflect the fact that a foreign key no longer necessarily points to a primary key (#562). -
*_pk()
and*_fk()
functions now verify that the dots are actually empty (#536). -
dm_get_pk()
is deprecated in favor ofdm_get_all_pks()
(#561). -
dm_has_fk()
anddm_get_fk()
are deprecated in favor ofdm_get_all_fks()
(#561).
Features
-
dm_add_fk()
gainsref_columns
argument that supports creating foreign keys to non-primary keys (#402). -
dm_get_all_pks()
gainstable
argument for filtering the returned primary keys (#560). -
dm_get_all_fks()
gainsparent_table
argument for filtering the returned foreign keys (#560). -
dm_rm_fk()
gains an optionalref_columns
argument. This function now supports removal of multiple foreign keys filtered by parent or child table or columns, with a message (#559). -
dm_rm_pk()
gainscolumns
argument and allows filtering by columns and by tables or removing all primary keys. Therm_referencing_fks
argument has been deprecated in favor of the newfail_fk
argument (#558). -
dm_get_all_fks()
has been optimized for speed and no longer sorts the keys (#560). - dm operations are now slightly faster overall.
Internal
- The internal data structure for a dm object has changed to accommodate foreign keys to other columns than the primary key. An upgrade message is shown when working with a dm object from an earlier version, e.g. if it was loaded from a cache or an
.rds
file (#402). - Drop
"dm_v1"
class from dm objects again, this would have made every S3 dispatch more costly. Relying on an internal"version"
attribute instead (#547).
dm 0.2.0
CRAN release: 2021-05-03
Breaking changes
- Deprecate
dm_get_src()
tbl.dm()
,src_tbls.dm()
,copy_to.dm()
. These functions have better alternatives and use the notion of a “data source” which is being phased out of dplyr (#527). -
*_pk()
and*_fk()
functions gain an ellipsis argument that comes beforecheck
,force
andrm_referencing_fks
arguments (#520).
Features
-
dm_add_pk()
anddm_add_fk()
support compound keys via thec()
notation, e.g.dm_add_pk(dm, table, c(col1, col2))
.dm_nycflights13()
returns a data model with compound keys by default. Usecompound = FALSE
to return the data model from dm v0.1.13 or earlier (#3). -
dm_get_all_fks()
includesparent_pk_cols
column that describes the primary key columns of the parent table (#335). -
dm_from_src()
supports theschema
argument also for MariaDB and MySQL databases (#516). - dm objects now inherit from
"dm_v1"
in addition to"dm"
, to allow backward-compatible changes of the internal format (#521). - Use hack to create compound primary keys on the database (#522).
-
dm_examine_constraints()
and other check functions count the number of rows that violate constraints for primary and foreign keys (#335). -
copy_dm_to(set_key_constraints = FALSE)
downgrades unique indexes to regular indexes (#335). -
rows_truncate()
implemented for data frames (#335). -
dm_enum_fk_candidates()
enumerates column in the order they apper in the table (#335).
dm 0.1.13
CRAN release: 2021-04-25
Features
-
dm_draw()
gainscolumn_types
argument, ifTRUE
the column type is shown for each displayed column (#444, @samssann). -
copy_dm_to()
gainsschema
argument (#432). -
dm_from_src()
gainsdbname
argument for MSSQL (#472).
Bug fixes
- Fix
rows_update()
when multiple columns are updated (#488, @samssann).
Performance
-
enum_fk_candidates()
now only checks distinct values, this improves performance for large tables. As a consequence, only the number of distinct values is reported for mismatches, not the number of mismatching rows/entries (#494).
Documentation
- Fix description of filtering behavior in
?dm_zoom_to
(#403).
Internal
- Move repository to https://github.com/cynkra/dm (#500).
- Enable more Postgres tests (#497).
- Test DuckDB on GitHub Actions (#498).
dm 0.1.12
CRAN release: 2021-02-15
-
dm_from_src()
gainsdbname
argument for MSSQL (#472). - Implement
count()
andtally()
for dplyr 1.0.3 compatibility (#475). - Use databases.pacha.dev instead of db-edu.pacha.dev (#478, @pachamaltese).
dm 0.1.10
CRAN release: 2021-01-07
- Columns with missing values are no longer primary keys (#469).
- Fix
dm_from_src()
for MSSQL whenlearn_keys = FALSE
(#427). - Tests use
expect_snapshot()
everywhere (#456). - Fix compatibility with testthat 3.0.1 (#457).
dm 0.1.9
CRAN release: 2020-11-18
- New
vignette("howto-dm-copy", package = "dm")
andvignette("howto-dm-rows", package = "dm")
discuss updating data on the database. In part derived fromvignette("howto-dm-db", package = "dm")
(#411, @jawond). - New
dm_mutate_tbl()
(#448). -
dm_financial()
falls back to db-edu.pacha.dev if relational.fit.cvut.cz is unavailable (#446, @pachamaltese). - Use testthat 3e (#455).
dm 0.1.7
CRAN release: 2020-09-02
- Bump RMariaDB required version to 1.0.10 to work around timeout with
R CMD check
. -
dm_from_src()
acceptsschema
argument for MSSQL databases (#367).
dm 0.1.6
CRAN release: 2020-07-29
Breaking changes
-
dm_get_src()
returnsNULL
for local data sources (#394). - Local target in
copy_dm_to()
gives a deprecation message (#395).
Features
-
copy_dm_to()
gives a better error message for badtable_names
(#397). -
dm
objects with local data sources no longer show the “Table source” part in the output. - Error messages now refer to “tables”, not “elements” (#413).
- New
dm_bind()
for binding two or more ‘dm’ objects together (#417).
Bug fixes
- For databases, the underlying SQL table names are quoted early to avoid later SQL syntax errors (#419).
-
dm_financial()
no longer prints message aboutlearn_keys = FALSE
. -
dm_rows_update()
and related functions now use the primary keys defined inx
for establishing matching rows.
Internal
- Use
withCallingHandlers()
where appropriate (#422). - Consistent definition of
.dm
and.dm_zoomed
methods (#300). - Examples involving
dm_financial()
are not run if connection can’t be established (#418). - Fix database tests on CI (#416).
dm 0.1.5
CRAN release: 2020-07-03
Features
-
dm_paste()
generates self-contained code (#401). - Errors regarding cycles in the relationship graph now show the shortest cycle (#405).
- Implement
rows_truncate()
for databases. -
collect()
works on a zoomed dm, with a message. - The data model is drawn in a more compact way if it comprises of multiple connected components.
-
dm_add_pk(check = TRUE)
gives a better error message.
Bug fixes
-
rows_insert()
works if column names consist of SQL keywords (#409). - Cycles in other connected components don’t affect filtering in a cycle-free component.
- Avoid
src_sqlite()
in examples (#372).
dm 0.1.4
CRAN release: 2020-06-07
Features
New
dm_rows_insert()
,dm_rows_update()
,dm_rows_patch()
,dm_rows_upsert()
,dm_rows_delete()
anddm_rows_truncate()
, calling the correspondingrows_*()
method for every table (#319).New
rows_truncate()
(#319).Added
rows_insert()
androws_update()
methods for SQLite, Postgres, MariaDB and MSSQL (#319).Missing arguments now give a better error message (#388).
copy_dm_to()
also accepts a function as thetable_names
argument. Theunique_table_names()
argument is deprecated (#80).
dm 0.1.2
CRAN release: 2020-05-04
Features
-
dm_from_src()
now works for databases other than Postgres and MSSQL (#288), gives a warning if tables cannot be accessed withtable_name = NULL
(#348), and gainslearn_keys
argument to control querying of primary and foreign keys from the database (#340). -
dm_examine_constraints()
now prints a different message if a dm has no constraints defined. - Disambiguation message now only lists column names for easier copy-pasting.
- New methods for
"dm_zoomed"
:head()
,tail()
,pull()
,group_data()
,group_indices()
,group_vars()
,group_keys()
andgroups()
(#236, #203). -
dm_paste()
supports writing colors and the table definition via the newoptions
argument. The definition can be written to a file via the newpath
argument. Theselect
argument is soft-deprecated (#218, #302). -
dm_add_tbl()
usesrlang::list2()
internally, now accepts:=
to specify table names. - New
dm_ptype()
(#301). - New
dm_financial()
anddm_financial_sqlite()
. - Printing dm objects from database sources with many tables is now faster (#308, @gadenbuie).
-
check_key()
now also works on a zoomed dm. - Key columns are always selected in a join operation, with a message (#153).
- Support alpha colors for the table colors (#279).
Bug fixes
- Fix visualization of column that acts as a foreign key more than once (#37).
-
dm_add_pk()
,dm_rm_pk()
,dm_add_fk()
anddm_rm_fk()
are now stricter when keys exists or when attempting to remove keys that don’t exist. A more relaxed mode of operation may be added later (#214). -
examine_cardinality()
,dm_examine_constraints()
andenum_pk_candidates()
now work for columns namedn
. -
dm_set_key_constraints()
(and by extensiondm_copy_to(set_key_constraints = TRUE)
) now quote identifiers for the SQL that creates foreign keys on the database. -
collect()
gives a better error message when called on a"dm_zoomed"
(#294). -
check_subset()
gives a clean error message if the tables are complex expressions. -
dm_from_src(schema = "...")
works on Postgres ifsearch_path
is not set on the connection. -
compute.dm_zoomed()
no longer throws an error. - Remove unused DT import (#295).
Internal
- Testing on local data frames (by default), optionally also SQLite, Postgres, RMariaDB, and SQL Server. Currently requires development versions and various pull requests (#334, #327, #312, #76).
-
dm_nycflights13(subset = TRUE)
memoizes subset and also reduces the size of theweather
table. - Expand definitions of deprecated functions (#204).
dm 0.1.1
CRAN release: 2020-03-12
- Implement
format.dm()
. - Adapt to tidyselect 1.0.0 (#257).
- Zooming and unzooming is now faster if no columns are removed.
- Table names must be unique.
-
dm_examine_constraints()
formats the problems nicely. - New class for prettier printing of keys (#244).
- Add experimental schema support for
dm_from_src()
for Postgres through the newschema
andtable_type
arguments (#256).
dm 0.1.0
- Package is now in the “maturing” lifecycle (#154).
-
filter.dm_zoomed()
no longer sets the filter. -
examine_()
functions never throw an error (#238). - API overhaul:
dm_zoom_to()
,dm_insert_zoomed()
,dm_update_zoomed()
anddm_discard_zoomed()
;check_()
->examine_()
;dm_get_filter()
->dm_get_filters()
;dm_from_src()
+dm_learn_from_db()
->dm_from_src()
(#233). - New
$.dm_zoomed()
,[.dm_zoomed()
,[[.dm_zoomed()
,length.dm_zoomed()
,names.dm_zoomed()
,tbl_vars.dm_zoomed()
(#199, #216). - New
as.list()
methods (#213). - Help pages for dplyr methods (#209).
- New migration guide from dm <= 0.0.5 (#234).
- New {tidyselect} interface for setting colors (#162) and support for hex color codes as well as R standard colors.
- Prepare
dm_examine_constraints()
and other key-related functions for compound keys (#239). - Avoid warnings in
R CMD check
with dev versions of dependencies. - Improve error messages for missing tables (#220).
dm 0.0.6
- Change
cdm_
prefix todm_
. The old names are still available (#117). - New
pull_tbl()
extracts a single table from adm
(#206). - New
dm_apply_filters_to_tbl()
that applies filters in related tables to a table, similar todm_apply_filters()
;tbl()
,$
and[[
no longer apply filter conditions defined in related tables (#161). - New
dm_paste()
(#160). - New
check_cardinality()
returns the nature of the relationship betweenparent_table$pk_col
andchild_table$fk_col
(#15). - New zoom vignette (#171).
-
check_key()
no longer maps empty selection list to all columns. -
check_key()
supports tidyselect (#188). -
dm_rm_tbl()
supports tidyselect (#127). -
decompose_table()
uses tidyselect (#194). - Implement
copy_to()
fordm
objects (#129). - Relax test for cycles in relationship graph (#198).
- Return
ref_table
column indm_check_constraints()
(#178). -
str()
shows simplified views (#123). - Edits to README (#172, @bbecane).
- Extend
validate_dm()
(#173). - Fix zooming into table that uses an FK column as primary key (#193).
- Fix corner case in
dm_rm_fk()
(#175). - More efficient
check_key()
for databases (#208). - Testing for R >= 3.3 and for debug versions.
- Remove {stringr} dependency (#183).
dm 0.0.5
Features
-
cdm_filter()
andfilter.dm_zoomed()
apply the filter instantly, the expression is recorded only for display purposes and for terminating the search for filtered tables incdm_apply_filters()
. This now allows using a variety of operations on filtereddm
objects (#124). -
dimnames()
,colnames()
,dim()
,distinct()
,arrange()
,slice()
,separate()
andunite()
implemented for zoomed dm-s (#130). - Joins on zoomed dm objects now supported (#121). Joins use the same column name disambiguation algorithm as
cdm_flatten_to_tbl()
(#147). -
slice.dm_zoomed()
: user decides in arg.keep_pk
if PK column is tracked or not (#152). - Supported {dplyr} and {tidyr} verbs are reexported.
-
enum_pk_candidates()
works with zoomed dm-s (#156). - New
enum_fk_candidates()
(#156). - Add name repair argument for both
cdm_insert_zoomed_tbl()
andcdm_add_tbl()
, defaulting to renaming of old and new tables when adding tables with duplicate names (#132). - Redesign constructors and validators:
dm()
is akin totibble()
,dm_from_src()
works likedm()
did previously,new_dm()
only accepts a list of tables and no longer validates,validate_dm()
checks internal consistency (#69). -
compute.dm()
applies filters and callscompute()
on all tables (#135).
Documentation
- New demo.
- Add explanation for empty
dm
(#100).
Bug fixes
- Avoid asterisk when printing local
dm_zoomed
(#131). -
cdm_select_tbl()
works again when multiple foreign keys are defined between two tables (#122).
dm 0.0.4
- Many {dplyr} verbs now work on tables in a
dm
. Zooming to a table viecdm_zoom_to_tbl()
creates a zoomeddm
on which the {dplyr} verbs can be applied. The resulting table can be put back into thedm
withcdm_update_zoomed_tbl()
(overwriting the original table) orcdm_insert_zoomed_tbl()
(creating a new table), respectively (#89). -
cdm_select_to_tbl()
removes foreign key constraints if the corresponding columns are removed. - Integrate code from {datamodelr} in this package (@bergant, #111).
- Reorder tables in
"dm"
usingcdm_select_tbl()
(#108). - More accurate documentation of filtering operation (#98).
- Support empty
dm
objects viadm()
andnew_dm()
(#96). -
cdm_flatten_to_tbl()
now flattens all immediate neighbors by default (#95). - New
cdm_add_tbl()
andcdm_rm_tbl()
(#90). - New
cdm_get_con()
(#84). - A
dm
object is defined using a nested tibble, one row per table (#57).
dm 0.0.3
-
cdm_enum_pk_candidates()
andcdm_enum_fk_candidates()
both show candidates first (#85). -
cdm_flatten_to_tbl()
works only in the immediate neighborhood (#75). - New
cdm_squash_to_tbl()
implements recursive flattening for left, inner and full join (#75). - Updated readme and introduction vignette (#72, @cutterkom).
- New
cdm_check_constraints()
to check referential integrity of adm
(#56). -
cdm_copy_to()
gainstable_names
argument (#79). -
check_key()
now deals correctly with named column lists (#83). - Improve error message when calling
cdm_add_pk()
with a missing column.
dm 0.0.2.9002
- Use caching to improve loading times.
- Run some tests only for one source (#76).
-
cdm_enum_fk_candidates()
checks for class compatibility implicitly vialeft_join()
. -
cdm_enum_fk_candidates()
contains a more detailed entry in column why if no error & no candidate (percentage of mismatched vals etc.). - Improve error messages for
cdm_join_to_tbl()
andcdm_flatten_to_tbl()
in the presence of cycles or disconnected tables (#74).
dm 0.0.2.9001
- Remove the
src
component from dm (#38). - Internal: Add function checking if all tables have same src.
- Internal: Add 2 classed errors.
-
cdm_get_src()
for local dm always returns a src based on.GlobalEnv
. -
cdm_flatten()
gains...
argument to specify which tables to include. Currently, all tables must form a connected subtree rooted atstart
. Disambiguation of column names now happens after selecting relevant tables. The resulting SQL query is more efficient for inner and outer joins if filtering is applied. Flattening with aright_join
with more than two tables is not well-defined and gives an error (#62). - Add a vignette for joining functions (#60, @cutterkom).
- Shorten message in
cdm_disambiguate_cols()
.
dm 0.0.2.9000
-
cdm_flatten_to_tbl()
disambiguates only the necessary columns. - When flattening, the column name of the LHS (child) table is used (#52).
- Fix formatting in
enum_pk_candidates()
for character data. -
cdm_add_pk()
andcdm_add_fk()
no longer check data integrity by default. - Explicitly checking that the
join
argument is a function, to avoid surprises when the caller passes data. -
cdm_copy_to()
works correctly with filtereddm
objects. -
cdm_apply_filters()
actually resets the filter conditions. - A more detailed README file and a vignette for filtering (#29, @cutterkom).
-
cdm_draw()
no longer supports thetable_names
argument, usecdm_select_tbl()
. - Copying a
dm
to a database now creates indexes for all primary and foreign keys.
dm 0.0.2
Breaking changes
- Requires tidyr >= 1.0.0.
-
cdm_nrow()
returns named list (#49). - Remove
cdm_semi_join()
. - Remove
cdm_find_conn_tbls()
and theall_connected
argument tocdm_select()
(#35). - Unexport
cdm_set_key_constraints()
. - Rename
cdm_select()
tocdm_select_tbl()
, now uses {tidyselect}. -
cdm_nycflights13()
now hascycle = FALSE
as default. - Rename
cdm_check_for_*()
tocdm_enum_*()
.
Performance
-
cdm_filter()
only records the filtering operation, the filter is applied only when querying a table viatbl()
or when callingcompute()
or the newcdm_apply_filters()
(#32).
New functions
- New
cdm_flatten_to_tbl()
flattens adm
to a wide table with starting from a specified table (#13). Renamecdm_join_tbl()
tocdm_join_to_tbl()
. - New
cdm_disambiguate_cols()
(#40). - New
cdm_rename()
(#41) andcdm_select()
(#50) for renaming and selecting columns ofdm
tables. - New
length.dm()
andlength<-.dm()
(#53). -
$
,[[
,[
,names()
,str()
andlength()
now implemented for dm objects (read-only). - New
enum_pk_candidates()
.
Minor changes
-
browse_docs()
opens the pkgdown website (#36). -
as_dm()
now also accepts a list of remote tables (#30). - Use {tidyselect} syntax for
cdm_rename_tbl()
andcdm_select_tbl()
(#14). - The tibbles returned by
cdm_enum_fk_candidates()
andcdm_enum_pk_candidates()
contain awhy
column that explains the reasons for rejection in a human-readable form (#12). - Improve compatibility with RPostgres.
-
create_graph_from_dm()
no longer fails in the presence of cycles (#10). - Only suggest {RSQLite}.
-
cdm_filter()
no longer requires a primary key. -
decompose_table()
adds the new column in the table to the end. -
tbl()
now fails if the table is not part of the data model.
Documentation
- Add setup article (#7).
Internal
- Using simpler internal data structure to store primary and foreign key relations (#26).
- New
nse_function()
replacesh()
for marking functions as NSE to avoid R CMD check warnings. - Simplified internal data structure so that creation of new operations that update a dm becomes easier.
- When copying a dm to a database,
NOT NULL
constraints are set at creation of the table. This removes the necessity to store column types. - Using {RPostgres} instead of {RPostgreSQL} for testing.
dm 0.0.1
Initial GitHub release.
Creating dm
objects and basic functions:
dm()
new_dm()
validate_dm()
cdm_get_src()
cdm_get_tables()
cdm_get_data_model()
is_dm()
as_dm()
Primary keys
cdm_add_pk()
cdm_has_pk()
cdm_get_pk()
cdm_get_all_pks()
cdm_rm_pk()
cdm_check_for_pk_candidates()
Foreign keys
cdm_add_fk()
cdm_has_fk()
cdm_get_fk()
cdm_get_all_fks()
cdm_rm_fk()
cdm_check_for_fk_candidates()
Filtering
cdm_filter()
cdm_semi_join()
cdm_nrow()
Interaction with DBs
cdm_copy_to()
cdm_set_key_constraints()
cdm_learn_from_db()
Utilizing foreign key relations
cdm_is_referenced()
cdm_get_referencing_tables()
cdm_select()
cdm_find_conn_tbls()
Miscellaneous
cdm_nycflights13()
cdm_rename_table()
cdm_rename_tables()