Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
195e4b5
Merge pull request #347 from datashield/v6.3.1-RC2-dev
StuartWheater Sep 30, 2024
a4cf3a7
Update to align with V6.3.5 dev changes
StuartWheater Oct 6, 2025
56b1b45
Add serverside checks for class and object existence
timcadman Oct 9, 2025
4a9df9f
add functions in helper file
timcadman Oct 9, 2025
3280cd1
ran check, added dependencies
timcadman Oct 9, 2025
cee9a81
just use glue within messages for consistency
timcadman Oct 9, 2025
29d73aa
renamed snake case to camel case to match naming convention
timcadman Oct 9, 2025
245bbba
require testthat version 3
timcadman Oct 9, 2025
01b0f0f
test: unit tests for helper files
timcadman Oct 9, 2025
4a17212
look in correct environment, ie 2 steps up
timcadman Oct 9, 2025
6771731
write unit tests for unhappy path
timcadman Oct 9, 2025
c5e961e
added template with checklist for PRs
timcadman Oct 9, 2025
cc8d8f8
Added 'pull_request_template'
StuartWheater Oct 21, 2025
aa9fb8f
Updates for context() + copyright
StuartWheater Oct 26, 2025
809f7cb
Change version
StuartWheater Oct 27, 2025
3db8b60
Merge pull request #435 from StuartWheater/v6.3.5-dev-feat/testthat-r…
StuartWheater Oct 28, 2025
1e0b07f
Add "pull request template" to .Rbuildignore
StuartWheater Oct 28, 2025
8afb219
Merge branch 'v7.0-dev-feat/performance' into v6.3.5-dev-feat/testtha…
StuartWheater Oct 28, 2025
bef1b36
Merge pull request #436 from datashield/v6.3.5-dev-feat/testthat-rework
StuartWheater Oct 28, 2025
bb70048
Merge branch 'v7.0-dev-feat/performance' into v7.0-dev-feat/performance
StuartWheater Oct 28, 2025
511b95d
Comment out 'context(...)'
StuartWheater Oct 28, 2025
b6e7cb5
Update for 'testthat' and other changes
StuartWheater Oct 28, 2025
c5ff70c
Updates due to 'testthat' changes
StuartWheater Oct 28, 2025
597b2a3
Merge pull request #437 from StuartWheater/v7.0-dev-feat/performance
StuartWheater Oct 28, 2025
004919f
Experimental dealing with 'pull_request_template' issue
StuartWheater Oct 28, 2025
8216d19
Merge branch 'v7.0-dev-feat/performance' of github.com:datashield/dsB…
StuartWheater Oct 28, 2025
e5188a0
Fix regex for pull_request_template in .Rbuildignore
StuartWheater Oct 28, 2025
d37331a
test: call function within correct environment
timcadman Oct 30, 2025
7660589
fixed pull request template in buildignore
timcadman Oct 30, 2025
4867280
added note explaining test setup
timcadman Oct 30, 2025
953a7a1
Fixed version
StuartWheater Nov 24, 2025
53eb5dc
Merge branch 'v7.0-dev' into v7.0-dev-feat/performance
timcadman Dec 9, 2025
2247567
Removed colnames functionality to keep in separate branch
timcadman Jan 8, 2026
6eaba77
Merge pull request #434 from datashield/v7.0-dev-feat/performance
timcadman Jan 9, 2026
e3e98bc
refactor colnames
timcadman Jan 9, 2026
c130e19
added reusable functions
timcadman Jan 9, 2026
490c1f8
added tests
timcadman Jan 9, 2026
4305357
redocumented package
timcadman Jan 9, 2026
57de79f
added PR template
timcadman Jan 9, 2026
40eb96a
use get instead of eval as more secure
timcadman Jan 9, 2026
27537d5
extend messaging to account for >2 classes
timcadman Jan 15, 2026
77c8e38
moved pr template to correct folder
timcadman Jan 15, 2026
6857bf4
fixed namespace which mistakenly included colnamesDS2
timcadman Jan 15, 2026
a79e87a
Merge pull request #451 from datashield/v7.0-dev-feat-colnames
timcadman Jan 20, 2026
a701315
refactor: batch 1 server functions use shared helpers
timcadman Mar 10, 2026
bbe1177
Fix server-side helpers and DATASHIELD method whitelisting
timcadman Mar 12, 2026
15394de
remove object assigned message
timcadman Mar 12, 2026
9f03b43
restore mistakenly deleted tests
timcadman Mar 12, 2026
91f997d
Revert whitespace-only changes and fix test assertions
timcadman Mar 13, 2026
4d2d618
refactor: make column extraction clearer
timcadman Mar 16, 2026
b801087
added error handling for missing columns
timcadman Mar 16, 2026
f6f6b0d
test: removed redundant checks
timcadman Mar 16, 2026
e3c1f92
revert: shouldn't have touched these files
timcadman Mar 16, 2026
c6211ae
test: updated test expectations to fit changed message
timcadman Mar 27, 2026
433af42
test: added performance profile|
timcadman Mar 27, 2026
9facea0
added PR template to buildignore
timcadman Mar 27, 2026
bd18856
feat: validate input type in .loadServersideObject
timcadman Mar 27, 2026
0c59ba9
refactor: make perf test tolerances configurable via profile
timcadman Mar 27, 2026
b3fcee9
chore: comment out deprecated context calls
timcadman Mar 27, 2026
e276da3
Merge pull request #464 from datashield/perf-batch-1-v5
timcadman Mar 31, 2026
5873240
docs: updated authorship for last PR
timcadman Apr 11, 2026
c72dc11
refactor: replace eval(parse()) with .loadServersideObject()
timcadman Mar 31, 2026
4b67c4a
refactor: dimDS/lengthDS return class for client-side consistency check
timcadman Mar 31, 2026
2c2c9f5
refactor: isNaDS/numNaDS/levelsDS accept string name via .loadServers…
timcadman Mar 31, 2026
ac3747e
test: update batch 2 tests for refactored server functions
timcadman Mar 31, 2026
48b374a
refactor: also return class for client-side consistency checks
timcadman Apr 13, 2026
fa49156
refactor: remove validity message as inconsistent with other functions
timcadman Apr 13, 2026
97e2687
docs: redocumented
timcadman Apr 13, 2026
0f5814a
docs: updated authorship
timcadman Apr 13, 2026
b3f190c
test: updated expectations now some functions return a list
timcadman Apr 13, 2026
48b6a4e
revert: delete mistakenly committed file
timcadman Apr 13, 2026
885f61f
revert: allow data frames to pass class check and update tests
timcadman Apr 13, 2026
c74488d
refactor: remove class from levelsDS return, update test
timcadman Apr 13, 2026
24d7959
Merge pull request #469 from datashield/docs/affiliation
timcadman Apr 14, 2026
d2f5b78
Merge pull request #470 from datashield/refactor/perf-batch-2
timcadman Apr 14, 2026
bb3054c
Added 'libuv1-dev' dependency
StuartWheater May 14, 2026
5f65065
Add array and matrix to lengthDS permitted classes
timcadman Jun 1, 2026
2e6b5f4
bumped roxygen to version 8
timcadman Jun 2, 2026
c56e03c
test: add density test for levels
timcadman Jun 2, 2026
d88aa00
Merge pull request #483 from datashield/fix/lengthDS-permitted-classes
timcadman Jun 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@
^\.circleci/config\.yml$
^\.github$
^cran-comments\.md$
^pull_request_template$
PULL_REQUEST_TEMPLATE.md
15 changes: 15 additions & 0 deletions .github/pull_request_template
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## Description
[Write a description of the changes you have made]

## Checklist (all items may not apply)

### Refactor
- [ ] Replace `eval(parse(text = x), envir = parent.frame())` with `.loadServersideObject()`
- [ ] Where appropriate, check object class using `.checkClass()
- [ ] Check whether additional checks are required on the server-side

### Testing
- [ ] Writen server-side unit tests for unhappy flow
- [ ] Run and passed `devtools::test(filter = "smk-|disc|arg")`
- [ ] Run and passed `devtools::check(args = '--no-tests')` (we run tests separately to skip performance checks)
- [ ] Run and passed `devtools::build`
10 changes: 6 additions & 4 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Description: Base 'DataSHIELD' functions for the server side. 'DataSHIELD' is a
been designed to only share non disclosive summary statistics, with built in automated output
checking based on statistical disclosure control. With data sites setting the threshold values for
the automated output checks. For more details, see 'citation("dsBase")'.
Version: 6.3.4
Version: 7.0.0.9000
Authors@R: c(person(given = "Paul",
family = "Burton",
role = c("aut"),
Expand Down Expand Up @@ -71,10 +71,12 @@ Imports:
gamlss,
gamlss.dist,
mice,
childsds
childsds,
glue
Suggests:
spelling,
testthat
RoxygenNote: 7.3.3
testthat (>= 3.0.0)
RoxygenNote: 8.0.0
Encoding: UTF-8
Language: en-GB
Config/testthat/edition: 3
4 changes: 4 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ export(densityGridDS)
export(dimDS)
export(dmtC2SDS)
export(elsplineDS)
export(expDS)
export(extractQuantilesDS1)
export(extractQuantilesDS2)
export(gamlssDS)
Expand Down Expand Up @@ -72,6 +73,7 @@ export(listDS)
export(listDisclosureSettingsDS)
export(lmerSLMADS.assign)
export(lmerSLMADS2)
export(logDS)
export(lsDS)
export(lsplineDS)
export(matrixDS)
Expand Down Expand Up @@ -140,3 +142,5 @@ import(gamlss.dist)
import(mice)
importFrom(gamlss.dist,pST3)
importFrom(gamlss.dist,qST3)
importFrom(glue,glue)
importFrom(glue,glue_collapse)
20 changes: 20 additions & 0 deletions PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## Instructions & checklist for PR author

### Description of changes
[Add descriptions of changes made]

### Refactor instructions
- [ ] Replaced `x <- eval(parse(text = x.name), envir = parent.frame())` with `x <- .loadServersideObject(x)`
- [ ] If necessary, check the class of the object using `.checkClass()`

### Testing instructions
- [ ] Writen server-side unit tests for unhappy flow
- [ ] Run `devtools::test(filter = "smk-|disc|arg")` and check it passes
- [ ] Run `devtools::check(args = '--no-tests')` and check it passes (we run tests separately to skip performance checks)
- [ ] Run `devtools::build()` and check it builds without errors

## Instructions & checklist for PR reviewers
- [ ] Run `devtools::test(filter = "smk-|disc|arg")` and check it passes
- [ ] Run `devtools::check(args = '--no-tests')` and check it passes (we run tests separately to skip performance checks)
- [ ] Run `devtools::build()` and check it builds without errors

7 changes: 3 additions & 4 deletions R/absDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,14 @@
#' which is written to the serverside. The output object is of class numeric
#' or integer.
#' @author Demetris Avraam for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
absDS <- function(x) {
x.var <- eval(parse(text = x), envir = parent.frame())
x.var <- .loadServersideObject(x)
.checkClass(obj = x.var, obj_name = x, permitted_classes = c("numeric", "integer"))

# compute the absolute values of x
out <- abs(x.var)

# assign the outcome to the data servers
return(out)
}
# ASSIGN FUNCTION
Expand Down
3 changes: 2 additions & 1 deletion R/asCharacterDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@
#' "ascharacter.newobj") which is written to the serverside. For further
#' details see help on the clientside function \code{ds.asCharacter}
#' @author Amadou Gaye, Paul Burton, Demetris Avraam for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
asCharacterDS <- function(x.name) {
x <- eval(parse(text = x.name), envir = parent.frame())
x <- .loadServersideObject(x.name)

output <- as.character(x)
return(output)
Expand Down
9 changes: 2 additions & 7 deletions R/asDataMatrixDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,12 @@
#' "asdatamatrix.newobj") which is written to the serverside. For further
#' details see help on the clientside function \code{ds.asDataMatrix}
#' @author Paul Burton for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
asDataMatrixDS <- function(x.name) {
if (is.character(x.name)) {
x <- eval(parse(text = x.name), envir = parent.frame())
} else {
studysideMessage <- "ERROR: x.name must be specified as a character string"
stop(studysideMessage, call. = FALSE)
}
x <- .loadServersideObject(x.name)

output <- data.matrix(x)

return(output)
}
# ASSIGN FUNCTION
Expand Down
11 changes: 2 additions & 9 deletions R/asIntegerDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,14 @@
#' "asinteger.newobj") which is written to the serverside. For further
#' details see help on the clientside function \code{ds.asInteger}.
#' @author Amadou Gaye, Paul Burton, Demetris Avraam, for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
asIntegerDS <- function(x.name){

if(is.character(x.name)){
x <- eval(parse(text=x.name), envir = parent.frame())
}else{
studysideMessage <- "ERROR: x.name must be specified as a character string"
stop(studysideMessage, call. = FALSE)
}
x <- .loadServersideObject(x.name)

output <- as.integer(as.character(x))

return(output)

}
# ASSIGN FUNCTION
# asIntegerDS
23 changes: 5 additions & 18 deletions R/asListDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -20,26 +20,13 @@
#' coerces objects to list depends on the class of the object, but in general
#' the class of the output object should usually be 'list'
#' @author Amadou Gaye, Paul Burton for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
asListDS <- function (x.name, newobj){
x <- .loadServersideObject(x.name)

newobj.class <- NULL
if(is.character(x.name)){
active.text<-paste0(newobj,"<-as.list(",x.name,")")
eval(parse(text=active.text), envir = parent.frame())

active.text2<-paste0("class(",newobj,")")
assign("newobj.class", eval(parse(text=active.text2), envir = parent.frame()))

}else{
studysideMessage<-"ERROR: x.name must be specified as a character string"
stop(studysideMessage, call. = FALSE)
}

return.message<-paste0("New object <",newobj,"> created")
object.class.text<-paste0("Class of <",newobj,"> is '",newobj.class,"'")

return(list(return.message=return.message,class.of.newobj=object.class.text))
result <- as.list(x)
assign(newobj, result, envir = parent.frame())
}
# AGGEGATE FUNCTION
# AGGREGATE FUNCTION
# asListDS
25 changes: 7 additions & 18 deletions R/asLogicalDS.R
Original file line number Diff line number Diff line change
@@ -1,32 +1,21 @@
#' @title Coerces an R object into class numeric
#' @description this function is based on the native R function \code{as.numeric}
#' @title Coerces an R object into class logical
#' @description this function is based on the native R function \code{as.logical}
#' @details See help for function \code{as.logical} in native R
#' @param x.name the name of the input object to be coerced to class
#' numeric. Must be specified in inverted commas. But this argument is
#' logical. Must be specified in inverted commas. But this argument is
#' usually specified directly by <x.name> argument of the clientside function
#' \code{ds.aslogical}
#' \code{ds.asLogical}
#' @return the object specified by the <newobj> argument (or its default name
#' <x.name>.logic) which is written to the serverside. For further
#' details see help on the clientside function \code{ds.asLogical}
#' @author Amadou Gaye, Paul Burton for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
asLogicalDS <- function (x.name){

if(is.character(x.name)){
x<-eval(parse(text=x.name), envir = parent.frame())

}else{
studysideMessage<-"ERROR: x.name must be specified as a character string"
stop(studysideMessage, call. = FALSE)
}

if(!is.numeric(x)&&!is.integer(x)&&!is.character(x)&&!is.matrix(x)){
studysideMessage<-"ERROR: for ds.asLogical function, x.name must specify an input object of class numeric, integer, character or matrix"
stop(studysideMessage, call. = FALSE)
}
x <- .loadServersideObject(x.name)
.checkClass(obj = x, obj_name = x.name, permitted_classes = c("numeric", "integer", "character", "matrix"))

output <- as.logical(x)

return(output)
}
#ASSIGN FUNCTION
Expand Down
11 changes: 2 additions & 9 deletions R/asMatrixDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,12 @@
#' <x.name>.mat) which is written to the serverside. For further
#' details see help on the clientside function \code{ds.asMatrix}
#' @author Amadou Gaye, Paul Burton for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
asMatrixDS <- function (x.name){

if(is.character(x.name)){
x<-eval(parse(text=x.name), envir = parent.frame())

}else{
studysideMessage<-"ERROR: x.name must be specified as a character string"
stop(studysideMessage, call. = FALSE)
}
x <- .loadServersideObject(x.name)

output <- as.matrix(x)

return(output)
}
#ASSIGN FUNCTION
Expand Down
10 changes: 2 additions & 8 deletions R/asNumericDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,11 @@
#' <x.name>.num) which is written to the serverside. For further
#' details see help on the clientside function \code{ds.asNumeric}.
#' @author Amadou Gaye, Paul Burton, Demetris Avraam, for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
asNumericDS <- function(x.name){

if(is.character(x.name)){
x <- eval(parse(text=x.name), envir = parent.frame())
}else{
studysideMessage <- "ERROR: x.name must be specified as a character string"
stop(studysideMessage, call. = FALSE)
}
x <- .loadServersideObject(x.name)

# Check that it doesn't match any non-number
numbers_only <- function(vec) !grepl("\\D", vec)
Expand All @@ -36,7 +31,6 @@ asNumericDS <- function(x.name){
}

return(output)

}
# ASSIGN FUNCTION
# asNumericDS
9 changes: 2 additions & 7 deletions R/classDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,13 @@
#' @param x a string character, the name of an object
#' @return the class of the input object
#' @author Stuart Wheater, for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
classDS <- function(x){

x.val <- eval(parse(text=x), envir = parent.frame())

# find the class of the input object
x.val <- .loadServersideObject(x)
out <- class(x.val)

# return the class
return(out)

}
#AGGREGATE FUNCTION
# classDS
9 changes: 2 additions & 7 deletions R/colnamesDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,10 @@
#' @export
#'
colnamesDS <- function(x){

x.val <- eval(parse(text=x), envir = parent.frame())

# find the dim of the input dataframe or matrix
x.val <- .loadServersideObject(x)
.checkClass(obj = x.val, obj_name = x, permitted_classes = c("data.frame", "matrix"))
out <- colnames(x.val)

# return the dimension
return(out)

}
#AGGREGATE FUNCTION
# colnamesDS
6 changes: 3 additions & 3 deletions R/completeCasesDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
#' without problems no studysideMessage will have been saved and ds.message("newobj")
#' will return the message: "ALL OK: there are no studysideMessage(s) on this datasource".
#' @author Paul Burton for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
completeCasesDS <- function(x1.transmit){
Expand Down Expand Up @@ -111,10 +112,9 @@ completeCasesDS <- function(x1.transmit){
}

#Activate target object
#x1.transmit is the name of a serverside data.frame, matrix or vector
x1.use <- eval(parse(text=x1.transmit), envir = parent.frame())
x1.use <- .loadServersideObject(x1.transmit)
complete.rows <- stats::complete.cases(x1.use)

if(is.matrix(x1.use) || is.data.frame(x1.use)){
output.object <- x1.use[complete.rows,]
}else if(is.atomic(x1.use) || is.factor(x1.use)){
Expand Down
16 changes: 6 additions & 10 deletions R/dimDS.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,16 @@
#' @description This function is similar to R function \code{dim}.
#' @details The function returns the dimension of the input dataframe or matrix
#' @param x a string character, the name of a dataframe or matrix
#' @return the dimension of the input object
#' @return a list with two elements: \code{dim} (the dimension of the input object)
#' and \code{class} (the class of the input object, for client-side consistency checking)
#' @author Demetris Avraam, for DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
dimDS <- function(x){

x.var <- eval(parse(text=x), envir = parent.frame())

# find the dim of the input dataframe or matrix
out <- dim(x.var)

# return the dimension
return(out)

x.val <- .loadServersideObject(x)
.checkClass(obj = x.val, obj_name = x, permitted_classes = c("data.frame", "matrix"))
list(dim = dim(x.val), class = class(x.val))
}
#AGGREGATE FUNCTION
# dimDS
22 changes: 22 additions & 0 deletions R/expDS.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#'
#' @title Computes the exponential values of the input variable
#' @description This function is similar to R function \code{exp}.
#' @details The function computes the exponential values of an input numeric
#' or integer vector.
#' @param x a string character, the name of a numeric or integer vector
#' @return the object specified by the \code{newobj} argument
#' of \code{ds.exp} (or default name \code{exp.newobj})
#' which is written to the serverside. The output object is of class numeric.
#' @author DataSHIELD Development Team
#' @author Tim Cadman, Genomics Coordination Centre, UMCG, Netherlands
#' @export
#'
expDS <- function(x) {
x.var <- .loadServersideObject(x)
.checkClass(obj = x.var, obj_name = x, permitted_classes = c("numeric", "integer"))

out <- exp(x.var)
return(out)
}
# ASSIGN FUNCTION
# expDS
Loading