Groupware by pbleser-oc · Pull Request #1994 · opencloud-eu/opencloud

pbleser-oc · 2025-12-09T08:30:50Z

Description

Ongoing implementation of the Groupware backend service, which initially happened on its own branch but should now occur on the main branch.

Changes are almost exclusively on packages that are distinct to the Groupware backend, namely

pkg/jmap, pkg/jscalendar and pkg/jscontact: contain the implementation of the JMAP protocol and data models for Core, Mail, Contacts, Calendars, Blobs, ...
services/groupware: contains the Groupware backend service which is currently configured to not be started by default, and must be explicitly included in the START_ADDITIONAL_SERVICES property

Changes to common areas include:

devtools/deployments/opencloud_full: addition of a container of the Stalwart mail server which is used for Groupware functionality
pkg/structs: add more helper functions that are then used in pkg/jmap and services/groupware

Note that it is not fully functional yet and is going to be under continued and ongoing development along with its UI counterparts.

Specifically, the following aspects are lacking and only implemented in a skeletal way as a proof of concept:

API documentation: originally developed using go-swagger and redocly-cli, but which is having severe limitations and might be superseded by an ongoing implementation of a custom source code parser which renders OpenAPI
unit tests: very little coverage if at all, which is mostly due to most of the implementation of the Groupware backend doing relatively little logic and is mostly implementing the JMAP model
integration tests: some coverage exists that performs tests against a Stalwart container using testcontainers, but will be made more exhaustive
k6 black box tests: none implemented yet
system documentation: none implemented yet, too early to do so

The Groupware backend is not meant to be used productively yet.

butonic · 2025-12-10T16:06:38Z

+curl -ks -D- -X POST \
+"https://keycloak.opencloud.test/realms/openCloud/protocol/openid-connect/token" \
+-d username=alan -d password=demo -d grant_type=password \
+-d client_id=groupware -d scope=openid


when working from top to bottom this fails. using a client_id=web works ... so I need to add a client id for the goupware somewhere ...

I did add a JSON file devtools/deployments/opencloud_full/config/keycloak/clients/groupware.json, expecting it to be automatically picked up, but that might require re-creating the Keycloak container... ?
In any case, that was a beginner's mistake, there is really no reason to have a distinct client ID for the groupware, as the Web UI is going to use web instead.

Removed that config file and updated the documentation accordingly to use web instead, in cfbbe02

butonic · 2025-12-10T16:08:15Z

+to which one should receive the following response:
+
+```java
+A OK [CAPABILITY IMAP4rev2 ...] Authentication successful


hm, I need to dig into:

stalwart-1 | 2025-12-10T16:07:05Z TRACE Raw IMAP input received (imap.raw-input) listenerId = "imaptls", localPort = 993, remoteIp = 172.39.0.1, remotePort = 45034, size = 19, contents = "A LOGIN alan demo\r\n" stalwart-1 | 2025-12-10T16:07:05Z ERROR LDAP error (store.ldap-error) listenerId = "imaptls", localPort = 993, remoteIp = 172.39.0.1, remotePort = 45034, reason = "I/O error: Connection refused (os error 111)", causedBy = "crates/directory/src/core/dispatch.rs:25", id = "A"

butonic · 2025-12-11T13:53:56Z

@@ -0,0 +1,49 @@
+package jmap


I think we should drop the jmap_ prefix from all files in this package

butonic · 2025-12-11T14:46:38Z

+      AUTH_BASIC_LDAP_BIND_PASSWORD: "admin"
+      USERS_LDAP_BIND_PASSWORD: "admin"
+      GROUPS_LDAP_BIND_PASSWORD: "admin"
+      IDM_LDAPS_ADDR: 0.0.0.0:9235


I configured everything on .compose.test to prevent collisions with my .opencloud.test deployment. The stalwart URL then needs to be set for the groupware:

Suggested change

IDM_LDAPS_ADDR: 0.0.0.0:9235

IDM_LDAPS_ADDR: 0.0.0.0:9235

GROUPWARE_JMAP_BASE_URL: https://${STALWART_DOMAIN:-stalwart.opencloud.test}

oh and we need to set FRONTEND_GROUPWARE_ENABLED: "true" and enable the mail app in the web config when stalwart is enabled ... but that connot be configured with a simple env var ... 😞 we need to replace the config file then ...

butonic

Ok, I still don't have a menu entry for mail in the web UI even though the config.json contains the mail app. I assume that is a problem with my setup/config. in any case, I can see the mail UI when navigation to {$OC_URL}/mail manually. \o/

I did run into some minor issues when following the docs. If you could address them ~~(and enlighten me on how to get menu to show the groupware apps)~~ I'm happy to merge this. I mostly want others to be able follow the DEVELOPMENT.md and have a working setup. Kudos for that, btw.

The groupware service itself follows our ... boilerplate ... service code ... and implements the JMAP handling. I assume that will have to evolve, but we can merge it and iterate.

Tip

the web repo has a compose file with all the apps enabled. That gave me the final hint to get the menu entries

* add Groupware APIs for creating and deleting addressbooks * add Groupware APIs for creating and deleting calendars * add JMAP APIs for creating and deleting addressbooks, calendars * add JMAP APIs to retrieve Principals * fix API tagging * move addressbook JMAP APIs into its own file * move addressbook Groupware APIs into its own file

…hods and more intelligence to the various request and response types to improve the use of template functions, reducing the risks of typos and copy/paste mistakes

* move ContactCard from jscontact to jmap, as it is actually a JMAP specification item, but also was causing too many issues with circular references from jscontact -> jmap * introduce Foo, Idable, GetRequest, GetResponse, etc... types and generics * first attempt at a Foo factory type for Mailboxes, needs to be expanded to further minimize repetition * add more specialized template functions to avoid repetition * introduce ChangesTemplate[T] for *Changes structs

* introduce jmap.Context to hold multiple parameters and shorten function calls * introduce SearchResultsTemplate

* make the JMAP internal API a bit more future-proof by passing ObjectType objects instead of the JMAP namespaces * remove the new attempt to contain operations even further using the Factory objects * move CalendarEvent operations to its own file, like everything else * fix email tests * ignore WS error when closing an already closed connection

* jmap: add UpdateContactCard and UpdateCalendarEvent funcs * use JSON marshalling and unmarshalling into a map for toPatch() implementations * add updating ContactCards and CalendarEvents to tests * add deleting ContactCards and CalendarEvents to tests * make query response totals work when the value is 0 * tests: use CreateContactCard and CreateCalendarEvent funcs to create objects in tests instead of using a different JMAP stack that works with untyped maps

…events

* adds creating addressbooks, calendars, mailboxes * adds deleting mailbox, event, identity * adds modifying an email * introduce template functions for the Groupware API in templates.go, and use those in route function implementations whenever possible * add capability checking for mail, quota, blobs * adds Changes interface * adds JmapResponse interface

…use of template functions

…AP specification

* JMAP query limit of 0 is synonymous with "no limit", but we actually want to be able to perform queries without any results, for cases where we only want to count the total number of objects, and also because it makes more sense semantically * introduce query parameter validation checks, in order to only allow query parameters that are actually supported, which is going to be useful during development of clients

* add query parameters 'anchor' and 'offset': - anchor is an object identifier - offset is a numeric offset relative to the anchor

* the "position" attribute is always set to 0 when using an anchor for pagination, but it does not accurately reflect the position and thus we decided to remove the attribute from the resulting object for clarity * omit "canCalculateChanges" attribute when its value is "true", which is going to be the case in 100% of calls against Stalwart

* in the JMAP API as well as in several places in the Groupware framework, use a single jmap.Response[T] object to return the payload, the language, the session state and the etag/state instead of individual multi-valued return values

…suppliers * refactor APIs in JMAP and Groupware in order to implement pagination across multiple accountIds and multiple suppliers (currently implemented using a mock supplier for contacts) * requires go 1.26 due to use of self-reflecting generics type constraints * still missing: query criteria and sorting parameters * still missing: multi-accountId support for emails * errors are now all just 'error' in the APIs, instead of the specialized implementations, and are interpreted dynamically where necessary in order to transform them into HTTP responses * remove position, anchor, anchorOffset as individual query parameters as we now only support a 'next=...' token for subsequent pages (except in emails for now), and use jmap.QueryParams instead; those tokens have a header character for the format, followed by a JSON encoded QueryParams map, all wrapped into base62 to make it clearer that it is meant to be an opaque token, and not a parameter clients should tinker with or construct themselves * introduce QueryParamsSupplier as an interface to provide QueryParams for various scenarios (single supplier, multiple supplier, ...) per accountId * implement multi-supplier template methods slist and squery

* remove the following methods on the jmap.QueryCommand interface, as they ended up not being needed: - GetPosition() - GetAnchor() - GetAnchorOffset() - GetLimit()

…SupplierId Purpose is to make APIs and parameters easier to understand, since plain strings are used all over the place for all sorts of identifiers.

* add new configuration setting GROUPWARE_SEND_DURATIONS_RESPONSE (defaults to false) * keep track of lists of durations of backend calls * when enabled, report them as response headers Durations (human readable) and Durations-Nanos (as raw nanosecond values for machine consumption)

pbleser-oc self-assigned this Dec 9, 2025

pbleser-oc added Status:Needs-Review Type:Organizational Type:Maintenance E.g. technical debt, packaging, etc. and removed Type:Organizational labels Dec 9, 2025

pbleser-oc requested a review from butonic December 10, 2025 08:32

pbleser-oc marked this pull request as ready for review December 10, 2025 08:32

butonic reviewed Dec 10, 2025

View reviewed changes

Comment thread services/auth-api/pkg/service/http/v0/service.go Outdated

butonic reviewed Dec 10, 2025

View reviewed changes

Comment thread services/groupware/DEVELOPER.md Outdated

butonic reviewed Dec 10, 2025

View reviewed changes

butonic reviewed Dec 11, 2025

View reviewed changes

Comment thread devtools/deployments/opencloud_full/.env

butonic reviewed Dec 11, 2025

View reviewed changes

Comment thread services/groupware/DEVELOPER.md

butonic reviewed Dec 11, 2025

View reviewed changes

Comment thread services/groupware/pkg/groupware/api.go

butonic reviewed Dec 11, 2025

View reviewed changes

butonic requested changes Dec 11, 2025

View reviewed changes

pbleser-oc force-pushed the groupware branch from a9ef8f3 to 3d8cad1 Compare December 17, 2025 08:34

pbleser-oc force-pushed the groupware branch 2 times, most recently from 284aa10 to 0fce463 Compare January 22, 2026 08:47

pbleser-oc force-pushed the groupware branch 2 times, most recently from fed43d7 to aaa0140 Compare February 10, 2026 16:04

pbleser-oc force-pushed the groupware branch from aaa0140 to 4909efd Compare February 11, 2026 10:10

pbleser-oc force-pushed the groupware branch 2 times, most recently from c415019 to 31458e3 Compare February 27, 2026 13:46

pbleser-oc force-pushed the groupware branch from 70cfec5 to 9f95acb Compare March 6, 2026 08:37

pbleser-oc force-pushed the groupware branch from 0a06a6f to 58c4af5 Compare March 24, 2026 09:09

pbleser-oc force-pushed the groupware branch from 186022a to f144a7c Compare April 3, 2026 13:43

pbleser-oc force-pushed the groupware branch 2 times, most recently from 18d4522 to 911f8ad Compare April 20, 2026 08:35

pbleser-oc added 28 commits June 3, 2026 18:39

groupware: add changes support for quotas, identities, submissions

90e929c

groupware: bump @redocly/cli 2.24.0 -> 2.25.2

adfb2c5

groupware: add GetIdentityChanges

744a262

groupware: significant refactorings of the JMAP framework, adding met…

c215de9

…hods and more intelligence to the various request and response types to improve the use of template functions, reducing the risks of typos and copy/paste mistakes

groupware: refactor for conciseness

9f4f7fb

* introduce jmap.Context to hold multiple parameters and shorten function calls * introduce SearchResultsTemplate

groupware: add endpoints to patch addressbooks, contacts, calendars, …

870149a

…events

structs: add tests and documentation

88a48e6

groupware: documentation changes after API generator updates for the …

efb1ce4

…use of template functions

groupware: fixes after rebasing and Docker library API changes

0b08a26

groupware: fix getall template result type: []T instead of RESP

cc7008a

groupware: rename 'offset' to 'position' to be consistent with the JM…

3beb222

…AP specification

groupware: add documentation for SearchResultsTemplate + update pnpm

2ddda02

groupware: add support for anchor and anchor offset pagination

a3bbeb4

* add query parameters 'anchor' and 'offset': - anchor is an object identifier - offset is a numeric offset relative to the anchor

groupware: fix test logic error in the JMAP ws integration test

9e00005

groupware: API documentation updates

86d0760

groupware: go mod tidying after rebasing

32ed5fd

groupware: remove unneeded jmap.QueryCommand methods

ffa3e31

* remove the following methods on the jmap.QueryCommand interface, as they ended up not being needed: - GetPosition() - GetAnchor() - GetAnchorOffset() - GetLimit()

groupware: add strongly typed aliases for AccountId, PrincipalId and …

c97da79

…SupplierId Purpose is to make APIs and parameters easier to understand, since plain strings are used all over the place for all sorts of identifiers.

pbleser-oc force-pushed the groupware branch from 1172e2e to a42bcbf Compare June 3, 2026 16:39

groupware: fix unit tests

edd4bb8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Groupware#1994

Groupware#1994
pbleser-oc wants to merge 244 commits into
mainfrom
groupware

pbleser-oc commented Dec 9, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

butonic Dec 10, 2025

Uh oh!

pbleser-oc Feb 5, 2026

Uh oh!

butonic Dec 10, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

butonic Dec 11, 2025

Uh oh!

butonic Dec 11, 2025 •

edited

Loading

Uh oh!

butonic Dec 11, 2025

Uh oh!

butonic left a comment •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	IDM_LDAPS_ADDR: 0.0.0.0:9235
	IDM_LDAPS_ADDR: 0.0.0.0:9235
	GROUPWARE_JMAP_BASE_URL: https://${STALWART_DOMAIN:-stalwart.opencloud.test}

Conversation

pbleser-oc commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

Uh oh!

Uh oh!

butonic Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

pbleser-oc Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

butonic Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

butonic Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

butonic Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

butonic Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

butonic left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pbleser-oc commented Dec 9, 2025 •

edited

Loading

butonic Dec 11, 2025 •

edited

Loading

butonic left a comment •

edited

Loading