MDEV-30124: Add format validation for JSON Schema by varundeepsaini · Pull Request #4750 · MariaDB/server

varundeepsaini · 2026-03-07T05:21:56Z

Summary

Implements optional format validation for the JSON_SCHEMA_VALID() function per JSON Schema Draft 2020-12.

Adds a new session variable json_schema_format_validation (default OFF)
When OFF, the format keyword is treated as annotation only (existing behavior)
When ON, validates strings against 18 format types: date-time, date, time, duration, email, idn-email, hostname, idn-hostname, ipv4, ipv6, uri, uri-reference, iri, iri-reference, uuid, json-pointer, relative-json-pointer, regex
Unknown format values always pass validation

Test plan

Existing format annotation tests continue to pass unchanged
New tests cover all 18 formats with valid/invalid inputs when json_schema_format_validation=ON
Verified non-string types always pass regardless of format
Verified unknown format values are treated as annotation

gkodinov

Thank you for your contribution! This is a preliminary review.

In general I would strive to reuse as much of the existing infrastructure in the server as possible: charset handling, parsers for various data types, 3d party libraries etc.

The declarations of the formats are far from simple: some have quoting, some escaping etc.

I've jotted down some of the issues that I find at a first glance. It looks like this can benefit from some sort of standardized testing of the parser too. I'm sure test sets exist for most of these parsers.

CLAassistant · 2026-03-30T19:30:54Z

All committers have signed the CLA.

vuvova · 2026-04-21T09:55:25Z

this is GSoC project, as far as I remember, let's change it to draft until at least the coding period starts

varundeepsaini · 2026-04-21T10:27:33Z

@vuvova
yeah my bad, this PR was raised before I drafted the proposal and got to know about MDEV-30219 (the broader ticket)

vuvova · 2026-04-21T11:22:45Z

@varundeepsaini no problem, you can still work on this draft if you'd like. I just don't think we can promise anything until we know the status of your GSoC proposal, which depends on Google's decision.

gkodinov

One implementation question below.
FWIW, for more complex tasks as the one below (involving data structures etc), I'd always start with a quick design sketch: what new data structures are added, what is being done at each pass: validation, parsing etc.
Helps a lot understand the big picture and be more effective in self-review.

varundeepsaini · 2026-07-23T05:53:41Z

A quick overview

New session var json_schema_format_validation (default OFF). ON = validate format keyword (date, email, uuid, ipv4/6, regex, etc, 17 types) per Draft 2020-12. OFF = format stays annotation-only, old behavior.

Data structures:

json_schema_format_type enum, Json_schema_format class (subclass of Json_schema_keyword)
json_schema_format_hash: name->validator lookup, built once at startup (mirrors existing all_keywords_hash)
validator resolved once at parse time, validate() calls it direct (no switch, per reviewer ask)
regex format caches last value+result, skips pcre2_compile on repeat

Two-phase flow (same as every other keyword):

handle_keyword() = parse time, once per query
validate() = per document/row

Validators: ipv4/ipv6 via inet_pton, regex via pcre2, rest (date/email/uri/hostname/uuid/json-pointer etc) hand-rolled RFC grammar checks. 15 hand-written parsers.

gkodinov

Looking much better with the new hash! Not sure why you couldn't use subclasses and stay within the original keyword hash. But OK: no harm no faul. Some further comments below.

gkodinov · 2026-07-23T10:59:05Z

 static HASH all_keywords_hash;
+static HASH json_schema_format_hash;
+
+/* Needed by json_schema_format_array below. */


all of these can be private instance methods.

gkodinov · 2026-07-23T11:00:19Z

+static bool validate_format_regex(Json_schema_format *self,
+                                  const char *val, int len);
+
+struct st_json_schema_format_map


these two can be local variables into setup_json_schema_format_hash

Moved the map into setup_format_hash(), but kept it a static local
the hash stores pointers into the array rather than copies, so a plain local would leave dangling pointers after setup returns

varundeepsaini · 2026-07-29T14:44:14Z

@gkodinov tried removing all overkill parts of the patch

why you couldn't use subclasses and stay within the original keyword hash

The keyword hash maps keyword names, and the 18 format names live in the keyword's value
so 18 subclasses would still need their own name lookup somewhere.
A separate small hash felt like the smaller change

JSON_SCHEMA_VALID() now validates the format keyword per JSON Schema Draft 2020-12 when the new session variable json_schema_format_validation is ON. By default it stays an annotation, matching the previous behaviour. The 18 Draft 2020-12 formats are supported: date-time, date, time, duration, email, idn-email, hostname, idn-hostname, ipv4, ipv6, uri, uri-reference, iri, iri-reference, uuid, json-pointer, relative-json-pointer, regex. Each format has a small syntactic validator, a private method of Json_schema_format. IP addresses are parsed with inet_pton, regex with pcre2_compile, and the rest with hand-written checks that follow the relevant RFC grammar. The format name is resolved through Json_schema_format::format_hash, built once at server startup (mirrors the existing all_keywords_hash used for the other schema keywords). Each hash entry carries a member pointer to the format's validator, set on Json_schema_format at schema parse time, so validate() dispatches with a single indirect call per document. An unknown format name raises ER_JSON_INVALID_VALUE_FOR_KEYWORD at schema parse time when json_schema_format_validation is ON (2020-12 7.2.2: assertion mode must refuse unknown formats). When OFF it stays a legal annotation (7.2.1), preserving the previous behaviour.

varundeepsaini marked this pull request as draft March 7, 2026 05:22

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch 3 times, most recently from 293b984 to e5142f5 Compare March 7, 2026 08:22

varundeepsaini marked this pull request as ready for review March 7, 2026 09:34

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch 2 times, most recently from 5a6ab32 to 602a323 Compare March 7, 2026 16:45

gkodinov added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Mar 9, 2026

gkodinov requested changes Mar 9, 2026

View reviewed changes

Comment thread sql/json_schema.cc Outdated

Comment thread sql/json_schema.cc Outdated

Comment thread sql/json_schema.cc Outdated

Comment thread sql/json_schema.cc

Comment thread sql/json_schema.cc

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch from 602a323 to 1524d38 Compare April 1, 2026 11:50

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch from 1524d38 to b88e9c4 Compare April 21, 2026 09:53

vuvova marked this pull request as draft April 21, 2026 09:54

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch from b88e9c4 to ceef2ab Compare April 21, 2026 10:29

varundeepsaini requested a review from gkodinov July 19, 2026 17:59

varundeepsaini marked this pull request as ready for review July 19, 2026 17:59

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch from ceef2ab to c2c0068 Compare July 19, 2026 18:10

gkodinov requested changes Jul 20, 2026

View reviewed changes

Comment thread sql/json_schema.cc Outdated

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch 2 times, most recently from 4d6e1a0 to 7e68eaa Compare July 23, 2026 05:58

varundeepsaini requested a review from gkodinov July 23, 2026 05:59

gkodinov requested changes Jul 23, 2026

View reviewed changes

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch from 7e68eaa to 5163d5e Compare July 29, 2026 14:43

varundeepsaini requested a review from gkodinov July 29, 2026 14:44

varundeepsaini force-pushed the MDEV-30124-json-schema-format-validation branch from 5163d5e to cd929bf Compare July 29, 2026 14:53

Uh oh!

Uh oh!

Conversation

varundeepsaini commented Mar 7, 2026

Summary

Test plan

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vuvova commented Apr 21, 2026

Uh oh!

varundeepsaini commented Apr 21, 2026

Uh oh!

vuvova commented Apr 21, 2026

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

varundeepsaini commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gkodinov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gkodinov Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gkodinov Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

varundeepsaini Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

varundeepsaini commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

CLAassistant commented Mar 30, 2026 •

edited

Loading

varundeepsaini commented Jul 23, 2026 •

edited

Loading

varundeepsaini commented Jul 29, 2026 •

edited

Loading