sorcery/res_pjsip: Refactor for realtime performance

There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.

A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0.  One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.

This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare.  The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.

They do now.

The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator.  For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'".  If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.

The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container.  However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.

So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function.  Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex.  If the operator is like or regex, the
right string should be a %-pattern or a regex expression.  If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.

To use this new function on ast_variables, 2 new functions were added to
config.c.  One that compares 2 ast_variables, and one that compares 2
ast_variable lists.  The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list.  The latter will traverse the right list and return true if all
the variables in it match the left list.

Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines.  The realtime backend just passes
the variable list unaltered to the engine.  The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.

Only one more change to sorcery was done...  A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)

Now on to res_pjsip...

pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors.  Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.

res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.

res_pjsip_registrar_expire was completely refactored.  It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them.  A new
contact_expiration_check_interval was added to global with a default of
30 seconds.

Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.

There are still objects that can't be filtered at the database like
identifies, transports, and registrations.  These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.

Back to allow_unqualified_fetch.  If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :)  Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache.  Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts.  It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.

Example sorcery.conf:

[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error

ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer

Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67

4 files changed, 275 insertions, 67 deletions

diff --git a/tests/test_config.c b/tests/test_config.c
index 3ad57729f..40de2652b 100644
--- a/tests/test_config.c
+++ b/tests/test_config.c
@@ -1672,6 +1672,66 @@ out:
 	return res;
 }
 
+AST_TEST_DEFINE(variable_lists_match)
+{
+	RAII_VAR(struct ast_variable *, left, NULL, ast_variables_destroy);
+	RAII_VAR(struct ast_variable *, right, NULL, ast_variables_destroy);
+	struct ast_variable *var;
+
+	switch (cmd) {
+	case TEST_INIT:
+		info->name = "variable_lists_match";
+		info->category = "/main/config/";
+		info->summary = "Test ast_variable_lists_match";
+		info->description =	"Test ast_variable_lists_match";
+		return AST_TEST_NOT_RUN;
+	case TEST_EXECUTE:
+		break;
+	}
+
+	var = ast_variable_new("aaa", "111", "");
+	ast_test_validate(test, var);
+	left = var;
+	var = ast_variable_new("bbb", "222", "");
+	ast_test_validate(test, var);
+	ast_variable_list_append(&left, var);
+
+	var = ast_variable_new("aaa", "111", "");
+	ast_test_validate(test, var);
+	right = var;
+
+	ast_test_validate(test, ast_variable_lists_match(left, right, 0));
+	ast_test_validate(test, !ast_variable_lists_match(left, right, 1));
+
+	var = ast_variable_new("bbb", "222", "");
+	ast_test_validate(test, var);
+	ast_variable_list_append(&right, var);
+
+	ast_test_validate(test, ast_variable_lists_match(left, right, 0));
+	ast_test_validate(test, ast_variable_lists_match(left, right, 1));
+
+	var = ast_variable_new("ccc >", "333", "");
+	ast_test_validate(test, var);
+	ast_variable_list_append(&right, var);
+
+	ast_test_validate(test, !ast_variable_lists_match(left, right, 0));
+	ast_test_validate(test, !ast_variable_lists_match(left, right, 1));
+
+	var = ast_variable_new("ccc", "444", "");
+	ast_test_validate(test, var);
+	ast_variable_list_append(&left, var);
+
+	ast_test_validate(test, ast_variable_lists_match(left, right, 0));
+	ast_test_validate(test, !ast_variable_lists_match(left, right, 1));
+
+	ast_test_validate(test, !ast_variable_lists_match(left, NULL, 0));
+	ast_test_validate(test, ast_variable_lists_match(NULL, NULL, 0));
+	ast_test_validate(test, !ast_variable_lists_match(NULL, right, 0));
+	ast_test_validate(test, ast_variable_lists_match(left, left, 0));
+
+	return AST_TEST_PASS;
+}
+
 static int unload_module(void)
 {
 	AST_TEST_UNREGISTER(config_basic_ops);
@@ -1682,6 +1742,7 @@ static int unload_module(void)
 	AST_TEST_UNREGISTER(ast_parse_arg_test);
 	AST_TEST_UNREGISTER(config_options_test);
 	AST_TEST_UNREGISTER(config_dialplan_function);
+	AST_TEST_UNREGISTER(variable_lists_match);
 	return 0;
 }
 
@@ -1695,6 +1756,7 @@ static int load_module(void)
 	AST_TEST_REGISTER(ast_parse_arg_test);
 	AST_TEST_REGISTER(config_options_test);
 	AST_TEST_REGISTER(config_dialplan_function);
+	AST_TEST_REGISTER(variable_lists_match);
 	return AST_MODULE_LOAD_SUCCESS;
 }
 
diff --git a/tests/test_sorcery_astdb.c b/tests/test_sorcery_astdb.c
index b87ed74f8..b7566b7ae 100644
--- a/tests/test_sorcery_astdb.c
+++ b/tests/test_sorcery_astdb.c
@@ -298,7 +298,7 @@ AST_TEST_DEFINE(object_retrieve_multiple_field)
 	RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
 	RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
 	RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
-	RAII_VAR(struct ast_variable *, fields, ast_variable_new("joe", "6", ""), ast_variables_destroy);
+	RAII_VAR(struct ast_variable *, fields, ast_variable_new("joe >=", "6", ""), ast_variables_destroy);
 
 	switch (cmd) {
 	case TEST_INIT:
@@ -345,7 +345,7 @@ AST_TEST_DEFINE(object_retrieve_multiple_field)
 	ao2_cleanup(objects);
 	ast_variables_destroy(fields);
 
-	if (!(fields = ast_variable_new("joe", "7", ""))) {
+	if (!(fields = ast_variable_new("joe <", "6", ""))) {
 		ast_test_status_update(test, "Failed to create fields for multiple retrieval\n");
 		return AST_TEST_FAIL;
 	} else if (!(objects = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_MULTIPLE, fields))) {
diff --git a/tests/test_sorcery_realtime.c b/tests/test_sorcery_realtime.c
index eaee9ffbf..3ed14623e 100644
--- a/tests/test_sorcery_realtime.c
+++ b/tests/test_sorcery_realtime.c
@@ -42,60 +42,12 @@ ASTERISK_FILE_VERSION(__FILE__, "")
 /*! \brief Configuration structure which contains all stored objects */
 static struct ast_config *realtime_objects;
 
-/*! \brief Helper function which finds a given variable */
-static const struct ast_variable *realtime_find_variable(const struct ast_variable *fields, const char *name)
-{
-	const struct ast_variable *variable;
-
-	for (variable = fields; variable; variable = variable->next) {
-		if (!strcmp(variable->name, name)) {
-			return variable;
-		}
-	}
-
-	return NULL;
-}
-
-/*! \brief Helper function which returns if an object is matching or not */
-static int realtime_is_object_matching(const char *object_id, const struct ast_variable *fields)
-{
-	const struct ast_variable *field;
-
-	for (field = fields; field; field = field->next) {
-		char *name = ast_strdupa(field->name), *like;
-		const char *value;
-
-		/* If we are doing a pattern matching we need to remove the LIKE from the name */
-		if ((like = strstr(name, " LIKE"))) {
-			char *field_value = ast_strdupa(field->value);
-
-			*like = '\0';
-
-			value = ast_strdupa(ast_variable_retrieve(realtime_objects, object_id, name));
-
-			field_value = ast_strip_quoted(field_value, "%", "%");
-
-			if (strncmp(value, field_value, strlen(field_value))) {
-				return 0;
-			}
-		} else {
-			value = ast_variable_retrieve(realtime_objects, object_id, name);
-
-			if (ast_strlen_zero(value) || strcmp(value, field->value)) {
-				return 0;
-			}
-		}
-	}
-
-	return 1;
-}
-
 static struct ast_variable *realtime_sorcery(const char *database, const char *table, const struct ast_variable *fields)
 {
 	char *object_id = NULL;
 
 	while ((object_id = ast_category_browse(realtime_objects, object_id))) {
-		if (!realtime_is_object_matching(object_id, fields)) {
+		if (!ast_variable_lists_match(ast_category_root(realtime_objects, object_id), fields, 0)) {
 			continue;
 		}
 
@@ -116,8 +68,9 @@ static struct ast_config *realtime_sorcery_multi(const char *database, const cha
 
 	while ((object_id = ast_category_browse(realtime_objects, object_id))) {
 		struct ast_category *object;
+		const struct ast_variable *object_fields = ast_category_root(realtime_objects, object_id);
 
-		if (!realtime_is_object_matching(object_id, fields)) {
+		if (!ast_variable_lists_match(object_fields, fields, 0)) {
 			continue;
 		}
 
@@ -154,7 +107,7 @@ static int realtime_sorcery_update(const char *database, const char *table, cons
 static int realtime_sorcery_store(const char *database, const char *table, const struct ast_variable *fields)
 {
 	/* The key field is explicit within res_sorcery_realtime */
-	const struct ast_variable *keyfield = realtime_find_variable(fields, "id");
+	const struct ast_variable *keyfield = ast_variable_find_variable_in_list(fields, "id");
 	struct ast_category *object;
 
 	if (!keyfield || ast_category_exist(realtime_objects, keyfield->value, NULL) || !(object = ast_category_new(keyfield->value, "", 0))) {
@@ -201,7 +154,7 @@ static void *test_sorcery_object_alloc(const char *id)
 	return ast_sorcery_generic_alloc(sizeof(struct test_sorcery_object), NULL);
 }
 
-static struct ast_sorcery *alloc_and_initialize_sorcery(void)
+static struct ast_sorcery *alloc_and_initialize_sorcery(char *table)
 {
 	struct ast_sorcery *sorcery;
 
@@ -209,7 +162,7 @@ static struct ast_sorcery *alloc_and_initialize_sorcery(void)
 		return NULL;
 	}
 
-	if ((ast_sorcery_apply_default(sorcery, "test", "realtime", "sorcery_realtime_test") != AST_SORCERY_APPLY_SUCCESS) ||
+	if ((ast_sorcery_apply_default(sorcery, "test", "realtime", table) != AST_SORCERY_APPLY_SUCCESS) ||
 		ast_sorcery_internal_object_register(sorcery, "test", test_sorcery_object_alloc, NULL, NULL) ||
 		!(realtime_objects = ast_config_new())) {
 		ast_sorcery_unref(sorcery);
@@ -246,7 +199,7 @@ AST_TEST_DEFINE(object_create)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -281,7 +234,7 @@ AST_TEST_DEFINE(object_retrieve_id)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -344,7 +297,7 @@ AST_TEST_DEFINE(object_retrieve_field)
 		return AST_TEST_FAIL;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -402,7 +355,7 @@ AST_TEST_DEFINE(object_retrieve_multiple_all)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -440,6 +393,63 @@ AST_TEST_DEFINE(object_retrieve_multiple_all)
 	return AST_TEST_PASS;
 }
 
+AST_TEST_DEFINE(object_retrieve_multiple_all_nofetch)
+{
+	RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
+	RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
+	RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
+
+	switch (cmd) {
+	case TEST_INIT:
+		info->name = "object_retrieve_multiple_all_nofetch";
+		info->category = "/res/sorcery_realtime/";
+		info->summary = "sorcery multiple object retrieval unit test";
+		info->description =
+			"Test multiple object retrieval in sorcery using realtime wizard";
+		return AST_TEST_NOT_RUN;
+	case TEST_EXECUTE:
+		break;
+	}
+
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test,allow_unqualified_fetch=no"))) {
+		ast_test_status_update(test, "Failed to open sorcery structure\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah"))) {
+		ast_test_status_update(test, "Failed to allocate a known object type\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (ast_sorcery_create(sorcery, obj)) {
+		ast_test_status_update(test, "Failed to create object using realtime wizard\n");
+		return AST_TEST_FAIL;
+	}
+
+	ao2_cleanup(obj);
+
+	if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah2"))) {
+		ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (ast_sorcery_create(sorcery, obj)) {
+		ast_test_status_update(test, "Failed to create second object using realtime wizard\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (!(objects = ast_sorcery_retrieve_by_fields(sorcery, "test", AST_RETRIEVE_FLAG_MULTIPLE | AST_RETRIEVE_FLAG_ALL, NULL))) {
+		ast_test_status_update(test, "Failed to retrieve a container of all objects\n");
+		return AST_TEST_FAIL;
+	} else if (ao2_container_count(objects) != 0) {
+		ast_test_status_update(test, "Received a container with objects in it when there should be none\n");
+		return AST_TEST_FAIL;
+	}
+
+	return AST_TEST_PASS;
+}
+
+
 AST_TEST_DEFINE(object_retrieve_multiple_field)
 {
 	RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
@@ -464,7 +474,7 @@ AST_TEST_DEFINE(object_retrieve_multiple_field)
 		return AST_TEST_FAIL;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -524,7 +534,7 @@ AST_TEST_DEFINE(object_retrieve_regex)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -574,6 +584,74 @@ AST_TEST_DEFINE(object_retrieve_regex)
 	return AST_TEST_PASS;
 }
 
+AST_TEST_DEFINE(object_retrieve_regex_nofetch)
+{
+	RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
+	RAII_VAR(struct test_sorcery_object *, obj, NULL, ao2_cleanup);
+	RAII_VAR(struct ao2_container *, objects, NULL, ao2_cleanup);
+
+	switch (cmd) {
+	case TEST_INIT:
+		info->name = "object_retrieve_regex_nofetch";
+		info->category = "/res/sorcery_realtime/";
+		info->summary = "sorcery multiple object retrieval using regex unit test";
+		info->description =
+			"Test multiple object retrieval in sorcery using regular expression for matching using realtime wizard";
+		return AST_TEST_NOT_RUN;
+	case TEST_EXECUTE:
+		break;
+	}
+
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test,allow_unqualified_fetch=no"))) {
+		ast_test_status_update(test, "Failed to open sorcery structure\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah-98joe"))) {
+		ast_test_status_update(test, "Failed to allocate a known object type\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (ast_sorcery_create(sorcery, obj)) {
+		ast_test_status_update(test, "Failed to create object using realtime wizard\n");
+		return AST_TEST_FAIL;
+	}
+
+	ao2_cleanup(obj);
+
+	if (!(obj = ast_sorcery_alloc(sorcery, "test", "blah-93joe"))) {
+		ast_test_status_update(test, "Failed to allocate second instance of a known object type\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (ast_sorcery_create(sorcery, obj)) {
+		ast_test_status_update(test, "Failed to create second object using astdb wizard\n");
+		return AST_TEST_FAIL;
+	}
+
+	ao2_cleanup(obj);
+
+	if (!(obj = ast_sorcery_alloc(sorcery, "test", "neener-93joe"))) {
+		ast_test_status_update(test, "Failed to allocate third instance of a known object type\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (ast_sorcery_create(sorcery, obj)) {
+		ast_test_status_update(test, "Failed to create third object using astdb wizard\n");
+		return AST_TEST_FAIL;
+	}
+
+	if (!(objects = ast_sorcery_retrieve_by_regex(sorcery, "test", ""))) {
+		ast_test_status_update(test, "Failed to retrieve a container of objects\n");
+		return AST_TEST_FAIL;
+	} else if (ao2_container_count(objects) != 0) {
+		ast_test_status_update(test, "Received a container with incorrect number of objects in it: %d instead of 0\n", ao2_container_count(objects));
+		return AST_TEST_FAIL;
+	}
+
+	return AST_TEST_PASS;
+}
+
 AST_TEST_DEFINE(object_update)
 {
 	RAII_VAR(struct ast_sorcery *, sorcery, NULL, deinitialize_sorcery);
@@ -592,7 +670,7 @@ AST_TEST_DEFINE(object_update)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -650,7 +728,7 @@ AST_TEST_DEFINE(object_update_uncreated)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -685,7 +763,7 @@ AST_TEST_DEFINE(object_delete)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -732,7 +810,7 @@ AST_TEST_DEFINE(object_delete_uncreated)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -770,7 +848,7 @@ AST_TEST_DEFINE(object_allocate_on_retrieval)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -823,7 +901,7 @@ AST_TEST_DEFINE(object_filter)
 		break;
 	}
 
-	if (!(sorcery = alloc_and_initialize_sorcery())) {
+	if (!(sorcery = alloc_and_initialize_sorcery("sorcery_realtime_test"))) {
 		ast_test_status_update(test, "Failed to open sorcery structure\n");
 		return AST_TEST_FAIL;
 	}
@@ -859,8 +937,10 @@ static int unload_module(void)
 	AST_TEST_UNREGISTER(object_retrieve_id);
 	AST_TEST_UNREGISTER(object_retrieve_field);
 	AST_TEST_UNREGISTER(object_retrieve_multiple_all);
+	AST_TEST_UNREGISTER(object_retrieve_multiple_all_nofetch);
 	AST_TEST_UNREGISTER(object_retrieve_multiple_field);
 	AST_TEST_UNREGISTER(object_retrieve_regex);
+	AST_TEST_UNREGISTER(object_retrieve_regex_nofetch);
 	AST_TEST_UNREGISTER(object_update);
 	AST_TEST_UNREGISTER(object_update_uncreated);
 	AST_TEST_UNREGISTER(object_delete);
@@ -879,8 +959,10 @@ static int load_module(void)
 	AST_TEST_REGISTER(object_retrieve_id);
 	AST_TEST_REGISTER(object_retrieve_field);
 	AST_TEST_REGISTER(object_retrieve_multiple_all);
+	AST_TEST_REGISTER(object_retrieve_multiple_all_nofetch);
 	AST_TEST_REGISTER(object_retrieve_multiple_field);
 	AST_TEST_REGISTER(object_retrieve_regex);
+	AST_TEST_REGISTER(object_retrieve_regex_nofetch);
 	AST_TEST_REGISTER(object_update);
 	AST_TEST_REGISTER(object_update_uncreated);
 	AST_TEST_REGISTER(object_delete);
diff --git a/tests/test_strings.c b/tests/test_strings.c
index 5e3446d99..0c69020f8 100644
--- a/tests/test_strings.c
+++ b/tests/test_strings.c
@@ -523,6 +523,68 @@ AST_TEST_DEFINE(escape_test)
 	return AST_TEST_PASS;
 }
 
+AST_TEST_DEFINE(strings_match)
+{
+	switch (cmd) {
+	case TEST_INIT:
+		info->name = "strings_match";
+		info->category = "/main/strings/";
+		info->summary = "Test ast_strings_match";
+		info->description = "Test ast_strings_match";
+		return AST_TEST_NOT_RUN;
+	case TEST_EXECUTE:
+		break;
+	}
+
+	ast_test_validate(test, ast_strings_match("aaa", NULL, "aaa"));
+	ast_test_validate(test, ast_strings_match("aaa", "", "aaa"));
+	ast_test_validate(test, ast_strings_match("aaa", "=", "aaa"));
+	ast_test_validate(test, !ast_strings_match("aaa", "!=", "aaa"));
+	ast_test_validate(test, !ast_strings_match("aaa", NULL, "aba"));
+	ast_test_validate(test, !ast_strings_match("aaa", "", "aba"));
+	ast_test_validate(test, !ast_strings_match("aaa", "=", "aba"));
+	ast_test_validate(test, ast_strings_match("aaa", "!=", "aba"));
+
+	ast_test_validate(test, ast_strings_match("aaa", "<=", "aba"));
+	ast_test_validate(test, ast_strings_match("aaa", "<=", "aaa"));
+	ast_test_validate(test, !ast_strings_match("aaa", "<", "aaa"));
+
+	ast_test_validate(test, !ast_strings_match("aaa", ">=", "aba"));
+	ast_test_validate(test, ast_strings_match("aaa", ">=", "aaa"));
+	ast_test_validate(test, !ast_strings_match("aaa", ">", "aaa"));
+
+	ast_test_validate(test, !ast_strings_match("aaa", "=", "aa"));
+	ast_test_validate(test, ast_strings_match("aaa", ">", "aa"));
+	ast_test_validate(test, !ast_strings_match("aaa", "<", "aa"));
+
+	ast_test_validate(test, ast_strings_match("1", "=", "1"));
+	ast_test_validate(test, !ast_strings_match("1", "!=", "1"));
+	ast_test_validate(test, !ast_strings_match("2", "=", "1"));
+	ast_test_validate(test, ast_strings_match("2", ">", "1"));
+	ast_test_validate(test, ast_strings_match("2", ">=", "1"));
+	ast_test_validate(test, ast_strings_match("2", ">", "1.9888"));
+	ast_test_validate(test, ast_strings_match("2.9", ">", "1"));
+	ast_test_validate(test, ast_strings_match("2", ">", "1"));
+	ast_test_validate(test, ast_strings_match("2.999", "<", "3"));
+	ast_test_validate(test, ast_strings_match("2", ">", "#"));
+
+	ast_test_validate(test, ast_strings_match("abcccc", "like", "%a%c"));
+	ast_test_validate(test, !ast_strings_match("abcccx", "like", "%a%c"));
+	ast_test_validate(test, ast_strings_match("abcccc", "regex", "a[bc]+c"));
+	ast_test_validate(test, !ast_strings_match("abcccx", "regex", "^a[bxdfgtc]+c$"));
+
+	ast_test_validate(test, !ast_strings_match("neener-93joe", "LIKE", "%blah-%"));
+	ast_test_validate(test, ast_strings_match("blah-93joe", "LIKE", "%blah-%"));
+
+	ast_test_validate(test, !ast_strings_match("abcccx", "regex", NULL));
+	ast_test_validate(test, !ast_strings_match("abcccx", NULL, NULL));
+	ast_test_validate(test, !ast_strings_match(NULL, "regex", NULL));
+	ast_test_validate(test, !ast_strings_match(NULL, NULL, "abc"));
+	ast_test_validate(test, !ast_strings_match(NULL, NULL, NULL));
+
+	return AST_TEST_PASS;
+}
+
 static int unload_module(void)
 {
 	AST_TEST_UNREGISTER(str_test);
@@ -531,6 +593,7 @@ static int unload_module(void)
 	AST_TEST_UNREGISTER(strsep_test);
 	AST_TEST_UNREGISTER(escape_semicolons_test);
 	AST_TEST_UNREGISTER(escape_test);
+	AST_TEST_UNREGISTER(strings_match);
 	return 0;
 }
 
@@ -542,6 +605,7 @@ static int load_module(void)
 	AST_TEST_REGISTER(strsep_test);
 	AST_TEST_REGISTER(escape_semicolons_test);
 	AST_TEST_REGISTER(escape_test);
+	AST_TEST_REGISTER(strings_match);
 	return AST_MODULE_LOAD_SUCCESS;
 }