| Age | Commit message (Collapse) | Author |
|---|
|
If the SQL UPDATE statement changes nothing then SQLRowCount returns 0.
This value should be treated as success.
But the function sorcery_realtime_update treats it as failed.
This bug was found using stress tests on PJSIP.
If there are 2 consecutive SIP REGISTER requests with the same contact data
during 1 second then res_pjsip_registrar adds contact location on 1st request
and tries to update contact location on 2nd.
The update fails and res_pjsip_registrar even removes correct contact location.
The test "object_update_uncreated" was removed from test_sorcery_realtime.c
because it's now a valid situation.
This patch also adds missing debug of extra SQL parameter.
ASTERISK-26172 #close
Change-Id: I05a7f3051455336c9dda29efc229decf86071303
|
|
There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.
A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0. One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.
This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare. The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.
They do now.
The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator. For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'". If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.
The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container. However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.
So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function. Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex. If the operator is like or regex, the
right string should be a %-pattern or a regex expression. If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.
To use this new function on ast_variables, 2 new functions were added to
config.c. One that compares 2 ast_variables, and one that compares 2
ast_variable lists. The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list. The latter will traverse the right list and return true if all
the variables in it match the left list.
Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines. The realtime backend just passes
the variable list unaltered to the engine. The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.
Only one more change to sorcery was done... A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)
Now on to res_pjsip...
pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors. Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.
res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.
res_pjsip_registrar_expire was completely refactored. It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them. A new
contact_expiration_check_interval was added to global with a default of
30 seconds.
Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.
There are still objects that can't be filtered at the database like
identifies, transports, and registrations. These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.
Back to allow_unqualified_fetch. If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :) Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache. Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts. It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.
Example sorcery.conf:
[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error
ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer
Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
|
|
res_sorcery_realtime's search-by-regex callback performed a check to
ensure that the passed-in regex began with a caret (^). If it did not,
then no results would be returned.
This callback only started to become used when "like" support was added
to PJSIP CLI commands. The CLI command for listing objects would pass an
empty regex ("") to the sorcery backend if no "like" statement was
present. For most sorcery backends, this resulted in returning all
objects. However, for realtime, this resulted in returning no objects.
This commit seeks to fix the regression by removing the requirement from
res_sorcery_realtime for the passed-in-regex to begin with a caret.
ASTERISK-25689 #close
Reported by Marcelo Terres
Change-Id: I22b4dc5d7f3f11bb29ac2e42ef94682e9bab3b20
|
|
Analyzing the code shows that the unit test summary and description
strings should not end with a new-line character. Where these strings are
used in the code a new-line is provided for output.
Change-Id: I129284f5e7ca93d82532334076da4c462d3d9fba
|
|
This patch provides the capability to manipulate templates and categories
with non-unique names via AMI.
Summary of changes:
GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list
of name_regex=value_regex expressions which will cause only categories whose
variables match all expressions to be considered. The special variable name
TEMPLATES can be used to control whether templates are included. Passing
'include' as the value will include templates along with normal categories.
Passing 'restrict' as the value will restrict the operation to ONLY templates.
Not specifying a TEMPLATES expression results in the current default behavior
which is to not include templates.
UpdateConfig: NewCat now includes options for allowing duplicate category
names, indicating if the category should be created as a template, and
specifying templates the category should inherit from. The rest of the
actions now accept a filter string as defined above. If there are non-unique
category names, you can now update specific ones based on variable values.
To facilitate the new capabilities in manager, corresponding changes had to be
made to config, most notably the addition of filter criteria to many of the
APIs. In some cases it was easy to change the references to use the new
prototype but others would have required touching too many files for this
patch so a wrapper with the original prototype was created. Macros couldn't
be used in this case because it would break binary compatibility with modules
such as res_digium_phone that are linked to real symbols.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4033/
........
Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@425384 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
........
Merged revisions 419129 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........
Merged revisions 419162 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 419163 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419175 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This commit contains several changes to sorcery:
1) Application of sorcery configuration based on module name is automatically performed
when sorcery is opened for a module.
2) Sorcery will not attempt to apply the same wizard to an object type more than once.
3) Sorcery gives more exact results when attempting to apply a wizard, whether as the
default or based on configuration.
Sorcery unit tests still pass for me after making these changes.
Review: https://reviewboard.asterisk.org/r/3326
........
Merged revisions 411159 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@411656 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
These changes were still up for review and have not been approved
yet. I must have had the changes in my working copy when making
a different change.
........
Merged revisions 410696 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410699 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Playing back a file to a channel in an ARI bridge would attempt to wait until
the playback concluded before returning. The method used involved signaling the
waiting thread in the ARI custom playback function.
The problem with this is that there were some corner cases that were not accounted for:
* If a bridge channel could not be found, then we never would attempt the playback but
would still attempt to wait for the playback to complete.
* If the bridge playfile action failed to queue, we would still attempt to wait for the
playback to complete.
* If the bridge playfile action were queued but some circumstance caused the playback
not to occur (the bridge dies, the channel is removed from the bridge), then we would
never be notified.
The solution to this is to move the waiting logic into the bridge code. A new bridge
API function is added to queue a synchronous action on a bridge. The waiting thread
is notified when the queued frame has been freed, either due to an error occurring
or due to successful playback. As a failsafe, the waiting thread has a 10 minute
timeout just in case there is a frame leak somewhere.
Review: https://reviewboard.asterisk.org/r/3338
........
Merged revisions 410673 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410684 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
The store realtime callback needs to return a positive value for
sorcery to treat the store as a success.
........
Merged revisions 410625 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410626 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
When retrieving data from a database or other realtime backend, it's quite
possible to retrieve variables that Asterisk does not care about but that
are legitimate to exist. Asterisk does not need to throw a hissy fit when
these variables are encountered but rather just filter them out.
Review: https://reviewboard.asterisk.org/r/3305
........
Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
When strict XML documentation checking was re-enabled, the test objects used in
sorcery would fail to register as the types were not marked internal and the
nodoc option wasn't used for the options. This fixes that problem, such that,
as one would hope, they once again pass.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397571 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392647 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This change does the following:
1. Adds the sorcery realtime module
2. Adds unit tests for the sorcery realtime module
3. Changes the realtime core to use an ast_variable list instead of variadic arguments
4. Changes all realtime drivers to accept an ast_variable list
Review: https://reviewboard.asterisk.org/r/2424/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386731 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
