Age | Commit message (Collapse) | Author |
|
Sorcery creates taskprocessors for object types to process object observer
callbacks. An API call is needed to be able to set the congestion levels
of these taskprocessors for selected object types.
* Updated PJSIP's contact and contact_status sorcery object type observer
default congestion levels based upon stress testing. Increased the
congestion levels to reduce the potential for bursty register/unregister
and subscribe/unsubscribe activity from triggering the taskprocessor
overload alert.
ASTERISK-26088
Reported by: Richard Mudgett
Change-Id: I4542e83b556f0714009bfeff89505c801f1218c6
|
|
There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.
A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0. One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.
This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare. The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.
They do now.
The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator. For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'". If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.
The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container. However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.
So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function. Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex. If the operator is like or regex, the
right string should be a %-pattern or a regex expression. If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.
To use this new function on ast_variables, 2 new functions were added to
config.c. One that compares 2 ast_variables, and one that compares 2
ast_variable lists. The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list. The latter will traverse the right list and return true if all
the variables in it match the left list.
Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines. The realtime backend just passes
the variable list unaltered to the engine. The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.
Only one more change to sorcery was done... A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)
Now on to res_pjsip...
pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors. Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.
res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.
res_pjsip_registrar_expire was completely refactored. It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them. A new
contact_expiration_check_interval was added to global with a default of
30 seconds.
Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.
There are still objects that can't be filtered at the database like
identifies, transports, and registrations. These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.
Back to allow_unqualified_fetch. If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :) Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache. Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts. It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.
Example sorcery.conf:
[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error
ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer
Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
|
|
Gerrit is complaining of conflicts when trying to create a patch series
of all of the cherry-picked master commits, so I have instead squashed
it all into one commit.
ASTERISK-25067 #close
Reported by: Matt Jordan
Change-Id: I6dda90343fae24a75dc5beec84980024e8d61eb9
|
|
Find and unlink the specified sorcery object type to complement
ast_sorcery_object_register(). Without this function you cannot
completely unload individual modules that use sorcery for configuration.
ASTERISK-24907
Reported by: Kevin Harwell
Change-Id: I1c04634fe9a90921bf676725c7d6bb2aeaab1c88
|
|
Currently you can 'apply' a wizard to an object type but the wizard
always goes at the end of the object type's wizard list. This patch
adds a new ast_sorcery_insert_wizard_mapping function that allows
you to insert a wizard anyplace in the list. I.E. You could
add a caching wizard to an object type and place it before all
wizards.
ast_sorcery_get_wizard_mapping_count and
ast_sorcery_get_wizard_mapping were added to allow examination
of the mapping list.
ast_sorcery_remove_mapping was added to remove a mapping by name.
As part of this patch, the object type's wizard list was converted
from an ao2_container to an AST_VECTOR_RW.
A new test was added to test_sorcery for this capability.
ASTERISK-25044 #close
Change-Id: I9d2469a9296b2698082c0989e25e6848dc403b57
|
|
Add new global, instance and wizard observers.
instance_created
wizard_registered
wizard_unregistered
instance_destroying
instance_loading
instance_loaded
wizard_mapped
object_type_registered
object_type_loading
object_type_loaded
wizard_loading
wizard_loaded
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4215/
........
Merged revisions 428999 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@429000 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This commit contains several changes to sorcery:
1) Application of sorcery configuration based on module name is automatically performed
when sorcery is opened for a module.
2) Sorcery will not attempt to apply the same wizard to an object type more than once.
3) Sorcery gives more exact results when attempting to apply a wizard, whether as the
default or based on configuration.
Sorcery unit tests still pass for me after making these changes.
Review: https://reviewboard.asterisk.org/r/3326
........
Merged revisions 411159 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@411656 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
These changes were still up for review and have not been approved
yet. I must have had the changes in my working copy when making
a different change.
........
Merged revisions 410696 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410699 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Playing back a file to a channel in an ARI bridge would attempt to wait until
the playback concluded before returning. The method used involved signaling the
waiting thread in the ARI custom playback function.
The problem with this is that there were some corner cases that were not accounted for:
* If a bridge channel could not be found, then we never would attempt the playback but
would still attempt to wait for the playback to complete.
* If the bridge playfile action failed to queue, we would still attempt to wait for the
playback to complete.
* If the bridge playfile action were queued but some circumstance caused the playback
not to occur (the bridge dies, the channel is removed from the bridge), then we would
never be notified.
The solution to this is to move the waiting logic into the bridge code. A new bridge
API function is added to queue a synchronous action on a bridge. The waiting thread
is notified when the queued frame has been freed, either due to an error occurring
or due to successful playback. As a failsafe, the waiting thread has a 10 minute
timeout just in case there is a frame leak somewhere.
Review: https://reviewboard.asterisk.org/r/3338
........
Merged revisions 410673 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410684 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Created the 'pjsip show channel' and 'pjsip show contact' commands.
Refactored out the hated ast_hashtab. Replaced with ao2_container.
Cleaned up function naming. Internal only, no public name changes.
Cleaned up whitespace and brace formatting in cli code.
Changed some NULL checking from "if"s to ast_asserts.
Fixed some register/unregister ordering to reduce deadlock potential.
Fixed ast_sip_location_add_contact where the 'name' buffer was too short.
Fixed some self-assignment issues in res_pjsip_outbound_registration.
(closes issue ASTERISK-23276)
Review: http://reviewboard.asterisk.org/r/3283/
........
Merged revisions 410287 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410288 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This fixes mistakes I previously made in merging
gtjoseph's changes with mine.
........
Merged revisions 410211 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410212 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
When retrieving data from a database or other realtime backend, it's quite
possible to retrieve variables that Asterisk does not care about but that
are legitimate to exist. Asterisk does not need to throw a hissy fit when
these variables are encountered but rather just filter them out.
Review: https://reviewboard.asterisk.org/r/3305
........
Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
In order to prevent confusion over the allow and disallow
list of codecs being the same an option for registering a
field as an alias is added. The alias field will be read
from the configuration file, but afterwards is not listed
as a known field. With disallow set as an alias, the CLI
command pjsip show endpoint # will list the allow= field,
but not the disallow field.
(closes issue ASTERISK-23092)
Review: https://reviewboard.asterisk.org/r/3193/
........
Merged revisions 410190 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410191 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This patch creates the AST_SORCERY dialplan function which allows someone to
retrieve any value from a sorcery-based config file. It's similar to
AST_CONFIG.
The creation of the function itself was fairly straightforward but it required
changes to the underlying sorcery infrastructure that rippled into individual
sorcery objects. The changes stemmed from inconsistencies in how sorcery
created ast_variable objectsets from sorcery objects and the inconsistency
in how individual objects used that feature especially when it came to
parameters that can be specified multiple times like contact in aor and match
in identify. You can read more here...
http://lists.digium.com/pipermail/asterisk-dev/2014-February/065202.html
So, what this patch does, besides actually creating the AST_SORCERY function,
is the following...
* Creates ast_variable_list_append which is a helper to append one ast_variable
list to another.
* Modifies the ast_sorcery_object_field_register functions to accept the
already-defined sorcery_fields_handler callback.
* Modifies ast_sorcery_objectset_create to accept a parameter indicating return
type preference...a single ast_variable with all values concatenated or an
ast_variable list with multiple entries. Also fixed a few bugs.
* Modifies individual sorcery object implementations to use the new function
definition of the ast_sorcery_object_field_register functions.
* Modifies location.c and res_pjsip_endpoint_identifier_ip.c to implement
sorcery_fields_handler handlers so they return multiple occurrences as an
ast_variable_list.
* Added a whole bunch of tests to test_sorcery.
(closes issue ASTERISK-22537)
Review: http://reviewboard.asterisk.org/r/3254/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410042 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
In order to retrieve an arbitrary sorcery instance from a dialplan function
(or any place else) there needs to be a registry of sorcery instances.
ast_sorcery_init now creates a hashtab as a registry.
ast_sorcery_open now checks the hashtab for an existing sorcery instance
matching the caller's module name. If it finds one, it bumps the
refcount and returns it. If not, it creates a new sorcery instance,
adds it to the hashtab, then returns it.
ast_sorcery_retrieve_by_module_name is a new function that does a hashtab
lookup by module name. It can be called by the future dialplan function.
res_pjsip/config_system needed a small change to share the main res_pjsip
sorcery instance.
tests/test_sorcery was updated to include a test for the registry.
(closes issue ASTERISK-22537)
Review: http://reviewboard.asterisk.org/r/3184/
........
Merged revisions 408518 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@408519 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Implements the following cli commands:
pjsip list aors
pjsip list auths
pjsip list channels
pjsip list contacts
pjsip list endpoints
pjsip show aor(s)
pjsip show auth(s)
pjsip show channels
pjsip show endpoint(s)
Also...
Minor modifications made to the AMI command implementations to facilitate
reuse.
New function ast_variable_list_sort added to config.c and config.h to implement
variable list sorting.
(issue ASTERISK-22610)
patches:
pjsip_cli_v2.patch uploaded by george.joseph (License 6322)
........
Merged revisions 404480 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@404507 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
* Make ast_sorcery_observer_remove() accept a const callbacks struct.
* Make ast_sorcery_observer_remove() tolerant of the sorcery parameter
being NULL. Now it can be called within a module unload routine if the
sorcery initialization fails.
* Fix ast_sorcery_observer_add() to fail if the container link fails.
........
Merged revisions 403324 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403327 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Created the following AMI commands and corresponding events for res_pjsip:
PJSIPShowEndpoints - Provides a listing of all pjsip endpoints and a few
select attributes on each.
Events:
EndpointList - for each endpoint a few attributes.
EndpointlistComplete - after all endpoints have been listed.
PJSIPShowEndpoint - Provides a detail list of attributes for a specified
endpoint.
Events:
EndpointDetail - attributes on an endpoint.
AorDetail - raised for each AOR on an endpoint.
AuthDetail - raised for each associated inbound and outbound auth
TransportDetail - transport attributes.
IdentifyDetail - attributes for the identify object associated with
the endpoint.
EndpointDetailComplete - last event raised after all detail events.
PJSIPShowRegistrationsInbound - Provides a detail listing of all inbound
registrations.
Events:
InboundRegistrationDetail - inbound registration attributes for each
registration.
InboundRegistrationDetailComplete - raised after all detail records have
been listed.
PJSIPShowRegistrationsOutbound - Provides a detail listing of all outbound
registrations.
Events:
OutboundRegistrationDetail - outbound registration attributes for each
registration.
OutboundRegistrationDetailComplete - raised after all detail records
have been listed.
PJSIPShowSubscriptionsInbound - A detail listing of all inbound subscriptions
and their attributes.
Events:
SubscriptionDetail - on each subscription detailed attributes
SubscriptionDetailComplete - raised after all detail records have
been listed.
PJSIPShowSubscriptionsOutbound - A detail listing of all outboundbound
subscriptions and their attributes.
Events:
SubscriptionDetail - on each subscription detailed attributes
SubscriptionDetailComplete - raised after all detail records have
been listed.
(issue ASTERISK-22609)
Reported by: Matt Jordan
Review: https://reviewboard.asterisk.org/r/2959/
........
Merged revisions 403131 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403133 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Some configuration objects currently won't place nice if reloaded.
Specifically, in this case the pjsip transport objects. Now when
registering an object in sorcery one may specify that the object is
allowed to be reloaded or not. If the object is set to not reload
then upon reloading of the configuration the objects of that type
will not be reloaded. The initially loaded objects of that type
however will remain.
While the transport objects will not longer be reloaded it is still
possible for a user to configure an endpoint to an invalid transport.
A couple of log messages were added to help diagnose this problem if
it occurs.
(closes issue ASTERISK-22382)
Reported by: Rusty Newton
(closes issue ASTERISK-22384)
Reported by: Rusty Newton
Review: https://reviewboard.asterisk.org/r/2807/
........
Merged revisions 398139 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@398140 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
sorcery types.
This also adds a _nodoc equivalent.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397599 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
There are times when a configuration option should not have documentation.
1. Some options are registered with a particular object merely as a warning to
users. These options aren't even really 'deprecated' - which has its own
separate API call - they are actually provided by a different configuration
file. The options are merely registered so that the user gets a warning that
a different configuration file provides the item.
2. Some object types - most notably some used by modules that use sorcery - are
completely internal and should never be shown to the user.
3. Sorcery itself has several 'hidden' fields that should never be shown to a
user.
This patch updates the configuration framework and sorcery with additional API
calls that allow a module to register types as internal and options as not
requiring documentation. This bypasses the XML documentation checking.
This patch also re-enables the strict XML documentation checking in trunk, as
well as updates some documentation that was missing.
Review: https://reviewboard.asterisk.org/r/2785/
(closes issue ASTERISK-22359)
Reported by: Matt Jordan
(closes issue ASTERISK-22112)
Reported by: Rusty Newton
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@397524 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
observer.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392879 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Sorcery specific object information is now opaque and allocated with the object.
This means that modules do not need to be recompiled if the sorcery specific part
is changed. It also means that sorcery can store additional information on objects
and ensure it is freed or the reference count decreased when the object goes away.
To facilitate the above a generic sorcery allocator function has been added which
also ensures that allocated objects do not have a lock.
Extended fields have been added thanks to all of the above which allows specific fields
to be marked as extended, and thus simply stored as-is within the object. Type safety
is *NOT* enforced on these fields. A consumer of them has to query and ultimately perform
their own safety check. What does this mean? Extra modules can extend already defined
structures without having to modify them.
Tests have also been included to verify extended field functionality.
Review: https://reviewboard.asterisk.org/r/2585/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392586 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This change adds the ability for modules to add themselves as observers
to sorcery object types. Observers can be notified when objects are
created, updated, or deleted as well as when the object type is loaded or
reloaded. Observer notifications are done using a thread pool in a serialized
fashion so the caller of the sorcery API calls is minimally impacted.
This also adds the ability to create JSON changesets of a sorcery object.
Tests are also present to confirm all of the above functionality.
Review: https://reviewboard.asterisk.org/r/2477/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@387662 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
The pimp_my_sip branch is being merged at this point because
it offers basic functionality, and from an API standpoint, things
are complete.
SIP work is *not* feature-complete; however, with the completion
of the SUBSCRIBE/NOTIFY API, all APIs (except a PUBLISH API) have
been created, and thus it is possible for developers to attempt
to create new SIP work.
API documentation can be found in the doxygen in the code, but
usability documentation is still lacking.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386540 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@385548 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@383405 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
using a regex.
Review: https://reviewboard.asterisk.org/r/2332/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@382340 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
their id.
Review: https://reviewboard.asterisk.org/r/2329/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@381614 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This patch does two things:
1. It disables (temporarily) strict XML documentation checking for module
configurations. We should re-enable it before making any release from
trunk.
2. Pass the module flag AST_MODULE through sorcery. This means several of the
API calls are now macros and will do this automatically for you. The config
framework needs the module that objects are registering to so it can
properly construct the documentation. (This was already a required field,
but sorcery was getting by without it)
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@381567 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
This commit adds native implementation support for copying and diffing objects,
as well as the ability to load or reload on a per-object type level.
Review: https://reviewboard.asterisk.org/r/2320/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@381134 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|
|
Sorcery is a unifying data access layer which provides a pluggable mechanism to allow
object creation, retrieval, updating, and deletion using different backends (or wizards).
This is a fancy way of saying "one interface to rule them all" where them is configuration,
realtime, and anything else that comes along.
Review: https://reviewboard.asterisk.org/r/2259/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@380069 65c4cc65-6c06-0410-ace0-fbb531ad65f3
|