SDP: Rework SDP offer/answer model and update capabilities merges.

The SDP offer/answer model requires an answer to an offer before a new SDP
can be processed.  This allows our local SDP creation to be deferred until
we know that we need to create an offer or an answer SDP.  Once the local
SDP is created it won't change until the SDP negotiation is restarted.

An offer SDP in an initial SIP INVITE can receive more than one answer
SDP.  In this case, we need to merge each answer SDP with our original
offer capabilities to get the currently negotiated capabilities.  To
satisfy this requirement means that we cannot update our proposed
capabilities until the negotiations are restarted.

Local topology updates from ast_sdp_state_update_local_topology() are
merged together until the next offer SDP is created.  These accumulated
updates are then merged with the current negotiated capabilities to create
the new proposed capabilities that the offer SDP is built.

Local topology updates are merged in several passes to attempt to be smart
about how streams from the system are matched with the previously
negotiated stream slots.  To allow for T.38 support when merging, type
matching considers audio and image types to be equivalent.  First streams
are matched by stream name and type.  Then streams are matched by stream
type only.  Any remaining unmatched existing streams are declined.  Any
new active streams are either backfilled into pre-merge declined slots or
appended onto the end of the merged topology.  Any excess new streams
above the maximum supported number of streams are simply discarded.

Remote topology negotiation merges depend if the topology is an offer or
answer.  An offer remote topology negotiation dictates the stream slot
ordering and new streams can be added.  A remote offer can do anything to
the previously negotiated streams except reduce the number of stream
slots.  An answer remote topology negotiation is limited to what our offer
requested.  The answer can only decline streams, pick codecs from the
offered list, or indicate the remote's stream hold state.

I had originally kept the RTP instance if the remote offer SDP changed a
stream type between audio and video since they both use RTP.  However, I
later removed this support in favor of simply creating a new RTP instance
since the stream's purpose has to be changing anyway.  Any RTP packets
from the old stream type might cause mischief for the bridged peer.

* Added ast_sdp_state_restart_negotiations() to restart the SDP
offer/answer negotiations.  We will thus know to create a new local SDP
when it is time to create an offer or answer.

* Removed ast_sdp_state_reset().  Save the current topology before
starting T.38.  To recover from T.38 simply update the local topology to
the saved topology and restart the SDP negotiations to get the offer SDP
renegotiating the previous configuration.

* Allow initial topology for ast_sdp_state_alloc() to be NULL so an
initial remote offer SDP can dictate the streams we start with.  We can
always update the local topology later if it turns out we need to offer
SDP first because the remote chose to defer sending us a SDP.

* Made the ast_sdp_state_alloc() initial topology limit to max_streams,
limit to configured codecs, handle declined streams, and discard
unsupported types.

* Convert struct ast_sdp to ao2 object.  Needed to easily save off a
remote SDP to refer to later for various reasons such as generating
declined m= lines in the local SDP.

* Improve converting remote SDP streams to a topology including stream
state.  A stream state of AST_STREAM_STATE_REMOVED indicates the stream is
declined/dead.

* Improve merging streams to take into account the stream state.

* Added query for remote hold state.

* Added maximum streams allowed SDP config option.

* Added ability to create new streams as needed.  New streams are created
with configured default audio, video, or image codecs depending on stream
type.

* Added global locally_held state along with a per stream local hold
state.  Historically, Asterisk only has a global locally held state
because when the we put the remote on hold we do it for all active
streams.

* Added queries for a rejected offer and current SDP negotiation role.
The rejected query allows the using module to know how to respond to a
failed remote SDP set.  Should the using module respond with a 488 Not
Acceptable Here or 500 Internal Error to the offer SDP?

* Moved sdp_state_capabilities.connection_address to ast_sdp_state.  There
seems no reason to keep it in the sdp_state_capabilities struct since it
was only used by the ast_sdp_state.proposed_capabilities instance.

* Callbacks are now available to allow the using module some customization
of negotiated streams and to complete setting up streams for use.  See the
typedef doxygen for each callback for what is allowable and when they are
called.
    * Added topology answerer modify callback.
    * Added topology pre and post apply callbacks.
    * Added topology offerer modify callback.
    * Added topology offerer configure callback.

* Had to rework the unit tests because I changed how SDP topologies are
merged.  Replaced several unit tests with new negotiation tests.

Change-Id: If07fe6d79fbdce33968a9401d41d908385043a06

4 files changed, 339 insertions, 35 deletions

diff --git a/include/asterisk/sdp.h b/include/asterisk/sdp.h
index 224a0e5a3..768469544 100644
--- a/include/asterisk/sdp.h
+++ b/include/asterisk/sdp.h
@@ -254,16 +254,6 @@ void ast_sdp_s_free(struct ast_sdp_s_line *s_line);
 void ast_sdp_t_free(struct ast_sdp_t_line *t_line);
 
 /*!
- * \brief Free an SDP
- * Frees the sdp and all resources it contains
- *
- * \param sdp The sdp to free
- *
- * \since 15
- */
-void ast_sdp_free(struct ast_sdp *sdp);
-
-/*!
  * \brief Allocate an SDP Attribute
  *
  * \param name Attribute Name
@@ -544,7 +534,7 @@ int ast_sdp_m_add_format(struct ast_sdp_m_line *m_line, const struct ast_sdp_opt
 	int rtp_code, int asterisk_format, const struct ast_format *format, int code);
 
 /*!
- * \brief Create an SDP
+ * \brief Create an SDP ao2 object
  *
  * \param o_line Origin
  * \param c_line Connection
diff --git a/include/asterisk/sdp_options.h b/include/asterisk/sdp_options.h
index b8c1bbd56..e45ae8cb1 100644
--- a/include/asterisk/sdp_options.h
+++ b/include/asterisk/sdp_options.h
@@ -20,6 +20,7 @@
 #define _ASTERISK_SDP_OPTIONS_H
 
 #include "asterisk/udptl.h"
+#include "asterisk/format_cap.h"
 
 struct ast_sdp_options;
 
@@ -80,6 +81,149 @@ enum ast_sdp_options_encryption {
 };
 
 /*!
+ * \brief Callback when processing an offer SDP for our answer SDP.
+ * \since 15.0.0
+ *
+ * \details
+ * This callback is called after merging our last negotiated topology
+ * with the remote's offer topology and before we have sent our answer
+ * SDP.  At this point you can alter new_topology streams.  You can
+ * decline, remove formats, or rename streams.  Changing anything else
+ * on the streams is likely to not end well.
+ *
+ * * To decline a stream simply set the stream state to
+ *   AST_STREAM_STATE_REMOVED.  You could implement a maximum number
+ *   of active streams of a given type policy.
+ *
+ * * To remove formats use the format API to remove any formats from a
+ *   stream.  The streams have the current joint negotiated formats.
+ *   Most likely you would want to remove all but the first format.
+ *
+ * * To rename a stream you need to clone the stream and give it a
+ *   new name and then set it in new_topology using
+ *   ast_stream_topology_set_stream().
+ *
+ * \note Removing all formats is an error.  You should decline the
+ * stream instead.
+ *
+ * \param context User supplied context data pointer for the SDP
+ * state.
+ * \param old_topology Active negotiated topology.  NULL if this is
+ * the first SDP negotiation.  The old topology is available so you
+ * can tell if any streams are new or changing type.
+ * \param new_topology New negotiated topology that we intend to
+ * generate the answer SDP.
+ *
+ * \return Nothing
+ */
+typedef void (*ast_sdp_answerer_modify_cb)(void *context,
+	const struct ast_stream_topology *old_topology,
+	struct ast_stream_topology *new_topology);
+
+/*!
+ * \internal
+ * \brief Callback when generating a topology for our SDP offer.
+ * \since 15.0.0
+ *
+ * \details
+ * This callback is called after merging any topology updates from the
+ * system by ast_sdp_state_update_local_topology() and before we have
+ * sent our offer SDP.  At this point you can alter new_topology
+ * streams.  You can decline, add/remove/update formats, or rename
+ * streams.  Changing anything else on the streams is likely to not
+ * end well.
+ *
+ * * To decline a stream simply set the stream state to
+ *   AST_STREAM_STATE_REMOVED.  You could implement a maximum number
+ *   of active streams of a given type policy.
+ *
+ * * To update formats use the format API to change formats of the
+ *   streams.  The streams have the current proposed formats.  You
+ *   could do whatever you want for formats but you should stay within
+ *   the configured formats for the stream type's endpoint.  However,
+ *   you should use ast_sdp_state_update_local_topology() instead of
+ *   this backdoor method.
+ *
+ * * To rename a stream you need to clone the stream and give it a
+ *   new name and then set it in new_topology using
+ *   ast_stream_topology_set_stream().
+ *
+ * \note Removing all formats is an error.  You should decline the
+ * stream instead.
+ *
+ * \note Declined new streams that are in slots higher than present in
+ * old_topology are removed so the SDP can be smaller.  The remote has
+ * never seen those slots so we shouldn't bother keeping them.
+ *
+ * \param context User supplied context data pointer for the SDP
+ * state.
+ * \param old_topology Active negotiated topology.  NULL if this is
+ * the first SDP negotiation.  The old topology is available so you
+ * can tell if any streams are new or changing type.
+ * \param new_topology Merged topology that we intend to generate the
+ * offer SDP.
+ *
+ * \return Nothing
+ */
+typedef void (*ast_sdp_offerer_modify_cb)(void *context,
+	const struct ast_stream_topology *old_topology,
+	struct ast_stream_topology *new_topology);
+
+/*!
+ * \brief Callback when generating an offer SDP to configure extra stream data.
+ * \since 15.0.0
+ *
+ * \details
+ * This callback is called after any ast_sdp_offerer_modify_cb
+ * callback and before we have sent our offer SDP.  The callback can
+ * call several SDP API calls to configure the proposed capabilities
+ * of streams before we create the SDP offer.  For example, the
+ * callback could configure a stream specific connection address, T.38
+ * parameters, RTP instance, or UDPTL instance parameters.
+ *
+ * \param context User supplied context data pointer for the SDP
+ * state.
+ * \param topology Topology ready to configure extra stream options.
+ *
+ * \return Nothing
+ */
+typedef void (*ast_sdp_offerer_config_cb)(void *context, const struct ast_stream_topology *topology);
+
+/*!
+ * \brief Callback before applying a topology.
+ * \since 15.0.0
+ *
+ * \details
+ * This callback is called before the topology is applied so the
+ * using module can do what is necessary before the topology becomes
+ * active.
+ *
+ * \param context User supplied context data pointer for the SDP
+ * state.
+ * \param topology Topology ready to be applied.
+ *
+ * \return Nothing
+ */
+typedef void (*ast_sdp_preapply_cb)(void *context, const struct ast_stream_topology *topology);
+
+/*!
+ * \brief Callback after applying a topology.
+ * \since 15.0.0
+ *
+ * \details
+ * This callback is called after the topology is applied so the
+ * using module can do what is necessary after the topology becomes
+ * active.
+ *
+ * \param context User supplied context data pointer for the SDP
+ * state.
+ * \param topology Topology already applied.
+ *
+ * \return Nothing
+ */
+typedef void (*ast_sdp_postapply_cb)(void *context, const struct ast_stream_topology *topology);
+
+/*!
  * \since 15.0.0
  * \brief Allocate a new SDP options structure.
  *
@@ -204,6 +348,24 @@ void ast_sdp_options_set_rtp_engine(struct ast_sdp_options *options,
  */
 const char *ast_sdp_options_get_rtp_engine(const struct ast_sdp_options *options);
 
+void ast_sdp_options_set_state_context(struct ast_sdp_options *options, void *state_context);
+void *ast_sdp_options_get_state_context(const struct ast_sdp_options *options);
+
+void ast_sdp_options_set_answerer_modify_cb(struct ast_sdp_options *options, ast_sdp_answerer_modify_cb answerer_modify_cb);
+ast_sdp_answerer_modify_cb ast_sdp_options_get_answerer_modify_cb(const struct ast_sdp_options *options);
+
+void ast_sdp_options_set_offerer_modify_cb(struct ast_sdp_options *options, ast_sdp_offerer_modify_cb offerer_modify_cb);
+ast_sdp_offerer_modify_cb ast_sdp_options_get_offerer_modify_cb(const struct ast_sdp_options *options);
+
+void ast_sdp_options_set_offerer_config_cb(struct ast_sdp_options *options, ast_sdp_offerer_config_cb offerer_config_cb);
+ast_sdp_offerer_config_cb ast_sdp_options_get_offerer_config_cb(const struct ast_sdp_options *options);
+
+void ast_sdp_options_set_preapply_cb(struct ast_sdp_options *options, ast_sdp_preapply_cb preapply_cb);
+ast_sdp_preapply_cb ast_sdp_options_get_preapply_cb(const struct ast_sdp_options *options);
+
+void ast_sdp_options_set_postapply_cb(struct ast_sdp_options *options, ast_sdp_postapply_cb postapply_cb);
+ast_sdp_postapply_cb ast_sdp_options_get_postapply_cb(const struct ast_sdp_options *options);
+
 /*!
  * \since 15.0.0
  * \brief Set SDP Options rtp_symmetric
@@ -505,6 +667,26 @@ unsigned int ast_sdp_options_get_udptl_far_max_datagram(const struct ast_sdp_opt
 
 /*!
  * \since 15.0.0
+ * \brief Set SDP Options max_streams
+ *
+ * \param options SDP Options
+ * \param max_streams
+ */
+void ast_sdp_options_set_max_streams(struct ast_sdp_options *options,
+	unsigned int max_streams);
+
+/*!
+ * \since 15.0.0
+ * \brief Get SDP Options max_streams
+ *
+ * \param options SDP Options
+ *
+ * \returns max_streams
+ */
+unsigned int ast_sdp_options_get_max_streams(const struct ast_sdp_options *options);
+
+/*!
+ * \since 15.0.0
  * \brief Enable setting SSRC level attributes on SDPs
  *
  * \param options SDP Options
@@ -547,4 +729,46 @@ void ast_sdp_options_set_sched_type(struct ast_sdp_options *options,
 struct ast_sched_context *ast_sdp_options_get_sched_type(const struct ast_sdp_options *options,
 	enum ast_media_type type);
 
+/*!
+ * \brief Set all allowed stream types to create new streams.
+ * \since 15.0.0
+ *
+ * \param options SDP Options
+ * \param cap Format capabilities to set all allowed stream types at once.
+ *            Could be NULL to disable creating any new streams.
+ *
+ * \return Nothing
+ */
+void ast_sdp_options_set_format_caps(struct ast_sdp_options *options,
+	struct ast_format_cap *cap);
+
+/*!
+ * \brief Set the SDP options format cap used to create new streams of the type.
+ * \since 15.0.0
+ *
+ * \param options SDP Options
+ * \param type Media type the format cap represents.
+ * \param cap Format capabilities to use for the specified media type.
+ *            Could be NULL to disable creating new streams of type.
+ *
+ * \return Nothing
+ */
+void ast_sdp_options_set_format_cap_type(struct ast_sdp_options *options,
+	enum ast_media_type type, struct ast_format_cap *cap);
+
+/*!
+ * \brief Get the SDP options format cap used to create new streams of the type.
+ * \since 15.0.0
+ *
+ * \param options SDP Options
+ * \param type Media type the format cap represents.
+ *
+ * \retval NULL if stream not allowed to be created.
+ * \retval cap to use in negotiating the new stream.
+ *
+ * \note The returned cap does not have its own ao2 ref.
+ */
+struct ast_format_cap *ast_sdp_options_get_format_cap_type(const struct ast_sdp_options *options,
+	enum ast_media_type type);
+
 #endif /* _ASTERISK_SDP_OPTIONS_H */
diff --git a/include/asterisk/sdp_state.h b/include/asterisk/sdp_state.h
index b8209e1d5..ec9d502e2 100644
--- a/include/asterisk/sdp_state.h
+++ b/include/asterisk/sdp_state.h
@@ -30,12 +30,22 @@ struct ast_control_t38_parameters;
 /*!
  * \brief Allocate a new SDP state
  *
+ * \details
  * SDP state keeps tabs on everything SDP-related for a media session.
  * Most SDP operations will require the state to be provided.
  * Ownership of the SDP options is taken on by the SDP state.
  * A good strategy is to call this during session creation.
+ *
+ * \param topology Initial stream topology to offer.
+ *                NULL if we are going to be the answerer.  We can always
+ *                update the local topology later if it turns out we need
+ *                to be the offerer.
+ * \param options SDP options for the duration of the session.
+ *
+ * \retval SDP state struct
+ * \retval NULL on failure
  */
-struct ast_sdp_state *ast_sdp_state_alloc(struct ast_stream_topology *streams,
+struct ast_sdp_state *ast_sdp_state_alloc(struct ast_stream_topology *topology,
 	struct ast_sdp_options *options);
 
 /*!
@@ -86,6 +96,8 @@ int ast_sdp_state_get_stream_connection_address(const struct ast_sdp_state *sdp_
  *
  * If this is called prior to receiving a remote SDP, then this will just mirror
  * the local configured endpoint capabilities.
+ *
+ * \note Cannot return NULL.  It is a BUG if it does.
  */
 const struct ast_stream_topology *ast_sdp_state_get_joint_topology(
 	const struct ast_sdp_state *sdp_state);
@@ -93,6 +105,7 @@ const struct ast_stream_topology *ast_sdp_state_get_joint_topology(
 /*!
  * \brief Get the local topology
  *
+ * \note Cannot return NULL.  It is a BUG if it does.
  */
 const struct ast_stream_topology *ast_sdp_state_get_local_topology(
 	const struct ast_sdp_state *sdp_state);
@@ -114,9 +127,10 @@ const struct ast_sdp_options *ast_sdp_state_get_options(
  * \retval NULL Failure
  *
  * \note
- * This function will allocate a new SDP with RTP instances if it has not already
- * been allocated.
- *
+ * This function will return the last local SDP created if one were
+ * previously requested for the current negotiation.  Otherwise it
+ * creates our SDP offer/answer depending on what role we are playing
+ * in the current negotiation.
  */
 const struct ast_sdp *ast_sdp_state_get_local_sdp(struct ast_sdp_state *sdp_state);
 
@@ -152,6 +166,7 @@ const void *ast_sdp_state_get_local_sdp_impl(struct ast_sdp_state *sdp_state);
  *
  * \retval 0 Success
  * \retval non-0 Failure
+ *         Use ast_sdp_state_is_offer_rejected() to see if the SDP offer was rejected.
  *
  * \since 15
  */
@@ -165,39 +180,72 @@ int ast_sdp_state_set_remote_sdp(struct ast_sdp_state *sdp_state, const struct a
  *
  * \retval 0 Success
  * \retval non-0 Failure
+ *         Use ast_sdp_state_is_offer_rejected() to see if the SDP offer was rejected.
  *
  * \since 15
  */
 int ast_sdp_state_set_remote_sdp_from_impl(struct ast_sdp_state *sdp_state, const void *remote);
 
 /*!
- * \brief Reset the SDP state and stream capabilities as if the SDP state had just been allocated.
+ * \brief Was the set remote offer rejected.
+ * \since 15.0.0
  *
  * \param sdp_state
- * \param remote The implementation's representation of an SDP.
  *
- * \retval 0 Success
+ * \retval 0 if not rejected.
+ * \retval non-zero if rejected.
+ */
+int ast_sdp_state_is_offer_rejected(struct ast_sdp_state *sdp_state);
+
+/*!
+ * \brief Are we the SDP offerer.
+ * \since 15.0.0
  *
- * \note
- * This is most useful for when a channel driver is sending a session refresh message
- * and needs to re-advertise its initial capabilities instead of the previously-negotiated
- * joint capabilities.
+ * \param sdp_state
  *
- * \since 15
+ * \retval 0 if we are not the offerer.
+ * \retval non-zero we are the offerer.
  */
-int ast_sdp_state_reset(struct ast_sdp_state *sdp_state);
+int ast_sdp_state_is_offerer(struct ast_sdp_state *sdp_state);
+
+/*!
+ * \brief Are we the SDP answerer.
+ * \since 15.0.0
+ *
+ * \param sdp_state
+ *
+ * \retval 0 if we are not the answerer.
+ * \retval non-zero we are the answerer.
+ */
+int ast_sdp_state_is_answerer(struct ast_sdp_state *sdp_state);
+
+/*!
+ * \brief Restart the SDP offer/answer negotiations.
+ *
+ * \param sdp_state
+ *
+ * \retval 0 Success
+ * \retval non-0 Failure
+ */
+int ast_sdp_state_restart_negotiations(struct ast_sdp_state *sdp_state);
 
 /*!
  * \brief Update the local stream topology on the SDP state.
  *
+ * \details
+ * Basically we are saving off any topology updates until we create the
+ * next SDP offer.  Repeated updates merge with the previous updated
+ * topology.
+ *
  * \param sdp_state
- * \param streams The new stream topology.
+ * \param topology The new stream topology.
  *
  * \retval 0 Success
+ * \retval non-0 Failure
  *
  * \since 15
  */
-int ast_sdp_state_update_local_topology(struct ast_sdp_state *sdp_state, struct ast_stream_topology *streams);
+int ast_sdp_state_update_local_topology(struct ast_sdp_state *sdp_state, struct ast_stream_topology *topology);
 
 /*!
  * \brief Set the local address (IP address) to use for connection addresses
@@ -231,7 +279,26 @@ int ast_sdp_state_set_connection_address(struct ast_sdp_state *sdp_state, int st
 
 /*!
  * \since 15.0.0
- * \brief Set a stream to be held or unheld
+ * \brief Set the global locally held state.
+ *
+ * \param sdp_state
+ * \param locally_held
+ */
+void ast_sdp_state_set_global_locally_held(struct ast_sdp_state *sdp_state, unsigned int locally_held);
+
+/*!
+ * \since 15.0.0
+ * \brief Get the global locally held state.
+ *
+ * \param sdp_state
+ *
+ * \returns locally_held
+ */
+unsigned int ast_sdp_state_get_global_locally_held(const struct ast_sdp_state *sdp_state);
+
+/*!
+ * \since 15.0.0
+ * \brief Set a stream to be held or unheld locally
  *
  * \param sdp_state
  * \param stream_index The stream to set the held value for
@@ -242,25 +309,37 @@ void ast_sdp_state_set_locally_held(struct ast_sdp_state *sdp_state,
 
 /*!
  * \since 15.0.0
- * \brief Set the UDPTL session parameters
+ * \brief Get whether a stream is locally held or not
  *
  * \param sdp_state
- * \param stream_index The stream to set the UDPTL session parameters for
- * \param params
+ * \param stream_index The stream to get the held state for
+ *
+ * \returns locally_held
  */
-void ast_sdp_state_set_t38_parameters(struct ast_sdp_state *sdp_state,
-	int stream_index, struct ast_control_t38_parameters *params);
+unsigned int ast_sdp_state_get_locally_held(const struct ast_sdp_state *sdp_state,
+	int stream_index);
 
 /*!
  * \since 15.0.0
- * \brief Get whether a stream is held or not
+ * \brief Get whether a stream is remotely held or not
  *
  * \param sdp_state
  * \param stream_index The stream to get the held state for
  *
- * \returns locally_held
+ * \returns remotely_held
  */
-unsigned int ast_sdp_state_get_locally_held(const struct ast_sdp_state *sdp_state,
+unsigned int ast_sdp_state_get_remotely_held(const struct ast_sdp_state *sdp_state,
 	int stream_index);
 
+/*!
+ * \since 15.0.0
+ * \brief Set the UDPTL session parameters
+ *
+ * \param sdp_state
+ * \param stream_index The stream to set the UDPTL session parameters for
+ * \param params
+ */
+void ast_sdp_state_set_t38_parameters(struct ast_sdp_state *sdp_state,
+	int stream_index, struct ast_control_t38_parameters *params);
+
 #endif /* _ASTERISK_SDP_STATE_H */
diff --git a/include/asterisk/stream.h b/include/asterisk/stream.h
index 00169a3f1..21af53fae 100644
--- a/include/asterisk/stream.h
+++ b/include/asterisk/stream.h
@@ -228,6 +228,17 @@ void ast_stream_set_state(struct ast_stream *stream, enum ast_stream_state state
 const char *ast_stream_state2str(enum ast_stream_state state);
 
 /*!
+ * \brief Convert a string to a stream state
+ *
+ * \param str The string to convert
+ *
+ * \return The stream state
+ *
+ * \since 15.0.0
+ */
+enum ast_stream_state ast_stream_str2state(const char *str);
+
+/*!
  * \brief Get the opaque stream data
  *
  * \param stream The media stream