mirror of
https://github.com/asterisk/asterisk.git
synced 2026-07-14 15:37:49 -07:00
George Joseph
23fa44d2eb
There is a LOT of work in this commit but the TL;DR is that it takes CEL processing from using 38% of the CPU instructions used by a call, which is more than that used by the call processing itself, down to less than 10% of the instructions. So here's the deal... cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom all shared one ugly trait...they all used ast_str_substitute_variables() or pbx_substitute_variables_helper() to resolve the dialplan functions used in their config files. Not only are they both extraordinarily expensive, they both require a dummy channel to be allocated and destroyed for each record written. For CDRs, that's not too bad because we only write one CDR per call. For CELs however, it's a disaster. As far as source code goes, the modules basically all did the same thing. Unfortunately, they did it badly. The config files simply contained long opaque strings which were intepreted by ast_str_substitute_variables() or pbx_substitute_variables_helper(), the very functions that ate all the instructions. This meant introducing a new "advanced" config format much like the advanced manager event filtering added to manager.conf in 2024. Fortunately however, if the legacy config was recognizable, we were able to parse it as an advanced config and gain the benefit. If not, then it goes the legacy, and very expensive, route. Given the commonality among the modules, instead of making the same improvements to 4 modules then trying to maintain them over time, a single module "res_cdrel_custom" was created that contains all of the common code. A few bonuses became possible in the process... * The cdr_custom and cel_custom modules now support JSON formatted output. * The cdr_sqlite_custom and cel_sqlite3_custom modules no longer have to share an Sqlite3 database. Summary of changes: A new module "res/res_cdrel_custom.c" has been created and the existing cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom modules are now just stubs that call the code in res_cdrel_custom. res_cdrel_custom contains: * A common configuration facility. * Getters for both CDR and CEL fields that share the same abstraction. * Formatters for all data types found in the ast_cdr and ast_event structures that share the same abstraction. * Common writers for the text file and database backends that, you guessed it, share the same abstraction. The result is that while there is certainly a net increase in the number of lines in the code base, most of it is in the configuration handling at load-time. The run-time instruction path length is now significanty shorter. ``` Scenario Instructions Latency ===================================================== CEL pre changes 38.49% 37.51% CEL Advanced 9.68% 6.06% CEL Legacy (auto-conv to adv) 9.95% 6.13% CEL Sqlite3 pre changes 39.41% 39.90% CEL Sqlite3 Advanced 25.68% 24.24% CEL Sqlite3 Legacy (auto conv) 25.88% 24.53% CDR pre changes 4.79% 2.95% CDR Advanced 0.79% 0.47% CDR Legacy (auto conv to adv) 0.86% 0.51% CDR Sqlite3 pre changes 4.47% 2.89% CEL Sqlite3 Advanced 2.16% 1.29% CEL Sqlite3 Legacy (auto conv) 2.19% 1.30% ``` Notes: * We only write one CDR per call but every little bit helps. * Sqlite3 still takes a fair amount of resources but the new config makes a decent improvement. * Legacy configs that we can't auto convert will still take the "pre changes" path. If you're interested in more implementation details, see the comments at the top of the res_cdrel_custom.c file. One minor fix to CEL is also included...Although TenantID was added to the ast_event structure, it was always rendered as an empty string. It's now properly rendered. UserNote: Significant performance improvements have been made to the cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom modules. See the new sample config files for those modules to see how to benefit from them.
118 lines
4.0 KiB
C
118 lines
4.0 KiB
C
/*
|
|
* Asterisk -- An open source telephony toolkit.
|
|
*
|
|
* Copyright (C) 2026, Sangoma Technologies Corporation
|
|
*
|
|
* George Joseph <gjoseph@sangoma.com>
|
|
*
|
|
* See http://www.asterisk.org for more information about
|
|
* the Asterisk project. Please do not directly contact
|
|
* any of the maintainers of this project for assistance;
|
|
* the project provides a web site, mailing lists and IRC
|
|
* channels for your use.
|
|
*
|
|
* This program is free software, distributed under the terms of
|
|
* the GNU General Public License Version 2. See the LICENSE file
|
|
* at the top of the source tree.
|
|
*/
|
|
|
|
/*!
|
|
* \file
|
|
* \author George Joseph <gjoseph@sangoma.com>
|
|
*
|
|
* \brief Protected header for the CDR and CEL Custom Backends
|
|
*
|
|
* \warning This file should be included only by CDR and CEL backends.
|
|
*
|
|
*/
|
|
|
|
#ifndef _RES_CDREL_CUSTOM_H
|
|
#define _RES_CDREL_CUSTOM_H
|
|
|
|
/*! \enum Backend Types */
|
|
enum cdrel_backend_type {
|
|
cdrel_backend_text = 0, /*!< Text file: DSV or JSON */
|
|
cdrel_backend_db, /*!< Database (currently only sqlite3) */
|
|
cdrel_backend_type_end, /*!< Sentinel */
|
|
};
|
|
|
|
/*! \enum Record Types */
|
|
enum cdrel_record_type {
|
|
cdrel_record_cdr = 0, /*!< Call Detail Records */
|
|
cdrel_record_cel, /*!< Channel Event Log records */
|
|
cdrel_record_type_end, /*!< Sentinel */
|
|
};
|
|
|
|
/*! \struct Forward declaration of a configuration */
|
|
struct cdrel_config;
|
|
|
|
/*! \struct Vector to hold all configurations in a config file */
|
|
AST_VECTOR(cdrel_configs, struct cdrel_config *);
|
|
|
|
/*!
|
|
* \brief Perform initial module load.
|
|
*
|
|
* Needs to be called by each "custom" module
|
|
*
|
|
* \param backend_type One of \ref cdrel_backend_type.
|
|
* \param record_type One of \ref cdrel_record_type.
|
|
* \param config_filename The config file name.
|
|
* \param backend_name The name to register the backend as.
|
|
* \param logging_cb The logging callback to register with CDR or CEL.
|
|
* \returns A pointer to a VECTOR or config objects read from the config file.
|
|
*/
|
|
struct cdrel_configs *cdrel_load_module(enum cdrel_backend_type backend_type,
|
|
enum cdrel_record_type record_type, const char *config_filename,
|
|
const char *backend_name, void *logging_cb);
|
|
|
|
/*!
|
|
* \brief Perform module reload.
|
|
*
|
|
* Needs to be called by each "custom" module
|
|
*
|
|
* \warning This function MUST be called with the module's config_lock held
|
|
* for writing to prevent reloads from happening while we're logging.
|
|
*
|
|
* \param backend_type One of \ref cdrel_backend_type.
|
|
* \param record_type One of \ref cdrel_record_type.
|
|
* \param configs A pointer to the VECTOR of config objects returned by \ref cdrel_load_module.
|
|
* \param config_filename The config file name.
|
|
* \retval AST_MODULE_LOAD_SUCCESS
|
|
* \retval AST_MODULE_LOAD_DECLINE
|
|
*/
|
|
int cdrel_reload_module(enum cdrel_backend_type backend_type, enum cdrel_record_type record_type,
|
|
struct cdrel_configs **configs, const char *config_filename);
|
|
|
|
/*!
|
|
* \brief Perform module unload.
|
|
*
|
|
* Needs to be called by each "custom" module
|
|
*
|
|
* \warning This function MUST be called with the module's config_lock held
|
|
* for writing to prevent the module from being unloaded while we're logging.
|
|
*
|
|
* \param backend_type One of \ref cdrel_backend_type.
|
|
* \param record_type One of \ref cdrel_record_type.
|
|
* \param configs A pointer to the VECTOR of config objects returned by \ref cdrel_load_module.
|
|
* \param backend_name The backend name to unregister.
|
|
* \retval 0 Success.
|
|
* \retval -1 Failure.
|
|
*/
|
|
int cdrel_unload_module(enum cdrel_backend_type backend_type, enum cdrel_record_type record_type,
|
|
struct cdrel_configs *configs, const char *backend_name);
|
|
|
|
/*!
|
|
* \brief Log a record. The module's \ref logging_cb must call this.
|
|
*
|
|
* \warning This function MUST be called with the module's config_lock held
|
|
* for reading to prevent reloads from happening while we're logging.
|
|
*
|
|
* \param configs A pointer to the VECTOR of config objects returned by \ref cdrel_load_module.
|
|
* \param data A pointer to an ast_cdr or ast_event object to log.
|
|
* \retval 0 Success.
|
|
* \retval -1 Failure.
|
|
*/
|
|
int cdrel_logger(struct cdrel_configs *configs, void *data);
|
|
|
|
#endif /* _RES_CDREL_CUSTOM_H */
|