Commit Graph

531 Commits

Author SHA1 Message Date
George Joseph
5ce084579f CI: Update jenkinsfiles with new Gerrit URLs
The recent upgrade of Gerrit to 2.16 elimiated referencing a
repository in a way the jenkinsfiles were relying on so
the URL references were changed to a more consistent and supported
format.

Change-Id: I2e8e3f213b9a96bb1b27665eca4a9a24bc49820e
2019-02-27 09:37:14 -07:00
George Joseph
bae3fd04c1 taskprocessor: Enable subsystems and overload by subsystem
To prevent one subsystem's taskprocessors from causing others
to stall, new capabilities have been added to taskprocessors.

* Any taskprocessor name that has a '/' will have the part
  before the '/' saved as its "subsystem".
  Examples:
  "sorcery/acl-0000006a" and "sorcery/aor-00000019"
  will be grouped to subsystem "sorcery".
  "pjsip/distributor-00000025" and "pjsip/distributor-00000026"
  will bn grouped to subsystem "pjsip".
  Taskprocessors with no '/' have an empty subsystem.

* When a taskprocessor enters high-water alert status and it
  has a non-empty subsystem, the subsystem alert count will
  be incremented.

* When a taskprocessor leaves high-water alert status and it
  has a non-empty subsystem, the subsystem alert count will be
  decremented.

* A new api ast_taskprocessor_get_subsystem_alert() has been
  added that returns the number of taskprocessors in alert for
  the subsystem.

* A new CLI command "core show taskprocessor alerted subsystems"
  has been added.

* A new unit test was addded.

REMINDER: The taskprocessor code itself doesn't take any action
based on high-water alerts or overloading.  It's up to taskprocessor
users to check and take action themselves.  Currently only the pjsip
distributor does this.

* A new pjsip/global option "taskprocessor_overload_trigger"
  has been added that allows the user to select the trigger
  mechanism the distributor uses to pause accepting new requests.
  "none": Don't pause on any overload condition.
  "global": Pause on ANY taskprocessor overload (the default and
  current behavior)
  "pjsip_only": Pause only on pjsip taskprocessor overloads.

* The core pjsip pool was renamed from "SIP" to "pjsip" so it can
  be properly grouped into the "pjsip" subsystem.

* stasis taskprocessor names were changed to "stasis" as the
  subsystem.

* Sorcery core taskprocessor names were changed to "sorcery" to
  match the object taskprocessors.

Change-Id: I8c19068bb2fc26610a9f0b8624bdf577a04fcd56
2019-02-20 10:23:26 -07:00
Joshua Colp
46df13e2d0 CI: Use tmpfs option to Docker instead of mount.
Some tests require Asterisk to execute scripts which
are stored in /tmp. When mount is used for tmpfs there
is no ability to allow scripts to be executed from
that location.

This change switches to using tmpfs which can be told
to allow executables to be run from /tmp.

Change-Id: I0e598ca2b76af1f7f2d29f0da7b1731a214a291a
2019-02-19 16:22:46 +00:00
Joshua Colp
9a6464e54f ci: Rerun unit tests when non-code changes occur.
This change makes it so that even if non-code changes
occur (such as commit message changing) unit tests
will still be run and result in a verification.

ASTERISK-28251

Change-Id: I6491fff7c93e5d5cd8e41054486968bf66c4f608
2019-02-08 13:09:38 +00:00
Friendly Automation
a03202e551 Merge "stasis: Allow filtering by formatter" into 13 2018-12-12 11:09:22 -06:00
George Joseph
21a5ad0fd9 stasis: Allow filtering by formatter
A subscriber can now indicate that it only wants messages
that have formatters of a specific type.  For instance,
manager can indicate that it only wants messages that have a
"to_ami" formatter.  You can combine this with the existing
filter for message type to get only messages with specific
formatters or messages of specific types.

ASTERISK-28186

Change-Id: Ifdb7a222a73b6b56c6bb9e4ee93dc8a394a5494c
2018-12-07 06:57:15 -07:00
George Joseph
30934a0145 CI: Various updates to buildAsterisk.sh
* Added ---no-configure, --no-menuselect, --no-make and --no-alembic
  options that prevent those actions from being performed.  Useful
  for testing and re-running portions of the build after fixing
  earlier failures.

* Added "set -e" to abort the script on command failure.
  Not sure why this wasn't there in the first place.

* Fixed a few echos that were redirecting to stderr when they shouldn't
  have been.

* Catch more alembic failures by actually trying to generate the SQL.

Change-Id: I9f395fa4e9254be7299e7c1014f1a13db78faffb
2018-12-05 08:37:45 -07:00
Chris-Savinovich
2e15657485 test_websocket_client.c: Disable websocket_client_create_and_connect test.
This test was occasionally failing, with:

  WARNING[5812]: http.c:1939 httpd_helper_thread: Failed to set
      TCP_NODELAY on HTTP connection: Bad file descriptor
  ERROR[5812]: iostream.c:91 ast_iostream_nonblock: Failed to get
      fcntl() flags for file descriptor: Bad file descriptor
  ERROR[5812]: iostream.c:569 ast_iostream_close: close() failed: Bad
      file descriptor

Disabled for now by making the test explicit only.

Change-Id: I778f6cbb6104c6b4e89737a2eaf1a9540888d351
2018-12-02 08:55:10 -05:00
George Joseph
922fbafda0 test_cel: Plug a few ref leaks
These are only a few of the leaks.  The large number of macros
and return paths in this file would make a weeks worth of work
to plug them all.

Change-Id: Ie2369fa944023d44767871c5c30974cb077ffb56
2018-11-26 17:19:51 -05:00
Corey Farrell
0a9904e1c6 astobj2: Eliminate usage of legacy ao2_container_alloc routine.
Replace usage of ao2_container_alloc with ao2_container_alloc_hash or
ao2_container_alloc_list.

ao2_container_alloc is now restricted to modules only and is being
removed from Asterisk 17.

Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088
2018-11-21 09:56:07 -05:00
Jenkins2
35a5d63195 Merge "CI: Get job timeouts from environment" into 13 2018-11-19 15:13:55 -06:00
George Joseph
d2397e643d CI: Get job timeouts from environment
The job timeouts were hard coded in the jenkinsfiles which
means changes had to go through gerrit.  Now they are taken
from the following environment variables (and their defaults) that
can be set in Jenkins configuration...

TIMEOUT_GATES =      "60 MINUTES"
TIMEOUT_DAILIES =    "3 HOURS"
TIMEOUT_REF_DEBUG =  "24 HOURS"
TIMEOUT_UNITTESTS =  "30 MINUTES"

Change-Id: I673a551c1780bf665a3bc160b245da574aa4bbab
2018-11-19 10:59:07 -07:00
Joshua Colp
e0b5d74e93 Merge "test_res_pjsip_scheduler: Fix possible write after free in scheduler_policy." into 13 2018-11-19 09:38:31 -06:00
George Joseph
0deaf81bff backtrace: Refactor ast_bt_get_symbols so it doesn't crash
We've been seeing crashes in libbfd when we attempt to generate
a stack trace from multiple threads.  It turns out that libbfd
is NOT thread-safe.  It can cache the bfd structure and give it to
multiple threads without protecting itself.  To get around this,
we've added a global mutex around the bfd functions and also have
refactored the use of those functions to be more efficient and
to provide more information about inlined functions.

Also added a few more tests to test_pbx.c.  One just calls
ast_assert() and the other calls ast_log_backtrace().  Neither are
run by default.

WARNING:  This change necessitated changing the return value of
ast_bt_get_symbols() from an array of strings to a VECTOR of
strings.  However, the use of this function outside Asterisk is not
likely.

ASTERISK-28140

Change-Id: I79d02862ddaa2423a0809caa4b3b85c128131621
2018-11-19 05:47:35 -07:00
George Joseph
7ec470c1ca CI: Add tmpfs to all jenkinsfiles
Change-Id: Ida29d70d48d5f39aabf0b25c66b51f79324a8cba
2018-11-18 09:38:40 -07:00
George Joseph
73efe86436 CI: Mount a tmpfs on /tmp for testsuite docker containers
Change-Id: I0566d81b0852f22066cd76d58eae5f1fda5602aa
2018-11-17 14:40:46 -07:00
George Joseph
a335f4c9ad CI: Pass work directory to runTestsuite
The testsuite can now use a user-specified work directory for
all it's temp files.  This allows the docker containers to use
a tmpfs backed directory for the temp files instead of it's
own write-layer image.

* runTestsuite.sh now accepts a --work-dir command line argument
  that gets exported as AST_WORK_DIR before running the testsuite.

* gates.jenkinsfile now specifies --work-dir to be
  <testsuite_dir>/astroot.

Since the Asterisk CI docker hosts now mount /srv/jenkins/workspace
on a tmpfs, asterisk should be compiled and the testsuite run all in
memory.

Change-Id: If5ee905a15821296c355bb84cda38950ad8edc45
2018-11-17 12:07:32 -07:00
George Joseph
97633c09c5 CI: Allow runUnittests to use 'expect' to run the tests
There seems to be a race condition between starting the asterisk
daemon and attempting to use 'asterisk -r' that can cause the
control socket file to not be created.  Since all of the Jenkins
slaves have 'expect' installed, the runUnittests script can use
it to start asterisk in the forground and issue the commands
interactively.  This is much more reliable and it can also make
startup errors more visible since they'll be in the Jenkins console
output.

If 'expect' isn't installed, the original daemon/asterisk -r
process is used.

Also added a "core show settings" before running the tests
and added "notice,warning,error" to the console log.

Change-Id: Idd656085f854afede813ac241b9e312b31358160
2018-11-16 08:03:25 -07:00
Corey Farrell
8e7a8e8734 test_res_pjsip_scheduler: Fix possible write after free in scheduler_policy.
It's possible for a 4th task to be spawned before we cancel.  This
results in a write to the already freed test_data1.  Wait long enough to
verify success of the cancelation before freeing test_data1.

Change-Id: I057e2fcbe97f8a175e50890be89c28c20490a20f
2018-11-13 12:02:35 -05:00
Corey Farrell
54a1fbe428 astobj2: Eliminate usage of legacy container allocation macros.
These macros have been documented as legacy for a long time but are
still used in new code because they exist.  Remove all references to:
* ao2_container_alloc_options
* ao2_t_container_alloc_options
* ao2_t_container_alloc

These macro's are still available for use but only in modules.  Only
ao2_container_alloc remains due to it's use in over 100 places.

Change-Id: I1a26258b5bf3deb081aaeed11a0baa175c933c7a
2018-10-19 17:32:58 -04:00
George Joseph
b7cb2c90e6 Merge "taskprocessor: Warn on unused result from pushing task." into 13 2018-10-19 08:40:18 -05:00
Richard Mudgett
ce2e04801a modules: Add missing run time module support levels.
Change-Id: I29b9dbfa4bbfc49f21eba356858e38b1d3041824
2018-10-17 14:42:07 -05:00
Corey Farrell
01716f70e6 taskprocessor: Warn on unused result from pushing task.
Add attribute_warn_unused_result to ast_taskprocessor_push,
ast_taskprocessor_push_local and ast_threadpool_push.  This will help
ensure we perform the necessary cleanup upon failure.

Change-Id: I7e4079bd7b21cfe52fb431ea79e41314520c3f6d
2018-10-17 08:27:29 -05:00
George Joseph
737f6aec2f Merge "CI: Disable binary modules when REF_DEBUG is enabled." into 13 2018-10-16 10:11:59 -05:00
Corey Farrell
348a23b187 CI: Disable binary modules when REF_DEBUG is enabled.
In Asterisk 13 binary modules are linked to the non-REF_DEBUG variants
of ao2 functions.  Cause false leaks to be reported when binary modules
are used.

Change-Id: I192b1b912ad20b6f46e51dcc0f6fcf2d6cef9e70
2018-10-10 04:46:14 -04:00
Corey Farrell
3be3d4a730 CI: Use bundled jansson if needed.
Use pkg-config to determine if jansson is installed, enabled bundled
version otherwise.

Change-Id: Ib555a8b72ff6f6925f9280ef035caa0b91ca4bd2
2018-10-09 02:32:06 -05:00
George Joseph
1311aeecfb CI: Fix missing () in gates.jenkinsfile
Change-Id: I2f252e0f8c7f1a6328438fbd2be5d6574b7dfa5b
2018-10-04 12:03:05 -06:00
Jenkins2
8f6d527349 Merge "CI: Add timestamps and timeouts to jenkinsfiles" into 13 2018-10-04 12:50:19 -05:00
George Joseph
2065fe0c04 CI: Add timestamps and timeouts to jenkinsfiles
Change-Id: Ide83574dc957bc1df28e30a69079140050dfc35f
2018-10-04 09:13:22 -06:00
George Joseph
d85100aed0 Merge "jenkins: Fix cleanup command redirection." into 13 2018-10-04 09:40:59 -05:00
George Joseph
2c0123aad8 Merge "CI: Use brace expansion instead of calling out to seq" into 13 2018-10-04 09:40:09 -05:00
Sean Bright
e71c1ba045 CI: Look up configured kernel.core_pattern sysctl
Change-Id: I8246a0147df8d821fbbcabc1db1887104b8bedc4
2018-10-03 16:53:29 -05:00
Corey Farrell
e1ce997437 jenkins: Fix cleanup command redirection.
Fix redirection to /dev/null of cleanup commands.  The '2' was being
interpreted as part of the command instead of part of the redirect.

Change-Id: I2e3a591b165e0288c4b82b9ef475fdfd5392a90a
2018-10-03 16:32:19 -05:00
Sean Bright
7c65723f91 CI: Use brace expansion instead of calling out to seq
Also make the shebang in publishAsteriskDocs.sh the first line.

Change-Id: I3fdd6f22e652e4fb5b5fe85df46fa34eb6d0cf08
2018-10-03 09:11:12 -05:00
Sean Bright
71e9a94673 CI: Use bindport instead of port in test http.conf
Change-Id: Ife9a6879da63a56e5b8348a2024eeed4e7b1615b
2018-10-03 09:59:52 -04:00
George Joseph
97f4290421 CI: Add --test-timeout option to runTestsuite.sh
The default is 600 seconds.
Also added timeouts to the *TestGroups.json files.

Change-Id: I8ab6a69e704b6a10f06a0e52ede02312a2b72fe0
2018-09-26 07:12:28 -06:00
George Joseph
0b88512a4d CI: Fix typo in testsuite git checkout
Change-Id: I30024515e5b00a5044fd39fbff27d818f016b719
2018-09-17 06:10:18 -06:00
George Joseph
52324ef1a1 CI: Use proper credentials for Security testsuite checkout
Can't do anonymous http checkout from Security-testsuite.
Need to use same credentials as the gerrit review checkout.

Change-Id: I87af68c995cb8926f5e87f9af245600d76984f05
2018-09-14 11:31:28 -06:00
Jenkins2
641e5b7e63 Merge "optional_api: Remove unused nonoptreq fields" into 13 2018-09-13 13:06:41 -05:00
Corey Farrell
d1f6a323a0 CI: Use .gitreview to default BRANCH_NAME.
This ensures that binary modules are avoided in the master branch even
if BRANCH_NAME is not set.

Change-Id: I79162d2063f22fa9d6b31fde4827ace2dd5bf0da
2018-09-12 19:12:07 -05:00
Walter Doekes
d226458c5b optional_api: Remove unused nonoptreq fields
As they're not actively used, they only grow stale. The moduleinfo field itself
is kept in Asterisk 13/15 for ABI compatibility.

ASTERISK-28046 #close

Change-Id: I8df66a7007f807840414bb348511a8c14c05a9fc
2018-09-12 19:15:33 +02:00
Corey Farrell
abb04622f7 Create --disable-binary-modules option.
This new option can be passed for ./configure or
./tests/CI/buildAsterisk.sh to prevent download/install of binary
modules.

Normally enabling the categories MENUSELECT_CODECS or MENUSELECT_RES
will result in binary modules being enabled even if the build target is
incompatible with those modules.  This includes CI scripts which enable
categories before disabling specific modules.

If more binary modules are offered in the future this will help avoid
accidentally downloading them if unwanted or incompatible.  Adding a
binary module will only require creating a new menuselect entry similar
to the existing ones, it will not be necessary to modify the CI scripts.

Change-Id: I6b1bd1c75a2e48f05b8b8a45b7a7a2d00a079166
2018-08-27 13:51:29 -05:00
Joshua Colp
9ce4708da9 Merge "CI: Disable res_odbc if REF_DEBUG is requested." into 13 2018-08-17 06:16:14 -05:00
George Joseph
33a1593fba CI: Final version of setting correct gerrit creds
Change-Id: I7729ecceedceb12f52bf18dae259846aa1d993b3
2018-08-16 12:28:03 -06:00
George Joseph
ab095ca4c6 CI: Yet Another stab at creds
Change-Id: I768850780d39151c5dd8e2bb1a4b24776727958e
2018-08-16 12:23:05 -06:00
George Joseph
b1e294c670 CI: Another stab at creds
Change-Id: I5b1dc3b354789e676c27a4e1fc42f5c3343cc994
2018-08-16 12:11:11 -06:00
George Joseph
45d096fe8d CI: Add https credentials to gerrit checkouts
If the review to be tested is in a project with restricted access,
we need to use the jenkins user's gerrit https credentials when we
do the checkout or the checkout will fail.

Change-Id: I9dc9994763c5ebfeb9f1cff60fb53f6902b7fd5f
2018-08-16 11:08:21 -06:00
Corey Farrell
7f4420a222 CI: Disable res_odbc if REF_DEBUG is requested.
This is for 13/15 only, res_odbc never unloads so it is impossible to do
REF_DEBUG testing with that module.

Change-Id: I2c1e32b80564e8fb08b6b5900ede6b5d304ebd10
2018-08-15 11:46:46 -05:00
Corey Farrell
f3a8bfff36 CI: Add support for coverage processing.
Enable coverage with `./tests/CI/buildAsterisk.sh --coverage`.  This
will cause Asterisk to be compiled with coverage support.  It also
initializes 'before' coverage data for all sources.  Accept
--tested-only to disable modules which are not run by any test.
Enabling coverage also sets tested-only true by default.  To build
everything with coverage enabled use `--coverage --tested-only=0`.

./tests/CI/processCoverage.sh is used to process the coverage and
generate HTML reports.

Fix utils/check_expr2 which failed to compiled with coverage enabled.

Add status output 5 times per stage of astobj2_test_perf to ensure
remote CLI does not timeout when compiled with coverage.  Remote CLI
disconnects if no output is received for 60 seconds.  When coverage is
enabled it takes about 70 seconds for my laptop to run the stages of
this test, so with the change a message is printed every 14 seconds.

Change-Id: I890f7d5665087426ad7d3e363187691b9afc2222
2018-08-08 11:06:54 -05:00
George Joseph
7f203d2bd8 CI: Add optional uninstall step before installing asterisk
Change-Id: I7dedf1e925eafc3a0adf01dd9dfbe44eb642aab7
2018-07-30 08:44:12 -05:00