Commit Graph

12 Commits

Author SHA1 Message Date
George Joseph
a39dd50436 ast_coredumper: Refactor the pid determination process
In order to get a dump of the running process, we need to find the
pid of the main asterisk process.  This can be tricky if there are
also instances of "asterisk -r" running or if an alternate location
for asterisk.conf was specified on the command line with the -C
option that also specified an alternation location for the pid file.

So now...

1. We find the asterisk executable with "which" or the --asterisk-bin
   command line option.
2. If there's only 1 process with an executable path that matches,
   we use that pid.  If not...
3. We try "<asterisk-bin> -rx 'core show settings'" and parse the
   output to find the pidfile, then read that for the pid.  If that
   didn't work...
4. We get a list of all the pids matching <asterisk-bin> and look
   in /proc/<pid>/cmdline for a -C argument and retry the "core show
   settings" using the same -C option.  We can't parse the output
   of "ps" to get the -C path because it may contain spaces.  The
   contents of /proc/<pid>/cmdline are delimited by NULLs.  For BSDs
   we may have to mount /proc first. :(

ASTERISK-28221
Reported by: Andrew Nagy

Change-Id: I8aa1f3f912f949df2b5348908803c636bde1d57c
2018-12-24 14:16:53 -05:00
George Joseph
e91cce7673 Merge "ast_coredumper: Remove .gdbinit file on exit" into 13 2018-10-04 09:42:44 -05:00
Sean Bright
4a19205ac7 ast_coredumper: Remove .gdbinit file on exit
Change-Id: I1297de78628773ca368e687c6f148bf74857cae9
2018-10-03 18:02:27 -04:00
George Joseph
856d0f1312 ast_coredumper: Don't use "declare -n"
Change-Id: I7ddfed4cd6549a0cd458e4d5cf9ac95d784de6cb
2018-10-03 14:29:31 -06:00
George Joseph
acfdfcd19e ast_coredumper: Fix output directory and variable precedence
The OUTPUTDIR variable in ast_debug_tools.conf.sample is now set
to "/tmp" instead of "/some/directory".

Variables set on the command line or that are already in the
environment now take predecence over variables set in the config files.

ASTERISK-27846
Reported by: Ted G

Change-Id: Ie8baec52d531886bf5849ec1d59bb59dc87ad387
2018-05-24 13:00:06 -06:00
George Joseph
68df9e609e ast_coredumper: Minor fixes
* Fix --tarball-config so the option doesn't cause an error.

* Allow for missing /etc/os-release.

* Add a sleep between tarballing the coredump and removing the
  output directory to allow the filesystem to settle.

Change-Id: I73e03b13087978bcc7f6bc9f45753990f82d9d77
2018-02-23 13:58:54 -07:00
George Joseph
062a4390ac ast_coredumper: Add ability to use directory other than /tmp
The OUTPUTDIR environment variable can now be set either in the
environment itself or in ast_debug_tools.conf.  If set, it's used
for all work products instead of /tmp.

Also added the --tarball-config option that includes the contents
of /etc/asterisk when either --tarball-coredumps or --tarball-results
are used.

Change-Id: I66b2553319df61caea5b313d084f51978f730b4c
2017-11-15 08:43:31 -07:00
Jenkins2
ae2687494a Merge "ast_coredumper: allow setting asterisk binary explicitly" into 13 2017-10-31 07:00:50 -05:00
Tzafrir Cohen
1b535d8457 ast_coredumper: allow setting asterisk binary explicitly
Adds an extra option, --asterisk-bin=<path> to ast_coredumper. If
provided, the binary given to gdb will be the parameter, rather than
asterisk from the PATH.

ASTERISK-27380 #close

Change-Id: I25f5b91eb75059b0fb2f142e468c26b283b0a9f3
2017-10-30 09:58:48 -05:00
George Joseph
3f7ad66245 ast_coredumper: Add gzipping of binaries and display of signal info
The --tarball-coredump option now creates a gzipped tarball of
coredumps processed, their results txt files and copies of
/etc/os-release, /usr/sbin/asterisk, /usr/lib(64)/libasterisk* and
/usr/lib(64)/asterisk as those files are needed to properly examine
the coredump.  The file will be named
/tmp/asterisk.<timestamp>.coredumps.tar.gz or
/tmp/asterisk-<uniqueid>.coredumps.tar.gz if --tarball-uniqueid was
specified.

Added dumps of *_siginfo to the top of the txt files so you can
tell what signal was invoked.

Change-Id: Ib9ee6d83592d4b1bc90cb3419a05376a88d1ded9
2017-10-25 11:26:06 -06:00
George Joseph
c628a7acac debug_utilities: Create ast_loggrabber
ast_loggrabber gathers log files from customizable search patterns,
optionally converts POSIX timestamps to a readable format and
tarballs the results.

Also a few tweaks were made to ast_coredumper.

Change-Id: I8bfe1468ada24c1344ce4abab7b002a59a659495
(cherry picked from commit 5fa1c56d7e)
2017-01-20 11:21:47 -06:00
George Joseph
47474cfd54 debug_utilities: Create the ast_coredumper utility
This utility allows easy manipulation of asterisk coredumps.

* Configurable search paths and patterns for existing coredumps
* Can generate a consistent coredump from the running instance
* Can dump the lock_infos table from a coredump
* Dumps backtraces to separate files...
  - thread apply 1 bt full -> <coredump>.thread1.txt
  - thread apply all bt -> <coredump>.brief.txt
  - thread apply all bt full -> <coredump>.full.txt
  - lock_infos table -> <coredump>.locks.txt
* Can tarball corefiles and optionally delete them after processing
* Can tarball results files and optionally delete them after processing
* Converts ':' in coredump and results file names '-' to facilitate
  uploading.  Jira for instance, won't accept file names with colons
  in them.

Tested on Fedora24+, Ubuntu14+, Debian6+, CentOS6+ and FreeBSD9+[1].

[1] For *BSDs, the "devel/gdb" package might have to be installed to
get a recent gdb.  The utility will check all instances of gdb
it finds in $PATH and if one isn't found that can run python, it
prints a friendly error.

Change-Id: I935d37ab9db85ef923f32b05579897f0893d33cd
2017-01-11 12:07:33 -06:00