Adding ability to open trace files from fds #838
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add the ability to open scap files or inspectors from a fd. Makes it
possible to open other objects that return a fd and treat those objects
as if they were a file.
- Add new functions scap_open_offline_fd, scap_dump_open_fd that take
all the same arguments as their filename versions but take a fd
instead of a filename. scap_open_offline_int now takes a gzfile which
has been opened by one of the top-level functions.
- scap_open's args struct now contains a fd. If non-zero, it takes
precedence over filename and the fd is used to open a gzfile via
gzdopen().
- New function scap_event_reset_count sets the read event count to
0. Used when rewinding a scap file to start over.
- Add some more useful error messages to
scap_read_{proclist,userlist,fdlist} when reading more than expected.
- also add fdopen variants to sinsp_dumper/sinsp. They call the
appropriate scap fd variants. most of sinsp::open moves to sinsp::open_int().
- Add is_open() and written_events() methods to sinsp_dumper(). Useful
when writing files via fd to track progress.
- In sinsp::init(), after rewinding the file also reset the event
count.
- Add a sinsp::fseek() which just uses scap_fseek().
Add scap_dump_ftell, which maps to gztell, and sinsp_dumper::next_write_position, which uses scap_dump_ftell.
Add an argument to scap_dump_open_fd that controls whether or not to do a scan of /proc when initializing the dump file. In sinsp_dumper::fdopen, set this to false. Also write the thread table to the file, following recent changes to sinsp_dumper::open.
mstemm
pushed a commit
that referenced
this pull request
May 30, 2017
* Bline2 (#830) * generate a dump file when a process different from docker accesses the docker oom files * more debug info * slightly better logging filtering * better log filtering * a bit more filtering * minor changes * reenable falcobl * minor changes for better baseliner support * reduce noisiness * some baseline flags * cleanups * cleanups * cleanup * when dumping the sinsp thread table to disk, do it on a thread by thread basis to avoid excessive memory usage * temporarily remove the falcobl project * Fix parsing of cgroups with empty subsys * Dumpermem (#839) * when dumping the sinsp thread table to disk, do it on a thread by thread basis to avoid excessive memory usage * make sure to free the structures allocated in sinsp_thread_manager::dump_threads_to_file * fixed a buffer overflow in sinsp_threadinfo::args_to_scap * more buffer overflow protection in sinsp_threadinfo::env_to_scap and sinsp_threadinfo::cgroups_to_scap * make sure not to scan proc when crating a memory dumper * sinsp-generate the process list when dumping to memory * sinsp-generate the process list when dumping to memory * warnings and compile errors * cleanups * some optimizations in the code that writes the sinsp thread table to file * more thread table dump optimizations * Adding ability to open trace files from fds (#838) * Adding ability to open trace files from fds Add the ability to open scap files or inspectors from a fd. Makes it possible to open other objects that return a fd and treat those objects as if they were a file. - Add new functions scap_open_offline_fd, scap_dump_open_fd that take all the same arguments as their filename versions but take a fd instead of a filename. scap_open_offline_int now takes a gzfile which has been opened by one of the top-level functions. - scap_open's args struct now contains a fd. If non-zero, it takes precedence over filename and the fd is used to open a gzfile via gzdopen(). - New function scap_event_reset_count sets the read event count to 0. Used when rewinding a scap file to start over. - Add some more useful error messages to scap_read_{proclist,userlist,fdlist} when reading more than expected. - also add fdopen variants to sinsp_dumper/sinsp. They call the appropriate scap fd variants. most of sinsp::open moves to sinsp::open_int(). - Add is_open() and written_events() methods to sinsp_dumper(). Useful when writing files via fd to track progress. - In sinsp::init(), after rewinding the file also reset the event count. - Add a sinsp::fseek() which just uses scap_fseek(). * Add ftell/ability to read bytes written. Add scap_dump_ftell, which maps to gztell, and sinsp_dumper::next_write_position, which uses scap_dump_ftell. * Add ability to skip proc scan for fd opens. Add an argument to scap_dump_open_fd that controls whether or not to do a scan of /proc when initializing the dump file. In sinsp_dumper::fdopen, set this to false. Also write the thread table to the file, following recent changes to sinsp_dumper::open. * Add a filtercheck for process tty. (#847) This allows it to be used in falco rules. * Try changing the build environment (#849) * Try changing the build environment Adding sudo: required, as falco does, should allow the kernel driver to build. * Run sysdig without -N 65ef7a6 got rid of -N, so don't add that option. * Set close-on-exec flag for driver fds. (#851) While debugging some agent unit tests, I noticed that after a fork + exec the number of driver references was higher than expected. Some of these problems can be fixed within the tests, but it's also useful to set the close-on-exec flag for the driver fd so it's closed during an exec. Also fix some error messages that were referring to a hardcoded sysdig_probe instead of PROBE_DEVICE_NAME. * Use strtok_r instead of strtok. (#850) Ensures thread safety.
mstemm
pushed a commit
that referenced
this pull request
Jun 6, 2017
* Adding ability to open trace files from fds
Add the ability to open scap files or inspectors from a fd. Makes it
possible to open other objects that return a fd and treat those objects
as if they were a file.
- Add new functions scap_open_offline_fd, scap_dump_open_fd that take
all the same arguments as their filename versions but take a fd
instead of a filename. scap_open_offline_int now takes a gzfile which
has been opened by one of the top-level functions.
- scap_open's args struct now contains a fd. If non-zero, it takes
precedence over filename and the fd is used to open a gzfile via
gzdopen().
- New function scap_event_reset_count sets the read event count to
0. Used when rewinding a scap file to start over.
- Add some more useful error messages to
scap_read_{proclist,userlist,fdlist} when reading more than expected.
- also add fdopen variants to sinsp_dumper/sinsp. They call the
appropriate scap fd variants. most of sinsp::open moves to sinsp::open_int().
- Add is_open() and written_events() methods to sinsp_dumper(). Useful
when writing files via fd to track progress.
- In sinsp::init(), after rewinding the file also reset the event
count.
- Add a sinsp::fseek() which just uses scap_fseek().
* Add ftell/ability to read bytes written.
Add scap_dump_ftell, which maps to gztell, and
sinsp_dumper::next_write_position, which uses scap_dump_ftell.
* Add ability to skip proc scan for fd opens.
Add an argument to scap_dump_open_fd that controls whether or not to do
a scan of /proc when initializing the dump file.
In sinsp_dumper::fdopen, set this to false. Also write the thread table
to the file, following recent changes to sinsp_dumper::open.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add the ability to open scap files or inspectors from a fd. Makes it
possible to open other objects that return a fd and treat those objects
as if they were a file.
all the same arguments as their filename versions but take a fd
instead of a filename. scap_open_offline_int now takes a gzfile which
has been opened by one of the top-level functions.
precedence over filename and the fd is used to open a gzfile via
gzdopen().
0. Used when rewinding a scap file to start over.
scap_read_{proclist,userlist,fdlist} when reading more than expected.
appropriate scap fd variants. most of sinsp::open moves to sinsp::open_int().
when writing files via fd to track progress.
count.