The 0.1 release of Kiskadee  brought a minimal set of functionalities that permitted us to monitor some targets and analysis some packages, serving more as a proof of concept than as a real monitoring system. One of the limitations of the 0.1 release was that Kiskadee was saving all the analysis in a single field on the packages table. Every time that we need to generate a new analysis for a same package version, we appended the result on the analysis field. The next two code snippet was picked up from the Kiskadee source code.
reports =  ... with kiskadee.helpers.chdir(path): kiskadee.logger.debug('ANALYSIS: Unpacked!') analyzers = plugin.analyzers() for analyzer in analyzers: kiskadee.logger.debug('ANALYSIS: running %s ...' % analyzer) analysis = kiskadee.analyzers.run(analyzer, path) firehose_report = kiskadee.converter.to_firehose(analysis, analyzer) reports.append(str(firehose_report)) kiskadee.logger.debug('ANALYSIS: DONE running %s' % analyzer) return reports
all_analyses = '\n'.join(reports)
Note that we generate a analysis for each plugin analyzer, and append it in the reports array. In other part of the runner code we generate, using a join function, a single string with all the analysis made. A bad implementation to a issue that should be solved by changing the Kiskadee models in order to support several analysis for a same package version, and that was what we have done with the PR #24.With this pull request, Kiskadee will be able to save different analysis from a same package version made by different analyzers. A lot of refactoring was made in the Runner component, specially because was difficult to implement new tests for it. The responsibility of save a analysis was removed from the analyze method, and moved to the _save_source_analysis.
def _save_source_analysis(source_to_analysis, analysis, analyzer, session): if analysis is None: return None source_name = source_to_analysis['name'] source_version = source_to_analysis['version'] package = ( session.query(kiskadee.model.Package) .filter(kiskadee.model.Package.name == source_name).first() ) version_id = package.versions[-1].id _analysis = kiskadee.model.Analysis() try: _analyzer = session.query(kiskadee.model.Analyzer).\ filter(kiskadee.model.Analyzer.name == analyzer).first() _analysis.analyzer_id = _analyzer.id _analysis.version_id = version_id _analysis.raw = analysis session.add(_analysis) except Exception as err: kiskadee.logger.debug( "The required analyzer was not registered on Kiskadee" ) kiskadee.logger.debug(err)
With this pull request two new models was added to Kiskadee, analysis and analyzers. The Diagram 1 presents the current database structure of Kiskadee.
With the PR #24 we will fix some other issues:
- Docker not run properly on the Jenkins VM. Issue #23
- Docker and selinux are not good friends so we have to change the way that we were creating a container volume to run the analyzers.
- CI integration. Issue #7
- We now have a Jenkins instance running our tests for every push made to the pagure repository. In the future we want to add a continuous deploy to every code that be merged on master branch.