Additional functionality to core crawler

[MikeMeliz]

Description

This PR introduces:

• Fulfills a long-standing TODO, of catching links with Regex
  • Ability to introduce your own patterns, into a separate file called regex_patterns.txt
• Better support for images, and the ability to have a separate output for them
• Better support for scripts, and the ability to have a separate output for them
• Introduced two new arguments, in preparation of bringing visualization:
  • --json : which generates a json file with the output, alongside txt output
  • --xml : which generates an xml file with the output, alongside txt output

Motivation and Context

Long standing TODO list, and closing the gaps with other solutions.
Addressing a minimal approach to #9.

How Has This Been Tested?

python3 torcrawl.py -w -u www.google.com -c -d 1 -p 1 -j
python3 torcrawl.py -w -u www.google.com -c -d 1 -p 1 -x

========================================== tests coverage ===========================================
__________________________ coverage: platform darwin, python 3.9.6-final-0 __________________________

Coverage XML written to file coverage.xml
======================================== 22 passed in 1.22s =========================================
  py: OK (1.44=setup[0.03]+cmd[1.41] seconds)
  congratulations :) (1.45 seconds)

Screenshots (if appropriate):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
78.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud