--- /dev/null
+vendor
+/yukari
--- /dev/null
+# v0.2.5 - 2024.03.24
+* Rename `config.readConfig` to `config.ReadConfig`
+* Assume default values if no `yukari.ini(5)` is loaded
+
+# v0.2.4 - 2024.03.24
+* Replace invalid favicon with one sourced from [here](https://en.touhouwiki.net/wiki/File:Th123YukariSigil.png), as well as using `//go:embed` for it
+* Add rc.d files for FreeBSD and OpenBSD, respectively
+
+# v0.2.3 - 2024.03.21
+* Document the configuration file format, which is INI-style (which is compatible to the old format in the codebase, though it's now called as `config.Config.<key>`)
+* Manual page has been rewritten (using `mdoc(7)`)
+* 'YukariSukima' is an incorrect transliteration, use 'Yukari no Sukima' to indicate possession/ownership
+* Remove the 'proxified and sanitized view' text as it should already be obvious
+* The font family used earlier is horrible, changed it to `sans-serif`
+* Bump required Go toolchain version to 1.16 in order to use `//go:embed`
+* Rename some all-uppercase constants/variables to camelCase (I think?), also rename CLIENT to Gap (lol)
+
+# v0.2.1 - 2023.08.26
+Applied some suggestions from the [issue tracker](https://github.com/asciimoo/morty/issues), and rebrand this fork.
+
+# v0.2.0 - 2018.05.28
+
+Man page added
+
+# v0.1.0 - 2018.01.30
+
+Initial release
--- /dev/null
+ GNU AFFERO GENERAL PUBLIC LICENSE
+ Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+ The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+ Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+ A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+ The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+ An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ TERMS AND CONDITIONS
+
+ 0. Definitions.
+
+ "This License" refers to version 3 of the GNU Affero General Public License.
+
+ "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+ "The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+ To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy. The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+ A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+ To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+ To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+ An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+ 1. Source Code.
+
+ The "source code" for a work means the preferred form of the work
+for making modifications to it. "Object code" means any non-source
+form of a work.
+
+ A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+ The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+ The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+ The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+ The Corresponding Source for a work in source code form is that
+same work.
+
+ 2. Basic Permissions.
+
+ All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+ You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force. You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright. Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+ Conveying under any other circumstances is permitted solely under
+the conditions stated below. Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+ No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+ When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+ 4. Conveying Verbatim Copies.
+
+ You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+ You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+ 5. Conveying Modified Source Versions.
+
+ You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+ a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+
+ b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under section
+ 7. This requirement modifies the requirement in section 4 to
+ "keep intact all notices".
+
+ c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+
+ d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+ A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+ 6. Conveying Non-Source Forms.
+
+ You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+ a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+
+ b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the
+ Corresponding Source from a network server at no charge.
+
+ c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+
+ d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+
+ e) Convey the object code using peer-to-peer transmission, provided
+ you inform other peers where the object code and Corresponding
+ Source of the work are being offered to the general public at no
+ charge under subsection 6d.
+
+ A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+ A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling. In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage. For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product. A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+ "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source. The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+ If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+ The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed. Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+ Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+ 7. Additional Terms.
+
+ "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+ When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+ Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+ a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+
+ b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+
+ c) Prohibiting misrepresentation of the origin of that material, or
+ requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+
+ d) Limiting the use for publicity purposes of names of licensors or
+ authors of the material; or
+
+ e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+
+ f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions of
+ it) with contractual assumptions of liability to the recipient, for
+ any liability that these contractual assumptions directly impose on
+ those licensors and authors.
+
+ All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+ If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+ Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+ 8. Termination.
+
+ You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+ However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+ Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+ Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+ 9. Acceptance Not Required for Having Copies.
+
+ You are not required to accept this License in order to receive or
+run a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+ 10. Automatic Licensing of Downstream Recipients.
+
+ Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+ An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+ You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+ 11. Patents.
+
+ A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+ A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+ Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+ In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+ If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+ If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+ A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License. You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+ Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+ 12. No Surrender of Others' Freedom.
+
+ If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all. For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+ 13. Remote Network Interaction; Use with the GNU General Public License.
+
+ Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software. This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+ Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+ 14. Revised Versions of this License.
+
+ The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time. Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+ If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+ Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+ 15. Disclaimer of Warranty.
+
+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. Limitation of Liability.
+
+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+ 17. Interpretation of Sections 15 and 16.
+
+ If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+ If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+ You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<http://www.gnu.org/licenses/>.
--- /dev/null
+GO ?= go
+RM ?= rm
+GOFLAGS ?= -v -mod=vendor
+PREFIX ?= /usr/local
+BINDIR ?= bin
+MANDIR ?= share/man
+MKDIR ?= mkdir
+CP ?= cp
+SYSCONFDIR ?= /etc
+
+VERSION = `git describe --abbrev=0 --tags 2>/dev/null || echo "$VERSION"`
+COMMIT = `git rev-parse --short HEAD || echo "$COMMIT"`
+BRANCH = `git rev-parse --abbrev-ref HEAD`
+BUILD = `git show -s --pretty=format:%cI`
+
+GOARCH ?= amd64
+GOOS ?= linux
+
+all: yukari
+
+yukari: vendor
+ env GOARCH=${GOARCH} GOOS=${GOOS} ${GO} build ${GOFLAGS} ./cmd/yukari
+clean:
+ ${RM} -f yukari
+install:
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${BINDIR}
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man5
+
+ ${CP} -f yukari ${DESTDIR}${PREFIX}/${BINDIR}
+ ${CP} -f yukari.1 ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${CP} -f yukari.ini.5 ${DESTDIR}${PREFIX}/${MANDIR}/man5
+test:
+ go test
+bench:
+ go test -benchmem -bench .
+vendor:
+ go mod vendor
+.PHONY: yukari clean install
--- /dev/null
+# Yukari's Gap
+Web content sanitizer proxy as a service[^1], fork of [MortyProxy](https://github.com/asciimoo/morty) with some suggestions from the issue tracker applied, named after [the youkai you shouldn't ever come near](https://en.touhouwiki.net/wiki/Yukari_Yakumo)
+
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes. It also replaces external resource references to prevent third party information leaks.
+
+The main goal of this tool is to provide a result proxy for [searx](https://asciimoo.github.com/searx/), but it can be used as a standalone sanitizer service too.
+
+Features:
+
+* HTML sanitization
+* Rewrites HTML/CSS external references to locals
+* JavaScript blocking
+* No Cookies forwarded
+* No Referrers
+* No Caching/Etag
+* Supports GET/POST forms and IFrames
+* Optional HMAC URL verifier key to prevent service abuse
+
+## Installation and setup
+Requirement: Go version 1.16 or higher (thus making it incompatible with MortyProxy's own requirement, but also to use `go embed`)
+
+```
+$ go install marisa.chaotic.ninja/yukari/cmd/yukari@latest
+$ "$GOPATH/bin/yukari" --help
+```
+### Usage
+See `yukari(1)`
+
+### Test
+
+```
+$ make test
+```
+
+### Benchmark
+
+```
+$ make bench
+```
+
+## Bugs
+Bugs or suggestions? Mail [yukari-dev@chaotic.ninja](mailto:yukari-dev@chaotic.ninja)
+
+---
+
+[^1]: or WCPaaS, mind you, also I didn't come up with that, it was already there when I arrived
--- /dev/null
+package main
+
+import (
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+var ALLOWED_CONTENTTYPE_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // html
+ contenttype.NewFilterEquals("text", "html", ""),
+ contenttype.NewFilterEquals("application", "xhtml", "xml"),
+ // css
+ contenttype.NewFilterEquals("text", "css", ""),
+ // images
+ contenttype.NewFilterEquals("image", "gif", ""),
+ contenttype.NewFilterEquals("image", "png", ""),
+ contenttype.NewFilterEquals("image", "jpeg", ""),
+ contenttype.NewFilterEquals("image", "pjpeg", ""),
+ contenttype.NewFilterEquals("image", "webp", ""),
+ contenttype.NewFilterEquals("image", "tiff", ""),
+ contenttype.NewFilterEquals("image", "vnd.microsoft.icon", ""),
+ contenttype.NewFilterEquals("image", "bmp", ""),
+ contenttype.NewFilterEquals("image", "x-ms-bmp", ""),
+ contenttype.NewFilterEquals("image", "x-icon", ""),
+ contenttype.NewFilterEquals("image", "svg", "xml"),
+ // fonts
+ contenttype.NewFilterEquals("application", "font-otf", ""),
+ contenttype.NewFilterEquals("application", "font-ttf", ""),
+ contenttype.NewFilterEquals("application", "font-woff", ""),
+ contenttype.NewFilterEquals("application", "vnd.ms-fontobject", ""),
+})
+
+var ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // texts
+ contenttype.NewFilterEquals("text", "csv", ""),
+ contenttype.NewFilterEquals("text", "tab-separated-values", ""),
+ contenttype.NewFilterEquals("text", "plain", ""),
+ // API
+ contenttype.NewFilterEquals("application", "json", ""),
+ // Documents
+ contenttype.NewFilterEquals("application", "x-latex", ""),
+ contenttype.NewFilterEquals("application", "pdf", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.text", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.spreadsheet", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.presentation", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.graphics", ""),
+ // Compressed archives
+ contenttype.NewFilterEquals("application", "zip", ""),
+ contenttype.NewFilterEquals("application", "gzip", ""),
+ contenttype.NewFilterEquals("application", "x-compressed", ""),
+ contenttype.NewFilterEquals("application", "x-gtar", ""),
+ contenttype.NewFilterEquals("application", "x-compress", ""),
+ // Generic binary
+ contenttype.NewFilterEquals("application", "octet-stream", ""),
+})
+
+var ALLOWED_CONTENTTYPE_PARAMETERS map[string]bool = map[string]bool{
+ "charset": true,
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "crypto/hmac"
+ "crypto/sha256"
+ _ "embed"
+ "encoding/base64"
+ "encoding/hex"
+ "errors"
+ "flag"
+ "fmt"
+ "html/template"
+ "io"
+ "log"
+ "mime"
+ "net/url"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "time"
+ "unicode/utf8"
+
+ "github.com/valyala/fasthttp"
+ "github.com/valyala/fasthttp/fasthttpproxy"
+ "golang.org/x/net/html"
+ "golang.org/x/net/html/charset"
+ "golang.org/x/text/encoding"
+
+ "marisa.chaotic.ninja/yukari"
+ "marisa.chaotic.ninja/yukari/config"
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+const (
+ STATE_DEFAULT int = 0
+ STATE_IN_STYLE int = 1
+ STATE_IN_NOSCRIPT int = 2
+)
+
+const MaxRedirectCount = 5
+
+var Gap *fasthttp.Client = &fasthttp.Client{
+ MaxResponseBodySize: 10 * 1024 * 1024, // 10M
+ ReadBufferSize: 16 * 1024, // 16K
+}
+
+var cssURLRegex *regexp.Regexp = regexp.MustCompile("url\\((['\"]?)[ \\t\\f]*([\u0009\u0021\u0023-\u0026\u0028\u002a-\u007E]+)(['\"]?)\\)?")
+
+type Proxy struct {
+ Key []byte
+ RequestTimeout time.Duration
+ FollowRedirect bool
+}
+
+type RequestConfig struct {
+ Key []byte
+ BaseURL *url.URL
+ BodyInjected bool
+}
+
+type HTMLBodyExtParam struct {
+ BaseURL string
+ HasYukariKey bool
+ URLParamName string
+}
+
+type HTMLFormExtParam struct {
+ BaseURL string
+ YukariHash string
+ URLParamName string
+ HashParamName string
+}
+type HTMLMainPageFormParam struct {
+ URLParamName string
+}
+
+var htmlFormExtension *template.Template
+var htmlBodyExtension *template.Template
+var htmlMainPageForm *template.Template
+
+//go:embed templates/yukari_content_type.html
+var htmlHeadContentType string
+//go:embed templates/yukari_start.html
+var htmlPageStart string
+//go:embed templates/yukari_stop.html
+var htmlPageStop string
+//go:embed favicon.ico
+var faviconBytes []byte
+
+func init() {
+ var err error
+ htmlFormExtension, err = template.New("html_form_extension").Parse(
+ `<input type="hidden" name="yukariurl" value="{{.BaseURL}}" />{{if .YukariHash}}<input type="hidden" name="yukarihash" value="{{.YukariHash}}" />{{end}}`)
+ if err != nil {
+ panic(err)
+ }
+ htmlBodyExtension, err = template.New("html_body_extension").Parse(`
+<div id="yukariheader">
+ <form method="get">
+ <span><a href="/">Yukari's Gap</a></span>
+ <input type="url" value="{{.BaseURL}}" name="{{.URLParamName}}" {{if .HasYukariKey }}readonly="true"{{end}} />
+ </form>
+</div>
+<style>
+body{ position: absolute !important; top: 42px !important; left: 0 !important; right: 0 !important; bottom: 0 !important; }
+#yukariheader { position: fixed; margin: 0; box-sizing: border-box; -webkit-box-sizing: border-box; top: 0; left: 0; right: 0; z-index: 2147483647 !important; font-size: 12px; line-height: normal; border-width: 0px 0px 2px 0; border-style: solid; border-color: #9826FF; background: #33004A; padding: 4px; color: #D881FF; height: 42px; }
+#yukariheader * { padding: 0; margin: 0; }
+#yukariheader p { padding: 0 0 0.7em 0; display: block; }
+#yukariheader a { color: #8934DB; font-weight: bold; display: inline; }
+#yukariheader label { text-align: right; cursor: pointer; position: fixed; right: 4px; top: 4px; display: block; color: #444; }
+#yukariheader > form > span { font-size: 24px; font-weight: bold; margin-right: 20px; margin-left: 20px; }
+#yukariheader input[type=url] { width: 50%; padding: 4px; font-size: 16px; }
+</style>
+`)
+ if err != nil {
+ panic(err)
+ }
+ htmlMainPageForm, err = template.New("html_main_page_form").Parse(`
+ <form action="post">
+ Visit url: <input placeholder="https://url.." name="{{.URLParamName}}" autofocus />
+ <input type="submit" value="go" />
+ </form>`)
+ if err != nil {
+ panic(err)
+ }
+}
+
+func (p *Proxy) RequestHandler(ctx *fasthttp.RequestCtx) {
+
+ if appRequestHandler(ctx) {
+ return
+ }
+
+ requestHash := popRequestParam(ctx, []byte(config.Config.HashParameter))
+
+ requestURI := popRequestParam(ctx, []byte(config.Config.UrlParameter))
+
+ if requestURI == nil {
+ p.serveMainPage(ctx, 200, nil)
+ return
+ }
+
+ if p.Key != nil {
+ if !verifyRequestURI(requestURI, requestHash, p.Key) {
+ // HTTP status code 403 : Forbidden
+ error_message := fmt.Sprintf(`invalid "%s" parameter. hint: Hash URL Parameter`, config.Config.HashParameter)
+ p.serveMainPage(ctx, 403, errors.New(error_message))
+ return
+ }
+ }
+
+ requestURIQuery := ctx.QueryArgs().QueryString()
+ if len(requestURIQuery) > 0 {
+ if bytes.ContainsRune(requestURI, '?') {
+ requestURI = append(requestURI, '&')
+ } else {
+ requestURI = append(requestURI, '?')
+ }
+ requestURI = append(requestURI, requestURIQuery...)
+ }
+
+ p.ProcessUri(ctx, string(requestURI), 0)
+}
+
+func (p *Proxy) ProcessUri(ctx *fasthttp.RequestCtx, requestURIStr string, redirectCount int) {
+ parsedURI, err := url.Parse(requestURIStr)
+
+ if err != nil {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+
+ if parsedURI.Scheme == "" {
+ requestURIStr = "https://" + requestURIStr
+ parsedURI, err = url.Parse(requestURIStr)
+ if err != nil {
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+ }
+
+ // Serve an intermediate page for protocols other than HTTP(S)
+ if (parsedURI.Scheme != "http" && parsedURI.Scheme != "https") || strings.HasSuffix(parsedURI.Host, ".onion") || strings.HasSuffix(parsedURI.Host, ".i2p") {
+ p.serveExitYukariPage(ctx, parsedURI)
+ return
+ }
+
+ req := fasthttp.AcquireRequest()
+ defer fasthttp.ReleaseRequest(req)
+ req.SetConnectionClose()
+
+ if config.Config.Debug {
+ log.Println(string(ctx.Method()), requestURIStr)
+ }
+
+ req.SetRequestURI(requestURIStr)
+ req.Header.SetUserAgentBytes([]byte("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"))
+
+ resp := fasthttp.AcquireResponse()
+ defer fasthttp.ReleaseResponse(resp)
+
+ req.Header.SetMethodBytes(ctx.Method())
+ if ctx.IsPost() || ctx.IsPut() {
+ req.SetBody(ctx.PostBody())
+ }
+
+ err = Gap.DoTimeout(req, resp, p.RequestTimeout)
+
+ if err != nil {
+ if err == fasthttp.ErrTimeout {
+ // HTTP status code 504 : Gateway Time-Out
+ p.serveMainPage(ctx, 504, err)
+ } else {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ }
+ return
+ }
+
+ if resp.StatusCode() != 200 {
+ switch resp.StatusCode() {
+ case 301, 302, 303, 307, 308:
+ loc := resp.Header.Peek("Location")
+ if loc != nil {
+ if p.FollowRedirect && ctx.IsGet() {
+ // GET method: Yukari follows the redirect
+ if redirectCount < MaxRedirectCount {
+ if config.Config.Debug {
+ log.Println("follow redirect to", string(loc))
+ }
+ p.ProcessUri(ctx, string(loc), redirectCount+1)
+ } else {
+ p.serveMainPage(ctx, 310, errors.New("Too many redirects"))
+ }
+ return
+ } else {
+ // Other HTTP methods: Yukari does NOT follow the redirect
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ url, err := rc.ProxifyURI(loc)
+ if err == nil {
+ ctx.SetStatusCode(resp.StatusCode())
+ ctx.Response.Header.Add("Location", url)
+ if config.Config.Debug {
+ log.Println("redirect to", string(loc))
+ }
+ return
+ }
+ }
+ }
+ }
+ error_message := fmt.Sprintf("invalid response: %d (%s)", resp.StatusCode(), requestURIStr)
+ p.serveMainPage(ctx, resp.StatusCode(), errors.New(error_message))
+ return
+ }
+
+ contentTypeBytes := resp.Header.Peek("Content-Type")
+
+ if contentTypeBytes == nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ contentTypeString := string(contentTypeBytes)
+
+ // decode Content-Type header
+ contentType, error := contenttype.ParseContentType(contentTypeString)
+ if error != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ // content-disposition
+ contentDispositionBytes := ctx.Request.Header.Peek("Content-Disposition")
+
+ // check content type
+ if !ALLOWED_CONTENTTYPE_FILTER(contentType) {
+ // it is not a usual content type
+ if ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER(contentType) {
+ // force attachment for allowed content type
+ contentDispositionBytes = contentDispositionForceAttachment(contentDispositionBytes, parsedURI)
+ } else {
+ // deny access to forbidden content type
+ // HTTP status code 403 : Forbidden
+ p.serveMainPage(ctx, 403, errors.New("forbidden content type "+parsedURI.String()))
+ return
+ }
+ }
+
+ // HACK : replace */xhtml by text/html
+ if contentType.SubType == "xhtml" {
+ contentType.TopLevelType = "text"
+ contentType.SubType = "html"
+ contentType.Suffix = ""
+ }
+
+ // conversion to UTF-8
+ var responseBody []byte
+
+ if contentType.TopLevelType == "text" {
+ e, ename, _ := charset.DetermineEncoding(resp.Body(), contentTypeString)
+ if (e != encoding.Nop) && (!strings.EqualFold("utf-8", ename)) {
+ responseBody, err = e.NewDecoder().Bytes(resp.Body())
+ if err != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, err)
+ return
+ }
+ } else {
+ responseBody = resp.Body()
+ }
+ // update the charset or specify it
+ contentType.Parameters["charset"] = "UTF-8"
+ } else {
+ responseBody = resp.Body()
+ }
+
+ //
+ contentType.FilterParameters(ALLOWED_CONTENTTYPE_PARAMETERS)
+
+ // set the content type
+ ctx.SetContentType(contentType.String())
+
+ // output according to MIME type
+ switch {
+ case contentType.SubType == "css" && contentType.Suffix == "":
+ sanitizeCSS(&RequestConfig{Key: p.Key, BaseURL: parsedURI}, ctx, responseBody)
+ case contentType.SubType == "html" && contentType.Suffix == "":
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ sanitizeHTML(rc, ctx, responseBody)
+ if !rc.BodyInjected {
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+ default:
+ if contentDispositionBytes != nil {
+ ctx.Response.Header.AddBytesV("Content-Disposition", contentDispositionBytes)
+ }
+ ctx.Write(responseBody)
+ }
+}
+
+// force content-disposition to attachment
+func contentDispositionForceAttachment(contentDispositionBytes []byte, url *url.URL) []byte {
+ var contentDispositionParams map[string]string
+
+ if contentDispositionBytes != nil {
+ var err error
+ _, contentDispositionParams, err = mime.ParseMediaType(string(contentDispositionBytes))
+ if err != nil {
+ contentDispositionParams = make(map[string]string)
+ }
+ } else {
+ contentDispositionParams = make(map[string]string)
+ }
+
+ _, fileNameDefined := contentDispositionParams["filename"]
+ if !fileNameDefined {
+ // TODO : sanitize filename
+ contentDispositionParams["fileName"] = filepath.Base(url.Path)
+ }
+
+ return []byte(mime.FormatMediaType("attachment", contentDispositionParams))
+}
+
+func appRequestHandler(ctx *fasthttp.RequestCtx) bool {
+ // serve robots.txt
+ if bytes.Equal(ctx.Path(), []byte("/robots.txt")) {
+ ctx.SetContentType("text/plain")
+ ctx.Write([]byte("User-Agent: *\nDisallow: /\n"))
+ return true
+ }
+
+ // server favicon.ico
+ if bytes.Equal(ctx.Path(), []byte("/favicon.ico")) {
+ ctx.SetContentType("image/vnd.microsoft.icon")
+ ctx.Write(faviconBytes)
+ return true
+ }
+
+ return false
+}
+
+func popRequestParam(ctx *fasthttp.RequestCtx, paramName []byte) []byte {
+ param := ctx.QueryArgs().PeekBytes(paramName)
+
+ if param == nil {
+ param = ctx.PostArgs().PeekBytes(paramName)
+ ctx.PostArgs().DelBytes(paramName)
+ }
+ ctx.QueryArgs().DelBytes(paramName)
+
+ return param
+}
+
+func sanitizeCSS(rc *RequestConfig, out io.Writer, css []byte) {
+ // TODO
+
+ urlSlices := cssURLRegex.FindAllSubmatchIndex(css, -1)
+
+ if urlSlices == nil {
+ out.Write(css)
+ return
+ }
+
+ startIndex := 0
+
+ for _, s := range urlSlices {
+ urlStart := s[4]
+ urlEnd := s[5]
+
+ if uri, err := rc.ProxifyURI(css[urlStart:urlEnd]); err == nil {
+ out.Write(css[startIndex:urlStart])
+ out.Write([]byte(uri))
+ startIndex = urlEnd
+ } else if config.Config.Debug {
+ log.Println("cannot proxify css uri:", string(css[urlStart:urlEnd]))
+ }
+ }
+ if startIndex < len(css) {
+ out.Write(css[startIndex:len(css)])
+ }
+}
+
+func sanitizeHTML(rc *RequestConfig, out io.Writer, htmlDoc []byte) {
+ r := bytes.NewReader(htmlDoc)
+ decoder := html.NewTokenizer(r)
+ decoder.AllowCDATA(true)
+
+ unsafeElements := make([][]byte, 0, 8)
+ state := STATE_DEFAULT
+ for {
+ token := decoder.Next()
+ if token == html.ErrorToken {
+ err := decoder.Err()
+ if err != io.EOF {
+ log.Println("failed to parse HTML")
+ }
+ break
+ }
+
+ if len(unsafeElements) == 0 {
+
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, hasAttrs := decoder.TagName()
+ safe := !inArray(tag, UNSAFE_ELEMENTS)
+ if !safe {
+ if token != html.SelfClosingTagToken {
+ var unsafeTag []byte = make([]byte, len(tag))
+ copy(unsafeTag, tag)
+ unsafeElements = append(unsafeElements, unsafeTag)
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("base")) {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ if bytes.Equal(attrName, []byte("href")) {
+ parsedURI, err := url.Parse(string(attrValue))
+ if err == nil {
+ rc.BaseURL = parsedURI
+ }
+ }
+ if !moreAttr {
+ break
+ }
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("noscript")) {
+ state = STATE_IN_NOSCRIPT
+ break
+ }
+ var attrs [][][]byte
+ if hasAttrs {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ attrs = append(attrs, [][]byte{
+ attrName,
+ attrValue,
+ []byte(html.EscapeString(string(attrValue))),
+ })
+ if !moreAttr {
+ break
+ }
+ }
+ }
+ if bytes.Equal(tag, []byte("link")) {
+ sanitizeLinkTag(rc, out, attrs)
+ break
+ }
+
+ if bytes.Equal(tag, []byte("meta")) {
+ sanitizeMetaTag(rc, out, attrs)
+ break
+ }
+
+ fmt.Fprintf(out, "<%s", tag)
+
+ if hasAttrs {
+ sanitizeAttrs(rc, out, attrs)
+ }
+
+ if token == html.SelfClosingTagToken {
+ fmt.Fprintf(out, " />")
+ } else {
+ fmt.Fprintf(out, ">")
+ if bytes.Equal(tag, []byte("style")) {
+ state = STATE_IN_STYLE
+ }
+ }
+
+ if bytes.Equal(tag, []byte("head")) {
+ fmt.Fprintf(out, htmlHeadContentType)
+ }
+
+ if bytes.Equal(tag, []byte("form")) {
+ var formURL *url.URL
+ for _, attr := range attrs {
+ if bytes.Equal(attr[0], []byte("action")) {
+ formURL, _ = url.Parse(string(attr[1]))
+ formURL = mergeURIs(rc.BaseURL, formURL)
+ break
+ }
+ }
+ if formURL == nil {
+ formURL = rc.BaseURL
+ }
+ urlStr := formURL.String()
+ var key string
+ if rc.Key != nil {
+ key = hash(urlStr, rc.Key)
+ }
+ err := htmlFormExtension.Execute(out, HTMLFormExtParam{urlStr, key, config.Config.UrlParameter, config.Config.HashParameter})
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ writeEndTag := true
+ switch string(tag) {
+ case "body":
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(out, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ rc.BodyInjected = true
+ case "style":
+ state = STATE_DEFAULT
+ case "noscript":
+ state = STATE_DEFAULT
+ writeEndTag = false
+ }
+ // skip noscript tags - only the tag, not the content, because javascript is sanitized
+ if writeEndTag {
+ fmt.Fprintf(out, "</%s>", tag)
+ }
+
+ case html.TextToken:
+ switch state {
+ case STATE_DEFAULT:
+ fmt.Fprintf(out, "%s", decoder.Raw())
+ case STATE_IN_STYLE:
+ sanitizeCSS(rc, out, decoder.Raw())
+ case STATE_IN_NOSCRIPT:
+ sanitizeHTML(rc, out, decoder.Raw())
+ }
+
+ case html.CommentToken:
+ // ignore comment. TODO : parse IE conditional comment
+
+ case html.DoctypeToken:
+ out.Write(decoder.Raw())
+ }
+ } else {
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, _ := decoder.TagName()
+ if inArray(tag, UNSAFE_ELEMENTS) {
+ unsafeElements = append(unsafeElements, tag)
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ if bytes.Equal(unsafeElements[len(unsafeElements)-1], tag) {
+ unsafeElements = unsafeElements[:len(unsafeElements)-1]
+ }
+ }
+ }
+ }
+}
+
+func sanitizeLinkTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ exclude := false
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("rel")) {
+ if !inArray(attrValue, LINK_REL_SAFE_VALUES) {
+ exclude = true
+ break
+ }
+ }
+ if bytes.Equal(attrName, []byte("as")) {
+ if bytes.Equal(attrValue, []byte("script")) {
+ exclude = true
+ break
+ }
+ }
+ }
+
+ if !exclude {
+ out.Write([]byte("<link"))
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+ out.Write([]byte(">"))
+ }
+}
+
+func sanitizeMetaTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ var http_equiv []byte
+ var content []byte
+
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("http-equiv")) {
+ http_equiv = bytes.ToLower(attrValue)
+ // exclude some <meta http-equiv="..." ..>
+ if !inArray(http_equiv, LINK_HTTP_EQUIV_SAFE_VALUES) {
+ return
+ }
+ }
+ if bytes.Equal(attrName, []byte("content")) {
+ content = attrValue
+ }
+ if bytes.Equal(attrName, []byte("charset")) {
+ // exclude <meta charset="...">
+ return
+ }
+ }
+
+ out.Write([]byte("<meta"))
+ urlIndex := bytes.Index(bytes.ToLower(content), []byte("url="))
+ if bytes.Equal(http_equiv, []byte("refresh")) && urlIndex != -1 {
+ contentUrl := content[urlIndex+4:]
+ // special case of <meta http-equiv="refresh" content="0; url='example.com/url.with.quote.outside'">
+ if len(contentUrl) >= 2 && (contentUrl[0] == byte('\'') || contentUrl[0] == byte('"')) {
+ if contentUrl[0] == contentUrl[len(contentUrl)-1] {
+ contentUrl = contentUrl[1 : len(contentUrl)-1]
+ }
+ }
+ // output proxify result
+ if uri, err := rc.ProxifyURI(contentUrl); err == nil {
+ fmt.Fprintf(out, ` http-equiv="refresh" content="%surl=%s"`, content[:urlIndex], uri)
+ }
+ } else {
+ if len(http_equiv) > 0 {
+ fmt.Fprintf(out, ` http-equiv="%s"`, http_equiv)
+ }
+ sanitizeAttrs(rc, out, attrs)
+ }
+ out.Write([]byte(">"))
+}
+
+func sanitizeAttrs(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+}
+
+func sanitizeAttr(rc *RequestConfig, out io.Writer, attrName, attrValue, escapedAttrValue []byte) {
+ if inArray(attrName, SAFE_ATTRIBUTES) {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, escapedAttrValue)
+ return
+ }
+ switch string(attrName) {
+ case "src", "href", "action":
+ if uri, err := rc.ProxifyURI(attrValue); err == nil {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, uri)
+ } else if config.Config.Debug {
+ log.Println("cannot proxify uri:", string(attrValue))
+ }
+ case "style":
+ cssAttr := bytes.NewBuffer(nil)
+ sanitizeCSS(rc, cssAttr, attrValue)
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, html.EscapeString(string(cssAttr.Bytes())))
+ }
+}
+
+func mergeURIs(u1, u2 *url.URL) *url.URL {
+ if u2 == nil {
+ return u1
+ }
+ return u1.ResolveReference(u2)
+}
+
+// Sanitized URI : removes all runes bellow 32 (included) as the begining and end of URI, and lower case the scheme.
+// avoid memory allocation (except for the scheme)
+func sanitizeURI(uri []byte) ([]byte, string) {
+ first_rune_index := 0
+ first_rune_seen := false
+ scheme_last_index := -1
+ buffer := bytes.NewBuffer(make([]byte, 0, 10))
+
+ // remove trailing space and special characters
+ uri = bytes.TrimRight(uri, "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0A\x0B\x0C\x0D\x0E\x0F\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1A\x1B\x1C\x1D\x1E\x1F\x20")
+
+ // loop over byte by byte
+ for i, c := range uri {
+ // ignore special characters and space (c <= 32)
+ if c > 32 {
+ // append to the lower case of the rune to buffer
+ if c < utf8.RuneSelf && 'A' <= c && c <= 'Z' {
+ c = c + 'a' - 'A'
+ }
+
+ buffer.WriteByte(c)
+
+ // update the first rune index that is not a special rune
+ if !first_rune_seen {
+ first_rune_index = i
+ first_rune_seen = true
+ }
+
+ if c == ':' {
+ // colon rune found, we have found the scheme
+ scheme_last_index = i
+ break
+ } else if c == '/' || c == '?' || c == '\\' || c == '#' {
+ // special case : most probably a relative URI
+ break
+ }
+ }
+ }
+
+ if scheme_last_index != -1 {
+ // scheme found
+ // copy the "lower case without special runes scheme" before the ":" rune
+ scheme_start_index := scheme_last_index - buffer.Len() + 1
+ copy(uri[scheme_start_index:], buffer.Bytes())
+ // and return the result
+ return uri[scheme_start_index:], buffer.String()
+ } else {
+ // scheme NOT found
+ return uri[first_rune_index:], ""
+ }
+}
+
+func (rc *RequestConfig) ProxifyURI(uri []byte) (string, error) {
+ // sanitize URI
+ uri, scheme := sanitizeURI(uri)
+
+ // remove javascript protocol
+ if scheme == "javascript:" {
+ return "", nil
+ }
+
+ // TODO check malicious data: - e.g. data:script
+ if scheme == "data:" {
+ if bytes.HasPrefix(uri, []byte("data:image/png")) ||
+ bytes.HasPrefix(uri, []byte("data:image/jpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/pjpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/gif")) ||
+ bytes.HasPrefix(uri, []byte("data:image/webp")) {
+ // should be safe
+ return string(uri), nil
+ } else {
+ // unsafe data
+ return "", nil
+ }
+ }
+
+ // parse the uri
+ u, err := url.Parse(string(uri))
+ if err != nil {
+ return "", err
+ }
+
+ // get the fragment (with the prefix "#")
+ fragment := ""
+ if len(u.Fragment) > 0 {
+ fragment = "#" + u.Fragment
+ }
+
+ // reset the fragment: it is not included in the yukariurl
+ u.Fragment = ""
+
+ // merge the URI with the document URI
+ u = mergeURIs(rc.BaseURL, u)
+
+ // simple internal link ?
+ // some web pages describe the whole link https://same:auth@same.host/same.path?same.query#new.fragment
+ if u.Scheme == rc.BaseURL.Scheme &&
+ (rc.BaseURL.User == nil || (u.User != nil && u.User.String() == rc.BaseURL.User.String())) &&
+ u.Host == rc.BaseURL.Host &&
+ u.Path == rc.BaseURL.Path &&
+ u.RawQuery == rc.BaseURL.RawQuery {
+ // the fragment is the only difference between the document URI and the uri parameter
+ return fragment, nil
+ }
+
+ // return full URI and fragment (if not empty)
+ yukari_uri := u.String()
+
+ if rc.Key == nil {
+ return fmt.Sprintf("./?%s=%s%s", config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+ }
+ return fmt.Sprintf("./?%s=%s&%s=%s%s", config.Config.HashParameter, hash(yukari_uri, rc.Key), config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+}
+
+func inArray(b []byte, a [][]byte) bool {
+ for _, b2 := range a {
+ if bytes.Equal(b, b2) {
+ return true
+ }
+ }
+ return false
+}
+
+func hash(msg string, key []byte) string {
+ mac := hmac.New(sha256.New, key)
+ mac.Write([]byte(msg))
+ return hex.EncodeToString(mac.Sum(nil))
+}
+
+func verifyRequestURI(uri, hashMsg, key []byte) bool {
+ h := make([]byte, hex.DecodedLen(len(hashMsg)))
+ _, err := hex.Decode(h, hashMsg)
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("hmac error:", err)
+ }
+ return false
+ }
+ mac := hmac.New(sha256.New, key)
+ mac.Write(uri)
+ return hmac.Equal(h, mac.Sum(nil))
+}
+
+func (p *Proxy) serveExitYukariPage(ctx *fasthttp.RequestCtx, uri *url.URL) {
+ ctx.SetContentType("text/html")
+ ctx.SetStatusCode(403)
+ ctx.Write([]byte(htmlPageStart))
+ ctx.Write([]byte("<h2>You are about to exit Yukari no Sukima</h2>"))
+ ctx.Write([]byte("<p>Following</p><p><a href=\""))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("\" rel=\"noreferrer\">"))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("</a></p><p>the content of this URL will be <b>NOT</b> sanitized.</p>"))
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func (p *Proxy) serveMainPage(ctx *fasthttp.RequestCtx, statusCode int, err error) {
+ ctx.SetContentType("text/html; charset=UTF-8")
+ ctx.SetStatusCode(statusCode)
+ ctx.Write([]byte(htmlPageStart))
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("error:", err)
+ }
+ ctx.Write([]byte("<h2>Error: "))
+ ctx.Write([]byte(html.EscapeString(err.Error())))
+ ctx.Write([]byte("</h2>"))
+ }
+ if p.Key == nil {
+ p := HTMLMainPageFormParam{config.Config.UrlParameter}
+ err := htmlMainPageForm.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject main page form", err)
+ }
+ }
+ } else {
+ ctx.Write([]byte(`<h3>Warning! This instance does not support direct URL opening.</h3>`))
+ }
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func main() {
+ config.Config.ListenAddress = "127.0.0.1:3000"
+ config.Config.Key = ""
+ config.Config.IPV6 = true
+ config.Config.Debug = false
+ config.Config.RequestTimeout = 5
+ config.Config.FollowRedirect = false
+ config.Config.UrlParameter = "yukariurl"
+ config.Config.HashParameter = "yukarihash"
+ config.Config.MaxConnsPerHost = 5
+ config.Config.ProxyEnv = false
+
+ var configFile string
+ var proxy string
+ var socks5 string
+ var version bool
+
+ flag.StringVar(&configFile, "f", "", "Configuration file")
+ flag.StringVar(&proxy, "proxy", "", "Use the specified HTTP proxy (ie: '[user:pass@]hostname:port'). Overrides: -socks5, IPv6")
+ flag.StringVar(&socks5, "socks5", "", "Use a SOCKS5 proxy (ie: 'hostname:port'). Overrides: IPv6.")
+ flag.BoolVar(&version, "version", false, "Show version")
+ flag.Parse()
+
+ if configFile != "" {
+ config.ReadConfig(configFile)
+ }
+
+ if version {
+ yukari.FullVersion()
+ return
+ }
+
+ if config.Config.ProxyEnv && os.Getenv("HTTP_PROXY") == "" && os.Getenv("HTTPS_PROXY") == "" {
+ log.Fatal("Error -proxyenv is used but no environment variables named 'HTTP_PROXY' and/or 'HTTPS_PROXY' could be found.")
+ os.Exit(1)
+ }
+
+ if config.Config.ProxyEnv {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpProxyHTTPDialer()
+ log.Println("Using environment defined proxy(ies).")
+ } else if proxy != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpHTTPDialer(proxy)
+ log.Println("Using custom HTTP proxy.")
+ } else if socks5 != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpSocksDialer(socks5)
+ log.Println("Using Socks5 proxy.")
+ } else if config.Config.IPV6 {
+ Gap.Dial = fasthttp.DialDualStack
+ log.Println("Using dual stack (IPv4/IPv6) direct connections.")
+ } else {
+ Gap.Dial = fasthttp.Dial
+ log.Println("Using IPv4 only direct connections.")
+ }
+
+ p := &Proxy{RequestTimeout: time.Duration(config.Config.RequestTimeout) * time.Second,
+ FollowRedirect: config.Config.FollowRedirect}
+
+ if config.Config.Key != "" {
+ var err error
+ p.Key, err = base64.StdEncoding.DecodeString(config.Config.Key)
+ if err != nil {
+ log.Fatal("Error parsing -key", err.Error())
+ os.Exit(1)
+ }
+ }
+ log.Println("ゆかり様、お願いします…！")
+ log.Println("Listening on", config.Config.ListenAddress)
+
+ if err := fasthttp.ListenAndServe(config.Config.ListenAddress, p.RequestHandler); err != nil {
+ log.Fatal("Error in ListenAndServe:", err)
+ }
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "net/url"
+ "testing"
+)
+
+type AttrTestCase struct {
+ AttrName []byte
+ AttrValue []byte
+ ExpectedOutput []byte
+}
+
+type SanitizeURITestCase struct {
+ Input []byte
+ ExpectedOutput []byte
+ ExpectedScheme string
+}
+
+type StringTestCase struct {
+ Input string
+ ExpectedOutput string
+}
+
+var attrTestData []*AttrTestCase = []*AttrTestCase{
+ &AttrTestCase{
+ []byte("href"),
+ []byte("./x"),
+ []byte(` href="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fx"`),
+ },
+ &AttrTestCase{
+ []byte("src"),
+ []byte("http://x.com/y"),
+ []byte(` src="./?yukariurl=http%3A%2F%2Fx.com%2Fy"`),
+ },
+ &AttrTestCase{
+ []byte("action"),
+ []byte("/z"),
+ []byte(` action="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fz"`),
+ },
+ &AttrTestCase{
+ []byte("onclick"),
+ []byte("console.log(document.cookies)"),
+ nil,
+ },
+}
+
+var sanitizeUriTestData []*SanitizeURITestCase = []*SanitizeURITestCase{
+ &SanitizeURITestCase{
+ []byte("http://example.com/"),
+ []byte("http://example.com/"),
+ "http:",
+ },
+ &SanitizeURITestCase{
+ []byte("HtTPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte(" Ht TPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte("javascript:void(0)"),
+ []byte("javascript:void(0)"),
+ "javascript:",
+ },
+ &SanitizeURITestCase{
+ []byte(" /path/to/a/file/without/protocol "),
+ []byte("/path/to/a/file/without/protocol"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" #fragment "),
+ []byte("#fragment"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" qwertyuiop "),
+ []byte("qwertyuiop"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(""),
+ []byte(""),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(":"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte(" :"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte("schéma:"),
+ []byte("schéma:"),
+ "schéma:",
+ },
+}
+
+var urlTestData []*StringTestCase = []*StringTestCase{
+ &StringTestCase{
+ "http://x.com/",
+ "./?yukariurl=http%3A%2F%2Fx.com%2F",
+ },
+ &StringTestCase{
+ "http://a@x.com/",
+ "./?yukariurl=http%3A%2F%2Fa%40x.com%2F",
+ },
+ &StringTestCase{
+ "#a",
+ "#a",
+ },
+}
+
+func TestAttrSanitizer(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range attrTestData {
+ out := bytes.NewBuffer(nil)
+ sanitizeAttr(rc, out, testCase.AttrName, testCase.AttrValue, testCase.AttrValue)
+ res, _ := out.ReadBytes(byte(0))
+ if !bytes.Equal(res, testCase.ExpectedOutput) {
+ t.Errorf(
+ `Attribute parse error. Name: "%s", Value: "%s", Expected: %s, Got: "%s"`,
+ testCase.AttrName,
+ testCase.AttrValue,
+ testCase.ExpectedOutput,
+ res,
+ )
+ }
+ }
+}
+
+func TestSanitizeURI(t *testing.T) {
+ for _, testCase := range sanitizeUriTestData {
+ newUrl, scheme := sanitizeURI(testCase.Input)
+ if !bytes.Equal(newUrl, testCase.ExpectedOutput) {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ if scheme != testCase.ExpectedScheme {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedScheme,
+ scheme,
+ )
+ }
+ }
+}
+
+func TestURLProxifier(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range urlTestData {
+ newUrl, err := rc.ProxifyURI([]byte(testCase.Input))
+ if err != nil {
+ t.Errorf("Failed to parse URL: %s", testCase.Input)
+ }
+ if newUrl != testCase.ExpectedOutput {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ }
+}
+
+var BENCH_SIMPLE_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <title>test</title>
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeSimpleHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_SIMPLE_HTML)
+ }
+}
+
+var BENCH_COMPLEX_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <noscript><meta http-equiv="refresh" content="0; URL=./xy"></noscript>
+ <title>test 2</title>
+ <script> alert('xy'); </script>
+ <link rel="stylesheet" href="./core.bundle.css">
+ <style>
+ html { background: url(./a.jpg); }
+ </style
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ <img src="b.png" alt="imgtitle" />
+ <form action="/z">
+ <input type="submit" style="background: url(http://aa.bb/cc)" >
+ </form>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeComplexHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_COMPLEX_HTML)
+ }
+}
--- /dev/null
+package main
+
+var SAFE_ATTRIBUTES [][]byte = [][]byte{
+ []byte("abbr"),
+ []byte("accesskey"),
+ []byte("align"),
+ []byte("alt"),
+ []byte("as"),
+ []byte("autocomplete"),
+ []byte("charset"),
+ []byte("checked"),
+ []byte("class"),
+ []byte("content"),
+ []byte("contenteditable"),
+ []byte("contextmenu"),
+ []byte("dir"),
+ []byte("for"),
+ []byte("height"),
+ []byte("hidden"),
+ []byte("hreflang"),
+ []byte("id"),
+ []byte("lang"),
+ []byte("media"),
+ []byte("method"),
+ []byte("name"),
+ []byte("nowrap"),
+ []byte("placeholder"),
+ []byte("property"),
+ []byte("rel"),
+ []byte("spellcheck"),
+ []byte("tabindex"),
+ []byte("target"),
+ []byte("title"),
+ []byte("translate"),
+ []byte("type"),
+ []byte("value"),
+ []byte("width"),
+}
--- /dev/null
+package main
+
+var LINK_HTTP_EQUIV_SAFE_VALUES [][]byte = [][]byte{
+ // X-UA-Compatible will be added automaticaly, so it can be skipped
+ []byte("date"),
+ []byte("last-modified"),
+ []byte("refresh"), // URL rewrite
+ []byte("content-language"),
+}
+
+var LINK_REL_SAFE_VALUES [][]byte = [][]byte{
+ []byte("alternate"),
+ []byte("archives"),
+ []byte("author"),
+ []byte("copyright"),
+ []byte("first"),
+ []byte("help"),
+ []byte("icon"),
+ []byte("index"),
+ []byte("last"),
+ []byte("license"),
+ []byte("manifest"),
+ []byte("next"),
+ []byte("pingback"),
+ []byte("prev"),
+ []byte("publisher"),
+ []byte("search"),
+ []byte("shortcut icon"),
+ []byte("stylesheet"),
+ []byte("up"),
+}
--- /dev/null
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="referrer" content="no-referrer">
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
+<html>
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=1">
+ <style>
+ html {
+ height: 100%;
+ }
+ body {
+ min-height: 100%;
+ display: flex;
+ flex-direction: column;
+ font-family: sans-serif;
+ text-align: center;
+ color: #BC48FC;
+ background: #240039;
+ margin: 0;
+ padding: 0;
+ font-size: 1.1em;
+ }
+ input {
+ border: 1px solid #888;
+ padding: 0.3em;
+ color: #BC48FC;
+ background: #202020;
+ font-size: 1.1.em;
+ }
+ input[placeholder] {
+ width: 80%;
+ }
+ a {
+ text-decoration: none;
+ color: #9529B9;
+ }
+ h1, h2 {
+ font-weight: 200;
+ margin-bottom: 2rem;
+ }
+ h1 {
+ font-size: 3em;
+ }
+ .container {
+ flex: 1;
+ min-height: 100%;
+ margin-bottom: 1em;
+ }
+ .footer {
+ margin: 1em;
+ }
+ .footer p {
+ font-size: 0.8em;
+ }
+ </style>
+ <title>Yukari's Gap</title>
+ </head>
+ <body>
+ <div class="container">
+ <h1>Yukari's Gap</h1>
+
--- /dev/null
+</div>
+<div class="footer">
+ <p>
+ Yukari's Gap rewrites web pages to exclude malicious HTML tags and CSS/HTML attributes. <br>
+ It also replaces external resource references to prevent third-party information leaks. <br>
+ </p>
+</div>
+</body>
+</html>
--- /dev/null
+package main
+
+var UNSAFE_ELEMENTS [][]byte = [][]byte{
+ []byte("applet"),
+ []byte("canvas"),
+ []byte("embed"),
+ []byte("math"),
+ []byte("script"),
+ []byte("svg"),
+}
--- /dev/null
+package config
+
+import (
+ "gopkg.in/ini.v1"
+)
+
+var Config struct {
+ Debug bool
+ ListenAddress string
+ Key string
+ IPV6 bool
+ RequestTimeout uint
+ FollowRedirect bool
+ MaxConnsPerHost uint
+ UrlParameter string
+ HashParameter string
+ ProxyEnv bool
+}
+
+func ReadConfig(file string) error {
+ cfg, err := ini.Load(file)
+ if err != nil {
+ return err
+ }
+ Config.Debug, _ = cfg.Section("yukari").Key("debug").Bool()
+ Config.ListenAddress = cfg.Section("yukari").Key("listen").String()
+ Config.Key = cfg.Section("yukari").Key("key").String()
+ Config.IPV6, _ = cfg.Section("yukari").Key("ipv6").Bool()
+ Config.RequestTimeout, _ = cfg.Section("yukari").Key("timeout").Uint()
+ Config.FollowRedirect, _ = cfg.Section("yukari").Key("followredirect").Bool()
+ Config.MaxConnsPerHost, _ = cfg.Section("yukari").Key("max_conns_per_host").Uint()
+ Config.UrlParameter = cfg.Section("yukari").Key("urlparam").String()
+ Config.HashParameter = cfg.Section("yukari").Key("hashparam").String()
+ Config.ProxyEnv, _ = cfg.Section("yukari").Key("proxyenv").Bool()
+ return nil
+}
--- /dev/null
+package contenttype
+
+import (
+ "mime"
+ "strings"
+)
+
+type ContentType struct {
+ TopLevelType string
+ SubType string
+ Suffix string
+ Parameters map[string]string
+}
+
+func (contenttype *ContentType) String() string {
+ var mimetype string
+ if contenttype.Suffix == "" {
+ if contenttype.SubType == "" {
+ mimetype = contenttype.TopLevelType
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType
+ }
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType + "+" + contenttype.Suffix
+ }
+ return mime.FormatMediaType(mimetype, contenttype.Parameters)
+}
+
+func (contenttype *ContentType) Equals(other ContentType) bool {
+ if contenttype.TopLevelType != other.TopLevelType ||
+ contenttype.SubType != other.SubType ||
+ contenttype.Suffix != other.Suffix ||
+ len(contenttype.Parameters) != len(other.Parameters) {
+ return false
+ }
+ for k, v := range contenttype.Parameters {
+ if other.Parameters[k] != v {
+ return false
+ }
+ }
+ return true
+}
+
+func (contenttype *ContentType) FilterParameters(parameters map[string]bool) {
+ for k, _ := range contenttype.Parameters {
+ if !parameters[k] {
+ delete(contenttype.Parameters, k)
+ }
+ }
+}
+
+func ParseContentType(contenttype string) (ContentType, error) {
+ mimetype, params, err := mime.ParseMediaType(contenttype)
+ if err != nil {
+ return ContentType{"", "", "", params}, err
+ }
+ splitted_mimetype := strings.SplitN(strings.ToLower(mimetype), "/", 2)
+ if len(splitted_mimetype) <= 1 {
+ return ContentType{splitted_mimetype[0], "", "", params}, nil
+ } else {
+ splitted_subtype := strings.SplitN(splitted_mimetype[1], "+", 2)
+ if len(splitted_subtype) == 1 {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], "", params}, nil
+ } else {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], splitted_subtype[1], params}, nil
+ }
+ }
+
+}
+
+type Filter func(contenttype ContentType) bool
+
+func NewFilterContains(partialMimeType string) Filter {
+ return func(contenttype ContentType) bool {
+ return strings.Contains(contenttype.TopLevelType, partialMimeType) ||
+ strings.Contains(contenttype.SubType, partialMimeType) ||
+ strings.Contains(contenttype.Suffix, partialMimeType)
+ }
+}
+
+func NewFilterEquals(TopLevelType, SubType, Suffix string) Filter {
+ return func(contenttype ContentType) bool {
+ return ((TopLevelType != "*" && TopLevelType == contenttype.TopLevelType) || (TopLevelType == "*")) &&
+ ((SubType != "*" && SubType == contenttype.SubType) || (SubType == "*")) &&
+ ((Suffix != "*" && Suffix == contenttype.Suffix) || (Suffix == "*"))
+ }
+}
+
+func NewFilterOr(contentTypeFilterList []Filter) Filter {
+ return func(contenttype ContentType) bool {
+ for _, contentTypeFilter := range contentTypeFilterList {
+ if contentTypeFilter(contenttype) {
+ return true
+ }
+ }
+ return false
+ }
+}
--- /dev/null
+package contenttype
+
+import (
+ "bytes"
+ "fmt"
+ "testing"
+)
+
+type ParseContentTypeTestCase struct {
+ Input string
+ ExpectedOutput *ContentType /* or nil if an error is expected */
+ ExpectedString *string /* or nil if equals to Input */
+}
+
+var parseContentTypeTestCases []ParseContentTypeTestCase = []ParseContentTypeTestCase{
+ ParseContentTypeTestCase{
+ "text/html",
+ &ContentType{"text", "html", "", map[string]string{}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/svg+xml; charset=UTF-8",
+ &ContentType{"text", "svg", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/",
+ nil,
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text; charset=UTF-8",
+ &ContentType{"text", "", "", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/+xml; charset=UTF-8",
+ &ContentType{"text", "", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+}
+
+type ContentTypeEqualsTestCase struct {
+ A, B ContentType
+ Equals bool
+}
+
+var Map_Empty map[string]string = map[string]string{}
+var Map_A map[string]string = map[string]string{"a": "value_a"}
+var Map_B map[string]string = map[string]string{"b": "value_b"}
+var Map_AB map[string]string = map[string]string{"a": "value_a", "b": "value_b"}
+
+var ContentType_E ContentType = ContentType{"a", "b", "c", Map_Empty}
+var ContentType_A ContentType = ContentType{"a", "b", "c", Map_A}
+var ContentType_B ContentType = ContentType{"a", "b", "c", Map_B}
+var ContentType_AB ContentType = ContentType{"a", "b", "c", Map_AB}
+
+var contentTypeEqualsTestCases []ContentTypeEqualsTestCase = []ContentTypeEqualsTestCase{
+ // TopLevelType, SubType, Suffix
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "c", Map_Empty}, true},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"o", "b", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "o", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "o", Map_Empty}, false},
+ // Parameters
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_A, true},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_B, true},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_AB, true},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_B, false},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_AB, false},
+}
+
+type FilterTestCase struct {
+ Description string
+ Input Filter
+ TrueValues []ContentType
+ FalseValues []ContentType
+}
+
+var filterTestCases []FilterTestCase = []FilterTestCase{
+ FilterTestCase{
+ "contains xml",
+ NewFilterContains("xml"),
+ []ContentType{
+ ContentType{"xml", "", "", Map_Empty},
+ ContentType{"text", "xml", "", Map_Empty},
+ ContentType{"text", "html", "xml", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "svg", "", map[string]string{"script": "javascript"}},
+ ContentType{"java", "script", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/xhtml",
+ NewFilterEquals("application", "xhtml", "*"),
+ []ContentType{
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_AB},
+ },
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "xhtml", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals application/*",
+ NewFilterEquals("application", "*", ""),
+ []ContentType{
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "xhtml", "", Map_Empty},
+ ContentType{"text", "xhtml", "xml", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications */javascript",
+ NewFilterEquals("*", "javascript", ""),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"text", "javascript", "zip", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/* or */javascript",
+ NewFilterOr([]Filter{
+ NewFilterEquals("application", "*", ""),
+ NewFilterEquals("*", "javascript", ""),
+ }),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ },
+ },
+}
+
+type FilterParametersTestCase struct {
+ Input map[string]string
+ Filter map[string]bool
+ Output map[string]string
+}
+
+var filterParametersTestCases []FilterParametersTestCase = []FilterParametersTestCase{
+ FilterParametersTestCase{
+ map[string]string{},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true},
+ map[string]string{"A": "value_A"},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{"A": "value_A", "B": "value_B"},
+ },
+}
+
+func TestContentTypeEquals(t *testing.T) {
+ for _, testCase := range contentTypeEqualsTestCases {
+ if !testCase.A.Equals(testCase.B) && testCase.Equals {
+ t.Errorf(`Must be equals "%s"="%s"`, testCase.A, testCase.B)
+ } else if testCase.A.Equals(testCase.B) && !testCase.Equals {
+ t.Errorf(`Mustn't be equals "%s"!="%s"`, testCase.A, testCase.B)
+ }
+ }
+}
+
+func TestParseContentType(t *testing.T) {
+ for _, testCase := range parseContentTypeTestCases {
+ // test ParseContentType
+ contentType, err := ParseContentType(testCase.Input)
+ if testCase.ExpectedOutput == nil {
+ // error expected
+ if err == nil {
+ // but there is no error
+ t.Errorf(`Expecting error for "%s"`, testCase.Input)
+ }
+ } else {
+ // no expected error
+ if err != nil {
+ t.Errorf(`Unexpecting error for "%s" : %s`, testCase.Input, err)
+ } else if !contentType.Equals(*testCase.ExpectedOutput) {
+ // the parsed contentType doesn't matched
+ t.Errorf(`Unexpecting result for "%s", instead got "%s"`, testCase.ExpectedOutput.String(), contentType.String())
+ } else {
+ // ParseContentType is fine, checking String()
+ contentTypeString := contentType.String()
+ expectedString := testCase.Input
+ if testCase.ExpectedString != nil {
+ expectedString = *testCase.ExpectedString
+ }
+ if contentTypeString != expectedString {
+ t.Errorf(`Error with String() output of "%s", got "%s", ContentType{"%s", "%s", "%s", "%s"}`, expectedString, contentTypeString, contentType.TopLevelType, contentType.SubType, contentType.Suffix, contentType.Parameters)
+ }
+ }
+ }
+ }
+}
+
+func FilterToString(m map[string]bool) string {
+ b := new(bytes.Buffer)
+ for key, value := range m {
+ if value {
+ fmt.Fprintf(b, "'%s'=true;", key)
+ } else {
+ fmt.Fprintf(b, "'%s'=false;", key)
+ }
+ }
+ return b.String()
+}
+
+func TestFilters(t *testing.T) {
+ for _, testCase := range filterTestCases {
+ for _, contentType := range testCase.TrueValues {
+ if !testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" must accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ for _, contentType := range testCase.FalseValues {
+ if testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" mustn't accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ }
+}
+
+func TestFilterParameters(t *testing.T) {
+ for _, testCase := range filterParametersTestCases {
+ // copy Input since the map will be modified
+ InputCopy := make(map[string]string)
+ for k, v := range testCase.Input {
+ InputCopy[k] = v
+ }
+ // apply filter
+ contentType := ContentType{"", "", "", InputCopy}
+ contentType.FilterParameters(testCase.Filter)
+ // test
+ contentTypeOutput := ContentType{"", "", "", testCase.Output}
+ if !contentTypeOutput.Equals(contentType) {
+ t.Errorf(`FilterParameters error : %s becomes %s with this filter %s`, testCase.Input, contentType.Parameters, FilterToString(testCase.Filter))
+ }
+ }
+}
--- /dev/null
+[yukari]
+debug=false
+listen="127.0.0.1:3000"
+key=""
+ipv6=true
+timeout=5
+followredirect=false
+max_conns_per_host=5
+urlparam="yukariurl"
+hashparam="yukarihash"
+proxyenv=false
--- /dev/null
+module marisa.chaotic.ninja/yukari
+
+go 1.16
+
+require (
+ github.com/stretchr/testify v1.9.0 // indirect
+ github.com/valyala/fasthttp v1.34.0
+ golang.org/x/net v0.7.0
+ golang.org/x/text v0.7.0
+ gopkg.in/ini.v1 v1.67.0
+)
--- /dev/null
+github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
+github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/klauspost/compress v1.15.0 h1:xqfchp4whNFxn5A4XFyyYtitiWI8Hy5EW59jEwcyL6U=
+github.com/klauspost/compress v1.15.0/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
+github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
+github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
+github.com/valyala/fasthttp v1.34.0 h1:d3AAQJ2DRcxJYHm7OXNXtXt2as1vMDfxeIcFvhmGGm4=
+github.com/valyala/fasthttp v1.34.0/go.mod h1:epZA5N+7pY6ZaEKRmstzOuYJx9HI8DI1oaCGZpdH4h0=
+github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20220214200702-86341886e292/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
+golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
+gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
--- /dev/null
+#!/bin/sh
+# $TheSupernovaDuo$
+
+# PROVIDE: yukari
+# REQUIRE: DAEMON NETWORKING
+# KEYWORD: shutdown
+
+. /etc/rc.subr
+
+name="yukari"
+rcvar="yukari_enable"
+
+load_rc_config "${name}"
+
+: ${yukari_enable="NO"}
+: ${yukari_config=""}
+
+pidfile="/var/run/${name}.pid"
+command="/usr/sbin/daemon"
+procname="/usr/local/bin/${name}"
+command_args="-S -m 3 -s info -l daemon -p ${pidfile} /usr/bin/env ${procname} ${yukari_args}"
+
+run_rc_command "$1"
--- /dev/null
+cmd: /usr/local/bin/yukari
+user: www
--- /dev/null
+#!/bin/ksh
+# $TheSupernovaDuo
+daemon="/usr/local/bin/yukari"
+
+. /etc/rc.d/rc.subr
+
+rc_bg=YES
+rc_reload=NO
+
+rc_cmd "$1"
--- /dev/null
+package yukari
+
+import (
+ "fmt"
+)
+
+var (
+ // Version release version
+ Version = "0.0.1"
+
+ // Commit will be overwritten automatically by the build system
+ Commit = "HEAD"
+)
+
+// FullVersion display the full version and build
+func FullVersion() string {
+ return fmt.Sprintf("%s@%s", Version, Commit)
+}
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI 1
+.Os
+.Sh NAME
+.Nm yukari
+.Nd Privacy-aware Web Content Sanitizer Proxy As A Service (WCSPAAS)
+.Sh SYNOPSIS
+.Nm
+.Op Fl f Ar string
+.Op Fl proxy Ar string
+.Op Fl proxyenv Ar bool
+.Op Fl socks5 Ar string
+.Op Fl version
+.Sh DESCRIPTION
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes.
+It also replaces external resource references in order to prevent third-party
+information leaks.
+.Pp
+The main goal of Yukari's Gap is to provide a result proxy for SearX, but it
+can be used as a standalone sanitizer service, too.
+.Sh FEATURES
+.Bl -tag -width Ds
+.It HTML sanitization
+.It Rewrites HTML/CSS external references to locals
+.It JavaScript blocking
+.It No Cookies forwarded
+.It No Referrers
+.It No Caching/ETag
+.It Supports GET/POST forms and IFrames
+.It Optional HMAC URL verifier key to prevent service abuse
+.El
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It Fl f Ar path
+Load configuration file from path
+.It Fl proxy Ar string
+Use the specified HTTP proxy (ie: [user:pass@]hostname:port),
+this overrides the
+.Fl socks5
+option and the IPv6 setting
+.It Fl proxyenv Ar bool
+Use a HTTP proxy as set in the environment (such as
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+).
+Overrides the
+.Fl proxy ,
+.Fl socks5 ,
+flags and the IPv6 setting
+.It Fl socks5 Ar string
+Use a SOCKS5 proxy (ie: hostname:port), this
+overrides the IPv6 setting
+.El
+.Sh SEE ALSO
+.Xr SearX 1
+.Sh AUTHORS
+.An Adam Tauber Aq Mt asciimoo@gmail.com
+.An Alexandre Flament Aq Mt alex@al-f.net
+.Sh MAINTAINERS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
+.Sh BUGS
+Bugs or suggestions?
+Send an email to
+.Aq Mt yukari-dev@chaotic.ninja
+.Sh LICENSE
+This program is free software: you can redistribute it and/or modify it
+under the terms of the GNU Affero General Public License as published
+by the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+.Pp
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.
+See the GNU Affero General Public License for more details.
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI.INI 5
+.Os
+.Sh NAME
+.Nm yukari.ini
+.Nd INI-style configuration file for
+.Xr yukari 1
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It debug (bool)
+Enable/disable proxy and redirection logs (default true)
+.It listen (string)
+Listen address (default "127.0.0.1:3000")
+.It key (string)
+HMAC url validation key (base64 encoded) - leave blank to disable validation
+.It ipv6 (bool)
+Enable IPv6 support for queries
+(can be overrided by the proxy options, default true)
+.It timeout (uint)
+Request timeout (default 5)
+.It followredirect (bool)
+Follow HTTP GET redirect (default false)
+.It max_conns_per_host (uint)
+How much connections are allowed per Host/IP (default 4)
+.It urlparam (string)
+User-defined requesting string URL parameter name
+(ie: '/?url=...' or '/?u=...') (default "yukariurl")
+.It hashparam (string)
+User-defined requesting string HASH parameter name
+(ie: '/?hash=...' or '/?h=...') (default "yukarihash")
+.It proxyenv (bool)
+Use a HTTP proxy as set in the environment
+(
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+) (overrides ipv6, default false)
+.El
+.Sh AUTHORS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
--- /dev/null
+vendor
+/yukari
--- /dev/null
+# v0.2.5 - 2024.03.24
+* Rename `config.readConfig` to `config.ReadConfig`
+* Assume default values if no `yukari.ini(5)` is loaded
+
+# v0.2.4 - 2024.03.24
+* Replace invalid favicon with one sourced from [here](https://en.touhouwiki.net/wiki/File:Th123YukariSigil.png), as well as using `//go:embed` for it
+* Add rc.d files for FreeBSD and OpenBSD, respectively
+
+# v0.2.3 - 2024.03.21
+* Document the configuration file format, which is INI-style (which is compatible to the old format in the codebase, though it's now called as `config.Config.<key>`)
+* Manual page has been rewritten (using `mdoc(7)`)
+* 'YukariSukima' is an incorrect transliteration, use 'Yukari no Sukima' to indicate possession/ownership
+* Remove the 'proxified and sanitized view' text as it should already be obvious
+* The font family used earlier is horrible, changed it to `sans-serif`
+* Bump required Go toolchain version to 1.16 in order to use `//go:embed`
+* Rename some all-uppercase constants/variables to camelCase (I think?), also rename CLIENT to Gap (lol)
+
+# v0.2.1 - 2023.08.26
+Applied some suggestions from the [issue tracker](https://github.com/asciimoo/morty/issues), and rebrand this fork.
+
+# v0.2.0 - 2018.05.28
+
+Man page added
+
+# v0.1.0 - 2018.01.30
+
+Initial release
--- /dev/null
+ GNU AFFERO GENERAL PUBLIC LICENSE
+ Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+ The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+ Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+ A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+ The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+ An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ TERMS AND CONDITIONS
+
+ 0. Definitions.
+
+ "This License" refers to version 3 of the GNU Affero General Public License.
+
+ "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+ "The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+ To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy. The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+ A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+ To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+ To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+ An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+ 1. Source Code.
+
+ The "source code" for a work means the preferred form of the work
+for making modifications to it. "Object code" means any non-source
+form of a work.
+
+ A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+ The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+ The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+ The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+ The Corresponding Source for a work in source code form is that
+same work.
+
+ 2. Basic Permissions.
+
+ All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+ You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force. You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright. Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+ Conveying under any other circumstances is permitted solely under
+the conditions stated below. Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+ No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+ When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+ 4. Conveying Verbatim Copies.
+
+ You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+ You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+ 5. Conveying Modified Source Versions.
+
+ You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+ a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+
+ b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under section
+ 7. This requirement modifies the requirement in section 4 to
+ "keep intact all notices".
+
+ c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+
+ d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+ A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+ 6. Conveying Non-Source Forms.
+
+ You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+ a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+
+ b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the
+ Corresponding Source from a network server at no charge.
+
+ c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+
+ d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+
+ e) Convey the object code using peer-to-peer transmission, provided
+ you inform other peers where the object code and Corresponding
+ Source of the work are being offered to the general public at no
+ charge under subsection 6d.
+
+ A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+ A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling. In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage. For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product. A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+ "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source. The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+ If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+ The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed. Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+ Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+ 7. Additional Terms.
+
+ "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+ When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+ Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+ a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+
+ b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+
+ c) Prohibiting misrepresentation of the origin of that material, or
+ requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+
+ d) Limiting the use for publicity purposes of names of licensors or
+ authors of the material; or
+
+ e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+
+ f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions of
+ it) with contractual assumptions of liability to the recipient, for
+ any liability that these contractual assumptions directly impose on
+ those licensors and authors.
+
+ All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+ If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+ Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+ 8. Termination.
+
+ You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+ However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+ Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+ Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+ 9. Acceptance Not Required for Having Copies.
+
+ You are not required to accept this License in order to receive or
+run a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+ 10. Automatic Licensing of Downstream Recipients.
+
+ Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+ An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+ You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+ 11. Patents.
+
+ A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+ A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+ Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+ In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+ If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+ If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+ A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License. You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+ Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+ 12. No Surrender of Others' Freedom.
+
+ If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all. For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+ 13. Remote Network Interaction; Use with the GNU General Public License.
+
+ Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software. This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+ Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+ 14. Revised Versions of this License.
+
+ The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time. Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+ If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+ Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+ 15. Disclaimer of Warranty.
+
+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. Limitation of Liability.
+
+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+ 17. Interpretation of Sections 15 and 16.
+
+ If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+ If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+ You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<http://www.gnu.org/licenses/>.
--- /dev/null
+GO ?= go
+RM ?= rm
+GOFLAGS ?= -v -mod=vendor
+PREFIX ?= /usr/local
+BINDIR ?= bin
+MANDIR ?= share/man
+MKDIR ?= mkdir
+CP ?= cp
+SYSCONFDIR ?= /etc
+
+VERSION = `git describe --abbrev=0 --tags 2>/dev/null || echo "$VERSION"`
+COMMIT = `git rev-parse --short HEAD || echo "$COMMIT"`
+BRANCH = `git rev-parse --abbrev-ref HEAD`
+BUILD = `git show -s --pretty=format:%cI`
+
+GOARCH ?= amd64
+GOOS ?= linux
+
+all: yukari
+
+yukari: vendor
+ env GOARCH=${GOARCH} GOOS=${GOOS} ${GO} build ${GOFLAGS} ./cmd/yukari
+clean:
+ ${RM} -f yukari
+install:
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${BINDIR}
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man5
+
+ ${CP} -f yukari ${DESTDIR}${PREFIX}/${BINDIR}
+ ${CP} -f yukari.1 ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${CP} -f yukari.ini.5 ${DESTDIR}${PREFIX}/${MANDIR}/man5
+test:
+ go test
+bench:
+ go test -benchmem -bench .
+vendor:
+ go mod vendor
+.PHONY: yukari clean install
--- /dev/null
+# Yukari's Gap
+Web content sanitizer proxy as a service[^1], fork of [MortyProxy](https://github.com/asciimoo/morty) with some suggestions from the issue tracker applied, named after [the youkai you shouldn't ever come near](https://en.touhouwiki.net/wiki/Yukari_Yakumo)
+
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes. It also replaces external resource references to prevent third party information leaks.
+
+The main goal of this tool is to provide a result proxy for [searx](https://asciimoo.github.com/searx/), but it can be used as a standalone sanitizer service too.
+
+Features:
+
+* HTML sanitization
+* Rewrites HTML/CSS external references to locals
+* JavaScript blocking
+* No Cookies forwarded
+* No Referrers
+* No Caching/Etag
+* Supports GET/POST forms and IFrames
+* Optional HMAC URL verifier key to prevent service abuse
+
+## Installation and setup
+Requirement: Go version 1.16 or higher (thus making it incompatible with MortyProxy's own requirement, but also to use `go embed`)
+
+```
+$ go install marisa.chaotic.ninja/yukari/cmd/yukari@latest
+$ "$GOPATH/bin/yukari" --help
+```
+### Usage
+See `yukari(1)`
+
+### Test
+
+```
+$ make test
+```
+
+### Benchmark
+
+```
+$ make bench
+```
+
+## Bugs
+Bugs or suggestions? Mail [yukari-dev@chaotic.ninja](mailto:yukari-dev@chaotic.ninja)
+
+---
+
+[^1]: or WCPaaS, mind you, also I didn't come up with that, it was already there when I arrived
--- /dev/null
+package main
+
+import (
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+var ALLOWED_CONTENTTYPE_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // html
+ contenttype.NewFilterEquals("text", "html", ""),
+ contenttype.NewFilterEquals("application", "xhtml", "xml"),
+ // css
+ contenttype.NewFilterEquals("text", "css", ""),
+ // images
+ contenttype.NewFilterEquals("image", "gif", ""),
+ contenttype.NewFilterEquals("image", "png", ""),
+ contenttype.NewFilterEquals("image", "jpeg", ""),
+ contenttype.NewFilterEquals("image", "pjpeg", ""),
+ contenttype.NewFilterEquals("image", "webp", ""),
+ contenttype.NewFilterEquals("image", "tiff", ""),
+ contenttype.NewFilterEquals("image", "vnd.microsoft.icon", ""),
+ contenttype.NewFilterEquals("image", "bmp", ""),
+ contenttype.NewFilterEquals("image", "x-ms-bmp", ""),
+ contenttype.NewFilterEquals("image", "x-icon", ""),
+ contenttype.NewFilterEquals("image", "svg", "xml"),
+ // fonts
+ contenttype.NewFilterEquals("application", "font-otf", ""),
+ contenttype.NewFilterEquals("application", "font-ttf", ""),
+ contenttype.NewFilterEquals("application", "font-woff", ""),
+ contenttype.NewFilterEquals("application", "vnd.ms-fontobject", ""),
+})
+
+var ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // texts
+ contenttype.NewFilterEquals("text", "csv", ""),
+ contenttype.NewFilterEquals("text", "tab-separated-values", ""),
+ contenttype.NewFilterEquals("text", "plain", ""),
+ // API
+ contenttype.NewFilterEquals("application", "json", ""),
+ // Documents
+ contenttype.NewFilterEquals("application", "x-latex", ""),
+ contenttype.NewFilterEquals("application", "pdf", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.text", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.spreadsheet", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.presentation", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.graphics", ""),
+ // Compressed archives
+ contenttype.NewFilterEquals("application", "zip", ""),
+ contenttype.NewFilterEquals("application", "gzip", ""),
+ contenttype.NewFilterEquals("application", "x-compressed", ""),
+ contenttype.NewFilterEquals("application", "x-gtar", ""),
+ contenttype.NewFilterEquals("application", "x-compress", ""),
+ // Generic binary
+ contenttype.NewFilterEquals("application", "octet-stream", ""),
+})
+
+var ALLOWED_CONTENTTYPE_PARAMETERS map[string]bool = map[string]bool{
+ "charset": true,
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "crypto/hmac"
+ "crypto/sha256"
+ _ "embed"
+ "encoding/base64"
+ "encoding/hex"
+ "errors"
+ "flag"
+ "fmt"
+ "html/template"
+ "io"
+ "log"
+ "mime"
+ "net/url"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "time"
+ "unicode/utf8"
+
+ "github.com/valyala/fasthttp"
+ "github.com/valyala/fasthttp/fasthttpproxy"
+ "golang.org/x/net/html"
+ "golang.org/x/net/html/charset"
+ "golang.org/x/text/encoding"
+
+ "marisa.chaotic.ninja/yukari"
+ "marisa.chaotic.ninja/yukari/config"
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+const (
+ STATE_DEFAULT int = 0
+ STATE_IN_STYLE int = 1
+ STATE_IN_NOSCRIPT int = 2
+)
+
+const MaxRedirectCount = 5
+
+var Gap *fasthttp.Client = &fasthttp.Client{
+ MaxResponseBodySize: 10 * 1024 * 1024, // 10M
+ ReadBufferSize: 16 * 1024, // 16K
+}
+
+var cssURLRegex *regexp.Regexp = regexp.MustCompile("url\\((['\"]?)[ \\t\\f]*([\u0009\u0021\u0023-\u0026\u0028\u002a-\u007E]+)(['\"]?)\\)?")
+
+type Proxy struct {
+ Key []byte
+ RequestTimeout time.Duration
+ FollowRedirect bool
+}
+
+type RequestConfig struct {
+ Key []byte
+ BaseURL *url.URL
+ BodyInjected bool
+}
+
+type HTMLBodyExtParam struct {
+ BaseURL string
+ HasYukariKey bool
+ URLParamName string
+}
+
+type HTMLFormExtParam struct {
+ BaseURL string
+ YukariHash string
+ URLParamName string
+ HashParamName string
+}
+type HTMLMainPageFormParam struct {
+ URLParamName string
+}
+
+var htmlFormExtension *template.Template
+var htmlBodyExtension *template.Template
+var htmlMainPageForm *template.Template
+
+//go:embed templates/yukari_content_type.html
+var htmlHeadContentType string
+//go:embed templates/yukari_start.html
+var htmlPageStart string
+//go:embed templates/yukari_stop.html
+var htmlPageStop string
+//go:embed favicon.ico
+var faviconBytes []byte
+
+func init() {
+ var err error
+ htmlFormExtension, err = template.New("html_form_extension").Parse(
+ `<input type="hidden" name="yukariurl" value="{{.BaseURL}}" />{{if .YukariHash}}<input type="hidden" name="yukarihash" value="{{.YukariHash}}" />{{end}}`)
+ if err != nil {
+ panic(err)
+ }
+ htmlBodyExtension, err = template.New("html_body_extension").Parse(`
+<div id="yukariheader">
+ <form method="get">
+ <span><a href="/">Yukari's Gap</a></span>
+ <input type="url" value="{{.BaseURL}}" name="{{.URLParamName}}" {{if .HasYukariKey }}readonly="true"{{end}} />
+ </form>
+</div>
+<style>
+body{ position: absolute !important; top: 42px !important; left: 0 !important; right: 0 !important; bottom: 0 !important; }
+#yukariheader { position: fixed; margin: 0; box-sizing: border-box; -webkit-box-sizing: border-box; top: 0; left: 0; right: 0; z-index: 2147483647 !important; font-size: 12px; line-height: normal; border-width: 0px 0px 2px 0; border-style: solid; border-color: #9826FF; background: #33004A; padding: 4px; color: #D881FF; height: 42px; }
+#yukariheader * { padding: 0; margin: 0; }
+#yukariheader p { padding: 0 0 0.7em 0; display: block; }
+#yukariheader a { color: #8934DB; font-weight: bold; display: inline; }
+#yukariheader label { text-align: right; cursor: pointer; position: fixed; right: 4px; top: 4px; display: block; color: #444; }
+#yukariheader > form > span { font-size: 24px; font-weight: bold; margin-right: 20px; margin-left: 20px; }
+#yukariheader input[type=url] { width: 50%; padding: 4px; font-size: 16px; }
+</style>
+`)
+ if err != nil {
+ panic(err)
+ }
+ htmlMainPageForm, err = template.New("html_main_page_form").Parse(`
+ <form action="post">
+ Visit url: <input placeholder="https://url.." name="{{.URLParamName}}" autofocus />
+ <input type="submit" value="go" />
+ </form>`)
+ if err != nil {
+ panic(err)
+ }
+}
+
+func (p *Proxy) RequestHandler(ctx *fasthttp.RequestCtx) {
+
+ if appRequestHandler(ctx) {
+ return
+ }
+
+ requestHash := popRequestParam(ctx, []byte(config.Config.HashParameter))
+
+ requestURI := popRequestParam(ctx, []byte(config.Config.UrlParameter))
+
+ if requestURI == nil {
+ p.serveMainPage(ctx, 200, nil)
+ return
+ }
+
+ if p.Key != nil {
+ if !verifyRequestURI(requestURI, requestHash, p.Key) {
+ // HTTP status code 403 : Forbidden
+ error_message := fmt.Sprintf(`invalid "%s" parameter. hint: Hash URL Parameter`, config.Config.HashParameter)
+ p.serveMainPage(ctx, 403, errors.New(error_message))
+ return
+ }
+ }
+
+ requestURIQuery := ctx.QueryArgs().QueryString()
+ if len(requestURIQuery) > 0 {
+ if bytes.ContainsRune(requestURI, '?') {
+ requestURI = append(requestURI, '&')
+ } else {
+ requestURI = append(requestURI, '?')
+ }
+ requestURI = append(requestURI, requestURIQuery...)
+ }
+
+ p.ProcessUri(ctx, string(requestURI), 0)
+}
+
+func (p *Proxy) ProcessUri(ctx *fasthttp.RequestCtx, requestURIStr string, redirectCount int) {
+ parsedURI, err := url.Parse(requestURIStr)
+
+ if err != nil {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+
+ if parsedURI.Scheme == "" {
+ requestURIStr = "https://" + requestURIStr
+ parsedURI, err = url.Parse(requestURIStr)
+ if err != nil {
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+ }
+
+ // Serve an intermediate page for protocols other than HTTP(S)
+ if (parsedURI.Scheme != "http" && parsedURI.Scheme != "https") || strings.HasSuffix(parsedURI.Host, ".onion") || strings.HasSuffix(parsedURI.Host, ".i2p") {
+ p.serveExitYukariPage(ctx, parsedURI)
+ return
+ }
+
+ req := fasthttp.AcquireRequest()
+ defer fasthttp.ReleaseRequest(req)
+ req.SetConnectionClose()
+
+ if config.Config.Debug {
+ log.Println(string(ctx.Method()), requestURIStr)
+ }
+
+ req.SetRequestURI(requestURIStr)
+ req.Header.SetUserAgentBytes([]byte("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"))
+
+ resp := fasthttp.AcquireResponse()
+ defer fasthttp.ReleaseResponse(resp)
+
+ req.Header.SetMethodBytes(ctx.Method())
+ if ctx.IsPost() || ctx.IsPut() {
+ req.SetBody(ctx.PostBody())
+ }
+
+ err = Gap.DoTimeout(req, resp, p.RequestTimeout)
+
+ if err != nil {
+ if err == fasthttp.ErrTimeout {
+ // HTTP status code 504 : Gateway Time-Out
+ p.serveMainPage(ctx, 504, err)
+ } else {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ }
+ return
+ }
+
+ if resp.StatusCode() != 200 {
+ switch resp.StatusCode() {
+ case 301, 302, 303, 307, 308:
+ loc := resp.Header.Peek("Location")
+ if loc != nil {
+ if p.FollowRedirect && ctx.IsGet() {
+ // GET method: Yukari follows the redirect
+ if redirectCount < MaxRedirectCount {
+ if config.Config.Debug {
+ log.Println("follow redirect to", string(loc))
+ }
+ p.ProcessUri(ctx, string(loc), redirectCount+1)
+ } else {
+ p.serveMainPage(ctx, 310, errors.New("Too many redirects"))
+ }
+ return
+ } else {
+ // Other HTTP methods: Yukari does NOT follow the redirect
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ url, err := rc.ProxifyURI(loc)
+ if err == nil {
+ ctx.SetStatusCode(resp.StatusCode())
+ ctx.Response.Header.Add("Location", url)
+ if config.Config.Debug {
+ log.Println("redirect to", string(loc))
+ }
+ return
+ }
+ }
+ }
+ }
+ error_message := fmt.Sprintf("invalid response: %d (%s)", resp.StatusCode(), requestURIStr)
+ p.serveMainPage(ctx, resp.StatusCode(), errors.New(error_message))
+ return
+ }
+
+ contentTypeBytes := resp.Header.Peek("Content-Type")
+
+ if contentTypeBytes == nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ contentTypeString := string(contentTypeBytes)
+
+ // decode Content-Type header
+ contentType, error := contenttype.ParseContentType(contentTypeString)
+ if error != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ // content-disposition
+ contentDispositionBytes := ctx.Request.Header.Peek("Content-Disposition")
+
+ // check content type
+ if !ALLOWED_CONTENTTYPE_FILTER(contentType) {
+ // it is not a usual content type
+ if ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER(contentType) {
+ // force attachment for allowed content type
+ contentDispositionBytes = contentDispositionForceAttachment(contentDispositionBytes, parsedURI)
+ } else {
+ // deny access to forbidden content type
+ // HTTP status code 403 : Forbidden
+ p.serveMainPage(ctx, 403, errors.New("forbidden content type "+parsedURI.String()))
+ return
+ }
+ }
+
+ // HACK : replace */xhtml by text/html
+ if contentType.SubType == "xhtml" {
+ contentType.TopLevelType = "text"
+ contentType.SubType = "html"
+ contentType.Suffix = ""
+ }
+
+ // conversion to UTF-8
+ var responseBody []byte
+
+ if contentType.TopLevelType == "text" {
+ e, ename, _ := charset.DetermineEncoding(resp.Body(), contentTypeString)
+ if (e != encoding.Nop) && (!strings.EqualFold("utf-8", ename)) {
+ responseBody, err = e.NewDecoder().Bytes(resp.Body())
+ if err != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, err)
+ return
+ }
+ } else {
+ responseBody = resp.Body()
+ }
+ // update the charset or specify it
+ contentType.Parameters["charset"] = "UTF-8"
+ } else {
+ responseBody = resp.Body()
+ }
+
+ //
+ contentType.FilterParameters(ALLOWED_CONTENTTYPE_PARAMETERS)
+
+ // set the content type
+ ctx.SetContentType(contentType.String())
+
+ // output according to MIME type
+ switch {
+ case contentType.SubType == "css" && contentType.Suffix == "":
+ sanitizeCSS(&RequestConfig{Key: p.Key, BaseURL: parsedURI}, ctx, responseBody)
+ case contentType.SubType == "html" && contentType.Suffix == "":
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ sanitizeHTML(rc, ctx, responseBody)
+ if !rc.BodyInjected {
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+ default:
+ if contentDispositionBytes != nil {
+ ctx.Response.Header.AddBytesV("Content-Disposition", contentDispositionBytes)
+ }
+ ctx.Write(responseBody)
+ }
+}
+
+// force content-disposition to attachment
+func contentDispositionForceAttachment(contentDispositionBytes []byte, url *url.URL) []byte {
+ var contentDispositionParams map[string]string
+
+ if contentDispositionBytes != nil {
+ var err error
+ _, contentDispositionParams, err = mime.ParseMediaType(string(contentDispositionBytes))
+ if err != nil {
+ contentDispositionParams = make(map[string]string)
+ }
+ } else {
+ contentDispositionParams = make(map[string]string)
+ }
+
+ _, fileNameDefined := contentDispositionParams["filename"]
+ if !fileNameDefined {
+ // TODO : sanitize filename
+ contentDispositionParams["fileName"] = filepath.Base(url.Path)
+ }
+
+ return []byte(mime.FormatMediaType("attachment", contentDispositionParams))
+}
+
+func appRequestHandler(ctx *fasthttp.RequestCtx) bool {
+ // serve robots.txt
+ if bytes.Equal(ctx.Path(), []byte("/robots.txt")) {
+ ctx.SetContentType("text/plain")
+ ctx.Write([]byte("User-Agent: *\nDisallow: /\n"))
+ return true
+ }
+
+ // server favicon.ico
+ if bytes.Equal(ctx.Path(), []byte("/favicon.ico")) {
+ ctx.SetContentType("image/vnd.microsoft.icon")
+ ctx.Write(faviconBytes)
+ return true
+ }
+
+ return false
+}
+
+func popRequestParam(ctx *fasthttp.RequestCtx, paramName []byte) []byte {
+ param := ctx.QueryArgs().PeekBytes(paramName)
+
+ if param == nil {
+ param = ctx.PostArgs().PeekBytes(paramName)
+ ctx.PostArgs().DelBytes(paramName)
+ }
+ ctx.QueryArgs().DelBytes(paramName)
+
+ return param
+}
+
+func sanitizeCSS(rc *RequestConfig, out io.Writer, css []byte) {
+ // TODO
+
+ urlSlices := cssURLRegex.FindAllSubmatchIndex(css, -1)
+
+ if urlSlices == nil {
+ out.Write(css)
+ return
+ }
+
+ startIndex := 0
+
+ for _, s := range urlSlices {
+ urlStart := s[4]
+ urlEnd := s[5]
+
+ if uri, err := rc.ProxifyURI(css[urlStart:urlEnd]); err == nil {
+ out.Write(css[startIndex:urlStart])
+ out.Write([]byte(uri))
+ startIndex = urlEnd
+ } else if config.Config.Debug {
+ log.Println("cannot proxify css uri:", string(css[urlStart:urlEnd]))
+ }
+ }
+ if startIndex < len(css) {
+ out.Write(css[startIndex:len(css)])
+ }
+}
+
+func sanitizeHTML(rc *RequestConfig, out io.Writer, htmlDoc []byte) {
+ r := bytes.NewReader(htmlDoc)
+ decoder := html.NewTokenizer(r)
+ decoder.AllowCDATA(true)
+
+ unsafeElements := make([][]byte, 0, 8)
+ state := STATE_DEFAULT
+ for {
+ token := decoder.Next()
+ if token == html.ErrorToken {
+ err := decoder.Err()
+ if err != io.EOF {
+ log.Println("failed to parse HTML")
+ }
+ break
+ }
+
+ if len(unsafeElements) == 0 {
+
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, hasAttrs := decoder.TagName()
+ safe := !inArray(tag, UNSAFE_ELEMENTS)
+ if !safe {
+ if token != html.SelfClosingTagToken {
+ var unsafeTag []byte = make([]byte, len(tag))
+ copy(unsafeTag, tag)
+ unsafeElements = append(unsafeElements, unsafeTag)
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("base")) {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ if bytes.Equal(attrName, []byte("href")) {
+ parsedURI, err := url.Parse(string(attrValue))
+ if err == nil {
+ rc.BaseURL = parsedURI
+ }
+ }
+ if !moreAttr {
+ break
+ }
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("noscript")) {
+ state = STATE_IN_NOSCRIPT
+ break
+ }
+ var attrs [][][]byte
+ if hasAttrs {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ attrs = append(attrs, [][]byte{
+ attrName,
+ attrValue,
+ []byte(html.EscapeString(string(attrValue))),
+ })
+ if !moreAttr {
+ break
+ }
+ }
+ }
+ if bytes.Equal(tag, []byte("link")) {
+ sanitizeLinkTag(rc, out, attrs)
+ break
+ }
+
+ if bytes.Equal(tag, []byte("meta")) {
+ sanitizeMetaTag(rc, out, attrs)
+ break
+ }
+
+ fmt.Fprintf(out, "<%s", tag)
+
+ if hasAttrs {
+ sanitizeAttrs(rc, out, attrs)
+ }
+
+ if token == html.SelfClosingTagToken {
+ fmt.Fprintf(out, " />")
+ } else {
+ fmt.Fprintf(out, ">")
+ if bytes.Equal(tag, []byte("style")) {
+ state = STATE_IN_STYLE
+ }
+ }
+
+ if bytes.Equal(tag, []byte("head")) {
+ fmt.Fprintf(out, htmlHeadContentType)
+ }
+
+ if bytes.Equal(tag, []byte("form")) {
+ var formURL *url.URL
+ for _, attr := range attrs {
+ if bytes.Equal(attr[0], []byte("action")) {
+ formURL, _ = url.Parse(string(attr[1]))
+ formURL = mergeURIs(rc.BaseURL, formURL)
+ break
+ }
+ }
+ if formURL == nil {
+ formURL = rc.BaseURL
+ }
+ urlStr := formURL.String()
+ var key string
+ if rc.Key != nil {
+ key = hash(urlStr, rc.Key)
+ }
+ err := htmlFormExtension.Execute(out, HTMLFormExtParam{urlStr, key, config.Config.UrlParameter, config.Config.HashParameter})
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ writeEndTag := true
+ switch string(tag) {
+ case "body":
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(out, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ rc.BodyInjected = true
+ case "style":
+ state = STATE_DEFAULT
+ case "noscript":
+ state = STATE_DEFAULT
+ writeEndTag = false
+ }
+ // skip noscript tags - only the tag, not the content, because javascript is sanitized
+ if writeEndTag {
+ fmt.Fprintf(out, "</%s>", tag)
+ }
+
+ case html.TextToken:
+ switch state {
+ case STATE_DEFAULT:
+ fmt.Fprintf(out, "%s", decoder.Raw())
+ case STATE_IN_STYLE:
+ sanitizeCSS(rc, out, decoder.Raw())
+ case STATE_IN_NOSCRIPT:
+ sanitizeHTML(rc, out, decoder.Raw())
+ }
+
+ case html.CommentToken:
+ // ignore comment. TODO : parse IE conditional comment
+
+ case html.DoctypeToken:
+ out.Write(decoder.Raw())
+ }
+ } else {
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, _ := decoder.TagName()
+ if inArray(tag, UNSAFE_ELEMENTS) {
+ unsafeElements = append(unsafeElements, tag)
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ if bytes.Equal(unsafeElements[len(unsafeElements)-1], tag) {
+ unsafeElements = unsafeElements[:len(unsafeElements)-1]
+ }
+ }
+ }
+ }
+}
+
+func sanitizeLinkTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ exclude := false
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("rel")) {
+ if !inArray(attrValue, LINK_REL_SAFE_VALUES) {
+ exclude = true
+ break
+ }
+ }
+ if bytes.Equal(attrName, []byte("as")) {
+ if bytes.Equal(attrValue, []byte("script")) {
+ exclude = true
+ break
+ }
+ }
+ }
+
+ if !exclude {
+ out.Write([]byte("<link"))
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+ out.Write([]byte(">"))
+ }
+}
+
+func sanitizeMetaTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ var http_equiv []byte
+ var content []byte
+
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("http-equiv")) {
+ http_equiv = bytes.ToLower(attrValue)
+ // exclude some <meta http-equiv="..." ..>
+ if !inArray(http_equiv, LINK_HTTP_EQUIV_SAFE_VALUES) {
+ return
+ }
+ }
+ if bytes.Equal(attrName, []byte("content")) {
+ content = attrValue
+ }
+ if bytes.Equal(attrName, []byte("charset")) {
+ // exclude <meta charset="...">
+ return
+ }
+ }
+
+ out.Write([]byte("<meta"))
+ urlIndex := bytes.Index(bytes.ToLower(content), []byte("url="))
+ if bytes.Equal(http_equiv, []byte("refresh")) && urlIndex != -1 {
+ contentUrl := content[urlIndex+4:]
+ // special case of <meta http-equiv="refresh" content="0; url='example.com/url.with.quote.outside'">
+ if len(contentUrl) >= 2 && (contentUrl[0] == byte('\'') || contentUrl[0] == byte('"')) {
+ if contentUrl[0] == contentUrl[len(contentUrl)-1] {
+ contentUrl = contentUrl[1 : len(contentUrl)-1]
+ }
+ }
+ // output proxify result
+ if uri, err := rc.ProxifyURI(contentUrl); err == nil {
+ fmt.Fprintf(out, ` http-equiv="refresh" content="%surl=%s"`, content[:urlIndex], uri)
+ }
+ } else {
+ if len(http_equiv) > 0 {
+ fmt.Fprintf(out, ` http-equiv="%s"`, http_equiv)
+ }
+ sanitizeAttrs(rc, out, attrs)
+ }
+ out.Write([]byte(">"))
+}
+
+func sanitizeAttrs(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+}
+
+func sanitizeAttr(rc *RequestConfig, out io.Writer, attrName, attrValue, escapedAttrValue []byte) {
+ if inArray(attrName, SAFE_ATTRIBUTES) {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, escapedAttrValue)
+ return
+ }
+ switch string(attrName) {
+ case "src", "href", "action":
+ if uri, err := rc.ProxifyURI(attrValue); err == nil {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, uri)
+ } else if config.Config.Debug {
+ log.Println("cannot proxify uri:", string(attrValue))
+ }
+ case "style":
+ cssAttr := bytes.NewBuffer(nil)
+ sanitizeCSS(rc, cssAttr, attrValue)
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, html.EscapeString(string(cssAttr.Bytes())))
+ }
+}
+
+func mergeURIs(u1, u2 *url.URL) *url.URL {
+ if u2 == nil {
+ return u1
+ }
+ return u1.ResolveReference(u2)
+}
+
+// Sanitized URI : removes all runes bellow 32 (included) as the begining and end of URI, and lower case the scheme.
+// avoid memory allocation (except for the scheme)
+func sanitizeURI(uri []byte) ([]byte, string) {
+ first_rune_index := 0
+ first_rune_seen := false
+ scheme_last_index := -1
+ buffer := bytes.NewBuffer(make([]byte, 0, 10))
+
+ // remove trailing space and special characters
+ uri = bytes.TrimRight(uri, "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0A\x0B\x0C\x0D\x0E\x0F\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1A\x1B\x1C\x1D\x1E\x1F\x20")
+
+ // loop over byte by byte
+ for i, c := range uri {
+ // ignore special characters and space (c <= 32)
+ if c > 32 {
+ // append to the lower case of the rune to buffer
+ if c < utf8.RuneSelf && 'A' <= c && c <= 'Z' {
+ c = c + 'a' - 'A'
+ }
+
+ buffer.WriteByte(c)
+
+ // update the first rune index that is not a special rune
+ if !first_rune_seen {
+ first_rune_index = i
+ first_rune_seen = true
+ }
+
+ if c == ':' {
+ // colon rune found, we have found the scheme
+ scheme_last_index = i
+ break
+ } else if c == '/' || c == '?' || c == '\\' || c == '#' {
+ // special case : most probably a relative URI
+ break
+ }
+ }
+ }
+
+ if scheme_last_index != -1 {
+ // scheme found
+ // copy the "lower case without special runes scheme" before the ":" rune
+ scheme_start_index := scheme_last_index - buffer.Len() + 1
+ copy(uri[scheme_start_index:], buffer.Bytes())
+ // and return the result
+ return uri[scheme_start_index:], buffer.String()
+ } else {
+ // scheme NOT found
+ return uri[first_rune_index:], ""
+ }
+}
+
+func (rc *RequestConfig) ProxifyURI(uri []byte) (string, error) {
+ // sanitize URI
+ uri, scheme := sanitizeURI(uri)
+
+ // remove javascript protocol
+ if scheme == "javascript:" {
+ return "", nil
+ }
+
+ // TODO check malicious data: - e.g. data:script
+ if scheme == "data:" {
+ if bytes.HasPrefix(uri, []byte("data:image/png")) ||
+ bytes.HasPrefix(uri, []byte("data:image/jpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/pjpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/gif")) ||
+ bytes.HasPrefix(uri, []byte("data:image/webp")) {
+ // should be safe
+ return string(uri), nil
+ } else {
+ // unsafe data
+ return "", nil
+ }
+ }
+
+ // parse the uri
+ u, err := url.Parse(string(uri))
+ if err != nil {
+ return "", err
+ }
+
+ // get the fragment (with the prefix "#")
+ fragment := ""
+ if len(u.Fragment) > 0 {
+ fragment = "#" + u.Fragment
+ }
+
+ // reset the fragment: it is not included in the yukariurl
+ u.Fragment = ""
+
+ // merge the URI with the document URI
+ u = mergeURIs(rc.BaseURL, u)
+
+ // simple internal link ?
+ // some web pages describe the whole link https://same:auth@same.host/same.path?same.query#new.fragment
+ if u.Scheme == rc.BaseURL.Scheme &&
+ (rc.BaseURL.User == nil || (u.User != nil && u.User.String() == rc.BaseURL.User.String())) &&
+ u.Host == rc.BaseURL.Host &&
+ u.Path == rc.BaseURL.Path &&
+ u.RawQuery == rc.BaseURL.RawQuery {
+ // the fragment is the only difference between the document URI and the uri parameter
+ return fragment, nil
+ }
+
+ // return full URI and fragment (if not empty)
+ yukari_uri := u.String()
+
+ if rc.Key == nil {
+ return fmt.Sprintf("./?%s=%s%s", config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+ }
+ return fmt.Sprintf("./?%s=%s&%s=%s%s", config.Config.HashParameter, hash(yukari_uri, rc.Key), config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+}
+
+func inArray(b []byte, a [][]byte) bool {
+ for _, b2 := range a {
+ if bytes.Equal(b, b2) {
+ return true
+ }
+ }
+ return false
+}
+
+func hash(msg string, key []byte) string {
+ mac := hmac.New(sha256.New, key)
+ mac.Write([]byte(msg))
+ return hex.EncodeToString(mac.Sum(nil))
+}
+
+func verifyRequestURI(uri, hashMsg, key []byte) bool {
+ h := make([]byte, hex.DecodedLen(len(hashMsg)))
+ _, err := hex.Decode(h, hashMsg)
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("hmac error:", err)
+ }
+ return false
+ }
+ mac := hmac.New(sha256.New, key)
+ mac.Write(uri)
+ return hmac.Equal(h, mac.Sum(nil))
+}
+
+func (p *Proxy) serveExitYukariPage(ctx *fasthttp.RequestCtx, uri *url.URL) {
+ ctx.SetContentType("text/html")
+ ctx.SetStatusCode(403)
+ ctx.Write([]byte(htmlPageStart))
+ ctx.Write([]byte("<h2>You are about to exit Yukari no Sukima</h2>"))
+ ctx.Write([]byte("<p>Following</p><p><a href=\""))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("\" rel=\"noreferrer\">"))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("</a></p><p>the content of this URL will be <b>NOT</b> sanitized.</p>"))
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func (p *Proxy) serveMainPage(ctx *fasthttp.RequestCtx, statusCode int, err error) {
+ ctx.SetContentType("text/html; charset=UTF-8")
+ ctx.SetStatusCode(statusCode)
+ ctx.Write([]byte(htmlPageStart))
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("error:", err)
+ }
+ ctx.Write([]byte("<h2>Error: "))
+ ctx.Write([]byte(html.EscapeString(err.Error())))
+ ctx.Write([]byte("</h2>"))
+ }
+ if p.Key == nil {
+ p := HTMLMainPageFormParam{config.Config.UrlParameter}
+ err := htmlMainPageForm.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject main page form", err)
+ }
+ }
+ } else {
+ ctx.Write([]byte(`<h3>Warning! This instance does not support direct URL opening.</h3>`))
+ }
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func main() {
+ config.Config.ListenAddress = "127.0.0.1:3000"
+ config.Config.Key = ""
+ config.Config.IPV6 = true
+ config.Config.Debug = false
+ config.Config.RequestTimeout = 5
+ config.Config.FollowRedirect = false
+ config.Config.UrlParameter = "yukariurl"
+ config.Config.HashParameter = "yukarihash"
+ config.Config.MaxConnsPerHost = 5
+ config.Config.ProxyEnv = false
+
+ var configFile string
+ var proxy string
+ var socks5 string
+ var version bool
+
+ flag.StringVar(&configFile, "f", "", "Configuration file")
+ flag.StringVar(&proxy, "proxy", "", "Use the specified HTTP proxy (ie: '[user:pass@]hostname:port'). Overrides: -socks5, IPv6")
+ flag.StringVar(&socks5, "socks5", "", "Use a SOCKS5 proxy (ie: 'hostname:port'). Overrides: IPv6.")
+ flag.BoolVar(&version, "version", false, "Show version")
+ flag.Parse()
+
+ if configFile != "" {
+ config.ReadConfig(configFile)
+ }
+
+ if version {
+ yukari.FullVersion()
+ return
+ }
+
+ if config.Config.ProxyEnv && os.Getenv("HTTP_PROXY") == "" && os.Getenv("HTTPS_PROXY") == "" {
+ log.Fatal("Error -proxyenv is used but no environment variables named 'HTTP_PROXY' and/or 'HTTPS_PROXY' could be found.")
+ os.Exit(1)
+ }
+
+ if config.Config.ProxyEnv {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpProxyHTTPDialer()
+ log.Println("Using environment defined proxy(ies).")
+ } else if proxy != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpHTTPDialer(proxy)
+ log.Println("Using custom HTTP proxy.")
+ } else if socks5 != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpSocksDialer(socks5)
+ log.Println("Using Socks5 proxy.")
+ } else if config.Config.IPV6 {
+ Gap.Dial = fasthttp.DialDualStack
+ log.Println("Using dual stack (IPv4/IPv6) direct connections.")
+ } else {
+ Gap.Dial = fasthttp.Dial
+ log.Println("Using IPv4 only direct connections.")
+ }
+
+ p := &Proxy{RequestTimeout: time.Duration(config.Config.RequestTimeout) * time.Second,
+ FollowRedirect: config.Config.FollowRedirect}
+
+ if config.Config.Key != "" {
+ var err error
+ p.Key, err = base64.StdEncoding.DecodeString(config.Config.Key)
+ if err != nil {
+ log.Fatal("Error parsing -key", err.Error())
+ os.Exit(1)
+ }
+ }
+ log.Println("ゆかり様、お願いします…！")
+ log.Println("Listening on", config.Config.ListenAddress)
+
+ if err := fasthttp.ListenAndServe(config.Config.ListenAddress, p.RequestHandler); err != nil {
+ log.Fatal("Error in ListenAndServe:", err)
+ }
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "net/url"
+ "testing"
+)
+
+type AttrTestCase struct {
+ AttrName []byte
+ AttrValue []byte
+ ExpectedOutput []byte
+}
+
+type SanitizeURITestCase struct {
+ Input []byte
+ ExpectedOutput []byte
+ ExpectedScheme string
+}
+
+type StringTestCase struct {
+ Input string
+ ExpectedOutput string
+}
+
+var attrTestData []*AttrTestCase = []*AttrTestCase{
+ &AttrTestCase{
+ []byte("href"),
+ []byte("./x"),
+ []byte(` href="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fx"`),
+ },
+ &AttrTestCase{
+ []byte("src"),
+ []byte("http://x.com/y"),
+ []byte(` src="./?yukariurl=http%3A%2F%2Fx.com%2Fy"`),
+ },
+ &AttrTestCase{
+ []byte("action"),
+ []byte("/z"),
+ []byte(` action="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fz"`),
+ },
+ &AttrTestCase{
+ []byte("onclick"),
+ []byte("console.log(document.cookies)"),
+ nil,
+ },
+}
+
+var sanitizeUriTestData []*SanitizeURITestCase = []*SanitizeURITestCase{
+ &SanitizeURITestCase{
+ []byte("http://example.com/"),
+ []byte("http://example.com/"),
+ "http:",
+ },
+ &SanitizeURITestCase{
+ []byte("HtTPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte(" Ht TPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte("javascript:void(0)"),
+ []byte("javascript:void(0)"),
+ "javascript:",
+ },
+ &SanitizeURITestCase{
+ []byte(" /path/to/a/file/without/protocol "),
+ []byte("/path/to/a/file/without/protocol"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" #fragment "),
+ []byte("#fragment"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" qwertyuiop "),
+ []byte("qwertyuiop"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(""),
+ []byte(""),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(":"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte(" :"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte("schéma:"),
+ []byte("schéma:"),
+ "schéma:",
+ },
+}
+
+var urlTestData []*StringTestCase = []*StringTestCase{
+ &StringTestCase{
+ "http://x.com/",
+ "./?yukariurl=http%3A%2F%2Fx.com%2F",
+ },
+ &StringTestCase{
+ "http://a@x.com/",
+ "./?yukariurl=http%3A%2F%2Fa%40x.com%2F",
+ },
+ &StringTestCase{
+ "#a",
+ "#a",
+ },
+}
+
+func TestAttrSanitizer(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range attrTestData {
+ out := bytes.NewBuffer(nil)
+ sanitizeAttr(rc, out, testCase.AttrName, testCase.AttrValue, testCase.AttrValue)
+ res, _ := out.ReadBytes(byte(0))
+ if !bytes.Equal(res, testCase.ExpectedOutput) {
+ t.Errorf(
+ `Attribute parse error. Name: "%s", Value: "%s", Expected: %s, Got: "%s"`,
+ testCase.AttrName,
+ testCase.AttrValue,
+ testCase.ExpectedOutput,
+ res,
+ )
+ }
+ }
+}
+
+func TestSanitizeURI(t *testing.T) {
+ for _, testCase := range sanitizeUriTestData {
+ newUrl, scheme := sanitizeURI(testCase.Input)
+ if !bytes.Equal(newUrl, testCase.ExpectedOutput) {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ if scheme != testCase.ExpectedScheme {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedScheme,
+ scheme,
+ )
+ }
+ }
+}
+
+func TestURLProxifier(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range urlTestData {
+ newUrl, err := rc.ProxifyURI([]byte(testCase.Input))
+ if err != nil {
+ t.Errorf("Failed to parse URL: %s", testCase.Input)
+ }
+ if newUrl != testCase.ExpectedOutput {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ }
+}
+
+var BENCH_SIMPLE_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <title>test</title>
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeSimpleHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_SIMPLE_HTML)
+ }
+}
+
+var BENCH_COMPLEX_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <noscript><meta http-equiv="refresh" content="0; URL=./xy"></noscript>
+ <title>test 2</title>
+ <script> alert('xy'); </script>
+ <link rel="stylesheet" href="./core.bundle.css">
+ <style>
+ html { background: url(./a.jpg); }
+ </style
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ <img src="b.png" alt="imgtitle" />
+ <form action="/z">
+ <input type="submit" style="background: url(http://aa.bb/cc)" >
+ </form>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeComplexHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_COMPLEX_HTML)
+ }
+}
--- /dev/null
+package main
+
+var SAFE_ATTRIBUTES [][]byte = [][]byte{
+ []byte("abbr"),
+ []byte("accesskey"),
+ []byte("align"),
+ []byte("alt"),
+ []byte("as"),
+ []byte("autocomplete"),
+ []byte("charset"),
+ []byte("checked"),
+ []byte("class"),
+ []byte("content"),
+ []byte("contenteditable"),
+ []byte("contextmenu"),
+ []byte("dir"),
+ []byte("for"),
+ []byte("height"),
+ []byte("hidden"),
+ []byte("hreflang"),
+ []byte("id"),
+ []byte("lang"),
+ []byte("media"),
+ []byte("method"),
+ []byte("name"),
+ []byte("nowrap"),
+ []byte("placeholder"),
+ []byte("property"),
+ []byte("rel"),
+ []byte("spellcheck"),
+ []byte("tabindex"),
+ []byte("target"),
+ []byte("title"),
+ []byte("translate"),
+ []byte("type"),
+ []byte("value"),
+ []byte("width"),
+}
--- /dev/null
+package main
+
+var LINK_HTTP_EQUIV_SAFE_VALUES [][]byte = [][]byte{
+ // X-UA-Compatible will be added automaticaly, so it can be skipped
+ []byte("date"),
+ []byte("last-modified"),
+ []byte("refresh"), // URL rewrite
+ []byte("content-language"),
+}
+
+var LINK_REL_SAFE_VALUES [][]byte = [][]byte{
+ []byte("alternate"),
+ []byte("archives"),
+ []byte("author"),
+ []byte("copyright"),
+ []byte("first"),
+ []byte("help"),
+ []byte("icon"),
+ []byte("index"),
+ []byte("last"),
+ []byte("license"),
+ []byte("manifest"),
+ []byte("next"),
+ []byte("pingback"),
+ []byte("prev"),
+ []byte("publisher"),
+ []byte("search"),
+ []byte("shortcut icon"),
+ []byte("stylesheet"),
+ []byte("up"),
+}
--- /dev/null
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="referrer" content="no-referrer">
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
+<html>
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=1">
+ <style>
+ html {
+ height: 100%;
+ }
+ body {
+ min-height: 100%;
+ display: flex;
+ flex-direction: column;
+ font-family: sans-serif;
+ text-align: center;
+ color: #BC48FC;
+ background: #240039;
+ margin: 0;
+ padding: 0;
+ font-size: 1.1em;
+ }
+ input {
+ border: 1px solid #888;
+ padding: 0.3em;
+ color: #BC48FC;
+ background: #202020;
+ font-size: 1.1.em;
+ }
+ input[placeholder] {
+ width: 80%;
+ }
+ a {
+ text-decoration: none;
+ color: #9529B9;
+ }
+ h1, h2 {
+ font-weight: 200;
+ margin-bottom: 2rem;
+ }
+ h1 {
+ font-size: 3em;
+ }
+ .container {
+ flex: 1;
+ min-height: 100%;
+ margin-bottom: 1em;
+ }
+ .footer {
+ margin: 1em;
+ }
+ .footer p {
+ font-size: 0.8em;
+ }
+ </style>
+ <title>Yukari's Gap</title>
+ </head>
+ <body>
+ <div class="container">
+ <h1>Yukari's Gap</h1>
+
--- /dev/null
+</div>
+<div class="footer">
+ <p>
+ Yukari's Gap rewrites web pages to exclude malicious HTML tags and CSS/HTML attributes. <br>
+ It also replaces external resource references to prevent third-party information leaks. <br>
+ </p>
+</div>
+</body>
+</html>
--- /dev/null
+package main
+
+var UNSAFE_ELEMENTS [][]byte = [][]byte{
+ []byte("applet"),
+ []byte("canvas"),
+ []byte("embed"),
+ []byte("math"),
+ []byte("script"),
+ []byte("svg"),
+}
--- /dev/null
+package config
+
+import (
+ "gopkg.in/ini.v1"
+)
+
+var Config struct {
+ Debug bool
+ ListenAddress string
+ Key string
+ IPV6 bool
+ RequestTimeout uint
+ FollowRedirect bool
+ MaxConnsPerHost uint
+ UrlParameter string
+ HashParameter string
+ ProxyEnv bool
+}
+
+func ReadConfig(file string) error {
+ cfg, err := ini.Load(file)
+ if err != nil {
+ return err
+ }
+ Config.Debug, _ = cfg.Section("yukari").Key("debug").Bool()
+ Config.ListenAddress = cfg.Section("yukari").Key("listen").String()
+ Config.Key = cfg.Section("yukari").Key("key").String()
+ Config.IPV6, _ = cfg.Section("yukari").Key("ipv6").Bool()
+ Config.RequestTimeout, _ = cfg.Section("yukari").Key("timeout").Uint()
+ Config.FollowRedirect, _ = cfg.Section("yukari").Key("followredirect").Bool()
+ Config.MaxConnsPerHost, _ = cfg.Section("yukari").Key("max_conns_per_host").Uint()
+ Config.UrlParameter = cfg.Section("yukari").Key("urlparam").String()
+ Config.HashParameter = cfg.Section("yukari").Key("hashparam").String()
+ Config.ProxyEnv, _ = cfg.Section("yukari").Key("proxyenv").Bool()
+ return nil
+}
--- /dev/null
+package contenttype
+
+import (
+ "mime"
+ "strings"
+)
+
+type ContentType struct {
+ TopLevelType string
+ SubType string
+ Suffix string
+ Parameters map[string]string
+}
+
+func (contenttype *ContentType) String() string {
+ var mimetype string
+ if contenttype.Suffix == "" {
+ if contenttype.SubType == "" {
+ mimetype = contenttype.TopLevelType
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType
+ }
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType + "+" + contenttype.Suffix
+ }
+ return mime.FormatMediaType(mimetype, contenttype.Parameters)
+}
+
+func (contenttype *ContentType) Equals(other ContentType) bool {
+ if contenttype.TopLevelType != other.TopLevelType ||
+ contenttype.SubType != other.SubType ||
+ contenttype.Suffix != other.Suffix ||
+ len(contenttype.Parameters) != len(other.Parameters) {
+ return false
+ }
+ for k, v := range contenttype.Parameters {
+ if other.Parameters[k] != v {
+ return false
+ }
+ }
+ return true
+}
+
+func (contenttype *ContentType) FilterParameters(parameters map[string]bool) {
+ for k, _ := range contenttype.Parameters {
+ if !parameters[k] {
+ delete(contenttype.Parameters, k)
+ }
+ }
+}
+
+func ParseContentType(contenttype string) (ContentType, error) {
+ mimetype, params, err := mime.ParseMediaType(contenttype)
+ if err != nil {
+ return ContentType{"", "", "", params}, err
+ }
+ splitted_mimetype := strings.SplitN(strings.ToLower(mimetype), "/", 2)
+ if len(splitted_mimetype) <= 1 {
+ return ContentType{splitted_mimetype[0], "", "", params}, nil
+ } else {
+ splitted_subtype := strings.SplitN(splitted_mimetype[1], "+", 2)
+ if len(splitted_subtype) == 1 {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], "", params}, nil
+ } else {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], splitted_subtype[1], params}, nil
+ }
+ }
+
+}
+
+type Filter func(contenttype ContentType) bool
+
+func NewFilterContains(partialMimeType string) Filter {
+ return func(contenttype ContentType) bool {
+ return strings.Contains(contenttype.TopLevelType, partialMimeType) ||
+ strings.Contains(contenttype.SubType, partialMimeType) ||
+ strings.Contains(contenttype.Suffix, partialMimeType)
+ }
+}
+
+func NewFilterEquals(TopLevelType, SubType, Suffix string) Filter {
+ return func(contenttype ContentType) bool {
+ return ((TopLevelType != "*" && TopLevelType == contenttype.TopLevelType) || (TopLevelType == "*")) &&
+ ((SubType != "*" && SubType == contenttype.SubType) || (SubType == "*")) &&
+ ((Suffix != "*" && Suffix == contenttype.Suffix) || (Suffix == "*"))
+ }
+}
+
+func NewFilterOr(contentTypeFilterList []Filter) Filter {
+ return func(contenttype ContentType) bool {
+ for _, contentTypeFilter := range contentTypeFilterList {
+ if contentTypeFilter(contenttype) {
+ return true
+ }
+ }
+ return false
+ }
+}
--- /dev/null
+package contenttype
+
+import (
+ "bytes"
+ "fmt"
+ "testing"
+)
+
+type ParseContentTypeTestCase struct {
+ Input string
+ ExpectedOutput *ContentType /* or nil if an error is expected */
+ ExpectedString *string /* or nil if equals to Input */
+}
+
+var parseContentTypeTestCases []ParseContentTypeTestCase = []ParseContentTypeTestCase{
+ ParseContentTypeTestCase{
+ "text/html",
+ &ContentType{"text", "html", "", map[string]string{}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/svg+xml; charset=UTF-8",
+ &ContentType{"text", "svg", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/",
+ nil,
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text; charset=UTF-8",
+ &ContentType{"text", "", "", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/+xml; charset=UTF-8",
+ &ContentType{"text", "", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+}
+
+type ContentTypeEqualsTestCase struct {
+ A, B ContentType
+ Equals bool
+}
+
+var Map_Empty map[string]string = map[string]string{}
+var Map_A map[string]string = map[string]string{"a": "value_a"}
+var Map_B map[string]string = map[string]string{"b": "value_b"}
+var Map_AB map[string]string = map[string]string{"a": "value_a", "b": "value_b"}
+
+var ContentType_E ContentType = ContentType{"a", "b", "c", Map_Empty}
+var ContentType_A ContentType = ContentType{"a", "b", "c", Map_A}
+var ContentType_B ContentType = ContentType{"a", "b", "c", Map_B}
+var ContentType_AB ContentType = ContentType{"a", "b", "c", Map_AB}
+
+var contentTypeEqualsTestCases []ContentTypeEqualsTestCase = []ContentTypeEqualsTestCase{
+ // TopLevelType, SubType, Suffix
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "c", Map_Empty}, true},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"o", "b", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "o", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "o", Map_Empty}, false},
+ // Parameters
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_A, true},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_B, true},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_AB, true},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_B, false},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_AB, false},
+}
+
+type FilterTestCase struct {
+ Description string
+ Input Filter
+ TrueValues []ContentType
+ FalseValues []ContentType
+}
+
+var filterTestCases []FilterTestCase = []FilterTestCase{
+ FilterTestCase{
+ "contains xml",
+ NewFilterContains("xml"),
+ []ContentType{
+ ContentType{"xml", "", "", Map_Empty},
+ ContentType{"text", "xml", "", Map_Empty},
+ ContentType{"text", "html", "xml", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "svg", "", map[string]string{"script": "javascript"}},
+ ContentType{"java", "script", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/xhtml",
+ NewFilterEquals("application", "xhtml", "*"),
+ []ContentType{
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_AB},
+ },
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "xhtml", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals application/*",
+ NewFilterEquals("application", "*", ""),
+ []ContentType{
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "xhtml", "", Map_Empty},
+ ContentType{"text", "xhtml", "xml", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications */javascript",
+ NewFilterEquals("*", "javascript", ""),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"text", "javascript", "zip", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/* or */javascript",
+ NewFilterOr([]Filter{
+ NewFilterEquals("application", "*", ""),
+ NewFilterEquals("*", "javascript", ""),
+ }),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ },
+ },
+}
+
+type FilterParametersTestCase struct {
+ Input map[string]string
+ Filter map[string]bool
+ Output map[string]string
+}
+
+var filterParametersTestCases []FilterParametersTestCase = []FilterParametersTestCase{
+ FilterParametersTestCase{
+ map[string]string{},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true},
+ map[string]string{"A": "value_A"},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{"A": "value_A", "B": "value_B"},
+ },
+}
+
+func TestContentTypeEquals(t *testing.T) {
+ for _, testCase := range contentTypeEqualsTestCases {
+ if !testCase.A.Equals(testCase.B) && testCase.Equals {
+ t.Errorf(`Must be equals "%s"="%s"`, testCase.A, testCase.B)
+ } else if testCase.A.Equals(testCase.B) && !testCase.Equals {
+ t.Errorf(`Mustn't be equals "%s"!="%s"`, testCase.A, testCase.B)
+ }
+ }
+}
+
+func TestParseContentType(t *testing.T) {
+ for _, testCase := range parseContentTypeTestCases {
+ // test ParseContentType
+ contentType, err := ParseContentType(testCase.Input)
+ if testCase.ExpectedOutput == nil {
+ // error expected
+ if err == nil {
+ // but there is no error
+ t.Errorf(`Expecting error for "%s"`, testCase.Input)
+ }
+ } else {
+ // no expected error
+ if err != nil {
+ t.Errorf(`Unexpecting error for "%s" : %s`, testCase.Input, err)
+ } else if !contentType.Equals(*testCase.ExpectedOutput) {
+ // the parsed contentType doesn't matched
+ t.Errorf(`Unexpecting result for "%s", instead got "%s"`, testCase.ExpectedOutput.String(), contentType.String())
+ } else {
+ // ParseContentType is fine, checking String()
+ contentTypeString := contentType.String()
+ expectedString := testCase.Input
+ if testCase.ExpectedString != nil {
+ expectedString = *testCase.ExpectedString
+ }
+ if contentTypeString != expectedString {
+ t.Errorf(`Error with String() output of "%s", got "%s", ContentType{"%s", "%s", "%s", "%s"}`, expectedString, contentTypeString, contentType.TopLevelType, contentType.SubType, contentType.Suffix, contentType.Parameters)
+ }
+ }
+ }
+ }
+}
+
+func FilterToString(m map[string]bool) string {
+ b := new(bytes.Buffer)
+ for key, value := range m {
+ if value {
+ fmt.Fprintf(b, "'%s'=true;", key)
+ } else {
+ fmt.Fprintf(b, "'%s'=false;", key)
+ }
+ }
+ return b.String()
+}
+
+func TestFilters(t *testing.T) {
+ for _, testCase := range filterTestCases {
+ for _, contentType := range testCase.TrueValues {
+ if !testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" must accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ for _, contentType := range testCase.FalseValues {
+ if testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" mustn't accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ }
+}
+
+func TestFilterParameters(t *testing.T) {
+ for _, testCase := range filterParametersTestCases {
+ // copy Input since the map will be modified
+ InputCopy := make(map[string]string)
+ for k, v := range testCase.Input {
+ InputCopy[k] = v
+ }
+ // apply filter
+ contentType := ContentType{"", "", "", InputCopy}
+ contentType.FilterParameters(testCase.Filter)
+ // test
+ contentTypeOutput := ContentType{"", "", "", testCase.Output}
+ if !contentTypeOutput.Equals(contentType) {
+ t.Errorf(`FilterParameters error : %s becomes %s with this filter %s`, testCase.Input, contentType.Parameters, FilterToString(testCase.Filter))
+ }
+ }
+}
--- /dev/null
+[yukari]
+debug=false
+listen="127.0.0.1:3000"
+key=""
+ipv6=true
+timeout=5
+followredirect=false
+max_conns_per_host=5
+urlparam="yukariurl"
+hashparam="yukarihash"
+proxyenv=false
--- /dev/null
+module marisa.chaotic.ninja/yukari
+
+go 1.16
+
+require (
+ github.com/stretchr/testify v1.9.0 // indirect
+ github.com/valyala/fasthttp v1.34.0
+ golang.org/x/net v0.7.0
+ golang.org/x/text v0.7.0
+ gopkg.in/ini.v1 v1.67.0
+)
--- /dev/null
+github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
+github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/klauspost/compress v1.15.0 h1:xqfchp4whNFxn5A4XFyyYtitiWI8Hy5EW59jEwcyL6U=
+github.com/klauspost/compress v1.15.0/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
+github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
+github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
+github.com/valyala/fasthttp v1.34.0 h1:d3AAQJ2DRcxJYHm7OXNXtXt2as1vMDfxeIcFvhmGGm4=
+github.com/valyala/fasthttp v1.34.0/go.mod h1:epZA5N+7pY6ZaEKRmstzOuYJx9HI8DI1oaCGZpdH4h0=
+github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20220214200702-86341886e292/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
+golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
+gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
--- /dev/null
+#!/bin/sh
+# $TheSupernovaDuo$
+
+# PROVIDE: yukari
+# REQUIRE: DAEMON NETWORKING
+# KEYWORD: shutdown
+
+. /etc/rc.subr
+
+name="yukari"
+rcvar="yukari_enable"
+
+load_rc_config "${name}"
+
+: ${yukari_enable="NO"}
+: ${yukari_config=""}
+
+pidfile="/var/run/${name}.pid"
+command="/usr/sbin/daemon"
+procname="/usr/local/bin/${name}"
+command_args="-S -m 3 -s info -l daemon -p ${pidfile} /usr/bin/env ${procname} ${yukari_args}"
+
+run_rc_command "$1"
--- /dev/null
+cmd: /usr/local/bin/yukari
+user: www
--- /dev/null
+#!/bin/ksh
+# $TheSupernovaDuo
+daemon="/usr/local/bin/yukari"
+
+. /etc/rc.d/rc.subr
+
+rc_bg=YES
+rc_reload=NO
+
+rc_cmd "$1"
--- /dev/null
+package yukari
+
+import (
+ "fmt"
+)
+
+var (
+ // Version release version
+ Version = "0.0.1"
+
+ // Commit will be overwritten automatically by the build system
+ Commit = "HEAD"
+)
+
+// FullVersion display the full version and build
+func FullVersion() string {
+ return fmt.Sprintf("%s@%s", Version, Commit)
+}
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI 1
+.Os
+.Sh NAME
+.Nm yukari
+.Nd Privacy-aware Web Content Sanitizer Proxy As A Service (WCSPAAS)
+.Sh SYNOPSIS
+.Nm
+.Op Fl f Ar string
+.Op Fl proxy Ar string
+.Op Fl proxyenv Ar bool
+.Op Fl socks5 Ar string
+.Op Fl version
+.Sh DESCRIPTION
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes.
+It also replaces external resource references in order to prevent third-party
+information leaks.
+.Pp
+The main goal of Yukari's Gap is to provide a result proxy for SearX, but it
+can be used as a standalone sanitizer service, too.
+.Sh FEATURES
+.Bl -tag -width Ds
+.It HTML sanitization
+.It Rewrites HTML/CSS external references to locals
+.It JavaScript blocking
+.It No Cookies forwarded
+.It No Referrers
+.It No Caching/ETag
+.It Supports GET/POST forms and IFrames
+.It Optional HMAC URL verifier key to prevent service abuse
+.El
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It Fl f Ar path
+Load configuration file from path
+.It Fl proxy Ar string
+Use the specified HTTP proxy (ie: [user:pass@]hostname:port),
+this overrides the
+.Fl socks5
+option and the IPv6 setting
+.It Fl proxyenv Ar bool
+Use a HTTP proxy as set in the environment (such as
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+).
+Overrides the
+.Fl proxy ,
+.Fl socks5 ,
+flags and the IPv6 setting
+.It Fl socks5 Ar string
+Use a SOCKS5 proxy (ie: hostname:port), this
+overrides the IPv6 setting
+.El
+.Sh SEE ALSO
+.Xr SearX 1
+.Sh AUTHORS
+.An Adam Tauber Aq Mt asciimoo@gmail.com
+.An Alexandre Flament Aq Mt alex@al-f.net
+.Sh MAINTAINERS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
+.Sh BUGS
+Bugs or suggestions?
+Send an email to
+.Aq Mt yukari-dev@chaotic.ninja
+.Sh LICENSE
+This program is free software: you can redistribute it and/or modify it
+under the terms of the GNU Affero General Public License as published
+by the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+.Pp
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.
+See the GNU Affero General Public License for more details.
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI.INI 5
+.Os
+.Sh NAME
+.Nm yukari.ini
+.Nd INI-style configuration file for
+.Xr yukari 1
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It debug (bool)
+Enable/disable proxy and redirection logs (default true)
+.It listen (string)
+Listen address (default "127.0.0.1:3000")
+.It key (string)
+HMAC url validation key (base64 encoded) - leave blank to disable validation
+.It ipv6 (bool)
+Enable IPv6 support for queries
+(can be overrided by the proxy options, default true)
+.It timeout (uint)
+Request timeout (default 5)
+.It followredirect (bool)
+Follow HTTP GET redirect (default false)
+.It max_conns_per_host (uint)
+How much connections are allowed per Host/IP (default 4)
+.It urlparam (string)
+User-defined requesting string URL parameter name
+(ie: '/?url=...' or '/?u=...') (default "yukariurl")
+.It hashparam (string)
+User-defined requesting string HASH parameter name
+(ie: '/?hash=...' or '/?h=...') (default "yukarihash")
+.It proxyenv (bool)
+Use a HTTP proxy as set in the environment
+(
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+) (overrides ipv6, default false)
+.El
+.Sh AUTHORS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
--- /dev/null
+vendor
+/yukari
--- /dev/null
+# v0.2.5 - 2024.03.24
+* Rename `config.readConfig` to `config.ReadConfig`
+* Assume default values if no `yukari.ini(5)` is loaded
+
+# v0.2.4 - 2024.03.24
+* Replace invalid favicon with one sourced from [here](https://en.touhouwiki.net/wiki/File:Th123YukariSigil.png), as well as using `//go:embed` for it
+* Add rc.d files for FreeBSD and OpenBSD, respectively
+
+# v0.2.3 - 2024.03.21
+* Document the configuration file format, which is INI-style (which is compatible to the old format in the codebase, though it's now called as `config.Config.<key>`)
+* Manual page has been rewritten (using `mdoc(7)`)
+* 'YukariSukima' is an incorrect transliteration, use 'Yukari no Sukima' to indicate possession/ownership
+* Remove the 'proxified and sanitized view' text as it should already be obvious
+* The font family used earlier is horrible, changed it to `sans-serif`
+* Bump required Go toolchain version to 1.16 in order to use `//go:embed`
+* Rename some all-uppercase constants/variables to camelCase (I think?), also rename CLIENT to Gap (lol)
+
+# v0.2.1 - 2023.08.26
+Applied some suggestions from the [issue tracker](https://github.com/asciimoo/morty/issues), and rebrand this fork.
+
+# v0.2.0 - 2018.05.28
+
+Man page added
+
+# v0.1.0 - 2018.01.30
+
+Initial release
--- /dev/null
+ GNU AFFERO GENERAL PUBLIC LICENSE
+ Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+ The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+ Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+ A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+ The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+ An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ TERMS AND CONDITIONS
+
+ 0. Definitions.
+
+ "This License" refers to version 3 of the GNU Affero General Public License.
+
+ "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+ "The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+ To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy. The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+ A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+ To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+ To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+ An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+ 1. Source Code.
+
+ The "source code" for a work means the preferred form of the work
+for making modifications to it. "Object code" means any non-source
+form of a work.
+
+ A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+ The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+ The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+ The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+ The Corresponding Source for a work in source code form is that
+same work.
+
+ 2. Basic Permissions.
+
+ All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+ You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force. You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright. Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+ Conveying under any other circumstances is permitted solely under
+the conditions stated below. Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+ No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+ When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+ 4. Conveying Verbatim Copies.
+
+ You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+ You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+ 5. Conveying Modified Source Versions.
+
+ You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+ a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+
+ b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under section
+ 7. This requirement modifies the requirement in section 4 to
+ "keep intact all notices".
+
+ c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+
+ d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+ A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+ 6. Conveying Non-Source Forms.
+
+ You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+ a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+
+ b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the
+ Corresponding Source from a network server at no charge.
+
+ c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+
+ d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+
+ e) Convey the object code using peer-to-peer transmission, provided
+ you inform other peers where the object code and Corresponding
+ Source of the work are being offered to the general public at no
+ charge under subsection 6d.
+
+ A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+ A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling. In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage. For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product. A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+ "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source. The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+ If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+ The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed. Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+ Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+ 7. Additional Terms.
+
+ "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+ When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+ Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+ a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+
+ b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+
+ c) Prohibiting misrepresentation of the origin of that material, or
+ requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+
+ d) Limiting the use for publicity purposes of names of licensors or
+ authors of the material; or
+
+ e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+
+ f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions of
+ it) with contractual assumptions of liability to the recipient, for
+ any liability that these contractual assumptions directly impose on
+ those licensors and authors.
+
+ All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+ If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+ Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+ 8. Termination.
+
+ You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+ However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+ Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+ Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+ 9. Acceptance Not Required for Having Copies.
+
+ You are not required to accept this License in order to receive or
+run a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+ 10. Automatic Licensing of Downstream Recipients.
+
+ Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+ An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+ You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+ 11. Patents.
+
+ A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+ A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+ Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+ In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+ If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+ If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+ A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License. You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+ Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+ 12. No Surrender of Others' Freedom.
+
+ If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all. For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+ 13. Remote Network Interaction; Use with the GNU General Public License.
+
+ Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software. This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+ Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+ 14. Revised Versions of this License.
+
+ The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time. Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+ If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+ Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+ 15. Disclaimer of Warranty.
+
+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. Limitation of Liability.
+
+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+ 17. Interpretation of Sections 15 and 16.
+
+ If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+ If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+ You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<http://www.gnu.org/licenses/>.
--- /dev/null
+GO ?= go
+RM ?= rm
+GOFLAGS ?= -v -mod=vendor
+PREFIX ?= /usr/local
+BINDIR ?= bin
+MANDIR ?= share/man
+MKDIR ?= mkdir
+CP ?= cp
+SYSCONFDIR ?= /etc
+
+VERSION = `git describe --abbrev=0 --tags 2>/dev/null || echo "$VERSION"`
+COMMIT = `git rev-parse --short HEAD || echo "$COMMIT"`
+BRANCH = `git rev-parse --abbrev-ref HEAD`
+BUILD = `git show -s --pretty=format:%cI`
+
+GOARCH ?= amd64
+GOOS ?= linux
+
+all: yukari
+
+yukari: vendor
+ env GOARCH=${GOARCH} GOOS=${GOOS} ${GO} build ${GOFLAGS} ./cmd/yukari
+clean:
+ ${RM} -f yukari
+install:
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${BINDIR}
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man5
+
+ ${CP} -f yukari ${DESTDIR}${PREFIX}/${BINDIR}
+ ${CP} -f yukari.1 ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${CP} -f yukari.ini.5 ${DESTDIR}${PREFIX}/${MANDIR}/man5
+test:
+ go test
+bench:
+ go test -benchmem -bench .
+vendor:
+ go mod vendor
+.PHONY: yukari clean install
--- /dev/null
+# Yukari's Gap
+Web content sanitizer proxy as a service[^1], fork of [MortyProxy](https://github.com/asciimoo/morty) with some suggestions from the issue tracker applied, named after [the youkai you shouldn't ever come near](https://en.touhouwiki.net/wiki/Yukari_Yakumo)
+
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes. It also replaces external resource references to prevent third party information leaks.
+
+The main goal of this tool is to provide a result proxy for [searx](https://asciimoo.github.com/searx/), but it can be used as a standalone sanitizer service too.
+
+Features:
+
+* HTML sanitization
+* Rewrites HTML/CSS external references to locals
+* JavaScript blocking
+* No Cookies forwarded
+* No Referrers
+* No Caching/Etag
+* Supports GET/POST forms and IFrames
+* Optional HMAC URL verifier key to prevent service abuse
+
+## Installation and setup
+Requirement: Go version 1.16 or higher (thus making it incompatible with MortyProxy's own requirement, but also to use `go embed`)
+
+```
+$ go install marisa.chaotic.ninja/yukari/cmd/yukari@latest
+$ "$GOPATH/bin/yukari" --help
+```
+### Usage
+See `yukari(1)`
+
+### Test
+
+```
+$ make test
+```
+
+### Benchmark
+
+```
+$ make bench
+```
+
+## Bugs
+Bugs or suggestions? Mail [yukari-dev@chaotic.ninja](mailto:yukari-dev@chaotic.ninja)
+
+---
+
+[^1]: or WCPaaS, mind you, also I didn't come up with that, it was already there when I arrived
--- /dev/null
+package main
+
+import (
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+var ALLOWED_CONTENTTYPE_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // html
+ contenttype.NewFilterEquals("text", "html", ""),
+ contenttype.NewFilterEquals("application", "xhtml", "xml"),
+ // css
+ contenttype.NewFilterEquals("text", "css", ""),
+ // images
+ contenttype.NewFilterEquals("image", "gif", ""),
+ contenttype.NewFilterEquals("image", "png", ""),
+ contenttype.NewFilterEquals("image", "jpeg", ""),
+ contenttype.NewFilterEquals("image", "pjpeg", ""),
+ contenttype.NewFilterEquals("image", "webp", ""),
+ contenttype.NewFilterEquals("image", "tiff", ""),
+ contenttype.NewFilterEquals("image", "vnd.microsoft.icon", ""),
+ contenttype.NewFilterEquals("image", "bmp", ""),
+ contenttype.NewFilterEquals("image", "x-ms-bmp", ""),
+ contenttype.NewFilterEquals("image", "x-icon", ""),
+ contenttype.NewFilterEquals("image", "svg", "xml"),
+ // fonts
+ contenttype.NewFilterEquals("application", "font-otf", ""),
+ contenttype.NewFilterEquals("application", "font-ttf", ""),
+ contenttype.NewFilterEquals("application", "font-woff", ""),
+ contenttype.NewFilterEquals("application", "vnd.ms-fontobject", ""),
+})
+
+var ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // texts
+ contenttype.NewFilterEquals("text", "csv", ""),
+ contenttype.NewFilterEquals("text", "tab-separated-values", ""),
+ contenttype.NewFilterEquals("text", "plain", ""),
+ // API
+ contenttype.NewFilterEquals("application", "json", ""),
+ // Documents
+ contenttype.NewFilterEquals("application", "x-latex", ""),
+ contenttype.NewFilterEquals("application", "pdf", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.text", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.spreadsheet", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.presentation", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.graphics", ""),
+ // Compressed archives
+ contenttype.NewFilterEquals("application", "zip", ""),
+ contenttype.NewFilterEquals("application", "gzip", ""),
+ contenttype.NewFilterEquals("application", "x-compressed", ""),
+ contenttype.NewFilterEquals("application", "x-gtar", ""),
+ contenttype.NewFilterEquals("application", "x-compress", ""),
+ // Generic binary
+ contenttype.NewFilterEquals("application", "octet-stream", ""),
+})
+
+var ALLOWED_CONTENTTYPE_PARAMETERS map[string]bool = map[string]bool{
+ "charset": true,
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "crypto/hmac"
+ "crypto/sha256"
+ _ "embed"
+ "encoding/base64"
+ "encoding/hex"
+ "errors"
+ "flag"
+ "fmt"
+ "html/template"
+ "io"
+ "log"
+ "mime"
+ "net/url"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "time"
+ "unicode/utf8"
+
+ "github.com/valyala/fasthttp"
+ "github.com/valyala/fasthttp/fasthttpproxy"
+ "golang.org/x/net/html"
+ "golang.org/x/net/html/charset"
+ "golang.org/x/text/encoding"
+
+ "marisa.chaotic.ninja/yukari"
+ "marisa.chaotic.ninja/yukari/config"
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+const (
+ STATE_DEFAULT int = 0
+ STATE_IN_STYLE int = 1
+ STATE_IN_NOSCRIPT int = 2
+)
+
+const MaxRedirectCount = 5
+
+var Gap *fasthttp.Client = &fasthttp.Client{
+ MaxResponseBodySize: 10 * 1024 * 1024, // 10M
+ ReadBufferSize: 16 * 1024, // 16K
+}
+
+var cssURLRegex *regexp.Regexp = regexp.MustCompile("url\\((['\"]?)[ \\t\\f]*([\u0009\u0021\u0023-\u0026\u0028\u002a-\u007E]+)(['\"]?)\\)?")
+
+type Proxy struct {
+ Key []byte
+ RequestTimeout time.Duration
+ FollowRedirect bool
+}
+
+type RequestConfig struct {
+ Key []byte
+ BaseURL *url.URL
+ BodyInjected bool
+}
+
+type HTMLBodyExtParam struct {
+ BaseURL string
+ HasYukariKey bool
+ URLParamName string
+}
+
+type HTMLFormExtParam struct {
+ BaseURL string
+ YukariHash string
+ URLParamName string
+ HashParamName string
+}
+type HTMLMainPageFormParam struct {
+ URLParamName string
+}
+
+var htmlFormExtension *template.Template
+var htmlBodyExtension *template.Template
+var htmlMainPageForm *template.Template
+
+//go:embed templates/yukari_content_type.html
+var htmlHeadContentType string
+//go:embed templates/yukari_start.html
+var htmlPageStart string
+//go:embed templates/yukari_stop.html
+var htmlPageStop string
+//go:embed favicon.ico
+var faviconBytes []byte
+
+func init() {
+ var err error
+ htmlFormExtension, err = template.New("html_form_extension").Parse(
+ `<input type="hidden" name="yukariurl" value="{{.BaseURL}}" />{{if .YukariHash}}<input type="hidden" name="yukarihash" value="{{.YukariHash}}" />{{end}}`)
+ if err != nil {
+ panic(err)
+ }
+ htmlBodyExtension, err = template.New("html_body_extension").Parse(`
+<div id="yukariheader">
+ <form method="get">
+ <span><a href="/">Yukari's Gap</a></span>
+ <input type="url" value="{{.BaseURL}}" name="{{.URLParamName}}" {{if .HasYukariKey }}readonly="true"{{end}} />
+ </form>
+</div>
+<style>
+body{ position: absolute !important; top: 42px !important; left: 0 !important; right: 0 !important; bottom: 0 !important; }
+#yukariheader { position: fixed; margin: 0; box-sizing: border-box; -webkit-box-sizing: border-box; top: 0; left: 0; right: 0; z-index: 2147483647 !important; font-size: 12px; line-height: normal; border-width: 0px 0px 2px 0; border-style: solid; border-color: #9826FF; background: #33004A; padding: 4px; color: #D881FF; height: 42px; }
+#yukariheader * { padding: 0; margin: 0; }
+#yukariheader p { padding: 0 0 0.7em 0; display: block; }
+#yukariheader a { color: #8934DB; font-weight: bold; display: inline; }
+#yukariheader label { text-align: right; cursor: pointer; position: fixed; right: 4px; top: 4px; display: block; color: #444; }
+#yukariheader > form > span { font-size: 24px; font-weight: bold; margin-right: 20px; margin-left: 20px; }
+#yukariheader input[type=url] { width: 50%; padding: 4px; font-size: 16px; }
+</style>
+`)
+ if err != nil {
+ panic(err)
+ }
+ htmlMainPageForm, err = template.New("html_main_page_form").Parse(`
+ <form action="post">
+ Visit url: <input placeholder="https://url.." name="{{.URLParamName}}" autofocus />
+ <input type="submit" value="go" />
+ </form>`)
+ if err != nil {
+ panic(err)
+ }
+}
+
+func (p *Proxy) RequestHandler(ctx *fasthttp.RequestCtx) {
+
+ if appRequestHandler(ctx) {
+ return
+ }
+
+ requestHash := popRequestParam(ctx, []byte(config.Config.HashParameter))
+
+ requestURI := popRequestParam(ctx, []byte(config.Config.UrlParameter))
+
+ if requestURI == nil {
+ p.serveMainPage(ctx, 200, nil)
+ return
+ }
+
+ if p.Key != nil {
+ if !verifyRequestURI(requestURI, requestHash, p.Key) {
+ // HTTP status code 403 : Forbidden
+ error_message := fmt.Sprintf(`invalid "%s" parameter. hint: Hash URL Parameter`, config.Config.HashParameter)
+ p.serveMainPage(ctx, 403, errors.New(error_message))
+ return
+ }
+ }
+
+ requestURIQuery := ctx.QueryArgs().QueryString()
+ if len(requestURIQuery) > 0 {
+ if bytes.ContainsRune(requestURI, '?') {
+ requestURI = append(requestURI, '&')
+ } else {
+ requestURI = append(requestURI, '?')
+ }
+ requestURI = append(requestURI, requestURIQuery...)
+ }
+
+ p.ProcessUri(ctx, string(requestURI), 0)
+}
+
+func (p *Proxy) ProcessUri(ctx *fasthttp.RequestCtx, requestURIStr string, redirectCount int) {
+ parsedURI, err := url.Parse(requestURIStr)
+
+ if err != nil {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+
+ if parsedURI.Scheme == "" {
+ requestURIStr = "https://" + requestURIStr
+ parsedURI, err = url.Parse(requestURIStr)
+ if err != nil {
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+ }
+
+ // Serve an intermediate page for protocols other than HTTP(S)
+ if (parsedURI.Scheme != "http" && parsedURI.Scheme != "https") || strings.HasSuffix(parsedURI.Host, ".onion") || strings.HasSuffix(parsedURI.Host, ".i2p") {
+ p.serveExitYukariPage(ctx, parsedURI)
+ return
+ }
+
+ req := fasthttp.AcquireRequest()
+ defer fasthttp.ReleaseRequest(req)
+ req.SetConnectionClose()
+
+ if config.Config.Debug {
+ log.Println(string(ctx.Method()), requestURIStr)
+ }
+
+ req.SetRequestURI(requestURIStr)
+ req.Header.SetUserAgentBytes([]byte("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"))
+
+ resp := fasthttp.AcquireResponse()
+ defer fasthttp.ReleaseResponse(resp)
+
+ req.Header.SetMethodBytes(ctx.Method())
+ if ctx.IsPost() || ctx.IsPut() {
+ req.SetBody(ctx.PostBody())
+ }
+
+ err = Gap.DoTimeout(req, resp, p.RequestTimeout)
+
+ if err != nil {
+ if err == fasthttp.ErrTimeout {
+ // HTTP status code 504 : Gateway Time-Out
+ p.serveMainPage(ctx, 504, err)
+ } else {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ }
+ return
+ }
+
+ if resp.StatusCode() != 200 {
+ switch resp.StatusCode() {
+ case 301, 302, 303, 307, 308:
+ loc := resp.Header.Peek("Location")
+ if loc != nil {
+ if p.FollowRedirect && ctx.IsGet() {
+ // GET method: Yukari follows the redirect
+ if redirectCount < MaxRedirectCount {
+ if config.Config.Debug {
+ log.Println("follow redirect to", string(loc))
+ }
+ p.ProcessUri(ctx, string(loc), redirectCount+1)
+ } else {
+ p.serveMainPage(ctx, 310, errors.New("Too many redirects"))
+ }
+ return
+ } else {
+ // Other HTTP methods: Yukari does NOT follow the redirect
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ url, err := rc.ProxifyURI(loc)
+ if err == nil {
+ ctx.SetStatusCode(resp.StatusCode())
+ ctx.Response.Header.Add("Location", url)
+ if config.Config.Debug {
+ log.Println("redirect to", string(loc))
+ }
+ return
+ }
+ }
+ }
+ }
+ error_message := fmt.Sprintf("invalid response: %d (%s)", resp.StatusCode(), requestURIStr)
+ p.serveMainPage(ctx, resp.StatusCode(), errors.New(error_message))
+ return
+ }
+
+ contentTypeBytes := resp.Header.Peek("Content-Type")
+
+ if contentTypeBytes == nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ contentTypeString := string(contentTypeBytes)
+
+ // decode Content-Type header
+ contentType, error := contenttype.ParseContentType(contentTypeString)
+ if error != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ // content-disposition
+ contentDispositionBytes := ctx.Request.Header.Peek("Content-Disposition")
+
+ // check content type
+ if !ALLOWED_CONTENTTYPE_FILTER(contentType) {
+ // it is not a usual content type
+ if ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER(contentType) {
+ // force attachment for allowed content type
+ contentDispositionBytes = contentDispositionForceAttachment(contentDispositionBytes, parsedURI)
+ } else {
+ // deny access to forbidden content type
+ // HTTP status code 403 : Forbidden
+ p.serveMainPage(ctx, 403, errors.New("forbidden content type "+parsedURI.String()))
+ return
+ }
+ }
+
+ // HACK : replace */xhtml by text/html
+ if contentType.SubType == "xhtml" {
+ contentType.TopLevelType = "text"
+ contentType.SubType = "html"
+ contentType.Suffix = ""
+ }
+
+ // conversion to UTF-8
+ var responseBody []byte
+
+ if contentType.TopLevelType == "text" {
+ e, ename, _ := charset.DetermineEncoding(resp.Body(), contentTypeString)
+ if (e != encoding.Nop) && (!strings.EqualFold("utf-8", ename)) {
+ responseBody, err = e.NewDecoder().Bytes(resp.Body())
+ if err != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, err)
+ return
+ }
+ } else {
+ responseBody = resp.Body()
+ }
+ // update the charset or specify it
+ contentType.Parameters["charset"] = "UTF-8"
+ } else {
+ responseBody = resp.Body()
+ }
+
+ //
+ contentType.FilterParameters(ALLOWED_CONTENTTYPE_PARAMETERS)
+
+ // set the content type
+ ctx.SetContentType(contentType.String())
+
+ // output according to MIME type
+ switch {
+ case contentType.SubType == "css" && contentType.Suffix == "":
+ sanitizeCSS(&RequestConfig{Key: p.Key, BaseURL: parsedURI}, ctx, responseBody)
+ case contentType.SubType == "html" && contentType.Suffix == "":
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ sanitizeHTML(rc, ctx, responseBody)
+ if !rc.BodyInjected {
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+ default:
+ if contentDispositionBytes != nil {
+ ctx.Response.Header.AddBytesV("Content-Disposition", contentDispositionBytes)
+ }
+ ctx.Write(responseBody)
+ }
+}
+
+// force content-disposition to attachment
+func contentDispositionForceAttachment(contentDispositionBytes []byte, url *url.URL) []byte {
+ var contentDispositionParams map[string]string
+
+ if contentDispositionBytes != nil {
+ var err error
+ _, contentDispositionParams, err = mime.ParseMediaType(string(contentDispositionBytes))
+ if err != nil {
+ contentDispositionParams = make(map[string]string)
+ }
+ } else {
+ contentDispositionParams = make(map[string]string)
+ }
+
+ _, fileNameDefined := contentDispositionParams["filename"]
+ if !fileNameDefined {
+ // TODO : sanitize filename
+ contentDispositionParams["fileName"] = filepath.Base(url.Path)
+ }
+
+ return []byte(mime.FormatMediaType("attachment", contentDispositionParams))
+}
+
+func appRequestHandler(ctx *fasthttp.RequestCtx) bool {
+ // serve robots.txt
+ if bytes.Equal(ctx.Path(), []byte("/robots.txt")) {
+ ctx.SetContentType("text/plain")
+ ctx.Write([]byte("User-Agent: *\nDisallow: /\n"))
+ return true
+ }
+
+ // server favicon.ico
+ if bytes.Equal(ctx.Path(), []byte("/favicon.ico")) {
+ ctx.SetContentType("image/vnd.microsoft.icon")
+ ctx.Write(faviconBytes)
+ return true
+ }
+
+ return false
+}
+
+func popRequestParam(ctx *fasthttp.RequestCtx, paramName []byte) []byte {
+ param := ctx.QueryArgs().PeekBytes(paramName)
+
+ if param == nil {
+ param = ctx.PostArgs().PeekBytes(paramName)
+ ctx.PostArgs().DelBytes(paramName)
+ }
+ ctx.QueryArgs().DelBytes(paramName)
+
+ return param
+}
+
+func sanitizeCSS(rc *RequestConfig, out io.Writer, css []byte) {
+ // TODO
+
+ urlSlices := cssURLRegex.FindAllSubmatchIndex(css, -1)
+
+ if urlSlices == nil {
+ out.Write(css)
+ return
+ }
+
+ startIndex := 0
+
+ for _, s := range urlSlices {
+ urlStart := s[4]
+ urlEnd := s[5]
+
+ if uri, err := rc.ProxifyURI(css[urlStart:urlEnd]); err == nil {
+ out.Write(css[startIndex:urlStart])
+ out.Write([]byte(uri))
+ startIndex = urlEnd
+ } else if config.Config.Debug {
+ log.Println("cannot proxify css uri:", string(css[urlStart:urlEnd]))
+ }
+ }
+ if startIndex < len(css) {
+ out.Write(css[startIndex:len(css)])
+ }
+}
+
+func sanitizeHTML(rc *RequestConfig, out io.Writer, htmlDoc []byte) {
+ r := bytes.NewReader(htmlDoc)
+ decoder := html.NewTokenizer(r)
+ decoder.AllowCDATA(true)
+
+ unsafeElements := make([][]byte, 0, 8)
+ state := STATE_DEFAULT
+ for {
+ token := decoder.Next()
+ if token == html.ErrorToken {
+ err := decoder.Err()
+ if err != io.EOF {
+ log.Println("failed to parse HTML")
+ }
+ break
+ }
+
+ if len(unsafeElements) == 0 {
+
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, hasAttrs := decoder.TagName()
+ safe := !inArray(tag, UNSAFE_ELEMENTS)
+ if !safe {
+ if token != html.SelfClosingTagToken {
+ var unsafeTag []byte = make([]byte, len(tag))
+ copy(unsafeTag, tag)
+ unsafeElements = append(unsafeElements, unsafeTag)
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("base")) {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ if bytes.Equal(attrName, []byte("href")) {
+ parsedURI, err := url.Parse(string(attrValue))
+ if err == nil {
+ rc.BaseURL = parsedURI
+ }
+ }
+ if !moreAttr {
+ break
+ }
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("noscript")) {
+ state = STATE_IN_NOSCRIPT
+ break
+ }
+ var attrs [][][]byte
+ if hasAttrs {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ attrs = append(attrs, [][]byte{
+ attrName,
+ attrValue,
+ []byte(html.EscapeString(string(attrValue))),
+ })
+ if !moreAttr {
+ break
+ }
+ }
+ }
+ if bytes.Equal(tag, []byte("link")) {
+ sanitizeLinkTag(rc, out, attrs)
+ break
+ }
+
+ if bytes.Equal(tag, []byte("meta")) {
+ sanitizeMetaTag(rc, out, attrs)
+ break
+ }
+
+ fmt.Fprintf(out, "<%s", tag)
+
+ if hasAttrs {
+ sanitizeAttrs(rc, out, attrs)
+ }
+
+ if token == html.SelfClosingTagToken {
+ fmt.Fprintf(out, " />")
+ } else {
+ fmt.Fprintf(out, ">")
+ if bytes.Equal(tag, []byte("style")) {
+ state = STATE_IN_STYLE
+ }
+ }
+
+ if bytes.Equal(tag, []byte("head")) {
+ fmt.Fprintf(out, htmlHeadContentType)
+ }
+
+ if bytes.Equal(tag, []byte("form")) {
+ var formURL *url.URL
+ for _, attr := range attrs {
+ if bytes.Equal(attr[0], []byte("action")) {
+ formURL, _ = url.Parse(string(attr[1]))
+ formURL = mergeURIs(rc.BaseURL, formURL)
+ break
+ }
+ }
+ if formURL == nil {
+ formURL = rc.BaseURL
+ }
+ urlStr := formURL.String()
+ var key string
+ if rc.Key != nil {
+ key = hash(urlStr, rc.Key)
+ }
+ err := htmlFormExtension.Execute(out, HTMLFormExtParam{urlStr, key, config.Config.UrlParameter, config.Config.HashParameter})
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ writeEndTag := true
+ switch string(tag) {
+ case "body":
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(out, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ rc.BodyInjected = true
+ case "style":
+ state = STATE_DEFAULT
+ case "noscript":
+ state = STATE_DEFAULT
+ writeEndTag = false
+ }
+ // skip noscript tags - only the tag, not the content, because javascript is sanitized
+ if writeEndTag {
+ fmt.Fprintf(out, "</%s>", tag)
+ }
+
+ case html.TextToken:
+ switch state {
+ case STATE_DEFAULT:
+ fmt.Fprintf(out, "%s", decoder.Raw())
+ case STATE_IN_STYLE:
+ sanitizeCSS(rc, out, decoder.Raw())
+ case STATE_IN_NOSCRIPT:
+ sanitizeHTML(rc, out, decoder.Raw())
+ }
+
+ case html.CommentToken:
+ // ignore comment. TODO : parse IE conditional comment
+
+ case html.DoctypeToken:
+ out.Write(decoder.Raw())
+ }
+ } else {
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, _ := decoder.TagName()
+ if inArray(tag, UNSAFE_ELEMENTS) {
+ unsafeElements = append(unsafeElements, tag)
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ if bytes.Equal(unsafeElements[len(unsafeElements)-1], tag) {
+ unsafeElements = unsafeElements[:len(unsafeElements)-1]
+ }
+ }
+ }
+ }
+}
+
+func sanitizeLinkTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ exclude := false
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("rel")) {
+ if !inArray(attrValue, LINK_REL_SAFE_VALUES) {
+ exclude = true
+ break
+ }
+ }
+ if bytes.Equal(attrName, []byte("as")) {
+ if bytes.Equal(attrValue, []byte("script")) {
+ exclude = true
+ break
+ }
+ }
+ }
+
+ if !exclude {
+ out.Write([]byte("<link"))
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+ out.Write([]byte(">"))
+ }
+}
+
+func sanitizeMetaTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ var http_equiv []byte
+ var content []byte
+
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("http-equiv")) {
+ http_equiv = bytes.ToLower(attrValue)
+ // exclude some <meta http-equiv="..." ..>
+ if !inArray(http_equiv, LINK_HTTP_EQUIV_SAFE_VALUES) {
+ return
+ }
+ }
+ if bytes.Equal(attrName, []byte("content")) {
+ content = attrValue
+ }
+ if bytes.Equal(attrName, []byte("charset")) {
+ // exclude <meta charset="...">
+ return
+ }
+ }
+
+ out.Write([]byte("<meta"))
+ urlIndex := bytes.Index(bytes.ToLower(content), []byte("url="))
+ if bytes.Equal(http_equiv, []byte("refresh")) && urlIndex != -1 {
+ contentUrl := content[urlIndex+4:]
+ // special case of <meta http-equiv="refresh" content="0; url='example.com/url.with.quote.outside'">
+ if len(contentUrl) >= 2 && (contentUrl[0] == byte('\'') || contentUrl[0] == byte('"')) {
+ if contentUrl[0] == contentUrl[len(contentUrl)-1] {
+ contentUrl = contentUrl[1 : len(contentUrl)-1]
+ }
+ }
+ // output proxify result
+ if uri, err := rc.ProxifyURI(contentUrl); err == nil {
+ fmt.Fprintf(out, ` http-equiv="refresh" content="%surl=%s"`, content[:urlIndex], uri)
+ }
+ } else {
+ if len(http_equiv) > 0 {
+ fmt.Fprintf(out, ` http-equiv="%s"`, http_equiv)
+ }
+ sanitizeAttrs(rc, out, attrs)
+ }
+ out.Write([]byte(">"))
+}
+
+func sanitizeAttrs(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+}
+
+func sanitizeAttr(rc *RequestConfig, out io.Writer, attrName, attrValue, escapedAttrValue []byte) {
+ if inArray(attrName, SAFE_ATTRIBUTES) {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, escapedAttrValue)
+ return
+ }
+ switch string(attrName) {
+ case "src", "href", "action":
+ if uri, err := rc.ProxifyURI(attrValue); err == nil {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, uri)
+ } else if config.Config.Debug {
+ log.Println("cannot proxify uri:", string(attrValue))
+ }
+ case "style":
+ cssAttr := bytes.NewBuffer(nil)
+ sanitizeCSS(rc, cssAttr, attrValue)
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, html.EscapeString(string(cssAttr.Bytes())))
+ }
+}
+
+func mergeURIs(u1, u2 *url.URL) *url.URL {
+ if u2 == nil {
+ return u1
+ }
+ return u1.ResolveReference(u2)
+}
+
+// Sanitized URI : removes all runes bellow 32 (included) as the begining and end of URI, and lower case the scheme.
+// avoid memory allocation (except for the scheme)
+func sanitizeURI(uri []byte) ([]byte, string) {
+ first_rune_index := 0
+ first_rune_seen := false
+ scheme_last_index := -1
+ buffer := bytes.NewBuffer(make([]byte, 0, 10))
+
+ // remove trailing space and special characters
+ uri = bytes.TrimRight(uri, "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0A\x0B\x0C\x0D\x0E\x0F\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1A\x1B\x1C\x1D\x1E\x1F\x20")
+
+ // loop over byte by byte
+ for i, c := range uri {
+ // ignore special characters and space (c <= 32)
+ if c > 32 {
+ // append to the lower case of the rune to buffer
+ if c < utf8.RuneSelf && 'A' <= c && c <= 'Z' {
+ c = c + 'a' - 'A'
+ }
+
+ buffer.WriteByte(c)
+
+ // update the first rune index that is not a special rune
+ if !first_rune_seen {
+ first_rune_index = i
+ first_rune_seen = true
+ }
+
+ if c == ':' {
+ // colon rune found, we have found the scheme
+ scheme_last_index = i
+ break
+ } else if c == '/' || c == '?' || c == '\\' || c == '#' {
+ // special case : most probably a relative URI
+ break
+ }
+ }
+ }
+
+ if scheme_last_index != -1 {
+ // scheme found
+ // copy the "lower case without special runes scheme" before the ":" rune
+ scheme_start_index := scheme_last_index - buffer.Len() + 1
+ copy(uri[scheme_start_index:], buffer.Bytes())
+ // and return the result
+ return uri[scheme_start_index:], buffer.String()
+ } else {
+ // scheme NOT found
+ return uri[first_rune_index:], ""
+ }
+}
+
+func (rc *RequestConfig) ProxifyURI(uri []byte) (string, error) {
+ // sanitize URI
+ uri, scheme := sanitizeURI(uri)
+
+ // remove javascript protocol
+ if scheme == "javascript:" {
+ return "", nil
+ }
+
+ // TODO check malicious data: - e.g. data:script
+ if scheme == "data:" {
+ if bytes.HasPrefix(uri, []byte("data:image/png")) ||
+ bytes.HasPrefix(uri, []byte("data:image/jpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/pjpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/gif")) ||
+ bytes.HasPrefix(uri, []byte("data:image/webp")) {
+ // should be safe
+ return string(uri), nil
+ } else {
+ // unsafe data
+ return "", nil
+ }
+ }
+
+ // parse the uri
+ u, err := url.Parse(string(uri))
+ if err != nil {
+ return "", err
+ }
+
+ // get the fragment (with the prefix "#")
+ fragment := ""
+ if len(u.Fragment) > 0 {
+ fragment = "#" + u.Fragment
+ }
+
+ // reset the fragment: it is not included in the yukariurl
+ u.Fragment = ""
+
+ // merge the URI with the document URI
+ u = mergeURIs(rc.BaseURL, u)
+
+ // simple internal link ?
+ // some web pages describe the whole link https://same:auth@same.host/same.path?same.query#new.fragment
+ if u.Scheme == rc.BaseURL.Scheme &&
+ (rc.BaseURL.User == nil || (u.User != nil && u.User.String() == rc.BaseURL.User.String())) &&
+ u.Host == rc.BaseURL.Host &&
+ u.Path == rc.BaseURL.Path &&
+ u.RawQuery == rc.BaseURL.RawQuery {
+ // the fragment is the only difference between the document URI and the uri parameter
+ return fragment, nil
+ }
+
+ // return full URI and fragment (if not empty)
+ yukari_uri := u.String()
+
+ if rc.Key == nil {
+ return fmt.Sprintf("./?%s=%s%s", config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+ }
+ return fmt.Sprintf("./?%s=%s&%s=%s%s", config.Config.HashParameter, hash(yukari_uri, rc.Key), config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+}
+
+func inArray(b []byte, a [][]byte) bool {
+ for _, b2 := range a {
+ if bytes.Equal(b, b2) {
+ return true
+ }
+ }
+ return false
+}
+
+func hash(msg string, key []byte) string {
+ mac := hmac.New(sha256.New, key)
+ mac.Write([]byte(msg))
+ return hex.EncodeToString(mac.Sum(nil))
+}
+
+func verifyRequestURI(uri, hashMsg, key []byte) bool {
+ h := make([]byte, hex.DecodedLen(len(hashMsg)))
+ _, err := hex.Decode(h, hashMsg)
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("hmac error:", err)
+ }
+ return false
+ }
+ mac := hmac.New(sha256.New, key)
+ mac.Write(uri)
+ return hmac.Equal(h, mac.Sum(nil))
+}
+
+func (p *Proxy) serveExitYukariPage(ctx *fasthttp.RequestCtx, uri *url.URL) {
+ ctx.SetContentType("text/html")
+ ctx.SetStatusCode(403)
+ ctx.Write([]byte(htmlPageStart))
+ ctx.Write([]byte("<h2>You are about to exit Yukari no Sukima</h2>"))
+ ctx.Write([]byte("<p>Following</p><p><a href=\""))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("\" rel=\"noreferrer\">"))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("</a></p><p>the content of this URL will be <b>NOT</b> sanitized.</p>"))
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func (p *Proxy) serveMainPage(ctx *fasthttp.RequestCtx, statusCode int, err error) {
+ ctx.SetContentType("text/html; charset=UTF-8")
+ ctx.SetStatusCode(statusCode)
+ ctx.Write([]byte(htmlPageStart))
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("error:", err)
+ }
+ ctx.Write([]byte("<h2>Error: "))
+ ctx.Write([]byte(html.EscapeString(err.Error())))
+ ctx.Write([]byte("</h2>"))
+ }
+ if p.Key == nil {
+ p := HTMLMainPageFormParam{config.Config.UrlParameter}
+ err := htmlMainPageForm.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject main page form", err)
+ }
+ }
+ } else {
+ ctx.Write([]byte(`<h3>Warning! This instance does not support direct URL opening.</h3>`))
+ }
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func main() {
+ config.Config.ListenAddress = "127.0.0.1:3000"
+ config.Config.Key = ""
+ config.Config.IPV6 = true
+ config.Config.Debug = false
+ config.Config.RequestTimeout = 5
+ config.Config.FollowRedirect = false
+ config.Config.UrlParameter = "yukariurl"
+ config.Config.HashParameter = "yukarihash"
+ config.Config.MaxConnsPerHost = 5
+ config.Config.ProxyEnv = false
+
+ var configFile string
+ var proxy string
+ var socks5 string
+ var version bool
+
+ flag.StringVar(&configFile, "f", "", "Configuration file")
+ flag.StringVar(&proxy, "proxy", "", "Use the specified HTTP proxy (ie: '[user:pass@]hostname:port'). Overrides: -socks5, IPv6")
+ flag.StringVar(&socks5, "socks5", "", "Use a SOCKS5 proxy (ie: 'hostname:port'). Overrides: IPv6.")
+ flag.BoolVar(&version, "version", false, "Show version")
+ flag.Parse()
+
+ if configFile != "" {
+ config.ReadConfig(configFile)
+ }
+
+ if version {
+ yukari.FullVersion()
+ return
+ }
+
+ if config.Config.ProxyEnv && os.Getenv("HTTP_PROXY") == "" && os.Getenv("HTTPS_PROXY") == "" {
+ log.Fatal("Error -proxyenv is used but no environment variables named 'HTTP_PROXY' and/or 'HTTPS_PROXY' could be found.")
+ os.Exit(1)
+ }
+
+ if config.Config.ProxyEnv {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpProxyHTTPDialer()
+ log.Println("Using environment defined proxy(ies).")
+ } else if proxy != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpHTTPDialer(proxy)
+ log.Println("Using custom HTTP proxy.")
+ } else if socks5 != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpSocksDialer(socks5)
+ log.Println("Using Socks5 proxy.")
+ } else if config.Config.IPV6 {
+ Gap.Dial = fasthttp.DialDualStack
+ log.Println("Using dual stack (IPv4/IPv6) direct connections.")
+ } else {
+ Gap.Dial = fasthttp.Dial
+ log.Println("Using IPv4 only direct connections.")
+ }
+
+ p := &Proxy{RequestTimeout: time.Duration(config.Config.RequestTimeout) * time.Second,
+ FollowRedirect: config.Config.FollowRedirect}
+
+ if config.Config.Key != "" {
+ var err error
+ p.Key, err = base64.StdEncoding.DecodeString(config.Config.Key)
+ if err != nil {
+ log.Fatal("Error parsing -key", err.Error())
+ os.Exit(1)
+ }
+ }
+ log.Println("ゆかり様、お願いします…！")
+ log.Println("Listening on", config.Config.ListenAddress)
+
+ if err := fasthttp.ListenAndServe(config.Config.ListenAddress, p.RequestHandler); err != nil {
+ log.Fatal("Error in ListenAndServe:", err)
+ }
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "net/url"
+ "testing"
+)
+
+type AttrTestCase struct {
+ AttrName []byte
+ AttrValue []byte
+ ExpectedOutput []byte
+}
+
+type SanitizeURITestCase struct {
+ Input []byte
+ ExpectedOutput []byte
+ ExpectedScheme string
+}
+
+type StringTestCase struct {
+ Input string
+ ExpectedOutput string
+}
+
+var attrTestData []*AttrTestCase = []*AttrTestCase{
+ &AttrTestCase{
+ []byte("href"),
+ []byte("./x"),
+ []byte(` href="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fx"`),
+ },
+ &AttrTestCase{
+ []byte("src"),
+ []byte("http://x.com/y"),
+ []byte(` src="./?yukariurl=http%3A%2F%2Fx.com%2Fy"`),
+ },
+ &AttrTestCase{
+ []byte("action"),
+ []byte("/z"),
+ []byte(` action="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fz"`),
+ },
+ &AttrTestCase{
+ []byte("onclick"),
+ []byte("console.log(document.cookies)"),
+ nil,
+ },
+}
+
+var sanitizeUriTestData []*SanitizeURITestCase = []*SanitizeURITestCase{
+ &SanitizeURITestCase{
+ []byte("http://example.com/"),
+ []byte("http://example.com/"),
+ "http:",
+ },
+ &SanitizeURITestCase{
+ []byte("HtTPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte(" Ht TPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte("javascript:void(0)"),
+ []byte("javascript:void(0)"),
+ "javascript:",
+ },
+ &SanitizeURITestCase{
+ []byte(" /path/to/a/file/without/protocol "),
+ []byte("/path/to/a/file/without/protocol"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" #fragment "),
+ []byte("#fragment"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" qwertyuiop "),
+ []byte("qwertyuiop"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(""),
+ []byte(""),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(":"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte(" :"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte("schéma:"),
+ []byte("schéma:"),
+ "schéma:",
+ },
+}
+
+var urlTestData []*StringTestCase = []*StringTestCase{
+ &StringTestCase{
+ "http://x.com/",
+ "./?yukariurl=http%3A%2F%2Fx.com%2F",
+ },
+ &StringTestCase{
+ "http://a@x.com/",
+ "./?yukariurl=http%3A%2F%2Fa%40x.com%2F",
+ },
+ &StringTestCase{
+ "#a",
+ "#a",
+ },
+}
+
+func TestAttrSanitizer(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range attrTestData {
+ out := bytes.NewBuffer(nil)
+ sanitizeAttr(rc, out, testCase.AttrName, testCase.AttrValue, testCase.AttrValue)
+ res, _ := out.ReadBytes(byte(0))
+ if !bytes.Equal(res, testCase.ExpectedOutput) {
+ t.Errorf(
+ `Attribute parse error. Name: "%s", Value: "%s", Expected: %s, Got: "%s"`,
+ testCase.AttrName,
+ testCase.AttrValue,
+ testCase.ExpectedOutput,
+ res,
+ )
+ }
+ }
+}
+
+func TestSanitizeURI(t *testing.T) {
+ for _, testCase := range sanitizeUriTestData {
+ newUrl, scheme := sanitizeURI(testCase.Input)
+ if !bytes.Equal(newUrl, testCase.ExpectedOutput) {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ if scheme != testCase.ExpectedScheme {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedScheme,
+ scheme,
+ )
+ }
+ }
+}
+
+func TestURLProxifier(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range urlTestData {
+ newUrl, err := rc.ProxifyURI([]byte(testCase.Input))
+ if err != nil {
+ t.Errorf("Failed to parse URL: %s", testCase.Input)
+ }
+ if newUrl != testCase.ExpectedOutput {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ }
+}
+
+var BENCH_SIMPLE_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <title>test</title>
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeSimpleHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_SIMPLE_HTML)
+ }
+}
+
+var BENCH_COMPLEX_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <noscript><meta http-equiv="refresh" content="0; URL=./xy"></noscript>
+ <title>test 2</title>
+ <script> alert('xy'); </script>
+ <link rel="stylesheet" href="./core.bundle.css">
+ <style>
+ html { background: url(./a.jpg); }
+ </style
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ <img src="b.png" alt="imgtitle" />
+ <form action="/z">
+ <input type="submit" style="background: url(http://aa.bb/cc)" >
+ </form>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeComplexHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_COMPLEX_HTML)
+ }
+}
--- /dev/null
+package main
+
+var SAFE_ATTRIBUTES [][]byte = [][]byte{
+ []byte("abbr"),
+ []byte("accesskey"),
+ []byte("align"),
+ []byte("alt"),
+ []byte("as"),
+ []byte("autocomplete"),
+ []byte("charset"),
+ []byte("checked"),
+ []byte("class"),
+ []byte("content"),
+ []byte("contenteditable"),
+ []byte("contextmenu"),
+ []byte("dir"),
+ []byte("for"),
+ []byte("height"),
+ []byte("hidden"),
+ []byte("hreflang"),
+ []byte("id"),
+ []byte("lang"),
+ []byte("media"),
+ []byte("method"),
+ []byte("name"),
+ []byte("nowrap"),
+ []byte("placeholder"),
+ []byte("property"),
+ []byte("rel"),
+ []byte("spellcheck"),
+ []byte("tabindex"),
+ []byte("target"),
+ []byte("title"),
+ []byte("translate"),
+ []byte("type"),
+ []byte("value"),
+ []byte("width"),
+}
--- /dev/null
+package main
+
+var LINK_HTTP_EQUIV_SAFE_VALUES [][]byte = [][]byte{
+ // X-UA-Compatible will be added automaticaly, so it can be skipped
+ []byte("date"),
+ []byte("last-modified"),
+ []byte("refresh"), // URL rewrite
+ []byte("content-language"),
+}
+
+var LINK_REL_SAFE_VALUES [][]byte = [][]byte{
+ []byte("alternate"),
+ []byte("archives"),
+ []byte("author"),
+ []byte("copyright"),
+ []byte("first"),
+ []byte("help"),
+ []byte("icon"),
+ []byte("index"),
+ []byte("last"),
+ []byte("license"),
+ []byte("manifest"),
+ []byte("next"),
+ []byte("pingback"),
+ []byte("prev"),
+ []byte("publisher"),
+ []byte("search"),
+ []byte("shortcut icon"),
+ []byte("stylesheet"),
+ []byte("up"),
+}
--- /dev/null
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="referrer" content="no-referrer">
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
+<html>
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=1">
+ <style>
+ html {
+ height: 100%;
+ }
+ body {
+ min-height: 100%;
+ display: flex;
+ flex-direction: column;
+ font-family: sans-serif;
+ text-align: center;
+ color: #BC48FC;
+ background: #240039;
+ margin: 0;
+ padding: 0;
+ font-size: 1.1em;
+ }
+ input {
+ border: 1px solid #888;
+ padding: 0.3em;
+ color: #BC48FC;
+ background: #202020;
+ font-size: 1.1.em;
+ }
+ input[placeholder] {
+ width: 80%;
+ }
+ a {
+ text-decoration: none;
+ color: #9529B9;
+ }
+ h1, h2 {
+ font-weight: 200;
+ margin-bottom: 2rem;
+ }
+ h1 {
+ font-size: 3em;
+ }
+ .container {
+ flex: 1;
+ min-height: 100%;
+ margin-bottom: 1em;
+ }
+ .footer {
+ margin: 1em;
+ }
+ .footer p {
+ font-size: 0.8em;
+ }
+ </style>
+ <title>Yukari's Gap</title>
+ </head>
+ <body>
+ <div class="container">
+ <h1>Yukari's Gap</h1>
+
--- /dev/null
+</div>
+<div class="footer">
+ <p>
+ Yukari's Gap rewrites web pages to exclude malicious HTML tags and CSS/HTML attributes. <br>
+ It also replaces external resource references to prevent third-party information leaks. <br>
+ </p>
+</div>
+</body>
+</html>
--- /dev/null
+package main
+
+var UNSAFE_ELEMENTS [][]byte = [][]byte{
+ []byte("applet"),
+ []byte("canvas"),
+ []byte("embed"),
+ []byte("math"),
+ []byte("script"),
+ []byte("svg"),
+}
--- /dev/null
+package config
+
+import (
+ "gopkg.in/ini.v1"
+)
+
+var Config struct {
+ Debug bool
+ ListenAddress string
+ Key string
+ IPV6 bool
+ RequestTimeout uint
+ FollowRedirect bool
+ MaxConnsPerHost uint
+ UrlParameter string
+ HashParameter string
+ ProxyEnv bool
+}
+
+func ReadConfig(file string) error {
+ cfg, err := ini.Load(file)
+ if err != nil {
+ return err
+ }
+ Config.Debug, _ = cfg.Section("yukari").Key("debug").Bool()
+ Config.ListenAddress = cfg.Section("yukari").Key("listen").String()
+ Config.Key = cfg.Section("yukari").Key("key").String()
+ Config.IPV6, _ = cfg.Section("yukari").Key("ipv6").Bool()
+ Config.RequestTimeout, _ = cfg.Section("yukari").Key("timeout").Uint()
+ Config.FollowRedirect, _ = cfg.Section("yukari").Key("followredirect").Bool()
+ Config.MaxConnsPerHost, _ = cfg.Section("yukari").Key("max_conns_per_host").Uint()
+ Config.UrlParameter = cfg.Section("yukari").Key("urlparam").String()
+ Config.HashParameter = cfg.Section("yukari").Key("hashparam").String()
+ Config.ProxyEnv, _ = cfg.Section("yukari").Key("proxyenv").Bool()
+ return nil
+}
--- /dev/null
+package contenttype
+
+import (
+ "mime"
+ "strings"
+)
+
+type ContentType struct {
+ TopLevelType string
+ SubType string
+ Suffix string
+ Parameters map[string]string
+}
+
+func (contenttype *ContentType) String() string {
+ var mimetype string
+ if contenttype.Suffix == "" {
+ if contenttype.SubType == "" {
+ mimetype = contenttype.TopLevelType
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType
+ }
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType + "+" + contenttype.Suffix
+ }
+ return mime.FormatMediaType(mimetype, contenttype.Parameters)
+}
+
+func (contenttype *ContentType) Equals(other ContentType) bool {
+ if contenttype.TopLevelType != other.TopLevelType ||
+ contenttype.SubType != other.SubType ||
+ contenttype.Suffix != other.Suffix ||
+ len(contenttype.Parameters) != len(other.Parameters) {
+ return false
+ }
+ for k, v := range contenttype.Parameters {
+ if other.Parameters[k] != v {
+ return false
+ }
+ }
+ return true
+}
+
+func (contenttype *ContentType) FilterParameters(parameters map[string]bool) {
+ for k, _ := range contenttype.Parameters {
+ if !parameters[k] {
+ delete(contenttype.Parameters, k)
+ }
+ }
+}
+
+func ParseContentType(contenttype string) (ContentType, error) {
+ mimetype, params, err := mime.ParseMediaType(contenttype)
+ if err != nil {
+ return ContentType{"", "", "", params}, err
+ }
+ splitted_mimetype := strings.SplitN(strings.ToLower(mimetype), "/", 2)
+ if len(splitted_mimetype) <= 1 {
+ return ContentType{splitted_mimetype[0], "", "", params}, nil
+ } else {
+ splitted_subtype := strings.SplitN(splitted_mimetype[1], "+", 2)
+ if len(splitted_subtype) == 1 {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], "", params}, nil
+ } else {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], splitted_subtype[1], params}, nil
+ }
+ }
+
+}
+
+type Filter func(contenttype ContentType) bool
+
+func NewFilterContains(partialMimeType string) Filter {
+ return func(contenttype ContentType) bool {
+ return strings.Contains(contenttype.TopLevelType, partialMimeType) ||
+ strings.Contains(contenttype.SubType, partialMimeType) ||
+ strings.Contains(contenttype.Suffix, partialMimeType)
+ }
+}
+
+func NewFilterEquals(TopLevelType, SubType, Suffix string) Filter {
+ return func(contenttype ContentType) bool {
+ return ((TopLevelType != "*" && TopLevelType == contenttype.TopLevelType) || (TopLevelType == "*")) &&
+ ((SubType != "*" && SubType == contenttype.SubType) || (SubType == "*")) &&
+ ((Suffix != "*" && Suffix == contenttype.Suffix) || (Suffix == "*"))
+ }
+}
+
+func NewFilterOr(contentTypeFilterList []Filter) Filter {
+ return func(contenttype ContentType) bool {
+ for _, contentTypeFilter := range contentTypeFilterList {
+ if contentTypeFilter(contenttype) {
+ return true
+ }
+ }
+ return false
+ }
+}
--- /dev/null
+package contenttype
+
+import (
+ "bytes"
+ "fmt"
+ "testing"
+)
+
+type ParseContentTypeTestCase struct {
+ Input string
+ ExpectedOutput *ContentType /* or nil if an error is expected */
+ ExpectedString *string /* or nil if equals to Input */
+}
+
+var parseContentTypeTestCases []ParseContentTypeTestCase = []ParseContentTypeTestCase{
+ ParseContentTypeTestCase{
+ "text/html",
+ &ContentType{"text", "html", "", map[string]string{}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/svg+xml; charset=UTF-8",
+ &ContentType{"text", "svg", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/",
+ nil,
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text; charset=UTF-8",
+ &ContentType{"text", "", "", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/+xml; charset=UTF-8",
+ &ContentType{"text", "", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+}
+
+type ContentTypeEqualsTestCase struct {
+ A, B ContentType
+ Equals bool
+}
+
+var Map_Empty map[string]string = map[string]string{}
+var Map_A map[string]string = map[string]string{"a": "value_a"}
+var Map_B map[string]string = map[string]string{"b": "value_b"}
+var Map_AB map[string]string = map[string]string{"a": "value_a", "b": "value_b"}
+
+var ContentType_E ContentType = ContentType{"a", "b", "c", Map_Empty}
+var ContentType_A ContentType = ContentType{"a", "b", "c", Map_A}
+var ContentType_B ContentType = ContentType{"a", "b", "c", Map_B}
+var ContentType_AB ContentType = ContentType{"a", "b", "c", Map_AB}
+
+var contentTypeEqualsTestCases []ContentTypeEqualsTestCase = []ContentTypeEqualsTestCase{
+ // TopLevelType, SubType, Suffix
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "c", Map_Empty}, true},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"o", "b", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "o", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "o", Map_Empty}, false},
+ // Parameters
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_A, true},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_B, true},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_AB, true},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_B, false},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_AB, false},
+}
+
+type FilterTestCase struct {
+ Description string
+ Input Filter
+ TrueValues []ContentType
+ FalseValues []ContentType
+}
+
+var filterTestCases []FilterTestCase = []FilterTestCase{
+ FilterTestCase{
+ "contains xml",
+ NewFilterContains("xml"),
+ []ContentType{
+ ContentType{"xml", "", "", Map_Empty},
+ ContentType{"text", "xml", "", Map_Empty},
+ ContentType{"text", "html", "xml", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "svg", "", map[string]string{"script": "javascript"}},
+ ContentType{"java", "script", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/xhtml",
+ NewFilterEquals("application", "xhtml", "*"),
+ []ContentType{
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_AB},
+ },
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "xhtml", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals application/*",
+ NewFilterEquals("application", "*", ""),
+ []ContentType{
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "xhtml", "", Map_Empty},
+ ContentType{"text", "xhtml", "xml", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications */javascript",
+ NewFilterEquals("*", "javascript", ""),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"text", "javascript", "zip", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/* or */javascript",
+ NewFilterOr([]Filter{
+ NewFilterEquals("application", "*", ""),
+ NewFilterEquals("*", "javascript", ""),
+ }),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ },
+ },
+}
+
+type FilterParametersTestCase struct {
+ Input map[string]string
+ Filter map[string]bool
+ Output map[string]string
+}
+
+var filterParametersTestCases []FilterParametersTestCase = []FilterParametersTestCase{
+ FilterParametersTestCase{
+ map[string]string{},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true},
+ map[string]string{"A": "value_A"},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{"A": "value_A", "B": "value_B"},
+ },
+}
+
+func TestContentTypeEquals(t *testing.T) {
+ for _, testCase := range contentTypeEqualsTestCases {
+ if !testCase.A.Equals(testCase.B) && testCase.Equals {
+ t.Errorf(`Must be equals "%s"="%s"`, testCase.A, testCase.B)
+ } else if testCase.A.Equals(testCase.B) && !testCase.Equals {
+ t.Errorf(`Mustn't be equals "%s"!="%s"`, testCase.A, testCase.B)
+ }
+ }
+}
+
+func TestParseContentType(t *testing.T) {
+ for _, testCase := range parseContentTypeTestCases {
+ // test ParseContentType
+ contentType, err := ParseContentType(testCase.Input)
+ if testCase.ExpectedOutput == nil {
+ // error expected
+ if err == nil {
+ // but there is no error
+ t.Errorf(`Expecting error for "%s"`, testCase.Input)
+ }
+ } else {
+ // no expected error
+ if err != nil {
+ t.Errorf(`Unexpecting error for "%s" : %s`, testCase.Input, err)
+ } else if !contentType.Equals(*testCase.ExpectedOutput) {
+ // the parsed contentType doesn't matched
+ t.Errorf(`Unexpecting result for "%s", instead got "%s"`, testCase.ExpectedOutput.String(), contentType.String())
+ } else {
+ // ParseContentType is fine, checking String()
+ contentTypeString := contentType.String()
+ expectedString := testCase.Input
+ if testCase.ExpectedString != nil {
+ expectedString = *testCase.ExpectedString
+ }
+ if contentTypeString != expectedString {
+ t.Errorf(`Error with String() output of "%s", got "%s", ContentType{"%s", "%s", "%s", "%s"}`, expectedString, contentTypeString, contentType.TopLevelType, contentType.SubType, contentType.Suffix, contentType.Parameters)
+ }
+ }
+ }
+ }
+}
+
+func FilterToString(m map[string]bool) string {
+ b := new(bytes.Buffer)
+ for key, value := range m {
+ if value {
+ fmt.Fprintf(b, "'%s'=true;", key)
+ } else {
+ fmt.Fprintf(b, "'%s'=false;", key)
+ }
+ }
+ return b.String()
+}
+
+func TestFilters(t *testing.T) {
+ for _, testCase := range filterTestCases {
+ for _, contentType := range testCase.TrueValues {
+ if !testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" must accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ for _, contentType := range testCase.FalseValues {
+ if testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" mustn't accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ }
+}
+
+func TestFilterParameters(t *testing.T) {
+ for _, testCase := range filterParametersTestCases {
+ // copy Input since the map will be modified
+ InputCopy := make(map[string]string)
+ for k, v := range testCase.Input {
+ InputCopy[k] = v
+ }
+ // apply filter
+ contentType := ContentType{"", "", "", InputCopy}
+ contentType.FilterParameters(testCase.Filter)
+ // test
+ contentTypeOutput := ContentType{"", "", "", testCase.Output}
+ if !contentTypeOutput.Equals(contentType) {
+ t.Errorf(`FilterParameters error : %s becomes %s with this filter %s`, testCase.Input, contentType.Parameters, FilterToString(testCase.Filter))
+ }
+ }
+}
--- /dev/null
+[yukari]
+debug=false
+listen="127.0.0.1:3000"
+key=""
+ipv6=true
+timeout=5
+followredirect=false
+max_conns_per_host=5
+urlparam="yukariurl"
+hashparam="yukarihash"
+proxyenv=false
--- /dev/null
+module marisa.chaotic.ninja/yukari
+
+go 1.16
+
+require (
+ github.com/stretchr/testify v1.9.0 // indirect
+ github.com/valyala/fasthttp v1.34.0
+ golang.org/x/net v0.7.0
+ golang.org/x/text v0.7.0
+ gopkg.in/ini.v1 v1.67.0
+)
--- /dev/null
+github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
+github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/klauspost/compress v1.15.0 h1:xqfchp4whNFxn5A4XFyyYtitiWI8Hy5EW59jEwcyL6U=
+github.com/klauspost/compress v1.15.0/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
+github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
+github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
+github.com/valyala/fasthttp v1.34.0 h1:d3AAQJ2DRcxJYHm7OXNXtXt2as1vMDfxeIcFvhmGGm4=
+github.com/valyala/fasthttp v1.34.0/go.mod h1:epZA5N+7pY6ZaEKRmstzOuYJx9HI8DI1oaCGZpdH4h0=
+github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20220214200702-86341886e292/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
+golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
+gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
--- /dev/null
+#!/bin/sh
+# $TheSupernovaDuo$
+
+# PROVIDE: yukari
+# REQUIRE: DAEMON NETWORKING
+# KEYWORD: shutdown
+
+. /etc/rc.subr
+
+name="yukari"
+rcvar="yukari_enable"
+
+load_rc_config "${name}"
+
+: ${yukari_enable="NO"}
+: ${yukari_config=""}
+
+pidfile="/var/run/${name}.pid"
+command="/usr/sbin/daemon"
+procname="/usr/local/bin/${name}"
+command_args="-S -m 3 -s info -l daemon -p ${pidfile} /usr/bin/env ${procname} ${yukari_args}"
+
+run_rc_command "$1"
--- /dev/null
+cmd: /usr/local/bin/yukari
+user: www
--- /dev/null
+#!/bin/ksh
+# $TheSupernovaDuo
+daemon="/usr/local/bin/yukari"
+
+. /etc/rc.d/rc.subr
+
+rc_bg=YES
+rc_reload=NO
+
+rc_cmd "$1"
--- /dev/null
+package yukari
+
+import (
+ "fmt"
+)
+
+var (
+ // Version release version
+ Version = "0.0.1"
+
+ // Commit will be overwritten automatically by the build system
+ Commit = "HEAD"
+)
+
+// FullVersion display the full version and build
+func FullVersion() string {
+ return fmt.Sprintf("%s@%s", Version, Commit)
+}
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI 1
+.Os
+.Sh NAME
+.Nm yukari
+.Nd Privacy-aware Web Content Sanitizer Proxy As A Service (WCSPAAS)
+.Sh SYNOPSIS
+.Nm
+.Op Fl f Ar string
+.Op Fl proxy Ar string
+.Op Fl proxyenv Ar bool
+.Op Fl socks5 Ar string
+.Op Fl version
+.Sh DESCRIPTION
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes.
+It also replaces external resource references in order to prevent third-party
+information leaks.
+.Pp
+The main goal of Yukari's Gap is to provide a result proxy for SearX, but it
+can be used as a standalone sanitizer service, too.
+.Sh FEATURES
+.Bl -tag -width Ds
+.It HTML sanitization
+.It Rewrites HTML/CSS external references to locals
+.It JavaScript blocking
+.It No Cookies forwarded
+.It No Referrers
+.It No Caching/ETag
+.It Supports GET/POST forms and IFrames
+.It Optional HMAC URL verifier key to prevent service abuse
+.El
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It Fl f Ar path
+Load configuration file from path
+.It Fl proxy Ar string
+Use the specified HTTP proxy (ie: [user:pass@]hostname:port),
+this overrides the
+.Fl socks5
+option and the IPv6 setting
+.It Fl proxyenv Ar bool
+Use a HTTP proxy as set in the environment (such as
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+).
+Overrides the
+.Fl proxy ,
+.Fl socks5 ,
+flags and the IPv6 setting
+.It Fl socks5 Ar string
+Use a SOCKS5 proxy (ie: hostname:port), this
+overrides the IPv6 setting
+.El
+.Sh SEE ALSO
+.Xr SearX 1
+.Sh AUTHORS
+.An Adam Tauber Aq Mt asciimoo@gmail.com
+.An Alexandre Flament Aq Mt alex@al-f.net
+.Sh MAINTAINERS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
+.Sh BUGS
+Bugs or suggestions?
+Send an email to
+.Aq Mt yukari-dev@chaotic.ninja
+.Sh LICENSE
+This program is free software: you can redistribute it and/or modify it
+under the terms of the GNU Affero General Public License as published
+by the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+.Pp
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.
+See the GNU Affero General Public License for more details.
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI.INI 5
+.Os
+.Sh NAME
+.Nm yukari.ini
+.Nd INI-style configuration file for
+.Xr yukari 1
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It debug (bool)
+Enable/disable proxy and redirection logs (default true)
+.It listen (string)
+Listen address (default "127.0.0.1:3000")
+.It key (string)
+HMAC url validation key (base64 encoded) - leave blank to disable validation
+.It ipv6 (bool)
+Enable IPv6 support for queries
+(can be overrided by the proxy options, default true)
+.It timeout (uint)
+Request timeout (default 5)
+.It followredirect (bool)
+Follow HTTP GET redirect (default false)
+.It max_conns_per_host (uint)
+How much connections are allowed per Host/IP (default 4)
+.It urlparam (string)
+User-defined requesting string URL parameter name
+(ie: '/?url=...' or '/?u=...') (default "yukariurl")
+.It hashparam (string)
+User-defined requesting string HASH parameter name
+(ie: '/?hash=...' or '/?h=...') (default "yukarihash")
+.It proxyenv (bool)
+Use a HTTP proxy as set in the environment
+(
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+) (overrides ipv6, default false)
+.El
+.Sh AUTHORS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
--- /dev/null
+vendor
+/yukari
--- /dev/null
+# v0.2.5 - 2024.03.24
+* Rename `config.readConfig` to `config.ReadConfig`
+* Assume default values if no `yukari.ini(5)` is loaded
+
+# v0.2.4 - 2024.03.24
+* Replace invalid favicon with one sourced from [here](https://en.touhouwiki.net/wiki/File:Th123YukariSigil.png), as well as using `//go:embed` for it
+* Add rc.d files for FreeBSD and OpenBSD, respectively
+
+# v0.2.3 - 2024.03.21
+* Document the configuration file format, which is INI-style (which is compatible to the old format in the codebase, though it's now called as `config.Config.<key>`)
+* Manual page has been rewritten (using `mdoc(7)`)
+* 'YukariSukima' is an incorrect transliteration, use 'Yukari no Sukima' to indicate possession/ownership
+* Remove the 'proxified and sanitized view' text as it should already be obvious
+* The font family used earlier is horrible, changed it to `sans-serif`
+* Bump required Go toolchain version to 1.16 in order to use `//go:embed`
+* Rename some all-uppercase constants/variables to camelCase (I think?), also rename CLIENT to Gap (lol)
+
+# v0.2.1 - 2023.08.26
+Applied some suggestions from the [issue tracker](https://github.com/asciimoo/morty/issues), and rebrand this fork.
+
+# v0.2.0 - 2018.05.28
+
+Man page added
+
+# v0.1.0 - 2018.01.30
+
+Initial release
--- /dev/null
+ GNU AFFERO GENERAL PUBLIC LICENSE
+ Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+ The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+ Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+ A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+ The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+ An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ TERMS AND CONDITIONS
+
+ 0. Definitions.
+
+ "This License" refers to version 3 of the GNU Affero General Public License.
+
+ "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+ "The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+ To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy. The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+ A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+ To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+ To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+ An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+ 1. Source Code.
+
+ The "source code" for a work means the preferred form of the work
+for making modifications to it. "Object code" means any non-source
+form of a work.
+
+ A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+ The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+ The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+ The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+ The Corresponding Source for a work in source code form is that
+same work.
+
+ 2. Basic Permissions.
+
+ All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+ You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force. You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright. Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+ Conveying under any other circumstances is permitted solely under
+the conditions stated below. Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+ No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+ When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+ 4. Conveying Verbatim Copies.
+
+ You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+ You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+ 5. Conveying Modified Source Versions.
+
+ You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+ a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+
+ b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under section
+ 7. This requirement modifies the requirement in section 4 to
+ "keep intact all notices".
+
+ c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+
+ d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+ A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+ 6. Conveying Non-Source Forms.
+
+ You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+ a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+
+ b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the
+ Corresponding Source from a network server at no charge.
+
+ c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+
+ d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+
+ e) Convey the object code using peer-to-peer transmission, provided
+ you inform other peers where the object code and Corresponding
+ Source of the work are being offered to the general public at no
+ charge under subsection 6d.
+
+ A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+ A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling. In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage. For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product. A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+ "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source. The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+ If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+ The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed. Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+ Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+ 7. Additional Terms.
+
+ "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+ When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+ Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+ a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+
+ b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+
+ c) Prohibiting misrepresentation of the origin of that material, or
+ requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+
+ d) Limiting the use for publicity purposes of names of licensors or
+ authors of the material; or
+
+ e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+
+ f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions of
+ it) with contractual assumptions of liability to the recipient, for
+ any liability that these contractual assumptions directly impose on
+ those licensors and authors.
+
+ All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+ If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+ Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+ 8. Termination.
+
+ You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+ However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+ Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+ Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+ 9. Acceptance Not Required for Having Copies.
+
+ You are not required to accept this License in order to receive or
+run a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+ 10. Automatic Licensing of Downstream Recipients.
+
+ Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+ An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+ You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+ 11. Patents.
+
+ A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+ A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+ Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+ In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+ If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+ If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+ A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License. You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+ Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+ 12. No Surrender of Others' Freedom.
+
+ If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all. For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+ 13. Remote Network Interaction; Use with the GNU General Public License.
+
+ Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software. This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+ Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+ 14. Revised Versions of this License.
+
+ The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time. Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+ If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+ Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+ 15. Disclaimer of Warranty.
+
+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. Limitation of Liability.
+
+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+ 17. Interpretation of Sections 15 and 16.
+
+ If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+ If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+ You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<http://www.gnu.org/licenses/>.
--- /dev/null
+GO ?= go
+RM ?= rm
+GOFLAGS ?= -v -mod=vendor
+PREFIX ?= /usr/local
+BINDIR ?= bin
+MANDIR ?= share/man
+MKDIR ?= mkdir
+CP ?= cp
+SYSCONFDIR ?= /etc
+
+VERSION = `git describe --abbrev=0 --tags 2>/dev/null || echo "$VERSION"`
+COMMIT = `git rev-parse --short HEAD || echo "$COMMIT"`
+BRANCH = `git rev-parse --abbrev-ref HEAD`
+BUILD = `git show -s --pretty=format:%cI`
+
+GOARCH ?= amd64
+GOOS ?= linux
+
+all: yukari
+
+yukari: vendor
+ env GOARCH=${GOARCH} GOOS=${GOOS} ${GO} build ${GOFLAGS} ./cmd/yukari
+clean:
+ ${RM} -f yukari
+install:
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${BINDIR}
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${MKDIR} -p ${DESTDIR}${PREFIX}/${MANDIR}/man5
+
+ ${CP} -f yukari ${DESTDIR}${PREFIX}/${BINDIR}
+ ${CP} -f yukari.1 ${DESTDIR}${PREFIX}/${MANDIR}/man1
+ ${CP} -f yukari.ini.5 ${DESTDIR}${PREFIX}/${MANDIR}/man5
+test:
+ go test
+bench:
+ go test -benchmem -bench .
+vendor:
+ go mod vendor
+.PHONY: yukari clean install
--- /dev/null
+# Yukari's Gap
+Web content sanitizer proxy as a service[^1], fork of [MortyProxy](https://github.com/asciimoo/morty) with some suggestions from the issue tracker applied, named after [the youkai you shouldn't ever come near](https://en.touhouwiki.net/wiki/Yukari_Yakumo)
+
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes. It also replaces external resource references to prevent third party information leaks.
+
+The main goal of this tool is to provide a result proxy for [searx](https://asciimoo.github.com/searx/), but it can be used as a standalone sanitizer service too.
+
+Features:
+
+* HTML sanitization
+* Rewrites HTML/CSS external references to locals
+* JavaScript blocking
+* No Cookies forwarded
+* No Referrers
+* No Caching/Etag
+* Supports GET/POST forms and IFrames
+* Optional HMAC URL verifier key to prevent service abuse
+
+## Installation and setup
+Requirement: Go version 1.16 or higher (thus making it incompatible with MortyProxy's own requirement, but also to use `go embed`)
+
+```
+$ go install marisa.chaotic.ninja/yukari/cmd/yukari@latest
+$ "$GOPATH/bin/yukari" --help
+```
+### Usage
+See `yukari(1)`
+
+### Test
+
+```
+$ make test
+```
+
+### Benchmark
+
+```
+$ make bench
+```
+
+## Bugs
+Bugs or suggestions? Mail [yukari-dev@chaotic.ninja](mailto:yukari-dev@chaotic.ninja)
+
+---
+
+[^1]: or WCPaaS, mind you, also I didn't come up with that, it was already there when I arrived
--- /dev/null
+package main
+
+import (
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+var ALLOWED_CONTENTTYPE_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // html
+ contenttype.NewFilterEquals("text", "html", ""),
+ contenttype.NewFilterEquals("application", "xhtml", "xml"),
+ // css
+ contenttype.NewFilterEquals("text", "css", ""),
+ // images
+ contenttype.NewFilterEquals("image", "gif", ""),
+ contenttype.NewFilterEquals("image", "png", ""),
+ contenttype.NewFilterEquals("image", "jpeg", ""),
+ contenttype.NewFilterEquals("image", "pjpeg", ""),
+ contenttype.NewFilterEquals("image", "webp", ""),
+ contenttype.NewFilterEquals("image", "tiff", ""),
+ contenttype.NewFilterEquals("image", "vnd.microsoft.icon", ""),
+ contenttype.NewFilterEquals("image", "bmp", ""),
+ contenttype.NewFilterEquals("image", "x-ms-bmp", ""),
+ contenttype.NewFilterEquals("image", "x-icon", ""),
+ contenttype.NewFilterEquals("image", "svg", "xml"),
+ // fonts
+ contenttype.NewFilterEquals("application", "font-otf", ""),
+ contenttype.NewFilterEquals("application", "font-ttf", ""),
+ contenttype.NewFilterEquals("application", "font-woff", ""),
+ contenttype.NewFilterEquals("application", "vnd.ms-fontobject", ""),
+})
+
+var ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER contenttype.Filter = contenttype.NewFilterOr([]contenttype.Filter{
+ // texts
+ contenttype.NewFilterEquals("text", "csv", ""),
+ contenttype.NewFilterEquals("text", "tab-separated-values", ""),
+ contenttype.NewFilterEquals("text", "plain", ""),
+ // API
+ contenttype.NewFilterEquals("application", "json", ""),
+ // Documents
+ contenttype.NewFilterEquals("application", "x-latex", ""),
+ contenttype.NewFilterEquals("application", "pdf", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.text", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.spreadsheet", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.presentation", ""),
+ contenttype.NewFilterEquals("application", "vnd.oasis.opendocument.graphics", ""),
+ // Compressed archives
+ contenttype.NewFilterEquals("application", "zip", ""),
+ contenttype.NewFilterEquals("application", "gzip", ""),
+ contenttype.NewFilterEquals("application", "x-compressed", ""),
+ contenttype.NewFilterEquals("application", "x-gtar", ""),
+ contenttype.NewFilterEquals("application", "x-compress", ""),
+ // Generic binary
+ contenttype.NewFilterEquals("application", "octet-stream", ""),
+})
+
+var ALLOWED_CONTENTTYPE_PARAMETERS map[string]bool = map[string]bool{
+ "charset": true,
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "crypto/hmac"
+ "crypto/sha256"
+ _ "embed"
+ "encoding/base64"
+ "encoding/hex"
+ "errors"
+ "flag"
+ "fmt"
+ "html/template"
+ "io"
+ "log"
+ "mime"
+ "net/url"
+ "os"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "time"
+ "unicode/utf8"
+
+ "github.com/valyala/fasthttp"
+ "github.com/valyala/fasthttp/fasthttpproxy"
+ "golang.org/x/net/html"
+ "golang.org/x/net/html/charset"
+ "golang.org/x/text/encoding"
+
+ "marisa.chaotic.ninja/yukari"
+ "marisa.chaotic.ninja/yukari/config"
+ "marisa.chaotic.ninja/yukari/contenttype"
+)
+
+const (
+ STATE_DEFAULT int = 0
+ STATE_IN_STYLE int = 1
+ STATE_IN_NOSCRIPT int = 2
+)
+
+const MaxRedirectCount = 5
+
+var Gap *fasthttp.Client = &fasthttp.Client{
+ MaxResponseBodySize: 10 * 1024 * 1024, // 10M
+ ReadBufferSize: 16 * 1024, // 16K
+}
+
+var cssURLRegex *regexp.Regexp = regexp.MustCompile("url\\((['\"]?)[ \\t\\f]*([\u0009\u0021\u0023-\u0026\u0028\u002a-\u007E]+)(['\"]?)\\)?")
+
+type Proxy struct {
+ Key []byte
+ RequestTimeout time.Duration
+ FollowRedirect bool
+}
+
+type RequestConfig struct {
+ Key []byte
+ BaseURL *url.URL
+ BodyInjected bool
+}
+
+type HTMLBodyExtParam struct {
+ BaseURL string
+ HasYukariKey bool
+ URLParamName string
+}
+
+type HTMLFormExtParam struct {
+ BaseURL string
+ YukariHash string
+ URLParamName string
+ HashParamName string
+}
+type HTMLMainPageFormParam struct {
+ URLParamName string
+}
+
+var htmlFormExtension *template.Template
+var htmlBodyExtension *template.Template
+var htmlMainPageForm *template.Template
+
+//go:embed templates/yukari_content_type.html
+var htmlHeadContentType string
+//go:embed templates/yukari_start.html
+var htmlPageStart string
+//go:embed templates/yukari_stop.html
+var htmlPageStop string
+//go:embed favicon.ico
+var faviconBytes []byte
+
+func init() {
+ var err error
+ htmlFormExtension, err = template.New("html_form_extension").Parse(
+ `<input type="hidden" name="yukariurl" value="{{.BaseURL}}" />{{if .YukariHash}}<input type="hidden" name="yukarihash" value="{{.YukariHash}}" />{{end}}`)
+ if err != nil {
+ panic(err)
+ }
+ htmlBodyExtension, err = template.New("html_body_extension").Parse(`
+<div id="yukariheader">
+ <form method="get">
+ <span><a href="/">Yukari's Gap</a></span>
+ <input type="url" value="{{.BaseURL}}" name="{{.URLParamName}}" {{if .HasYukariKey }}readonly="true"{{end}} />
+ </form>
+</div>
+<style>
+body{ position: absolute !important; top: 42px !important; left: 0 !important; right: 0 !important; bottom: 0 !important; }
+#yukariheader { position: fixed; margin: 0; box-sizing: border-box; -webkit-box-sizing: border-box; top: 0; left: 0; right: 0; z-index: 2147483647 !important; font-size: 12px; line-height: normal; border-width: 0px 0px 2px 0; border-style: solid; border-color: #9826FF; background: #33004A; padding: 4px; color: #D881FF; height: 42px; }
+#yukariheader * { padding: 0; margin: 0; }
+#yukariheader p { padding: 0 0 0.7em 0; display: block; }
+#yukariheader a { color: #8934DB; font-weight: bold; display: inline; }
+#yukariheader label { text-align: right; cursor: pointer; position: fixed; right: 4px; top: 4px; display: block; color: #444; }
+#yukariheader > form > span { font-size: 24px; font-weight: bold; margin-right: 20px; margin-left: 20px; }
+#yukariheader input[type=url] { width: 50%; padding: 4px; font-size: 16px; }
+</style>
+`)
+ if err != nil {
+ panic(err)
+ }
+ htmlMainPageForm, err = template.New("html_main_page_form").Parse(`
+ <form action="post">
+ Visit url: <input placeholder="https://url.." name="{{.URLParamName}}" autofocus />
+ <input type="submit" value="go" />
+ </form>`)
+ if err != nil {
+ panic(err)
+ }
+}
+
+func (p *Proxy) RequestHandler(ctx *fasthttp.RequestCtx) {
+
+ if appRequestHandler(ctx) {
+ return
+ }
+
+ requestHash := popRequestParam(ctx, []byte(config.Config.HashParameter))
+
+ requestURI := popRequestParam(ctx, []byte(config.Config.UrlParameter))
+
+ if requestURI == nil {
+ p.serveMainPage(ctx, 200, nil)
+ return
+ }
+
+ if p.Key != nil {
+ if !verifyRequestURI(requestURI, requestHash, p.Key) {
+ // HTTP status code 403 : Forbidden
+ error_message := fmt.Sprintf(`invalid "%s" parameter. hint: Hash URL Parameter`, config.Config.HashParameter)
+ p.serveMainPage(ctx, 403, errors.New(error_message))
+ return
+ }
+ }
+
+ requestURIQuery := ctx.QueryArgs().QueryString()
+ if len(requestURIQuery) > 0 {
+ if bytes.ContainsRune(requestURI, '?') {
+ requestURI = append(requestURI, '&')
+ } else {
+ requestURI = append(requestURI, '?')
+ }
+ requestURI = append(requestURI, requestURIQuery...)
+ }
+
+ p.ProcessUri(ctx, string(requestURI), 0)
+}
+
+func (p *Proxy) ProcessUri(ctx *fasthttp.RequestCtx, requestURIStr string, redirectCount int) {
+ parsedURI, err := url.Parse(requestURIStr)
+
+ if err != nil {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+
+ if parsedURI.Scheme == "" {
+ requestURIStr = "https://" + requestURIStr
+ parsedURI, err = url.Parse(requestURIStr)
+ if err != nil {
+ p.serveMainPage(ctx, 500, err)
+ return
+ }
+ }
+
+ // Serve an intermediate page for protocols other than HTTP(S)
+ if (parsedURI.Scheme != "http" && parsedURI.Scheme != "https") || strings.HasSuffix(parsedURI.Host, ".onion") || strings.HasSuffix(parsedURI.Host, ".i2p") {
+ p.serveExitYukariPage(ctx, parsedURI)
+ return
+ }
+
+ req := fasthttp.AcquireRequest()
+ defer fasthttp.ReleaseRequest(req)
+ req.SetConnectionClose()
+
+ if config.Config.Debug {
+ log.Println(string(ctx.Method()), requestURIStr)
+ }
+
+ req.SetRequestURI(requestURIStr)
+ req.Header.SetUserAgentBytes([]byte("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36"))
+
+ resp := fasthttp.AcquireResponse()
+ defer fasthttp.ReleaseResponse(resp)
+
+ req.Header.SetMethodBytes(ctx.Method())
+ if ctx.IsPost() || ctx.IsPut() {
+ req.SetBody(ctx.PostBody())
+ }
+
+ err = Gap.DoTimeout(req, resp, p.RequestTimeout)
+
+ if err != nil {
+ if err == fasthttp.ErrTimeout {
+ // HTTP status code 504 : Gateway Time-Out
+ p.serveMainPage(ctx, 504, err)
+ } else {
+ // HTTP status code 500 : Internal Server Error
+ p.serveMainPage(ctx, 500, err)
+ }
+ return
+ }
+
+ if resp.StatusCode() != 200 {
+ switch resp.StatusCode() {
+ case 301, 302, 303, 307, 308:
+ loc := resp.Header.Peek("Location")
+ if loc != nil {
+ if p.FollowRedirect && ctx.IsGet() {
+ // GET method: Yukari follows the redirect
+ if redirectCount < MaxRedirectCount {
+ if config.Config.Debug {
+ log.Println("follow redirect to", string(loc))
+ }
+ p.ProcessUri(ctx, string(loc), redirectCount+1)
+ } else {
+ p.serveMainPage(ctx, 310, errors.New("Too many redirects"))
+ }
+ return
+ } else {
+ // Other HTTP methods: Yukari does NOT follow the redirect
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ url, err := rc.ProxifyURI(loc)
+ if err == nil {
+ ctx.SetStatusCode(resp.StatusCode())
+ ctx.Response.Header.Add("Location", url)
+ if config.Config.Debug {
+ log.Println("redirect to", string(loc))
+ }
+ return
+ }
+ }
+ }
+ }
+ error_message := fmt.Sprintf("invalid response: %d (%s)", resp.StatusCode(), requestURIStr)
+ p.serveMainPage(ctx, resp.StatusCode(), errors.New(error_message))
+ return
+ }
+
+ contentTypeBytes := resp.Header.Peek("Content-Type")
+
+ if contentTypeBytes == nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ contentTypeString := string(contentTypeBytes)
+
+ // decode Content-Type header
+ contentType, error := contenttype.ParseContentType(contentTypeString)
+ if error != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, errors.New("invalid content type"))
+ return
+ }
+
+ // content-disposition
+ contentDispositionBytes := ctx.Request.Header.Peek("Content-Disposition")
+
+ // check content type
+ if !ALLOWED_CONTENTTYPE_FILTER(contentType) {
+ // it is not a usual content type
+ if ALLOWED_CONTENTTYPE_ATTACHMENT_FILTER(contentType) {
+ // force attachment for allowed content type
+ contentDispositionBytes = contentDispositionForceAttachment(contentDispositionBytes, parsedURI)
+ } else {
+ // deny access to forbidden content type
+ // HTTP status code 403 : Forbidden
+ p.serveMainPage(ctx, 403, errors.New("forbidden content type "+parsedURI.String()))
+ return
+ }
+ }
+
+ // HACK : replace */xhtml by text/html
+ if contentType.SubType == "xhtml" {
+ contentType.TopLevelType = "text"
+ contentType.SubType = "html"
+ contentType.Suffix = ""
+ }
+
+ // conversion to UTF-8
+ var responseBody []byte
+
+ if contentType.TopLevelType == "text" {
+ e, ename, _ := charset.DetermineEncoding(resp.Body(), contentTypeString)
+ if (e != encoding.Nop) && (!strings.EqualFold("utf-8", ename)) {
+ responseBody, err = e.NewDecoder().Bytes(resp.Body())
+ if err != nil {
+ // HTTP status code 503 : Service Unavailable
+ p.serveMainPage(ctx, 503, err)
+ return
+ }
+ } else {
+ responseBody = resp.Body()
+ }
+ // update the charset or specify it
+ contentType.Parameters["charset"] = "UTF-8"
+ } else {
+ responseBody = resp.Body()
+ }
+
+ //
+ contentType.FilterParameters(ALLOWED_CONTENTTYPE_PARAMETERS)
+
+ // set the content type
+ ctx.SetContentType(contentType.String())
+
+ // output according to MIME type
+ switch {
+ case contentType.SubType == "css" && contentType.Suffix == "":
+ sanitizeCSS(&RequestConfig{Key: p.Key, BaseURL: parsedURI}, ctx, responseBody)
+ case contentType.SubType == "html" && contentType.Suffix == "":
+ rc := &RequestConfig{Key: p.Key, BaseURL: parsedURI}
+ sanitizeHTML(rc, ctx, responseBody)
+ if !rc.BodyInjected {
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+ default:
+ if contentDispositionBytes != nil {
+ ctx.Response.Header.AddBytesV("Content-Disposition", contentDispositionBytes)
+ }
+ ctx.Write(responseBody)
+ }
+}
+
+// force content-disposition to attachment
+func contentDispositionForceAttachment(contentDispositionBytes []byte, url *url.URL) []byte {
+ var contentDispositionParams map[string]string
+
+ if contentDispositionBytes != nil {
+ var err error
+ _, contentDispositionParams, err = mime.ParseMediaType(string(contentDispositionBytes))
+ if err != nil {
+ contentDispositionParams = make(map[string]string)
+ }
+ } else {
+ contentDispositionParams = make(map[string]string)
+ }
+
+ _, fileNameDefined := contentDispositionParams["filename"]
+ if !fileNameDefined {
+ // TODO : sanitize filename
+ contentDispositionParams["fileName"] = filepath.Base(url.Path)
+ }
+
+ return []byte(mime.FormatMediaType("attachment", contentDispositionParams))
+}
+
+func appRequestHandler(ctx *fasthttp.RequestCtx) bool {
+ // serve robots.txt
+ if bytes.Equal(ctx.Path(), []byte("/robots.txt")) {
+ ctx.SetContentType("text/plain")
+ ctx.Write([]byte("User-Agent: *\nDisallow: /\n"))
+ return true
+ }
+
+ // server favicon.ico
+ if bytes.Equal(ctx.Path(), []byte("/favicon.ico")) {
+ ctx.SetContentType("image/vnd.microsoft.icon")
+ ctx.Write(faviconBytes)
+ return true
+ }
+
+ return false
+}
+
+func popRequestParam(ctx *fasthttp.RequestCtx, paramName []byte) []byte {
+ param := ctx.QueryArgs().PeekBytes(paramName)
+
+ if param == nil {
+ param = ctx.PostArgs().PeekBytes(paramName)
+ ctx.PostArgs().DelBytes(paramName)
+ }
+ ctx.QueryArgs().DelBytes(paramName)
+
+ return param
+}
+
+func sanitizeCSS(rc *RequestConfig, out io.Writer, css []byte) {
+ // TODO
+
+ urlSlices := cssURLRegex.FindAllSubmatchIndex(css, -1)
+
+ if urlSlices == nil {
+ out.Write(css)
+ return
+ }
+
+ startIndex := 0
+
+ for _, s := range urlSlices {
+ urlStart := s[4]
+ urlEnd := s[5]
+
+ if uri, err := rc.ProxifyURI(css[urlStart:urlEnd]); err == nil {
+ out.Write(css[startIndex:urlStart])
+ out.Write([]byte(uri))
+ startIndex = urlEnd
+ } else if config.Config.Debug {
+ log.Println("cannot proxify css uri:", string(css[urlStart:urlEnd]))
+ }
+ }
+ if startIndex < len(css) {
+ out.Write(css[startIndex:len(css)])
+ }
+}
+
+func sanitizeHTML(rc *RequestConfig, out io.Writer, htmlDoc []byte) {
+ r := bytes.NewReader(htmlDoc)
+ decoder := html.NewTokenizer(r)
+ decoder.AllowCDATA(true)
+
+ unsafeElements := make([][]byte, 0, 8)
+ state := STATE_DEFAULT
+ for {
+ token := decoder.Next()
+ if token == html.ErrorToken {
+ err := decoder.Err()
+ if err != io.EOF {
+ log.Println("failed to parse HTML")
+ }
+ break
+ }
+
+ if len(unsafeElements) == 0 {
+
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, hasAttrs := decoder.TagName()
+ safe := !inArray(tag, UNSAFE_ELEMENTS)
+ if !safe {
+ if token != html.SelfClosingTagToken {
+ var unsafeTag []byte = make([]byte, len(tag))
+ copy(unsafeTag, tag)
+ unsafeElements = append(unsafeElements, unsafeTag)
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("base")) {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ if bytes.Equal(attrName, []byte("href")) {
+ parsedURI, err := url.Parse(string(attrValue))
+ if err == nil {
+ rc.BaseURL = parsedURI
+ }
+ }
+ if !moreAttr {
+ break
+ }
+ }
+ break
+ }
+ if bytes.Equal(tag, []byte("noscript")) {
+ state = STATE_IN_NOSCRIPT
+ break
+ }
+ var attrs [][][]byte
+ if hasAttrs {
+ for {
+ attrName, attrValue, moreAttr := decoder.TagAttr()
+ attrs = append(attrs, [][]byte{
+ attrName,
+ attrValue,
+ []byte(html.EscapeString(string(attrValue))),
+ })
+ if !moreAttr {
+ break
+ }
+ }
+ }
+ if bytes.Equal(tag, []byte("link")) {
+ sanitizeLinkTag(rc, out, attrs)
+ break
+ }
+
+ if bytes.Equal(tag, []byte("meta")) {
+ sanitizeMetaTag(rc, out, attrs)
+ break
+ }
+
+ fmt.Fprintf(out, "<%s", tag)
+
+ if hasAttrs {
+ sanitizeAttrs(rc, out, attrs)
+ }
+
+ if token == html.SelfClosingTagToken {
+ fmt.Fprintf(out, " />")
+ } else {
+ fmt.Fprintf(out, ">")
+ if bytes.Equal(tag, []byte("style")) {
+ state = STATE_IN_STYLE
+ }
+ }
+
+ if bytes.Equal(tag, []byte("head")) {
+ fmt.Fprintf(out, htmlHeadContentType)
+ }
+
+ if bytes.Equal(tag, []byte("form")) {
+ var formURL *url.URL
+ for _, attr := range attrs {
+ if bytes.Equal(attr[0], []byte("action")) {
+ formURL, _ = url.Parse(string(attr[1]))
+ formURL = mergeURIs(rc.BaseURL, formURL)
+ break
+ }
+ }
+ if formURL == nil {
+ formURL = rc.BaseURL
+ }
+ urlStr := formURL.String()
+ var key string
+ if rc.Key != nil {
+ key = hash(urlStr, rc.Key)
+ }
+ err := htmlFormExtension.Execute(out, HTMLFormExtParam{urlStr, key, config.Config.UrlParameter, config.Config.HashParameter})
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ writeEndTag := true
+ switch string(tag) {
+ case "body":
+ p := HTMLBodyExtParam{rc.BaseURL.String(), false, config.Config.UrlParameter}
+ if len(rc.Key) > 0 {
+ p.HasYukariKey = true
+ }
+ err := htmlBodyExtension.Execute(out, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject body extension", err)
+ }
+ }
+ rc.BodyInjected = true
+ case "style":
+ state = STATE_DEFAULT
+ case "noscript":
+ state = STATE_DEFAULT
+ writeEndTag = false
+ }
+ // skip noscript tags - only the tag, not the content, because javascript is sanitized
+ if writeEndTag {
+ fmt.Fprintf(out, "</%s>", tag)
+ }
+
+ case html.TextToken:
+ switch state {
+ case STATE_DEFAULT:
+ fmt.Fprintf(out, "%s", decoder.Raw())
+ case STATE_IN_STYLE:
+ sanitizeCSS(rc, out, decoder.Raw())
+ case STATE_IN_NOSCRIPT:
+ sanitizeHTML(rc, out, decoder.Raw())
+ }
+
+ case html.CommentToken:
+ // ignore comment. TODO : parse IE conditional comment
+
+ case html.DoctypeToken:
+ out.Write(decoder.Raw())
+ }
+ } else {
+ switch token {
+ case html.StartTagToken, html.SelfClosingTagToken:
+ tag, _ := decoder.TagName()
+ if inArray(tag, UNSAFE_ELEMENTS) {
+ unsafeElements = append(unsafeElements, tag)
+ }
+
+ case html.EndTagToken:
+ tag, _ := decoder.TagName()
+ if bytes.Equal(unsafeElements[len(unsafeElements)-1], tag) {
+ unsafeElements = unsafeElements[:len(unsafeElements)-1]
+ }
+ }
+ }
+ }
+}
+
+func sanitizeLinkTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ exclude := false
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("rel")) {
+ if !inArray(attrValue, LINK_REL_SAFE_VALUES) {
+ exclude = true
+ break
+ }
+ }
+ if bytes.Equal(attrName, []byte("as")) {
+ if bytes.Equal(attrValue, []byte("script")) {
+ exclude = true
+ break
+ }
+ }
+ }
+
+ if !exclude {
+ out.Write([]byte("<link"))
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+ out.Write([]byte(">"))
+ }
+}
+
+func sanitizeMetaTag(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ var http_equiv []byte
+ var content []byte
+
+ for _, attr := range attrs {
+ attrName := attr[0]
+ attrValue := attr[1]
+ if bytes.Equal(attrName, []byte("http-equiv")) {
+ http_equiv = bytes.ToLower(attrValue)
+ // exclude some <meta http-equiv="..." ..>
+ if !inArray(http_equiv, LINK_HTTP_EQUIV_SAFE_VALUES) {
+ return
+ }
+ }
+ if bytes.Equal(attrName, []byte("content")) {
+ content = attrValue
+ }
+ if bytes.Equal(attrName, []byte("charset")) {
+ // exclude <meta charset="...">
+ return
+ }
+ }
+
+ out.Write([]byte("<meta"))
+ urlIndex := bytes.Index(bytes.ToLower(content), []byte("url="))
+ if bytes.Equal(http_equiv, []byte("refresh")) && urlIndex != -1 {
+ contentUrl := content[urlIndex+4:]
+ // special case of <meta http-equiv="refresh" content="0; url='example.com/url.with.quote.outside'">
+ if len(contentUrl) >= 2 && (contentUrl[0] == byte('\'') || contentUrl[0] == byte('"')) {
+ if contentUrl[0] == contentUrl[len(contentUrl)-1] {
+ contentUrl = contentUrl[1 : len(contentUrl)-1]
+ }
+ }
+ // output proxify result
+ if uri, err := rc.ProxifyURI(contentUrl); err == nil {
+ fmt.Fprintf(out, ` http-equiv="refresh" content="%surl=%s"`, content[:urlIndex], uri)
+ }
+ } else {
+ if len(http_equiv) > 0 {
+ fmt.Fprintf(out, ` http-equiv="%s"`, http_equiv)
+ }
+ sanitizeAttrs(rc, out, attrs)
+ }
+ out.Write([]byte(">"))
+}
+
+func sanitizeAttrs(rc *RequestConfig, out io.Writer, attrs [][][]byte) {
+ for _, attr := range attrs {
+ sanitizeAttr(rc, out, attr[0], attr[1], attr[2])
+ }
+}
+
+func sanitizeAttr(rc *RequestConfig, out io.Writer, attrName, attrValue, escapedAttrValue []byte) {
+ if inArray(attrName, SAFE_ATTRIBUTES) {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, escapedAttrValue)
+ return
+ }
+ switch string(attrName) {
+ case "src", "href", "action":
+ if uri, err := rc.ProxifyURI(attrValue); err == nil {
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, uri)
+ } else if config.Config.Debug {
+ log.Println("cannot proxify uri:", string(attrValue))
+ }
+ case "style":
+ cssAttr := bytes.NewBuffer(nil)
+ sanitizeCSS(rc, cssAttr, attrValue)
+ fmt.Fprintf(out, " %s=\"%s\"", attrName, html.EscapeString(string(cssAttr.Bytes())))
+ }
+}
+
+func mergeURIs(u1, u2 *url.URL) *url.URL {
+ if u2 == nil {
+ return u1
+ }
+ return u1.ResolveReference(u2)
+}
+
+// Sanitized URI : removes all runes bellow 32 (included) as the begining and end of URI, and lower case the scheme.
+// avoid memory allocation (except for the scheme)
+func sanitizeURI(uri []byte) ([]byte, string) {
+ first_rune_index := 0
+ first_rune_seen := false
+ scheme_last_index := -1
+ buffer := bytes.NewBuffer(make([]byte, 0, 10))
+
+ // remove trailing space and special characters
+ uri = bytes.TrimRight(uri, "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0A\x0B\x0C\x0D\x0E\x0F\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1A\x1B\x1C\x1D\x1E\x1F\x20")
+
+ // loop over byte by byte
+ for i, c := range uri {
+ // ignore special characters and space (c <= 32)
+ if c > 32 {
+ // append to the lower case of the rune to buffer
+ if c < utf8.RuneSelf && 'A' <= c && c <= 'Z' {
+ c = c + 'a' - 'A'
+ }
+
+ buffer.WriteByte(c)
+
+ // update the first rune index that is not a special rune
+ if !first_rune_seen {
+ first_rune_index = i
+ first_rune_seen = true
+ }
+
+ if c == ':' {
+ // colon rune found, we have found the scheme
+ scheme_last_index = i
+ break
+ } else if c == '/' || c == '?' || c == '\\' || c == '#' {
+ // special case : most probably a relative URI
+ break
+ }
+ }
+ }
+
+ if scheme_last_index != -1 {
+ // scheme found
+ // copy the "lower case without special runes scheme" before the ":" rune
+ scheme_start_index := scheme_last_index - buffer.Len() + 1
+ copy(uri[scheme_start_index:], buffer.Bytes())
+ // and return the result
+ return uri[scheme_start_index:], buffer.String()
+ } else {
+ // scheme NOT found
+ return uri[first_rune_index:], ""
+ }
+}
+
+func (rc *RequestConfig) ProxifyURI(uri []byte) (string, error) {
+ // sanitize URI
+ uri, scheme := sanitizeURI(uri)
+
+ // remove javascript protocol
+ if scheme == "javascript:" {
+ return "", nil
+ }
+
+ // TODO check malicious data: - e.g. data:script
+ if scheme == "data:" {
+ if bytes.HasPrefix(uri, []byte("data:image/png")) ||
+ bytes.HasPrefix(uri, []byte("data:image/jpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/pjpeg")) ||
+ bytes.HasPrefix(uri, []byte("data:image/gif")) ||
+ bytes.HasPrefix(uri, []byte("data:image/webp")) {
+ // should be safe
+ return string(uri), nil
+ } else {
+ // unsafe data
+ return "", nil
+ }
+ }
+
+ // parse the uri
+ u, err := url.Parse(string(uri))
+ if err != nil {
+ return "", err
+ }
+
+ // get the fragment (with the prefix "#")
+ fragment := ""
+ if len(u.Fragment) > 0 {
+ fragment = "#" + u.Fragment
+ }
+
+ // reset the fragment: it is not included in the yukariurl
+ u.Fragment = ""
+
+ // merge the URI with the document URI
+ u = mergeURIs(rc.BaseURL, u)
+
+ // simple internal link ?
+ // some web pages describe the whole link https://same:auth@same.host/same.path?same.query#new.fragment
+ if u.Scheme == rc.BaseURL.Scheme &&
+ (rc.BaseURL.User == nil || (u.User != nil && u.User.String() == rc.BaseURL.User.String())) &&
+ u.Host == rc.BaseURL.Host &&
+ u.Path == rc.BaseURL.Path &&
+ u.RawQuery == rc.BaseURL.RawQuery {
+ // the fragment is the only difference between the document URI and the uri parameter
+ return fragment, nil
+ }
+
+ // return full URI and fragment (if not empty)
+ yukari_uri := u.String()
+
+ if rc.Key == nil {
+ return fmt.Sprintf("./?%s=%s%s", config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+ }
+ return fmt.Sprintf("./?%s=%s&%s=%s%s", config.Config.HashParameter, hash(yukari_uri, rc.Key), config.Config.UrlParameter, url.QueryEscape(yukari_uri), fragment), nil
+}
+
+func inArray(b []byte, a [][]byte) bool {
+ for _, b2 := range a {
+ if bytes.Equal(b, b2) {
+ return true
+ }
+ }
+ return false
+}
+
+func hash(msg string, key []byte) string {
+ mac := hmac.New(sha256.New, key)
+ mac.Write([]byte(msg))
+ return hex.EncodeToString(mac.Sum(nil))
+}
+
+func verifyRequestURI(uri, hashMsg, key []byte) bool {
+ h := make([]byte, hex.DecodedLen(len(hashMsg)))
+ _, err := hex.Decode(h, hashMsg)
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("hmac error:", err)
+ }
+ return false
+ }
+ mac := hmac.New(sha256.New, key)
+ mac.Write(uri)
+ return hmac.Equal(h, mac.Sum(nil))
+}
+
+func (p *Proxy) serveExitYukariPage(ctx *fasthttp.RequestCtx, uri *url.URL) {
+ ctx.SetContentType("text/html")
+ ctx.SetStatusCode(403)
+ ctx.Write([]byte(htmlPageStart))
+ ctx.Write([]byte("<h2>You are about to exit Yukari no Sukima</h2>"))
+ ctx.Write([]byte("<p>Following</p><p><a href=\""))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("\" rel=\"noreferrer\">"))
+ ctx.Write([]byte(html.EscapeString(uri.String())))
+ ctx.Write([]byte("</a></p><p>the content of this URL will be <b>NOT</b> sanitized.</p>"))
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func (p *Proxy) serveMainPage(ctx *fasthttp.RequestCtx, statusCode int, err error) {
+ ctx.SetContentType("text/html; charset=UTF-8")
+ ctx.SetStatusCode(statusCode)
+ ctx.Write([]byte(htmlPageStart))
+ if err != nil {
+ if config.Config.Debug {
+ log.Println("error:", err)
+ }
+ ctx.Write([]byte("<h2>Error: "))
+ ctx.Write([]byte(html.EscapeString(err.Error())))
+ ctx.Write([]byte("</h2>"))
+ }
+ if p.Key == nil {
+ p := HTMLMainPageFormParam{config.Config.UrlParameter}
+ err := htmlMainPageForm.Execute(ctx, p)
+ if err != nil {
+ if config.Config.Debug {
+ fmt.Println("failed to inject main page form", err)
+ }
+ }
+ } else {
+ ctx.Write([]byte(`<h3>Warning! This instance does not support direct URL opening.</h3>`))
+ }
+ ctx.Write([]byte(htmlPageStop))
+}
+
+func main() {
+ config.Config.ListenAddress = "127.0.0.1:3000"
+ config.Config.Key = ""
+ config.Config.IPV6 = true
+ config.Config.Debug = false
+ config.Config.RequestTimeout = 5
+ config.Config.FollowRedirect = false
+ config.Config.UrlParameter = "yukariurl"
+ config.Config.HashParameter = "yukarihash"
+ config.Config.MaxConnsPerHost = 5
+ config.Config.ProxyEnv = false
+
+ var configFile string
+ var proxy string
+ var socks5 string
+ var version bool
+
+ flag.StringVar(&configFile, "f", "", "Configuration file")
+ flag.StringVar(&proxy, "proxy", "", "Use the specified HTTP proxy (ie: '[user:pass@]hostname:port'). Overrides: -socks5, IPv6")
+ flag.StringVar(&socks5, "socks5", "", "Use a SOCKS5 proxy (ie: 'hostname:port'). Overrides: IPv6.")
+ flag.BoolVar(&version, "version", false, "Show version")
+ flag.Parse()
+
+ if configFile != "" {
+ config.ReadConfig(configFile)
+ }
+
+ if version {
+ yukari.FullVersion()
+ return
+ }
+
+ if config.Config.ProxyEnv && os.Getenv("HTTP_PROXY") == "" && os.Getenv("HTTPS_PROXY") == "" {
+ log.Fatal("Error -proxyenv is used but no environment variables named 'HTTP_PROXY' and/or 'HTTPS_PROXY' could be found.")
+ os.Exit(1)
+ }
+
+ if config.Config.ProxyEnv {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpProxyHTTPDialer()
+ log.Println("Using environment defined proxy(ies).")
+ } else if proxy != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpHTTPDialer(proxy)
+ log.Println("Using custom HTTP proxy.")
+ } else if socks5 != "" {
+ config.Config.IPV6 = false
+ Gap.Dial = fasthttpproxy.FasthttpSocksDialer(socks5)
+ log.Println("Using Socks5 proxy.")
+ } else if config.Config.IPV6 {
+ Gap.Dial = fasthttp.DialDualStack
+ log.Println("Using dual stack (IPv4/IPv6) direct connections.")
+ } else {
+ Gap.Dial = fasthttp.Dial
+ log.Println("Using IPv4 only direct connections.")
+ }
+
+ p := &Proxy{RequestTimeout: time.Duration(config.Config.RequestTimeout) * time.Second,
+ FollowRedirect: config.Config.FollowRedirect}
+
+ if config.Config.Key != "" {
+ var err error
+ p.Key, err = base64.StdEncoding.DecodeString(config.Config.Key)
+ if err != nil {
+ log.Fatal("Error parsing -key", err.Error())
+ os.Exit(1)
+ }
+ }
+ log.Println("ゆかり様、お願いします…！")
+ log.Println("Listening on", config.Config.ListenAddress)
+
+ if err := fasthttp.ListenAndServe(config.Config.ListenAddress, p.RequestHandler); err != nil {
+ log.Fatal("Error in ListenAndServe:", err)
+ }
+}
--- /dev/null
+package main
+
+import (
+ "bytes"
+ "net/url"
+ "testing"
+)
+
+type AttrTestCase struct {
+ AttrName []byte
+ AttrValue []byte
+ ExpectedOutput []byte
+}
+
+type SanitizeURITestCase struct {
+ Input []byte
+ ExpectedOutput []byte
+ ExpectedScheme string
+}
+
+type StringTestCase struct {
+ Input string
+ ExpectedOutput string
+}
+
+var attrTestData []*AttrTestCase = []*AttrTestCase{
+ &AttrTestCase{
+ []byte("href"),
+ []byte("./x"),
+ []byte(` href="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fx"`),
+ },
+ &AttrTestCase{
+ []byte("src"),
+ []byte("http://x.com/y"),
+ []byte(` src="./?yukariurl=http%3A%2F%2Fx.com%2Fy"`),
+ },
+ &AttrTestCase{
+ []byte("action"),
+ []byte("/z"),
+ []byte(` action="./?yukariurl=http%3A%2F%2F127.0.0.1%2Fz"`),
+ },
+ &AttrTestCase{
+ []byte("onclick"),
+ []byte("console.log(document.cookies)"),
+ nil,
+ },
+}
+
+var sanitizeUriTestData []*SanitizeURITestCase = []*SanitizeURITestCase{
+ &SanitizeURITestCase{
+ []byte("http://example.com/"),
+ []byte("http://example.com/"),
+ "http:",
+ },
+ &SanitizeURITestCase{
+ []byte("HtTPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte(" Ht TPs://example.com/ \t"),
+ []byte("https://example.com/"),
+ "https:",
+ },
+ &SanitizeURITestCase{
+ []byte("javascript:void(0)"),
+ []byte("javascript:void(0)"),
+ "javascript:",
+ },
+ &SanitizeURITestCase{
+ []byte(" /path/to/a/file/without/protocol "),
+ []byte("/path/to/a/file/without/protocol"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" #fragment "),
+ []byte("#fragment"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(" qwertyuiop "),
+ []byte("qwertyuiop"),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(""),
+ []byte(""),
+ "",
+ },
+ &SanitizeURITestCase{
+ []byte(":"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte(" :"),
+ []byte(":"),
+ ":",
+ },
+ &SanitizeURITestCase{
+ []byte("schéma:"),
+ []byte("schéma:"),
+ "schéma:",
+ },
+}
+
+var urlTestData []*StringTestCase = []*StringTestCase{
+ &StringTestCase{
+ "http://x.com/",
+ "./?yukariurl=http%3A%2F%2Fx.com%2F",
+ },
+ &StringTestCase{
+ "http://a@x.com/",
+ "./?yukariurl=http%3A%2F%2Fa%40x.com%2F",
+ },
+ &StringTestCase{
+ "#a",
+ "#a",
+ },
+}
+
+func TestAttrSanitizer(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range attrTestData {
+ out := bytes.NewBuffer(nil)
+ sanitizeAttr(rc, out, testCase.AttrName, testCase.AttrValue, testCase.AttrValue)
+ res, _ := out.ReadBytes(byte(0))
+ if !bytes.Equal(res, testCase.ExpectedOutput) {
+ t.Errorf(
+ `Attribute parse error. Name: "%s", Value: "%s", Expected: %s, Got: "%s"`,
+ testCase.AttrName,
+ testCase.AttrValue,
+ testCase.ExpectedOutput,
+ res,
+ )
+ }
+ }
+}
+
+func TestSanitizeURI(t *testing.T) {
+ for _, testCase := range sanitizeUriTestData {
+ newUrl, scheme := sanitizeURI(testCase.Input)
+ if !bytes.Equal(newUrl, testCase.ExpectedOutput) {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ if scheme != testCase.ExpectedScheme {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedScheme,
+ scheme,
+ )
+ }
+ }
+}
+
+func TestURLProxifier(t *testing.T) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ for _, testCase := range urlTestData {
+ newUrl, err := rc.ProxifyURI([]byte(testCase.Input))
+ if err != nil {
+ t.Errorf("Failed to parse URL: %s", testCase.Input)
+ }
+ if newUrl != testCase.ExpectedOutput {
+ t.Errorf(
+ `URL proxifier error. Expected: "%s", Got: "%s"`,
+ testCase.ExpectedOutput,
+ newUrl,
+ )
+ }
+ }
+}
+
+var BENCH_SIMPLE_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <title>test</title>
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeSimpleHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_SIMPLE_HTML)
+ }
+}
+
+var BENCH_COMPLEX_HTML []byte = []byte(`<!doctype html>
+<html>
+ <head>
+ <noscript><meta http-equiv="refresh" content="0; URL=./xy"></noscript>
+ <title>test 2</title>
+ <script> alert('xy'); </script>
+ <link rel="stylesheet" href="./core.bundle.css">
+ <style>
+ html { background: url(./a.jpg); }
+ </style
+ </head>
+ <body>
+ <h1>Test heading</h1>
+ <img src="b.png" alt="imgtitle" />
+ <form action="/z">
+ <input type="submit" style="background: url(http://aa.bb/cc)" >
+ </form>
+ </body>
+</html>`)
+
+func BenchmarkSanitizeComplexHTML(b *testing.B) {
+ u, _ := url.Parse("http://127.0.0.1/")
+ rc := &RequestConfig{BaseURL: u}
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ out := bytes.NewBuffer(nil)
+ sanitizeHTML(rc, out, BENCH_COMPLEX_HTML)
+ }
+}
--- /dev/null
+package main
+
+var SAFE_ATTRIBUTES [][]byte = [][]byte{
+ []byte("abbr"),
+ []byte("accesskey"),
+ []byte("align"),
+ []byte("alt"),
+ []byte("as"),
+ []byte("autocomplete"),
+ []byte("charset"),
+ []byte("checked"),
+ []byte("class"),
+ []byte("content"),
+ []byte("contenteditable"),
+ []byte("contextmenu"),
+ []byte("dir"),
+ []byte("for"),
+ []byte("height"),
+ []byte("hidden"),
+ []byte("hreflang"),
+ []byte("id"),
+ []byte("lang"),
+ []byte("media"),
+ []byte("method"),
+ []byte("name"),
+ []byte("nowrap"),
+ []byte("placeholder"),
+ []byte("property"),
+ []byte("rel"),
+ []byte("spellcheck"),
+ []byte("tabindex"),
+ []byte("target"),
+ []byte("title"),
+ []byte("translate"),
+ []byte("type"),
+ []byte("value"),
+ []byte("width"),
+}
--- /dev/null
+package main
+
+var LINK_HTTP_EQUIV_SAFE_VALUES [][]byte = [][]byte{
+ // X-UA-Compatible will be added automaticaly, so it can be skipped
+ []byte("date"),
+ []byte("last-modified"),
+ []byte("refresh"), // URL rewrite
+ []byte("content-language"),
+}
+
+var LINK_REL_SAFE_VALUES [][]byte = [][]byte{
+ []byte("alternate"),
+ []byte("archives"),
+ []byte("author"),
+ []byte("copyright"),
+ []byte("first"),
+ []byte("help"),
+ []byte("icon"),
+ []byte("index"),
+ []byte("last"),
+ []byte("license"),
+ []byte("manifest"),
+ []byte("next"),
+ []byte("pingback"),
+ []byte("prev"),
+ []byte("publisher"),
+ []byte("search"),
+ []byte("shortcut icon"),
+ []byte("stylesheet"),
+ []byte("up"),
+}
--- /dev/null
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+<meta http-equiv="X-UA-Compatible" content="IE=edge">
+<meta name="referrer" content="no-referrer">
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
+<html>
+ <head>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=1">
+ <style>
+ html {
+ height: 100%;
+ }
+ body {
+ min-height: 100%;
+ display: flex;
+ flex-direction: column;
+ font-family: sans-serif;
+ text-align: center;
+ color: #BC48FC;
+ background: #240039;
+ margin: 0;
+ padding: 0;
+ font-size: 1.1em;
+ }
+ input {
+ border: 1px solid #888;
+ padding: 0.3em;
+ color: #BC48FC;
+ background: #202020;
+ font-size: 1.1.em;
+ }
+ input[placeholder] {
+ width: 80%;
+ }
+ a {
+ text-decoration: none;
+ color: #9529B9;
+ }
+ h1, h2 {
+ font-weight: 200;
+ margin-bottom: 2rem;
+ }
+ h1 {
+ font-size: 3em;
+ }
+ .container {
+ flex: 1;
+ min-height: 100%;
+ margin-bottom: 1em;
+ }
+ .footer {
+ margin: 1em;
+ }
+ .footer p {
+ font-size: 0.8em;
+ }
+ </style>
+ <title>Yukari's Gap</title>
+ </head>
+ <body>
+ <div class="container">
+ <h1>Yukari's Gap</h1>
+
--- /dev/null
+</div>
+<div class="footer">
+ <p>
+ Yukari's Gap rewrites web pages to exclude malicious HTML tags and CSS/HTML attributes. <br>
+ It also replaces external resource references to prevent third-party information leaks. <br>
+ </p>
+</div>
+</body>
+</html>
--- /dev/null
+package main
+
+var UNSAFE_ELEMENTS [][]byte = [][]byte{
+ []byte("applet"),
+ []byte("canvas"),
+ []byte("embed"),
+ []byte("math"),
+ []byte("script"),
+ []byte("svg"),
+}
--- /dev/null
+package config
+
+import (
+ "gopkg.in/ini.v1"
+)
+
+var Config struct {
+ Debug bool
+ ListenAddress string
+ Key string
+ IPV6 bool
+ RequestTimeout uint
+ FollowRedirect bool
+ MaxConnsPerHost uint
+ UrlParameter string
+ HashParameter string
+ ProxyEnv bool
+}
+
+func ReadConfig(file string) error {
+ cfg, err := ini.Load(file)
+ if err != nil {
+ return err
+ }
+ Config.Debug, _ = cfg.Section("yukari").Key("debug").Bool()
+ Config.ListenAddress = cfg.Section("yukari").Key("listen").String()
+ Config.Key = cfg.Section("yukari").Key("key").String()
+ Config.IPV6, _ = cfg.Section("yukari").Key("ipv6").Bool()
+ Config.RequestTimeout, _ = cfg.Section("yukari").Key("timeout").Uint()
+ Config.FollowRedirect, _ = cfg.Section("yukari").Key("followredirect").Bool()
+ Config.MaxConnsPerHost, _ = cfg.Section("yukari").Key("max_conns_per_host").Uint()
+ Config.UrlParameter = cfg.Section("yukari").Key("urlparam").String()
+ Config.HashParameter = cfg.Section("yukari").Key("hashparam").String()
+ Config.ProxyEnv, _ = cfg.Section("yukari").Key("proxyenv").Bool()
+ return nil
+}
--- /dev/null
+package contenttype
+
+import (
+ "mime"
+ "strings"
+)
+
+type ContentType struct {
+ TopLevelType string
+ SubType string
+ Suffix string
+ Parameters map[string]string
+}
+
+func (contenttype *ContentType) String() string {
+ var mimetype string
+ if contenttype.Suffix == "" {
+ if contenttype.SubType == "" {
+ mimetype = contenttype.TopLevelType
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType
+ }
+ } else {
+ mimetype = contenttype.TopLevelType + "/" + contenttype.SubType + "+" + contenttype.Suffix
+ }
+ return mime.FormatMediaType(mimetype, contenttype.Parameters)
+}
+
+func (contenttype *ContentType) Equals(other ContentType) bool {
+ if contenttype.TopLevelType != other.TopLevelType ||
+ contenttype.SubType != other.SubType ||
+ contenttype.Suffix != other.Suffix ||
+ len(contenttype.Parameters) != len(other.Parameters) {
+ return false
+ }
+ for k, v := range contenttype.Parameters {
+ if other.Parameters[k] != v {
+ return false
+ }
+ }
+ return true
+}
+
+func (contenttype *ContentType) FilterParameters(parameters map[string]bool) {
+ for k, _ := range contenttype.Parameters {
+ if !parameters[k] {
+ delete(contenttype.Parameters, k)
+ }
+ }
+}
+
+func ParseContentType(contenttype string) (ContentType, error) {
+ mimetype, params, err := mime.ParseMediaType(contenttype)
+ if err != nil {
+ return ContentType{"", "", "", params}, err
+ }
+ splitted_mimetype := strings.SplitN(strings.ToLower(mimetype), "/", 2)
+ if len(splitted_mimetype) <= 1 {
+ return ContentType{splitted_mimetype[0], "", "", params}, nil
+ } else {
+ splitted_subtype := strings.SplitN(splitted_mimetype[1], "+", 2)
+ if len(splitted_subtype) == 1 {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], "", params}, nil
+ } else {
+ return ContentType{splitted_mimetype[0], splitted_subtype[0], splitted_subtype[1], params}, nil
+ }
+ }
+
+}
+
+type Filter func(contenttype ContentType) bool
+
+func NewFilterContains(partialMimeType string) Filter {
+ return func(contenttype ContentType) bool {
+ return strings.Contains(contenttype.TopLevelType, partialMimeType) ||
+ strings.Contains(contenttype.SubType, partialMimeType) ||
+ strings.Contains(contenttype.Suffix, partialMimeType)
+ }
+}
+
+func NewFilterEquals(TopLevelType, SubType, Suffix string) Filter {
+ return func(contenttype ContentType) bool {
+ return ((TopLevelType != "*" && TopLevelType == contenttype.TopLevelType) || (TopLevelType == "*")) &&
+ ((SubType != "*" && SubType == contenttype.SubType) || (SubType == "*")) &&
+ ((Suffix != "*" && Suffix == contenttype.Suffix) || (Suffix == "*"))
+ }
+}
+
+func NewFilterOr(contentTypeFilterList []Filter) Filter {
+ return func(contenttype ContentType) bool {
+ for _, contentTypeFilter := range contentTypeFilterList {
+ if contentTypeFilter(contenttype) {
+ return true
+ }
+ }
+ return false
+ }
+}
--- /dev/null
+package contenttype
+
+import (
+ "bytes"
+ "fmt"
+ "testing"
+)
+
+type ParseContentTypeTestCase struct {
+ Input string
+ ExpectedOutput *ContentType /* or nil if an error is expected */
+ ExpectedString *string /* or nil if equals to Input */
+}
+
+var parseContentTypeTestCases []ParseContentTypeTestCase = []ParseContentTypeTestCase{
+ ParseContentTypeTestCase{
+ "text/html",
+ &ContentType{"text", "html", "", map[string]string{}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/svg+xml; charset=UTF-8",
+ &ContentType{"text", "svg", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/",
+ nil,
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text; charset=UTF-8",
+ &ContentType{"text", "", "", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+ ParseContentTypeTestCase{
+ "text/+xml; charset=UTF-8",
+ &ContentType{"text", "", "xml", map[string]string{"charset": "UTF-8"}},
+ nil,
+ },
+}
+
+type ContentTypeEqualsTestCase struct {
+ A, B ContentType
+ Equals bool
+}
+
+var Map_Empty map[string]string = map[string]string{}
+var Map_A map[string]string = map[string]string{"a": "value_a"}
+var Map_B map[string]string = map[string]string{"b": "value_b"}
+var Map_AB map[string]string = map[string]string{"a": "value_a", "b": "value_b"}
+
+var ContentType_E ContentType = ContentType{"a", "b", "c", Map_Empty}
+var ContentType_A ContentType = ContentType{"a", "b", "c", Map_A}
+var ContentType_B ContentType = ContentType{"a", "b", "c", Map_B}
+var ContentType_AB ContentType = ContentType{"a", "b", "c", Map_AB}
+
+var contentTypeEqualsTestCases []ContentTypeEqualsTestCase = []ContentTypeEqualsTestCase{
+ // TopLevelType, SubType, Suffix
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "c", Map_Empty}, true},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"o", "b", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "o", "c", Map_Empty}, false},
+ ContentTypeEqualsTestCase{ContentType_E, ContentType{"a", "b", "o", Map_Empty}, false},
+ // Parameters
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_A, true},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_B, true},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_AB, true},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_B, false},
+ ContentTypeEqualsTestCase{ContentType_B, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_A, false},
+ ContentTypeEqualsTestCase{ContentType_AB, ContentType_E, false},
+ ContentTypeEqualsTestCase{ContentType_A, ContentType_AB, false},
+}
+
+type FilterTestCase struct {
+ Description string
+ Input Filter
+ TrueValues []ContentType
+ FalseValues []ContentType
+}
+
+var filterTestCases []FilterTestCase = []FilterTestCase{
+ FilterTestCase{
+ "contains xml",
+ NewFilterContains("xml"),
+ []ContentType{
+ ContentType{"xml", "", "", Map_Empty},
+ ContentType{"text", "xml", "", Map_Empty},
+ ContentType{"text", "html", "xml", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "svg", "", map[string]string{"script": "javascript"}},
+ ContentType{"java", "script", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/xhtml",
+ NewFilterEquals("application", "xhtml", "*"),
+ []ContentType{
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_Empty},
+ ContentType{"application", "xhtml", "zip", Map_AB},
+ },
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "xhtml", "", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals application/*",
+ NewFilterEquals("application", "*", ""),
+ []ContentType{
+ ContentType{"application", "xhtml", "", Map_Empty},
+ ContentType{"application", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "xhtml", "", Map_Empty},
+ ContentType{"text", "xhtml", "xml", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications */javascript",
+ NewFilterEquals("*", "javascript", ""),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"text", "javascript", "zip", Map_Empty},
+ },
+ },
+ FilterTestCase{
+ "equals applications/* or */javascript",
+ NewFilterOr([]Filter{
+ NewFilterEquals("application", "*", ""),
+ NewFilterEquals("*", "javascript", ""),
+ }),
+ []ContentType{
+ ContentType{"application", "javascript", "", Map_Empty},
+ ContentType{"text", "javascript", "", Map_Empty},
+ ContentType{"application", "xhtml", "", Map_Empty},
+ },
+ []ContentType{
+ ContentType{"text", "html", "", Map_Empty},
+ ContentType{"application", "xhtml", "xml", Map_Empty},
+ },
+ },
+}
+
+type FilterParametersTestCase struct {
+ Input map[string]string
+ Filter map[string]bool
+ Output map[string]string
+}
+
+var filterParametersTestCases []FilterParametersTestCase = []FilterParametersTestCase{
+ FilterParametersTestCase{
+ map[string]string{},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{},
+ map[string]string{},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true},
+ map[string]string{"A": "value_A"},
+ },
+ FilterParametersTestCase{
+ map[string]string{"A": "value_A", "B": "value_B"},
+ map[string]bool{"A": true, "B": true},
+ map[string]string{"A": "value_A", "B": "value_B"},
+ },
+}
+
+func TestContentTypeEquals(t *testing.T) {
+ for _, testCase := range contentTypeEqualsTestCases {
+ if !testCase.A.Equals(testCase.B) && testCase.Equals {
+ t.Errorf(`Must be equals "%s"="%s"`, testCase.A, testCase.B)
+ } else if testCase.A.Equals(testCase.B) && !testCase.Equals {
+ t.Errorf(`Mustn't be equals "%s"!="%s"`, testCase.A, testCase.B)
+ }
+ }
+}
+
+func TestParseContentType(t *testing.T) {
+ for _, testCase := range parseContentTypeTestCases {
+ // test ParseContentType
+ contentType, err := ParseContentType(testCase.Input)
+ if testCase.ExpectedOutput == nil {
+ // error expected
+ if err == nil {
+ // but there is no error
+ t.Errorf(`Expecting error for "%s"`, testCase.Input)
+ }
+ } else {
+ // no expected error
+ if err != nil {
+ t.Errorf(`Unexpecting error for "%s" : %s`, testCase.Input, err)
+ } else if !contentType.Equals(*testCase.ExpectedOutput) {
+ // the parsed contentType doesn't matched
+ t.Errorf(`Unexpecting result for "%s", instead got "%s"`, testCase.ExpectedOutput.String(), contentType.String())
+ } else {
+ // ParseContentType is fine, checking String()
+ contentTypeString := contentType.String()
+ expectedString := testCase.Input
+ if testCase.ExpectedString != nil {
+ expectedString = *testCase.ExpectedString
+ }
+ if contentTypeString != expectedString {
+ t.Errorf(`Error with String() output of "%s", got "%s", ContentType{"%s", "%s", "%s", "%s"}`, expectedString, contentTypeString, contentType.TopLevelType, contentType.SubType, contentType.Suffix, contentType.Parameters)
+ }
+ }
+ }
+ }
+}
+
+func FilterToString(m map[string]bool) string {
+ b := new(bytes.Buffer)
+ for key, value := range m {
+ if value {
+ fmt.Fprintf(b, "'%s'=true;", key)
+ } else {
+ fmt.Fprintf(b, "'%s'=false;", key)
+ }
+ }
+ return b.String()
+}
+
+func TestFilters(t *testing.T) {
+ for _, testCase := range filterTestCases {
+ for _, contentType := range testCase.TrueValues {
+ if !testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" must accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ for _, contentType := range testCase.FalseValues {
+ if testCase.Input(contentType) {
+ t.Errorf(`Filter "%s" mustn't accept the value "%s"`, testCase.Description, contentType)
+ }
+ }
+ }
+}
+
+func TestFilterParameters(t *testing.T) {
+ for _, testCase := range filterParametersTestCases {
+ // copy Input since the map will be modified
+ InputCopy := make(map[string]string)
+ for k, v := range testCase.Input {
+ InputCopy[k] = v
+ }
+ // apply filter
+ contentType := ContentType{"", "", "", InputCopy}
+ contentType.FilterParameters(testCase.Filter)
+ // test
+ contentTypeOutput := ContentType{"", "", "", testCase.Output}
+ if !contentTypeOutput.Equals(contentType) {
+ t.Errorf(`FilterParameters error : %s becomes %s with this filter %s`, testCase.Input, contentType.Parameters, FilterToString(testCase.Filter))
+ }
+ }
+}
--- /dev/null
+[yukari]
+debug=false
+listen="127.0.0.1:3000"
+key=""
+ipv6=true
+timeout=5
+followredirect=false
+max_conns_per_host=5
+urlparam="yukariurl"
+hashparam="yukarihash"
+proxyenv=false
--- /dev/null
+module marisa.chaotic.ninja/yukari
+
+go 1.16
+
+require (
+ github.com/stretchr/testify v1.9.0 // indirect
+ github.com/valyala/fasthttp v1.34.0
+ golang.org/x/net v0.7.0
+ golang.org/x/text v0.7.0
+ gopkg.in/ini.v1 v1.67.0
+)
--- /dev/null
+github.com/andybalholm/brotli v1.0.4 h1:V7DdXeJtZscaqfNuAdSRuRFzuiKlHSC/Zh3zl9qY3JY=
+github.com/andybalholm/brotli v1.0.4/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/klauspost/compress v1.15.0 h1:xqfchp4whNFxn5A4XFyyYtitiWI8Hy5EW59jEwcyL6U=
+github.com/klauspost/compress v1.15.0/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
+github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
+github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
+github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
+github.com/valyala/fasthttp v1.34.0 h1:d3AAQJ2DRcxJYHm7OXNXtXt2as1vMDfxeIcFvhmGGm4=
+github.com/valyala/fasthttp v1.34.0/go.mod h1:epZA5N+7pY6ZaEKRmstzOuYJx9HI8DI1oaCGZpdH4h0=
+github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20220214200702-86341886e292/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
+golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
+gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
--- /dev/null
+#!/bin/sh
+# $TheSupernovaDuo$
+
+# PROVIDE: yukari
+# REQUIRE: DAEMON NETWORKING
+# KEYWORD: shutdown
+
+. /etc/rc.subr
+
+name="yukari"
+rcvar="yukari_enable"
+
+load_rc_config "${name}"
+
+: ${yukari_enable="NO"}
+: ${yukari_config=""}
+
+pidfile="/var/run/${name}.pid"
+command="/usr/sbin/daemon"
+procname="/usr/local/bin/${name}"
+command_args="-S -m 3 -s info -l daemon -p ${pidfile} /usr/bin/env ${procname} ${yukari_args}"
+
+run_rc_command "$1"
--- /dev/null
+cmd: /usr/local/bin/yukari
+user: www
--- /dev/null
+#!/bin/ksh
+# $TheSupernovaDuo
+daemon="/usr/local/bin/yukari"
+
+. /etc/rc.d/rc.subr
+
+rc_bg=YES
+rc_reload=NO
+
+rc_cmd "$1"
--- /dev/null
+package yukari
+
+import (
+ "fmt"
+)
+
+var (
+ // Version release version
+ Version = "0.0.1"
+
+ // Commit will be overwritten automatically by the build system
+ Commit = "HEAD"
+)
+
+// FullVersion display the full version and build
+func FullVersion() string {
+ return fmt.Sprintf("%s@%s", Version, Commit)
+}
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI 1
+.Os
+.Sh NAME
+.Nm yukari
+.Nd Privacy-aware Web Content Sanitizer Proxy As A Service (WCSPAAS)
+.Sh SYNOPSIS
+.Nm
+.Op Fl f Ar string
+.Op Fl proxy Ar string
+.Op Fl proxyenv Ar bool
+.Op Fl socks5 Ar string
+.Op Fl version
+.Sh DESCRIPTION
+Yukari's Gap rewrites web pages to exclude malicious HTML tags and attributes.
+It also replaces external resource references in order to prevent third-party
+information leaks.
+.Pp
+The main goal of Yukari's Gap is to provide a result proxy for SearX, but it
+can be used as a standalone sanitizer service, too.
+.Sh FEATURES
+.Bl -tag -width Ds
+.It HTML sanitization
+.It Rewrites HTML/CSS external references to locals
+.It JavaScript blocking
+.It No Cookies forwarded
+.It No Referrers
+.It No Caching/ETag
+.It Supports GET/POST forms and IFrames
+.It Optional HMAC URL verifier key to prevent service abuse
+.El
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It Fl f Ar path
+Load configuration file from path
+.It Fl proxy Ar string
+Use the specified HTTP proxy (ie: [user:pass@]hostname:port),
+this overrides the
+.Fl socks5
+option and the IPv6 setting
+.It Fl proxyenv Ar bool
+Use a HTTP proxy as set in the environment (such as
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+).
+Overrides the
+.Fl proxy ,
+.Fl socks5 ,
+flags and the IPv6 setting
+.It Fl socks5 Ar string
+Use a SOCKS5 proxy (ie: hostname:port), this
+overrides the IPv6 setting
+.El
+.Sh SEE ALSO
+.Xr SearX 1
+.Sh AUTHORS
+.An Adam Tauber Aq Mt asciimoo@gmail.com
+.An Alexandre Flament Aq Mt alex@al-f.net
+.Sh MAINTAINERS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
+.Sh BUGS
+Bugs or suggestions?
+Send an email to
+.Aq Mt yukari-dev@chaotic.ninja
+.Sh LICENSE
+This program is free software: you can redistribute it and/or modify it
+under the terms of the GNU Affero General Public License as published
+by the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+.Pp
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.
+See the GNU Affero General Public License for more details.
--- /dev/null
+.\" $TheSupernovaDuo$
+.Dd $Mdocdate$
+.Dt YUKARI.INI 5
+.Os
+.Sh NAME
+.Nm yukari.ini
+.Nd INI-style configuration file for
+.Xr yukari 1
+.Sh OPTIONS
+.Bl -tag -width Ds
+.It debug (bool)
+Enable/disable proxy and redirection logs (default true)
+.It listen (string)
+Listen address (default "127.0.0.1:3000")
+.It key (string)
+HMAC url validation key (base64 encoded) - leave blank to disable validation
+.It ipv6 (bool)
+Enable IPv6 support for queries
+(can be overrided by the proxy options, default true)
+.It timeout (uint)
+Request timeout (default 5)
+.It followredirect (bool)
+Follow HTTP GET redirect (default false)
+.It max_conns_per_host (uint)
+How much connections are allowed per Host/IP (default 4)
+.It urlparam (string)
+User-defined requesting string URL parameter name
+(ie: '/?url=...' or '/?u=...') (default "yukariurl")
+.It hashparam (string)
+User-defined requesting string HASH parameter name
+(ie: '/?hash=...' or '/?h=...') (default "yukarihash")
+.It proxyenv (bool)
+Use a HTTP proxy as set in the environment
+(
+.Ev HTTP_PROXY ,
+.Ev HTTPS_PROXY ,
+.Ev NO_PROXY
+) (overrides ipv6, default false)
+.El
+.Sh AUTHORS
+.An Izuru Yakumo Aq Mt yakumo.izuru@chaotic.ninja
projects / yukari.git / commitdiff