Added snappy-1.1.3 compression library
This commit is contained in:
parent
7c9fc12b63
commit
27319d4532
1
external/snappy-1.1.3/AUTHORS
vendored
Normal file
1
external/snappy-1.1.3/AUTHORS
vendored
Normal file
|
@ -0,0 +1 @@
|
|||
opensource@google.com
|
54
external/snappy-1.1.3/COPYING
vendored
Normal file
54
external/snappy-1.1.3/COPYING
vendored
Normal file
|
@ -0,0 +1,54 @@
|
|||
Copyright 2011, Google Inc.
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
* Redistributions in binary form must reproduce the above
|
||||
copyright notice, this list of conditions and the following disclaimer
|
||||
in the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
* Neither the name of Google Inc. nor the names of its
|
||||
contributors may be used to endorse or promote products derived from
|
||||
this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
===
|
||||
|
||||
Some of the benchmark data in testdata/ is licensed differently:
|
||||
|
||||
- fireworks.jpeg is Copyright 2013 Steinar H. Gunderson, and
|
||||
is licensed under the Creative Commons Attribution 3.0 license
|
||||
(CC-BY-3.0). See https://creativecommons.org/licenses/by/3.0/
|
||||
for more information.
|
||||
|
||||
- kppkn.gtb is taken from the Gaviota chess tablebase set, and
|
||||
is licensed under the MIT License. See
|
||||
https://sites.google.com/site/gaviotachessengine/Home/endgame-tablebases-1
|
||||
for more information.
|
||||
|
||||
- paper-100k.pdf is an excerpt (bytes 92160 to 194560) from the paper
|
||||
“Combinatorial Modeling of Chromatin Features Quantitatively Predicts DNA
|
||||
Replication Timing in _Drosophila_” by Federico Comoglio and Renato Paro,
|
||||
which is licensed under the CC-BY license. See
|
||||
http://www.ploscompbiol.org/static/license for more ifnormation.
|
||||
|
||||
- alice29.txt, asyoulik.txt, plrabn12.txt and lcet10.txt are from Project
|
||||
Gutenberg. The first three have expired copyrights and are in the public
|
||||
domain; the latter does not have expired copyright, but is still in the
|
||||
public domain according to the license information
|
||||
(http://www.gutenberg.org/ebooks/53).
|
370
external/snappy-1.1.3/INSTALL
vendored
Normal file
370
external/snappy-1.1.3/INSTALL
vendored
Normal file
|
@ -0,0 +1,370 @@
|
|||
Installation Instructions
|
||||
*************************
|
||||
|
||||
Copyright (C) 1994-1996, 1999-2002, 2004-2013 Free Software Foundation,
|
||||
Inc.
|
||||
|
||||
Copying and distribution of this file, with or without modification,
|
||||
are permitted in any medium without royalty provided the copyright
|
||||
notice and this notice are preserved. This file is offered as-is,
|
||||
without warranty of any kind.
|
||||
|
||||
Basic Installation
|
||||
==================
|
||||
|
||||
Briefly, the shell command `./configure && make && make install'
|
||||
should configure, build, and install this package. The following
|
||||
more-detailed instructions are generic; see the `README' file for
|
||||
instructions specific to this package. Some packages provide this
|
||||
`INSTALL' file but do not implement all of the features documented
|
||||
below. The lack of an optional feature in a given package is not
|
||||
necessarily a bug. More recommendations for GNU packages can be found
|
||||
in *note Makefile Conventions: (standards)Makefile Conventions.
|
||||
|
||||
The `configure' shell script attempts to guess correct values for
|
||||
various system-dependent variables used during compilation. It uses
|
||||
those values to create a `Makefile' in each directory of the package.
|
||||
It may also create one or more `.h' files containing system-dependent
|
||||
definitions. Finally, it creates a shell script `config.status' that
|
||||
you can run in the future to recreate the current configuration, and a
|
||||
file `config.log' containing compiler output (useful mainly for
|
||||
debugging `configure').
|
||||
|
||||
It can also use an optional file (typically called `config.cache'
|
||||
and enabled with `--cache-file=config.cache' or simply `-C') that saves
|
||||
the results of its tests to speed up reconfiguring. Caching is
|
||||
disabled by default to prevent problems with accidental use of stale
|
||||
cache files.
|
||||
|
||||
If you need to do unusual things to compile the package, please try
|
||||
to figure out how `configure' could check whether to do them, and mail
|
||||
diffs or instructions to the address given in the `README' so they can
|
||||
be considered for the next release. If you are using the cache, and at
|
||||
some point `config.cache' contains results you don't want to keep, you
|
||||
may remove or edit it.
|
||||
|
||||
The file `configure.ac' (or `configure.in') is used to create
|
||||
`configure' by a program called `autoconf'. You need `configure.ac' if
|
||||
you want to change it or regenerate `configure' using a newer version
|
||||
of `autoconf'.
|
||||
|
||||
The simplest way to compile this package is:
|
||||
|
||||
1. `cd' to the directory containing the package's source code and type
|
||||
`./configure' to configure the package for your system.
|
||||
|
||||
Running `configure' might take a while. While running, it prints
|
||||
some messages telling which features it is checking for.
|
||||
|
||||
2. Type `make' to compile the package.
|
||||
|
||||
3. Optionally, type `make check' to run any self-tests that come with
|
||||
the package, generally using the just-built uninstalled binaries.
|
||||
|
||||
4. Type `make install' to install the programs and any data files and
|
||||
documentation. When installing into a prefix owned by root, it is
|
||||
recommended that the package be configured and built as a regular
|
||||
user, and only the `make install' phase executed with root
|
||||
privileges.
|
||||
|
||||
5. Optionally, type `make installcheck' to repeat any self-tests, but
|
||||
this time using the binaries in their final installed location.
|
||||
This target does not install anything. Running this target as a
|
||||
regular user, particularly if the prior `make install' required
|
||||
root privileges, verifies that the installation completed
|
||||
correctly.
|
||||
|
||||
6. You can remove the program binaries and object files from the
|
||||
source code directory by typing `make clean'. To also remove the
|
||||
files that `configure' created (so you can compile the package for
|
||||
a different kind of computer), type `make distclean'. There is
|
||||
also a `make maintainer-clean' target, but that is intended mainly
|
||||
for the package's developers. If you use it, you may have to get
|
||||
all sorts of other programs in order to regenerate files that came
|
||||
with the distribution.
|
||||
|
||||
7. Often, you can also type `make uninstall' to remove the installed
|
||||
files again. In practice, not all packages have tested that
|
||||
uninstallation works correctly, even though it is required by the
|
||||
GNU Coding Standards.
|
||||
|
||||
8. Some packages, particularly those that use Automake, provide `make
|
||||
distcheck', which can by used by developers to test that all other
|
||||
targets like `make install' and `make uninstall' work correctly.
|
||||
This target is generally not run by end users.
|
||||
|
||||
Compilers and Options
|
||||
=====================
|
||||
|
||||
Some systems require unusual options for compilation or linking that
|
||||
the `configure' script does not know about. Run `./configure --help'
|
||||
for details on some of the pertinent environment variables.
|
||||
|
||||
You can give `configure' initial values for configuration parameters
|
||||
by setting variables in the command line or in the environment. Here
|
||||
is an example:
|
||||
|
||||
./configure CC=c99 CFLAGS=-g LIBS=-lposix
|
||||
|
||||
*Note Defining Variables::, for more details.
|
||||
|
||||
Compiling For Multiple Architectures
|
||||
====================================
|
||||
|
||||
You can compile the package for more than one kind of computer at the
|
||||
same time, by placing the object files for each architecture in their
|
||||
own directory. To do this, you can use GNU `make'. `cd' to the
|
||||
directory where you want the object files and executables to go and run
|
||||
the `configure' script. `configure' automatically checks for the
|
||||
source code in the directory that `configure' is in and in `..'. This
|
||||
is known as a "VPATH" build.
|
||||
|
||||
With a non-GNU `make', it is safer to compile the package for one
|
||||
architecture at a time in the source code directory. After you have
|
||||
installed the package for one architecture, use `make distclean' before
|
||||
reconfiguring for another architecture.
|
||||
|
||||
On MacOS X 10.5 and later systems, you can create libraries and
|
||||
executables that work on multiple system types--known as "fat" or
|
||||
"universal" binaries--by specifying multiple `-arch' options to the
|
||||
compiler but only a single `-arch' option to the preprocessor. Like
|
||||
this:
|
||||
|
||||
./configure CC="gcc -arch i386 -arch x86_64 -arch ppc -arch ppc64" \
|
||||
CXX="g++ -arch i386 -arch x86_64 -arch ppc -arch ppc64" \
|
||||
CPP="gcc -E" CXXCPP="g++ -E"
|
||||
|
||||
This is not guaranteed to produce working output in all cases, you
|
||||
may have to build one architecture at a time and combine the results
|
||||
using the `lipo' tool if you have problems.
|
||||
|
||||
Installation Names
|
||||
==================
|
||||
|
||||
By default, `make install' installs the package's commands under
|
||||
`/usr/local/bin', include files under `/usr/local/include', etc. You
|
||||
can specify an installation prefix other than `/usr/local' by giving
|
||||
`configure' the option `--prefix=PREFIX', where PREFIX must be an
|
||||
absolute file name.
|
||||
|
||||
You can specify separate installation prefixes for
|
||||
architecture-specific files and architecture-independent files. If you
|
||||
pass the option `--exec-prefix=PREFIX' to `configure', the package uses
|
||||
PREFIX as the prefix for installing programs and libraries.
|
||||
Documentation and other data files still use the regular prefix.
|
||||
|
||||
In addition, if you use an unusual directory layout you can give
|
||||
options like `--bindir=DIR' to specify different values for particular
|
||||
kinds of files. Run `configure --help' for a list of the directories
|
||||
you can set and what kinds of files go in them. In general, the
|
||||
default for these options is expressed in terms of `${prefix}', so that
|
||||
specifying just `--prefix' will affect all of the other directory
|
||||
specifications that were not explicitly provided.
|
||||
|
||||
The most portable way to affect installation locations is to pass the
|
||||
correct locations to `configure'; however, many packages provide one or
|
||||
both of the following shortcuts of passing variable assignments to the
|
||||
`make install' command line to change installation locations without
|
||||
having to reconfigure or recompile.
|
||||
|
||||
The first method involves providing an override variable for each
|
||||
affected directory. For example, `make install
|
||||
prefix=/alternate/directory' will choose an alternate location for all
|
||||
directory configuration variables that were expressed in terms of
|
||||
`${prefix}'. Any directories that were specified during `configure',
|
||||
but not in terms of `${prefix}', must each be overridden at install
|
||||
time for the entire installation to be relocated. The approach of
|
||||
makefile variable overrides for each directory variable is required by
|
||||
the GNU Coding Standards, and ideally causes no recompilation.
|
||||
However, some platforms have known limitations with the semantics of
|
||||
shared libraries that end up requiring recompilation when using this
|
||||
method, particularly noticeable in packages that use GNU Libtool.
|
||||
|
||||
The second method involves providing the `DESTDIR' variable. For
|
||||
example, `make install DESTDIR=/alternate/directory' will prepend
|
||||
`/alternate/directory' before all installation names. The approach of
|
||||
`DESTDIR' overrides is not required by the GNU Coding Standards, and
|
||||
does not work on platforms that have drive letters. On the other hand,
|
||||
it does better at avoiding recompilation issues, and works well even
|
||||
when some directory options were not specified in terms of `${prefix}'
|
||||
at `configure' time.
|
||||
|
||||
Optional Features
|
||||
=================
|
||||
|
||||
If the package supports it, you can cause programs to be installed
|
||||
with an extra prefix or suffix on their names by giving `configure' the
|
||||
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
|
||||
|
||||
Some packages pay attention to `--enable-FEATURE' options to
|
||||
`configure', where FEATURE indicates an optional part of the package.
|
||||
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
|
||||
is something like `gnu-as' or `x' (for the X Window System). The
|
||||
`README' should mention any `--enable-' and `--with-' options that the
|
||||
package recognizes.
|
||||
|
||||
For packages that use the X Window System, `configure' can usually
|
||||
find the X include and library files automatically, but if it doesn't,
|
||||
you can use the `configure' options `--x-includes=DIR' and
|
||||
`--x-libraries=DIR' to specify their locations.
|
||||
|
||||
Some packages offer the ability to configure how verbose the
|
||||
execution of `make' will be. For these packages, running `./configure
|
||||
--enable-silent-rules' sets the default to minimal output, which can be
|
||||
overridden with `make V=1'; while running `./configure
|
||||
--disable-silent-rules' sets the default to verbose, which can be
|
||||
overridden with `make V=0'.
|
||||
|
||||
Particular systems
|
||||
==================
|
||||
|
||||
On HP-UX, the default C compiler is not ANSI C compatible. If GNU
|
||||
CC is not installed, it is recommended to use the following options in
|
||||
order to use an ANSI C compiler:
|
||||
|
||||
./configure CC="cc -Ae -D_XOPEN_SOURCE=500"
|
||||
|
||||
and if that doesn't work, install pre-built binaries of GCC for HP-UX.
|
||||
|
||||
HP-UX `make' updates targets which have the same time stamps as
|
||||
their prerequisites, which makes it generally unusable when shipped
|
||||
generated files such as `configure' are involved. Use GNU `make'
|
||||
instead.
|
||||
|
||||
On OSF/1 a.k.a. Tru64, some versions of the default C compiler cannot
|
||||
parse its `<wchar.h>' header file. The option `-nodtk' can be used as
|
||||
a workaround. If GNU CC is not installed, it is therefore recommended
|
||||
to try
|
||||
|
||||
./configure CC="cc"
|
||||
|
||||
and if that doesn't work, try
|
||||
|
||||
./configure CC="cc -nodtk"
|
||||
|
||||
On Solaris, don't put `/usr/ucb' early in your `PATH'. This
|
||||
directory contains several dysfunctional programs; working variants of
|
||||
these programs are available in `/usr/bin'. So, if you need `/usr/ucb'
|
||||
in your `PATH', put it _after_ `/usr/bin'.
|
||||
|
||||
On Haiku, software installed for all users goes in `/boot/common',
|
||||
not `/usr/local'. It is recommended to use the following options:
|
||||
|
||||
./configure --prefix=/boot/common
|
||||
|
||||
Specifying the System Type
|
||||
==========================
|
||||
|
||||
There may be some features `configure' cannot figure out
|
||||
automatically, but needs to determine by the type of machine the package
|
||||
will run on. Usually, assuming the package is built to be run on the
|
||||
_same_ architectures, `configure' can figure that out, but if it prints
|
||||
a message saying it cannot guess the machine type, give it the
|
||||
`--build=TYPE' option. TYPE can either be a short name for the system
|
||||
type, such as `sun4', or a canonical name which has the form:
|
||||
|
||||
CPU-COMPANY-SYSTEM
|
||||
|
||||
where SYSTEM can have one of these forms:
|
||||
|
||||
OS
|
||||
KERNEL-OS
|
||||
|
||||
See the file `config.sub' for the possible values of each field. If
|
||||
`config.sub' isn't included in this package, then this package doesn't
|
||||
need to know the machine type.
|
||||
|
||||
If you are _building_ compiler tools for cross-compiling, you should
|
||||
use the option `--target=TYPE' to select the type of system they will
|
||||
produce code for.
|
||||
|
||||
If you want to _use_ a cross compiler, that generates code for a
|
||||
platform different from the build platform, you should specify the
|
||||
"host" platform (i.e., that on which the generated programs will
|
||||
eventually be run) with `--host=TYPE'.
|
||||
|
||||
Sharing Defaults
|
||||
================
|
||||
|
||||
If you want to set default values for `configure' scripts to share,
|
||||
you can create a site shell script called `config.site' that gives
|
||||
default values for variables like `CC', `cache_file', and `prefix'.
|
||||
`configure' looks for `PREFIX/share/config.site' if it exists, then
|
||||
`PREFIX/etc/config.site' if it exists. Or, you can set the
|
||||
`CONFIG_SITE' environment variable to the location of the site script.
|
||||
A warning: not all `configure' scripts look for a site script.
|
||||
|
||||
Defining Variables
|
||||
==================
|
||||
|
||||
Variables not defined in a site shell script can be set in the
|
||||
environment passed to `configure'. However, some packages may run
|
||||
configure again during the build, and the customized values of these
|
||||
variables may be lost. In order to avoid this problem, you should set
|
||||
them in the `configure' command line, using `VAR=value'. For example:
|
||||
|
||||
./configure CC=/usr/local2/bin/gcc
|
||||
|
||||
causes the specified `gcc' to be used as the C compiler (unless it is
|
||||
overridden in the site shell script).
|
||||
|
||||
Unfortunately, this technique does not work for `CONFIG_SHELL' due to
|
||||
an Autoconf limitation. Until the limitation is lifted, you can use
|
||||
this workaround:
|
||||
|
||||
CONFIG_SHELL=/bin/bash ./configure CONFIG_SHELL=/bin/bash
|
||||
|
||||
`configure' Invocation
|
||||
======================
|
||||
|
||||
`configure' recognizes the following options to control how it
|
||||
operates.
|
||||
|
||||
`--help'
|
||||
`-h'
|
||||
Print a summary of all of the options to `configure', and exit.
|
||||
|
||||
`--help=short'
|
||||
`--help=recursive'
|
||||
Print a summary of the options unique to this package's
|
||||
`configure', and exit. The `short' variant lists options used
|
||||
only in the top level, while the `recursive' variant lists options
|
||||
also present in any nested packages.
|
||||
|
||||
`--version'
|
||||
`-V'
|
||||
Print the version of Autoconf used to generate the `configure'
|
||||
script, and exit.
|
||||
|
||||
`--cache-file=FILE'
|
||||
Enable the cache: use and save the results of the tests in FILE,
|
||||
traditionally `config.cache'. FILE defaults to `/dev/null' to
|
||||
disable caching.
|
||||
|
||||
`--config-cache'
|
||||
`-C'
|
||||
Alias for `--cache-file=config.cache'.
|
||||
|
||||
`--quiet'
|
||||
`--silent'
|
||||
`-q'
|
||||
Do not print messages saying which checks are being made. To
|
||||
suppress all normal output, redirect it to `/dev/null' (any error
|
||||
messages will still be shown).
|
||||
|
||||
`--srcdir=DIR'
|
||||
Look for the package's source code in directory DIR. Usually
|
||||
`configure' can determine that directory automatically.
|
||||
|
||||
`--prefix=DIR'
|
||||
Use DIR as the installation prefix. *note Installation Names::
|
||||
for more details, including other options available for fine-tuning
|
||||
the installation locations.
|
||||
|
||||
`--no-create'
|
||||
`-n'
|
||||
Run the configure checks, but stop before creating any output
|
||||
files.
|
||||
|
||||
`configure' also accepts some other, not widely useful, options. Run
|
||||
`configure --help' for more details.
|
140
external/snappy-1.1.3/NEWS
vendored
Normal file
140
external/snappy-1.1.3/NEWS
vendored
Normal file
|
@ -0,0 +1,140 @@
|
|||
Snappy v1.1.3, July 6th 2015:
|
||||
|
||||
This is the first release to be done from GitHub, which means that
|
||||
some minor things like the ChangeLog format has changed (git log
|
||||
format instead of svn log).
|
||||
|
||||
* Add support for Uncompress() from a Source to a Sink.
|
||||
|
||||
* Various minor changes to improve MSVC support; in particular,
|
||||
the unit tests now compile and run under MSVC.
|
||||
|
||||
|
||||
Snappy v1.1.2, February 28th 2014:
|
||||
|
||||
This is a maintenance release with no changes to the actual library
|
||||
source code.
|
||||
|
||||
* Stop distributing benchmark data files that have unclear
|
||||
or unsuitable licensing.
|
||||
|
||||
* Add support for padding chunks in the framing format.
|
||||
|
||||
|
||||
Snappy v1.1.1, October 15th 2013:
|
||||
|
||||
* Add support for uncompressing to iovecs (scatter I/O).
|
||||
The bulk of this patch was contributed by Mohit Aron.
|
||||
|
||||
* Speed up decompression by ~2%; much more so (~13-20%) on
|
||||
a few benchmarks on given compilers and CPUs.
|
||||
|
||||
* Fix a few issues with MSVC compilation.
|
||||
|
||||
* Support truncated test data in the benchmark.
|
||||
|
||||
|
||||
Snappy v1.1.0, January 18th 2013:
|
||||
|
||||
* Snappy now uses 64 kB block size instead of 32 kB. On average,
|
||||
this means it compresses about 3% denser (more so for some
|
||||
inputs), at the same or better speeds.
|
||||
|
||||
* libsnappy no longer depends on iostream.
|
||||
|
||||
* Some small performance improvements in compression on x86
|
||||
(0.5–1%).
|
||||
|
||||
* Various portability fixes for ARM-based platforms, for MSVC,
|
||||
and for GNU/Hurd.
|
||||
|
||||
|
||||
Snappy v1.0.5, February 24th 2012:
|
||||
|
||||
* More speed improvements. Exactly how big will depend on
|
||||
the architecture:
|
||||
|
||||
- 3–10% faster decompression for the base case (x86-64).
|
||||
|
||||
- ARMv7 and higher can now use unaligned accesses,
|
||||
and will see about 30% faster decompression and
|
||||
20–40% faster compression.
|
||||
|
||||
- 32-bit platforms (ARM and 32-bit x86) will see 2–5%
|
||||
faster compression.
|
||||
|
||||
These are all cumulative (e.g., ARM gets all three speedups).
|
||||
|
||||
* Fixed an issue where the unit test would crash on system
|
||||
with less than 256 MB address space available,
|
||||
e.g. some embedded platforms.
|
||||
|
||||
* Added a framing format description, for use over e.g. HTTP,
|
||||
or for a command-line compressor. We do not have any
|
||||
implementations of this at the current point, but there seems
|
||||
to be enough of a general interest in the topic.
|
||||
Also make the format description slightly clearer.
|
||||
|
||||
* Remove some compile-time warnings in -Wall
|
||||
(mostly signed/unsigned comparisons), for easier embedding
|
||||
into projects that use -Wall -Werror.
|
||||
|
||||
|
||||
Snappy v1.0.4, September 15th 2011:
|
||||
|
||||
* Speeded up the decompressor somewhat; typically about 2–8%
|
||||
for Core i7, in 64-bit mode (comparable for Opteron).
|
||||
Somewhat more for some tests, almost no gain for others.
|
||||
|
||||
* Make Snappy compile on certain platforms it didn't before
|
||||
(Solaris with SunPro C++, HP-UX, AIX).
|
||||
|
||||
* Correct some minor errors in the format description.
|
||||
|
||||
|
||||
Snappy v1.0.3, June 2nd 2011:
|
||||
|
||||
* Speeded up the decompressor somewhat; about 3-6% for Core 2,
|
||||
6-13% for Core i7, and 5-12% for Opteron (all in 64-bit mode).
|
||||
|
||||
* Added compressed format documentation. This text is new,
|
||||
but an earlier version from Zeev Tarantov was used as reference.
|
||||
|
||||
* Only link snappy_unittest against -lz and other autodetected
|
||||
libraries, not libsnappy.so (which doesn't need any such dependency).
|
||||
|
||||
* Fixed some display issues in the microbenchmarks, one of which would
|
||||
frequently make the test crash on GNU/Hurd.
|
||||
|
||||
|
||||
Snappy v1.0.2, April 29th 2011:
|
||||
|
||||
* Relicense to a BSD-type license.
|
||||
|
||||
* Added C bindings, contributed by Martin Gieseking.
|
||||
|
||||
* More Win32 fixes, in particular for MSVC.
|
||||
|
||||
* Replace geo.protodata with a newer version.
|
||||
|
||||
* Fix timing inaccuracies in the unit test when comparing Snappy
|
||||
to other algorithms.
|
||||
|
||||
|
||||
Snappy v1.0.1, March 25th 2011:
|
||||
|
||||
This is a maintenance release, mostly containing minor fixes.
|
||||
There is no new functionality. The most important fixes include:
|
||||
|
||||
* The COPYING file and all licensing headers now correctly state that
|
||||
Snappy is licensed under the Apache 2.0 license.
|
||||
|
||||
* snappy_unittest should now compile natively under Windows,
|
||||
as well as on embedded systems with no mmap().
|
||||
|
||||
* Various autotools nits have been fixed.
|
||||
|
||||
|
||||
Snappy v1.0, March 17th 2011:
|
||||
|
||||
* Initial version.
|
135
external/snappy-1.1.3/README
vendored
Normal file
135
external/snappy-1.1.3/README
vendored
Normal file
|
@ -0,0 +1,135 @@
|
|||
Snappy, a fast compressor/decompressor.
|
||||
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
Snappy is a compression/decompression library. It does not aim for maximum
|
||||
compression, or compatibility with any other compression library; instead,
|
||||
it aims for very high speeds and reasonable compression. For instance,
|
||||
compared to the fastest mode of zlib, Snappy is an order of magnitude faster
|
||||
for most inputs, but the resulting compressed files are anywhere from 20% to
|
||||
100% bigger. (For more information, see "Performance", below.)
|
||||
|
||||
Snappy has the following properties:
|
||||
|
||||
* Fast: Compression speeds at 250 MB/sec and beyond, with no assembler code.
|
||||
See "Performance" below.
|
||||
* Stable: Over the last few years, Snappy has compressed and decompressed
|
||||
petabytes of data in Google's production environment. The Snappy bitstream
|
||||
format is stable and will not change between versions.
|
||||
* Robust: The Snappy decompressor is designed not to crash in the face of
|
||||
corrupted or malicious input.
|
||||
* Free and open source software: Snappy is licensed under a BSD-type license.
|
||||
For more information, see the included COPYING file.
|
||||
|
||||
Snappy has previously been called "Zippy" in some Google presentations
|
||||
and the like.
|
||||
|
||||
|
||||
Performance
|
||||
===========
|
||||
|
||||
Snappy is intended to be fast. On a single core of a Core i7 processor
|
||||
in 64-bit mode, it compresses at about 250 MB/sec or more and decompresses at
|
||||
about 500 MB/sec or more. (These numbers are for the slowest inputs in our
|
||||
benchmark suite; others are much faster.) In our tests, Snappy usually
|
||||
is faster than algorithms in the same class (e.g. LZO, LZF, FastLZ, QuickLZ,
|
||||
etc.) while achieving comparable compression ratios.
|
||||
|
||||
Typical compression ratios (based on the benchmark suite) are about 1.5-1.7x
|
||||
for plain text, about 2-4x for HTML, and of course 1.0x for JPEGs, PNGs and
|
||||
other already-compressed data. Similar numbers for zlib in its fastest mode
|
||||
are 2.6-2.8x, 3-7x and 1.0x, respectively. More sophisticated algorithms are
|
||||
capable of achieving yet higher compression rates, although usually at the
|
||||
expense of speed. Of course, compression ratio will vary significantly with
|
||||
the input.
|
||||
|
||||
Although Snappy should be fairly portable, it is primarily optimized
|
||||
for 64-bit x86-compatible processors, and may run slower in other environments.
|
||||
In particular:
|
||||
|
||||
- Snappy uses 64-bit operations in several places to process more data at
|
||||
once than would otherwise be possible.
|
||||
- Snappy assumes unaligned 32- and 64-bit loads and stores are cheap.
|
||||
On some platforms, these must be emulated with single-byte loads
|
||||
and stores, which is much slower.
|
||||
- Snappy assumes little-endian throughout, and needs to byte-swap data in
|
||||
several places if running on a big-endian platform.
|
||||
|
||||
Experience has shown that even heavily tuned code can be improved.
|
||||
Performance optimizations, whether for 64-bit x86 or other platforms,
|
||||
are of course most welcome; see "Contact", below.
|
||||
|
||||
|
||||
Usage
|
||||
=====
|
||||
|
||||
Note that Snappy, both the implementation and the main interface,
|
||||
is written in C++. However, several third-party bindings to other languages
|
||||
are available; see the Google Code page at http://code.google.com/p/snappy/
|
||||
for more information. Also, if you want to use Snappy from C code, you can
|
||||
use the included C bindings in snappy-c.h.
|
||||
|
||||
To use Snappy from your own C++ program, include the file "snappy.h" from
|
||||
your calling file, and link against the compiled library.
|
||||
|
||||
There are many ways to call Snappy, but the simplest possible is
|
||||
|
||||
snappy::Compress(input.data(), input.size(), &output);
|
||||
|
||||
and similarly
|
||||
|
||||
snappy::Uncompress(input.data(), input.size(), &output);
|
||||
|
||||
where "input" and "output" are both instances of std::string.
|
||||
|
||||
There are other interfaces that are more flexible in various ways, including
|
||||
support for custom (non-array) input sources. See the header file for more
|
||||
information.
|
||||
|
||||
|
||||
Tests and benchmarks
|
||||
====================
|
||||
|
||||
When you compile Snappy, snappy_unittest is compiled in addition to the
|
||||
library itself. You do not need it to use the compressor from your own library,
|
||||
but it contains several useful components for Snappy development.
|
||||
|
||||
First of all, it contains unit tests, verifying correctness on your machine in
|
||||
various scenarios. If you want to change or optimize Snappy, please run the
|
||||
tests to verify you have not broken anything. Note that if you have the
|
||||
Google Test library installed, unit test behavior (especially failures) will be
|
||||
significantly more user-friendly. You can find Google Test at
|
||||
|
||||
http://code.google.com/p/googletest/
|
||||
|
||||
You probably also want the gflags library for handling of command-line flags;
|
||||
you can find it at
|
||||
|
||||
http://code.google.com/p/google-gflags/
|
||||
|
||||
In addition to the unit tests, snappy contains microbenchmarks used to
|
||||
tune compression and decompression performance. These are automatically run
|
||||
before the unit tests, but you can disable them using the flag
|
||||
--run_microbenchmarks=false if you have gflags installed (otherwise you will
|
||||
need to edit the source).
|
||||
|
||||
Finally, snappy can benchmark Snappy against a few other compression libraries
|
||||
(zlib, LZO, LZF, FastLZ and QuickLZ), if they were detected at configure time.
|
||||
To benchmark using a given file, give the compression algorithm you want to test
|
||||
Snappy against (e.g. --zlib) and then a list of one or more file names on the
|
||||
command line. The testdata/ directory contains the files used by the
|
||||
microbenchmark, which should provide a reasonably balanced starting point for
|
||||
benchmarking. (Note that baddata[1-3].snappy are not intended as benchmarks; they
|
||||
are used to verify correctness in the presence of corrupted data in the unit
|
||||
test.)
|
||||
|
||||
|
||||
Contact
|
||||
=======
|
||||
|
||||
Snappy is distributed through Google Code. For the latest version, a bug tracker,
|
||||
and other information, see
|
||||
|
||||
http://code.google.com/p/snappy/
|
138
external/snappy-1.1.3/include/snappy/snappy-c.h
vendored
Normal file
138
external/snappy-1.1.3/include/snappy/snappy-c.h
vendored
Normal file
|
@ -0,0 +1,138 @@
|
|||
/*
|
||||
* Copyright 2011 Martin Gieseking <martin.gieseking@uos.de>.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
* modification, are permitted provided that the following conditions are
|
||||
* met:
|
||||
*
|
||||
* * Redistributions of source code must retain the above copyright
|
||||
* notice, this list of conditions and the following disclaimer.
|
||||
* * Redistributions in binary form must reproduce the above
|
||||
* copyright notice, this list of conditions and the following disclaimer
|
||||
* in the documentation and/or other materials provided with the
|
||||
* distribution.
|
||||
* * Neither the name of Google Inc. nor the names of its
|
||||
* contributors may be used to endorse or promote products derived from
|
||||
* this software without specific prior written permission.
|
||||
*
|
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
*
|
||||
* Plain C interface (a wrapper around the C++ implementation).
|
||||
*/
|
||||
|
||||
#ifndef THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_C_H_
|
||||
#define THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_C_H_
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
/*
|
||||
* Return values; see the documentation for each function to know
|
||||
* what each can return.
|
||||
*/
|
||||
typedef enum {
|
||||
SNAPPY_OK = 0,
|
||||
SNAPPY_INVALID_INPUT = 1,
|
||||
SNAPPY_BUFFER_TOO_SMALL = 2
|
||||
} snappy_status;
|
||||
|
||||
/*
|
||||
* Takes the data stored in "input[0..input_length-1]" and stores
|
||||
* it in the array pointed to by "compressed".
|
||||
*
|
||||
* <compressed_length> signals the space available in "compressed".
|
||||
* If it is not at least equal to "snappy_max_compressed_length(input_length)",
|
||||
* SNAPPY_BUFFER_TOO_SMALL is returned. After successful compression,
|
||||
* <compressed_length> contains the true length of the compressed output,
|
||||
* and SNAPPY_OK is returned.
|
||||
*
|
||||
* Example:
|
||||
* size_t output_length = snappy_max_compressed_length(input_length);
|
||||
* char* output = (char*)malloc(output_length);
|
||||
* if (snappy_compress(input, input_length, output, &output_length)
|
||||
* == SNAPPY_OK) {
|
||||
* ... Process(output, output_length) ...
|
||||
* }
|
||||
* free(output);
|
||||
*/
|
||||
snappy_status snappy_compress(const char* input,
|
||||
size_t input_length,
|
||||
char* compressed,
|
||||
size_t* compressed_length);
|
||||
|
||||
/*
|
||||
* Given data in "compressed[0..compressed_length-1]" generated by
|
||||
* calling the snappy_compress routine, this routine stores
|
||||
* the uncompressed data to
|
||||
* uncompressed[0..uncompressed_length-1].
|
||||
* Returns failure (a value not equal to SNAPPY_OK) if the message
|
||||
* is corrupted and could not be decrypted.
|
||||
*
|
||||
* <uncompressed_length> signals the space available in "uncompressed".
|
||||
* If it is not at least equal to the value returned by
|
||||
* snappy_uncompressed_length for this stream, SNAPPY_BUFFER_TOO_SMALL
|
||||
* is returned. After successful decompression, <uncompressed_length>
|
||||
* contains the true length of the decompressed output.
|
||||
*
|
||||
* Example:
|
||||
* size_t output_length;
|
||||
* if (snappy_uncompressed_length(input, input_length, &output_length)
|
||||
* != SNAPPY_OK) {
|
||||
* ... fail ...
|
||||
* }
|
||||
* char* output = (char*)malloc(output_length);
|
||||
* if (snappy_uncompress(input, input_length, output, &output_length)
|
||||
* == SNAPPY_OK) {
|
||||
* ... Process(output, output_length) ...
|
||||
* }
|
||||
* free(output);
|
||||
*/
|
||||
snappy_status snappy_uncompress(const char* compressed,
|
||||
size_t compressed_length,
|
||||
char* uncompressed,
|
||||
size_t* uncompressed_length);
|
||||
|
||||
/*
|
||||
* Returns the maximal size of the compressed representation of
|
||||
* input data that is "source_length" bytes in length.
|
||||
*/
|
||||
size_t snappy_max_compressed_length(size_t source_length);
|
||||
|
||||
/*
|
||||
* REQUIRES: "compressed[]" was produced by snappy_compress()
|
||||
* Returns SNAPPY_OK and stores the length of the uncompressed data in
|
||||
* *result normally. Returns SNAPPY_INVALID_INPUT on parsing error.
|
||||
* This operation takes O(1) time.
|
||||
*/
|
||||
snappy_status snappy_uncompressed_length(const char* compressed,
|
||||
size_t compressed_length,
|
||||
size_t* result);
|
||||
|
||||
/*
|
||||
* Check if the contents of "compressed[]" can be uncompressed successfully.
|
||||
* Does not return the uncompressed data; if so, returns SNAPPY_OK,
|
||||
* or if not, returns SNAPPY_INVALID_INPUT.
|
||||
* Takes time proportional to compressed_length, but is usually at least a
|
||||
* factor of four faster than actual decompression.
|
||||
*/
|
||||
snappy_status snappy_validate_compressed_buffer(const char* compressed,
|
||||
size_t compressed_length);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
|
||||
#endif /* THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_C_H_ */
|
103
external/snappy-1.1.3/include/snappy/snappy-stubs-public.h
vendored
Normal file
103
external/snappy-1.1.3/include/snappy/snappy-stubs-public.h
vendored
Normal file
|
@ -0,0 +1,103 @@
|
|||
// Copyright 2011 Google Inc. All Rights Reserved.
|
||||
// Author: sesse@google.com (Steinar H. Gunderson)
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
//
|
||||
// Various type stubs for the open-source version of Snappy.
|
||||
//
|
||||
// This file cannot include config.h, as it is included from snappy.h,
|
||||
// which is a public header. Instead, snappy-stubs-public.h is generated by
|
||||
// from snappy-stubs-public.h.in at configure time.
|
||||
|
||||
#ifndef THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_STUBS_PUBLIC_H_
|
||||
#define THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_STUBS_PUBLIC_H_
|
||||
|
||||
#if 1
|
||||
#include <stdint.h>
|
||||
#endif
|
||||
|
||||
#if 1
|
||||
#include <stddef.h>
|
||||
#endif
|
||||
|
||||
#if defined(_MSC_VER) && (_MSC_VER <= 1900)
|
||||
#include <basetsd.h>
|
||||
typedef SSIZE_T ssize_t;
|
||||
#endif
|
||||
|
||||
#if 0
|
||||
#include <sys/uio.h>
|
||||
#endif
|
||||
|
||||
#define SNAPPY_MAJOR 1
|
||||
#define SNAPPY_MINOR 1
|
||||
#define SNAPPY_PATCHLEVEL 3
|
||||
#define SNAPPY_VERSION \
|
||||
((SNAPPY_MAJOR << 16) | (SNAPPY_MINOR << 8) | SNAPPY_PATCHLEVEL)
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace snappy {
|
||||
|
||||
#if 1
|
||||
typedef int8_t int8;
|
||||
typedef uint8_t uint8;
|
||||
typedef int16_t int16;
|
||||
typedef uint16_t uint16;
|
||||
typedef int32_t int32;
|
||||
typedef uint32_t uint32;
|
||||
typedef int64_t int64;
|
||||
typedef uint64_t uint64;
|
||||
#else
|
||||
typedef signed char int8;
|
||||
typedef unsigned char uint8;
|
||||
typedef short int16;
|
||||
typedef unsigned short uint16;
|
||||
typedef int int32;
|
||||
typedef unsigned int uint32;
|
||||
typedef long long int64;
|
||||
typedef unsigned long long uint64;
|
||||
#endif
|
||||
|
||||
typedef std::string string;
|
||||
|
||||
#define DISALLOW_COPY_AND_ASSIGN(TypeName) \
|
||||
TypeName(const TypeName&); \
|
||||
void operator=(const TypeName&)
|
||||
|
||||
#if !0
|
||||
// Windows does not have an iovec type, yet the concept is universally useful.
|
||||
// It is simple to define it ourselves, so we put it inside our own namespace.
|
||||
struct iovec {
|
||||
void* iov_base;
|
||||
size_t iov_len;
|
||||
};
|
||||
#endif
|
||||
|
||||
} // namespace snappy
|
||||
|
||||
#endif // THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_STUBS_PUBLIC_H_
|
203
external/snappy-1.1.3/include/snappy/snappy.h
vendored
Normal file
203
external/snappy-1.1.3/include/snappy/snappy.h
vendored
Normal file
|
@ -0,0 +1,203 @@
|
|||
// Copyright 2005 and onwards Google Inc.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
//
|
||||
// A light-weight compression algorithm. It is designed for speed of
|
||||
// compression and decompression, rather than for the utmost in space
|
||||
// savings.
|
||||
//
|
||||
// For getting better compression ratios when you are compressing data
|
||||
// with long repeated sequences or compressing data that is similar to
|
||||
// other data, while still compressing fast, you might look at first
|
||||
// using BMDiff and then compressing the output of BMDiff with
|
||||
// Snappy.
|
||||
|
||||
#ifndef THIRD_PARTY_SNAPPY_SNAPPY_H__
|
||||
#define THIRD_PARTY_SNAPPY_SNAPPY_H__
|
||||
|
||||
#include <stddef.h>
|
||||
#include <string>
|
||||
|
||||
#include "snappy/snappy-stubs-public.h"
|
||||
|
||||
namespace snappy {
|
||||
class Source;
|
||||
class Sink;
|
||||
|
||||
// ------------------------------------------------------------------------
|
||||
// Generic compression/decompression routines.
|
||||
// ------------------------------------------------------------------------
|
||||
|
||||
// Compress the bytes read from "*source" and append to "*sink". Return the
|
||||
// number of bytes written.
|
||||
size_t Compress(Source* source, Sink* sink);
|
||||
|
||||
// Find the uncompressed length of the given stream, as given by the header.
|
||||
// Note that the true length could deviate from this; the stream could e.g.
|
||||
// be truncated.
|
||||
//
|
||||
// Also note that this leaves "*source" in a state that is unsuitable for
|
||||
// further operations, such as RawUncompress(). You will need to rewind
|
||||
// or recreate the source yourself before attempting any further calls.
|
||||
bool GetUncompressedLength(Source* source, uint32* result);
|
||||
|
||||
// ------------------------------------------------------------------------
|
||||
// Higher-level string based routines (should be sufficient for most users)
|
||||
// ------------------------------------------------------------------------
|
||||
|
||||
// Sets "*output" to the compressed version of "input[0,input_length-1]".
|
||||
// Original contents of *output are lost.
|
||||
//
|
||||
// REQUIRES: "input[]" is not an alias of "*output".
|
||||
size_t Compress(const char* input, size_t input_length, string* output);
|
||||
|
||||
// Decompresses "compressed[0,compressed_length-1]" to "*uncompressed".
|
||||
// Original contents of "*uncompressed" are lost.
|
||||
//
|
||||
// REQUIRES: "compressed[]" is not an alias of "*uncompressed".
|
||||
//
|
||||
// returns false if the message is corrupted and could not be decompressed
|
||||
bool Uncompress(const char* compressed, size_t compressed_length,
|
||||
string* uncompressed);
|
||||
|
||||
// Decompresses "compressed" to "*uncompressed".
|
||||
//
|
||||
// returns false if the message is corrupted and could not be decompressed
|
||||
bool Uncompress(Source* compressed, Sink* uncompressed);
|
||||
|
||||
// This routine uncompresses as much of the "compressed" as possible
|
||||
// into sink. It returns the number of valid bytes added to sink
|
||||
// (extra invalid bytes may have been added due to errors; the caller
|
||||
// should ignore those). The emitted data typically has length
|
||||
// GetUncompressedLength(), but may be shorter if an error is
|
||||
// encountered.
|
||||
size_t UncompressAsMuchAsPossible(Source* compressed, Sink* uncompressed);
|
||||
|
||||
// ------------------------------------------------------------------------
|
||||
// Lower-level character array based routines. May be useful for
|
||||
// efficiency reasons in certain circumstances.
|
||||
// ------------------------------------------------------------------------
|
||||
|
||||
// REQUIRES: "compressed" must point to an area of memory that is at
|
||||
// least "MaxCompressedLength(input_length)" bytes in length.
|
||||
//
|
||||
// Takes the data stored in "input[0..input_length]" and stores
|
||||
// it in the array pointed to by "compressed".
|
||||
//
|
||||
// "*compressed_length" is set to the length of the compressed output.
|
||||
//
|
||||
// Example:
|
||||
// char* output = new char[snappy::MaxCompressedLength(input_length)];
|
||||
// size_t output_length;
|
||||
// RawCompress(input, input_length, output, &output_length);
|
||||
// ... Process(output, output_length) ...
|
||||
// delete [] output;
|
||||
void RawCompress(const char* input,
|
||||
size_t input_length,
|
||||
char* compressed,
|
||||
size_t* compressed_length);
|
||||
|
||||
// Given data in "compressed[0..compressed_length-1]" generated by
|
||||
// calling the Snappy::Compress routine, this routine
|
||||
// stores the uncompressed data to
|
||||
// uncompressed[0..GetUncompressedLength(compressed)-1]
|
||||
// returns false if the message is corrupted and could not be decrypted
|
||||
bool RawUncompress(const char* compressed, size_t compressed_length,
|
||||
char* uncompressed);
|
||||
|
||||
// Given data from the byte source 'compressed' generated by calling
|
||||
// the Snappy::Compress routine, this routine stores the uncompressed
|
||||
// data to
|
||||
// uncompressed[0..GetUncompressedLength(compressed,compressed_length)-1]
|
||||
// returns false if the message is corrupted and could not be decrypted
|
||||
bool RawUncompress(Source* compressed, char* uncompressed);
|
||||
|
||||
// Given data in "compressed[0..compressed_length-1]" generated by
|
||||
// calling the Snappy::Compress routine, this routine
|
||||
// stores the uncompressed data to the iovec "iov". The number of physical
|
||||
// buffers in "iov" is given by iov_cnt and their cumulative size
|
||||
// must be at least GetUncompressedLength(compressed). The individual buffers
|
||||
// in "iov" must not overlap with each other.
|
||||
//
|
||||
// returns false if the message is corrupted and could not be decrypted
|
||||
bool RawUncompressToIOVec(const char* compressed, size_t compressed_length,
|
||||
const struct iovec* iov, size_t iov_cnt);
|
||||
|
||||
// Given data from the byte source 'compressed' generated by calling
|
||||
// the Snappy::Compress routine, this routine stores the uncompressed
|
||||
// data to the iovec "iov". The number of physical
|
||||
// buffers in "iov" is given by iov_cnt and their cumulative size
|
||||
// must be at least GetUncompressedLength(compressed). The individual buffers
|
||||
// in "iov" must not overlap with each other.
|
||||
//
|
||||
// returns false if the message is corrupted and could not be decrypted
|
||||
bool RawUncompressToIOVec(Source* compressed, const struct iovec* iov,
|
||||
size_t iov_cnt);
|
||||
|
||||
// Returns the maximal size of the compressed representation of
|
||||
// input data that is "source_bytes" bytes in length;
|
||||
size_t MaxCompressedLength(size_t source_bytes);
|
||||
|
||||
// REQUIRES: "compressed[]" was produced by RawCompress() or Compress()
|
||||
// Returns true and stores the length of the uncompressed data in
|
||||
// *result normally. Returns false on parsing error.
|
||||
// This operation takes O(1) time.
|
||||
bool GetUncompressedLength(const char* compressed, size_t compressed_length,
|
||||
size_t* result);
|
||||
|
||||
// Returns true iff the contents of "compressed[]" can be uncompressed
|
||||
// successfully. Does not return the uncompressed data. Takes
|
||||
// time proportional to compressed_length, but is usually at least
|
||||
// a factor of four faster than actual decompression.
|
||||
bool IsValidCompressedBuffer(const char* compressed,
|
||||
size_t compressed_length);
|
||||
|
||||
// Returns true iff the contents of "compressed" can be uncompressed
|
||||
// successfully. Does not return the uncompressed data. Takes
|
||||
// time proportional to *compressed length, but is usually at least
|
||||
// a factor of four faster than actual decompression.
|
||||
// On success, consumes all of *compressed. On failure, consumes an
|
||||
// unspecified prefix of *compressed.
|
||||
bool IsValidCompressed(Source* compressed);
|
||||
|
||||
// The size of a compression block. Note that many parts of the compression
|
||||
// code assumes that kBlockSize <= 65536; in particular, the hash table
|
||||
// can only store 16-bit offsets, and EmitCopy() also assumes the offset
|
||||
// is 65535 bytes or less. Note also that if you change this, it will
|
||||
// affect the framing format (see framing_format.txt).
|
||||
//
|
||||
// Note that there might be older data around that is compressed with larger
|
||||
// block sizes, so the decompression code should not rely on the
|
||||
// non-existence of long backreferences.
|
||||
static const int kBlockLog = 16;
|
||||
static const size_t kBlockSize = 1 << kBlockLog;
|
||||
|
||||
static const int kMaxHashTableBits = 14;
|
||||
static const size_t kMaxHashTableSize = 1 << kMaxHashTableBits;
|
||||
} // end namespace snappy
|
||||
|
||||
#endif // THIRD_PARTY_SNAPPY_SNAPPY_H__
|
90
external/snappy-1.1.3/snappy-c.cc
vendored
Normal file
90
external/snappy-1.1.3/snappy-c.cc
vendored
Normal file
|
@ -0,0 +1,90 @@
|
|||
// Copyright 2011 Martin Gieseking <martin.gieseking@uos.de>.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
#include "snappy/snappy.h"
|
||||
#include "snappy/snappy-c.h"
|
||||
|
||||
extern "C" {
|
||||
|
||||
snappy_status snappy_compress(const char* input,
|
||||
size_t input_length,
|
||||
char* compressed,
|
||||
size_t *compressed_length) {
|
||||
if (*compressed_length < snappy_max_compressed_length(input_length)) {
|
||||
return SNAPPY_BUFFER_TOO_SMALL;
|
||||
}
|
||||
snappy::RawCompress(input, input_length, compressed, compressed_length);
|
||||
return SNAPPY_OK;
|
||||
}
|
||||
|
||||
snappy_status snappy_uncompress(const char* compressed,
|
||||
size_t compressed_length,
|
||||
char* uncompressed,
|
||||
size_t* uncompressed_length) {
|
||||
size_t real_uncompressed_length;
|
||||
if (!snappy::GetUncompressedLength(compressed,
|
||||
compressed_length,
|
||||
&real_uncompressed_length)) {
|
||||
return SNAPPY_INVALID_INPUT;
|
||||
}
|
||||
if (*uncompressed_length < real_uncompressed_length) {
|
||||
return SNAPPY_BUFFER_TOO_SMALL;
|
||||
}
|
||||
if (!snappy::RawUncompress(compressed, compressed_length, uncompressed)) {
|
||||
return SNAPPY_INVALID_INPUT;
|
||||
}
|
||||
*uncompressed_length = real_uncompressed_length;
|
||||
return SNAPPY_OK;
|
||||
}
|
||||
|
||||
size_t snappy_max_compressed_length(size_t source_length) {
|
||||
return snappy::MaxCompressedLength(source_length);
|
||||
}
|
||||
|
||||
snappy_status snappy_uncompressed_length(const char *compressed,
|
||||
size_t compressed_length,
|
||||
size_t *result) {
|
||||
if (snappy::GetUncompressedLength(compressed,
|
||||
compressed_length,
|
||||
result)) {
|
||||
return SNAPPY_OK;
|
||||
} else {
|
||||
return SNAPPY_INVALID_INPUT;
|
||||
}
|
||||
}
|
||||
|
||||
snappy_status snappy_validate_compressed_buffer(const char *compressed,
|
||||
size_t compressed_length) {
|
||||
if (snappy::IsValidCompressedBuffer(compressed, compressed_length)) {
|
||||
return SNAPPY_OK;
|
||||
} else {
|
||||
return SNAPPY_INVALID_INPUT;
|
||||
}
|
||||
}
|
||||
|
||||
} // extern "C"
|
150
external/snappy-1.1.3/snappy-internal.h
vendored
Normal file
150
external/snappy-1.1.3/snappy-internal.h
vendored
Normal file
|
@ -0,0 +1,150 @@
|
|||
// Copyright 2008 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
//
|
||||
// Internals shared between the Snappy implementation and its unittest.
|
||||
|
||||
#ifndef THIRD_PARTY_SNAPPY_SNAPPY_INTERNAL_H_
|
||||
#define THIRD_PARTY_SNAPPY_SNAPPY_INTERNAL_H_
|
||||
|
||||
#include "snappy-stubs-internal.h"
|
||||
|
||||
namespace snappy {
|
||||
namespace internal {
|
||||
|
||||
class WorkingMemory {
|
||||
public:
|
||||
WorkingMemory() : large_table_(NULL) { }
|
||||
~WorkingMemory() { delete[] large_table_; }
|
||||
|
||||
// Allocates and clears a hash table using memory in "*this",
|
||||
// stores the number of buckets in "*table_size" and returns a pointer to
|
||||
// the base of the hash table.
|
||||
uint16* GetHashTable(size_t input_size, int* table_size);
|
||||
|
||||
private:
|
||||
uint16 small_table_[1<<10]; // 2KB
|
||||
uint16* large_table_; // Allocated only when needed
|
||||
|
||||
DISALLOW_COPY_AND_ASSIGN(WorkingMemory);
|
||||
};
|
||||
|
||||
// Flat array compression that does not emit the "uncompressed length"
|
||||
// prefix. Compresses "input" string to the "*op" buffer.
|
||||
//
|
||||
// REQUIRES: "input_length <= kBlockSize"
|
||||
// REQUIRES: "op" points to an array of memory that is at least
|
||||
// "MaxCompressedLength(input_length)" in size.
|
||||
// REQUIRES: All elements in "table[0..table_size-1]" are initialized to zero.
|
||||
// REQUIRES: "table_size" is a power of two
|
||||
//
|
||||
// Returns an "end" pointer into "op" buffer.
|
||||
// "end - op" is the compressed size of "input".
|
||||
char* CompressFragment(const char* input,
|
||||
size_t input_length,
|
||||
char* op,
|
||||
uint16* table,
|
||||
const int table_size);
|
||||
|
||||
// Return the largest n such that
|
||||
//
|
||||
// s1[0,n-1] == s2[0,n-1]
|
||||
// and n <= (s2_limit - s2).
|
||||
//
|
||||
// Does not read *s2_limit or beyond.
|
||||
// Does not read *(s1 + (s2_limit - s2)) or beyond.
|
||||
// Requires that s2_limit >= s2.
|
||||
//
|
||||
// Separate implementation for x86_64, for speed. Uses the fact that
|
||||
// x86_64 is little endian.
|
||||
#if defined(ARCH_K8)
|
||||
static inline int FindMatchLength(const char* s1,
|
||||
const char* s2,
|
||||
const char* s2_limit) {
|
||||
assert(s2_limit >= s2);
|
||||
int matched = 0;
|
||||
|
||||
// Find out how long the match is. We loop over the data 64 bits at a
|
||||
// time until we find a 64-bit block that doesn't match; then we find
|
||||
// the first non-matching bit and use that to calculate the total
|
||||
// length of the match.
|
||||
while (PREDICT_TRUE(s2 <= s2_limit - 8)) {
|
||||
if (UNALIGNED_LOAD64(s2) == UNALIGNED_LOAD64(s1 + matched)) {
|
||||
s2 += 8;
|
||||
matched += 8;
|
||||
} else {
|
||||
// On current (mid-2008) Opteron models there is a 3% more
|
||||
// efficient code sequence to find the first non-matching byte.
|
||||
// However, what follows is ~10% better on Intel Core 2 and newer,
|
||||
// and we expect AMD's bsf instruction to improve.
|
||||
uint64 x = UNALIGNED_LOAD64(s2) ^ UNALIGNED_LOAD64(s1 + matched);
|
||||
int matching_bits = Bits::FindLSBSetNonZero64(x);
|
||||
matched += matching_bits >> 3;
|
||||
return matched;
|
||||
}
|
||||
}
|
||||
while (PREDICT_TRUE(s2 < s2_limit)) {
|
||||
if (s1[matched] == *s2) {
|
||||
++s2;
|
||||
++matched;
|
||||
} else {
|
||||
return matched;
|
||||
}
|
||||
}
|
||||
return matched;
|
||||
}
|
||||
#else
|
||||
static inline int FindMatchLength(const char* s1,
|
||||
const char* s2,
|
||||
const char* s2_limit) {
|
||||
// Implementation based on the x86-64 version, above.
|
||||
assert(s2_limit >= s2);
|
||||
int matched = 0;
|
||||
|
||||
while (s2 <= s2_limit - 4 &&
|
||||
UNALIGNED_LOAD32(s2) == UNALIGNED_LOAD32(s1 + matched)) {
|
||||
s2 += 4;
|
||||
matched += 4;
|
||||
}
|
||||
if (LittleEndian::IsLittleEndian() && s2 <= s2_limit - 4) {
|
||||
uint32 x = UNALIGNED_LOAD32(s2) ^ UNALIGNED_LOAD32(s1 + matched);
|
||||
int matching_bits = Bits::FindLSBSetNonZero(x);
|
||||
matched += matching_bits >> 3;
|
||||
} else {
|
||||
while ((s2 < s2_limit) && (s1[matched] == *s2)) {
|
||||
++s2;
|
||||
++matched;
|
||||
}
|
||||
}
|
||||
return matched;
|
||||
}
|
||||
#endif
|
||||
|
||||
} // end namespace internal
|
||||
} // end namespace snappy
|
||||
|
||||
#endif // THIRD_PARTY_SNAPPY_SNAPPY_INTERNAL_H_
|
104
external/snappy-1.1.3/snappy-sinksource.cc
vendored
Normal file
104
external/snappy-1.1.3/snappy-sinksource.cc
vendored
Normal file
|
@ -0,0 +1,104 @@
|
|||
// Copyright 2011 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
#include <string.h>
|
||||
|
||||
#include "snappy-sinksource.h"
|
||||
|
||||
namespace snappy {
|
||||
|
||||
Source::~Source() { }
|
||||
|
||||
Sink::~Sink() { }
|
||||
|
||||
char* Sink::GetAppendBuffer(size_t length, char* scratch) {
|
||||
return scratch;
|
||||
}
|
||||
|
||||
char* Sink::GetAppendBufferVariable(
|
||||
size_t min_size, size_t desired_size_hint, char* scratch,
|
||||
size_t scratch_size, size_t* allocated_size) {
|
||||
*allocated_size = scratch_size;
|
||||
return scratch;
|
||||
}
|
||||
|
||||
void Sink::AppendAndTakeOwnership(
|
||||
char* bytes, size_t n,
|
||||
void (*deleter)(void*, const char*, size_t),
|
||||
void *deleter_arg) {
|
||||
Append(bytes, n);
|
||||
(*deleter)(deleter_arg, bytes, n);
|
||||
}
|
||||
|
||||
ByteArraySource::~ByteArraySource() { }
|
||||
|
||||
size_t ByteArraySource::Available() const { return left_; }
|
||||
|
||||
const char* ByteArraySource::Peek(size_t* len) {
|
||||
*len = left_;
|
||||
return ptr_;
|
||||
}
|
||||
|
||||
void ByteArraySource::Skip(size_t n) {
|
||||
left_ -= n;
|
||||
ptr_ += n;
|
||||
}
|
||||
|
||||
UncheckedByteArraySink::~UncheckedByteArraySink() { }
|
||||
|
||||
void UncheckedByteArraySink::Append(const char* data, size_t n) {
|
||||
// Do no copying if the caller filled in the result of GetAppendBuffer()
|
||||
if (data != dest_) {
|
||||
memcpy(dest_, data, n);
|
||||
}
|
||||
dest_ += n;
|
||||
}
|
||||
|
||||
char* UncheckedByteArraySink::GetAppendBuffer(size_t len, char* scratch) {
|
||||
return dest_;
|
||||
}
|
||||
|
||||
void UncheckedByteArraySink::AppendAndTakeOwnership(
|
||||
char* data, size_t n,
|
||||
void (*deleter)(void*, const char*, size_t),
|
||||
void *deleter_arg) {
|
||||
if (data != dest_) {
|
||||
memcpy(dest_, data, n);
|
||||
(*deleter)(deleter_arg, data, n);
|
||||
}
|
||||
dest_ += n;
|
||||
}
|
||||
|
||||
char* UncheckedByteArraySink::GetAppendBufferVariable(
|
||||
size_t min_size, size_t desired_size_hint, char* scratch,
|
||||
size_t scratch_size, size_t* allocated_size) {
|
||||
*allocated_size = desired_size_hint;
|
||||
return dest_;
|
||||
}
|
||||
|
||||
} // namespace snappy
|
182
external/snappy-1.1.3/snappy-sinksource.h
vendored
Normal file
182
external/snappy-1.1.3/snappy-sinksource.h
vendored
Normal file
|
@ -0,0 +1,182 @@
|
|||
// Copyright 2011 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
#ifndef THIRD_PARTY_SNAPPY_SNAPPY_SINKSOURCE_H_
|
||||
#define THIRD_PARTY_SNAPPY_SNAPPY_SINKSOURCE_H_
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
namespace snappy {
|
||||
|
||||
// A Sink is an interface that consumes a sequence of bytes.
|
||||
class Sink {
|
||||
public:
|
||||
Sink() { }
|
||||
virtual ~Sink();
|
||||
|
||||
// Append "bytes[0,n-1]" to this.
|
||||
virtual void Append(const char* bytes, size_t n) = 0;
|
||||
|
||||
// Returns a writable buffer of the specified length for appending.
|
||||
// May return a pointer to the caller-owned scratch buffer which
|
||||
// must have at least the indicated length. The returned buffer is
|
||||
// only valid until the next operation on this Sink.
|
||||
//
|
||||
// After writing at most "length" bytes, call Append() with the
|
||||
// pointer returned from this function and the number of bytes
|
||||
// written. Many Append() implementations will avoid copying
|
||||
// bytes if this function returned an internal buffer.
|
||||
//
|
||||
// If a non-scratch buffer is returned, the caller may only pass a
|
||||
// prefix of it to Append(). That is, it is not correct to pass an
|
||||
// interior pointer of the returned array to Append().
|
||||
//
|
||||
// The default implementation always returns the scratch buffer.
|
||||
virtual char* GetAppendBuffer(size_t length, char* scratch);
|
||||
|
||||
// For higher performance, Sink implementations can provide custom
|
||||
// AppendAndTakeOwnership() and GetAppendBufferVariable() methods.
|
||||
// These methods can reduce the number of copies done during
|
||||
// compression/decompression.
|
||||
|
||||
// Append "bytes[0,n-1] to the sink. Takes ownership of "bytes"
|
||||
// and calls the deleter function as (*deleter)(deleter_arg, bytes, n)
|
||||
// to free the buffer. deleter function must be non NULL.
|
||||
//
|
||||
// The default implementation just calls Append and frees "bytes".
|
||||
// Other implementations may avoid a copy while appending the buffer.
|
||||
virtual void AppendAndTakeOwnership(
|
||||
char* bytes, size_t n, void (*deleter)(void*, const char*, size_t),
|
||||
void *deleter_arg);
|
||||
|
||||
// Returns a writable buffer for appending and writes the buffer's capacity to
|
||||
// *allocated_size. Guarantees *allocated_size >= min_size.
|
||||
// May return a pointer to the caller-owned scratch buffer which must have
|
||||
// scratch_size >= min_size.
|
||||
//
|
||||
// The returned buffer is only valid until the next operation
|
||||
// on this ByteSink.
|
||||
//
|
||||
// After writing at most *allocated_size bytes, call Append() with the
|
||||
// pointer returned from this function and the number of bytes written.
|
||||
// Many Append() implementations will avoid copying bytes if this function
|
||||
// returned an internal buffer.
|
||||
//
|
||||
// If the sink implementation allocates or reallocates an internal buffer,
|
||||
// it should use the desired_size_hint if appropriate. If a caller cannot
|
||||
// provide a reasonable guess at the desired capacity, it should set
|
||||
// desired_size_hint = 0.
|
||||
//
|
||||
// If a non-scratch buffer is returned, the caller may only pass
|
||||
// a prefix to it to Append(). That is, it is not correct to pass an
|
||||
// interior pointer to Append().
|
||||
//
|
||||
// The default implementation always returns the scratch buffer.
|
||||
virtual char* GetAppendBufferVariable(
|
||||
size_t min_size, size_t desired_size_hint, char* scratch,
|
||||
size_t scratch_size, size_t* allocated_size);
|
||||
|
||||
private:
|
||||
// No copying
|
||||
Sink(const Sink&);
|
||||
void operator=(const Sink&);
|
||||
};
|
||||
|
||||
// A Source is an interface that yields a sequence of bytes
|
||||
class Source {
|
||||
public:
|
||||
Source() { }
|
||||
virtual ~Source();
|
||||
|
||||
// Return the number of bytes left to read from the source
|
||||
virtual size_t Available() const = 0;
|
||||
|
||||
// Peek at the next flat region of the source. Does not reposition
|
||||
// the source. The returned region is empty iff Available()==0.
|
||||
//
|
||||
// Returns a pointer to the beginning of the region and store its
|
||||
// length in *len.
|
||||
//
|
||||
// The returned region is valid until the next call to Skip() or
|
||||
// until this object is destroyed, whichever occurs first.
|
||||
//
|
||||
// The returned region may be larger than Available() (for example
|
||||
// if this ByteSource is a view on a substring of a larger source).
|
||||
// The caller is responsible for ensuring that it only reads the
|
||||
// Available() bytes.
|
||||
virtual const char* Peek(size_t* len) = 0;
|
||||
|
||||
// Skip the next n bytes. Invalidates any buffer returned by
|
||||
// a previous call to Peek().
|
||||
// REQUIRES: Available() >= n
|
||||
virtual void Skip(size_t n) = 0;
|
||||
|
||||
private:
|
||||
// No copying
|
||||
Source(const Source&);
|
||||
void operator=(const Source&);
|
||||
};
|
||||
|
||||
// A Source implementation that yields the contents of a flat array
|
||||
class ByteArraySource : public Source {
|
||||
public:
|
||||
ByteArraySource(const char* p, size_t n) : ptr_(p), left_(n) { }
|
||||
virtual ~ByteArraySource();
|
||||
virtual size_t Available() const;
|
||||
virtual const char* Peek(size_t* len);
|
||||
virtual void Skip(size_t n);
|
||||
private:
|
||||
const char* ptr_;
|
||||
size_t left_;
|
||||
};
|
||||
|
||||
// A Sink implementation that writes to a flat array without any bound checks.
|
||||
class UncheckedByteArraySink : public Sink {
|
||||
public:
|
||||
explicit UncheckedByteArraySink(char* dest) : dest_(dest) { }
|
||||
virtual ~UncheckedByteArraySink();
|
||||
virtual void Append(const char* data, size_t n);
|
||||
virtual char* GetAppendBuffer(size_t len, char* scratch);
|
||||
virtual char* GetAppendBufferVariable(
|
||||
size_t min_size, size_t desired_size_hint, char* scratch,
|
||||
size_t scratch_size, size_t* allocated_size);
|
||||
virtual void AppendAndTakeOwnership(
|
||||
char* bytes, size_t n, void (*deleter)(void*, const char*, size_t),
|
||||
void *deleter_arg);
|
||||
|
||||
// Return the current output pointer so that a caller can see how
|
||||
// many bytes were produced.
|
||||
// Note: this is not a Sink method.
|
||||
char* CurrentDestination() const { return dest_; }
|
||||
private:
|
||||
char* dest_;
|
||||
};
|
||||
|
||||
} // namespace snappy
|
||||
|
||||
#endif // THIRD_PARTY_SNAPPY_SNAPPY_SINKSOURCE_H_
|
42
external/snappy-1.1.3/snappy-stubs-internal.cc
vendored
Normal file
42
external/snappy-1.1.3/snappy-stubs-internal.cc
vendored
Normal file
|
@ -0,0 +1,42 @@
|
|||
// Copyright 2011 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
#include <algorithm>
|
||||
#include <string>
|
||||
|
||||
#include "snappy-stubs-internal.h"
|
||||
|
||||
namespace snappy {
|
||||
|
||||
void Varint::Append32(string* s, uint32 value) {
|
||||
char buf[Varint::kMax32];
|
||||
const char* p = Varint::Encode32(buf, value);
|
||||
s->append(buf, p - buf);
|
||||
}
|
||||
|
||||
} // namespace snappy
|
491
external/snappy-1.1.3/snappy-stubs-internal.h
vendored
Normal file
491
external/snappy-1.1.3/snappy-stubs-internal.h
vendored
Normal file
|
@ -0,0 +1,491 @@
|
|||
// Copyright 2011 Google Inc. All Rights Reserved.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
//
|
||||
// Various stubs for the open-source version of Snappy.
|
||||
|
||||
#ifndef THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_STUBS_INTERNAL_H_
|
||||
#define THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_STUBS_INTERNAL_H_
|
||||
|
||||
#ifdef HAVE_CONFIG_H
|
||||
#include "config.h"
|
||||
#endif
|
||||
|
||||
#include <string>
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
#ifdef HAVE_SYS_MMAN_H
|
||||
#include <sys/mman.h>
|
||||
#endif
|
||||
|
||||
#include "snappy/snappy-stubs-public.h"
|
||||
|
||||
#if defined(__x86_64__)
|
||||
|
||||
// Enable 64-bit optimized versions of some routines.
|
||||
#define ARCH_K8 1
|
||||
|
||||
#endif
|
||||
|
||||
// Needed by OS X, among others.
|
||||
#ifndef MAP_ANONYMOUS
|
||||
#define MAP_ANONYMOUS MAP_ANON
|
||||
#endif
|
||||
|
||||
// Pull in std::min, std::ostream, and the likes. This is safe because this
|
||||
// header file is never used from any public header files.
|
||||
using namespace std;
|
||||
|
||||
// The size of an array, if known at compile-time.
|
||||
// Will give unexpected results if used on a pointer.
|
||||
// We undefine it first, since some compilers already have a definition.
|
||||
#ifdef ARRAYSIZE
|
||||
#undef ARRAYSIZE
|
||||
#endif
|
||||
#define ARRAYSIZE(a) (sizeof(a) / sizeof(*(a)))
|
||||
|
||||
// Static prediction hints.
|
||||
#ifdef HAVE_BUILTIN_EXPECT
|
||||
#define PREDICT_FALSE(x) (__builtin_expect(x, 0))
|
||||
#define PREDICT_TRUE(x) (__builtin_expect(!!(x), 1))
|
||||
#else
|
||||
#define PREDICT_FALSE(x) x
|
||||
#define PREDICT_TRUE(x) x
|
||||
#endif
|
||||
|
||||
// This is only used for recomputing the tag byte table used during
|
||||
// decompression; for simplicity we just remove it from the open-source
|
||||
// version (anyone who wants to regenerate it can just do the call
|
||||
// themselves within main()).
|
||||
#define DEFINE_bool(flag_name, default_value, description) \
|
||||
bool FLAGS_ ## flag_name = default_value
|
||||
#define DECLARE_bool(flag_name) \
|
||||
extern bool FLAGS_ ## flag_name
|
||||
|
||||
namespace snappy {
|
||||
|
||||
static const uint32 kuint32max = static_cast<uint32>(0xFFFFFFFF);
|
||||
static const int64 kint64max = static_cast<int64>(0x7FFFFFFFFFFFFFFFLL);
|
||||
|
||||
// Potentially unaligned loads and stores.
|
||||
|
||||
// x86 and PowerPC can simply do these loads and stores native.
|
||||
|
||||
#if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
|
||||
|
||||
#define UNALIGNED_LOAD16(_p) (*reinterpret_cast<const uint16 *>(_p))
|
||||
#define UNALIGNED_LOAD32(_p) (*reinterpret_cast<const uint32 *>(_p))
|
||||
#define UNALIGNED_LOAD64(_p) (*reinterpret_cast<const uint64 *>(_p))
|
||||
|
||||
#define UNALIGNED_STORE16(_p, _val) (*reinterpret_cast<uint16 *>(_p) = (_val))
|
||||
#define UNALIGNED_STORE32(_p, _val) (*reinterpret_cast<uint32 *>(_p) = (_val))
|
||||
#define UNALIGNED_STORE64(_p, _val) (*reinterpret_cast<uint64 *>(_p) = (_val))
|
||||
|
||||
// ARMv7 and newer support native unaligned accesses, but only of 16-bit
|
||||
// and 32-bit values (not 64-bit); older versions either raise a fatal signal,
|
||||
// do an unaligned read and rotate the words around a bit, or do the reads very
|
||||
// slowly (trip through kernel mode). There's no simple #define that says just
|
||||
// “ARMv7 or higher”, so we have to filter away all ARMv5 and ARMv6
|
||||
// sub-architectures.
|
||||
//
|
||||
// This is a mess, but there's not much we can do about it.
|
||||
|
||||
#elif defined(__arm__) && \
|
||||
!defined(__ARM_ARCH_4__) && \
|
||||
!defined(__ARM_ARCH_4T__) && \
|
||||
!defined(__ARM_ARCH_5__) && \
|
||||
!defined(__ARM_ARCH_5T__) && \
|
||||
!defined(__ARM_ARCH_5TE__) && \
|
||||
!defined(__ARM_ARCH_5TEJ__) && \
|
||||
!defined(__ARM_ARCH_6__) && \
|
||||
!defined(__ARM_ARCH_6J__) && \
|
||||
!defined(__ARM_ARCH_6K__) && \
|
||||
!defined(__ARM_ARCH_6Z__) && \
|
||||
!defined(__ARM_ARCH_6ZK__) && \
|
||||
!defined(__ARM_ARCH_6T2__)
|
||||
|
||||
#define UNALIGNED_LOAD16(_p) (*reinterpret_cast<const uint16 *>(_p))
|
||||
#define UNALIGNED_LOAD32(_p) (*reinterpret_cast<const uint32 *>(_p))
|
||||
|
||||
#define UNALIGNED_STORE16(_p, _val) (*reinterpret_cast<uint16 *>(_p) = (_val))
|
||||
#define UNALIGNED_STORE32(_p, _val) (*reinterpret_cast<uint32 *>(_p) = (_val))
|
||||
|
||||
// TODO(user): NEON supports unaligned 64-bit loads and stores.
|
||||
// See if that would be more efficient on platforms supporting it,
|
||||
// at least for copies.
|
||||
|
||||
inline uint64 UNALIGNED_LOAD64(const void *p) {
|
||||
uint64 t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline void UNALIGNED_STORE64(void *p, uint64 v) {
|
||||
memcpy(p, &v, sizeof v);
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
// These functions are provided for architectures that don't support
|
||||
// unaligned loads and stores.
|
||||
|
||||
inline uint16 UNALIGNED_LOAD16(const void *p) {
|
||||
uint16 t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline uint32 UNALIGNED_LOAD32(const void *p) {
|
||||
uint32 t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline uint64 UNALIGNED_LOAD64(const void *p) {
|
||||
uint64 t;
|
||||
memcpy(&t, p, sizeof t);
|
||||
return t;
|
||||
}
|
||||
|
||||
inline void UNALIGNED_STORE16(void *p, uint16 v) {
|
||||
memcpy(p, &v, sizeof v);
|
||||
}
|
||||
|
||||
inline void UNALIGNED_STORE32(void *p, uint32 v) {
|
||||
memcpy(p, &v, sizeof v);
|
||||
}
|
||||
|
||||
inline void UNALIGNED_STORE64(void *p, uint64 v) {
|
||||
memcpy(p, &v, sizeof v);
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
// This can be more efficient than UNALIGNED_LOAD64 + UNALIGNED_STORE64
|
||||
// on some platforms, in particular ARM.
|
||||
inline void UnalignedCopy64(const void *src, void *dst) {
|
||||
if (sizeof(void *) == 8) {
|
||||
UNALIGNED_STORE64(dst, UNALIGNED_LOAD64(src));
|
||||
} else {
|
||||
const char *src_char = reinterpret_cast<const char *>(src);
|
||||
char *dst_char = reinterpret_cast<char *>(dst);
|
||||
|
||||
UNALIGNED_STORE32(dst_char, UNALIGNED_LOAD32(src_char));
|
||||
UNALIGNED_STORE32(dst_char + 4, UNALIGNED_LOAD32(src_char + 4));
|
||||
}
|
||||
}
|
||||
|
||||
// The following guarantees declaration of the byte swap functions.
|
||||
#ifdef WORDS_BIGENDIAN
|
||||
|
||||
#ifdef HAVE_SYS_BYTEORDER_H
|
||||
#include <sys/byteorder.h>
|
||||
#endif
|
||||
|
||||
#ifdef HAVE_SYS_ENDIAN_H
|
||||
#include <sys/endian.h>
|
||||
#endif
|
||||
|
||||
#ifdef _MSC_VER
|
||||
#include <stdlib.h>
|
||||
#define bswap_16(x) _byteswap_ushort(x)
|
||||
#define bswap_32(x) _byteswap_ulong(x)
|
||||
#define bswap_64(x) _byteswap_uint64(x)
|
||||
|
||||
#elif defined(__APPLE__)
|
||||
// Mac OS X / Darwin features
|
||||
#include <libkern/OSByteOrder.h>
|
||||
#define bswap_16(x) OSSwapInt16(x)
|
||||
#define bswap_32(x) OSSwapInt32(x)
|
||||
#define bswap_64(x) OSSwapInt64(x)
|
||||
|
||||
#elif defined(HAVE_BYTESWAP_H)
|
||||
#include <byteswap.h>
|
||||
|
||||
#elif defined(bswap32)
|
||||
// FreeBSD defines bswap{16,32,64} in <sys/endian.h> (already #included).
|
||||
#define bswap_16(x) bswap16(x)
|
||||
#define bswap_32(x) bswap32(x)
|
||||
#define bswap_64(x) bswap64(x)
|
||||
|
||||
#elif defined(BSWAP_64)
|
||||
// Solaris 10 defines BSWAP_{16,32,64} in <sys/byteorder.h> (already #included).
|
||||
#define bswap_16(x) BSWAP_16(x)
|
||||
#define bswap_32(x) BSWAP_32(x)
|
||||
#define bswap_64(x) BSWAP_64(x)
|
||||
|
||||
#else
|
||||
|
||||
inline uint16 bswap_16(uint16 x) {
|
||||
return (x << 8) | (x >> 8);
|
||||
}
|
||||
|
||||
inline uint32 bswap_32(uint32 x) {
|
||||
x = ((x & 0xff00ff00UL) >> 8) | ((x & 0x00ff00ffUL) << 8);
|
||||
return (x >> 16) | (x << 16);
|
||||
}
|
||||
|
||||
inline uint64 bswap_64(uint64 x) {
|
||||
x = ((x & 0xff00ff00ff00ff00ULL) >> 8) | ((x & 0x00ff00ff00ff00ffULL) << 8);
|
||||
x = ((x & 0xffff0000ffff0000ULL) >> 16) | ((x & 0x0000ffff0000ffffULL) << 16);
|
||||
return (x >> 32) | (x << 32);
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
#endif // WORDS_BIGENDIAN
|
||||
|
||||
// Convert to little-endian storage, opposite of network format.
|
||||
// Convert x from host to little endian: x = LittleEndian.FromHost(x);
|
||||
// convert x from little endian to host: x = LittleEndian.ToHost(x);
|
||||
//
|
||||
// Store values into unaligned memory converting to little endian order:
|
||||
// LittleEndian.Store16(p, x);
|
||||
//
|
||||
// Load unaligned values stored in little endian converting to host order:
|
||||
// x = LittleEndian.Load16(p);
|
||||
class LittleEndian {
|
||||
public:
|
||||
// Conversion functions.
|
||||
#ifdef WORDS_BIGENDIAN
|
||||
|
||||
static uint16 FromHost16(uint16 x) { return bswap_16(x); }
|
||||
static uint16 ToHost16(uint16 x) { return bswap_16(x); }
|
||||
|
||||
static uint32 FromHost32(uint32 x) { return bswap_32(x); }
|
||||
static uint32 ToHost32(uint32 x) { return bswap_32(x); }
|
||||
|
||||
static bool IsLittleEndian() { return false; }
|
||||
|
||||
#else // !defined(WORDS_BIGENDIAN)
|
||||
|
||||
static uint16 FromHost16(uint16 x) { return x; }
|
||||
static uint16 ToHost16(uint16 x) { return x; }
|
||||
|
||||
static uint32 FromHost32(uint32 x) { return x; }
|
||||
static uint32 ToHost32(uint32 x) { return x; }
|
||||
|
||||
static bool IsLittleEndian() { return true; }
|
||||
|
||||
#endif // !defined(WORDS_BIGENDIAN)
|
||||
|
||||
// Functions to do unaligned loads and stores in little-endian order.
|
||||
static uint16 Load16(const void *p) {
|
||||
return ToHost16(UNALIGNED_LOAD16(p));
|
||||
}
|
||||
|
||||
static void Store16(void *p, uint16 v) {
|
||||
UNALIGNED_STORE16(p, FromHost16(v));
|
||||
}
|
||||
|
||||
static uint32 Load32(const void *p) {
|
||||
return ToHost32(UNALIGNED_LOAD32(p));
|
||||
}
|
||||
|
||||
static void Store32(void *p, uint32 v) {
|
||||
UNALIGNED_STORE32(p, FromHost32(v));
|
||||
}
|
||||
};
|
||||
|
||||
// Some bit-manipulation functions.
|
||||
class Bits {
|
||||
public:
|
||||
// Return floor(log2(n)) for positive integer n. Returns -1 iff n == 0.
|
||||
static int Log2Floor(uint32 n);
|
||||
|
||||
// Return the first set least / most significant bit, 0-indexed. Returns an
|
||||
// undefined value if n == 0. FindLSBSetNonZero() is similar to ffs() except
|
||||
// that it's 0-indexed.
|
||||
static int FindLSBSetNonZero(uint32 n);
|
||||
static int FindLSBSetNonZero64(uint64 n);
|
||||
|
||||
private:
|
||||
DISALLOW_COPY_AND_ASSIGN(Bits);
|
||||
};
|
||||
|
||||
#ifdef HAVE_BUILTIN_CTZ
|
||||
|
||||
inline int Bits::Log2Floor(uint32 n) {
|
||||
return n == 0 ? -1 : 31 ^ __builtin_clz(n);
|
||||
}
|
||||
|
||||
inline int Bits::FindLSBSetNonZero(uint32 n) {
|
||||
return __builtin_ctz(n);
|
||||
}
|
||||
|
||||
inline int Bits::FindLSBSetNonZero64(uint64 n) {
|
||||
return __builtin_ctzll(n);
|
||||
}
|
||||
|
||||
#else // Portable versions.
|
||||
|
||||
inline int Bits::Log2Floor(uint32 n) {
|
||||
if (n == 0)
|
||||
return -1;
|
||||
int log = 0;
|
||||
uint32 value = n;
|
||||
for (int i = 4; i >= 0; --i) {
|
||||
int shift = (1 << i);
|
||||
uint32 x = value >> shift;
|
||||
if (x != 0) {
|
||||
value = x;
|
||||
log += shift;
|
||||
}
|
||||
}
|
||||
assert(value == 1);
|
||||
return log;
|
||||
}
|
||||
|
||||
inline int Bits::FindLSBSetNonZero(uint32 n) {
|
||||
int rc = 31;
|
||||
for (int i = 4, shift = 1 << 4; i >= 0; --i) {
|
||||
const uint32 x = n << shift;
|
||||
if (x != 0) {
|
||||
n = x;
|
||||
rc -= shift;
|
||||
}
|
||||
shift >>= 1;
|
||||
}
|
||||
return rc;
|
||||
}
|
||||
|
||||
// FindLSBSetNonZero64() is defined in terms of FindLSBSetNonZero().
|
||||
inline int Bits::FindLSBSetNonZero64(uint64 n) {
|
||||
const uint32 bottombits = static_cast<uint32>(n);
|
||||
if (bottombits == 0) {
|
||||
// Bottom bits are zero, so scan in top bits
|
||||
return 32 + FindLSBSetNonZero(static_cast<uint32>(n >> 32));
|
||||
} else {
|
||||
return FindLSBSetNonZero(bottombits);
|
||||
}
|
||||
}
|
||||
|
||||
#endif // End portable versions.
|
||||
|
||||
// Variable-length integer encoding.
|
||||
class Varint {
|
||||
public:
|
||||
// Maximum lengths of varint encoding of uint32.
|
||||
static const int kMax32 = 5;
|
||||
|
||||
// Attempts to parse a varint32 from a prefix of the bytes in [ptr,limit-1].
|
||||
// Never reads a character at or beyond limit. If a valid/terminated varint32
|
||||
// was found in the range, stores it in *OUTPUT and returns a pointer just
|
||||
// past the last byte of the varint32. Else returns NULL. On success,
|
||||
// "result <= limit".
|
||||
static const char* Parse32WithLimit(const char* ptr, const char* limit,
|
||||
uint32* OUTPUT);
|
||||
|
||||
// REQUIRES "ptr" points to a buffer of length sufficient to hold "v".
|
||||
// EFFECTS Encodes "v" into "ptr" and returns a pointer to the
|
||||
// byte just past the last encoded byte.
|
||||
static char* Encode32(char* ptr, uint32 v);
|
||||
|
||||
// EFFECTS Appends the varint representation of "value" to "*s".
|
||||
static void Append32(string* s, uint32 value);
|
||||
};
|
||||
|
||||
inline const char* Varint::Parse32WithLimit(const char* p,
|
||||
const char* l,
|
||||
uint32* OUTPUT) {
|
||||
const unsigned char* ptr = reinterpret_cast<const unsigned char*>(p);
|
||||
const unsigned char* limit = reinterpret_cast<const unsigned char*>(l);
|
||||
uint32 b, result;
|
||||
if (ptr >= limit) return NULL;
|
||||
b = *(ptr++); result = b & 127; if (b < 128) goto done;
|
||||
if (ptr >= limit) return NULL;
|
||||
b = *(ptr++); result |= (b & 127) << 7; if (b < 128) goto done;
|
||||
if (ptr >= limit) return NULL;
|
||||
b = *(ptr++); result |= (b & 127) << 14; if (b < 128) goto done;
|
||||
if (ptr >= limit) return NULL;
|
||||
b = *(ptr++); result |= (b & 127) << 21; if (b < 128) goto done;
|
||||
if (ptr >= limit) return NULL;
|
||||
b = *(ptr++); result |= (b & 127) << 28; if (b < 16) goto done;
|
||||
return NULL; // Value is too long to be a varint32
|
||||
done:
|
||||
*OUTPUT = result;
|
||||
return reinterpret_cast<const char*>(ptr);
|
||||
}
|
||||
|
||||
inline char* Varint::Encode32(char* sptr, uint32 v) {
|
||||
// Operate on characters as unsigneds
|
||||
unsigned char* ptr = reinterpret_cast<unsigned char*>(sptr);
|
||||
static const int B = 128;
|
||||
if (v < (1<<7)) {
|
||||
*(ptr++) = v;
|
||||
} else if (v < (1<<14)) {
|
||||
*(ptr++) = v | B;
|
||||
*(ptr++) = v>>7;
|
||||
} else if (v < (1<<21)) {
|
||||
*(ptr++) = v | B;
|
||||
*(ptr++) = (v>>7) | B;
|
||||
*(ptr++) = v>>14;
|
||||
} else if (v < (1<<28)) {
|
||||
*(ptr++) = v | B;
|
||||
*(ptr++) = (v>>7) | B;
|
||||
*(ptr++) = (v>>14) | B;
|
||||
*(ptr++) = v>>21;
|
||||
} else {
|
||||
*(ptr++) = v | B;
|
||||
*(ptr++) = (v>>7) | B;
|
||||
*(ptr++) = (v>>14) | B;
|
||||
*(ptr++) = (v>>21) | B;
|
||||
*(ptr++) = v>>28;
|
||||
}
|
||||
return reinterpret_cast<char*>(ptr);
|
||||
}
|
||||
|
||||
// If you know the internal layout of the std::string in use, you can
|
||||
// replace this function with one that resizes the string without
|
||||
// filling the new space with zeros (if applicable) --
|
||||
// it will be non-portable but faster.
|
||||
inline void STLStringResizeUninitialized(string* s, size_t new_size) {
|
||||
s->resize(new_size);
|
||||
}
|
||||
|
||||
// Return a mutable char* pointing to a string's internal buffer,
|
||||
// which may not be null-terminated. Writing through this pointer will
|
||||
// modify the string.
|
||||
//
|
||||
// string_as_array(&str)[i] is valid for 0 <= i < str.size() until the
|
||||
// next call to a string method that invalidates iterators.
|
||||
//
|
||||
// As of 2006-04, there is no standard-blessed way of getting a
|
||||
// mutable reference to a string's internal buffer. However, issue 530
|
||||
// (http://www.open-std.org/JTC1/SC22/WG21/docs/lwg-defects.html#530)
|
||||
// proposes this as the method. It will officially be part of the standard
|
||||
// for C++0x. This should already work on all current implementations.
|
||||
inline char* string_as_array(string* str) {
|
||||
return str->empty() ? NULL : &*str->begin();
|
||||
}
|
||||
|
||||
} // namespace snappy
|
||||
|
||||
#endif // THIRD_PARTY_SNAPPY_OPENSOURCE_SNAPPY_STUBS_INTERNAL_H_
|
1553
external/snappy-1.1.3/snappy.cc
vendored
Normal file
1553
external/snappy-1.1.3/snappy.cc
vendored
Normal file
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue
Block a user