<!-- Converted by db4-upgrade version 1.1 -->
<chapter xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="using_raster_dataman">
  <title>Raster Data Management, Queries, and Applications</title>
  <section xml:id="RT_Loading_Rasters">
    <title>Loading and Creating Rasters</title>
    <para>For most use cases, you will create PostGIS rasters by loading existing raster files using the packaged <varname>raster2pgsql</varname> raster loader.</para>

    <section xml:id="RT_Raster_Loader">
  	<title>Using raster2pgsql to load rasters</title>

    <para>The <varname>raster2pgsql</varname> is a raster loader executable that loads GDAL supported raster formats into SQL suitable for loading into a PostGIS raster table. It is capable of loading folders of raster files as well as creating overviews of rasters. </para>

    <para>Since the raster2pgsql is compiled as part of PostGIS most often (unless you compile your own GDAL library), the raster types supported by the executable will be the same as those compiled in the GDAL dependency library.  To get a list of raster types your particular <varname>raster2pgsql</varname> supports use the <varname>-G</varname> switch.</para>

    <note>
      <para>When creating overviews of a specific factor from a set of rasters that are aligned, it is possible for the overviews to not align.  Visit <link xlink:href="http://trac.osgeo.org/postgis/ticket/1764">http://trac.osgeo.org/postgis/ticket/1764</link> for an example where the overviews do not align.</para>
    </note>

    <section>
      <title>Example Usage</title>

    <para>An example session using the loader to create an input file and uploading it chunked in 100x100 tiles might look like this:</para>

    <para>The loader assigns SRID 4326, creates an index and standard
    constraints, analyzes the loaded table, records source filenames, and
    stores the input rasters in 100 by 100 pixel tiles.</para>
    <programlisting language="bash"><![CDATA[
raster2pgsql -s 4326 -I -C -M -F -t 100x100 *.tif public.demelevation > elev.sql
psql -d gisdb -f elev.sql
]]></programlisting>

    <note>
      <para>If you do not specify the schema as part of the target table name, the table will be created in the default schema of the database or user you are connecting with.</para>
    </note>

    <para>A conversion and upload can be done all in one step using UNIX pipes:</para>

    <programlisting language="bash">raster2pgsql -s 4326 -I -C -M -F -t 100x100 \
  *.tif public.demelevation |
  psql -d gisdb</programlisting>

    <para>Load rasters Massachusetts state plane meters aerial tiles
  into a schema called <varname>aerial</varname> and create a full view, 2 and 4 level overview tables, use copy mode for inserting (no intermediary file just straight to db), and -e don't force everything in a transaction (good if you want to see data in tables right away without waiting).  Break up the rasters into 128x128 pixel tiles and apply raster constraints. Use copy mode instead of table insert. (-F) Include a field called filename to hold the name of the file the tiles were cut from.</para>
    <programlisting language="bash">raster2pgsql -I -C -e -Y -F -s 26986 -t 128x128 -l 2,4 \
  bostonaerials2008/*.jpg aerials.boston |
  psql -U postgres -d gisdb -h localhost -p 5432</programlisting>

    <para>List the raster formats supported by this build:</para>
    <programlisting language="bash">raster2pgsql -G</programlisting>

<para>The <option>-G</option> command outputs a list something like:</para>
<screen role="text-primary">
Available GDAL raster formats:
  Virtual Raster
  GeoTIFF
  National Imagery Transmission Format
  Raster Product Format TOC format
  ECRG TOC format
  Erdas Imagine Images (.img)
  CEOS SAR Image
  CEOS Image
  ...
  Arc/Info Export E00 GRID
  ZMap Plus Grid
  NOAA NGS Geoid Height Grids</screen>

    </section>

    <section>
      <title>raster2pgsql options</title>

    <para>
      Long options that take values also accept the <option>--option=value</option> spelling.
    </para>

    <variablelist>
         <varlistentry>
          <term><option>-?, --help</option></term>
          <listitem>
            <para>
              Display help screen.  Help will also display if you don't pass in any arguments.
            </para>
          </listitem>
        </varlistentry>

       <varlistentry>
          <term><option>-G, --gdal-formats</option></term>
          <listitem>
            <para>
             Print the supported raster formats.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>(c|a|d|p) These are mutually exclusive options:</term>
          <listitem>
            <para>
              <variablelist>
                <varlistentry>
                  <term><option>-c</option></term>
                  <listitem>
                    <para>
                      Create new table and populate it with raster(s), <emphasis>this is the default mode</emphasis>
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-a</option></term>
                  <listitem>
                    <para>
                      Append raster(s) to an existing table.
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-d</option></term>
                  <listitem>
                    <para>
                      Drop table, create new one and populate it with raster(s)
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-p</option></term>
                  <listitem>
                    <para>
                      Prepare mode, only create the table.
                    </para>
                  </listitem>
                </varlistentry>
              </variablelist>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>--if-not-exists</option></term>
          <listitem>
            <para>
              Use <literal>IF NOT EXISTS</literal> for table creation in
              <option>-c</option> and <option>-p</option> modes. When
              <option>-I</option> or <option>--create-index</option> is also
              specified, use <literal>IF NOT EXISTS</literal> for index
              creation too. This option cannot be used with <option>-d</option>.
              Append mode requires an explicit creation action.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Atomic loader actions</term>
          <listitem>
            <para>
              The short modes are presets over explicit actions. The equivalent
              long options are <option>--drop-table</option>,
              <option>--create-table</option>, <option>--load-data</option>,
              <option>--create-index</option>,
              <option>--add-constraints</option>, <option>--vacuum</option>,
              <option>--analyze</option>, and <option>--no-transaction</option>.
              Add <option>--if-not-exists</option>
              to make active creation actions use <literal>IF NOT EXISTS</literal>.
              <option>--drop-table</option> emits
              <literal>DROP TABLE IF EXISTS</literal> before the selected
              actions. With no mode specified, the default create/load actions
              still apply.
            </para>
          </listitem>
        </varlistentry>

		<varlistentry>
			<term>Raster processing: Applying constraints for proper registering in raster catalogs</term>
			<listitem>
				<para>
					<variablelist>
						<varlistentry>
							<term><option>-C, --add-constraints</option></term>
							<listitem>
								<para>
									Apply raster constraints -- srid, pixelsize etc. to ensure raster is properly registered in <varname>raster_columns</varname> view.
								</para>
							</listitem>
						</varlistentry>
					   <varlistentry>
							<term><option>-x, --no-extent</option></term>
							<listitem>
								<para>
									Disable setting the max extent constraint.  Only applied if -C or --add-constraints is also used.
								</para>
							</listitem>
						</varlistentry>
						<varlistentry>
							<term><option>-r, --regular-blocking</option></term>
							<listitem>
								<para>
									Set the constraints (spatially unique and coverage tile) for regular blocking.  Only applied if -C or --add-constraints is also used.
								</para>
							</listitem>
						</varlistentry>
					</variablelist>
				</para>
			</listitem>
		</varlistentry>

       <varlistentry>
          <term>Raster processing: Optional parameters used to manipulate input raster dataset</term>
          <listitem>
            <para>
            <variablelist>
                <varlistentry>
                    <term><option>-s, --srid [&lt;FROM_SRID&gt;:]&lt;SRID&gt;</option></term>
                    <listitem>
                        <para>
                            Assign the output raster to the specified SRID.  When FROM_SRID is also provided, raster2pgsql marks the input raster with FROM_SRID and emits SQL that reprojects the raster to the target SRID.  If the source SRID is omitted or is zero, the raster metadata will be checked to determine an appropriate source SRID.  Reprojection cannot be used with copy mode.
                        </para>
                    </listitem>
                </varlistentry>

                <varlistentry>
                    <term><option>-b, --band BAND</option></term>
                    <listitem>
                        <para>
                           Index (1-based) of band to extract from raster.  For more than one band index, separate with comma (,).  If unspecified,
                           all bands of raster will be extracted.
                        </para>
                    </listitem>
                </varlistentry>

                <varlistentry>
                    <term><option>-t, --tile-size TILE_SIZE</option></term>
                    <listitem>
                        <para>
                            Cut raster into tiles to be inserted one per table row.  <varname>TILE_SIZE</varname> is expressed as WIDTHxHEIGHT or set to the value "auto" to allow the loader to compute an appropriate tile size using the first raster and applied to all rasters.
                        </para>
                    </listitem>
                </varlistentry>

                <varlistentry>
                    <term><option>-P, --pad</option></term>
                    <listitem>
                        <para>
                            Pad right-most and bottom-most tiles to guarantee that all tiles
                            have the same width and height.
                        </para>
                    </listitem>
                </varlistentry>



                <varlistentry>
                    <term><option>-R, --register</option></term>
                    <listitem>
                        <para>Register the raster as a filesystem (out-db) raster.</para>
                        <para>Only the metadata of the raster and path location to the raster is stored in the database (not the pixels).</para>
                    </listitem>
                </varlistentry>

                <varlistentry>
                    <term><option>-l, --overview-factor OVERVIEW_FACTOR</option></term>
										<listitem><para>Create overview of the raster.  For more than
     one factor, separate with comma(,).  Overview table name follows
			 the pattern o_<varname>overview factor</varname>_<varname>table</varname>, where <varname>overview factor</varname> is a placeholder for numerical overview factor and <varname>table</varname> is replaced with the base table name. If a source raster already has an overview with matching dimensions, that overview is reused. Created overview is
     stored in the database and is not affected by -R. Note that your generated sql file will contain both the main table and overview tables.</para>
                    </listitem>
                </varlistentry>

								<varlistentry>
									<term><option>-N, --nodata NODATA</option></term>
									<listitem>
										<para>
											NODATA value to use on bands without a NODATA value.
										</para>
									</listitem>
								</varlistentry>

              </variablelist>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Optional parameters used to manipulate database objects</term>
          <listitem>
            <para>
              <variablelist>
                <varlistentry>
                  <term><option>-f, --raster-column COLUMN</option></term>
                  <listitem>
                    <para>Specify name of destination raster column, default is 'rast'
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-F, --filename</option></term>
                  <listitem>
                    <para>Add a column with the name of the file</para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-n, --filename-column COLUMN</option></term>
                  <listitem>
                    <para>Specify the name of the filename column. Implies -F.</para>
                  </listitem>
                </varlistentry>

               <varlistentry>
                  <term><option>-q, --quote</option></term>
                  <listitem>
                    <para>Wrap PostgreSQL identifiers in quotes.</para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-I</option></term>
                  <listitem>
                    <para>
                      Alias for <option>--create-index</option>.
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>--create-index</option></term>
                  <listitem>
                    <para>
                      Create a GiST index on the raster column at the end of this
                      raster2pgsql run.  With repeated <option>-a</option> append
                      runs, create the index on the final run or after loading;
                      add <option>--if-not-exists</option> to make reruns tolerate
                      an existing index.
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-M</option></term>
                  <listitem>
                    <para>
                      Vacuum analyze the raster table.
                    </para>
                  </listitem>
                </varlistentry>


                <varlistentry>
                  <term><option>-k, --skip-nodata-check</option></term>
                  <listitem>
                    <para>
                        Keeps empty tiles and skips NODATA value checks for each raster band.
                        Note you save time in checking, but could end up with far more junk rows in your database and those junk rows are not marked as empty tiles.
                    </para>
                  </listitem>
                </varlistentry>


                <varlistentry>
                  <term><option>-T, --tablespace tablespace</option></term>
                  <listitem>
                    <para>
                      Specify the tablespace for the new table.
     Note that indices (including the primary key) will still use
     the default tablespace unless the -X flag is also used.
                    </para>
                  </listitem>
                </varlistentry>

                <varlistentry>
                  <term><option>-X, --index-tablespace tablespace</option></term>
                  <listitem>
                    <para>
                      Specify the tablespace for the table's new index.
     This applies to the primary key and the spatial index if the
     -I flag is used.
                    </para>
                  </listitem>
                 </varlistentry>

                <varlistentry>
                  <term><option>-Y, --copy [MAX_ROWS_PER_COPY]</option></term>
                  <listitem>
                    <para>
                      Use copy statements instead of insert statements.  Optionally specify <varname>max_rows_per_copy</varname>;
                      default 50 when not specified.</para>
                  </listitem>
                </varlistentry>

              </variablelist>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
            <term><option>-e, --no-transaction</option></term>
            <listitem><para>Execute each statement individually, do not use a transaction.</para></listitem>
        </varlistentry>

        <varlistentry>
            <term><option>-E, --endian ENDIAN</option></term>
            <listitem><para>Control endianness of generated binary output of raster; specify 0 for XDR and 1 for NDR (default); only NDR output is supported now</para></listitem>
        </varlistentry>

        <varlistentry>
            <term><option>-V, --wkb-version version</option></term>
            <listitem><para>Specify version of output format.  Default  is 0.  Only 0 is supported at this time.</para></listitem>
        </varlistentry>
    </variablelist>

    </section>




    </section>
    <section xml:id="RT_Creating_Rasters">
	<title>Creating rasters using PostGIS raster functions</title>
	<para>On many occasions, you'll want to create rasters and raster tables right in the database.  There are a plethora of functions to do that.  The general steps to follow.</para>
	<orderedlist>
		<listitem><para>Create a table with a raster column to hold the new raster records which can be accomplished with:</para>
			<programlisting language="sql">CREATE TABLE myrasters(rid serial primary key, rast raster);</programlisting>
		</listitem>
		<listitem>
			<para>There are many functions to help with that goal.  If you are creating rasters not as a derivative of other rasters, you will want to start with:
				<xref linkend="RT_ST_MakeEmptyRaster"/>, followed by <xref linkend="RT_ST_AddBand"/></para>
			<para>You can also create rasters from geometries.  To achieve that you'll want to use <xref linkend="RT_ST_AsRaster"/> perhaps accompanied with
			other functions such as <xref linkend="RT_ST_Union"/> or <xref linkend="RT_ST_MapAlgebraFct2"/> or any of the family of other map algebra functions.</para>
			<para>There are even many more options for creating new raster tables from existing tables.  For example you can create a raster table in a different projection from an existing one using <xref linkend="RT_ST_Transform"/> </para>
		</listitem>
		<listitem><para>Once you are done populating your table initially, you'll want to create a spatial index on the raster column with something like:</para>
			<programlisting language="sql">CREATE INDEX myrasters_rast_st_convexhull_idx ON myrasters USING gist(ST_ConvexHull(rast));</programlisting>
			<para>Note the use of <xref linkend="RT_ST_ConvexHull"/> since most raster operators are based on the convex hull of the rasters.</para>
			<note><para>Pre-2.0 versions of PostGIS raster were based on the envelop rather than the convex hull.  For the spatial indexes to work properly you'll need to drop those and replace with convex hull based index.</para></note></listitem>
		<listitem><para>Apply raster constraints using <xref linkend="RT_AddRasterConstraints"/></para></listitem>
	</orderedlist>
    </section>

    <section xml:id="RT_Cloud_Rasters">
    <title>Using "out db" cloud rasters</title>
    <para>
        The <varname>raster2pgsql</varname> tool uses GDAL to access raster data, and can take advantage of a key GDAL feature: the ability to read
        from rasters that are <link xlink:href="https://gdal.org/user/virtual_file_systems.html#network-based-file-systems">stored remotely</link> in cloud "object stores" (e.g. AWS S3, Google Cloud Storage).
    </para>
    <para>
        Efficient use of cloud stored rasters requires the use of a "cloud optimized" format. The most well-known and widely used is the "<link xlink:href="https://gdal.org/drivers/raster/cog.html">cloud optimized GeoTIFF</link>" format. Using a non-cloud format, like a JPEG, or an un-tiled TIFF will result in very poor performance, as the system will have to download the entire raster each time it needs to access a subset.
    </para>
    <para>
        First, load your raster into the cloud storage of your choice. Once it is loaded, you will have a URI to access it with, either an "http" URI, or sometimes a URI specific to the service. (e.g., "s3://bucket/object"). To access non-public buckets, you will need to supply GDAL config options to authenticate your connection. Note that this command is <emphasis>reading</emphasis> from the cloud raster and <emphasis>writing</emphasis> to the database.
    </para>
    <programlisting language="bash">AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxx \
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
raster2pgsql \
  -s 990000 \
  -t 256x256 \
  -I \
  -R \
  /vsis3/your.bucket.com/your_file.tif \
  your_table \
  | psql your_db</programlisting>
    <para>
        Once the table is loaded, you need to give the database permission to read from remote rasters, by setting two permissions, <xref linkend="postgis_enable_outdb_rasters"/> and <xref linkend="postgis_gdal_enabled_drivers"/>.
    </para>
    <programlisting language="sql">SET postgis.enable_outdb_rasters = true;
SET postgis.gdal_enabled_drivers TO 'ENABLE_ALL';
    </programlisting>
    <para>
        To make the changes sticky, set them directly on your database. You will need to re-connect to experience the new settings.
    </para>
    <programlisting language="sql">ALTER DATABASE your_db SET postgis.enable_outdb_rasters = true;
ALTER DATABASE your_db SET postgis.gdal_enabled_drivers TO 'ENABLE_ALL';
    </programlisting>
    <para>
        For non-public rasters, you may have to provide access keys to read from the cloud rasters. The same keys you used to write the <varname>raster2pgsql</varname> call can be set for use inside the database, with the <xref linkend="postgis_gdal_vsi_options"/> configuration. Note that multiple options can be set by space-separating the <varname>key=value</varname> pairs.
    </para>
    <programlisting language="sql">SET postgis.gdal_vsi_options = 'AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxx
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';</programlisting>
    <para>
        Once you have the data loaded and permissions set you can interact with the raster table like any other raster table, using the same functions. The database will handle all the mechanics of connecting to the cloud data when it needs to read pixel data.
    </para>
    </section>
  </section>
  <section xml:id="RT_Raster_Coverages">
		<title>Raster Coverages and Raster/Vector Analysis</title>
		<para>A coverage is a way to interpret spatial data as a mapping from a
		<emphasis>domain</emphasis> of locations to a <emphasis>range</emphasis> of
		values. Evaluating a coverage means asking for the value at a location;
		an inverse query asks for the locations whose values match a condition.
		PostGIS does not expose a generic ISO 19123 coverage SQL type, but geometry
		tables and raster values provide the building blocks for both directions.</para>
		<para>A coverage can be continuous, with values calculated or interpolated
		for arbitrary locations, or discrete, with values supplied by stored spatial
		objects or grid cells. The coverage model describes how locations map to
		values, independently of whether those values come from an equation, a table,
		or one or more raster values. A time column or function argument can add a
		temporal dimension without changing that distinction.</para>
		<para>In a vector coverage, a geometry column supplies the domain and other
		columns supply attributes in the range. A vector range can contain text,
		numbers, or another spatial value, provided its meaning and type are
		consistent wherever the coverage is defined. In a raster, each cell location
		is part of the domain and the band values at that cell form the range. Other
		columns in the same table row describe the raster value or tile as a whole;
		they do not attach separate values to individual cells.</para>
		<para>A logical raster coverage may be stored as one raster value or as many
		tiles in a table. Tiles that belong to one coverage should have compatible
		SRIDs, scales, alignment, band counts, pixel types, and NODATA conventions.
		The constraints exposed by <xref linkend="RT_Raster_Columns"/> record and
		validate those properties.</para>
		<formalpara>
		  <title>Raster table arrangements.</title>
		  <para>A raster column can represent an image warehouse, a tiled coverage,
		  or a collection of rasterized objects. These arrangements have different
		  assumptions about tile size, alignment, gaps, and overlaps. PostGIS does
		  not infer the intended arrangement from the column type alone.</para>
		</formalpara>
		<informaltable>
		  <tgroup cols="3">
		    <thead>
		      <row><entry>Arrangement</entry><entry>Row relationship</entry><entry>Typical use</entry></row>
		    </thead>
		    <tbody>
		      <row><entry>Image warehouse</entry><entry>Independent rasters; sizes,
		      alignment, and extents may differ or overlap</entry><entry>Cataloging
		      unrelated source images</entry></row>
		      <row><entry>Irregular tiled coverage</entry><entry>One logical coverage
		      with gaps allowed, variable tile sizes, and normally no overlap</entry><entry>Incremental or
		      source-driven mosaics</entry></row>
		      <row><entry>Regular tiled coverage</entry><entry>One alignment and tile
		      size with no overlap; gaps may remain at the boundary or inside the coverage</entry><entry>
		      Indexed analysis over a large coverage</entry></row>
		      <row><entry>Rectangular regular coverage</entry><entry>Equal aligned
		      tiles fill a rectangular extent without gaps or overlaps</entry><entry>Predictable
		      block processing and overview generation</entry></row>
		      <row><entry>Separate tiled images</entry><entry>Each image is internally
		      regular but stored in its own table or partition</entry><entry>Keeping
		      source-image identity and lifecycle separate</entry></row>
		      <row><entry>Raster-object coverage</entry><entry>Each row is a rasterized
		      feature; extents and sizes may differ and overlap</entry><entry>Object
		      models where cells replace or supplement vector geometry</entry></row>
		    </tbody>
		  </tgroup>
		</informaltable>
		<para>Use <xref linkend="RT_AddRasterConstraints"/> to record common
		SRID, scale, alignment, band, extent, and regular-blocking properties, and
		use spatial indexes and application constraints for relationships that the
		catalog cannot express.</para>
		<para>Queries against a tiled coverage normally have two stages:</para>
		<orderedlist>
		  <listitem><para>Use the raster spatial index and
		  <xref linkend="RT_ST_Intersects"/> to select candidate tiles.</para></listitem>
		  <listitem><para>Read, clip, aggregate, or vectorize values inside those
		  tiles with functions such as <xref linkend="RT_ST_Value"/>,
		  <xref linkend="RT_ST_Clip"/>,
		  <xref linkend="RT_ST_SummaryStatsAgg"/>, and
		  <xref linkend="RT_ST_DumpAsPolygons"/>.</para></listitem>
		</orderedlist>
		<para><xref linkend="RT_ST_DumpAsPolygons"/> returns
		<xref linkend="geomval"/> rows that pair geometries with cell values. This
		allows ordinary geometry overlay and filtering to answer inverse coverage
		queries, but polygonizing many cells is usually more expensive than keeping
		the operation in raster space.</para>
		<para>Choose boundary semantics deliberately. Clipping and aggregating raster
		pixels is usually the shorter and faster path. If a partial boundary cell must
		contribute in proportion to its covered area, intersect the geometry with the
		<xref linkend="geomval"/> rows and calculate an area-weighted result in an
		appropriate projected coordinate system. That exact partial-cell method costs
		more because it vectorizes the selected cells.</para>
		<para>Tile size affects both sides of the indexed query: smaller tiles reduce
		the amount of raster data inspected for each candidate row, while increasing
		row and index overhead. Benchmark representative queries instead of treating
		one tile size as a universal default.</para>
		<para>For a current end-to-end raster/vector pattern, load and tile data with
		<xref linkend="RT_Raster_Loader"/>, then see the building example in
		<xref linkend="RT_ST_SummaryStats"/>. It selects intersecting tiles, clips
		them to each polygon, and combines per-tile statistics into one result per
		feature. For point observations such as animal locations, first buffer the
		points in an appropriate projected coordinate system, then apply the same
		tile selection, clipping, and aggregation pattern to an elevation or other
		continuous raster coverage.</para>
  </section>
  <section xml:id="RT_Raster_Catalog">
		<title>Raster Catalogs</title>
		<para>There are two raster catalog views that come packaged with PostGIS.  Both views utilize information embedded in the constraints of the raster tables.  As a result
		the catalog views are always consistent with the raster data in the tables since the constraints are enforced. </para>
		 <orderedlist>
          <listitem>
            <para><varname>raster_columns</varname> this view catalogs all the raster table columns in your database.</para>
          </listitem>
          <listitem>
            <para><varname>raster_overviews</varname> this view catalogs all the raster table columns in your database that serve as overviews for a finer grained table.  Tables of this type are generated when you use the <varname>-l</varname> switch during load.</para>
          </listitem>
        </orderedlist>
        <section xml:id="RT_Raster_Columns">
	<title>Raster Columns Catalog</title>
	<para>The <varname>raster_columns</varname> is a catalog of all raster table columns in your database that are of type raster.  It is a view utilizing the constraints on the tables
	so the information is always consistent even if you restore one raster table from a backup of another database.  The following columns exist in the <varname>raster_columns</varname> catalog.</para>
	<para>If you created your tables not with the loader or forgot to specify the <varname>-C</varname> flag during load, you can enforce the constraints after the
	fact using <xref linkend="RT_AddRasterConstraints"/> so that the <varname>raster_columns</varname> catalog registers the common information about your raster tiles.</para>

			 <itemizedlist>
				<listitem>
					<para><varname>r_table_catalog</varname> The database the table is in.  This will always read the current database.</para>
				</listitem>
				<listitem>
					<para><varname>r_table_schema</varname> The database schema the raster table belongs to.</para>
				</listitem>
				<listitem>
					<para><varname>r_table_name</varname> raster table</para>
				</listitem>
				<listitem>
					<para><varname>r_raster_column</varname> the column in the <varname>r_table_name</varname> table that is of type raster.  There is nothing in PostGIS preventing you from having multiple raster columns per table so its possible to have a raster table listed multiple times with a different raster column for each.</para>
				</listitem>
				<listitem>
					<para><varname>srid</varname> The spatial reference identifier of the raster.  Should be an entry in the <xref linkend="spatial_ref_sys"/>.</para>
				</listitem>
				<listitem>
					<para><varname>scale_x</varname> The scaling between geometric spatial coordinates and pixel.  This is only available if all tiles in the raster column have the same <varname>scale_x</varname> and this constraint is applied. Refer to <xref linkend="RT_ST_ScaleX"/> for more details.</para>
				</listitem>
				<listitem>
					<para><varname>scale_y</varname> The scaling between geometric spatial coordinates and pixel.  This is only available if all tiles in the raster column have the same <varname>scale_y</varname> and the <varname>scale_y</varname> constraint is applied. Refer to <xref linkend="RT_ST_ScaleY"/> for more details.</para>
				</listitem>
				<listitem>
					<para><varname>blocksize_x</varname> The width (number of pixels across) of each raster tile . Refer to <xref linkend="RT_ST_Width"/> for more details.</para>
				</listitem>
				<listitem>
					<para><varname>blocksize_y</varname> The width (number of pixels down) of each raster tile . Refer to <xref linkend="RT_ST_Height"/> for more details.</para>
				</listitem>
				<listitem>
					<para><varname>same_alignment</varname> A boolean that is true if all the raster tiles have the same alignment . Refer to <xref linkend="RT_ST_SameAlignment"/> for more details.</para>
				</listitem>
				<listitem>
					<para><varname>regular_blocking</varname> If the raster column has the spatially unique and coverage tile constraints, the value with be TRUE.  Otherwise, it will be FALSE.</para>
				</listitem>
				<listitem>
					<para><varname>num_bands</varname> The number of bands in each tile of your raster set.  This is the same information as what is provided by <xref linkend="RT_ST_NumBands"/></para>
				</listitem>
				<listitem>
					<para><varname>pixel_types</varname> An array defining the pixel type for each band.  You will have the same number of elements in this array as you have number of bands.  The pixel_types are one of the following defined in <xref linkend="RT_ST_BandPixelType"/>.</para>
				</listitem>
				<listitem>
					<para><varname>nodata_values</varname> An array of double precision numbers denoting the <varname>nodata_value</varname> for each band.  You will have the same number of elements in this array as you have number of bands. These numbers define the pixel value for each band that should be ignored for most operations.  This is similar information provided by <xref linkend="RT_ST_BandNoDataValue"/>.</para>
				</listitem>
				<listitem>
					<para><varname>out_db</varname> An array of boolean flags indicating if the raster bands data is maintained outside the database. You will have the same number of elements in this array as you have number of bands.</para>
				</listitem>
				<listitem>
					<para><varname>extent</varname> This is the extent of all the raster rows in your raster set. If you plan to load more data that will change the extent of the set, you'll want to run the <xref linkend="RT_DropRasterConstraints"/> function before load and then reapply constraints with <xref linkend="RT_AddRasterConstraints"/> after load. </para>
				</listitem>
				<listitem>
					<para><varname>spatial_index</varname> A boolean that is true if raster column has a spatial index.</para>
				</listitem>
			 </itemizedlist>
        </section>
        <section xml:id="RT_Raster_Overviews">
	<title>Raster Overviews</title>
	<para><varname>raster_overviews</varname> catalogs information about raster table columns used for overviews and additional information about them that is useful to know when utilizing overviews. Overview tables are cataloged in both <varname>raster_columns</varname> and <varname>raster_overviews</varname> because they are rasters in their own right but also serve an additional special purpose of being a lower resolution caricature of a higher resolution table. These are generated along-side the main raster table when you use the <varname>-l</varname> switch in raster loading or can be generated manually using <xref linkend="RT_AddOverviewConstraints"/>.</para>
	<para>Overview tables contain the same constraints as other raster tables as well as additional informational only constraints specific to overviews.</para>
	<note><para>The information in <varname>raster_overviews</varname> does not duplicate the information in <varname>raster_columns</varname>.  If you need the information about an overview table present in <varname>raster_columns</varname> you can join the <varname>raster_overviews</varname> and <varname>raster_columns</varname> together to get the full set of information you need.</para> </note>
	<para>Two main reasons for overviews are:</para>
	 <orderedlist>
		<listitem><para>Low resolution representation of the core tables commonly used for fast mapping zoom-out.</para></listitem>
		<listitem><para>Computations are generally faster to do on them than their higher resolution parents because there are fewer records and each pixel covers more territory.  Though the computations are not as accurate as the high-res tables they support, they can be sufficient in many rule-of-thumb computations.</para></listitem>
	</orderedlist>

	<para>The <varname>raster_overviews</varname> catalog contains the following columns of information.</para>
			 <itemizedlist>
				<listitem>
					<para><varname>o_table_catalog</varname> The database the overview table is in.  This will always read the current database.</para>
				</listitem>
				<listitem>
					<para><varname>o_table_schema</varname> The database schema the overview raster table belongs to.</para>
				</listitem>
				<listitem>
					<para><varname>o_table_name</varname> raster overview table name</para>
				</listitem>
				<listitem>
					<para><varname>o_raster_column</varname> the raster column in the overview table.</para>
				</listitem>

				<listitem>
					<para><varname>r_table_catalog</varname> The database the raster table that this overview services is in.  This will always read the current database.</para>
				</listitem>
				<listitem>
					<para><varname>r_table_schema</varname> The database schema the raster table that this overview services belongs to.</para>
				</listitem>
				<listitem>
					<para><varname>r_table_name</varname> raster table that this overview services.</para>
				</listitem>
				<listitem>
					<para><varname>r_raster_column</varname> the raster column that this overview column services.</para>
				</listitem>
				<listitem>
					<para><varname>overview_factor</varname> - this is the pyramid level of the overview table.  The higher the number the lower the resolution of the table.
					raster2pgsql if given a folder of images, will compute overview of each image file and load separately.  Level 1 is assumed and always the original file. Level 2 is
					will have each tile represent 4 of the original.  So for example if you have a folder of 5000x5000 pixel image files that you chose to chunk 125x125, for each image file your base table will
						have (5000*5000)/(125*125) records = 1600, your (l=2) <varname>o_2</varname> table will have ceiling(1600/Power(2,2)) = 400 rows, your (l=3) <varname>o_3</varname> will have ceiling(1600/Power(2,3) ) = 200 rows.
						If your pixels aren't divisible by the size of your tiles, you'll get some scrap tiles (tiles not completely filled).  Note that each overview tile generated by raster2pgsql has the same number of
						pixels as its parent, but is of a lower resolution where each pixel of it represents (Power(2,overview_factor) pixels of the original).</para>
				</listitem>
			 </itemizedlist>

        </section>
   </section>
   <section xml:id="RT_Raster_Applications">
		<title>Building Custom Applications with PostGIS Raster</title>
		<para>The fact that PostGIS raster provides you with SQL functions to render rasters in known image formats gives  you a lot of options for rendering them.
		For example you can use OpenOffice / LibreOffice for rendering as demonstrated in <link xlink:href="http://www.postgresonline.com/journal/archives/244-Rendering-PostGIS-Raster-graphics-with-LibreOffice-Base-Reports.html">Rendering PostGIS Raster graphics with LibreOffice Base Reports</link>.  In addition you can use a wide variety of languages as demonstrated in this section.</para>
		<section xml:id="RT_PHP_Output">
			<title>PHP Example Outputting using ST_AsPNG in concert with other raster functions</title>
			<para>In this section, we'll demonstrate how to use the PHP PostgreSQL driver and the <xref linkend="RT_ST_AsGDALRaster"/> family of functions to
				output band 1,2,3 of a raster to a PHP request stream that can then be embedded in an img src html tag.</para>

			<para>The sample query demonstrates how to combine a whole bunch of raster functions together to grab all tiles that intersect
				a particular wgs 84 bounding box and then unions with <xref linkend="RT_ST_Union"/> the intersecting tiles together returning all bands, transforms to user specified projection using <xref linkend="RT_ST_Transform"/>,
				and then outputs the results as a png using <xref linkend="RT_ST_AsPNG"/>.</para>
				<para>You would call the script with <literal>http://mywebserver/test_raster.php?srid=2249</literal> to get the raster image in Massachusetts state plane feet.</para>
			<programlisting language="php"><![CDATA[<?php
$conn_str = 'dbname=mydb host=localhost port=5432 user=myuser password=mypwd';
$dbconn = pg_connect($conn_str);
header('Content-Type: image/png');

if (!empty($_REQUEST['srid']) &amp;&amp; is_numeric($_REQUEST['srid'])) {
    $input_srid = intval($_REQUEST['srid']);
} else {
    $input_srid = 26986;
}

$sql = "SET bytea_output = 'escape';
SELECT ST_AsPNG(
    ST_Transform(
        ST_AddBand(
            ST_Union(rast, 1),
            ARRAY[ST_Union(rast, 2), ST_Union(rast, 3)]
        ),
        $input_srid
    )
) AS new_rast
FROM aerials.boston
WHERE ST_Intersects(
    rast,
    ST_Transform(
        ST_MakeEnvelope(-71.1217, 42.227, -71.1210, 42.218, 4326),
        26986
    )
)";

$result = pg_query($sql);
$row = pg_fetch_row($result);
pg_free_result($result);
if ($row === false) return;
echo pg_unescape_bytea($row[0]);
?>
]]></programlisting>
		</section>
		<section xml:id="RT_Net_Output_CS">
			<title>ASP.NET C# Example Outputting using ST_AsPNG in concert with other raster functions</title>
			<para>In this section, we'll demonstrate how to use Npgsql PostgreSQL .NET driver and the <xref linkend="RT_ST_AsGDALRaster"/> family of functions to
				output band 1,2,3 of a raster to the HTTP response stream so it can be embedded in an <code>img</code> tag.</para>

			<para>You will need the npgsql .NET PostgreSQL driver for this exercise which you can get the latest of from <link xlink:href="http://npgsql.projects.postgresql.org/">http://npgsql.projects.postgresql.org/</link>.  Just download the latest and drop into your ASP.NET bin folder and you'll be good to go.</para>
			<para>The sample query demonstrates how to combine a whole bunch of raster functions together to grab all tiles that intersect
				a particular wgs 84 bounding box and then unions with <xref linkend="RT_ST_Union"/> the intersecting tiles together returning all bands, transforms to user specified projection using <xref linkend="RT_ST_Transform"/>,
				and then outputs the results as a png using <xref linkend="RT_ST_AsPNG"/>.</para>
				<para>This is same example as <xref linkend="RT_PHP_Output"/> except implemented in C#.</para>
				<para>You would call the handler with <literal>http://mywebserver/TestRaster.ashx?srid=2249</literal> to get the raster image in Massachusetts state plane feet.</para>
				<para>web.config connection string section:</para>
				<programlisting language="xml"><![CDATA[
<connectionStrings>
    <add name="DSN"
        connectionString="server=localhost;database=mydb;Port=5432;User Id=myuser;password=mypwd"/>
</connectionStrings>
]]></programlisting>
				<para>Code for <filename>TestRaster.ashx</filename>:</para>
			<programlisting language="csharp"><![CDATA[
<%@ WebHandler Language="C#" Class="TestRaster" %>
using System;
using System.Data;
using System.Web;
using Npgsql;

public class TestRaster : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        context.Response.ContentType = "image/png";
        context.Response.BinaryWrite(GetResults(context));
    }

    public bool IsReusable {
        get { return false; }
    }

    public byte[] GetResults(HttpContext context)
    {
        byte[] result = null;
        NpgsqlCommand command;
        string sql = null;
        int input_srid = 26986;
        try {
            string connectionString = System.Configuration.ConfigurationManager
                .ConnectionStrings["DSN"].ConnectionString;
            using (NpgsqlConnection conn = new NpgsqlConnection(connectionString)) {
                conn.Open();

                if (context.Request["srid"] != null)
                {
                    input_srid = Convert.ToInt32(context.Request["srid"]);
                }
                sql = @"SELECT ST_AsPNG(
                            ST_Transform(
                                ST_AddBand(
                                    ST_Union(rast, 1),
                                    ARRAY[ST_Union(rast, 2), ST_Union(rast, 3)]
                                ),
                                :input_srid
                            )
                        ) As new_rast
                        FROM aerials.boston
                        WHERE ST_Intersects(
                            rast,
                            ST_Transform(
                                ST_MakeEnvelope(
                                    -71.1217, 42.227,
                                    -71.1210, 42.218,
                                    4326
                                ),
                                26986
                            )
                        )";
                command = new NpgsqlCommand(sql, conn);
                command.Parameters.Add(new NpgsqlParameter("input_srid", input_srid));


                result = (byte[]) command.ExecuteScalar();
                conn.Close();
            }

        }
        catch (Exception ex)
        {
            result = null;
            context.Response.Write(ex.Message.Trim());
        }
        return result;
    }
}
]]></programlisting>
		</section>
		<section xml:id="RT_Java_Console_App">
			<title>Java console app that outputs raster query as Image file</title>
			<para>This is a simple java console app that takes a query that returns one image and outputs to specified file.</para>
			<para>You can download the latest PostgreSQL JDBC drivers from <link xlink:href="http://jdbc.postgresql.org/download.html">http://jdbc.postgresql.org/download.html</link> </para>
				<para>You can compile the following code with commands like:</para>
	<programlisting language="bash">CLASSPATH=".:../postgresql.jar" javac SaveQueryImage.java
jar cfm SaveQueryImage.jar Manifest.txt *.class</programlisting>
	<para>And call it from the command line with something like:</para>
	<programlisting language="bash">java -jar SaveQueryImage.jar \
  "SELECT ST_AsPNG(ST_AsRaster(ST_Buffer(ST_Point(1, 5), 10, 'quad_segs=2'), 150, 150, '8BUI', 100));" \
  "test.png"</programlisting>
	<para><filename>Manifest.txt</filename>:</para>
	<programlisting language="text">Class-Path: postgresql-9.0-801.jdbc4.jar
Main-Class: SaveQueryImage</programlisting>
			<para>Code for <filename>SaveQueryImage.java</filename>:</para>
			<programlisting language="java">
import java.sql.Connection;
import java.sql.SQLException;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.io.*;

public class SaveQueryImage {
  public static void main(String[] argv) {
      System.out.println("Checking if Driver is registered with DriverManager.");

      try {
        //java.sql.DriverManager.registerDriver (new org.postgresql.Driver());
        Class.forName("org.postgresql.Driver");
      }
      catch (ClassNotFoundException cnfe) {
        System.out.println("Couldn't find the driver!");
        cnfe.printStackTrace();
        System.exit(1);
      }

      Connection conn = null;

      try {
        conn = DriverManager.getConnection("jdbc:postgresql://localhost:5432/mydb", "myuser", "mypwd");
        conn.setAutoCommit(false);

        PreparedStatement sGetImg = conn.prepareStatement(argv[0]);

        ResultSet rs = sGetImg.executeQuery();

        FileOutputStream fout;
        try
        {
            rs.next();
            /** Output to file name requested by user **/
            fout = new FileOutputStream(new File(argv[1]) );
            fout.write(rs.getBytes(1));
            fout.close();
        }
        catch(Exception e)
        {
            System.out.println("Can't create file");
            e.printStackTrace();
        }

        rs.close();
        sGetImg.close();
        conn.close();
      }
      catch (SQLException se) {
        System.out.println("Couldn't connect: print out a stack trace and exit.");
        se.printStackTrace();
        System.exit(1);
      }
  }
}</programlisting>
		</section>

		<section xml:id="RT_PLPython">
			<title>Use PLPython to dump out images via SQL</title>
			<para>This is a plpython stored function that creates a file in the server directory for each record.
			Requires you have plpython installed.  Should work fine with both plpythonu and plpython3u.</para>
			<programlisting language="sql">CREATE OR REPLACE FUNCTION write_file (param_bytes bytea, param_filepath text)
RETURNS text
AS $$
f = open(param_filepath, 'wb+')
f.write(param_bytes)
return param_filepath
$$ LANGUAGE plpythonu;</programlisting>
<para>Write out 5 images to the PostgreSQL server in varying sizes. The PostgreSQL daemon account needs write access to the folder; this echoes back the file names created.</para>
<programlisting language="sql">SELECT write_file(
    ST_AsPNG(
        ST_AsRaster(
            ST_Buffer(ST_Point(1, 5), j * 5, 'quad_segs=2'),
            150 * j,
            150 * j,
            '8BUI',
            100
        )
    ),
    'C:/temp/slices' || j || '.png'
)
FROM generate_series(1, 5) AS j;</programlisting>
<screen role="text-primary">
     write_file
---------------------
 C:/temp/slices1.png
 C:/temp/slices2.png
 C:/temp/slices3.png
 C:/temp/slices4.png
 C:/temp/slices5.png
</screen>
		</section>
		<section xml:id="RasterOutput_PSQL">
			<title>Outputting Rasters with PSQL</title>
			<para>Sadly PSQL doesn't have easy to use built-in functionality for outputting binaries.  This is a bit of a hack that piggy backs on PostgreSQL somewhat legacy large object support.  To use it, first launch your <command>psql</command> command line connected to your database.
			</para>
			<para>Unlike the python approach, this approach creates the file on your local computer.</para>
			<programlisting language="sql">SELECT
    oid,
    lowrite(lo_open(oid, 131072), png) AS num_bytes
FROM (VALUES (
    lo_create(0),
    ST_AsPNG((SELECT rast FROM aerials.boston WHERE rid = 1))
)) AS v(oid, png);</programlisting>
<screen role="text-primary">   oid   | num_bytes
---------+-----------
 2630819 |     74860</screen>

			<para>Note the large object OID and export it to a file on your local computer, replacing the example path with the desired file path.</para>
			<programlisting language="psql">\lo_export 2630819 'C:/temp/aerial_samp.png'</programlisting>

			<para>This deletes the file from large object storage in the database.</para>
			<programlisting language="sql">SELECT lo_unlink(2630819);</programlisting>
		</section>
   </section>
</chapter>
