For single-image decoding you provide the data size and a pointer to the file data, and the decoded image is placed in the output buffer.

To use the nvJPEG library, start by calling the helper functions for initialization.

1. Create nvJPEG library handle with one of the helper functions nvjpegCreateSimple() or nvjpegCreateEx().

2. Create JPEG state with the helper function nvjpegJpegStateCreate(). See nvJPEG Type Declarations and nvjpegJpegStateCreate().

   Below is the list of helper functions available in the nvJPEG library:

   • nvjpegStatus_t nvjpegGetProperty(libraryPropertyType type, int *value);
   • [DEPRECATED] nvjpegStatus_t nvjpegCreate(nvjpegBackend_t backend, nvjpegHandle_t *handle , nvjpeg_dev_allocator allocator);
   • nvjpegStatus_t nvjpegCreateSimple(nvjpegHandle_t *handle);
   • nvjpegStatus_t nvjpegCreateEx(nvjpegBackend_t backend, nvjpegDevAllocator_t *dev_allocator, nvjpegPinnedAllocator_t *pinned_allocator, unsigned int flags, nvjpegHandle_t *handle);
   • nvjpegStatus_t nvjpegDestroy(nvjpegHandle_t handle);
   • nvjpegStatus_t nvjpegJpegStateCreate(nvjpegHandle_t handle, nvjpegJpegState_t *jpeg_handle);
   • nvjpegStatus_t nvjpegJpegStateDestroy(nvjpegJpegState handle);
   • Other helper functions such as nvjpegSet*() and nvjpegGet*() can be used to configure the library functionality on per-handle basis. Refer to the helper API reference for more details.

3. Retrieve the width and height information from the JPEG-encoded image by using the nvjpegGetImageInfo() function. See also nvjpegGetImageInfo().

   Below is the signature of nvjpegGetImageInfo() function:

   nvjpegStatus_t nvjpegGetImageInfo(
     nvjpegHandle_t              handle,
     const unsigned char         *data,
     size_t                      length,
     int                         *nComponents,
     nvjpegChromaSubsampling_t   *subsampling,
     int                         *widths,
     int                         *heights); 

   For each image to be decoded, pass the JPEG data pointer and data length to the above function. The nvjpegGetImageInfo() function is thread safe.

4. One of the outputs of the above nvjpegGetImageInfo() function is nvjpegChromaSubsampling_t. This parameter is an enum type, and its enumerator list is composed of the chroma subsampling property retrieved from the JPEG image. See nvJPEG Chroma Subsampling.

5. Use the nvjpegDecode() function in the nvJPEG library to decode this single JPEG image. See the signature of this function below:

   nvjpegStatus_t nvjpegDecode(
     nvjpegHandle_t          handle,
     nvjpegJpegState_t       jpeg_handle,
     const unsigned char     *data,
     size_t                  length, 
     nvjpegOutputFormat_t    output_format,
     nvjpegImage_t           *destination,
     cudaStream_t            stream);

   In the above nvjpegDecode() function, the parameters nvjpegOutputFormat_t, nvjpegImage_t, and cudaStream_t can be used to set the output behavior of the nvjpegDecode() function. You provide the cudaStream_t parameter to indicate the stream to which your asynchronous tasks are submitted.

6. The nvjpegOutputFormat_t parameter:

   The nvjpegOutputFormat_t parameter can be set to one of the output_format settings below:

   output_format	Meaning
   NVJPEG_OUTPUT_UNCHANGED	Return the decoded image planar format.
   NVJPEG_OUTPUT_RGB	Convert to planar RGB.
   NVJPEG_OUTPUT_BGR	Convert to planar BGR.
   NVJPEG_OUTPUT_RGBI	Convert to interleaved RGB.
   NVJPEG_OUTPUT_BGRI	Convert to interleaved BGR.
   NVJPEG_OUTPUT_Y	Return the Y component only.
   NVJPEG_OUTPUT_YUV	Return in the YUV planar format.

   For example, if the output_format is set to NVJPEG_OUTPUT_Y or NVJPEG_OUTPUT_RGBI, or NVJPEG_OUTPUT_BGRI then the output is written only to channel[0], and the other channels are not touched.

   Alternately, in the case of planar output, the data is written to the corresponding channels of the nvjpegImage_t destination structure.

   Finally, in the case of grayscale JPEG and RGB output, the luminance is used to create the grayscale RGB.

7. As mentioned above, an important benefit of the nvjpegGetImageInfo()function is the ability to utilize the image information retrieved from the the input JPEG image to allocate proper GPU memory for your decoding operation.

   The nvjpegGetImageInfo() function returns the widths, heights and nComponents parameters.

   nvjpegStatus_t nvjpegGetImageInfo(
     nvjpegHandle_t             handle, 
     const unsigned char        *data, 
     size_t                     length,     
     int                        *nComponents, 
     nvjpegChromaSubsampling_t  *subsampling, 
     int                        *widths, 
     int                        *heights); 

   You can use the retrieved parameters, widths, heights and nComponents, to calculate the required size for the output buffers, either for a single decoded JPEG, or for every decoded JPEG in a batch.

   To optimally set the destination parameter for the nvjpegDecode() function, use the following guidelines:

   For the output_format: NVJPEG_OUTPUT_Y	destination.pitch[0] should be at least: width[0]	destination.channel[0] should be at least of size: destination.pitch[0]*height[0]
   For the output_format	destination.pitch[c] should be at least:	destination.channel[c] should be at least of size:
   NVJPEG_OUTPUT_YUV	width[c] for c = 0, 1, 2	destination.pitch[c]*height[c] for c = 0, 1, 2
   NVJPEG_OUTPUT_RGB and NVJPEG_OUTPUT_BGR	width[0] for c = 0, 1, 2	destination.pitch[0]*height[0] for c = 0, 1, 2
   NVJPEG_OUTPUT_RGBI and NVJPEG_OUTPUT_BGRI	width[0]*3	destination.pitch[0]*height[0]
   NVJPEG_OUTPUT_UNCHANGED	width[c] for c = [ 0, nComponents - 1 ]	destination.pitch[c]*height[c] for c = [ 0, nComponents - 1]

8. Ensure that the nvjpegImage_t structure (or structures, in the case of batched decode) is filled with the pointers and pitches of allocated buffers. The nvjpegImage_t structure that holds the output pointers is defined as follows:

   typedef struct
   {
       unsigned char * channel[NVJPEG_MAX_COMPONENT]; 
       unsigned int pitch[NVJPEG_MAX_COMPONENT];
   } nvjpegImage_t;

   NVJPEG_MAX_COMPONENT is the maximum number of color components the nvJPEG library supports in the current release. For generic images, this is the maximum number of encoded channels that the library is able to decompress.

9. Finally, when you call the nvjpegDecode() function with the parameters as described above, the nvjpegDecode() function fills the output buffers with the decoded data.

Alternately, you can decode a single image in multiple phases. This gives you flexibility in controlling the flow, and optimizing the decoding process.

To decode an image in multiple phases, follow these steps:

1. Just as when you are decoding in a single phase, create the JPEG state with the helper function nvjpegJpegStateCreate().
2. Next, call the functions in the sequence below (see Decode API -- Multiple Phases.)

   • nvjpegDecodePhaseOne()
   • nvjpegDecodePhaseTwo()
   • nvjpegDecodePhaseThree()

3. At the conclusion of the third phase, the nvjpegDecodePhaseThree() function writes the decoded output at the memory location pointed to by its *destination parameter.

For the batched image decoding you provide pointers to multiple file data in the memory, and also provide the buffer sizes for each file data. The nvJPEG library will decode these multiple images, and will place the decoded data in the output buffers that you specified in the parameters.

For batched image decoding in single phase, follow these steps:

1. Call nvjpegDecodeBatchedInitialize() function to initialize the batched decoder. Specify the batch size in the batch_size parameter. See nvjpegDecodeBatchedInitialize().

2. Next, call nvjpegDecodeBatched() for each new batch. Make sure to pass the parameters that are correct to the specific batch of images. If the size of the batch changes, or if the batch decoding fails, then call the nvjpegDecodeBatchedInitialize() function again.

To decode a batch of images in multiple phases, follow these steps:

1. Create the JPEG state with the helper function nvjpegJpegStateCreate().
2. Call the nvjpegDecodeBatchedInitialize() function to initialize the batched decoder. Specify the batch size in the batch_size parameter, and specify the max_cpu_threads parameter to set the maximum number of CPU threads that work on single batch.
3. Batched processing is done by calling the functions for the specific phases in sequence:
   • In the first phase, call nvjpegDecodePhaseOne() for each image in the batch, according to the index of the image in the batch. Note that this could be done using multiple threads. If multiple threads are used then the thread index in the range [0, max_cpu_threads-1] should be provided to the nvjpegDecodeBatchedPhaseOne() function. Before proceeding to the next phase, ensure that the nvjpegDecodePhaseOne() calls for every image have finished.
   • Next, call nvjpegDecodePhaseTwo()..
   • Finally, call nvjpegDecodePhaseThree()..
4. If you have another batch of images of the same size to process, then repeat from 3.

typedef int (*tDevMalloc)(void**, size_t); 
typedef int (*tDevFree)(void*); 
typedef struct 
  { 
    tDevMalloc dev_malloc; 
    tDevFree dev_free; 
  } nvjpegDevAllocator_t;

Users can tell the library to use their own device memory allocator. The function prototypes for the memory allocation and memory freeing functions are similar to the cudaMalloc() and cudaFree() functions. They should return 0 in case of success, and non-zero otherwise. A pointer to the nvjpegDevAllocator_t structure, with properly filled fields, should be provided to the nvjpegCreate() function. NULL is accepted, in which case the default memory allocation functions cudaMalloc() and cudaFree() is used.

When the nvjpegDevAllocator_t *allocator parameter in the nvjpegCreate() or nvjpegCreateEx() function is set as a pointer to the above nvjpegDevAllocator_t structure, then this structure is used for allocating and releasing the device memory. The function prototypes for the memory allocation and memory freeing functions are similar to the cudaMalloc() and cudaFree() functions. They should return 0 in case of success, and non-zero otherwise.

However, if the nvjpegDevAllocator_t *allocator parameter in the nvjpegCreate() or nvjpegCreateEx() function is set to NULL, then the default memory allocation functions cudaMalloc() and cudaFree() will be used. When using nvjpegCreateSimple() function to create library handle the default device memory allocator will be used.

typedef int (*tPinnedMalloc)(void**, size_t, unsigned int flags);
typedef int (*tPinnedFree)(void*);
typedef struct 
{
tPinnedMalloc pinned_malloc;
tPinnedFree pinned_free;
} nvjpegPinnedAllocator_t;      

When the nvjpegPinnedAllocator_t *allocator parameter in the nvjpegCreateEx() function is set as a pointer to the above nvjpegPinnedAllocator_t structure, then this structure will be used for allocating and releasing host pinned memory for copying data to/from device. The function prototypes for the memory allocation and memory freeing functions are similar to the cudaHostAlloc() and cudaFreeHost() functions. They will return 0 in case of success, and non-zero otherwise.

However, if the nvjpegPinnedAllocator_t *allocator parameter in the nvjpegCreateEx() function is set to NULL, then the default memory allocation functions cudaHostAlloc() and cudaFreeHost() will be used. When using nvjpegCreate() or nvjpegCreateSimple() function to create library handle, the default host pinned memory allocator will be used.

struct nvjpegHandle;
typedef struct nvjpegHandle* nvjpegHandle_t;

The library handle is used in any consecutive nvJPEG library calls, and should be initialized first.

The library handle is thread safe, and can be used by multiple threads simultaneously.

struct nvjpegJpegState;
typedef struct nvjpegJpegState* nvjpegJpegState_t;

The nvjpegJpegState structure stores the temporary JPEG information. It should be initialized before any usage. This JPEG state handle can be reused after being used in another decoding. The same JPEG handle should be used across the decoding phases for the same image or batch. Multiple threads are allowed to share the JPEG state handle only when processing same batch during first phase (nvjpegDecodePhaseOne) .

typedef struct
  {
    unsigned char * channel[NVJPEG_MAX_COMPONENT];
    unsigned int pitch[NVJPEG_MAX_COMPONENT];
  } nvjpegImage_t;

The nvjpegImage_t struct holds the pointers to the output buffers, and holds the corresponding strides of those buffers for the image decoding.

See Single Image Decoding on how to set up the nvjpegImage_t struct.

typedef enum {
	NVJPEG_BACKEND_DEFAULT = 0, 
	NVJPEG_BACKEND_HYBRID = 1,
	NVJPEG_BACKEND_GPU_HYBRID = 2
} nvjpegBackend_t; 

The nvjpegBackend_t enum is used to select either default back-end by default, or use GPU decoding for baseline JPEG images, or use CPU for Huffman decoding.

The nvJPEG helper functions are used for initializing.

The helper functions for retrieving the encoded image information.

Functions for decoding single image or batched images in a single phase.

The nvJPEG library provides an ability to control the decoding process in phases. In the simple case of a single-image decode you can split the decoding into phases. For decoding multiple images, you can overlap the decoding phases of separate images within a single thread. Finally, for the batched decode you can use multiple threads to split the host tasks. Synchronization between phases should be handled with CUDA events and CUDA stream synchronization mechanisms, by the user.

The nvJPEG API adheres to the following return codes and their indicators:

typedef enum
{
 NVJPEG_STATUS_SUCCESS = 0,
 NVJPEG_STATUS_NOT_INITIALIZED = 1,
 NVJPEG_STATUS_INVALID_PARAMETER = 2,
 NVJPEG_STATUS_BAD_JPEG = 3,
 NVJPEG_STATUS_JPEG_NOT_SUPPORTED = 4,
 NVJPEG_STATUS_ALLOCATOR_FAILURE = 5,
 NVJPEG_STATUS_EXECUTION_FAILED = 6,
 NVJPEG_STATUS_ARCH_MISMATCH = 7,
 NVJPEG_STATUS_INTERNAL_ERROR = 8,
} nvjpegStatus_t;	

Description of the returned error codes:

One of the outputs of the nvjpegGetImageInfo() API is nvjpegChromaSubsampling_t. This parameter is an enum type, and its enumerator list comprises of the chroma subsampling property retrieved from the encoded JPEG image. Below are the chroma subsampling types the nvjpegGetImageInfo() function currently supports:

typedef enum
{
 NVJPEG_CSS_444,
 NVJPEG_CSS_422,
 NVJPEG_CSS_420,
 NVJPEG_CSS_440,
 NVJPEG_CSS_411,
 NVJPEG_CSS_410,
 NVJPEG_CSS_GRAY,
 NVJPEG_CSS_UNKNOWN
} nvjpegChromaSubsampling_t;	

Refer to the JPEG standard: https://jpeg.org/jpeg/

The user should create an encoding parameters structure with nvjpegEncoderParamsCreate() function. The function will be initialized with default parameters. User can use an appropriate nvjpegEncoderParamsSet*() function to set a specific parameter.

The quality parameter can be set, using nvjpegEncoderParamsSetQuality() function, to an integer value between 1 and 100, and this quality parameter will be used as a base for generating the JPEG quantization tables.

The parameters structure should be passed to compression functions.

The user should create the encoding state structure using nvjpegEncoderStateCreate() function. This function will hold intermediate buffers for the encoding process. This state should be passed to the compression functions.

The nvJPEG library provides a few interfaces for compressing the image in different formats and colorspaces. See below.

Often it is not feasible to accurately predict the final compressed data size of the final JPEG stream for any input data and parameters. The nvJPEG library, while encoding, will calculate the size of the final stream, allocate temporary buffer in the encoder state and save the compressed data in the encoding state's buffer. In order to get final compressed JPEG stream, the user should provide the memory buffer large enough to store this compressed data. There are two options for how to do this:

1. Use the upper bound on compressed JPEG stream size for the given parameters and image dimensions:
   1. Use the nvjpegEncodeRetrieveBitstream() function to retrieve the maximum possible JPEG stream size at any given time.
   2. Allocate the memory buffer at any given time.
   3. Encode the image using one of the encoding functions.
   4. Retrieve the compressed JPEG stream from the encoder state after successful encoding, using the nvjpegEncodeRetrieveBitstream() and the allocated buffer.
2. Wait for the encoding to complete, and retrieve the exact size of required buffer, as below:
   1. Encode the image using one of the encoding functions.
   2. Use the nvjpegEncodeRetrieveBitstream() function to retrieve the size in bytes of the compressed JPEG stream.
   3. Allocate the memory buffer of at least this size.
   4. Use the nvjpegEncodeRetrieveBitstream() function to populate your buffer with the compressed JPEG stream.

See below the example code, and the block diagram shown in Figure 1, for encoding with nvJPEG Encoder.

Figure 1. JPEG Encoding Using nvJPEG Encoder

nvjpegHandle_t nv_handle;
nvjpegEncoderState_t nv_enc_state;
nvjpegEncoderParams_t nv_enc_params;
cudaStream_t stream;
 
// initialize nvjpeg structures
nvjpegCreateSimple(&nv_handle);
nvjpegEncoderStateCreate(nv_handle, &nv_enc_state, stream);
nvjpegEncoderParamsCreate(nv_handle, &nv_enc_params, stream);
 
nvjpegImage_t nv_image;
// Fill nv_image with image data, let’s say 640x480 image in RGB format
 
// Compress image
nvjpegEncodeImage(nv_handle, nv_enc_state, nv_enc_params,
    &nv_image, NVJPEG_INPUT_RGB, 640, 480, stream);
 
// get compressed stream size
size_t length;
nvjpegEncodeRetrieveBitstream(nv_handle, nv_enc_state, NULL, &length, stream);
// get stream itself
cudaStreamSynchronize(stream);
std::vector<char> jpeg(length);
nvjpegEncodeRetrieveBitstream(nv_handle, nv_enc_state, jpeg.data(), &length, 0);
 
// write stream to file
cudaStreamSynchronize(stream);
std::ofstream output_file(“test.jpg”, std::ios::out | std::ios::binary);
output_file.write(jpeg.data(), length);
output_file.close();

typedef enum
{
    NVJPEG_INPUT_RGB         = 3, 
    NVJPEG_INPUT_BGR         = 4, 
    NVJPEG_INPUT_RGBI        = 5, 
    NVJPEG_INPUT_BGRI        = 6 
} nvjpegInputFormat_t;

The nvjpegInputFormat_t enum is used to select the color model and pixel format of the input image. It is used for conversion to YCbCr during encoding.

The nvjpegEncoderState_t is a structure that stores intermdiate buffers and variables used for compression.

The nvjpegEncoderParams_t is a structure that stores JPEG encode parameters.

Creates encoder state that stores intermediate buffers used in compression.

Signature:

nvjpegStatus_t nvjpegEncoderStateCreate(
	nvjpegHandle_t handle,
	nvjpegEncoderState_t *encoder_state,
	cudaStream_t stream);

Parameters:

Destroys the encoder state.

Signature:

nvjpegStatus_t nvjpegEncoderStateDestroy(
	nvjpegEncoderState_t encoder_state);

Parameters:

Creates the structure that holds the compression parameters.

Signature:

nvjpegStatus_t nvjpegEncoderParamsCreate(
	nvjpegHandle_t handle, 
	nvjpegEncoderParams_t *encoder_params,
	cudaStream_t stream);

Parameters:

Destroys the encoder parameters structure.

Signature:

nvjpegEncoderParamsDestroy(
	nvjpegEncoderParams_t encoder_params);

Parameters:

Sets the parameter quality in the encoder parameters structure.

Signature:

nvjpegStatus_t nvjpegEncoderParamsSetQuality(
	nvjpegEncoderParams_t encoder_params,
	const int quality,
	cudaStream_t stream);

Parameters:

Sets whether or not to use optimized Huffman. Using optimized Huffman produces smaller JPEG bitstream sizes with the same quality, but with slower performance.

Signature:

nvjpegStatus_t nvjpegEncoderParamsSetOptimizedHuffman(
	nvjpegEncoderParams_t encoder_params,
	const int optimized,
	cudaStream_t stream);

Parameters:

Sets which chroma subsampling will be used for JPEG compression.

Signature:

nvjpegStatus_t nvjpegEncoderParamsSetSamplingFactors(
	nvjpegEncoderParams_t encoder_params,
	const nvjpegChromaSubsampling_t chroma_subsampling,
	cudaStream_t stream);

Parameters:

Returns the maximum possible buffer size that is needed to store the compressed JPEG stream, for the given input parameters.

Signature:

nvjpegStatus_t nvjpegEncodeGetBufferSize(
	nvjpegHandle_t handle,
	const nvjpegEncoderParams_t encoder_params,
	int image_width,
	int image_height,
	size_t *max_stream_length);

Parameters:

Compresses the image in YUV colorspace to JPEG stream using the provided parameters, and stores it in the state structure.

Signature:

nvjpegStatus_t nvjpegEncodeYUV(
	nvjpegHandle_t handle,
	nvjpegEncoderState_t encoder_state,
	const nvjpegEncoderParams_t encoder_params,
	const nvjpegImage_t *source,
	nvjpegChromaSubsampling_t chroma_subsampling, 
	int image_width,
	int image_height,
	cudaStream_t stream);

Parameters:

Compresses the image in the provided format to the JPEG stream using the provided parameters, and stores it in the state structure.

Signature:

nvjpegStatus_t nvjpegEncodeImage(
	nvjpegHandle_t handle,
	nvjpegEncoderState_t encoder_state,
	const nvjpegEncoderParams_t encoder_params,
	const nvjpegImage_t *source,
	nvjpegInputFormat_t input_format, 
	int image_width,
	int image_height,
	cudaStream_t stream);

Parameters:

Retrieves the compressed stream from the encoder state that was previously used in one of the encoder functions.

Signature:

nvjpegStatus_t nvjpegEncodeRetrieveBitstream(
	nvjpegHandle_t handle,
	nvjpegEncoderState_t encoder_state,
	unsigned char *data,
	size_t *length,
	cudaStream_t stream);

Parameters: