Cameras¶

The cameras on Iristick smart glasses are accessible only through the camera package. This package contains a set of classes and interfaces for opening camera devices, starting capture sessions and capturing frames with capture requests.

This package is loosely modeled after the standard Android camera2 API.

Example code

The Camera Example provided in the SDK package shows how to use the Camera API. It shows a preview of both cameras on the display. The example shows how to adjust zoom and offset, and how to trigger auto-focus.

Querying cameras¶

Every camera device has a unique identifier. You can find the identifier of a camera with the findCamera() method of a Headset object, providing the desired type of camera.

@Override
public void onHeadsetConnected(@NonNull Headset headset) {
    String center = headset.findCamera(CameraCharacteristics.TYPE_WIDE_ANGLE);
    if (center != null) {
        // This headset has a wide-angle center camera.
        // Do something with it.
    }
    String zoom = headset.findCamera(CameraCharacteristics.TYPE_ZOOM);
    if (zoom != null) {
        // This headset has a zoom camera.
        // Do something with it.
    }
}

Alternatively, you can get a list of all available camera devices with the getCameraIdList() method of a Headset object. With an identifier, you can query the properties and capabilities of the corresponding camera with getCameraCharacteristics().

/**
 * Test if the camera device identified by cameraId supports 720p streaming.
 */
boolean supportsHD(@NonNull Headset headset, @NonNull String cameraId) {
    /* Get the characteristics for this specific camera. */
    CameraCharacteristics characteristics =
            headset.getCameraCharacteristics(cameraId);

    /* Query the supported stream mapping. */
    StreamConfigurationMap mapping =
            characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

    /* Test if 720p is in the list of supported sizes. */
    for (Point size : mapping.getSizes()) {
        if (size.x == 1280 && size.y == 720)
            return true;
    }
    return false;
}

Opening a camera¶

The camera device is opened asynchronously with a call to openCamera(), supplying the identifier of the camera and an implementation of CameraDevice.Listener. The listener's onOpened method will be called when the camera is successfully opened.

Important

Opening a camera device requires the CAMERA permission.

You can close the camera device with its close() method. This will close the active capture session.

When another app has already opened a camera device, that app will be disconnected from the camera device and must reopen it in order to capture images again.

@Override
public void onHeadsetConnected(@NonNull Headset headset) {
    /* Open wide-angle center camera. */
    String cameraId = headset.findCamera(CameraCharacteristics.TYPE_WIDE_ANGLE);
    if (cameraId == null)
        return;
    headset.openCamera(cameraId, new CameraDevice.Listener() {
        @Override
        public void onOpened(@NonNull CameraDevice camera) {
            /* This is called when the camera was successfully opened.
             * You can start a capture session here. */
        }

        @Override
        public void onClosed(@NonNull CameraDevice camera) {
            /* This is called when the camera device is completely closed. */
        }

        @Override
        public void onDisconnected(@NonNull CameraDevice camera) {
            /* This is called when the camera is no longer available. */
        }

        @Override
        public void onError(@NonNull CameraDevice camera, int error) {
            /* This is called when there is an error with the camera device. */
        }
    }, null);
}

Starting a capture session¶

After opening the camera, it must be configured for the output surfaces that are going to be used. The configuration is done by starting a capture session with the createCaptureSession() method. Any output surface that will be used as target of a capture request must be provided when creating the capture session. As for opening a camera device, the creation of a capture session happens asynchronously. The onConfigured() method of the provided listener will be called when the capture session is ready to accept capture requests.

The following surface formats are supported. Providing a surface with an unsupported format will result in onConfigureFailed() to be called with error code ERROR_INVALID_OUTPUTS.

Surface format	Pixel format
`PixelFormat.RGB_888`	24 bits per pixel (Red, Green, Blue)
`PixelFormat.RGBA_8888`	32 bits per pixel (Red, Green, Blue, Alpha)
`PixelFormat.RGBX_8888`	32 bits per pixel (Red, Green, Blue, Padding)
`PixelFormat.RGB_565`	16 bits per pixel (5b Red, 6b Green, 5b Blue)
`PixelFormat.L_8`	Monochrome 8 bits per pixel
`ImageFormat.Y8`	Monochrome 8 bits per pixel
`ImageFormat.YUY2`	Interleaved YUYV with 4:2:2 chroma subsampling
`ImageFormat.YV12`	Planar YUV with 4:2:0 chroma subsampling
`ImageFormat.JPEG`	Compressed JPEG

Only one capture session may be active per camera device. If you create a new session, the previous session will be closed.

You can close the capture session with its close() method. This will asynchronously remove any repeating requests and then wait for all pending capture requests to finish. If you want to stop the session as soon as possible, you can abort the captures first with abortCaptures().

Surface mPreviewSurface;

@Override
public void onOpened(@NonNull CameraDevice camera) {
    List<Surface> outputs = new ArrayList<>();
    outputs.add(mPreviewSurface);
    camera.createCaptureSession(outputs, new CaptureSession.Listener() {
        @Override
        public void onConfigured(@NonNull CaptureSession captureSession) {
            /* This is called when the capture session is fully configured.
             * You may start issuing capture requests here. */
        }

        @Override
        public void onConfigureFailed(@NonNull CaptureSession captureSession,
                                      int error) {
            /* This is called when configuring the capture session failed. */
        }

        @Override
        public void onClosed(@NonNull CaptureSession captureSession) {
            /* This is called when the capture session is closed.
             * It is no longer possible to issue capture requests. */
        }

        @Override
        public void onActive(@NonNull CaptureSession session) {
            /* This is called when the camera device actively starts processing
             * capture requests. */
        }

        @Override
        public void onCaptureQueueEmpty(@NonNull CaptureSession session) {
            /* This is called when the camera device has no more pending capture
             * requests and may fall back to the repeating request if set. */
        }

        @Override
        public void onReady(@NonNull CaptureSession session) {
            /* This is called when the camera device has finished processing all
             * capture requests and has no repeating request set. */
        }
    }, null);
}

Creating a capture request¶

Once the capture session is configured, the camera device is ready to process capture requests. A capture request allows to capture an individual frame of the camera. It may be submitted multiple times, or set as a repeating request, to capture consecutive frames.

A capture request specifies the target surfaces and specific camera settings. Capture requests are created with a CaptureRequest.Builder. The createCaptureRequest() method will create a new builder and configure it for a specific use case. Then, you can set the output surfaces and fine tune the settings. Finally, use the build() method to create an immutable capture request.

CaptureRequest createCaptureRequest(CameraDevice camera) {
    /* Create the builder. */
    CaptureRequest.Builder builder =
        camera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

    /* Add the target surface(s) */
    builder.addTarget(mPreviewSurface);

    /* Set capture settings (e.g., zoom level). */
    builder.set(CaptureRequest.SCALER_ZOOM, 2);

    /* Build the capture request. */
    return builder.build();
}

You can now submit capture requests to the capture session. For one-shot image captures or a small burst of captures, use capture() or captureBurst(). For repeated captures, such as video recording or a camera preview, use setRepeatingRequest().

You can stop the repeating request with stopRepeating().

In order to be notified of the progress of the submitted capture requests, you can provide a CaptureListener2. A capture sequence denotes a set of capture requests that have been submitted together. The capture listener allows tracking the progress of both individual captures and complete sequence.

class MyCaptureListener extends CaptureListener2 {
    @Override
    public void onCaptureStarted(@NonNull CaptureSession session,
                                 @NonNull CaptureRequest request,
                                 long frameNumber) {
        /* This is called when the camera device starts processing a capture
         * request. This is the appropriate time to play a shutter sound
         * effect or trigger UI indicators. */
    }

    @Override
    public void onCaptureBufferLost(@NonNull CaptureSession session,
                                    @NonNull CaptureRequest request,
                                    @NonNull Surface surface,
                                    long frameNumber) {
        /* This is called when a buffer for capture data could not be sent to
         * its destination surface. */
    }

    @Override
    public void onCaptureCompleted(@NonNull CaptureSession session,
                                   @NonNull CaptureRequest request,
                                   @NonNull CaptureResult result) {
        /* This is called when an image capture has completed successfully. */
    }

    @Override
    public void onPostProcessCompleted(@NonNull CaptureSession session,
                                       @NonNull CaptureRequest request,
                                       @NonNull CaptureResult result) {
        /* This may be called after onCaptureCompleted when post-processing
         * has been requested, e.g., when scanning barcodes.  The result object
         * will contain the same information as onCaptureCompleted plus the
         * results of the post-processing. */
    }

    @Override
    public void onCaptureFailed(@NonNull CaptureSession session,
                                @NonNull CaptureRequest request,
                                @NonNull CaptureFailure failure) {
        /* This is called when a capture failed. */
    }

    @Override
    public void onCaptureSequenceCompleted(@NonNull CaptureSession session,
                                           int sequenceId,
                                           long frameNumber) {
        /* This is called when all captures of a capture sequence are
         * finished, i.e., onCaptureCompleted or onCaptureFailed has been
         * called for all capture requests in the sequence. */
    }

    @Override
    public void onCaptureSequenceAborted(@NonNull CaptureSession session,
                                         int sequenceId) {
        /* This is called when a capture sequence aborts before any
         * CaptureResult or CaptureFailure for it have been returned
         * via this listener. */
    }
}