CS453 | mobile programming
  • outline
  • projects
  • syllabus
  • links

Camera - version 2 of camera api in Android Explained through Example

 

WHY? version 2 lets developer controll the camera options (e.g. focus) more than version 1

 

STEP1: Use CameraManager class tp get a CameraDevice (use CameraManger.open(cameraID)      

  • We use it to iterate all the cameras that are available in the system, each with a designated cameraId
  • Using the cameraId, we can get the properties of the specified camera device.
  • Those properties are represented by class CameraCharacteristics.
  • Things like "is it front or back camera", "output resolutions supported" can be queried there.

STEP 2: Create Camera Capture Session    

  • To capture or stream images from a camera device, the application must first create a camera capture session

 

STEP 3: Create Target Surface to recieve images        

  • The camera capture needs a surface to output what has been captured or being previewed.
  • A target Surface can be obtained from a variety of classes, including SurfaceView, SurfaceTexture via Surface(SurfaceTexture), MediaCodec, MediaRecorder, Allocation, and ImageReader

STEP 4: Create Capture Request using CaptureRequest.Builder   

  • construct a Capture Request, which defines all the capture parameters needed by a camera device to capture a single image.

 

STEP 6: Hand off Capture Request (of step 4) to the Capture Session (of step2) and based on Event CAPTURE either 1 image or Repeated (Endlessly) images    

  • capture session either for a one-shot capture or for an endlessly repeating use

 

STEP 7: Results from processing a request is production of a TotalCaptureResult object.     

 

  • After processing a request, the camera device will produce a TotalCaptureResult object, which contains information about the state of the camera device at time of capture, and the final settings used.

 

 

The Interface

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:paddingBottom="@dimen/activity_vertical_margin"
android:paddingLeft="@dimen/activity_horizontal_margin"
android:paddingRight="@dimen/activity_horizontal_margin"
android:paddingTop="@dimen/activity_vertical_margin"
tools:context="computervision.grewe.camera2api_androidstudio.MainActivity">

<TextView
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:text="Hello World!"
android:id="@+id/textView_takePicture" />

<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="Take Picture"
android:id="@+id/button_takePicture"
android:singleLine="true"
android:layout_below="@+id/textView_takePicture"
android:layout_centerHorizontal="true" />

<TextureView                      <------ THIS PART OF STEP 3
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:id="@+id/textureView"
android:layout_below="@+id/button_takePicture"
android:layout_alignParentStart="true"
android:layout_marginTop="49dp" />

</RelativeLayout>

 

 

 

 

The Manifest File

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="computervision.grewe.camera2api_androidstudio">

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-feature android:name="android.hardware.camera2.full" />

<application
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:supportsRtl="true"
android:theme="@style/AppTheme">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />

<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>

</manifest>

 

 

Strings File

<resources>
<string name="app_name">Camera2API_AndroidStudio</string>
<string name="take_picture">Take picture</string>
</resources>

 

The CODE- Activity Class

package computervision.grewe.camera2api_androidstudio;

 

import android.Manifest;

import android.content.Context;

import android.support.v7.app.AppCompatActivity;

import android.os.Bundle;

import android.content.pm.PackageManager;

import android.graphics.ImageFormat;

import android.graphics.SurfaceTexture;

import android.hardware.camera2.CameraAccessException;

import android.hardware.camera2.CameraCaptureSession;

import android.hardware.camera2.CameraCharacteristics;

import android.hardware.camera2.CameraDevice;

import android.hardware.camera2.CameraManager;

import android.hardware.camera2.CameraMetadata;

import android.hardware.camera2.CaptureRequest;

import android.hardware.camera2.TotalCaptureResult;

import android.hardware.camera2.params.StreamConfigurationMap;

import android.media.Image;

import android.media.ImageReader;

import android.os.Environment;

import android.os.Handler;

import android.os.HandlerThread;

import android.support.annotation.NonNull;

import android.support.v4.app.ActivityCompat;

import android.support.v7.app.AppCompatActivity;

import android.util.Log;

import android.util.Size;

import android.util.SparseIntArray;

import android.view.Surface;

import android.view.TextureView;

import android.view.View;

import android.widget.Button;

import android.widget.Toast;

import java.io.File;

import java.io.FileNotFoundException;

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.OutputStream;

import java.nio.ByteBuffer;

import java.util.ArrayList;

import java.util.Arrays;

import java.util.List;

 

 

 

public class MainActivity extends AppCompatActivity {

 

 

 

    private static final String TAG = "AndroidCameraApi";

    private Button takePictureButton;

    private TextureView textureView;

    private static final SparseIntArray ORIENTATIONS = new SparseIntArray();

    static {

        ORIENTATIONS.append(Surface.ROTATION_0, 90);

        ORIENTATIONS.append(Surface.ROTATION_90, 0);

        ORIENTATIONS.append(Surface.ROTATION_180, 270);

        ORIENTATIONS.append(Surface.ROTATION_270, 180);

    }

    private String cameraId;

    protected CameraDevice cameraDevice;

    protected CameraCaptureSession cameraCaptureSessions;

    protected CaptureRequest captureRequest;

    protected CaptureRequest.Builder captureRequestBuilder;

    private Size imageDimension;

    private ImageReader imageReader;

    private File file;

    private static final int REQUEST_CAMERA_PERMISSION = 200;

    private boolean mFlashSupported;

 

    //threads to run camera in background thread

    private Handler mBackgroundHandler;

    private HandlerThread mBackgroundThread;

 

 

    /**

     * onCreate method to setup interface and to setup the Listener for The TextureView in the

     * interface and create the event listening for the Take Picture button

     * @param savedInstanceState

     */

    @Override

    protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        setContentView(R.layout.activity_main);//load in the xml interface

        textureView = (TextureView) findViewById(R.id.textureView); //textureView

        assert textureView != null;

      textureView.setSurfaceTextureListener(textureListener);

        takePictureButton = (Button) findViewById(R.id.button_takePicture); //button to take paicture

        assert takePictureButton != null;

        takePictureButton.setOnClickListener(new View.OnClickListener() {

            @Override

            public void onClick(View v) {

                takePicture();

            }

        });

    }

 

 

 

    //anonymous class to listen to TextureView

    TextureView.SurfaceTextureListener textureListener = new TextureView.SurfaceTextureListener() {

        @Override

        public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {

            //open your camera here

            openCamera();

        }

        @Override

        public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {

            // Transform you image captured size according to the surface width and height

        }

        @Override

        public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {

            return false;

        }

        @Override

        public void onSurfaceTextureUpdated(SurfaceTexture surface) {

        }

    };

 

 

 

 

 

    //anonymous class to serve as callback for changes in state of the camera

    private final CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {

        @Override

        public void onOpened(CameraDevice camera) {

            //This is called when the camera is open

            Log.e(TAG, "onOpened");

            cameraDevice = camera;//save the opened camera in the Class variable.

            createCameraPreview();

        }

        @Override

        public void onDisconnected(CameraDevice camera) {

            cameraDevice.close();   //close the open camera

        }

        @Override

        public void onError(CameraDevice camera, int error) {

            cameraDevice.close();  //close open camera if an error

            cameraDevice = null;

        }

    };

 

 

 

    //anonymous class to serve as callback for Camera Capture -- will make a toast and setup the camera preview

    final CameraCaptureSession.CaptureCallback captureCallbackListener = new CameraCaptureSession.CaptureCallback() {

        @Override

        public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {

            super.onCaptureCompleted(session, request, result);

         Toast.makeText(MainActivity.this, "Saved:" + file, Toast.LENGTH_SHORT).show();

         createCameraPreview();

        }

    };

 

 

    /**

     *

     *method to start HandlerThread for camera to run in background

     */

    protected void startBackgroundThread() {

        mBackgroundThread = new HandlerThread("Camera Background");

        mBackgroundThread.start();

        mBackgroundHandler = new Handler(mBackgroundThread.getLooper());

    }

 

    /**

     * method to stop Backbround thread

     */

    protected void stopBackgroundThread() {

        mBackgroundThread.quitSafely();

        try {

            mBackgroundThread.join();

            mBackgroundThread = null;

            mBackgroundHandler = null;

        } catch (InterruptedException e) {

            e.printStackTrace();

        }

    }

 

 

    /**

     * method invoked when the "Take Picture" button is pressed to invoke the camera to take a picture

     * A) starts Camera Manager to get the open camera characteristics

     * B) set the capture size to width=640 x height =480 as default or use whatever the first size of images given in characteristics of camera

     * C) Create ImageReader instance with capture size

     * D) Use CaptureRequest.Builder to create a capture request of the cameraDevice (class variable  of CameraDevice instance) associated with the ImageReader as a target

     * E) Setup file on Storage director/pic.jpg to contain the results of taking a picture

     * F) Have OnImageAvailableListneer for  when an image is "written into the ImageReader" save to file of step #E

     * G) call createCamptuerSession(**) method on cameraDevice to do a single image Capture

     *

     */

    protected void takePicture() {

        if(null == cameraDevice) {

            Log.e(TAG, "cameraDevice is null");

            return;

        }

 

        //STEPA)  setup camera manager to get the opened camera characteristics

        CameraManager manager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);

        try {

            CameraCharacteristics characteristics = manager.getCameraCharacteristics(cameraDevice.getId());

            Size[] jpegSizes = null;

            if (characteristics != null) {

                jpegSizes = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputSizes(ImageFormat.JPEG);

            }

 

            //STEP B use default of 640x480 capture or if you can query device above use first resolution returned

            int width = 640;

            int height = 480;

            if (jpegSizes != null && 0 < jpegSizes.length) {

                width = jpegSizes[0].getWidth();

                height = jpegSizes[0].getHeight();

            }

 

            //STEP C: setup ImageReader with capture size

            ImageReader reader = ImageReader.newInstance(width, height, ImageFormat.JPEG, 1);

            List<Surface> outputSurfaces = new ArrayList<Surface>(2);

            outputSurfaces.add(reader.getSurface());

            outputSurfaces.add(new Surface(textureView.getSurfaceTexture()));

 

            //STEP D: Use CaptureRequest.Builder to create a capture request of the cameraDevice associated with the ImageReader as a target

            final CaptureRequest.Builder captureBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);

            captureBuilder.addTarget(reader.getSurface());

            captureBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);

            // Orientation

            int rotation = getWindowManager().getDefaultDisplay().getRotation();

               captureBuilder.set(CaptureRequest.JPEG_ORIENTATION, ORIENTATIONS.get(rotation));    

 

            //STEP E) Setup file on Storage director/pic.jpg to contain the results of taking a picture

            final File file = new File(Environment.getExternalStorageDirectory()+"/pic.jpg");

 

            //STEP F) Have OnImageAvailableListneer for  when an image is "written into the ImageReader" save to file of step #5

            ImageReader.OnImageAvailableListener readerListener = new ImageReader.OnImageAvailableListener() {

                @Override

                public void onImageAvailable(ImageReader reader) {  //when image available save it to the file

                    Image image = null;

                    try {

                        image = reader.acquireLatestImage();

                        ByteBuffer buffer = image.getPlanes()[0].getBuffer();

                        byte[] bytes = new byte[buffer.capacity()];

                        buffer.get(bytes);

                        save(bytes);

                    } catch (FileNotFoundException e) {

                        e.printStackTrace();

                    } catch (IOException e) {

                        e.printStackTrace();

                    } finally {

                        if (image != null) {

                            image.close();

                        }

                    }

                }

                private void save(byte[] bytes) throws IOException {

                    OutputStream output = null;

                    try {

                        output = new FileOutputStream(file);

                        output.write(bytes);

                    } finally {

                        if (null != output) {

                            output.close();

                        }

                    }

                }

            };

 

 

            //Step F continued: registraton of OnImageAvailableListener to our ImageReader instance, reader

            reader.setOnImageAvailableListener(readerListener, mBackgroundHandler);

 

 

            //annonymous inner class that is capture listener

            final CameraCaptureSession.CaptureCallback captureListener = new CameraCaptureSession.CaptureCallback() {

                @Override

                public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {

                    super.onCaptureCompleted(session, request, result);

                    Toast.makeText(MainActivity.this, "Saved:" + file, Toast.LENGTH_SHORT).show();

                    createCameraPreview();

                }

            };

 

            //STEP G: call createCaptuerSession(**) method on cameraDevice to do a single image capture

            cameraDevice.createCaptureSession(outputSurfaces, new CameraCaptureSession.StateCallback() {

                @Override

                public void onConfigured(CameraCaptureSession session) {

                    try {

                       //ask session to CAPTURE 1 picture with listener captureListner above

                        session.capture(captureBuilder.build(), captureListener, mBackgroundHandler);

                    } catch (CameraAccessException e) {

                        e.printStackTrace();

                    }

                }

                @Override

                public void onConfigureFailed(CameraCaptureSession session) {

                }

            }, mBackgroundHandler);

        } catch (CameraAccessException e) {

            e.printStackTrace();

        }

    }

 

 

    /**

     * createCameraPreview method setups TextureView widget to display the cameraDevice preview.

     */

    protected void createCameraPreview() {

        try {

            SurfaceTexture texture = textureView.getSurfaceTexture();

            assert texture != null;

            texture.setDefaultBufferSize(imageDimension.getWidth(), imageDimension.getHeight());

            Surface surface = new Surface(texture);

            captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

         captureRequestBuilder.addTarget(surface);

            cameraDevice.createCaptureSession(Arrays.asList(surface), new CameraCaptureSession.StateCallback(){

                @Override

                public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {

                    //The camera is already closed

                    if (null == cameraDevice) {

                        return;

                    }

                    // When the session is ready, we start displaying the preview.

                    cameraCaptureSessions = cameraCaptureSession;

                    updatePreview();

                }

                @Override

                public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {

                    Toast.makeText(MainActivity.this, "Configuration change", Toast.LENGTH_SHORT).show();

                }

            }, null);

        } catch (CameraAccessException e) {

            e.printStackTrace();

        }

    }

 

 

    /**

     * openCamera method attempts to create the cameraDevice instance using the CameraManager

     */

    private void openCamera() {

        CameraManager manager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);

        Log.e(TAG, "is camera open");

        try {

            cameraId = manager.getCameraIdList()[0];

            CameraCharacteristics characteristics = manager.getCameraCharacteristics(cameraId);

            StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

            assert map != null;

            imageDimension = map.getOutputSizes(SurfaceTexture.class)[0];

            // Add permission for camera and let user grant the permission

            if (ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED && ActivityCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {

                ActivityCompat.requestPermissions(MainActivity.this, new String[]{Manifest.permission.CAMERA, Manifest.permission.WRITE_EXTERNAL_STORAGE}, REQUEST_CAMERA_PERMISSION);

                return;

            }

            manager.openCamera(cameraId, stateCallback, null);

        } catch (CameraAccessException e) {

            e.printStackTrace();

        }

        Log.e(TAG, "openCamera X");

    }

 

 

    /**

     * method to respond to update of the Preview

     */

    protected void updatePreview() {

        if(null == cameraDevice) {

            Log.e(TAG, "updatePreview error, return");

        }

        captureRequestBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);

        try {

        //session asks to repeatedly capture for preview

            cameraCaptureSessions.setRepeatingRequest(captureRequestBuilder.build(), null, mBackgroundHandler);

        } catch (CameraAccessException e) {

            e.printStackTrace();

        }

    }

 

 

    /**

     * method to close the cameraDevice instance

     */

    private void closeCamera() {

        if (null != cameraDevice) {

            cameraDevice.close();

            cameraDevice = null;

        }

        if (null != imageReader) {

            imageReader.close();

            imageReader = null;

        }

    }

 

 

    /**

     * method to do a Toast in response to a situation where the application manifest is not setup with proper permissions

     * @param requestCode

     * @param permissions

     * @param grantResults

     */

    @Override

    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {

        if (requestCode == REQUEST_CAMERA_PERMISSION) {

            if (grantResults[0] == PackageManager.PERMISSION_DENIED) {

                // close the app

                Toast.makeText(MainActivity.this, "Sorry!!!, you can't use this app without granting permission", Toast.LENGTH_LONG).show();

                finish();

            }

        }

    }

 

 

    /**

     * Activity's onResume will call openCamera() and setup the GUI's textureView with a listener to display the image

     */

    @Override

    protected void onResume() {

        super.onResume();

        Log.e(TAG, "onResume");

        startBackgroundThread();

        if (textureView.isAvailable()) {

            openCamera();

        } else {

            textureView.setSurfaceTextureListener(textureListener);

        }

    }

 

    /**

     * Pause camera if onPause

     */

    @Override

    protected void onPause() {

        Log.e(TAG, "onPause");

        //closeCamera();

        stopBackgroundThread();

        super.onPause();

    }

}

 

 

 

Running the code

 

 

Special Note: the code above does NOT rotate the image in the display when you run it in horizontal mode

 

 

 

cs453:mobile programming

  • home
  • outline
  • projects
  • syllabus
  • links