Video and Image Capture with CameraX in Android

Table of Contents

• Permissions
• Image Capture
  - Save Photo
  - Capture Photo
• Video Capture

Getting Started

App level build.gradle file,

def camerax_version = "1.3.0-alpha02"
implementation "androidx.camera:camera-core:${camerax_version}"
implementation "androidx.camera:camera-camera2:${camerax_version}"
implementation "androidx.camera:camera-lifecycle:${camerax_version}"
implementation "androidx.camera:camera-video:${camerax_version}"

implementation "androidx.camera:camera-view:${camerax_version}"
implementation "androidx.camera:camera-extensions:${camerax_version}"

AndroidManifest.xml for permissions,

<uses-feature android:name="android.hardware.camera.any" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" android:maxSdkVersion="28" />

Adding android.hardware.camera.any makes sure that the device has a camera. Specifying .any means that it can be a front camera or a back camera.

Permissions

class MainActivity : AppCompatActivity() {
    companion object {
        private const val REQUEST_CODE_PERMISSIONS = 10
        private val REQUIRED_PERMISSIONS =
            mutableListOf (
                Manifest.permission.CAMERA,
                Manifest.permission.RECORD_AUDIO,
            ).apply {
                if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.P) {
                    add(Manifest.permission.WRITE_EXTERNAL_STORAGE)
                }
            }.toTypedArray()
    }

    private lateinit var viewModel: SharedViewModel

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        viewModel = ViewModelProvider(this).get(SharedViewModel::class.java)

        if (!allPermissionsGranted()) {
            ActivityCompat.requestPermissions(this, REQUIRED_PERMISSIONS, REQUEST_CODE_PERMISSIONS)
        } else {
            viewModel.setPermission(allPermissionsGranted())
        }

        //...
    }

    override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<out String>, grantResults: IntArray) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults)
        if (requestCode == REQUEST_CODE_PERMISSIONS) {
            viewModel.setPermission(allPermissionsGranted())
        }
    }

    private fun allPermissionsGranted() = REQUIRED_PERMISSIONS.all {
        ContextCompat.checkSelfPermission(this, it) == PackageManager.PERMISSION_GRANTED
    }
}

We’ll ask for necessary permissions on app launch but it’s not ideal for production.

Recommended Implementation: Generally, you should display a UI explaining to the user the features that will be enabled if they grant permissions for the app to post notifications. This UI should provide the user options to agree or deny, such as OK and No thanks buttons. If the user selects OK, directly request the permission. If the user selects No thanks, allow the user to continue without notifications.

Image Capture

class MainFragment : Fragment() {
    //...

    private var imageCapture: ImageCapture? = null

    private lateinit var cameraExecutor: ExecutorService

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        sharedViewModel.isPermissionGranted.observe(viewLifecycleOwner) {
            if (it) startCamera()
        }

        _binding.imageCaptureButton.setOnClickListener { capturePhoto() }
        _binding.imageSaveButton.setOnClickListener { savePhoto() }

        cameraExecutor = Executors.newSingleThreadExecutor()
    }

    private fun capturePhoto() {
        //...
    }

    private fun savePhoto() {
        //...
    }

    private fun startCamera() {
        val cameraProviderFuture = ProcessCameraProvider.getInstance(requireContext())

        cameraProviderFuture.addListener({
            val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()

            val preview = Preview.Builder()
                .build()
                .also {
                    it.setSurfaceProvider(_binding.viewFinder.surfaceProvider)
                }

            imageCapture = ImageCapture.Builder()
                .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
                .build()

            val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

            try {
                // Unbind use cases before rebinding
                cameraProvider.unbindAll()
                cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageCapture)
            } catch(exc: Exception) {
                Log.e(TAG, "Use case binding failed", exc)
            }
        }, ContextCompat.getMainExecutor(requireContext()))
    }

    override fun onDestroy() {
        super.onDestroy()
        cameraExecutor.shutdown()
    }
}

We’ll implement savePhoto and capturePhoto later.

ProcessCameraProvider, a singleton which can be used to bind the lifecycle of cameras to any LifecycleOwner within an application's process.

Only a single process camera provider can exist within a process, and it can be retrieved with getInstance.

setCaptureMode, is to configure the capture mode when taking a photo,

• CAPTURE_MODE_MINIMIZE_LATENCY optimize image capture for latency.
• CAPTURE_MODE_MAXIMIZE_QUALITY optimize image capture for image quality.

The capture mode defaults to CAPTURE_MODE_MINIMIZE_LATENCY.

Now, we’ll implement savePhoto and capturePhoto. Difference is, savePhoto saves the image to the provided file location but capturePhoto provides in-memory buffer of the captured image.

Save Photo

private fun savePhoto() {
    // If the use case is null, exit out of the function.
    // This will be null If we tap the photo button before image capture is set up.
    // Without the return statement, the app would crash if it was null.
    val imageCapture = imageCapture ?: return

    val name = SimpleDateFormat(FILENAME_FORMAT, Locale.US)
        .format(System.currentTimeMillis())
    val contentValues = ContentValues().apply {
        put(MediaStore.MediaColumns.DISPLAY_NAME, name)
        put(MediaStore.MediaColumns.MIME_TYPE, "image/jpeg")
        if(Build.VERSION.SDK_INT > Build.VERSION_CODES.P) {
            put(MediaStore.Images.Media.RELATIVE_PATH, "Pictures/PATH")
        }
    }

    val outputOptions = ImageCapture.OutputFileOptions
        .Builder(requireActivity().contentResolver,
            MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
            contentValues)
        .build()

    imageCapture.takePicture(
        outputOptions,
        ContextCompat.getMainExecutor(requireContext()),
        object : ImageCapture.OnImageSavedCallback {
            override fun onError(exc: ImageCaptureException) {
                Log.e(TAG, "Photo capture failed: ${exc.message}", exc)
            }

            override fun onImageSaved(output: ImageCapture.OutputFileResults){
                val msg = "Photo capture succeeded: ${output.savedUri}"
                Toast.makeText(context, msg, Toast.LENGTH_SHORT).show()
                Log.d(TAG, msg)
            }
        }
    )
}

contentValues, creates a MediaStore content value to hold the image. We use a timestamp so the display name in MediaStore will be unique.

outputOptions, creates output options object which contains file and metadata.

takePicture(OutputFileOptions, Executor, OnImageSavedCallback), this method saves the captured image to the provided file location.

Capture Photo

private fun capturePhoto() {
    val imageCapture = imageCapture ?: return

    imageCapture.takePicture(
        ContextCompat.getMainExecutor(requireContext()),
        object : ImageCapture.OnImageCapturedCallback() {
            override fun onCaptureSuccess(image: ImageProxy) {
                super.onCaptureSuccess(image)
                parentFragmentManager.beginTransaction()
                    .replace(R.id.container, ImageViewFragment.newInstance(image))
                    .addToBackStack(null)
                    .commit()
            }
        }
    )
}

takePicture(Executor, OnImageCapturedCallback), this method provides an in-memory buffer of the captured image.

That’s it. We pass image: ImageProxy to ImageViewFragment and display it to the user.

Note: To display ImageProxy as Image, we need to convert it to bitmap.

fun Image.toBitmap(): Bitmap {
    val buffer = planes[0].buffer
    buffer.rewind()
    val bytes = ByteArray(buffer.capacity())
    buffer.get(bytes)
    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

Video Capture

class VideoFragment : Fragment() {
    //...

    private var videoCapture: VideoCapture<Recorder>? = null
    private var recording: Recording? = null

    private lateinit var cameraExecutor: ExecutorService

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        sharedViewModel.isPermissionGranted.observe(viewLifecycleOwner) {
            if (it) startCamera()
        }

        _binding.videoCaptureButton.setOnClickListener { captureVideo() }

        cameraExecutor = Executors.newSingleThreadExecutor()
    }

    private fun captureVideo() {
        //...
    }

    private fun startCamera() {
        //...

        // Video
        val recorder = Recorder.Builder()
            .setQualitySelector(
                QualitySelector.from(
                    Quality.HIGHEST,
                    FallbackStrategy.higherQualityOrLowerThan(Quality.SD)
                )
            )
            .build()
        videoCapture = VideoCapture.withOutput(recorder)
        
        //...
        cameraProvider.bindToLifecycle(this, cameraSelector, preview, videoCapture)
    }

    override fun onDestroy() {
        super.onDestroy()
        cameraExecutor.shutdown()
    }
}

We’ll implement captureVideo later.

startCamera is same as what we did in Image Capture part. Only difference is, we need to remove imageCapture and add recorder and videoCapture variables.

private fun captureVideo() {
    val videoCapture = this.videoCapture ?: return

    _binding.videoCaptureButton.isEnabled = false

    val curRecording = recording
    if (curRecording != null) {
        curRecording.stop()
        recording = null
        return
    }

    val name = SimpleDateFormat(FILENAME_FORMAT, Locale.US)
        .format(System.currentTimeMillis())
    val contentValues = ContentValues().apply {
        put(MediaStore.MediaColumns.DISPLAY_NAME, name)
        put(MediaStore.MediaColumns.MIME_TYPE, "video/mp4")
        if (Build.VERSION.SDK_INT > Build.VERSION_CODES.P) {
            put(MediaStore.Video.Media.RELATIVE_PATH, "Movies/CameraX-Video")
        }
    }

    val mediaStoreOutputOptions = MediaStoreOutputOptions
        .Builder(requireActivity().contentResolver, MediaStore.Video.Media.EXTERNAL_CONTENT_URI)
        .setContentValues(contentValues)
        .build()

    recording = videoCapture.output
        .prepareRecording(requireContext(), mediaStoreOutputOptions)
        .apply {
            // Enable Audio for recording
            if (
                PermissionChecker.checkSelfPermission(requireContext(), Manifest.permission.RECORD_AUDIO) ==
                PermissionChecker.PERMISSION_GRANTED
            ) {
                withAudioEnabled()
            }
        }
        .start(ContextCompat.getMainExecutor(requireContext())) { recordEvent ->
            when(recordEvent) {
                is VideoRecordEvent.Start -> {
                    _binding.videoCaptureButton.apply {
                        text = getString(R.string.stop_capture)
                        isEnabled = true
                    }
                }
                is VideoRecordEvent.Finalize -> {
                    if (!recordEvent.hasError()) {
                        val msg = "Video capture succeeded: ${recordEvent.outputResults.outputUri}"
                        parentFragmentManager.beginTransaction()
                            .replace(R.id.container, VideoViewFragment.newInstance(recordEvent.outputResults.outputUri))
                            .addToBackStack(null)
                            .commit()
                        Log.d(TAG, msg)
                    } else {
                        recording?.close()
                        recording = null
                        Log.e(TAG, "Video capture ends with error: ${recordEvent.error}")
                    }
                    _binding.videoCaptureButton.apply {
                        text = getString(R.string.start_capture)
                        isEnabled = true
                    }
                }
            }
        }
}

curRecording, if there is an active recording in progress, we stop it and release the current recording. We notify when the captured video file is ready to be used by application.

To start recording, we create a new recording session. First, we create intended MediaStore video object with system timestamp as the display name.

.start starts the recording, making it an active recording. Only a single recording can be active at a time.

• Upon successful start, VideoRecordEvent.Start event will be triggered.
• Upon successful recording, VideoRecordEvent.Finalize will be triggered. Also, if error occurs while starting the recording, VideoRecordEvent.Finalize event will be the first event to be triggered. We can access the error with .getError method.

When we successfully record a video, we can get the saved video’s uri with .outputResults.outputUri.

Full Code

MrNtlu/Camera-Guide (github.com)

Sources:

• CameraX overview | Android Developers
• https://developer.android.com/codelabs/camerax-getting-started
• Image capture | Android Developers
• CameraX video capturing architecture | Android Developers
• Photo picker | Android Developers

You can contact me on,

• LinkedIn
• Github
• Twitter