1.1. AV Functions

The SDK provides some additional audio and video capabilities when play live video or playback.

1.1.1. Recording

Once the video has been successfully played (either live or playback), you can record the video currently playing to the phone.

Start video recording

Declaration

int startRecordLocalMp4(String folderPath, String fileName, Context context, OperationDelegateCallBack callBack);

Note: Recording video requires write card permission

Parameter

Example

if (Constants.hasStoragePermission()) {
    String picPath = Environment.getExternalStorageDirectory().getAbsolutePath() + "/Camera/";
    File file = new File(picPath);
    if (!file.exists()) {
        file.mkdirs();
    }
    String fileName = System.currentTimeMillis() + ".mp4";
    videoPath = picPath + fileName;
    mCameraP2P.startRecordLocalMp4(picPath, fileName, CameraPanelActivity.this, new OperationDelegateCallBack() {
        @Override
        public void onSuccess(int sessionId, int requestId, String data) {
            isRecording = true;
            mHandler.sendEmptyMessage(MSG_VIDEO_RECORD_BEGIN);

        }

        @Override
        public void onFailure(int sessionId, int requestId, int errCode) {
            mHandler.sendEmptyMessage(MSG_VIDEO_RECORD_FAIL);
        }
    });
    recordStatue(true);
} else {
    Constants.requestPermission(CameraPanelActivity.this, Manifest.permission.WRITE_EXTERNAL_STORAGE, Constants.EXTERNAL_STORAGE_REQ_CODE, "open_storage");
}

stop video recording

Declaration

int stopRecordLocalMp4(OperationDelegateCallBack callBack);

Parameter

Example

mCameraP2P.stopRecordLocalMp4(new OperationDelegateCallBack() {
    @Override
    public void onSuccess(int sessionId, int requestId, String data) {

    }

    @Override
    public void onFailure(int sessionId, int requestId, int errCode) {

    }
});

1.1.2. Video snapshot

When the video starts to play successfully (it can be live video or video playback), you can take a screenshot of the currently displayed video image.

Declaration

int snapshot(String absoluteFilePath, Context context, OperationDelegateCallBack callBack);

Parameter

Example

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
    String path = Environment.getExternalStorageDirectory().getAbsolutePath() + "/Camera/";
    File file = new File(path);
    if (!file.exists()) {
        file.mkdirs();
    }
    picPath = path;
}
mCameraP2P.snapshot(picPath, CameraPanelActivity.this, new OperationDelegateCallBack() {
    @Override
    public void onSuccess(int sessionId, int requestId, String data) {
        mHandler.sendMessage(MessageUtil.getMessage(MSG_SCREENSHOT, ARG1_OPERATE_SUCCESS, data));
    }

    @Override
    public void onFailure(int sessionId, int requestId, int errCode) {
        mHandler.sendMessage(MessageUtil.getMessage(MSG_SCREENSHOT, ARG1_OPERATE_FAIL));
    }
});

1.1.3. Video sound

When the video is successfully played (either live or playback), the video sound can be turned on, with the default sound turned off.

Declaration

Turn on/off video sound

void setMute(int mute, OperationDelegateCallBack callBack);

Example

mCameraP2P.setMute(1, new OperationDelegateCallBack() {
  @Override
  public void onSuccess(int sessionId, int requestId, String data) {
    //data is the result value after the operation
    previewMute = Integer.valueOf(data);
    mHandler.sendMessage(MessageUtil.getMessage(MSG_MUTE, ARG1_OPERATE_SUCCESS));
  }

  @Override
  public void onFailure(int sessionId, int requestId, int errCode) {
    mHandler.sendMessage(MessageUtil.getMessage(MSG_MUTE, ARG1_OPERATE_FAIL));
  }
});

Declaration

Switch between speaker and handset mode，p2p 1.0 not support.

void setLoudSpeakerStatus(boolean enable);

Parameter

1.1.4. Talk to camera

After the p2p connection is successful, the real-time calling function with the device can be enabled. Before starting talk to device, we need to ensure that the App has gained access to the phone's microphone.

Start talck

Open the audio channel from App to camera.

Example

if (Constants.hasRecordPermission()) {
    mCameraP2P.startAudioTalk(new OperationDelegateCallBack() {
        @Override
        public void onSuccess(int sessionId, int requestId, String data) {
            isSpeaking = true;
            mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_BEGIN, ARG1_OPERATE_SUCCESS));
            ToastUtil.shortToast(CameraPanelActivity.this, "start talk success");
        }

        @Override
        public void onFailure(int sessionId, int requestId, int errCode) {
            isSpeaking = false;
            mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_BEGIN, ARG1_OPERATE_FAIL));
            ToastUtil.shortToast(CameraPanelActivity.this, "operation fail");

        }
    });
} else {
    Constants.requestPermission(CameraPanelActivity.this, Manifest.permission.RECORD_AUDIO, Constants.EXTERNAL_AUDIO_REQ_CODE, "open_recording");
}

Stop talck

Close the audio channel from App to camera.

Example

mCameraP2P.stopAudioTalk(new OperationDelegateCallBack() {
    @Override
    public void onSuccess(int sessionId, int requestId, String data) {
        isSpeaking = false;
        mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_OVER, ARG1_OPERATE_SUCCESS));
    }

    @Override
    public void onFailure(int sessionId, int requestId, int errCode) {
        isSpeaking = false;
        mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_OVER, ARG1_OPERATE_FAIL));

    }
});

Two-way talk

In the video live broadcast, turn on the video sound. At this time, the sound played is the human voice and environmental sound collected by the camera in real time. Then, open the sound channel from App to the camera to implement two-way talk.

Some cameras may not have speakers or pickups, and such cameras are not capable of two-way talk.

One-way talk

The one-way talk function needs the developer to implement control. When the talk is on, turn off the video sound. After the talk is off, turn on the video sound again.

1.1.5. Definition switching

In the video live, you can switch the definition (a few cameras only support one kind of definition), currently only high-definition and standard clarity two kinds of definition, and only when the video live support. A memory card video recording saves only one definition stream of video at the time of recording.

definition mode

Get definition

Get the sharpness of the image transmitted from the camera.

Example

mCameraP2P.getVideoClarity(new OperationDelegateCallBack() {
​    @Override
​    public void onSuccess(int sessionId, int requestId, String data) {

​    }

​    @Override
​    public void onFailure(int sessionId, int requestId, int errCode) {

​    }
});

note: After the preview screen comes out, call this function

Set Video Clarity

Set the sharpness of the image played by the camera.

Example

mCameraP2P.setVideoClarity(2, new OperationDelegateCallBack() {
    @Override
    public void onSuccess(int sessionId, int requestId, String data) {
        videoClarity = Integer.valueOf(data);
        mHandler.sendMessage(MessageUtil.getMessage(MSG_GET_CLARITY, ARG1_OPERATE_SUCCESS));
    }

    @Override
    public void onFailure(int sessionId, int requestId, int errCode) {
        mHandler.sendMessage(MessageUtil.getMessage(MSG_GET_CLARITY, ARG1_OPERATE_FAIL));
    }
});

1.1.6. Original video data

Developers can get the YUV data of video from IPC SDK. The pixel format type of video data is YUV 420sp.

Declaration

To get the YUV data of video, you need to register a listener with ITuyaSmartCameraP2P, Developers only need to overwrite the callbacks you care about

void registerP2PCameraListener(AbsP2pCameraListener listener);

AbsP2pCameraListener main method:

Declaration

Callback video YUV data

public void onReceiveFrameYUVData(int sessionId, ByteBuffer y, ByteBuffer u, ByteBuffer v, int width, int height, int nFrameRate, int nIsKeyFrame, long timestamp, long nProgress, long nDuration, Object camera);

Parameter

Declaration

p2p session status callback

public void onSessionStatusChanged(Object camera, int sessionId, int sessionStatus)

Parameter

Example

private ITuyaSmartCameraP2P mCameraP2P;

@Override
protected void onResume() {
    super.onResume();
    if (null != mCameraP2P) {
        //register listener 
        mCameraP2P.registorOnP2PCameraListener(p2pCameraListener);
    }
}

@Override
protected void onPause() {
    super.onPause();
    if (null != mCameraP2P) {
        //unregister listener
        mCameraP2P.removeOnP2PCameraListener();
    }
}

private AbsP2pCameraListener p2pCameraListener = new AbsP2pCameraListener() {
    @Override
    public void onSessionStatusChanged(Object o, int i, int i1) {
        super.onSessionStatusChanged(o, i, i1);
        //call back when the connection status changes
    }
};

1.1.7. Audio data

If you need to perform additional processing on the audio data collected by the APP, such as voice change, you can get the audio data from IPC SDK, and the audio has been echo canceled. To receive audio data callbacks, you need to register a listener with ITuyaSmartCameraP2P.

Declaration

register listener

void registerSpeakerEchoProcessor(ISpeakerEchoProcessor processor);

Declaration

unregister listener

void unregisterSpeakerEchoProcessor();

Declaration

After the audio data is processed, it needs to be transmitted to the device for playback

void sendAudioTalkData(byte[] outbuf, int length);

Parameter

Example

mCameraP2P.registerSpeakerEchoProcessor(new ISpeakerEchoProcessor() {
    @Override
    public void receiveSpeakerEchoData(ByteBuffer pcm, int sampleRate) {

    }
});

1.1.8. Object outline

After the Object outline function is turned on, during the live video playing, if the device detects a moving object, it will draw a white rectangular frame on the corresponding object.

First, you need to turn on the Object outline function of the device. After it is turned on, the device will send the coordinates of the moving object along with the video frame. Use the DP point "198" to turn on the Object outline function on the device.Device control interface function see details Device Management.

Under the premise that the Object outline function on the device side is enabled, the Object outline function of the IPC SDK needs to be turned on when the live video is played. The SDK will draw a rectangular frame on the video image according to the coordinates of the moving object sent by the device.

Declaration

Enable Object ounline function

void setEnableIVA(boolean enableIVA);

Parameters

1.1.9. Device capability

Declaration

The ICameraConfigInfo class can parse the configuration information of the device to obtain some basic capabilities of the device.

ICameraConfigInfo getCameraConfig(String devId);

Example

ITuyaIPCCore cameraInstance = TuyaIPCSdk.getCameraInstance();
if (cameraInstance != null) {
    ICameraConfigInfo cameraConfig = cameraInstance.getCameraConfig(devId);
    if (cameraConfig != null) {
        int videoNum = cameraConfig.getVideoNum();
    }
}

Device capability class ICameraConfigInfo

Declaration

The value 1 indicates that the device only supports one definition, which is obtained through the defaultDefinition property, and the definition cannot be switched. When converted to 2, it indicates that it supports both SD and HD.

int getVideoNum();

Declaration

Get default definition of live video.

int getDefaultDefinition();

Declaration

Whether the device supports speaker. Return true means that the device support talk.

boolean isSupportSpeaker();

Declaration

Whether the device supports pickups, if the device supports pickups, the sound can be turned on when viewing the video stream on the App side.

boolean isSupportPickup();

Declaration

Get default talk mode.

int getDefaultTalkBackMode();

Declaration

Whether the device supports pickup. Return true means that the device support one-way talk and two-way talk.

boolean isSupportChangeTalkBackMode();

Declaration

Get p2p config raw data.

String getRawDataJsonStr();