1.1. AV Functions
The SDK provides some additional audio and video capabilities when play live video or playback.
1.1.1. Recording
Once the video has been successfully played (either live or playback), you can record the video currently playing to the phone.
Start video recording
Declaration
int startRecordLocalMp4(String folderPath, String fileName, Context context, OperationDelegateCallBack callBack);
Note: Recording video requires write card permission
Parameter
Parameter | Description |
---|---|
folderPath | file path to save video |
fileName | file name |
context | Context |
callBack | result callback |
Example
if (Constants.hasStoragePermission()) {
String picPath = Environment.getExternalStorageDirectory().getAbsolutePath() + "/Camera/";
File file = new File(picPath);
if (!file.exists()) {
file.mkdirs();
}
String fileName = System.currentTimeMillis() + ".mp4";
videoPath = picPath + fileName;
mCameraP2P.startRecordLocalMp4(picPath, fileName, CameraPanelActivity.this, new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
isRecording = true;
mHandler.sendEmptyMessage(MSG_VIDEO_RECORD_BEGIN);
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
mHandler.sendEmptyMessage(MSG_VIDEO_RECORD_FAIL);
}
});
recordStatue(true);
} else {
Constants.requestPermission(CameraPanelActivity.this, Manifest.permission.WRITE_EXTERNAL_STORAGE, Constants.EXTERNAL_STORAGE_REQ_CODE, "open_storage");
}
stop video recording
Declaration
int stopRecordLocalMp4(OperationDelegateCallBack callBack);
Parameter
Parameter | Description |
---|---|
callback | result callback |
Example
mCameraP2P.stopRecordLocalMp4(new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
}
});
1.1.2. Video snapshot
When the video starts to play successfully (it can be live video or video playback), you can take a screenshot of the currently displayed video image.
Declaration
int snapshot(String absoluteFilePath, Context context, OperationDelegateCallBack callBack);
Parameter
Parameter | Description |
---|---|
absoluteFilePath | Picture storage path |
context | Context |
callback | Result callback |
Example
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
String path = Environment.getExternalStorageDirectory().getAbsolutePath() + "/Camera/";
File file = new File(path);
if (!file.exists()) {
file.mkdirs();
}
picPath = path;
}
mCameraP2P.snapshot(picPath, CameraPanelActivity.this, new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
mHandler.sendMessage(MessageUtil.getMessage(MSG_SCREENSHOT, ARG1_OPERATE_SUCCESS, data));
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
mHandler.sendMessage(MessageUtil.getMessage(MSG_SCREENSHOT, ARG1_OPERATE_FAIL));
}
});
1.1.3. Video sound
When the video is successfully played (either live or playback), the video sound can be turned on, with the default sound turned off.
Declaration
Turn on/off video sound
void setMute(int mute, OperationDelegateCallBack callBack);
Audio mode | value |
---|---|
mute | 1 |
unmute | 0 |
Example
mCameraP2P.setMute(1, new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
//data is the result value after the operation
previewMute = Integer.valueOf(data);
mHandler.sendMessage(MessageUtil.getMessage(MSG_MUTE, ARG1_OPERATE_SUCCESS));
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
mHandler.sendMessage(MessageUtil.getMessage(MSG_MUTE, ARG1_OPERATE_FAIL));
}
});
Declaration
Switch between speaker and handset mode,p2p 1.0 not support.
void setLoudSpeakerStatus(boolean enable);
Parameter
Parameter | Description |
---|---|
enable | true switch to speaker mode,false switch to handset mode |
1.1.4. Talk to camera
After the p2p connection is successful, the real-time calling function with the device can be enabled. Before starting talk to device, we need to ensure that the App has gained access to the phone's microphone.
Start talck
Open the audio channel from App to camera.
Example
if (Constants.hasRecordPermission()) {
mCameraP2P.startAudioTalk(new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
isSpeaking = true;
mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_BEGIN, ARG1_OPERATE_SUCCESS));
ToastUtil.shortToast(CameraPanelActivity.this, "start talk success");
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
isSpeaking = false;
mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_BEGIN, ARG1_OPERATE_FAIL));
ToastUtil.shortToast(CameraPanelActivity.this, "operation fail");
}
});
} else {
Constants.requestPermission(CameraPanelActivity.this, Manifest.permission.RECORD_AUDIO, Constants.EXTERNAL_AUDIO_REQ_CODE, "open_recording");
}
Stop talck
Close the audio channel from App to camera.
Example
mCameraP2P.stopAudioTalk(new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
isSpeaking = false;
mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_OVER, ARG1_OPERATE_SUCCESS));
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
isSpeaking = false;
mHandler.sendMessage(MessageUtil.getMessage(MSG_TALK_BACK_OVER, ARG1_OPERATE_FAIL));
}
});
Two-way talk
In the video live broadcast, turn on the video sound. At this time, the sound played is the human voice and environmental sound collected by the camera in real time. Then, open the sound channel from App to the camera to implement two-way talk.
Some cameras may not have speakers or pickups, and such cameras are not capable of two-way talk.
One-way talk
The one-way talk function needs the developer to implement control. When the talk is on, turn off the video sound. After the talk is off, turn on the video sound again.
1.1.5. Definition switching
In the video live, you can switch the definition (a few cameras only support one kind of definition), currently only high-definition and standard clarity two kinds of definition, and only when the video live support. A memory card video recording saves only one definition stream of video at the time of recording.
definition mode
definition mode | value |
---|---|
SD | 2 |
HD | 4 |
Get definition
Get the sharpness of the image transmitted from the camera.
Example
mCameraP2P.getVideoClarity(new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
}
});
note: After the preview screen comes out, call this function
Set Video Clarity
Set the sharpness of the image played by the camera.
Example
mCameraP2P.setVideoClarity(2, new OperationDelegateCallBack() {
@Override
public void onSuccess(int sessionId, int requestId, String data) {
videoClarity = Integer.valueOf(data);
mHandler.sendMessage(MessageUtil.getMessage(MSG_GET_CLARITY, ARG1_OPERATE_SUCCESS));
}
@Override
public void onFailure(int sessionId, int requestId, int errCode) {
mHandler.sendMessage(MessageUtil.getMessage(MSG_GET_CLARITY, ARG1_OPERATE_FAIL));
}
});
1.1.6. Original video data
Developers can get the YUV data of video from IPC SDK. The pixel format type of video data is YUV 420sp.
Declaration
To get the YUV data of video, you need to register a listener with ITuyaSmartCameraP2P, Developers only need to overwrite the callbacks you care about
void registerP2PCameraListener(AbsP2pCameraListener listener);
AbsP2pCameraListener main method:
Declaration
Callback video YUV data
public void onReceiveFrameYUVData(int sessionId, ByteBuffer y, ByteBuffer u, ByteBuffer v, int width, int height, int nFrameRate, int nIsKeyFrame, long timestamp, long nProgress, long nDuration, Object camera);
Parameter
Parameter | Description |
---|---|
sessionId | p2p session Id |
y | Video Y data |
u | Video U data |
v | Video V data |
width | Video width |
height | Video height |
timestamp | Time stamp |
nFrameRate | Frame rate |
nIsKeyFrame | Whether I frame |
nProgress | ime progress (the progress of the message center video) |
nDuration | Duration (message center video duration) |
camera | / |
Declaration
p2p session status callback
public void onSessionStatusChanged(Object camera, int sessionId, int sessionStatus)
Parameter
Parameter | Description |
---|---|
camera | / |
sessionId | session Id |
sessionStatus | Session status |
session status code | Declaration |
---|---|
0 | connection succeeded |
-3 | connection time out |
-12 | connection closed by the device |
-13 | connection closed without response timeout |
Example
private ITuyaSmartCameraP2P mCameraP2P;
@Override
protected void onResume() {
super.onResume();
if (null != mCameraP2P) {
//register listener
mCameraP2P.registorOnP2PCameraListener(p2pCameraListener);
}
}
@Override
protected void onPause() {
super.onPause();
if (null != mCameraP2P) {
//unregister listener
mCameraP2P.removeOnP2PCameraListener();
}
}
private AbsP2pCameraListener p2pCameraListener = new AbsP2pCameraListener() {
@Override
public void onSessionStatusChanged(Object o, int i, int i1) {
super.onSessionStatusChanged(o, i, i1);
//call back when the connection status changes
}
};
1.1.7. Audio data
If you need to perform additional processing on the audio data collected by the APP, such as voice change, you can get the audio data from IPC SDK, and the audio has been echo canceled. To receive audio data callbacks, you need to register a listener with ITuyaSmartCameraP2P.
Declaration
register listener
void registerSpeakerEchoProcessor(ISpeakerEchoProcessor processor);
Declaration
unregister listener
void unregisterSpeakerEchoProcessor();
Declaration
After the audio data is processed, it needs to be transmitted to the device for playback
void sendAudioTalkData(byte[] outbuf, int length);
Parameter
Parameter | Declaration |
---|---|
outbuf | audio data |
length | length |
Example
mCameraP2P.registerSpeakerEchoProcessor(new ISpeakerEchoProcessor() {
@Override
public void receiveSpeakerEchoData(ByteBuffer pcm, int sampleRate) {
}
});
1.1.8. Object outline
After the Object outline function is turned on, during the live video playing, if the device detects a moving object, it will draw a white rectangular frame on the corresponding object.
First, you need to turn on the Object outline function of the device. After it is turned on, the device will send the coordinates of the moving object along with the video frame. Use the DP point "198" to turn on the Object outline function on the device.Device control interface function see details Device Management.
Under the premise that the Object outline function on the device side is enabled, the Object outline function of the IPC SDK needs to be turned on when the live video is played. The SDK will draw a rectangular frame on the video image according to the coordinates of the moving object sent by the device.
Declaration
Enable Object ounline function
void setEnableIVA(boolean enableIVA);
Parameters
Parameter | Description |
---|---|
enable | Whether to open the Object outline |
1.1.9. Device capability
Declaration
The ICameraConfigInfo
class can parse the configuration information of the device to obtain some basic capabilities of the device.
ICameraConfigInfo getCameraConfig(String devId);
Example
ITuyaIPCCore cameraInstance = TuyaIPCSdk.getCameraInstance();
if (cameraInstance != null) {
ICameraConfigInfo cameraConfig = cameraInstance.getCameraConfig(devId);
if (cameraConfig != null) {
int videoNum = cameraConfig.getVideoNum();
}
}
Device capability class ICameraConfigInfo
Declaration
The value 1
indicates that the device only supports one definition, which is obtained through the defaultDefinition
property, and the definition cannot be switched. When converted to 2
, it indicates that it supports both SD and HD.
int getVideoNum();
Declaration
Get default definition of live video.
int getDefaultDefinition();
Declaration
Whether the device supports speaker. Return true
means that the device support talk.
boolean isSupportSpeaker();
Declaration
Whether the device supports pickups, if the device supports pickups, the sound can be turned on when viewing the video stream on the App side.
boolean isSupportPickup();
Declaration
Get default talk mode.
int getDefaultTalkBackMode();
Declaration
Whether the device supports pickup. Return true
means that the device support one-way talk and two-way talk.
boolean isSupportChangeTalkBackMode();
Declaration
Get p2p config raw data.
String getRawDataJsonStr();