go语言做的网站,免费推广手段,设计公司网站 唐山,网站文件解压前言
今天主要对Android平台GB28181设备接入模块支持的接入数据类型#xff0c;做个简单的汇总#xff1a;
编码前数据#xff08;目前支持的有YV12/NV21/NV12/I420/RGB24/RGBA32/RGB565等数据类型#xff09;#xff0c;其中#xff0c;Android平台前后摄像头数据做个简单的汇总
编码前数据目前支持的有YV12/NV21/NV12/I420/RGB24/RGBA32/RGB565等数据类型其中Android平台前后摄像头数据或者屏幕数据或者Unity拿到的数据均属编码前数据编码后数据如无人机等264/HEVC数据或者本地解析的MP4音视频数据拉取RTSP或RTMP流并接入至GB28181平台比如其他IPC的RTSP流可通过Android平台GB28181接入到国标平台。
支持的音视频数据类型
编码前音视频数据
编码前音视频数据支持的类型接口如下除了常规的音视频数据外我们还设计支持了实时动态水印比如实时文字水印、图片水印音频这块我们实现了混音机制支持2路混音输入
/** SmartPublisherJniV2.java* SmartPublisherJniV2** WebSite: https://daniusdk.com* Github: https://github.com/daniulive/SmarterStreaming* * Created by DaniuLive on 2015/09/20.*//*** Set live video data(no encoded data).** param cameraType: CAMERA_FACING_BACK with 0, CAMERA_FACING_FRONT with 1* * param curOrg:* PORTRAIT 1; //竖屏* LANDSCAPE 2; //横屏 home键在右边的情况* LANDSCAPE_LEFT_HOME_KEY 3; //横屏 home键在左边的情况** return {0} if successful*/public native int SmartPublisherOnCaptureVideoData(long handle, byte[] data, int len, int cameraType, int curOrg);/*** YV12数据接口** param data: YV12 data** param width: 图像宽** param height: 图像高** param y_stride: y面步长** param v_stride: v面步长** param u_stride: u面步长** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** return {0} if successful*/public native int SmartPublisherOnYV12Data(long handle, byte[] data, int width, int height, int y_stride, int v_stride, int u_stride, int rotation_degree);/*** NV21数据接口** param data: nv21 data** param len: data length** param width: 图像宽** param height: 图像高** param y_stride: y面步长** param uv_stride: uv面步长** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** return {0} if successful*/public native int SmartPublisherOnNV21Data(long handle, byte[] data, int len, int width, int height, int y_stride, int uv_stride, int rotation_degree);/*** NV21数据接口** param data: nv21 data** param len: data length** param width: 图像宽** param height: 图像高** param y_stride: y面步长** param uv_stride: uv面步长** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** return {0} if successful*/public native int SmartPublisherOnNV21DataV2(long handle, byte[] data, int len, int width, int height, int y_stride, int uv_stride, int rotation_degree,int is_vertical_flip, int is_horizontal_flip);/*** NV21转换到I420并旋转** param src: nv21 data** param dst: 输出I420 data** param width: 图像宽** param height: 图像高** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** return {0} if successful*/public native int SmartPublisherNV21ToI420Rotate(long handle, byte[] src, int src_y_stride, int src_uv_stride, byte[] dst,int dst_y_stride, int dst_u_stride, int dst_v_stride,int width, int height,int rotation_degree);/*** Set live video data(no encoded data).** param data: I420 data* * param len: I420 data length* * param yStride: y stride* * param uStride: u stride* * param vStride: v stride** return {0} if successful*/public native int SmartPublisherOnCaptureVideoI420Data(long handle, byte[] data, int len, int yStride, int uStride, int vStride);/*** 传I420图像接口** param data: I420 data** param width: 图像宽** param height: 图像高** param y_stride: y stride** param u_stride: u stride** param v_stride: v stride** return {0} if successful*/public native int SmartPublisherOnCaptureVideoI420DataV2(long handle, byte[] data, int width, int height, int y_stride, int u_stride, int v_stride);/*** Set live video data(no encoded data).** param buffer: RGB24 data** param length: data length** param rowStride: stride information** param width: width** param height: height** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** param scale_width: 缩放宽必须是8的倍数, 0不缩放** param scale_height: 缩放高, 必须是8的倍数, 0不缩放** param scale_filter_mode: 缩放质量, 范围必须是[1,3], 传0使用默认质量** return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGB24Data(long handle, long buffer, int length, int rowStride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int rotation_degree,int scale_width, int scale_height, int scale_filter_mode);/*** Set live video data(no encoded data).** param buffer: RGBA data** param length: data length** param rowStride: stride information** param width: width** param height: height** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** param scale_width: 缩放宽必须是8的倍数, 0不缩放** param scale_height: 缩放高, 必须是8的倍数, 0不缩放** param scale_filter_mode: 缩放质量, 范围必须是[1,3], 传0使用默认质量** return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGBA32Data(long handle, long buffer, int length, int rowStride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int rotation_degree,int scale_width, int scale_height, int scale_filter_mode);/*** Set live video data(no encoded data).** param data: RGBA data* * param rowStride: stride information* * param width: width* * param height: height** return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height);/*** 投递裁剪过的RGBA数据** param data: RGBA data** param rowStride: stride information** param width: width** param height: height** param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数** return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);/*** Set live video data(no encoded data).** param data: ABGR flip vertical(垂直翻转) data** param rowStride: stride information** param width: width** param height: height** return {0} if successful*/public native int SmartPublisherOnCaptureVideoABGRFlipVerticalData(long handle, ByteBuffer data, int rowStride, int width, int height);/*** Set live video data(no encoded data).** param data: RGB565 data** param row_stride: stride information** param width: width** param height: height** return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGB565Data(long handle,ByteBuffer data, int row_stride, int width, int height);/** 专门为android.media.Image的android.graphics.ImageFormat.YUV_420_888格式提供的接口** param width: 必须是8的倍数** param height: 必须是8的倍数** param crop_left: 剪切左上角水平坐标, 一般根据android.media.Image.getCropRect() 填充** param crop_top: 剪切左上角垂直坐标, 一般根据android.media.Image.getCropRect() 填充** param crop_width: 必须是8的倍数, 填0将忽略这个参数, 一般根据android.media.Image.getCropRect() 填充** param crop_height: 必须是8的倍数, 填0将忽略这个参数一般根据android.media.Image.getCropRect() 填充** param y_plane 对应android.media.Image.Plane[0].getBuffer()** param y_row_stride 对应android.media.Image.Plane[0].getRowStride()** param u_plane 对应android.media.Image.Plane[1].getBuffer()** param v_plane 对应android.media.Image.Plane[2].getBuffer()** param uv_row_stride 对应android.media.Image.Plane[1].getRowStride()** param uv_pixel_stride 对应android.media.Image.Plane[1].getPixelStride()** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是8的倍数, 0不缩放** param scale_height: 缩放高, 必须是8的倍数, 0不缩放** param scale_filter_mode: 缩放质量, 范围必须是[1,3], 传0使用默认速度** return {0} if successful*/public native int SmartPublisherOnImageYUV420888(long handle, int width, int height,int crop_left, int crop_top, int crop_width, int crop_height,ByteBuffer y_plane, int y_row_stride,ByteBuffer u_plane, ByteBuffer v_plane, int uv_row_stride, int uv_pixel_stride,int rotation_degree, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode);/*** 启用或者停用视频层, 这个接口必须在StartXXX之后调用.** param index: 层索引, 必须大于0, 注意第0层不能停用** param is_enable: 是否启用, 0停用, 1启用** return {0} if successful*/public native int EnableLayer(long handle, int index, int is_enable);/*** 移除视频层, 这个接口必须在StartXXX之后调用.** param index: 层索引, 必须大于0, 注意第0层不能移除** return {0} if successful*/public native int RemoveLayer(long handle, int index);/*** 投递层RGBA8888图像,如果不需要Aplpha通道的话, 请使用RGBX8888接口, 效率高** param index: 层索引, 必须大于等于0, 注意:如果index是0的话将忽略Alpha通道** param left: 层叠加的左上角坐标, 对于第0层的话传0** param top: 层叠加的左上角坐标, 对于第0层的话传0** param rgba_plane: rgba 图像数据** param offset: 图像偏移, 这个主要目的是用来做clip的, 一般传0** param row_stride: stride information** param width: width, 必须大于1, 如果是奇数, 将减1** param height: height, 必须大于1, 如果是奇数, 将减1** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是偶数, 0或负数不缩放** param scale_height: 缩放高, 必须是偶数, 0或负数不缩放** param scale_filter_mode: 缩放质量, 传0使用默认速度,可选等级范围是:[1,3],值越大缩放质量越好, 但速度越慢** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270, 注意:旋转是在缩放, 垂直/水品反转之后再做, 请留意顺序** return {0} if successful*/public native int PostLayerImageRGBA8888ByteBuffer(long handle, int index, int left, int top,ByteBuffer rgba_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层RGBA8888图像, 详细说明请参考PostLayerImageRGBA8888ByteBuffer** return {0} if successful*/public native int PostLayerImageRGBA8888ByteArray(long handle, int index, int left, int top,byte[] rgba_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层RGBA8888图像, 详细说明请参考PostLayerImageRGBA8888ByteBuffer** return {0} if successful*/public native int PostLayerImageRGBA8888Native(long handle, int index, int left, int top,long rgba_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层RGBX8888图像** param index: 层索引, 必须大于等于0** param left: 层叠加的左上角坐标, 对于第0层的话传0** param top: 层叠加的左上角坐标, 对于第0层的话传0** param rgbx_plane: rgbx 图像数据** param offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param row_stride: stride information** param width: width, 必须大于1, 如果是奇数, 将减1** param height: height, 必须大于1, 如果是奇数, 将减1** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是偶数, 0或负数不缩放** param scale_height: 缩放高, 必须是偶数, 0或负数不缩放** param scale_filter_mode: 缩放质量, 传0使用默认速度,可选等级范围是:[1,3],值越大缩放质量越好, 但速度越慢** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270, 注意:旋转是在缩放, 垂直/水品反转之后再做, 请留意顺序** return {0} if successful*/public native int PostLayerImageRGBX8888ByteBuffer(long handle, int index, int left, int top,ByteBuffer rgbx_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层RGBX8888图像, 详细说明请参考PostLayerImageRGBX8888ByteBuffer** return {0} if successful*/public native int PostLayerImageRGBX8888ByteArray(long handle, int index, int left, int top,byte[] rgbx_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层RGBX8888图像, 详细说明请参考PostLayerImageRGBX8888ByteBuffer** return {0} if successful*/public native int PostLayerImageRGBX8888Native(long handle, int index, int left, int top,long rgbx_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层RGB888图像** param index: 层索引, 必须大于等于0** param left: 层叠加的左上角坐标, 对于第0层的话传0** param top: 层叠加的左上角坐标, 对于第0层的话传0** param rgb_plane: rgb888 图像数据** param offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param row_stride: stride information** param width: width, 必须大于1, 如果是奇数, 将减1** param height: height, 必须大于1, 如果是奇数, 将减1** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是偶数, 0或负数不缩放** param scale_height: 缩放高, 必须是偶数, 0或负数不缩放** param scale_filter_mode: 缩放质量, 传0使用默认速度,可选等级范围是:[1,3],值越大缩放质量越好, 但速度越慢** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270, 注意:旋转是在缩放, 垂直/水品反转之后再做, 请留意顺序** return {0} if successful*/public native int PostLayerImageRGB888Native(long handle, int index, int left, int top,long rgb_plane, int offset, int row_stride, int width, int height,int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层NV21图像** param index: 层索引, 必须大于等于0** param left: 层叠加的左上角坐标, 对于第0层的话传0** param top: 层叠加的左上角坐标, 对于第0层的话传0** param y_plane: y平面图像数据** param y_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param y_row_stride: stride information** param uv_plane: uv平面图像数据** param uv_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param uv_row_stride: stride information** param width: width, 必须大于1, 且必须是偶数** param height: height, 必须大于1, 且必须是偶数** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是偶数, 0或负数不缩放** param scale_height: 缩放高, 必须是偶数, 0或负数不缩放** param scale_filter_mode: 缩放质量, 传0使用默认速度,可选等级范围是:[1,3],值越大缩放质量越好, 但速度越慢** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270, 注意:旋转是在缩放, 垂直/水品反转之后再做, 请留意顺序** return {0} if successful*/public native int PostLayerImageNV21ByteBuffer(long handle, int index, int left, int top,ByteBuffer y_plane, int y_offset, int y_row_stride,ByteBuffer uv_plane, int uv_offset, int uv_row_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层NV21图像, 详细说明请参考PostLayerImageNV21ByteBuffer** return {0} if successful*/public native int PostLayerImageNV21ByteArray(long handle, int index, int left, int top,byte[] y_plane, int y_offset, int y_row_stride,byte[] uv_plane, int uv_offset, int uv_row_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层NV12图像, 详细说明请参考PostLayerImageNV21ByteBuffer** return {0} if successful*/public native int PostLayerImageNV12ByteBuffer(long handle, int index, int left, int top,ByteBuffer y_plane, int y_offset, int y_row_stride,ByteBuffer uv_plane, int uv_offset, int uv_row_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层NV12图像, 详细说明请参考PostLayerImageNV21ByteBuffer** return {0} if successful*/public native int PostLayerImageNV12ByteArray(long handle, int index, int left, int top,byte[] y_plane, int y_offset, int y_row_stride,byte[] uv_plane, int uv_offset, int uv_row_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层I420图像** param index: 层索引, 必须大于等于0** param left: 层叠加的左上角坐标, 对于第0层的话传0** param top: 层叠加的左上角坐标, 对于第0层的话传0** param y_plane: y平面图像数据** param y_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param y_row_stride: stride information** param u_plane: u平面图像数据** param u_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param u_row_stride: stride information* ** param v_plane: v平面图像数据** param v_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param v_row_stride: stride information** param width: width, 必须大于1, 且必须是偶数** param height: height, 必须大于1, 且必须是偶数** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是偶数, 0或负数不缩放** param scale_height: 缩放高, 必须是偶数, 0或负数不缩放** param scale_filter_mode: 缩放质量, 传0使用默认速度,可选等级范围是:[1,3],值越大缩放质量越好, 但速度越慢** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270, 注意:旋转是在缩放, 垂直/水品反转之后再做, 请留意顺序** return {0} if successful*/public native int PostLayerImageI420ByteBuffer(long handle, int index, int left, int top,ByteBuffer y_plane, int y_offset, int y_row_stride,ByteBuffer u_plane, int u_offset, int u_row_stride,ByteBuffer v_plane, int v_offset, int v_row_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层I420图像, 详细说明请参考PostLayerImageI420ByteBuffer** return {0} if successful*/public native int PostLayerImageI420ByteArray(long handle, int index, int left, int top,byte[] y_plane, int y_offset, int y_row_stride,byte[] u_plane, int u_offset, int u_row_stride,byte[] v_plane, int v_offset, int v_row_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 投递层YUV420888图像, 专门为android.media.Image的android.graphics.ImageFormat.YUV_420_888格式提供的接口** param index: 层索引, 必须大于等于0** param left: 层叠加的左上角坐标, 对于第0层的话传0** param top: 层叠加的左上角坐标, 对于第0层的话传0** param y_plane: 对应android.media.Image.Plane[0].getBuffer()** param y_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param y_row_stride: 对应android.media.Image.Plane[0].getRowStride()** param u_plane: android.media.Image.Plane[1].getBuffer()** param u_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param u_row_stride: android.media.Image.Plane[1].getRowStride()** param v_plane: 对应android.media.Image.Plane[2].getBuffer()** param v_offset: 图像偏移, 这个主要目的是用来做clip的,一般传0** param v_row_stride: 对应android.media.Image.Plane[2].getRowStride()** param uv_pixel_stride: 对应android.media.Image.Plane[1].getPixelStride()** param width: width, 必须大于1, 且必须是偶数** param height: height, 必须大于1, 且必须是偶数** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是偶数, 0或负数不缩放** param scale_height: 缩放高, 必须是偶数, 0或负数不缩放** param scale_filter_mode: 缩放质量, 传0使用默认速度,可选等级范围是:[1,3],值越大缩放质量越好, 但速度越慢** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270, 注意:旋转是在缩放, 垂直/水品反转之后再做, 请留意顺序** return {0} if successful*/public native int PostLayerImageYUV420888ByteBuffer(long handle, int index, int left, int top,ByteBuffer y_plane, int y_offset, int y_row_stride,ByteBuffer u_plane, int u_offset, int u_row_stride,ByteBuffer v_plane, int v_offset, int v_row_stride, int uv_pixel_stride,int width, int height, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode,int rotation_degree);/*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次* * param pcmdata: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* param size: pcm数据大小* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMData(long handle, ByteBuffer pcmdata, int size, int sample_rate, int channel, int per_channel_sample_number);/*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次** param pcmdata: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* param offset: pcmdata的偏移* param size: pcm数据大小* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMDataV2(long handle, ByteBuffer pcmdata, int offset, int size, int sample_rate, int channel, int per_channel_sample_number);/*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次** param pcm_short_array: pcm数据, short是native endian order* param offset: 数组偏移* param len: 数组项数* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMShortArray(long handle, short[] pcm_short_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);/*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次** param pcm_float_array: pcm数据* param offset: 数组偏移* param len: 数组项数* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMFloatArray(long handle, float[] pcm_float_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);/*** 请参考SmartPublisherOnPCMFloatArray*/public native int SmartPublisherOnPCMFloatNative(long handle, long pcm_float_data, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);/*** Set far end pcm data* * param pcmdata : 16bit pcm data* param sampleRate: audio sample rate* param channel: auido channel* param per_channel_sample_number: per channel sample numbers* param is_low_latency: if with 0, it is not low_latency, if with 1, it is low_latency* return {0} if successful*/public native int SmartPublisherOnFarEndPCMData(long handle, ByteBuffer pcmdata, int sampleRate, int channel, int per_channel_sample_number, int is_low_latency);/*** 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次** param stream_index: 当前只能传1, 传其他返回错误* param pcm_data: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* param offset: pcmdata的偏移* param size: pcm数据大小* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}* param channels: 通道, 当前通道支持单通道(1)和双通道(2)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMData(long handle, int stream_index, ByteBuffer pcm_data, int offset, int size, int sample_rate, int channels, int per_channel_sample_number);/*** 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次** param stream_index: 当前只能传1, 传其他返回错误* param pcm_short_array: pcm数据, short是native endian order* param offset: 数组偏移* param len: 数组项数* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}* param channels: 通道, 当前通道支持单通道(1)和双通道(2)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMShortArray(long handle, int stream_index, short[] pcm_short_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);/*** 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次** param stream_index: 当前只能传1, 传其他返回错误* param pcm_float_array: pcm数据* param offset: 数组偏移* param len: 数组项数* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}* param channels: 通道, 当前通道支持单通道(1)和双通道(2)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMFloatArray(long handle, int stream_index, float[] pcm_float_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);/** 请参考SmartPublisherOnMixPCMFloatArray*/public native int SmartPublisherOnMixPCMFloatNative(long handle, int stream_index, long pcm_float_data, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);
编码后音视频数据
像前文所说如无人机等264/HEVC数据或者本地解析的MP4音视频数据均可通过实时流的形式接入到GB28181平台设计接口如下
/*** 设置编码后视频数据(H.264)** param codec_id, H.264对应 1** param data 编码后的video数据** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0.** param timestamp video timestamp** param pts Presentation Time Stamp, 显示时间戳** return {0} if successful*/public native int SmartPublisherPostVideoEncodedData(long handle, int codec_id, ByteBuffer data, int size, int is_key_frame, long timestamp, long pts);/*** 设置编码后视频数据(H.264)** param codec_id, H.264对应 1** param data 编码后的video数据**param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0.** param timestamp video timestamp** param pts Presentation Time Stamp, 显示时间戳** return {0} if successful*/public native int SmartPublisherPostVideoEncodedDataV2(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp, long pts,byte[] sps, int sps_len,byte[] pps, int pps_len);/*** 设置编码后视频数据(H.264)如需录制编码后的数据用此接口且设置实际宽高** param codec_id, H.264对应 1** param data 编码后的video数据**param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0.** param timestamp video timestamp** param pts Presentation Time Stamp, 显示时间戳** param width, height: 编码后视频宽高** return {0} if successful*/public native int SmartPublisherPostVideoEncodedDataV3(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp, long pts,byte[] sps, int sps_len,byte[] pps, int pps_len,int width, int height);/*** 设置音频数据(AAC/PCMA/PCMU/SPEEX)** param codec_id:** NT_MEDIA_CODEC_ID_AUDIO_BASE 0x10000,* NT_MEDIA_CODEC_ID_PCMA NT_MEDIA_CODEC_ID_AUDIO_BASE,* NT_MEDIA_CODEC_ID_PCMU,* NT_MEDIA_CODEC_ID_AAC,* NT_MEDIA_CODEC_ID_SPEEX,* NT_MEDIA_CODEC_ID_SPEEX_NB,* NT_MEDIA_CODEC_ID_SPEEX_WB,* NT_MEDIA_CODEC_ID_SPEEX_UWB,** param data audio数据** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0, audio忽略** param timestamp video timestamp** param parameter_info 用于AAC special config信息填充** param parameter_info_size parameter info size** return {0} if successful*/public native int SmartPublisherPostAudioEncodedData(long handle, int codec_id, ByteBuffer data, int size, int is_key_frame, long timestamp,ByteBuffer parameter_info, int parameter_info_size);/*** 设置音频数据(AAC/PCMA/PCMU/SPEEX)** param codec_id:** NT_MEDIA_CODEC_ID_AUDIO_BASE 0x10000,* NT_MEDIA_CODEC_ID_PCMA NT_MEDIA_CODEC_ID_AUDIO_BASE,* NT_MEDIA_CODEC_ID_PCMU,* NT_MEDIA_CODEC_ID_AAC,* NT_MEDIA_CODEC_ID_SPEEX,* NT_MEDIA_CODEC_ID_SPEEX_NB,* NT_MEDIA_CODEC_ID_SPEEX_WB,* NT_MEDIA_CODEC_ID_SPEEX_UWB,** param data audio数据** param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0, audio忽略** param timestamp video timestamp** param parameter_info 用于AAC special config信息填充** param parameter_info_size parameter info size** return {0} if successful*/public native int SmartPublisherPostAudioEncodedDataV2(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp,byte[] parameter_info, int parameter_info_size);/*** 设置音频数据(AAC/PCMA/PCMU/SPEEX)** param codec_id:** NT_MEDIA_CODEC_ID_AUDIO_BASE 0x10000,* NT_MEDIA_CODEC_ID_PCMA NT_MEDIA_CODEC_ID_AUDIO_BASE,* NT_MEDIA_CODEC_ID_PCMU,* NT_MEDIA_CODEC_ID_AAC,* NT_MEDIA_CODEC_ID_SPEEX,* NT_MEDIA_CODEC_ID_SPEEX_NB,* NT_MEDIA_CODEC_ID_SPEEX_WB,* NT_MEDIA_CODEC_ID_SPEEX_UWB,** param data audio数据** param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0, audio忽略** param timestamp video timestamp** param parameter_info 用于AAC special config信息填充** param parameter_info_size parameter info size** param sample_rate 采样率,如果需要录像的话必须传正确的值**param channels 通道数, 如果需要录像的话必须传正确的值, 一般是1或者2** return {0} if successful*/public native int SmartPublisherPostAudioEncodedDataV3(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp,byte[] parameter_info, int parameter_info_size,int sample_rate, int channels);
拉取RTSP或RTMP流并接入至GB28181平台
这块我在之前的blog有专门提过比如其他IPC的RTSP流可通过Android平台GB28181接入到国标平台。 这张图很好的展示了这块当然如果做的更精细的话还可以把语音对讲和语音广播的也挪过来。
这里简单的说一下具体的实现其实和之前我们做RTSP转RTMP推送一样无非就是把RTSP或者RTMP流数据拉下来然后回调编码后的数据到上层上层根据GB28181数据格式要求实现PS打包然后通过对接GB28181平台信令和数据交互实时推过去即可。
其中拉流实现如下
private boolean StartPull(){if ( isPulling )return false;if (!OpenPullHandle())return false;libPlayer.SmartPlayerSetAudioDataCallback(playerHandle, new PlayerAudioDataCallback());libPlayer.SmartPlayerSetVideoDataCallback(playerHandle, new PlayerVideoDataCallback());int is_pull_trans_code 1;libPlayer.SmartPlayerSetPullStreamAudioTranscodeAAC(playerHandle, is_pull_trans_code);int startRet libPlayer.SmartPlayerStartPullStream(playerHandle);if (startRet ! 0) {Log.e(TAG, Failed to start pull stream!);if(!isPlaying !isRecording isPushing !isRTSPPublisherRunning){libPlayer.SmartPlayerClose(playerHandle);playerHandle 0;}return false;}isPulling true;return true;}private void StopPull(){if ( !isPulling )return;libPlayer.SmartPlayerStopPullStream(playerHandle);if ( !isPlaying !isRecording !isPushing !isRTSPPublisherRunning){libPlayer.SmartPlayerClose(playerHandle);playerHandle 0;}isPulling false;}
拉到的音视频数据投递到GB28181接入模块即可
class PlayerAudioDataCallback implements NTAudioDataCallback{private int audio_buffer_size 0;private int param_info_size 0;private ByteBuffer audio_buffer_ null;private ByteBuffer parameter_info_ null;Overridepublic ByteBuffer getAudioByteBuffer(int size){//Log.i(getAudioByteBuffer, size: size);if( size 1 ){return null;}if ( size audio_buffer_size audio_buffer_ ! null ){return audio_buffer_;}audio_buffer_size size 512;audio_buffer_size (audio_buffer_size0xf) (~0xf);audio_buffer_ ByteBuffer.allocateDirect(audio_buffer_size);// Log.i(getAudioByteBuffer, size: size buffer_size: audio_buffer_size);return audio_buffer_;}Overridepublic ByteBuffer getAudioParameterInfo(int size){//Log.i(getAudioParameterInfo, size: size);if(size 1){return null;}if ( size param_info_size parameter_info_ ! null ){return parameter_info_;}param_info_size size 32;param_info_size (param_info_size0xf) (~0xf);parameter_info_ ByteBuffer.allocateDirect(param_info_size);//Log.i(getAudioParameterInfo, size: size buffer_size: param_info_size);return parameter_info_;}public void onAudioDataCallback(int ret, int audio_codec_id, int sample_size, int is_key_frame, long timestamp, int sample_rate, int channel, int parameter_info_size, long reserve){//Log.i(onAudioDataCallback, ret: ret , audio_codec_id: audio_codec_id , sample_size: sample_size , timestamp: timestamp // ,sample_rate: sample_rate);if ( audio_buffer_ null)return;audio_buffer_.rewind();if ( ret 0 (isPushing || isRTSPPublisherRunning || isGB28181StreamRunning)) {libPublisher.SmartPublisherPostAudioEncodedData(publisherHandle, audio_codec_id, audio_buffer_, sample_size, is_key_frame, timestamp, parameter_info_, parameter_info_size);}// test/*byte[] test_buffer new byte[16];pcm_buffer_.get(test_buffer);Log.i(TAG, onGetPcmFrame data: bytesToHexString(test_buffer));*/}}class PlayerVideoDataCallback implements NTVideoDataCallback{private int video_buffer_size 0;private ByteBuffer video_buffer_ null;Overridepublic ByteBuffer getVideoByteBuffer(int size){//Log.i(getVideoByteBuffer, size: size);if( size 1 ){return null;}if ( size video_buffer_size video_buffer_ ! null ){return video_buffer_;}video_buffer_size size 1024;video_buffer_size (video_buffer_size0xf) (~0xf);video_buffer_ ByteBuffer.allocateDirect(video_buffer_size);// Log.i(getVideoByteBuffer, size: size buffer_size: video_buffer_size);return video_buffer_;}public void onVideoDataCallback(int ret, int video_codec_id, int sample_size, int is_key_frame, long timestamp, int width, int height, long presentation_timestamp){//Log.i(onVideoDataCallback, ret: ret , video_codec_id: video_codec_id , sample_size: sample_size , is_key_frame: is_key_frame , timestamp: timestamp // ,presentation_timestamp: presentation_timestamp);if ( video_buffer_ null)return;video_buffer_.rewind();if ( ret 0 (isPushing || isRTSPPublisherRunning || isGB28181StreamRunning) ) {libPublisher.SmartPublisherPostVideoEncodedData(publisherHandle, video_codec_id, video_buffer_, sample_size, is_key_frame, timestamp, presentation_timestamp);}}}
总结
通过以上描述大家可以看到GB/T 28181音视频数据源接入无非就是上述类型有RTSP或RTMP音视频开发经验的开发者来说实现GB28181数据接入基本上可以复用之前的设计。
