注册域名后如何建立网站,做内衣模特接广告网站,网络营销策划书1000字,免费ppt模板下载app许多开发者在做智慧教室同屏亦或会议同屏时#xff0c;基于Andriod平台采集屏幕并编码推送#xff0c;往往遇到各种各样的问题#xff0c;以下就我们开发过程中的一些技术考量做个分享#xff0c;权当抛砖引玉#xff1a;
协议选择、数据来源和处理
1. 内网环境下#…许多开发者在做智慧教室同屏亦或会议同屏时基于Andriod平台采集屏幕并编码推送往往遇到各种各样的问题以下就我们开发过程中的一些技术考量做个分享权当抛砖引玉
协议选择、数据来源和处理
1. 内网环境下组播还是RTMP
回答这个问题被无数的开发者问到为此单独写了篇博客论证https://blog.csdn.net/renhui1112/article/details/86741428感兴趣的可以参考下简单来说能RTMP的就RTMP如果真是内网环境下没有并发瓶颈的同屏可以启动内置RTSP服务走单播然后其他终端拉流也不失为一个好的方案。
2. 推送分辨率如何设定或缩放
回答一般来说好多Android设备特别是高分屏拿到的视频原始宽高非常大如果推原始分辨率编码和上行压力大所以一般建议适当缩放比如宽高缩放至2/3缩放一般建议等比例缩放此外缩放宽高建议16字节对齐。
废话不多说上实例代码 private void createScreenEnvironment() {sreenWindowWidth mWindowManager.getDefaultDisplay().getWidth();screenWindowHeight mWindowManager.getDefaultDisplay().getHeight();Log.i(TAG, screenWindowWidth: sreenWindowWidth ,screenWindowHeight: screenWindowHeight);if (sreenWindowWidth 800){if (screenResolution SCREEN_RESOLUTION_STANDARD){scale_rate SCALE_RATE_HALF;sreenWindowWidth align(sreenWindowWidth / 2, 16);screenWindowHeight align(screenWindowHeight / 2, 16);}else if(screenResolution SCREEN_RESOLUTION_LOW){scale_rate SCALE_RATE_TWO_FIFTHS;sreenWindowWidth align(sreenWindowWidth * 2 / 5, 16);}}Log.i(TAG, After adjust mWindowWidth: sreenWindowWidth , mWindowHeight: screenWindowHeight);int pf mWindowManager.getDefaultDisplay().getPixelFormat();Log.i(TAG, display format: pf);DisplayMetrics displayMetrics new DisplayMetrics();mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);mScreenDensity displayMetrics.densityDpi;mImageReader ImageReader.newInstance(sreenWindowWidth,screenWindowHeight, 0x1, 6);mMediaProjectionManager (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);}
3. 横竖屏自动适配
回答因为横竖屏状态下采集的屏幕宽高不一样如果横竖屏切换这个时候需要考虑到横竖屏适配问题确保比如竖屏状态下切换到横屏时推拉流两端可以自动适配横竖屏自动适配编码器需要重启拉流端需要能自动适配宽高变化自动播放。
4. 一定的补帧策略
回答好多人不理解为什么要补帧实际上屏幕采集的时候屏幕不动的话不会一直有数据下去这个时候比较好的做法是保存最后一帧数据设定一定的补帧间隔确保不会因为帧间距太大导致播放端几秒都收不到数据当然如果服务器可以缓存GOP这个问题迎刃而解。
5. 异常网络处理、事件回调机制
回答如果是走RTMP网络抖动或者其他网络异常需要有好重连机制和状态回馈机制。 class EventHandeV2 implements NTSmartEventCallbackV2 {Overridepublic void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {Log.i(TAG, EventHandeV2: handle handle id: id);String publisher_event ;switch (id) {case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:publisher_event 开始..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:publisher_event 连接中..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:publisher_event 连接失败..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:publisher_event 连接成功..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:publisher_event 连接断开..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:publisher_event 关闭..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:publisher_event 开始一个新的录像文件 : param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:publisher_event 已生成一个录像文件 : param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:publisher_event 发送时延: param1 帧数: param2;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:publisher_event 快照: param1 路径 param3;if (param1 0) {publisher_event publisher_event 截取快照成功..;} else {publisher_event publisher_event 截取快照失败..;}break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:publisher_event RTSP服务URL: param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:publisher_event RTSP status code received, codeID: param1 , RTSP URL: param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:publisher_event 服务器不支持RTSP推送, 推送的RTSP URL: param3;break;}String str 当前回调状态 publisher_event;Log.i(TAG, str);Message message new Message();message.what PUBLISHER_EVENT_MSG;message.obj publisher_event;handler.sendMessage(message);}}
6. 部分屏幕数据采集
回答我们遇到的好多场景下教室端会拿出来3/4的区域用来投递给学生看1/4的区域用来做一些指令等操作这个时候就需要考虑屏幕区域裁剪接口可做如下设计 /*** 投递裁剪过的RGBA数据** param data: RGBA data** param rowStride: stride information** param width: width** param height: height** param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数** return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);//实际裁剪比例可酌情自行调整int left 100;int cliped_left 0;int top 0;int cliped_top 0;int cliped_width width_;int cliped_height height_;if(scale_rate SCALE_RATE_HALF){cliped_left left / 2;cliped_top top / 2;//宽度裁剪后展示3/4比例cliped_width (width_ *3)/4;//高度不做裁剪cliped_height height_;}else if(scale_rate SCALE_RATE_TWO_FIFTHS){cliped_left left * 2 / 5;cliped_top top * 2 / 5;//宽度裁剪后展示3/4比例cliped_width (width_ *3)/4;//高度不做裁剪cliped_height height_;}if(cliped_width % 2 ! 0){cliped_width cliped_width 1;}if(cliped_height % 2 ! 0){cliped_height cliped_height 1;}if ( (cliped_left cliped_width) width_){Log.e(TAG, invalid cliped region settings, cliped_left: cliped_left cliped_width: cliped_width width: width_);return;}if ( (cliped_top cliped_height) height_){Log.e(TAG, invalid cliped region settings, cliped_top: cliped_top cliped_height: cliped_height height: height_);return;}//Log.i(TAG, clipLeft: cliped_left clipTop: cliped_top clipWidth: cliped_width clipHeight: cliped_height);libPublisher.SmartPublisherOnCaptureVideoClipedRGBAData(publisherHandle, last_buffer, row_stride_,width_, height_, cliped_left, cliped_top, cliped_width, cliped_height );
7. 文字、图片水印
回答好多场景下同屏者会把公司logo和一定的文字信息展示在推送端这个时候需要考虑到文字和图片水印问题具体可参考如下接口设置 /*** Set Text water-mark(设置文字水印)* * param fontSize: it should be MEDIUM, SMALL, BIG* * param waterPostion: it should be TOPLEFT, TOPRIGHT, BOTTOMLEFT, BOTTOMRIGHT.* * param xPading, yPading: the distance of the original picture.* * pre The interface is only used for setting font water-mark when publishing stream. /pre * * return {0} if successful*/public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);/*** Set Text water-mark font file name(设置文字水印字体路径)** param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf** return {0} if successful*/public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);/*** Set picture water-mark(设置png图片水印)* * param picPath: the picture working path, e.g: /sdcard/logo.png* * param waterPostion: it should be TOPLEFT, TOPRIGHT, BOTTOMLEFT, BOTTOMRIGHT.* * param picWidth, picHeight: picture width height* * param xPading, yPading: the distance of the original picture.* * pre The interface is only used for setting picture(logo) water-mark when publishing stream, with *.png format /pre * * return {0} if successful*/public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);
如何通过RTMP或RTSP协议发送屏幕数据
RTMP或RTSP方案有很多开源的选择不过大多是demo级如果产品化自填研坑需要很大的精力以下以大牛直播SDK为例介绍下如何对接数据好多开发者提到为什么大牛直播SDK的Android平台RTMP推送接口怎么这么多不像一些开源或者商业RTMP推送一样就几个接口简单明了。废话不多说感兴趣的可以参考下 Github
1. 初始化Publisher接口返回推送实例句柄 /*** Open publisher(启动推送实例)** param ctx: get by this.getApplicationContext()* * param audio_opt:* if 0: 不推送音频* if 1: 推送编码前音频(PCM)* if 2: 推送编码后音频(aac/pcma/pcmu/speex).* * param video_opt:* if 0: 不推送视频* if 1: 推送编码前视频(YUV420SP/YUV420P/RGBA/ARGB)* if 2: 推送编码后视频(H.264)** param width: capture width; height: capture height.** preThis function must be called firstly./pre** return the handle of publisher instance*/public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt, int width, int height);
2. 相关Event事件回调如网络状态、实时快照、录像状态等回调 /*** Set callback event(设置事件回调)* * param callbackv2: callback function* * return {0} if successful*/public native int SetSmartPublisherEventCallbackV2(long handle, NTSmartEventCallbackV2 callbackv2); class EventHandeV2 implements NTSmartEventCallbackV2 {Overridepublic void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {Log.i(TAG, EventHandeV2: handle handle id: id);String publisher_event ;switch (id) {case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:publisher_event 开始..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:publisher_event 连接中..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:publisher_event 连接失败..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:publisher_event 连接成功..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:publisher_event 连接断开..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:publisher_event 关闭..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:publisher_event 开始一个新的录像文件 : param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:publisher_event 已生成一个录像文件 : param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:publisher_event 发送时延: param1 帧数: param2;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:publisher_event 快照: param1 路径 param3;if (param1 0) {publisher_event publisher_event 截取快照成功..;} else {publisher_event publisher_event 截取快照失败..;}break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:publisher_event RTSP服务URL: param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:publisher_event RTSP status code received, codeID: param1 , RTSP URL: param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:publisher_event 服务器不支持RTSP推送, 推送的RTSP URL: param3;break;}String str 当前回调状态 publisher_event;Log.i(TAG, str);Message message new Message();message.what PUBLISHER_EVENT_MSG;message.obj publisher_event;handler.sendMessage(message);}}
3. 检测/设置软硬编码 /*** Set Video H.264 HW Encoder, if support HW encoder, it will return 0(设置H.264硬编码)* * param kbps: the kbps of different resolution(25 fps).* * return {0} if successful*/public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);/*** Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(设置H.265硬编码)** param kbps: the kbps of different resolution(25 fps).** return {0} if successful*/public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);
4. 设置文字水印、PNG图片水印 /*** Set Text water-mark(设置文字水印)* * param fontSize: it should be MEDIUM, SMALL, BIG* * param waterPostion: it should be TOPLEFT, TOPRIGHT, BOTTOMLEFT, BOTTOMRIGHT.* * param xPading, yPading: the distance of the original picture.* * pre The interface is only used for setting font water-mark when publishing stream. /pre * * return {0} if successful*/public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);/*** Set Text water-mark font file name(设置文字水印字体路径)** param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf** return {0} if successful*/public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);/*** Set picture water-mark(设置png图片水印)* * param picPath: the picture working path, e.g: /sdcard/logo.png* * param waterPostion: it should be TOPLEFT, TOPRIGHT, BOTTOMLEFT, BOTTOMRIGHT.* * param picWidth, picHeight: picture width height* * param xPading, yPading: the distance of the original picture.* * pre The interface is only used for setting picture(logo) water-mark when publishing stream, with *.png format /pre * * return {0} if successful*/public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);
5. 软编码可变码率设置如摄像头采集编码在设备性能允许的情况下可以考虑软编码可变码率 /*** Set software encode vbr mode(软编码可变码率).** preplease set before SmartPublisherStart while after SmartPublisherOpen./pre** is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr** video_quality: vbr video quality, range with (1,50), default 23** vbr_max_kbitrate: vbr max encode bit-rate(kbps)** return {0} if successful*/public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);6. 帧率、软编码码率、关键帧间隔编码速度、软编码profile等基础设定 /*** Set gop interval(设置I帧间隔)** preplease set before SmartPublisherStart while after SmartPublisherOpen./pre** gopInterval: encode I frame interval, the value always 0** return {0} if successful*/public native int SmartPublisherSetGopInterval(long handle, int gopInterval);/*** Set software encode video bit-rate(设置视频软编码bit-rate)** preplease set before SmartPublisherStart while after SmartPublisherOpen./pre** avgBitRate: average encode bit-rate(kbps)* * maxBitRate: max encode bit-rate(kbps)** return {0} if successful*/public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);/*** Set fps(设置帧率)** preplease set before SmartPublisherStart while after SmartPublisherOpen./pre** fps: the fps of video, range with (1,25).** return {0} if successful*/public native int SmartPublisherSetFPS(long handle, int fps);/*** Set software video encoder profile(设置视频编码profile).** preplease set before SmartPublisherStart while after SmartPublisherOpen./pre** profile: the software video encoder profile, range with (1,3).* * 1: baseline profile* 2: main profile* 3: high profile** return {0} if successful*/public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);/*** Set software video encoder speed(设置视频软编码编码速度)* * preplease set before SmartPublisherStart while after SmartPublisherOpen./pre* * param speed: range with(1, 6), the default speed is 6. * * if with 1, CPU is lowest.* if with 6, CPU is highest.* * return {0} if successful*/public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);
7. 设置audio编码类型如AAC编码、Speex编码并设置如AAC编码码率等参数 /*** Set audio encoder type(设置音频编码类型)* * param type: if with 1:AAC, if with 2: SPEEX* * return {0} if successful*/public native int SmartPublisherSetAudioCodecType(long handle, int type);/*** Set audio encoder bit-rate(设置音频编码码率), 当前只对AAC编码有效** param kbit_rate: 码率(单位是kbps), 如果是0的话将使用默认码率, 必须大于等于0** return {0} if successful*/public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);/*** Set speex encoder quality(设置speex编码质量)* * param quality: range with (0, 10), default value is 8* * return {0} if successful*/public native int SmartPublisherSetSpeexEncoderQuality(long handle, int quality);
8. 音频处理如自动增益控制、噪音抑制 /*** Set Audio Noise Suppression(设置音频噪音抑制)* * param isNS: if with 1:suppress, if with 0: does not suppress* * return {0} if successful*/public native int SmartPublisherSetNoiseSuppression(long handle, int isNS);/*** Set Audio AGC(设置音频自动增益控制)* * param isAGC: if with 1:AGC, if with 0: does not AGC* * return {0} if successful*/public native int SmartPublisherSetAGC(long handle, int isAGC);
9. 音频混音接口设置 /*** 设置混音,目前支持两路音频混音** param is_mix: 1混音, 0不混音, 默认不混音** return {0} if successful*/public native int SmartPublisherSetAudioMix(long handle, int is_mix);
10. 实时静音设置 /*** Set mute or not during publish stream(设置实时静音)* * param isMute: if with 1:mute, if with 0: does not mute* * return {0} if successful*/public native int SmartPublisherSetMute(long handle, int isMute);11. 音量调整 /*** 设置输入音量, 这个接口一般不建议调用, 在一些特殊情况下可能会用, 一般不建议放大音量** param index: 一般是0和1, 如果没有混音的只用0, 有混音的话, 0,1分别设置音量** param volume: 音量默认是1.0范围是[0.0, 5.0], 设置成0静音, 1音量不变** return {0} if successful*/public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);
12. 前置摄像头镜像 /*** Set mirror(设置前置摄像头镜像)* * param isMirror: if with 1:mirror mode, if with 0: normal mode* * Please note when with mirror mode, the publisher and player with the same echo direction* * return {0} if successful*/public native int SmartPublisherSetMirror(long handle, int isMirror);
13. 录像相关配置如只录制音频或视频单个文件最大size录像存放目录开始/暂停/停止录像 /*** 音频录制开关, 目的是为了更细粒度的去控制录像, 一般不需要调用这个接口, 这个接口使用场景比如同时推送音视频但只想录制视频可以调用这个接口关闭音频录制** param is_recoder: 0: do not recorder; 1: recorder; sdk默认是1** return {0} if successful*/public native int SmartPublisherSetRecorderAudio(long handle, int is_recoder);/*** 视频录制开关, 目的是为了更细粒度的去控制录像, 一般不需要调用这个接口, 这个接口使用场景比如同时推送音视频但只想录制音频可以调用这个接口关闭视频录制** param is_recoder: 0: do not recorder; 1: recorder; sdk默认是1** return {0} if successful*/public native int SmartPublisherSetRecorderVideo(long handle, int is_recoder);/*** Create file directory(创建录像存放目录)* * param path, E.g: /sdcard/daniulive/rec* * pre The interface is only used for recording the stream data to local side. /pre * * return {0} if successful*/public native int SmartPublisherCreateFileDirectory(String path);/*** Set recorder directory(设置录像存放目录)* * param path: the directory of recorder file.* * pre NOTE: make sure the path should be existed, or else the setting failed. /pre* * return {0} if successful*/public native int SmartPublisherSetRecorderDirectory(long handle, String path);/*** Set the size of every recorded file(设置单个录像文件大小如超过最大文件大小自动切换到下个文件录制)* * param size: (MB), (5M~500M), if not in this range, set default size with 200MB.* * return {0} if successful*/public native int SmartPublisherSetRecorderFileMaxSize(long handle, int size);/*** Start recorder(开始录像)** return {0} if successful*/public native int SmartPublisherStartRecorder(long handle);/*** Pause recorder(暂停/恢复录像)** is_pause: 1表示暂停, 0表示恢复录像, 输入其他值将调用失败** return {0} if successful*/public native int SmartPublisherPauseRecorder(long handle, int is_pause);/*** Stop recorder(停止录像)** return {0} if successful*/public native int SmartPublisherStopRecorder(long handle);
14. 实时快照 /*** Set if needs to save image during publishing stream(设置是否启用快照)** param is_save_image: if with 1, it will save current image via the interface of SmartPlayerSaveImage(), if with 0: does not it** return {0} if successful*/public native int SmartPublisherSaveImageFlag(long handle, int is_save_image);/*** Save current image during publishing stream(实时快照)** param imageName: image name, which including fully path, /sdcard/daniuliveimage/daniu.png, etc.** return {0} if successful*/public native int SmartPublisherSaveCurImage(long handle, String imageName);
15. 设置推送的RTMP URL /*** Set rtmp publish stream url(设置推送的RTMP url)** param url: rtmp publish url.** return {0} if successful*/public native int SmartPublisherSetURL(long handle, String url);
16. Android摄像头前后camera通过OnPreviewFrame()回调的数据接口 Overridepublic void onPreviewFrame(byte[] data, Camera camera) {frameCount;if (frameCount % 3000 0) {Log.i(OnPre, gc);System.gc();Log.i(OnPre, gc-);}if (data null) {Parameters params camera.getParameters();Size size params.getPreviewSize();int bufferSize (((size.width | 0x1f) 1) * size.height * ImageFormat.getBitsPerPixel(params.getPreviewFormat())) / 8;camera.addCallbackBuffer(new byte[bufferSize]);} else {if (isRTSPPublisherRunning || isPushingRtmp || isRecording || isPushingRtsp) {libPublisher.SmartPublisherOnCaptureVideoData(publisherHandle, data, data.length, currentCameraType, currentOrigentation);}camera.addCallbackBuffer(data);}} 对应接口定义
/*** Set live video data(no encoded data).** param cameraType: CAMERA_FACING_BACK with 0, CAMERA_FACING_FRONT with 1* * param curOrg:* PORTRAIT 1; //竖屏* LANDSCAPE 2; //横屏 home键在右边的情况* LANDSCAPE_LEFT_HOME_KEY 3; //横屏 home键在左边的情况** return {0} if successful*/public native int SmartPublisherOnCaptureVideoData(long handle, byte[] data, int len, int cameraType, int curOrg);
17. 部分定制设备只支持YV12的数据 /*** YV12数据接口** param data: YV12 data** param width: 图像宽** param height: 图像高** param y_stride: y面步长** param v_stride: v面步长** param u_stride: u面步长** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** return {0} if successful*/public native int SmartPublisherOnYV12Data(long handle, byte[] data, int width, int height, int y_stride, int v_stride, int u_stride, int rotation_degree);
18. 支持NV21数据接口
nv21数据接口除了用于常规的camera数据接入外部分定制摄像头出来的数据发生翻转这个接口也支持。 /*** NV21数据接口** param data: nv21 data** param len: data length** param width: 图像宽** param height: 图像高** param y_stride: y面步长** param uv_stride: uv面步长** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** return {0} if successful*/public native int SmartPublisherOnNV21Data(long handle, byte[] data, int len, int width, int height, int y_stride, int uv_stride, int rotation_degree);/*** NV21数据接口** param data: nv21 data** param len: data length** param width: 图像宽** param height: 图像高** param y_stride: y面步长** param uv_stride: uv面步长** rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** return {0} if successful*/public native int SmartPublisherOnNV21DataV2(long handle, byte[] data, int len, int width, int height, int y_stride, int uv_stride, int rotation_degree,int is_vertical_flip, int is_horizontal_flip);
19. 支持YUV数据接入 /*** Set live video data(no encoded data).** param data: I420 data* * param len: I420 data length* * param yStride: y stride* * param uStride: u stride* * param vStride: v stride** return {0} if successful*/public native int SmartPublisherOnCaptureVideoI420Data(long handle, byte[] data, int len, int yStride, int uStride, int vStride); 20. 支持RGBA数据接入支持裁剪后数据接入主要用于同屏场景 /*** Set live video data(no encoded data).** param data: RGBA data* * param rowStride: stride information* * param width: width* * param height: height** return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height);/*** 投递裁剪过的RGBA数据** param data: RGBA data** param rowStride: stride information** param width: width** param height: height** param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数** return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);/*** Set live video data(no encoded data).** param data: ABGR flip vertical(垂直翻转) data** param rowStride: stride information** param width: width** param height: height** return {0} if successful*/public native int SmartPublisherOnCaptureVideoABGRFlipVerticalData(long handle, ByteBuffer data, int rowStride, int width, int height);
21. 支持RGB565数据接入主要用于同屏场景 /*** Set live video data(no encoded data).** param data: RGB565 data** param row_stride: stride information** param width: width** param height: height** return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGB565Data(long handle,ByteBuffer data, int row_stride, int width, int height);
22. 支持camera数据接入主要用于camera2接口对接
/** 专门为android.media.Image的android.graphics.ImageFormat.YUV_420_888格式提供的接口** param width: 必须是8的倍数** param height: 必须是8的倍数** param crop_left: 剪切左上角水平坐标, 一般根据android.media.Image.getCropRect() 填充** param crop_top: 剪切左上角垂直坐标, 一般根据android.media.Image.getCropRect() 填充** param crop_width: 必须是8的倍数, 填0将忽略这个参数, 一般根据android.media.Image.getCropRect() 填充** param crop_height: 必须是8的倍数, 填0将忽略这个参数一般根据android.media.Image.getCropRect() 填充** param y_plane 对应android.media.Image.Plane[0].getBuffer()** param y_row_stride 对应android.media.Image.Plane[0].getRowStride()** param u_plane 对应android.media.Image.Plane[1].getBuffer()** param v_plane 对应android.media.Image.Plane[2].getBuffer()** param uv_row_stride 对应android.media.Image.Plane[1].getRowStride()** param uv_pixel_stride 对应android.media.Image.Plane[1].getPixelStride()** param rotation_degree: 顺时针旋转, 必须是0, 90, 180, 270** param is_vertical_flip: 是否垂直翻转, 0不翻转, 1翻转** param is_horizontal_flip是否水平翻转, 0不翻转, 1翻转** param scale_width: 缩放宽必须是8的倍数, 0不缩放** param scale_height: 缩放高, 必须是8的倍数, 0不缩放** param scale_filter_mode: 缩放质量, 范围必须是[1,3], 传0使用默认速度** return {0} if successful*/public native int SmartPublisherOnImageYUV420888(long handle, int width, int height,int crop_left, int crop_top, int crop_width, int crop_height,ByteBuffer y_plane, int y_row_stride,ByteBuffer u_plane, ByteBuffer v_plane, int uv_row_stride, int uv_pixel_stride,int rotation_degree, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode);
23. 支持PCM数据接入 /*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次* * param pcmdata: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* param size: pcm数据大小* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMData(long handle, ByteBuffer pcmdata, int size, int sample_rate, int channel, int per_channel_sample_number);/*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次** param pcmdata: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* param offset: pcmdata的偏移* param size: pcm数据大小* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMDataV2(long handle, ByteBuffer pcmdata, int offset, int size, int sample_rate, int channel, int per_channel_sample_number);/*** 传递PCM音频数据给SDK, 每10ms音频数据传入一次** param pcm_short_array: pcm数据, short是native endian order* param offset: 数组偏移* param len: 数组项数* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推荐44100* param channel: 通道, 当前通道支持单通道(1)和双通道(2)推荐单通道(1)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnPCMShortArray(long handle, short[] pcm_short_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);
24. 支持远端PCM数据接入和混音后PCM数据接入主要用于一对一互动
/*** Set far end pcm data* * param pcmdata : 16bit pcm data* param sampleRate: audio sample rate* param channel: auido channel* param per_channel_sample_number: per channel sample numbers* param is_low_latency: if with 0, it is not low_latency, if with 1, it is low_latency* return {0} if successful*/public native int SmartPublisherOnFarEndPCMData(long handle, ByteBuffer pcmdata, int sampleRate, int channel, int per_channel_sample_number, int is_low_latency);/*** 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次** param stream_index: 当前只能传1, 传其他返回错误* param pcm_data: pcm数据, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* param offset: pcmdata的偏移* param size: pcm数据大小* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}* param channels: 通道, 当前通道支持单通道(1)和双通道(2)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMData(long handle, int stream_index, ByteBuffer pcm_data, int offset, int size, int sample_rate, int channels, int per_channel_sample_number);/*** 传递PCM混音音频数据给SDK, 每10ms音频数据传入一次** param stream_index: 当前只能传1, 传其他返回错误* param pcm_short_array: pcm数据, short是native endian order* param offset: 数组偏移* param len: 数组项数* param sample_rate: 采样率当前只支持{44100, 8000, 16000, 24000, 32000, 48000}* param channels: 通道, 当前通道支持单通道(1)和双通道(2)* param per_channel_sample_number: 这个请传入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMShortArray(long handle, int stream_index, short[] pcm_short_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);
25. H264扩展SEI消息 /*** 设置发送队列大小为保证实时性默认大小为3, 必须设置一个大于0的数** param max_size: 队列最大长度** param reserve: 保留字段** NOTE: 1. 如果数据超过队列大小将丢掉队头数据; 2. 这个接口请在 StartPublisher 之前调用** return {0} if successful*/public native int SmartPublisherSetPostUserDataQueueMaxSize(long handle, int max_size, int reserve);/*** 清空用户数据队列, 有些情况可能会用到比如发送队列里面有4条消息再等待发送,又想把最新的消息快速发出去, 可以先清除掉正在排队消息, 再调用PostUserXXX** return {0} if successful*/public native int SmartPublisherClearPostUserDataQueue(long handle);/*** 发送二进制数据** NOTE:* 1.目前数据大小限制在256个字节以内太大可能会影响视频传输如果有特殊需求需要增大限制请联系我们* 2. 如果积累的数据超过了设置的队列大小之前的队头数据将被丢弃* 3. 必须再调用StartPublisher之后再发送数据** param data: 二进制数据** param size: 数据大小** param reserve: 保留字段** return {0} if successful*/public native int SmartPublisherPostUserData(long handle, byte[] data, int size, int reserve);/*** 发送utf8字符串** NOTE:* 1. 字符串长度不能超过256, 太大可能会影响视频传输如果有特殊需求需要增大限制请联系我们* 2. 如果积累的数据超过了设置的队列大小之前的队头数据将被丢弃* 3. 必须再调用StartPublisher之后再发送数据** param utf8_str: utf8字符串** param reserve: 保留字段** return {0} if successful*/public native int SmartPublisherPostUserUTF8StringData(long handle, String utf8_str, int reserve);
26. 关闭推送实例 /*** 关闭推送实例结束时必须调用close接口释放资源** return {0} if successful*/public native int SmartPublisherClose(long handle);
