一个演示用的基于表情识别和敲击识别的App
演示用app,表情识别基于微软人脸识别API,可以在点击表情后自动多次拍照,拍照界面已经做了隐藏处理,当与预定义的表情序列匹配后人之成功。
敲击识别的功能上也差不多,是基于加速传感器和录音机实现的,保证较高的准确率,可以设置识别的敲击次数。
下载地址:
表情识别
基本框架基于微软表情识别基于微软人脸识别API
注:API免费试用一个月申请地址https://azure.microsoft.com/zh-cn/try/cognitive-services/?unauthorized=1
我之前申请的API已经过期,可以在上面的网址申请一个在Cognitive-Face-Android-master\Sample\app\src\main\res\values\strings.xml 中
<!-- Please refer to https://azure.microsoft.com/en-us/services/cognitive-services/face/ to get your subscription key -->
<!-- If you have subscription key with its corresponding endpoint, you can add them here to use the service -->
<string name="subscription_key">26e66675269944f4a27656d35afd8026</string>
<string name="endpoint">https://westcentralus.api.cognitive.microsoft.com/face/v1.0</string>
这里添加自己的key即可。
思路就是在本地用前置摄像头拍一张图片,上传到微软的API,会返回一个人脸的各种信息,从中提取出表情的信息,在界面上显示表情所对应的字符。
相机是做了一个隐藏式的自动连续拍照的相机,用的谷歌官方的自定义相机API。
核心代码:
// Background task of face detection.
private class DetectionTask extends AsyncTask<InputStream, String, Face[]> {
private boolean mSucceed = true;
@Override
protected Face[] doInBackground(InputStream... params) {
// Get an instance of face service client to detect faces in image.
FaceServiceClient faceServiceClient = SampleApp.getFaceServiceClient();
try {
publishProgress("Detecting...");
// Start detection.
return faceServiceClient.detect(
params[0], /* Input stream of image to detect */
true, /* Whether to return face ID */
true, /* Whether to return face landmarks */
/* Which face attributes to analyze, currently we support:
age,gender,headPose,smile,facialHair */
new FaceServiceClient.FaceAttributeType[] {
FaceServiceClient.FaceAttributeType.Emotion});
} catch (Exception e) {
mSucceed = false;
publishProgress(e.getMessage());
addLog(e.getMessage());
return null;
}
}
@Override
protected void onPreExecute() {
mProgressDialog.show();
addLog("Request: Detecting in image " + mImageUri);
}
@Override
protected void onProgressUpdate(String... progress) {
mProgressDialog.setMessage(progress[0]);
setInfo(progress[0]);
}
@Override
protected void onPostExecute(Face[] result) {
if (mSucceed) {
addLog("Response: Success. Detected " + (result == null ? 0 : result.length)
+ " face(s) in " + mImageUri);
}
// Show the result on screen when detection is done.
setUiAfterDetection(result, mSucceed);
}
}
将传回的文本提取,根据不同表情返回相应的符号:
public View getView(final int position, View convertView, ViewGroup parent) {
if (convertView == null) {
LayoutInflater layoutInflater =
(LayoutInflater) getSystemService(Context.LAYOUT_INFLATER_SERVICE);
convertView = layoutInflater.inflate(R.layout.item_face_with_description, parent, false);
}
convertView.setId(position);
// Show the face thumbnail.
((ImageView) convertView.findViewById(R.id.face_thumbnail)).setImageBitmap(
faceThumbnails.get(position));
// Show the face details.
emotion = getEmotion(faces.get(position).faceAttributes.emotion).substring(0, 2);
input(view);
return convertView;
}
private String getEmotion(Emotion emotion)
{
String emotionType = "";
double emotionValue = 0.0;
if (emotion.anger > emotionValue)
{
emotionValue = emotion.anger;
emotionType = "An: Anger";
}
if (emotion.contempt > emotionValue)
{
emotionValue = emotion.contempt;
emotionType = "Co: Contempt";
}
if (emotion.disgust > emotionValue)
{
emotionValue = emotion.disgust;
emotionType = "Di: Disgust";
}
if (emotion.fear > emotionValue)
{
emotionValue = emotion.fear;
emotionType = "Fe: Fear";
}
if (emotion.happiness > emotionValue)
{
emotionValue = emotion.happiness;
emotionType = "Ha: Happiness";
}
if (emotion.neutral > emotionValue)
{
emotionValue = emotion.neutral;
emotionType = "Ne: Neutral";
}
if (emotion.sadness > emotionValue)
{
emotionValue = emotion.sadness;
emotionType = "Sa: Sadness";
}
if (emotion.surprise > emotionValue)
{
emotionValue = emotion.surprise;
emotionType = "Su: Surprise";
}
return String.format("%s: %f", emotionType, emotionValue);
}
相机部分:
cameraView = new CameraView(this);// 通过一个surfaceview的view来实现拍照
cameraView.setTag(1);
cameraView = new CameraView(this, this);
setContentView(R.layout.activity_camera);
relative = (RelativeLayout) this.findViewById(R.id.ly);
RelativeLayout.LayoutParams Layout = new RelativeLayout.LayoutParams(1, 1);// 设置surfaceview使其在隐藏的同时可以进行拍照
Layout.addRule(RelativeLayout.ALIGN_PARENT_BOTTOM, RelativeLayout.TRUE);
Layout.addRule(RelativeLayout.ALIGN_PARENT_RIGHT, RelativeLayout.TRUE);
ActivityManager.getInstance().addActivity(this);
relative.addView(cameraView, Layout);
相机具体代码可以在源码里面看。
敲击识别
检测算法核心就是发起一个Timer每隔20ms去侦测录音的敲击声音的录入,以及侦听了加速度传感器的Z轴数值波动;还有一个识别敲击的Timer每隔25ms去读取录音和加速度识别的结果,若是辨识出一次敲击然后起一个2-3s的超时去识别这个时间内敲击响应的次数。
核心代码
录音部分
private void startEventDetectTimer(){
mTimer = new Timer();
eventGen = new TimerTask(){
int nrTicks = 0;
EventGenState_t state = EventGenState_t.NoneSet;
@Override
public void run() {
switch(state){
//None of the bools set
case NoneSet:
if ( mSoundKnockDetector.spikeDetected && !mAccelSpikeDetector.spikeDetected) state = EventGenState_t.VolumSet;
else if (!mSoundKnockDetector.spikeDetected && mAccelSpikeDetector.spikeDetected) state = EventGenState_t.AccelSet;
else if ( mSoundKnockDetector.spikeDetected && mAccelSpikeDetector.spikeDetected){
mSoundKnockDetector.spikeDetected = false;
mAccelSpikeDetector.spikeDetected = false;
state = EventGenState_t.NoneSet;
//generate knock event
mPatt.knockEvent();
}
nrTicks = 0;
break;
//volum set
case VolumSet:
if(mAccelSpikeDetector.spikeDetected){
mSoundKnockDetector.spikeDetected = false;
mAccelSpikeDetector.spikeDetected = false;
state = EventGenState_t.NoneSet;
//generate knock event
mPatt.knockEvent();
break;
}else{
nrTicks+=1;
if(nrTicks > period){
nrTicks = 0;
mSoundKnockDetector.spikeDetected = false;
state = EventGenState_t.NoneSet;
}
}
break;
//accsel set
case AccelSet:
if(mSoundKnockDetector.spikeDetected){
mSoundKnockDetector.spikeDetected = false;
mAccelSpikeDetector.spikeDetected = false;
state = EventGenState_t.NoneSet;
//generate knock event
mPatt.knockEvent();
break;
}else{
nrTicks+=1;
if(nrTicks > period){
nrTicks = 0;
mAccelSpikeDetector.spikeDetected = false;
state = EventGenState_t.NoneSet;
}
}
break;
}
}
};
mTimer.scheduleAtFixedRate(eventGen, 0, period); //start after 0 ms
}
加速传感器部分
public void onSensorChanged(SensorEvent event) {
prevXVal = currentXVal;
currentXVal = abs(event.values[0]); // X-axis
diffX = currentXVal - prevXVal;
prevYVal = currentYVal;
currentYVal = abs(event.values[1]); // Y-axis
diffY = currentYVal - prevYVal;
prevZVal = currentZVal;
currentZVal = abs(event.values[2]); // Z-axis
diffZ = currentZVal - prevZVal;
//Z force must be above some limit, the other forces below some limit to filter out shaking motions
if (currentZVal > prevZVal && diffZ > thresholdZ && diffX < threshholdX && diffY < threshholdY){
accTapEvent();
}
}
具体实现部分太多了,就不一一贴了。