如何使用Firebase ML Kit在Android中标记图像？

我们已经看到Android中有许多应用程序，其中我们将检测图像中存在的对象是否可能是任何对象。在本文中，我们将介绍使用Firebase ML Kit在Android中实现图像标签的方法。

我们将在本文中构建什么?

我们将构建一个简单的应用程序，在该应用程序中，我们将捕获任何对象的图像，然后，我们将以准确度级别检测图像中存在的对象。下面提供了一个示例视频，以使您对我们在本文中将要做的事情有个大概的了解。注意，我们将使用Java语言实现该项目。

分步实施

步骤1：创建一个新项目

要在Android Studio中创建新项目，请参阅如何在Android Studio中创建/启动新项目。请注意，选择Java作为编程语言。

第2步：将您的应用连接到Firebase

在Android Studio中创建新项目后，将您的应用连接到Firebase。用于将您的应用程序连接到Firebase。导航到顶部栏上的“工具”。之后，单击Firebase。右侧将打开一个新窗口。在该窗口中，单击Firebase ML，然后在Android中单击“使用Firebase ML工具包”。您可以在屏幕截图下方看到该选项。

在下一个屏幕上单击此选项后，单击“连接到Firebase”选项以将您的应用程序连接到Firebase。

步骤3：添加语言翻译的依赖关系到build.gradle文件

导航到Gradle脚本> build.gradle(Module：app)并将以下依赖项添加到“依赖项”部分。

// dependency for firebase core.

implementation’com.google.firebase:firebase-core:15.0.2′

// Firebase ML dependency

implementation ‘com.google.firebase:firebase-ml-vision:24.0.3’

implementation ‘com.google.firebase:firebase-ml-vision-image-label-model:20.0.1’

步骤4：在您的Android Apps AndroidManifest文件中添加访问Internet和元数据的权限

导航至应用程序> AndroidManifest.xml文件，然后将以下代码添加到其中。在代码中添加了注释，以便更详细地了解。

将以下行添加到您的应用程序标记内。

以下是AndroidManifest.xml文件的完整代码。

步骤5：使用activity_main.xml文件

导航到应用程序> res>布局> activity_main.xml，然后将以下代码添加到该文件中。以下是activity_main.xml文件的代码。

第6步：创建用于存储数据的模式类

导航到应用程序> Java >应用程序的程序包名称>右键单击它>新建> Java类，并将您的类命名为DataModal，然后将以下代码添加到其中。在代码内部添加了注释，以更详细地了解代码。

Java

public class DataModal {
  
    // variables for our 
    // string and confidence.
    private String result;
    private float confidence;
  
    // constructor
    public DataModal(String result, float confidence) {
        this.result = result;
        this.confidence = confidence;
    }
  
    // getter and setter methods
    public float getConfidence() {
        return confidence;
    }
  
    public void setConfidence(float confidence) {
        this.confidence = confidence;
    }
      
    public String getResult() {
        return result;
    }
  
    public void setResult(String result) {
        this.result = result;
    }
      
}

第7步：创建一个布局文件以显示我们的回收站视图项目

导航到应用程序> res>布局>右键单击它>新建>布局资源文件，并将其命名为result_rv_item ，并向其添加以下代码。

步骤8：为我们的RecyclerView创建适配器类

导航到应用程序> Java >应用程序的包名称>右键单击它>新建> Java类，并将其命名为resultRVAdapter，然后将以下代码添加到其中。在代码内部添加了注释，以更详细地了解代码。

Java

import android.content.Context;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.TextView;
  
import androidx.annotation.NonNull;
import androidx.recyclerview.widget.RecyclerView;
  
import java.util.ArrayList;
  
public class resultRVAdapter extends RecyclerView.Adapter {
      
    // arraylist for storing our data and context
    private ArrayList dataModalArrayList;
    private Context context;
  
    // constructor for our variables
    public resultRVAdapter(ArrayList dataModalArrayList, Context context) {
        this.dataModalArrayList = dataModalArrayList;
        this.context = context;
    }
  
    @NonNull
    @Override
    public ViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
        // inside on create view holder method we are inflating our layout file which we created.
        View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.result_rv_item, parent, false);
        return new ViewHolder(view);
    }
  
    @Override
    public void onBindViewHolder(@NonNull ViewHolder holder, int position) {
        // inside on bind view holder method we are setting
        // data to each item of recycler view.
        DataModal modal = dataModalArrayList.get(position);
        holder.resultTV.setText(modal.getResult());
        holder.confidenceTV.setText("" + modal.getConfidence());
    }
  
    @Override
    public int getItemCount() {
        // returning the size of array list.
        return dataModalArrayList.size();
    }
  
    public class ViewHolder extends RecyclerView.ViewHolder {
        // creating variables for our text view.
        private TextView resultTV, confidenceTV;
  
        public ViewHolder(@NonNull View itemView) {
            super(itemView);
            // initializing our views with their ids.
            resultTV = itemView.findViewById(R.id.idTVResult);
            confidenceTV = itemView.findViewById(R.id.idTVConfidence);
        }
    }
}

步骤9：使用MainActivity。 Java文件

转到MainActivity。 Java文件并参考以下代码。下面是MainActivity的代码。 Java文件。在代码内部添加了注释，以更详细地了解代码。

Java

import android.content.Intent;
import android.graphics.Bitmap;
import android.os.Bundle;
import android.provider.MediaStore;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.Toast;
  
import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.recyclerview.widget.LinearLayoutManager;
import androidx.recyclerview.widget.RecyclerView;
  
import com.google.android.gms.tasks.OnFailureListener;
import com.google.android.gms.tasks.OnSuccessListener;
import com.google.firebase.ml.vision.FirebaseVision;
import com.google.firebase.ml.vision.common.FirebaseVisionImage;
import com.google.firebase.ml.vision.label.FirebaseVisionImageLabel;
import com.google.firebase.ml.vision.label.FirebaseVisionImageLabeler;
  
import java.util.ArrayList;
import java.util.List;
  
public class MainActivity extends AppCompatActivity {
  
    // variables for our image view, image bitmap,
    // buttons, recycler view, adapter and array list.
    private ImageView img;
    private Button snap, labelBtn;
    private Bitmap imageBitmap;
    private RecyclerView resultRV;
    private resultRVAdapter resultRvAdapter;
    private ArrayList dataModalArrayList;
  
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
          
        // initializing all our variables for views
        img = (ImageView) findViewById(R.id.image);
        snap = (Button) findViewById(R.id.snapbtn);
        labelBtn = findViewById(R.id.labelBtn);
        resultRV = findViewById(R.id.idRVResults);
          
        // initializing our array list
        dataModalArrayList = new ArrayList<>();
          
        // initializing our adapter class.
        resultRvAdapter = new resultRVAdapter(dataModalArrayList, MainActivity.this);
          
        // layout manager for our recycler view.
        LinearLayoutManager manager = new LinearLayoutManager(MainActivity.this);
          
        // on below line we are setting layout manager 
        // and adapter to our recycler view.
        resultRV.setLayoutManager(manager);
        resultRV.setAdapter(resultRvAdapter);
  
        // adding on click listener for our label button.
        labelBtn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                // calling method 
                // to label images.
                labelImage();
            }
        });
        // adding on click listener for our snap button.
        snap.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                // calling a method 
                // to capture an image.
                dispatchTakePictureIntent();
            }
        });
    }
  
    static final int REQUEST_IMAGE_CAPTURE = 1;
  
    private void dispatchTakePictureIntent() {
        // inside this method we are calling an implicit intent to capture an image.
        Intent takePictureIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
        if (takePictureIntent.resolveActivity(getPackageManager()) != null) {
            // calling a start activity for result when image is captured.
            startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE);
        }
    }
  
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);
        // inside on activity result method we are setting
        // our image to our image view from bitmap.
        if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
            Bundle extras = data.getExtras();
            imageBitmap = (Bitmap) extras.get("data");
            // on below line we are setting our
            // bitmap to our image view.
            img.setImageBitmap(imageBitmap);
        }
    }
  
    private void labelImage() {
        
        // inside the label image method we are calling a 
        // firebase vision image and passing our image bitmap to it.
        FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(imageBitmap);
          
        // on below line we are creating a labeler for our image bitmap
        // and creating a variable for our firebase vision image labeler.
        FirebaseVisionImageLabeler labeler = FirebaseVision.getInstance().getOnDeviceImageLabeler();
          
        // calling a method to process an image and adding on success listener method to it.
        labeler.processImage(image).addOnSuccessListener(new OnSuccessListener>() {
            @Override
            public void onSuccess(List firebaseVisionImageLabels) {
                  
                  // inside on success method we are running a loop to get the data from our list.
                for (FirebaseVisionImageLabel label : firebaseVisionImageLabels) {
                    
                    // on below line we are getting text from our list.
                    String text = label.getText();
                      
                    // on below line we are getting its entity id
                    String entityId = label.getEntityId();
                      
                    // on below line we are getting the
                    // confidence level of our modal.
                    float confidence = label.getConfidence();
                     
                    // after getting all data we are passing it to our array list.
                    dataModalArrayList.add(new DataModal(text, confidence));
                      
                    // after adding a new data we are notifying 
                    // our adapter that data has been updated.
                    resultRvAdapter.notifyDataSetChanged();
                }
            }
        }).addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception e) {
                // error handling for on failure method
                Toast.makeText(MainActivity.this, "Fail to get data..", Toast.LENGTH_SHORT).show();
            }
        });
    }
}

现在运行您的应用程序，并查看该应用程序的输出。

输出：

Note: If you are facing the following NDK at ~/Library/Android/sdk/ndk-bundle did not have a source.properties file error then please refer to this article.