Spring Data SQL Not UTF8 - Java

Introduction

In Spring Data, when working with SQL databases, it is common to use properties files for configuring the database connection. However, if the database uses a character set that is not UTF-8, Spring Data can have issues with encoding and decoding data. This can result in unexpected behavior, such as incorrect data being stored or retrieved from the database.

In this article, we will explore how to configure Spring Data to work with non-UTF-8 character sets in SQL databases.

Problem

When Spring Data is used to work with SQL databases, it relies on the JDBC driver to handle the database connection. By default, the JDBC driver assumes that the database is using the UTF-8 character set. As such, it will attempt to encode and decode data using UTF-8, regardless of the actual character set used by the database.

This can result in issues with data encoding and decoding when working with non-UTF-8 character sets.

Solution

To configure Spring Data to work with non-UTF-8 character sets in SQL databases, we need to set the character set used by the JDBC driver to match the character set used by the database.

Step 1: Add Character Set Properties to Database Configuration

We need to add the following properties to the database configuration properties file:

spring.datasource.url=jdbc:mysql://localhost:3306/mydatabase?useUnicode=true&characterEncoding=UTF-8
spring.datasource.username=root
spring.datasource.password=my_password
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

The important property here is useUnicode=true&characterEncoding=UTF-8, which tells the JDBC driver to use the UTF-8 character set when encoding and decoding data.

Step 2: Set Connection Properties in Application Properties

We also need to set the connection properties in our application properties file:

spring.datasource.hikari.connection-init-sql=SET NAMES utf8mb4 COLLATE utf8mb4_bin;
spring.datasource.hikari.connection-init-sql=SET CHARACTER SET utf8mb4;

The above properties tell the Hikari connection pool (which is used by default in Spring Boot) to set the character set used by the database.

Step 3: Test the Configuration

Finally, we need to test the configuration to ensure that everything is working correctly. We can do this by storing and retrieving some data from the database that includes non-ASCII characters.

@Repository
public interface MyRepository extends JpaRepository<MyEntity, Long> {

    @Query(value = "INSERT INTO my_table (my_column) VALUES (?1)",
            nativeQuery = true)
    void insertWithNonAsciiChar(String value);

    @Query(value = "SELECT my_column FROM my_table WHERE id = ?1",
            nativeQuery = true)
    String selectWithNonAsciiChar(int id);
}

@Service
public class MyService {

    private final MyRepository myRepository;

    @Autowired
    public MyService(MyRepository myRepository) {
        this.myRepository = myRepository;
    }

    public void testEncoding() {
        String data = "こんにちは";
        myRepository.insertWithNonAsciiChar(data);

        String selectedData = myRepository.selectWithNonAsciiChar(1);
        System.out.println(selectedData);
    }
}

If everything is configured correctly, we should see the Japanese greeting "こんにちは" printed out to the console.

Conclusion

In this article, we have explored how to configure Spring Data to work with non-UTF-8 character sets in SQL databases. By setting the character set used by the JDBC driver to match the character set used by the database, we can avoid issues with data encoding and decoding.