Tag: mysql 8

A small dive into the MySQL 8.0 X-DevAPI

A small dive into the MySQL 8.0 X-DevAPI

Introduction

What is the X-DevApi? From insidemysql.com there is a definition of the X-DevAPI and its features in the following paragraphs:

The X DevAPI is the common client-side API used by all connectors to abstract the details of the X Protocol. It specifies the common set of CRUD-style functions/methods used by all the official connectors to work with both document store collections and relational tables, a common expression language to establish query properties such as criteria, projections, aliases, and a standard set of additional database management features for handling things like transactions, indexes, etc.

The fact that most of these features share the same format and API between connectors, makes the X DevAPI a perfect fit for modern polyglot development environments such as microservices, and the fact that they are based on a well-documented format allows advanced users to extend client-side implementations and build new middleware components or extensions tailor-made for their use case.

With MySQL 8.0, the X-DevAPI can be used either with the MySQL Shell, or with the MySQL Connectors that supports the X-Protocol. For this blogpost I will be showing you code using the Node.js driver as interface with it.

As of the moment of writing of this post, the latest version for the driver (available on npm) is the 8.0.11.

Hands-on, the relational way

You can use the X-DevAPI to use it in a more “relational” way, consider a table teams from the worldcup schema:

[code lang=sql]
DESCRIBE teams;
[/code]

Description:

Field Type Null Key Default Extra
id int(11) NO PRI NULL auto_increment
name varchar(255) NO NULL
players json YES NULL

Notice that the field players is a JSON and it can be NULL.

const mysqlx = require('@mysql/xdevapi');
const options = require('./config')
mysqlx.getSession(options)
.then((session) => {
const db = session.getSchema("worldcup");
const tableTeams = db.getTable("teams");
return tableTeams;
})
.then((t) => {
t.select(['id', 'name', 'players'])
.where('name = :name')
.bind('name', 'Brazil')
.limit(1)
.execute(console.log);
})
.catch(err => {
console.log(err.stack);
process.exit(1);
});
view raw xdevapi1.js hosted with ❤ by GitHub

Returns on a browser (data clipped):

[
4,
"Brazil ",
{
"players": [{
"id": 70,
"dob": "05.03.1993",
"club": "FC Shakhtar Donetsk (UKR)",
"name": "FRED",
"height": 169,
"weight": 64,
"position": "MF",
"shirt_name": "FRED",
"shirt_number": 18
},
{
"id": 71,
"dob": "13.01.1988",
"club": "FC Shakhtar Donetsk (UKR)",
"name": "TAISON",
"height": 172,
"weight": 64,
"position": "FW",
"shirt_name": "TAISON",
"shirt_number": 21
}]
}
]
view raw xdevapi2.js hosted with ❤ by GitHub

Or on your terminal:

1__node_index_js
Extra padding on strings

This looks handy, specially if you are working as a document store (which I will talk in a bit), however there are some issues with JSON and VARCHAR fields for now:

  1. VARCHAR fields are getting padded. Take a look at the third line, where it is “Brazil” actually has trailing spaces. Notice on the terminal is how far off the closing quote is.
  2. JSON does work, but a bit in a limited way, in this case, it is only bringing the result set because the column is set to be possibly NULL, on the other hand if you execute the following you will have trouble:

[code lang=sql]
ALTER TABLE teams MODIFY players JSON NOT NULL;
[/code]

worldcup_json_problem
Error on NOT NULL for JSON columns

See how the error is in a string parser that tries to concatenate whatever to it. On my tests usually is empty space, I had zeroes padded on the right too (yes zero on a string), problem 1 looks like is causing problem 2.

They are aware of it, and apparently it will be fixed on a future release.

Hands-on, the document store way

MySQL 8.0 has been marketed extensively as an alternative for NoSQL, although you can actually use JSON with it, there are some caveats to the feature.

I restructured data to be inside a collection instead of a table.

To query it now, you do the following:

const mysqlx = require('@mysql/xdevapi');
const options = require('./config')
mysqlx.getSession(options)
.then((session) => {
const db = session.getSchema("worldcup");
let teamsCollection = db.getCollection("teams_2018");
return teamsCollection.find("name = :country1 OR name = :country2")
.bind("country1", "Brazil")
.bind("country2", "England")
.execute(console.log)
})
.catch(err => {
console.log(err.stack);
process.exit(1);
});
view raw xdevapi3.js hosted with ❤ by GitHub

Something feels wrong to me with this approach of having to write strings instead of dealing with JavaScript objects inside the .find() method, using the “literal” word for OR and AND. I think there is room for improvement here. At this point I am sorry to say this looks more like as an hasty ORM put together than a real document store interface.

By the way, problem 1 and problem 2 don’t happen here:

result_node
Result set for a query

Should I use it now?

I would recommend to wait a bit more. The current version is the first General Availability version, which means next one will have bug fixes and more stability added to it. This definitely is a nifty way for those using Node.js to be able to manipulate data inside MySQL more “natively”, with that in mind, I still feel that I am actually writing SQL to be able to use it the “document store” way.

Bonus

You can actually use the X-DevAPI on Google Cloud Functions.

You need to add the @mysql/xdevapi to package.json and put the code above inside a “wrapper” that looks like this:

/**
* Responds to any HTTP request that can provide a "message" field in the body.
*
* @param {!Object} req Cloud Function request context.
* @param {!Object} res Cloud Function response context.
*/
const options = {
host: "your ip address",
port: 33060,
password: "your password",
user: "your username"
};
const mysqlx = require('@mysql/xdevapi');
exports.getTeam = (req, res) => {
console.log(req.query)
if (req.query.name === undefined) {
res.status(400).send('No name defined!');
} else {
const name = req.query.name;
/** your code goes here **/
mysqlx.getSession(options)
.then((session) => {
const db = session.getSchema("worldcup");
let teamsCollection = db.getCollection("teams_2018");
return teamsCollection.find("name = :country")
.limit(1)
.bind("country", req.query.name)
.execute(row => {
res.status(200).send(row);
})
})
.catch(err => {
console.log(err.stack);
process.exit(1);
})
}
};
view raw xdevapi4.js hosted with ❤ by GitHub

Keep in mind that .execute() doesn’t return a promise, but rather receives a callback function to do your data processing of each individual row. This could be a bit annoying for you.


Image header of the post belongs to Pexel. And yes, I know the code shown is not SQL or JavaScript 🙂

How to use MySQL 8.0.4 with a GUI

If you want to have a look on what is about to come in the new version of the popular database and is used to Syntax Highlighting you don’t need to be chained to the Terminal.

Some of you may use tools like MySQL Workbench or Sequel Pro (as of the release of this post both tools had the following error occurring), and even if you are using the Terminal (if you are using an old version of mysql​, like 5.7) you may encounter this error:

Unable to connect to host 127.0.0.1, or the request timed out.

Be sure that the address is correct and that you have the necessary privileges, or try increasing the connection timeout (currently 10 seconds).

MySQL said: Authentication plugin ‘caching_sha2_password’ cannot be loaded: dlopen(/usr/local/lib/plugin/caching_sha2_password.so, 2): image not found

The reason for that is because since the 8.0.4 RC release, MySQL now uses SHA-2 Pluggable Authentication. In another words, how the database does authentication now changed.

Graphical User Interface

As of now, the only tool I could verify that it is working is Datagrip. But there is some steps to make sure you can successfully connect to the server. Follow the steps

1. The JDBC Connector

  1. Open the JDBC Connector page. Click on “Development Releases” tab and select your operating system, as of this post 8.0.9 was the latest version.
  2. Select the zip version of the file, if you are using macOS, select “Platform Independent”.
  3. The website it will ask for you to login, you don’t need to login, there is a link on the bottom of the page that says: “No thanks, just start my download.”.
  4. Unzip the zip​ file mysql-connector-java-8.0.9-rc.zip (the name may be different for you if the version is different)
  5. A folder will be created with the name of the compressed file, inside copy the jar file to a location where you can access it later easily, for example, I put mine in ~/workspace/drivers folder.

2. The GUI configuration

  1. Open Datagrip. Go to “File > Data Sources“. A window will open, right click on top of the “MySQL” name and select Duplicate.
  2. A new Driver is added with the name “MySQL [2]”, rename it to “MySQL 8.0”
    Data_Sources_and_Drivers.png
  3. Then, unselect “Use” on “Driver Files” and click on the + sign. Select the jar file you downloaded on the previous section.
    Data_Sources_and_Drivers.png
  4. Click in Apply.

3. Adding the source

  1. On the same window, click on the + sign on the top left. Select “MySQL 8.0
    Screen Shot 2018-02-22 at 14.50.28.png
  2. Fill out the details as you would for a connection: Data_Sources_and_Drivers
  3. Click on “Test Connection“.
  4. If everything worked, just click in “OK” to exit the screen.

From MySQL 8.0.0 to MySQL 8.0.1 – or any other dev milestone

Disclaimer: This post is aimed to you, the curious developer, sys-admin, technologist, whatever-title-you-use. DO NOT run the following lines on production. Not even in a stable environment, do this if you don’t care about the outcome of the current data.

If you want to keep up with the newest MySQL developer milestones I have news for you: there is no upgrade available for milestone versions. The way to go is to remove old version and install new one, according to their website:

Upgrades between milestone releases (or from a milestone release to a GA release) are not supported. For example, upgrading from 8.0.0 to 8.0.1 is not supported, as neither are GA status releases.

So if you, like me, had the 8.0.0 version and want to test the 8.0.1 (alhtough 8.0.3 milestone is already in development) you need to do something like the following (tutorial based on Debian/Ubuntu servers).

Stop your service:

[code lang=shell]
$ sudo service mysql stop
[/code]

Download Oracle’s repository and install it, as of now this is the current version, you can get the new package here:

[code lang=shell]
$ wget https://dev.mysql.com/get/mysql-apt-config_0.8.6-1_all.deb
$ sudo dpkg -i mysql-apt-config_0.8.6-1_all.deb
[/code]

Clean your old install, you will lose all the data. Be careful, back up is on you!

[code lang=shell]
$ sudo apt-get remove –purge mysql-server mysql-client mysql-common
$ sudo apt autoremove
$ sudo apt-get autoclean
$ sudo apt-get install mysql-server
[/code]

This is the way to go to test the new features such as Descending Indexes and others. Remember, the new default encoding was changed from latin1 to utf8mb4.

Short feature list:

The complete list is available here.