Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hbase10] Address #701 by mimicking the same locks from the HBase 0.9… #1028

Merged
merged 4 commits into from
Sep 21, 2017

Conversation

manolama
Copy link
Collaborator

…8 client in the HBase 10 client.

…he HBase 0.98 client in the HBase 10 client.
@@ -82,7 +84,7 @@
* @See #CONNECTION_LOCK.
*/
private static Connection connection = null;
private static final Object CONNECTION_LOCK = new Object();
//private static final Object CONNECTION_LOCK = new Object();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just delete the line. Don't comment out.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doh, thanks.

@@ -225,7 +224,7 @@ public void cleanup() throws DBException {

public void getHTable(String table) throws IOException {
final TableName tName = TableName.valueOf(table);
synchronized (CONNECTION_LOCK) {
synchronized (TABLE_LOCK) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#701 asked for this lock to be removed, if I am reading it correctly.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

He did mention it but I need confirmation from an HBase dev that the table fetch is thread safe in 1.X. I think it wasn't in the past.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connection objects in hbase 1.y are threadsafe, so getTable is fine. The returned Table object is not threadsafe, but making them is supposed to be lightweight and done per-thread when needed.

int threadCount = THREAD_COUNT.decrementAndGet();
if (threadCount <= 0 && connection != null) {
connection.close();
connection = null;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This synchronized block should not be needed. The closing of the connection is protected by the decrement of the atomic integer.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, yanked the sync.

@@ -145,8 +147,8 @@ public void init() throws DBException {
}

try {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we merge this try into the try after the "Terminate right now" comment?

@@ -179,7 +181,7 @@ public void init() throws DBException {
String table = getProperties().getProperty(TABLENAME_PROPERTY, TABLENAME_PROPERTY_DEFAULT);
try {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you merge the try above into this try you would have something like:

if( THREAD_COUNT.getAndIncrement() == 0) {
    String table = getProperties().getProperty(TABLENAME_PROPERTY, TABLENAME_PROPERTY_DEFAULT);
    try {
        if (connection == null) {
            // Initialize if not set up already.
            connection = ConnectionFactory.createConnection(config);

            // Validate the table name.
            final TableName tName = TableName.valueOf(table);
            connection.getTable(tName).getTableDescriptor();
        }
    } catch (java.io.IOException e) {
        throw new DBException(e);
   }
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, merged em.

@@ -225,7 +224,7 @@ public void cleanup() throws DBException {

public void getHTable(String table) throws IOException {
final TableName tName = TableName.valueOf(table);
synchronized (CONNECTION_LOCK) {
synchronized (TABLE_LOCK) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connection objects in hbase 1.y are threadsafe, so getTable is fine. The returned Table object is not threadsafe, but making them is supposed to be lightweight and done per-thread when needed.

@manolama
Copy link
Collaborator Author

Yanked the lock so we should be good now @busbey, thanks!

}
}
int threadCount = THREAD_COUNT.decrementAndGet();
if (threadCount <= 0 && connection != null) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be in a lock that covers checking if it's null and setting it to null, othewise we race the initialization above that only creates a new connection when it's null.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But Alan was right in that this should only fire when the atomic int returns a 0 at which point only one thread will perform the null check and close + null the connection.

Or are you thinking of the case where there may be some initialization issue wherein a run initializes multiple threads but the initializations fail and possibly cleanup is executed on a thread before initialization begins in another thread?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right exactly.

@manolama manolama merged commit 1a3c8cb into brianfrankcooper:master Sep 21, 2017
@manolama manolama mentioned this pull request Sep 26, 2017
@busbey busbey mentioned this pull request May 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants