Configuring VectorH for Use with Apache Knox
VectorH has a gateway facility that performs a function comparable to the Knox gateway service.
To enable the VectorH gateway function on the Knox gateway system, the following are required:
• The VectorH client must be installed on the system that is running the Knox gateway service.
• The VectorH client installation must be configured so that it provides a transparent gateway between JDBC client connections and the VectorH server that is running in the cluster.
Understanding the Connections
When configuring VectorH to work with Apache Knox, three connection points need to be considered: the JDBC Client, the Gateway machine, and the VectorH master node.
The JDBC Client connects to the Gateway machine and from there to the VectorH master node. The user is authenticated on the VectorH master node by using LDAP, like Knox does.
The following illustrates the work flow from the user perspective:
In more detail, the JDBC Client connects to the Data Access Server (GCD) of the Vector Client Runtime running on the Gateway machine.
The Data Access Server (GCD) asks the remote Name Server (GCN) to do the authentication, which means the connection will be forwarded through the local Communications Server (GCC) (also known as the Net Server) to the remote Communications Server (GCC) and from there to the local Name Server (GCN). The Name Server (GCN) calls ingvalidpam, which uses the configured PAM authentication using LDAP to check the credentials.
If the authentication is successful, the Data Access Server (GCD) connects again to the local Communications Server (GCC). The local Communications Server (GCC) connects to the remote Communications Server (GCC) on the VectorH master node. Finally, the Communications Server (GCC) on the VectorH master node connects locally to the DBMS Server.
So there are actually three connections:
1. From the JDBC Client to the Gateway
2. From the Gateway to the VectorH master node to do the (remote) LDAP authentication
3. From the Gateway to the VectorH master node if LDAP authentication was successful
An improved variant of this work flow could be the use of a Bridge Server.
The following illustrates the work flow from a low-level perspective:
Configure LDAP Authentication Using PAM
The Data Access Server (GCD) on the Gateway machine asks the Name Server (GCN) on the VectorH master node to perform the LDAP authentication using PAM.
To configure the LDAP authentication on the VectorH master node, perform the following steps:
1. Shut down the VectorH instance:
ingstop
2. Disable DBMS authentication:
iisetres ii.$(iipmhost).dbms.*.dbms_authentication OFF
You can check the new setting:
iigetres ii.$(iipmhost).dbms.*.dbms_authentication
3. Install OpenLDAP Client:
yum -y install openldap-clients nss-pam-ldapd
4. As root, do the following:
a. Enable LDAP authentication:
authconfig --enableldap --enableldapauth --ldapserver=<LDAP Server Host>:<LDAP Server Port> --ldapbasedn="dc=hadoop,dc=apache,dc=org" --update
Note: If message “getsebool: SELinux is disabled” is displayed, enable SELinux by setting “SELINUX=enforcing” in “/etc/selinux/config/”, reboot the machine, run "authconfig", and then switch back to "SELINUX=disabled" and reboot again.
Check your setting:
authconfig --test | grep ldap
b. Configure the local LDAP name service daemon (nslcd) by editing the file /etc/nslcd.conf, as follows:
# The user and group nslcd should run as (default setting)
uid nslcd
gid ldap
# The uri pointing to the LDAP server to use for name lookups.
# Multiple entries may be specified. The address that is used
# here should be resolvable without using LDAP (obviously).
# or "uri ldaps://<LDAP Server Hostname>:<LDAP Server Port>/" if TLS is used
uri ldap://<LDAP Server Hostname>:<LDAP Server Port>/
# The distinguished name of the search base.
base dc=your,dc=search,dc=base
# Customize certain database lookups.
base passwd ou=<your user accounts group>,...,dc=your,dc=search,dc=base
# The distinguished name to bind to the server with.
# Optional: default is to bind anonymously.
binddn uid=<admin user>, ..., dc=your,dc=search,dc=base
bindpw <admin password>
# the user account object class which contains the password field
filter passwd (objectClass=inetOrgPerson)
# This comment prevents repeated auto-migration of settings.
# "yes", if TLS is used
ssl no
tls_cacertdir /etc/openldap/cacerts
Note: There is no need to configure /etc/openldap/ldap.conf.
c. Create PAM configuration service file "ingres" in directory /etc/pam.d as a symbolic link to 'login':
ln -s /etc/pam.d/login /etc/pam.d/ingres
d. Add the following entries to /etc/pam.d/ingres.
Note: To avoid risk, test it first with a copy of /etc/pam.d/login, rename the copy to /etc/pam.d/ingres, and then add the entries.
auth sufficient pam_ldap.so
auth sufficient pam_nologin.so
account sufficient pam_ldap.so
e. Change the ownership and permissions on ingvalidpam:
chown root:root $II_SYSTEM/ingres/bin/ingvalidpam
chmod 4755 $II_SYSTEM/ingres/bin/ingvalidpam
5. As user actian, set II_SHADOW_PWD to point to the executable:
ingsetenv II_SHADOW_PWD $II_SYSTEM/ingres/bin/ingvalidpam
To check that the setting now exists:
ingprenv II_SHADOW_PWD
6. Start the VectorH instance:
ingstart
To test the configuration, call ingvalidpam directly. You are not prompted. Enter the LDAP user name followed by a space and the LDAP password. If that returns 0 (echo $?) it works. If it does not work, set II_INGVALIDPW_LOG to a filename (in the OS environment). It will show the value returned by the PAM system (which is not the return value from ingvalidpam). Those values are defined in /usr/include/security/pam_appl.h or actually in /usr/include/security/_pam_types.h (included from the first).
Define a JDBC Connection
The JDBC Client must include iijdbc.jar and run with a JRE version described in
Requirements.
On the Gateway machine the Vector Client Runtime and Apache Knox must be installed. The important connection target of the VectorH cluster is the VectorH master node through the Gateway.
The following are required for a successful JDBC connection:
• A configured LDAP authentication using PAM on the VectorH master node (see previous section)
• LDAP users defined in the master database, iidbdb. The users in iidbdb do not need passwords or special privileges.
To define a JDBC connection, perform the following steps.
1. On Gateway machine define a global vnode pointing to the VectorH master node (only the connection details are needed):
a. Start Vector Client Runtime and Data Access Server (DAS):
ingstart
ingstart -iigcd
b. Create a vnode using the netutil utility:
a. Issue the netutil command at the command prompt. The netutil utility starts.
b. Select "Create".
c. Enter a name for the vnode (for example, test), and then select OK.
d. Select "Global".
e. Skip adding login/password credentials by pressing Cancel.
f. For "Network Address" enter the hostname or IP address of the VectorH master node.
g. Leave the protocol as it is.
h. For "Listen Address" enter the VectorH server instance ID (default is "VH").
i. Select "Save", and then quit netutil by pressing F10.
j. Set ii.hostname.gcn.remote_vnode to the vnode name:
iisetres ii.$(iipmhost).gcn.remote_vnode <vnode name>
You can check the new setting:
iigetres ii.$(iipmhost).gcn.remote_vnode
k. Restart Name Server:
ingstop -iigcn; ingstart -iigcn
2. On the JDBC Client, use the following connection string:
jdbc:ingres://<gatewayHost>:<gatewayPort>/<vnode name>::<dbname>;user=<gatewayUser>;password=<gatewayPassword>
A vnode (virtual node) stores all the information required to connect to a remote VectorH server. For more information about vnode usage in JDBC, see JDBC Driver Properties in the Connectivity Guide.
CountryList.java
The CountryList.java example shows how the JDBC connection works.
Change package name to your appropriate java project path.
package jdbc.src;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;
public class CountryList {
/* Connection to VectorH Gateway - "Creation of a vnode at Gateway and provide the vnode in the JDBC connection string"
* connection string template: "jdbc:ingres://<gatewayHost>:<gatewayPort>/<VectorHdbname>;<gatewayUser>;<gatewayPassword>" */
private static String gatewayHost = "ussf-user01-c3m"; // Gateway machine where Knox and Ranger are installed
private static String gatewayPort = "CL7"; // DAS Port on Gateway machine
private static String gatewayUser = "user=tom"; // credentials for LDAP authentication which happen on VectorH Master
private static String gatewayPassword = "password=tom-password";// credentials for LDAP authentication which happen on VectorH Master
private static String VectorHdbname = "testvnode::testdb"; // "<vnode name>::<dbname>"
private static String connectionStr = "jdbc:ingres://" +
gatewayHost + ":" + gatewayPort + "/" +
VectorHdbname + ";" +
gatewayUser + ";" + gatewayPassword;
private static Connection conn = null;
private static Statement stmt = null;
private static String tableName = "country";
/**
* @param args
*/
public static void main(String[] args) {
System.out.println("create connection to" + connectionStr);
createConnection();
System.out.println("drop and create country table...");
dropCreateCountryTable();
System.out.println("retrieve countries...");
retrieveCountries();
System.out.println("inserting country data...");
insertCountry(1, "CH", "Switzerland");
insertCountry(2, "GER", "Germany");
insertCountry(3, "P", "Poland");
System.out.println("retrieve countries...");
retrieveCountries();
}
/**
* Create a connection to Ingres using the Ingres JDBC driver
* and the private variable connectionStr
*/
private static void createConnection()
{
try
{
Class.forName("com.ingres.jdbc.IngresDriver").newInstance();
conn = DriverManager.getConnection(connectionStr);
System.out.println("");
}
catch (Exception except)
{
except.printStackTrace();
}
}
/**
* Drop and create country table.
*/
private static void dropCreateCountryTable()
{
try
{
stmt = conn.createStatement();
stmt.execute("drop table IF EXISTS " + tableName);
stmt.execute("create table " + tableName +" (ct_id integer, ct_code varchar(32), ct_name varchar(32))");
stmt.close();
System.out.println("");
}
catch (SQLException sqlExcept)
{
sqlExcept.printStackTrace();
}
}
/**
* Insert a country.
*
* @param countryID
* @param countryCode
* @param countryName
*/
private static void insertCountry(Integer countryID, String countryCode, String countryName)
{
try
{
stmt = conn.createStatement();
stmt.execute("insert into " + tableName +
" values(" +
countryID + ","+ "'" +
countryCode + "','" +
countryName +"')");
stmt.close();
System.out.println("");
}
catch (SQLException sqlExcept)
{
sqlExcept.printStackTrace();
}
}
/**
* Retrieve and print all countries in the database.
*/
private static void retrieveCountries()
{
try
{
stmt = conn.createStatement();
ResultSet results = stmt.executeQuery("SELECT ct_id, ct_code, ct_name" + " FROM " + tableName);
ResultSetMetaData rsmd = results.getMetaData();
int numberCols = rsmd.getColumnCount();
for (int i=1; i<=numberCols; i++)
{
//print Column Names
System.out.print(rsmd.getColumnLabel(i)+"\t");
}
System.out.println("");
String line = "";
for (int i = 0; i < numberCols; i++) {
line += "=====\t";
}
System.out.println(line);
while(results.next())
{
String colValue = "";
for (int i = 1; i <= numberCols; i++) {
colValue += results.getString(i) + "\t";
}
System.out.println(colValue);
}
results.close();
stmt.close();
System.out.println("");
}
catch (SQLException sqlExcept)
{
sqlExcept.printStackTrace();
}
}
}
