Skip to content
This repository was archived by the owner on Oct 15, 2020. It is now read-only.

Commit 20f9253

Browse files
eamonnotooleeamonnotoole
committed
Allow connection reuse
We have to use the SDK in environments where the total number of connections is restricted, where repeatedly opening and closing connections can cause instability, and with fairly large latencies in opening a new connection. To aid stability and SDK performance in these environments we allow for the reuse of https connections as returned by get_connection(). Connections will be reused when 'reuse_connection' is set to True or when the ONEVIEWSDK_REUSE_CONNECTION is anything other than an empty string. We've added a unit-test for this new capability.
1 parent e54af3c commit 20f9253

8 files changed

Lines changed: 104 additions & 32 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
# 4.5.0 (Unreleased)
22
#### Notes
3-
Added the capability to set a connection timeout when connecting to the HPE OneView Appliance
3+
Added the capability to set a connection timeout when connecting to the HPE OneView Appliance.
44

55
Extends support of the SDK to OneView Rest API version 600 (OneView v4.0).
66

7+
Added the capability to reuse https connections to the HPE OneView Appliance.
8+
79
#### Features supported with current release:
810
- Connection template
911
- Enclosure

‎README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ export ONEVIEWSDK_AUTH_LOGIN_DOMAIN='authdomain'
9999
export ONEVIEWSDK_SSL_CERTIFICATE='<path_to_cert.crt_file>'
100100
export ONEVIEWSDK_PROXY='<proxy_host>:<proxy_port>'
101101
export ONEVIEWSDK_CONNECTION_TIMEOUT='<connection time-out in seconds>'
102+
export ONEVIEWSDK_REUSE_CONNECTION='<string>'
102103
```
103104

104105
:lock: Tip: Make sure no unauthorized person has access to the environment variables, since the password is stored in clear-text.
@@ -259,6 +260,22 @@ export ONEVIEWSDK_CONNECTION_TIMEOUT='<connection time-out in seconds>'
259260
"timeout": <timeout in seconds>
260261
```
261262

263+
264+
### Reuse https connections
265+
By default a new https connection is made and subsequently closed for each SDK transaction. To
266+
change this so that the https_connection is reused then either:
267+
268+
1. Set the appropriate environment variable:
269+
```bash
270+
export ONEVIEWSDK_REUSE_CONNECTION='<any non-null string, eg Yes>'
271+
```
272+
273+
2. Set the reuse_connection flag in the JSON configuration file:
274+
```bash
275+
"reuse_connection": true
276+
```
277+
278+
262279
## Exception handling
263280

264281
All exceptions raised by the OneView Python SDK inherit from HPOneViewException.

‎examples/config-rename.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,5 +21,6 @@
2121
"storage_system_password": "",
2222
"power_device_hostname": "",
2323
"power_device_username": "",
24-
"power_device_password": ""
24+
"power_device_password": "",
25+
"reuse_connection": true
2526
}

‎hpOneView/connection.py‎

100644100755
Lines changed: 43 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -*- coding: utf-8 -*
22
###
3-
# (C) Copyright (2012-2017) Hewlett Packard Enterprise Development LP
3+
# (C) Copyright (2012-2018) Hewlett Packard Enterprise Development LP
44
#
55
# Permission is hereby granted, free of charge, to any person obtaining a copy
66
# of this software and associated documentation files (the "Software"), to deal
@@ -54,7 +54,7 @@
5454

5555

5656
class connection(object):
57-
def __init__(self, applianceIp, api_version=300, sslBundle=False, timeout=None):
57+
def __init__(self, applianceIp, api_version=300, sslBundle=False, timeout=None, reuse_connection=False):
5858
self._session = None
5959
self._host = applianceIp
6060
self._cred = None
@@ -75,6 +75,10 @@ def __init__(self, applianceIp, api_version=300, sslBundle=False, timeout=None):
7575
self._numDisplayedRecords = 0
7676
self._validateVersion = False
7777
self._timeout = timeout
78+
self._reuse_connection = reuse_connection
79+
if self._reuse_connection:
80+
self._headers['Connection'] = 'keep-alive'
81+
self._conn = None
7882

7983
def validateVersion(self):
8084
version = self.get(uri['version'])
@@ -120,11 +124,11 @@ def do_http(self, method, path, body, custom_headers=None):
120124
if custom_headers:
121125
http_headers.update(custom_headers)
122126

123-
bConnected = False
124127
conn = None
128+
bConnected = False
125129
while bConnected is False:
126130
try:
127-
conn = self.get_connection()
131+
conn = self.get_reusable_connection()
128132
conn.request(method, path, body, http_headers)
129133
resp = conn.getresponse()
130134
tempbytes = ''
@@ -133,23 +137,25 @@ def do_http(self, method, path, body, custom_headers=None):
133137
tempbody = tempbytes.decode('utf-8')
134138
except UnicodeDecodeError: # Might be binary data
135139
tempbody = tempbytes
136-
conn.close()
140+
if not self._reuse_connection:
141+
self.close_reusable_connection(conn)
137142
bConnected = True
138143
return resp, tempbody
139144
if tempbody:
140145
try:
141146
body = json.loads(tempbody)
142147
except ValueError:
143148
body = tempbody
144-
conn.close()
149+
if not self._reuse_connection:
150+
self.close_reusable_connection(conn)
145151
bConnected = True
146152
except http.client.BadStatusLine:
147153
logger.warning('Bad Status Line. Trying again...')
148-
if conn:
149-
conn.close()
154+
self.close_reusable_connection(conn)
150155
time.sleep(1)
151156
continue
152157
except http.client.HTTPException:
158+
self.close_reusable_connection(conn)
153159
raise HPOneViewException('Failure during login attempt.\n %s' % traceback.format_exc())
154160

155161
return resp, body
@@ -165,7 +171,7 @@ def download_to_stream(self, stream_writer, url, body='', method='GET', custom_h
165171
successful_connected = False
166172
while not successful_connected:
167173
try:
168-
conn = self.get_connection()
174+
conn = self.get_reusable_connection()
169175
conn.request(method, url, body, http_headers)
170176
resp = conn.getresponse()
171177

@@ -178,15 +184,16 @@ def download_to_stream(self, stream_writer, url, body='', method='GET', custom_h
178184
if tempbytes: # filter out keep-alive new chunks
179185
stream_writer.write(tempbytes)
180186

181-
conn.close()
187+
if not self._reuse_connection:
188+
self.close_reusable_connection(conn)
182189
successful_connected = True
183190
except http.client.BadStatusLine:
184191
logger.warning('Bad Status Line. Trying again...')
185-
if conn:
186-
conn.close()
192+
self.close_reusable_connection(conn)
187193
time.sleep(1)
188194
continue
189195
except http.client.HTTPException:
196+
self.close_reusable_connection(conn)
190197
raise HPOneViewException('Failure during login attempt.\n %s' % traceback.format_exc())
191198

192199
return successful_connected
@@ -201,13 +208,28 @@ def __handle_download_error(self, resp, conn):
201208
body = tempbody
202209
except UnicodeDecodeError: # Might be binary data
203210
body = tempbytes
204-
conn.close()
211+
self.close_reusable_connection(conn)
205212
if not body:
206213
body = "Error " + str(resp.status)
207214

208-
conn.close()
215+
self.close_reusable_connection(conn)
209216
raise HPOneViewException(body)
210217

218+
def get_reusable_connection(self):
219+
if self._reuse_connection:
220+
if not self._conn:
221+
logger.debug('Creating new connection')
222+
self._conn = self.get_connection()
223+
conn = self._conn
224+
else:
225+
conn = self.get_connection()
226+
return conn
227+
228+
def close_reusable_connection(self, conn):
229+
if conn:
230+
conn.close()
231+
self._conn = None
232+
211233
def get_connection(self):
212234
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
213235
if self._sslTrustAll is False:
@@ -290,7 +312,7 @@ def post_multipart(self, uri, fields, files, baseName, verbose=False):
290312
mappedfile = mmap.mmap(inputfile.fileno(), 0, access=mmap.ACCESS_READ)
291313
if verbose is True:
292314
print(('Uploading ' + files + '...'))
293-
conn = self.get_connection()
315+
conn = self.get_reusable_connection()
294316
# conn.set_debuglevel(1)
295317
conn.connect()
296318
conn.putrequest('POST', uri)
@@ -300,6 +322,8 @@ def post_multipart(self, uri, fields, files, baseName, verbose=False):
300322
totalSize = os.path.getsize(files + '.b64')
301323
conn.putheader('Content-Length', totalSize)
302324
conn.putheader('X-API-Version', self._apiVersion)
325+
if self._reuse_connection:
326+
conn.putheader('Connection', 'keep-alive')
303327
conn.endheaders()
304328

305329
while mappedfile.tell() < mappedfile.size():
@@ -313,6 +337,7 @@ def post_multipart(self, uri, fields, files, baseName, verbose=False):
313337
mappedfile.close()
314338
inputfile.close()
315339
os.remove(files + '.b64')
340+
316341
response = conn.getresponse()
317342
body = response.read().decode('utf-8')
318343

@@ -322,9 +347,11 @@ def post_multipart(self, uri, fields, files, baseName, verbose=False):
322347
except ValueError:
323348
body = response.read().decode('utf-8')
324349

325-
conn.close()
350+
if not self._reuse_connection:
351+
self.close_reusable_connection(conn)
326352

327353
if response.status >= 400:
354+
self.close_reusable_connection(conn)
328355
raise HPOneViewException(body)
329356

330357
return response, body

‎hpOneView/image_streamer/image_streamer_client.py‎

100644100755
Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -*- coding: utf-8 -*-
22
###
3-
# (C) Copyright (2012-2017) Hewlett Packard Enterprise Development LP
3+
# (C) Copyright (2012-2018) Hewlett Packard Enterprise Development LP
44
#
55
# Permission is hereby granted, free of charge, to any person obtaining a copy
66
# of this software and associated documentation files (the "Software"), to deal
@@ -32,7 +32,6 @@
3232

3333
standard_library.install_aliases()
3434

35-
3635
from hpOneView.connection import connection
3736
from hpOneView.image_streamer.resources.golden_images import GoldenImages
3837
from hpOneView.image_streamer.resources.plan_scripts import PlanScripts
@@ -44,8 +43,8 @@
4443

4544

4645
class ImageStreamerClient(object):
47-
def __init__(self, ip, session_id, api_version, sslBundle=False):
48-
self.__connection = connection(ip, api_version, sslBundle)
46+
def __init__(self, ip, session_id, api_version, sslBundle=False, timeout=None, reuse_connection=False):
47+
self.__connection = connection(ip, api_version, sslBundle, timeout, reuse_connection)
4948
self.__connection.set_session_id(session_id)
5049
self.__golden_images = None
5150
self.__plan_scripts = None

‎hpOneView/oneview_client.py‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -*- coding: utf-8 -*-
22
###
3-
# (C) Copyright (2012-2017) Hewlett Packard Enterprise Development LP
3+
# (C) Copyright (2012-2018) Hewlett Packard Enterprise Development LP
44
#
55
# Permission is hereby granted, free of charge, to any person obtaining a copy
66
# of this software and associated documentation files (the "Software"), to deal
@@ -117,7 +117,7 @@ class OneViewClient(object):
117117

118118
def __init__(self, config):
119119
self.__connection = connection(config["ip"], config.get('api_version', self.DEFAULT_API_VERSION), config.get('ssl_certificate', False),
120-
config.get('timeout'))
120+
config.get('timeout'), config.get('reuse_connection', False))
121121
self.__image_streamer_ip = config.get("image_streamer_ip")
122122
self.__set_proxy(config)
123123
self.__connection.login(config["credentials"])
@@ -218,7 +218,7 @@ def from_environment_variables(cls):
218218
219219
Allowed variables: ONEVIEWSDK_IP (required), ONEVIEWSDK_USERNAME (required), ONEVIEWSDK_PASSWORD (required),
220220
ONEVIEWSDK_AUTH_LOGIN_DOMAIN, ONEVIEWSDK_API_VERSION, ONEVIEWSDK_IMAGE_STREAMER_IP, ONEVIEWSDK_SESSIONID, ONEVIEWSDK_SSL_CERTIFICATE,
221-
ONEVIEWSDK_CONNECTION_TIMEOUT and ONEVIEWSDK_PROXY.
221+
ONEVIEWSDK_CONNECTION_TIMEOUT, ONEVIEWSDK_PROXY and ONEVIEWSDK_REUSE_CONNECTION.
222222
223223
Returns:
224224
OneViewClient:
@@ -233,13 +233,14 @@ def from_environment_variables(cls):
233233
proxy = os.environ.get('ONEVIEWSDK_PROXY', '')
234234
sessionID = os.environ.get('ONEVIEWSDK_SESSIONID', '')
235235
timeout = os.environ.get('ONEVIEWSDK_CONNECTION_TIMEOUT')
236+
reuse_connection = bool(os.environ.get('ONEVIEWSDK_REUSE_CONNECTION', ''))
236237

237238
config = dict(ip=ip,
238239
image_streamer_ip=image_streamer_ip,
239240
api_version=api_version,
240241
ssl_certificate=ssl_certificate,
241242
credentials=dict(userName=username, authLoginDomain=auth_login_domain, password=password, sessionID=sessionID),
242-
proxy=proxy, timeout=timeout)
243+
proxy=proxy, timeout=timeout, reuse_connection=reuse_connection)
243244

244245
return cls(config)
245246

@@ -289,7 +290,9 @@ def create_image_streamer_client(self):
289290
image_streamer = ImageStreamerClient(self.__image_streamer_ip,
290291
self.__connection.get_session_id(),
291292
self.__connection._apiVersion,
292-
self.__connection._sslBundle)
293+
self.__connection._sslBundle,
294+
self.__connection._timeout,
295+
self.__connection._reuse_connection)
293296

294297
return image_streamer
295298

‎tests/unit/test_connection.py‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -*- coding: utf-8 -*-
22
###
3-
# (C) Copyright (2016-2017) Hewlett Packard Enterprise Development LP
3+
# (C) Copyright (2016-2018) Hewlett Packard Enterprise Development LP
44
#
55
# Permission is hereby granted, free of charge, to any person obtaining a copy
66
# of this software and associated documentation files (the "Software"), to deal
@@ -604,6 +604,25 @@ def test_download_to_stream_when_error_status_with_empty_body(self, mock_get_con
604604
else:
605605
self.fail()
606606

607+
@patch.object(connection, 'get_connection')
608+
def test_reuse_connection(self, mock_get_conn):
609+
mock_conn = Mock()
610+
mock_get_conn.return_value = mock_conn
611+
612+
mock_response = mock_conn.getresponse.return_value
613+
mock_response.read.return_value = json.dumps('').encode('utf-8')
614+
mock_response.status = 202
615+
616+
try:
617+
self.connection._reuse_connection = True
618+
self.connection.do_http('GET', '/rest', None)
619+
self.assertEqual(self.connection._conn, mock_conn)
620+
self.connection.close_reusable_connection(mock_conn)
621+
self.assertEqual(self.connection._conn, None)
622+
finally:
623+
self.connection._reuse_connection = False
624+
self.connection._conn = None
625+
607626
@patch.object(connection, 'get_connection')
608627
def test_download_to_stream_with_timeout_error(self, mock_get_connection):
609628

‎tests/unit/test_oneview_client.py‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# -*- coding: utf-8 -*-
22
###
3-
# (C) Copyright (2012-2017) Hewlett Packard Enterprise Development LP
3+
# (C) Copyright (2012-2018) Hewlett Packard Enterprise Development LP
44
#
55
# Permission is hereby granted, free of charge, to any person obtaining a copy
66
# of this software and associated documentation files (the "Software"), to deal
@@ -100,7 +100,8 @@
100100
'ONEVIEWSDK_API_VERSION': '201',
101101
'ONEVIEWSDK_AUTH_LOGIN_DOMAIN': 'authdomain',
102102
'ONEVIEWSDK_PROXY': '172.16.100.195:9999',
103-
'ONEVIEWSDK_CONNECTION_TIMEOUT': '20'
103+
'ONEVIEWSDK_CONNECTION_TIMEOUT': '20',
104+
'ONEVIEWSDK_REUSE_CONNECTION': 'Yes'
104105
}
105106

106107
OS_ENVIRON_CONFIG_FULL_WITH_SESSIONID = {
@@ -111,8 +112,8 @@
111112
'ONEVIEWSDK_SESSIONID': '123',
112113
'ONEVIEWSDK_API_VERSION': '201',
113114
'ONEVIEWSDK_PROXY': '172.16.100.195:9999',
114-
'ONEVIEWSDK_CONNECTION_TIMEOUT': '20'
115-
115+
'ONEVIEWSDK_CONNECTION_TIMEOUT': '20',
116+
'ONEVIEWSDK_REUSE_CONNECTION': 'Yes'
116117
}
117118

118119

@@ -300,6 +301,7 @@ def test_from_environment_variables_is_passing_right_arguments_to_the_constructo
300301
mock_cls.assert_called_once_with({'api_version': 201,
301302
'proxy': '172.16.100.195:9999',
302303
'timeout': '20',
304+
'reuse_connection': True,
303305
'ip': '172.16.100.199',
304306
'ssl_certificate': '',
305307
'image_streamer_ip': '172.172.172.172',
@@ -317,6 +319,7 @@ def test_from_environment_variables_is_passing_right_arguments_to_the_constructo
317319
mock_cls.assert_called_once_with({'api_version': 201,
318320
'proxy': '172.16.100.195:9999',
319321
'timeout': '20',
322+
'reuse_connection': True,
320323
'ip': '172.16.100.199',
321324
'image_streamer_ip': '172.172.172.172',
322325
'ssl_certificate': '',
@@ -334,6 +337,7 @@ def test_from_environment_variables_is_passing_right_arguments_to_the_constructo
334337
mock_cls.assert_called_once_with({'api_version': 300,
335338
'proxy': '',
336339
'timeout': None,
340+
'reuse_connection': False,
337341
'ip': '172.16.100.199',
338342
'image_streamer_ip': '',
339343
'ssl_certificate': '',

0 commit comments

Comments
 (0)