Martins Blog

Trying to explain complex things in simple terms

11.2.0.2 Bundled Patch 3 for Linux x86-64bit

Posted by Martin Bach on February 2, 2011

It has become a habit recently to issue explicit warnings when presenting information that-if it doesn’t work as described-might cause you harm. So to follow suit, here’s mine:

Warning! This is a Patch Set Update installation war story-it worked for me. That does by no means imply that the same works for you! So always apply the PSU or bundle patch on a non-critical test system first. And always take a backup of your ORACLE_HOME and inventory first before applying any patches-you have been warned. I like to be rather safe than sorry.

A good one I think …  First of all thanks to Coskan for the pointer to the Oracle blog entry: http://blogs.oracle.com/UPGRADE/2011/01/11202_bundled_patch_3_for_linu.html which actually prompted me to apply the patch in the first place.

My environment is a 2 node 11.2.0.2 cluster on OEL 5.5 64bit. Judging by the readme file I need the latest OPatch, 11.2.0.1.4, (patch p6880880)  and the bundle patch itself (patch 10387939). Unfortunately Oracle leaves it as an exercise to the reader to make sense of the patch readme and note 1274453.1 which is applicable if ACFS is involved on the cluster. Although I am using ACFS all my ACFS volumes were disabled before patching (i.e. they are not shared ORACLE_HOMEs). If you have ACFS in use then follow note 1274453.1 and shut all of them down before proceeding. Note 1274453.1 is divided into a number of sections. First it’s important to find the right case-I wanted to try the opatch auto feature this time, but patch GRID_HOME independently of the RDBMS_HOME. I knew from other users that opatch auto has gained a bad reputation in the past but was curious whether or not Oracle fixed it. Regardless of your configuration, you have to stop dbconsole instances on all nodes if there are any.

If applicable, the next step is to stop all database instances and their corresponding ACFS volumes. The ACFS mounts also have to be dismounted. If you are unsure you can use /sbin/acfsutil registry to list the ACFS volumes. Those registered with Clusterware show up in the output of crsctl status resource -t.

Important! If not already done so, update OPatch on all nodes for GI and RDBMS home. It’s a lot of manual work, I wish there was a cluster wide OPatch location…it would make my life so much easier. Just patching OPatch on 4 nodes is a pain, I don’t want to imagine a 8 or more nodes cluster right now.

Now proceed with the patch installation. I’ll only let it patch the GRID_HOME at this stage. The patch should be rolling which is good. Note that you’ll be root, which can cause its own problems in larger organisations. My patch is staged in /u01/app/oracle/product/stage, and the patch readme suggests I should use the opatch executable in the $GRID_HOME.

[root@node1 ~]# cd /u01/app/oracle/product/stage
[root@node1 stage]# cd 10387939
[root@node1 10387939]# export PATH=/u01/app/oragrid/product/11.2.0.2/OPatch:$PATH

Oracle Configuration Manager response file

Unlike with previous patchsets which prompted the user each time you ran opatch, this time the OCM configuration is not part of the patch set installation. Instead, you have to create an OCM configuration “response” file before you apply the patch. The file is created in the current directory. As the GRID_OWNER, execute $ORACLE_HOME/OPatch/ocm/bin/emocmrsp as shown here:

[root@node1 stage]# su - oragrid
[oragrid@node1 ~]$ . oraenv
ORACLE_SID = [oragrid] ? grid
The Oracle base for ORACLE_HOME=/u01/app/oragrid/product/11.2.0.2 is /u01/app/oragrid/product/admin
[oragrid@node1 ~]$ /u01/app/oragrid/product/11.2.0.2/OPatch/ocm/bin/emocmrsp
OCM Installation Response Generator 10.3.1.2.0 - Production
Copyright (c) 2005, 2009, Oracle and/or its affiliates.  All rights reserved.

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  y
The OCM configuration response file (ocm.rsp) was successfully created.
[oragrid@node1 ~]$ pwd
/home/oragrid

I don’t want automatic email notifications in this case so I left the email address blank. I also prefer to be uninformed of future patch sets. I then moved the file into /u01/app/oracle/product/stage/ocm.rsp as root for the next steps. It is good practise to save the detailed oracle inventory information for later. Again as the grid owner I ran this command:

[root@node1 stage]# su - oragrid
[oragrid@node1 ~]$ . oraenv
ORACLE_SID = [oragrid] ? grid
The Oracle base for ORACLE_HOME=/u01/app/oragrid/product/11.2.0.2 is /u01/app/oragrid/product/admin
[oragrid@node1 ~]$ /u01/app/oragrid/product/11.2.0.2/OPatch/opatch lsinv -detail > /tmp/gridhomedetail

What can (will) be patched?

The bundle.xml file lists the components that can be patched with a bundle patch. In my case, the bundle is applicable for these:

<bundle type = "EXA_BUNDLE3">
 <entities>
 <entity location="10387939">
 <target type="crs"/>
 <target type="rac"/>
 <target type="sidb"/>
 <target type="siha"/>
 </entity>
 <entity location="10157622">
 <target type="crs"/>
 <target type="siha"/>
 </entity>
 <entity location="10157622/custom/server/10157622">
 <target type="rac" />
 </entity>
 <entity location="10626132">
 <target type="crs"/>
 <target type="siha"/>
 </entity>
 </entities>
</bundle>

So in other words, the bundle exists of 3 patches – 10387939, 10157622 and 10626132. For each of these a target type is shown. SIHA is short for Single Instance High Availability, which has been renamed “Oracle Restart”. I assume SIDB is single instance DB, but don’t know for sure.

Patching the GRID_HOME

I’m now ready to apply the patch to the GRID home. The command to do so is simple, the below example worked for me:

[root@node1 stage]# opatch auto /u01/app/oracle/product/stage/10387939 -oh /u01/app/oragrid/product/11.2.0.2
Executing /usr/bin/perl /u01/app/oragrid/product/11.2.0.2/OPatch/crs/patch112.pl -patchdir /u01/app/oracle/product/stage -patchn 10387939 -oh

/u01/app/oragrid/product/11.2.0.2 -paramfile /u01/app/oragrid/product/11.2.0.2/crs/install/crsconfig_params
opatch auto log file location is /u01/app/oragrid/product/11.2.0.2/OPatch/crs/../../cfgtoollogs/opatchauto2011-02-02_15-50-04.log
Detected Oracle Clusterware install
Using configuration parameter file: /u01/app/oragrid/product/11.2.0.2/crs/install/crsconfig_params
OPatch  is bundled with OCM, Enter the absolute OCM response file path:

Before entering the path I opened another terminal to look at the log file, which is in $ORACLE_HOME/cfgtoollogs/opatchauto plus a timestamp. After entering the full path to my ocm.rsp file nothing happened in my opatch session-Oracle could have told us that it was about to apply the patch. That is because it actually is doing that. Go back to your other terminal and see the log messages fly by! The opatch auto command automatically does everything that the DBA had to do previously, including unlocking the CRS stack and calling opatch napply for the relevant bits and pieces. This is indeed a nice step forward. I can vividly remember having to apply portions of a PSU to the GRID_HOME and others to the RDBMS home, 6 or 7 steps in total before a patch was applied. That was indeed hard work.

Only after CRS has been shut down (after quite a while after entering the path to the ocm.rsp file!) will you be shown this line:

Successfully unlock /u01/app/oragrid/product/11.2.0.2

Even further down the line you see those, after the patches have been applied:

patch /u01/app/oracle/product/stage/10387939/10387939  apply successful for home  /u01/app/oragrid/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10157622  apply  failed  for home  /u01/app/oragrid/product/11.2.0.2
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9312: Existing ADVM/ACFS installation detected.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-4123: Oracle High Availability Services has been started.

And with that I got my prompt back. One patch failed to apply-the log file indicated a lack of space (2G are required to be free).Tomorrow I’ll post an update and remove/reapply the patch manually.

I checked the other node and found that the patch has indeed been applied on the local node only. I hasn’t propagated across which is good as the readme wasn’t really clear if the patch was rolling. From what I’ve seen I’d call it a local patch, similar to the “opatch napply -local” we did manually before the opatch auto option has been introduced. And even better, it worked. Querying opatch lsinventory I got this result:

Lsinventory Output file location : /u01/app/oragrid/product/11.2.0.2/cfgtoollogs/opatch/lsinv/lsinventory2011-02-02_16-01-27PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure                                           11.2.0.2.0
There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch  10387939     : applied on Wed Feb 02 15:56:50 GMT 2011
Unique Patch ID:  13350217
 Created on 30 Dec 2010, 22:55:01 hrs PST8PDT
 Bugs fixed:
 10158965, 9940990, 10190642, 10031806, 10228635, 10018789, 9744252
 10010252, 9956713, 10204358, 9715581, 9770451, 10094635, 10121589
 10170431, 9824198, 10071193, 10145612, 10035737, 9845644, 10086980
 10052141, 10039731, 10035521, 10219576, 10184634, 10207092, 10138589
 10209232, 8752691, 9965655, 9819413, 9500046, 10106828, 10220118, 9881076
 9869287, 10040531, 10122077, 10218814, 10261389, 10033603, 9788588
 9735237, 10126219, 10043801, 10073205, 10205715, 9709292, 10105926
 10079168, 10098253, 10005127, 10013431, 10228151, 10092153, 10142909
 10238786, 10260808, 10033071, 9791810, 10052956, 9309735, 10026972
 10080579, 10073683, 10004943, 10019218, 9539440, 10022980, 10061490
 10006008, 6523037, 9724970, 10142776, 10208386, 10113803, 10261680
 9671271, 10084145, 10051966, 10355493, 10227133, 10229719, 10046912
 10228393, 10353054, 10142788, 10221016, 9414040, 10127360, 10310299
 10094201, 9591812, 10129643, 10332589, 10026193, 10195991, 10260870
 10248523, 9951423, 10261072, 10299224, 10230571, 10222719, 10233732
 10113633, 10102506, 10094949, 10077191, 10329146, 8685446, 10048701
 10314582, 10149223, 10245259, 10151017, 9924349, 10245086, 11074393

Rac system comprising of multiple nodes
 Local node = node1
 Remote node = node2

--------------------------------------------------------------------------------

OPatch succeeded.

Patching the RDBMS home

Now with that I’ll try applying it to the RDBMS home on the same node:

[root@node1 stage]# opatch auto /u01/app/oracle/product/stage/10387939 -oh /u01/app/oracle/product/11.2.0.2
Executing /usr/bin/perl /u01/app/oragrid/product/11.2.0.2/OPatch/crs/patch112.pl -patchdir /u01/app/oracle/product/stage -patchn 10387939 -oh

/u01/app/oracle/product/11.2.0.2 -paramfile /u01/app/oragrid/product/11.2.0.2/crs/install/crsconfig_params
opatch auto log file location is /u01/app/oragrid/product/11.2.0.2/OPatch/crs/../../cfgtoollogs/opatchauto2011-02-02_16-05-29.log
Detected Oracle Clusterware install
Using configuration parameter file: /u01/app/oragrid/product/11.2.0.2/crs/install/crsconfig_params
OPatch  is bundled with OCM, Enter the absolute OCM response file path:
/u01/app/oracle/product/stage/ocm.rsp
patch /u01/app/oracle/product/stage/10387939/10387939  apply successful for home  /u01/app/oracle/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10157622/custom/server/10157622  apply successful for home  /u01/app/oracle/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10626132  apply successful for home  /u01/app/oracle/product/11.2.0.2
[root@node1 stage]#

Cool! That was indeed easy. Did it work?

$ORACLE_HOME/OPatch/opatch lsinv
[...]
--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Database 11g                                                  11.2.0.2.0
There are 1 products installed in this Oracle Home.

Interim patches (3) :

Patch  10626132     : applied on Wed Feb 02 16:08:43 GMT 2011
Unique Patch ID:  13350217
 Created on 31 Dec 2010, 00:18:12 hrs PST8PDT
 Bugs fixed:
 10626132

Patch  10157622     : applied on Wed Feb 02 16:08:27 GMT 2011
Unique Patch ID:  13350217
 Created on 19 Nov 2010, 01:41:19 hrs PST8PDT
 Bugs fixed:
 9979706, 9959110, 10016083, 10015460, 10014392, 9918485, 10157622
 10089120, 10057296, 9971646, 10053985, 10040647, 9978765, 9864003
 10069541, 10110969, 10107380, 9915329, 10044622, 10029119, 9812970
 10083009, 9812956, 10048027, 10036193, 10008467, 10040109, 10015210
 10083789, 10033106, 10073372, 9876201, 10042143, 9963327, 9679401
 10062301, 10018215, 10075643, 10007185, 10071992, 10057680, 10038791
 10124517, 10048487, 10078086, 9926027, 10052721, 9944948, 10028235
 10146768, 10011084, 10027079, 10028343, 10045436, 9907089, 10073075
 10175855, 10178670, 10072474, 10036834, 9975837, 10028637, 10029900, 9949676

Patch  10157622     : applied on Wed Feb 02 16:08:27 GMT 2011
Unique Patch ID:  13350217
 Created on 19 Nov 2010, 01:41:19 hrs PST8PDT
 Bugs fixed:
 9979706, 9959110, 10016083, 10015460, 10014392, 9918485, 10157622
 10089120, 10057296, 9971646, 10053985, 10040647, 9978765, 9864003
 10069541, 10110969, 10107380, 9915329, 10044622, 10029119, 9812970
 10083009, 9812956, 10048027, 10036193, 10008467, 10040109, 10015210
 10083789, 10033106, 10073372, 9876201, 10042143, 9963327, 9679401
 10062301, 10018215, 10075643, 10007185, 10071992, 10057680, 10038791
 10124517, 10048487, 10078086, 9926027, 10052721, 9944948, 10028235
 10146768, 10011084, 10027079, 10028343, 10045436, 9907089, 10073075
 10175855, 10178670, 10072474, 10036834, 9975837, 10028637, 10029900, 9949676

Patch  10387939     : applied on Wed Feb 02 16:07:18 GMT 2011
Unique Patch ID:  13350217
 Created on 30 Dec 2010, 22:55:01 hrs PST8PDT
 Bugs fixed:
 10158965, 9940990, 10190642, 10031806, 10228635, 10018789, 9744252
 10010252, 9956713, 10204358, 9715581, 9770451, 10094635, 10121589
 10170431, 9824198, 10071193, 10145612, 10035737, 9845644, 10086980
 10052141, 10039731, 10035521, 10219576, 10184634, 10207092, 10138589
 10209232, 8752691, 9965655, 9819413, 9500046, 10106828, 10220118, 9881076
 9869287, 10040531, 10122077, 10218814, 10261389, 10033603, 9788588
 9735237, 10126219, 10043801, 10073205, 10205715, 9709292, 10105926
 10079168, 10098253, 10005127, 10013431, 10228151, 10092153, 10142909
 10238786, 10260808, 10033071, 9791810, 10052956, 9309735, 10026972
 10080579, 10073683, 10004943, 10019218, 9539440, 10022980, 10061490
 10006008, 6523037, 9724970, 10142776, 10208386, 10113803, 10261680
 9671271, 10084145, 10051966, 10355493, 10227133, 10229719, 10046912
 10228393, 10353054, 10142788, 10221016, 9414040, 10127360, 10310299
 10094201, 9591812, 10129643, 10332589, 10026193, 10195991, 10260870
 10248523, 9951423, 10261072, 10299224, 10230571, 10222719, 10233732
 10113633, 10102506, 10094949, 10077191, 10329146, 8685446, 10048701
 10314582, 10149223, 10245259, 10151017, 9924349, 10245086, 11074393

Rac system comprising of multiple nodes
 Local node = node1
 Remote node = node2

--------------------------------------------------------------------------------

OPatch succeeded.

It did indeed. As a nice side effect I didn’t even have to worry about the srvctl start/stop home commands, they were automatically done for me. From the log:

2011-02-02 16:08:45: /u01/app/oracle/product/11.2.0.2/bin/srvctl start home -o /u01/app/oracle/product/11.2.0.2 -s

/u01/app/oracle/product/11.2.0.2/srvm/admin/stophome.txt -n node1 output is
2011-02-02 16:08:45: Started resources from datbase home /u01/app/oracle/product/11.2.0.2

Now I’ll simply repeat this on the remaining nodes and should be done. To test the stability of the process I didn’t limit the opatch command to a specific home but had it patch them all.

Patching all homes in one go

This takes a little longer but otherwise is just the same. I have added the output here for the sake of completeness:

[root@node2 stage]# which opatch
/u01/app/oragrid/product/11.2.0.2/OPatch/opatch
[root@node2 stage]# opatch auto /u01/app/oracle/product/stage/10387939
Executing /usr/bin/perl /u01/app/oragrid/product/11.2.0.2/OPatch/crs/patch112.pl -patchdir /u01/app/oracle/product/stage -patchn 10387939 -paramfile /u01/app/oragrid/product/11.2.0.2/crs/install/crsconfig_params
opatch auto log file location is /u01/app/oragrid/product/11.2.0.2/OPatch/crs/../../cfgtoollogs/opatchauto2011-02-02_16-36-09.log
Detected Oracle Clusterware install
Using configuration parameter file: /u01/app/oragrid/product/11.2.0.2/crs/install/crsconfig_params
OPatch  is bundled with OCM, Enter the absolute OCM response file path:
/u01/app/oracle/product/stage/ocm.rsp
patch /u01/app/oracle/product/stage/10387939/10387939  apply successful for home  /u01/app/oracle/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10157622/custom/server/10157622  apply successful for home  /u01/app/oracle/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10626132  apply successful for home  /u01/app/oracle/product/11.2.0.2
Successfully unlock /u01/app/oragrid/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10387939  apply successful for home  /u01/app/oragrid/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10157622  apply successful for home  /u01/app/oragrid/product/11.2.0.2
patch /u01/app/oracle/product/stage/10387939/10626132  apply successful for home  /u01/app/oragrid/product/11.2.0.2
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9312: Existing ADVM/ACFS installation detected.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-4123: Oracle High Availability Services has been started.
[root@node2 stage]#

Well that’s for the technical part. When I compared the number of patches applied in the $GRID_HOME on node 2 (which simply used “opatch auto”) then I found 3 interim patches applied, as compared to only 1 on node 1 when I only patched the GRID_HOME. I’ll have to raise this with Oracle…

[oragrid@node2 ~]$ /u01/app/oragrid/product/11.2.0.2/OPatch/opatch lsinv
Invoking OPatch 11.2.0.1.4

Oracle Interim Patch Installer version 11.2.0.1.4
Copyright (c) 2010, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oragrid/product/11.2.0.2
Central Inventory : /u01/app/oracle/product/oraInventory
 from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.4
OUI version       : 11.2.0.2.0
OUI location      : /u01/app/oragrid/product/11.2.0.2/oui
Log file location : /u01/app/oragrid/product/11.2.0.2/cfgtoollogs/opatch/opatch2011-02-02_17-13-40PM.log

Patch history file: /u01/app/oragrid/product/11.2.0.2/cfgtoollogs/opatch/opatch_history.txt

Lsinventory Output file location : /u01/app/oragrid/product/11.2.0.2/cfgtoollogs/opatch/lsinv/lsinventory2011-02-02_17-13-40PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure                                           11.2.0.2.0
There are 1 products installed in this Oracle Home.

Interim patches (3) :

Patch  10626132     : applied on Wed Feb 02 16:48:59 GMT 2011
Unique Patch ID:  13350217
 Created on 31 Dec 2010, 00:18:12 hrs PST8PDT
 Bugs fixed:
 10626132

Patch  10157622     : applied on Wed Feb 02 16:48:34 GMT 2011
Unique Patch ID:  13350217
 Created on 19 Nov 2010, 01:41:33 hrs PST8PDT
 Bugs fixed:
 9979706, 9959110, 10016083, 10015460, 10014392, 9918485, 10157622
 10089120, 10057296, 9971646, 10053985, 10040647, 9978765, 9864003
 10069541, 10110969, 10107380, 9915329, 10044622, 10029119, 9812970
 10083009, 9812956, 10048027, 10036193, 10008467, 10040109, 10015210
 10083789, 10033106, 10073372, 9876201, 10042143, 9963327, 9679401
 10062301, 10018215, 10075643, 10007185, 10071992, 10057680, 10038791
 10124517, 10048487, 10078086, 9926027, 10052721, 9944948, 10028235
 10146768, 10011084, 10027079, 10028343, 10045436, 9907089, 10073075
 10175855, 10072474, 10036834, 9975837, 10028637, 10029900, 9949676
 9974223, 10260251

Patch  10387939     : applied on Wed Feb 02 16:46:29 GMT 2011
Unique Patch ID:  13350217
 Created on 30 Dec 2010, 22:55:01 hrs PST8PDT
 Bugs fixed:
 10158965, 9940990, 10190642, 10031806, 10228635, 10018789, 9744252
 10010252, 9956713, 10204358, 9715581, 9770451, 10094635, 10121589
 10170431, 9824198, 10071193, 10145612, 10035737, 9845644, 10086980
 10052141, 10039731, 10035521, 10219576, 10184634, 10207092, 10138589
 10209232, 8752691, 9965655, 9819413, 9500046, 10106828, 10220118, 9881076
 9869287, 10040531, 10122077, 10218814, 10261389, 10033603, 9788588
 9735237, 10126219, 10043801, 10073205, 10205715, 9709292, 10105926
 10079168, 10098253, 10005127, 10013431, 10228151, 10092153, 10142909
 10238786, 10260808, 10033071, 9791810, 10052956, 9309735, 10026972
 10080579, 10073683, 10004943, 10019218, 9539440, 10022980, 10061490
 10006008, 6523037, 9724970, 10142776, 10208386, 10113803, 10261680
 9671271, 10084145, 10051966, 10355493, 10227133, 10229719, 10046912
 10228393, 10353054, 10142788, 10221016, 9414040, 10127360, 10310299
 10094201, 9591812, 10129643, 10332589, 10026193, 10195991, 10260870
 10248523, 9951423, 10261072, 10299224, 10230571, 10222719, 10233732
 10113633, 10102506, 10094949, 10077191, 10329146, 8685446, 10048701
 10314582, 10149223, 10245259, 10151017, 9924349, 10245086, 11074393

Rac system comprising of multiple nodes
 Local node = node2
 Remote node = node1

--------------------------------------------------------------------------------

OPatch succeeded.

One Response to “11.2.0.2 Bundled Patch 3 for Linux x86-64bit”

  1. awesome artical

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: