Maintenance.  It's critical to keeping any system in top performing shape and preventing unplanned service interruptions.  Oracle AgilePLM is often overlooked when it comes to performing regular maintenance, but a little effort will go a long way to keeping your AgilePLM system humming along.

Table of Contents

1. Run Averify Regularly
2. Regularly Clean Up the Log Files
3. Regularly Review Error Log Files
4. Schedule Regular Maintenance Days
5. Monitor Disk Usage and DB Tablespace
6. Perform Regular File Manager Maintenance
7. Regular File Exports/Backups
8. Keep Up with OS System Updates
9. Perform Regular Security Audits
10. Establish Performance Baselines
11. Learn How to Submit Effective Oracle Support Requests

1. Run Averify Regularly
Cleaning up Averify issues is key to keeping your Agile data consistent and the system performing at its best.Although Averify does not fix the database, this is perhaps the most overlooked aspect of Agile system maintenance and at the same time it is the most impactful task you can undertake.  No single maintenance task has such a direct impact on the reliability, performance and integrity of your Agile database. You can use Oracle Technology Network to ensure you always have the most up-to-date version of Averify.  Check the Oracle document: How to Download the Latest Version of Averify (Doc ID 571832.1) for instructions on getting the most recent Averify version.

The Oracle provided ADT generic Averify fix scripts go hand in hand with running Averify.  The generic scripts will repair a number of database integrity issues without involving Oracle support directly and are fully supported by Oracle support.  Details on these generic solutions can be found in OTN at this document:  Agile Product Lifecycle Management (PLM) Master Index for Averify Solutions (Doc ID 1121926.1)

Oracle has also published a Best Practices document here: Best Practice for Running Agile PLM Averify Utility (Doc ID 1270857.1)

You should be familiar with these resources before attempting to utilize the generic fixes and NEVER run fixes without testing them in a non-production environment first.

1. To use Averify, download the latest version and the unzip to a folder location on the server running Oracle Database.
2. Open the Averify.properties file (located in the config folder) and modify the connection properties to match your environment. See the graphic below for the workflow.
   
3. Stop the Agile server and any attached File Managers.
4. Open an administrator command prompt (run as administrator) and move to the Averify bin folder.
5. Run Averify.bat
6. Provide the credentials for the database connection specified in the averify.properties file.
7. Wait for Averify to complete.
8. Retrieve the Averify logs from the logs folder and examine them for errors requiring action.

If the oracle_averify_report.log file contains any error codes listed in the “Agile Product Lifecycle Management (PLM) Master Index for Averify Solutions (Doc ID 1121926.1)”, proceed to run the latest ADT Generic scripts as listed in that same document.

For errors not listed in the Master Index, contact Oracle Support.
 

2. Regularly Clean Up the Log Files

Keeping log files in check reduces storage requirements and makes troubleshooting easier.

During any restart of the Agile system, make it a practice to delete the existing log files or archive them if you prefer.  These logs can use significant disk space and can grow to sizes that make analysis impracticable.

This includes log files for the FileManager(s).

Agile logs are located by default at:

{AGILE_HOME}/agileDomain/server/servername/logs

FileManager logs are located by default at:

{AGILE_HOME}/FileManager/logs

Simply open an administrative command prompt (run as administrator), move to the folders and delete the .log files.  Agile will recreate all of the log files when you start it up.

Make this a regular part of your maintenance routine to minimize the effort of opening the files and finding the most relevant sections.

3. Regularly Review Error Log Files

Practice proactive issue management identifying potential issues before users report a problem. A motivated and professional system administrator strives to proactively identify potential or existing issues in the Agile environment and monitoring log files is a part of this.  However, the modern reality of busy IT departments makes such efforts short lived.  Instead, look for log monitoring systems that will notify appropriate personnel when specified conditions exist in a log file.  Several commercial solutions are available and a wily system administrator can craft their own.  Linux installations already contain all the pieces to make such monitoring practical and suitable alternatives are available for the Windows platform.

Agile will generate error statements in the stdout.log file that are not actionable. These result from any number of complex circumstances.  The key is to learn which are ‘normal’ and which are worthy of further investigation.  Unfortunately, the best means of learning this skill is to regularly review the log files and research messages using the Oracle Technology Network or OTN.

OTN is an amazing resource of practical advice and guidance that can enhance your troubleshooting efforts. However, not all of the advice you may be applicable to your particular version of Agile or its environment. Learn to wade through the Oracle documents by constructing more explicit search terms and utilize Powerview to limit responses to Agile related objects. If you are not yet comfortable using OTN and wish to learn more, Oracle has free webinars available on the OTN site to help you get started. Practice is still the best means of improvement in this area.

4. Schedule Regular Maintenance Days
Scheduled maintenance days allow you to perform a variety of systems checks, install OS patches and deploy Agile Hot Fixes.  Most IT departments already have fixed maintenance schedules for the system infrastructure and Agile administrators can take advantage of these planned downtimes to run Averify and other system checks on the system.  I have worked on Agile systems that not had Averify or other maintenance performed in years.  Not months, years.  Make sure yours is not one of them.

Maintenance can be performed monthly and often consists of running Averify, applying patches or hot fixes, fixing previously identified Averify errors and the aforementioned log maintenance.  It is also a good time to be sure your operating system is up-to-date.

5. Monitor Disk Usage and DB Tablespace
Monitor Database Tablespace Usage by running Averify regularly and watching the raw storage space.  Linux administrators are already familiar with the commands to monitor disk space (df anyone?) but a little more work is required on Windows systems.  Microsoft has made available the DU utility (written by Mark Russinovich who also authored the tantalizing Jeffery Aiken book series).  DU is available for free at this URL:  https://technet.microsoft.com/en-us/sysinternals/du.aspx

Failure to properly manage disk space can cause your Agile system to unexpectedly run out of Vault (File Repository) space or the database to become stalled resulting in user complaints and lost productivity.

6. Perform Regular File Manager Maintenance
Check the available disk space regularly and run Missing FiIes Utility and Dead File Utility on a regular basis.  Although disk space can be added ‘on the fly’, if your Agile system runs out of Vault space (File Repository) it will stop processing requests and this will inconvenience your users.  In Linux you can use the df -k or df -h (for human readable output) to check the space on the volumes of a Linux server.

In Windows, while it is possible to check disk space using the dir command, a more accurate and easier to use alternative is the Disk Usage (du) utility available for free from Microsoft.  Here is the link to download du and see the documentation:  https://technet.microsoft.com/en-us/sysinternals/du.aspx

DU has the added flexibility of formatting the output for use in batch files to generate disk usage statistics as part of an automated job on the server.

Microsoft PowerShell can also be used to track disk usage although it is a bit more complex and requires more setup.  You can find more on that topic here and elsewhere on the web.
https://gallery.technet.microsoft.com/scriptcenter/Check-Free-Disk-Space-of-921a2646

7. Regular File Exports/Backups

Prepare for refreshes and always make regular backups of your database.

While I would suggest that using export will take the place of a well-considered backup system it does have its advantages.  Firstly, Agile creates the necessary scripts as part of the database creation process making it very simple to use.  In Windows the scripts are called agile9exp.bat (uses old-school database dump) and agile9expdp.bat which uses the more efficient DataPump utilities.  Either will work but for Agile versions newer than 9.3.2 it is recommended that DataPump be used.  In Linux the same commands are agile9exp.sh and agile9expdp.sh.

Exports can be easily automated to occur on a regular basis streamlining the refresh process needed when chasing Averify errors or other system malfunctions. Zero Wait-State has directions for performing database refreshes which you can access freely here: https://zerowait-state.sharefile.com/d-sfa48f7c4ce242f3a

8. Keep Up with OS System Updates
To maintain both the functionality and security of your Agile systems you must stay current with OS patches and security bulletins.

Security experts all agree that the number one task you can perform to protect your system is to apply regular patches and updates.  Of course you will want to test any system level upgrades before applying them to a production environment.  It is important to establish a regular process for testing OS level updates and moving them to production in a timely manner. Timing is important because once a security patch is released a subsequent exploit is created and unleashed.  Good security is like an onion with many layers protecting the core.  Do not become complacent regarding OS updates and the need for a layered approach to system security.

9. Perform Regular Security Audits
Clean up the user accounts and look for potential incursion attempts while preventing security related issues.

Possible symptoms of an incursion attempt might be user accounts getting locked or repeated log entries indicating bad user credentials.

Regularly scanning the log files can sometimes lead you to identify a security breach but they are difficult to anticipate.  The best defense is to perform fast and accurate user maintenance.  Disable unused accounts and limit the proliferation of ‘test’ accounts as these often have weak passwords and are easily exploited.  Do not disable built-in Agile accounts such as ‘admin’, instead protect it with a strong password and very limited distribution.  Removing the account or disabling it can cause issues when attempting to refresh databases, submitting databases to Oracle in support of SR’s and make upgrades difficult.

Logging in to the Java Client with ‘admin’ privileges will allow you to run the “User History Report”, “Logon Failure Report”, “User Usage Report” and the “Users Configuration Report”.  These are useful tools for identifying potential misuse of the system and possible attacks on the system as well.

10. Establish Performance Baselines
Performance baselines for system functionality will help you recognize real problems versus perception.

How many times have you received a report of slow system performance without any backup information as to what specifically is is performing slowly or if indeed the is genuinely slow or just perceived that way today.  Recent versions of Agile are not perfect but are a very well tested and architected application that does not generally place undue workloads on database servers or application servers (not true of older Agile versions unfortunately). In order to properly trouble shoot performance problems, you must have a set of accurate (and up-to-date) baseline measurements to compare to.

Establish your performance baselines during normal system use within your environment and not during weekends of typical low usage periods.

Update the baseline information with each significant change to the system such as an OS upgrade, application of Hot Fixes or the deployment of custom PX code.  These will help you to chart the effect of changes to the system and can aid in trouble shooting.

Begin by creating standard use cases that can be easily and accurately duplicated.  Then measure the ‘performance’ of these use cases by recording system delay times, processing times and screen refresh times.  Repeat the measurements during slow and busy periods over several days to establish useful averages.

11. Learn How to Submit Effective Oracle Support Requests
Submitting complete hardware and software information along with log files will help to move your support request along faster.

Submitting roper Service Requests (SRs) to Oracle is an often overlooked skill that when performed badly can have a very negative affect on SR resolution.

Describe the problem as accurately as you can and with as much detail as you can without wandering down the path of obscure causes.  This will give the Oracle technician a fighting chance to quickly identify the cause of the problem or at least ask intelligent follow-up questions.

Always include system logs, ACollect reports and recent Averify reports in the submission.  This will help to minimize the up-front questions and requests from Oracle Support and allow them to get to work on your issue more quickly.  When submitting an SR, you cannot provide too much information.  Service Requests that clearly state the problem and how to recreate the issue are going to be more actively worked by support personnel.

Oracle Resources

• Best Practices for Maintaining an Agile Product Lifecycle Management (PLM) System (Doc ID 1522897.1)
• How to Download the Latest Version of Averify (Doc ID 571832.1) for instructions on getting the most recent Averify version
• Agile Product Lifecycle Management (PLM) Master Index for Averify Solutions (Doc ID 1121926.1)
• Best Practice for Running Agile PLM Averify Utility (Doc ID 1270857.1)
• Best Practices for Refreshing an Agile Product Lifecycle Management (PLM) System to a Test Environment (Doc ID 1527257.1)

ZWS Blog Resources:

• A Better Mousetrap (Agile PLM)
• Agile PLM 9.3.2 Changed The Database Import Process
• Database Refresh Process Documentation
• ZeroWait-State Password Script for refreshes

Zero Wait-State has programs available
to help you maintain your Agile system.
Click here to contact us.

If you need help with it, look for an article in the future, leave a comment below, or contact us, we’ll be glad to help!